I've given some thought about this today and hopefully can bring up some points that will be beneficial. I think there are a few things with your goals and Wofati style building that need some consideration. First about your goals:
1. Live debt free
I love this! Minimizing debt and doing things for yourself to minimize unnecessary future spending is the gift that keeps on giving. Paying attention and practicing is a great way to reach that goal.
2. Energy efficiency
This is probably closer to my heart than anything else with permaculture. Not wasting personal energy is nice, and minimizing outside energy inputs goes hand in hand with reducing spending. There are a number of things from electricity to thermal mass and insulation that can be used to increase efficiency, or misunderstood and reduce efficiency.
3. Self built as much as possible
Given the later line saying that you are not builders tells to me that you probably need some hands on time with building before building a house you expect to spend your life in, and you need something more straightforward to start with. The earthbag idea seems good for learning a few things while creating a structure that will serve you well in the future without outrageous upfront costs. A small scale model of the home you want to build might be another exercise that could enlighten you to some pros and cons before committing to the final build.
4. Build with what's available, natural, repurposed or reused
This is one of the main tenets of Wofati building. If anyone is interested in building a Wofati style structure, it seems paramount to me that you look closely at the Wofati definition and give each aspect serious consideration before trying to modify it, doubly so if your building skills are limited. Too many changes can end up defeating the original purpose of the design. The Wofati Page
gives a detailed explanation which I will summarize:
oodland" I'll let others debate on whether it is possible or not to build in this style without being in or near a woodland and on or not on a slope. I'd imagine earthworks and creating a food forest could be beneficial, and having plenty of cheap or free lumber for the structure is critically tied in with another part of the definition.
ehler" is for Mike Oehler inspired underground house design.
reaky-cheap" This ties in with being in or near a woodland. If you can get all of your logs without paying for the logs themselves, just the cost of working them in to shape and moving them, then you can achieve Freaky-cheap materials. You may need to pay for top tier skilled labor to make this work, which might be a great place to spend your money if you aren't well versed with timber framing or underground structures. I would be looking for examples of actual timber framing and references before hiring someone.
nertia" I believe this is where the rubber meets the road, and flies in direct contrast to so-called 'common knowledge' and conventional building. Just adding insulation randomly to a thermal mass can have deleterious consequences to the point of negating most of the benefits of the mass. From what I can tell, no insulation and more mass is both cheaper and more effective than using insulation. The mass regulates the temperature inside, and as such must
contact the interior space you are trying to regulate in order to work. Insulation inside, in the walls, or between the walls and the mass will break contact between the interior and the mass and defeat the thermal inertia.
In the case of the Wofati's at the Lab, they were made to face away from the sun to prove that the thermal inertia can work without passive solar gain. For anybody else that wants maximum efficiency, you probably want to take advantage of the passive solar heating by facing the opening South towards the sun. A large mass, plus a rocket mass heater, plus passive solar should yield a space that is more efficient to keep warm than nearly any other house design. If you go for a lighter roof without earth, you can likely still make use of the rest of the ideas. I personally would look into adding something to the ceiling inside to reflect radiant heat back to the floors and walls to keep it in. Rocket mass heaters, passive solar, and thermal mass give a great deal of radiant heat, and certain reflective materials can reflect over 97% of radiant heat back into the living space. I'm not certain about what kind of testing has been done in this space with all natural materials, but aluminum, mylar, and steel can all reflect radiant heat back quite well.
I'm about all out of steam for the night, so hopefully that gives some more food for thought.