to answer some of your questions:
1) yes, draft is affected by height differences (heat riser, chimney) as well as temperature. Both affect the density of air.
2) insulation creates a higher temperature difference between insulated heat riser and uninsulated barrel. The steel drum is there to radiate heat and create a high temp. difference.
3) you think internal combustion engines are efficient? They aren't. They are much more efficient than they used to. But there's a huge compromise to make between efficiency and portability. They don't run as hot, because it's not practical to have a red hot glowing engine in your car. If You look at f1-engines on test rigs (youtube), You see red-hot glowing exhausts.
Now, why are high temperatures important? To burn fuel to close to 100%. The higher the temp., the higher the probability to burn 100%of fuel in all conditions. The compromise between efficiency and material stress is usually balanced towards efficiency, as the material stress isn't as important if you don't use steel. Sure, You already built it, so I would use it, too. But I don't see the relevance if you use the right material to start with
Regarding your cooling solution and the shift of the highest burn temp: There's 2 variables changed between test #2 and #3: outside temp difference and burning room temp (cooling). The result was that the red glowing appeared after the cooled area. But this could well be due to the cooling area! If you expand the cooling area, you might solve the problem. Or maybe the RMH doesn't overheat anymore, but the temps are too low for 100% combustion.
That's a common problem in engineering: You try to fix a problem, but instead, you just shift it to another place and maybe even worsen it. But you think you solved it, because you are looking in the wrong place...
All the best,