I usually put about 2 oz. per gallon in a pump up sprayer and fill with water. Add a small amount of dish soap or dr Bronners castile soap.(to keep the oil and water mixed).
Then go to soaking all leaves and fruit top and bottom. This is best done when the sun is going down. I do it about 3 times, once a week. Don't look for dead worms right away, It takes a little while to work.
Within a week should be far fewer, another week maybe none at all. It only kills chewing critters, wont hurt bees or earthworms etc.... Messes up reproductive cycles too on many chewing pests.
If that's correct (too hard to be sure from your pictures), it is considered a harmful invasive species - http://www.invadingspecies.com/gypsy-moth/ - so if they were on my trees, I'd put on a pair of rubber gloves, clean them off my plants as best I'm able, and squish and compost them. There are some caterpillars that are targeted by predatory wasps that will have a little white dot on them that is the wasp's egg, but I don't know if any are targetting Gypsy moths. Some caterpillars taste yucky, so the birds won't eat them (tent caterpillars for instance).
I would be very cautious about using chemicals because chemicals tend to kill the predator species faster than the prey, so you would end up further behind. That can even be true of "natural" chemicals that exist in nature. If it kills a "bad" caterpillar, it might just as easily kill a "Monarch caterpillar" which is native and endangered if any happen to be around. Your ideas of improving the support system for birds and predator bugs is a good long-term plan, but right now you've got too many bad guys, so helping nature re-balance isn't an unreasonable approach.
I don't know as much about them, but some use "bait traps" for some moths - pheromones that attract and catch the males to reduce reproduction. I believe those traps are quite species specific, so you won't harm any innocent bystanders, and it sounds as if your neighborhood is so overwhelmed right now, that such action would be warranted. Again, first you need to know what caterpillar you've got. This might sound weird, but you could try containing a few in some sort of ventilated large jar until they pupate and emerge, then identification would be definite! It's a bit of a fun science project also. I have a sister who loves doing that with Monarch butterflies.