Asphalt contains lots of PAHs and othe
petroleum based products/chemicals but is not inherently dangerous to vegetation nearby due to negligible leaching properties...meaning that
water won't break down the asphalt and contaminate/impact soils/groundwater near them. However, why in the world would you want to grow anything on the asphalt? Unless it is high-dollar permeable asphalt the asphalt will act as an impermeable layer preventing proper drainage. Also, it will be really hard to get good biological breakdown/soil building if the soils beneath the asphalt/concrete can't interact with whatever you place above.
IMO I would rent a bobcat and rip all that junk up. The soil beneath will probably need some intensive tlc because it has been isolated and depleated under the asphalt for the life of the asphalt. Find other forums on here to figure out how to rehab the soil once it is saved from asphalt/concrete. The asphalt and
concrete can be recyled seperately.