I usually dig an exploratory shaft to the bottom of the map as soon as I have initial storage shops, and living areas dug out, finding all caverns and magma as well as exposing ore so I can get some mined before the smelters are built.
If you do this, an important thing to remember (which I often forget) is to build hatch covers before you breach the caverns. When you breach a cavern try to do it via a stair through the ceiling. If you hit a column or wall and for some reason you forget to remove the designation before they reach the cavern floor (I do this a lot), remove the rest of your staircase (the designation, not the carved stairs) and backtrack to the highest hatch blockable cavern access, go to the side a few tiles (I like 5 to leave room for traps and later redesigns) and continue downwards, missing the caverns. That way, you can seal off each cavern individually (building destroyers can not destroy locked hatches from below, though I suppose that could be considered an exploit). It's also a good idea to put a hatch at the lowest level of your working fortress (usually either Z-20 or Z-30 for me), and every 20 levels or so on your staircase, in case of mishaps and you suddenly want the staircase sealed NOW.
It is very important that each cavern is individually sealable. This was my downfall on my most recent fort. It was a haunted tropical beach (with haunted ocean, though the worst I saw from it were the zombie horseshoe crabs that were the greeting party for my starting 7). Through various mishaps, I had a tantrum spiral starting, 40 dwarves (I had very good luck getting migrants indoors), about 5 beds, and no more wood. Right as I was getting more from the caverns, a three tailed fire breathing salamander showed up. I was in the process of individually sealing the caverns, and the hauling had actually started for the necessary hatch cover, but due to the fact that I had idiotically made them out of native gold it was taking forever to get there, and the salamander was fast. I ended up having sealing the fortress basement hatch at Z-30, cutting off all access to the caverns and giving the salamander free reign of the main staircase below the hatch. He got in a fight with a tribe of ant people in cavern three (which had been busy fighting their own zombies after a six legged vomit beast ran into them earlier), and after that bounced back and forth between all three caverns, burning down what seemed like every tree in the underground before finally getting locked in eternal combat with a soldier ant person who happened to have a fungi wood shield and enough blocking skill to be seemingly fireproof, but was unable to harm the salamander with his tunnel tube spear. As far as I know, they are still there, eternally burning down the fungus forests of the third cavern. With no beds for hospitals, half my dwarves babbling (my fortress is littered with the clothes of the insane), and everyone too upset to actually dump the corpses, it quickly crumbled to its end as all the corpses piling up in the entrance hall began to rise. I was quite sad, as I had a fell mood in progress, and I had had an actual kobold invasion (vile force of darkness has arrived), which I had never even seen before. Of course, the kobolds were outnumbered by zombie elephants, but still, it was cool.