Make sure that your fortress can be sealed off from the downward stairwell, and make sure the stairwell can be sealed off from each cavern individually, without restricting traffic downward.
A good option for sealable access is floor hatches. Locked floor hatches can not be destroyed from below, and can be locked instantly by you, rather than relying on a dwarf to pull a lever. If you think that's too exploity, use lever controlled raising drawbridges.
It's good to place traps in the cavern access. I like to place quite a few cage traps, because cavern creatures can be useful and/or valuable, but can also come in good sized packs (elk birds, naked mole dogs, and crundles are common examples), and also a weapon trap or two beyond the cage traps in case they overload the cages or I accidentally fill all my cages).
If you wish to use a section of caverns frequently, as a tree farm (or normal farm), find a large open area with narrow chokepoints and wall them off. Make sure you block all map edge access and completely enclose or wall off water, in case of amphibious attackers (if you want to fish or gather water, use grates. Like floodgates, they are indestructible from underneath. Try to choose an area with a low ceiling, as you will eventually want to extend your wall upwards to block flying FBs.
As for the advantages to caverns, many have already been mentioned, but:
-Forests to chop if you need wood and the surface is inhospitable (active siege, or 1000s of zombies, or what have you).
-Plants to harvest if you need booze and you for some reason screwed up your farming
-Open space to plant farms
-Silk to harvest if you need cloth quick
-Animals for hunting and trapping, many of which are very useful especially cave crocodiles (by far the best egg layer, producing stacks of up to 50, with a high value. If hatched, they produce valuable (and copious) meat, leather, and bones, and are one of the best guard animals, since they are extremely renewable, physically imposing, and naturally aggressive, offsetting the inability to train them without modding) and giant cave spiders (infinite high value silk to keep your dwarves delirious with joy at their swanky clothes, the ability to trap [TRAPAVOID] creatures, and pretty impressive as guards).
-A huge amount of free exploratory mining. If you want gems, the caverns are by far the quickest way to get them.