I founded my current fortress deep within the natural caverns. It's actually my favorite kind of fortress to do! For me, the natural subterranean kingdom has a different feel to it. Convenient that it usually has all of the resources you need plus heaps of fertile land for farming.
It -is- dangerous, though, more so the deeper you go. Forgotten beasts can also crop up in the caverns. For me, when I found in a cavern, I do my best to quickly dig out enough space to have bedrooms for the founders, storage space for things I brought on embark, and space for workshops. I'll quickly move everything and everyone in, make a pasture for any animals just outside, and hurry up with securing the place. That early on, I've found it works better to just use engineering than a military, starting with immediately erecting walls at the entrance with retracting bridges to seal the openings. If you can safely entomb your dwarves in their cavern fortress as needed, just about any cavern resident or invader won't be able to get to them.
Early on, my current fortress faced a lot of attacks by cave ogres and trolls, as well as flying heads. They wrecked my farmland more times than I can count. Gradually, though, I worked out how to address these issues with engineering. I walled off farmland in the cavern and made it accessible only via an underground tunnel within the fortress. Voila, no more wrecked farmland, and I can wait out sieges without losing access to food!
So, that's one use for natural caverns: a cool fortress built upon a living environment, like a surface fortress but still underground! Oh, and no worries about being isolated, as caverns and visitors will happily make the trek down.
Otherwise, well, natural caverns are useful for their resources. New resources become available the deeper you go, but the dangers are also greater. I've found they're good sources of gems and metals, and the types of wood and plants you can get in caverns aren't available anywhere else. As has already been mentioned, if you're fortunate, there are cave/giant cave spiders down there producing silk. If you work carefully, you can wrangle them into a silk farming operation!