You could probably get along just fine without ever opening the caverns on the right embark, and if they aren't very fun for you then you don't really need to engage with them! If you're looking for the fun with them, then there are a few ways to make them more relevant to your fortress.
If you want some extra challenge, then leave your caverns open to your fortress, or only partially sealed off (without cage traps). You could use scheduling to station guards there, and have some kind of cute guard outpost to keep your fortress safe.
I think a lot of this has been mentioned, but if you want to make the caverns useful to you then you can put yourself in situations that make them useful or even necessary. For example, having moss and so on grow on dirt after you open the caverns is definitely useful, and if you embark somewhere without trees then opening the caverns is pretty much necessary. Caverns provide a fast way to uncover lots of ground, so they provide a fast way of mining lots of ore and gems (no time wasted in finding the ore and gems) which could be especially useful in a low metal embark. There's water underground, which I know I usually end up relying on! So there are lots of interesting embark situations that would make the caverns very useful.
In the end Dwarf Fortress is a game where you set your own goals, so you could always make those goals cavern related! Building in the caverns would be a fun one to start with.