Subterranean moors and bogs! The idea of it is ultimately to satisfy a worldgen hurdle that changing the cavern resources to include many more barren places would entail - there'd be worldgen rejections and logged errors from dwarves not having a place to farm underground.
On a technical level, indoor-farming entities could change the tiles underneath their starting sites into a biome with cave crops to prevent rejections. Flavorfully, they would engineer cellars and tunnels to bear fruit, transplanting dirt and organisms to jump-start a mycorrhizal network even if the biome mechanically just lets the plants and creatures spawn by virtue of it being a habitable biome.
Later on, moorcaves can be generated from the thriving caves we see currently being exploited by indoor farmers, unable to withstand the ecological shocks.
Should be ambiguous from which direction dwarves originated - if they dug down, or pierced the surface.
Since there's lots of decaying matter, it should be easy to find organic remnants such as lignite or petrified wood in a cave bog that's existed since year 0, while the artificial ones wouldn't be made on a geological timescale. Decaying matter without an output valve for acid such as a river could carve beautiful karsts in carbonate minerals, perhaps even than abiotic ones. Not to mention acidic aquifers, posing a challenge to chemically isolate (especially when it cuts into your lime reserves).
Acidity as a general worldgen parameter would be interesting, but I don't know enough about it other than to just wager to start with acidity from minerals/noise, make a second acidity parameter influenced by water/rivers/drainage & biota. The mineral acidity is to preferentially spawn minerals that cause acidity or alkaliness with the effects being more noticeable in sedimentary rock, and the water acidity is for the !!fun!! stuff.
Back to the fantasy stuff! Ice caves are a classic trope, frozen parts of the world tend to have these closer to the surface due to the similarity, while in other parts of the world they're found much deeper. They'd be cold enough to drop snow from their aquifers, and nether-caps would be an inexorable part of this locale. Like Eric Blank's idea for a soil-filled cave, much of the non-rock area could be filled with ice and dotted with tunnels.
Directly transplanted overworld biomes, complete with all their flora and fauna mysteriously surviving in the darkness.
Re: transplanted biomes; such a biome should be Savage in spirit, if not already so due to the way that caverns spawn all creatures regardless of alignment.
A more digetically underground desert would be fun, themed around dryness and being composed of minerals like rock salt or sandstone. Tunnel tubes are already the caverns' idea of "cactus", and plenty of sand to haul down to the magma smelters. Even without quicksand, some of the rock should have a high risk of caving in. Should follow similar spawning logic to the ice caves, where places already cut out to be a sandy desert get these shallowly, while it can also appear as part of the unexplainable bottom of the world.
Fungi tend to dominate semi-realistic fantasy cave ecosystems, but what about a cave dominated by some other sort of organism? "Nest of swarming bugs and their eggs" is a classic, slime is another one. A warren of underground hares might even be scarier than you expect. Creatures tagged with this biome token are just co-inhabitants, like rattlesnakes in gopher tortoise holes, while another tag allows such a creature to become dominant in "nest" biomes. It could even be done for a creature class rather than a single species, a spider nest could be populated by more than just GCSes and have the entire spider clade brood there.
And as for terrain features, dirt walls! Deep Rock has them, Minecraft Ideas Academy recommends them, lets you partition off sections of the cavern and have some exploratory digging without it being strip mining. Especially useful for spelunking in adventure mode, where I'm assuming mining through stone is going to be on a much slower timescale.