I generally leave it on, but I have been VERY tempted to turn it off.
My BIGGEST and most significant problem with it is that it's an INSTANT fort-destroyer way too early in the game. I can't even hollow out a little hovel in the ground without spawning anywhere from 3 to 10 rock wraiths (my dwarves are usually not even equipped to handle 1). They are NOT rare, and even if you could see the stone they'd spawn from ahead of time, it makes it so you have to start and stop and mine out a tile or two at a time. Far too tedious.
I am pretty much guaranteed to have all my miners die the second I get the bright idea of mining out a 10x10 or 15x15 space in stone. I've tried to "prepare" for this by giving them expensive, quality battle picks and assigning them iron-clad war beasts as guards and the wraiths simply rip through them all like a knife through butter. Within seconds, the wraiths are up and murdering everyone else.
I don't find this element 'fun' because it is very difficult to prep for and makes something that should be early game seem like a late-game feature.
---
COAL DUST - I'm okay with this feature. I DO wish that the delay between encountering it and dying from it was significantly longer, gave off symptoms, and that it was at least treatable with special care. That way, I could approach it like an engineering problem and rotate my dwarves without being resigned to making everyone a mason guild dwarf within my first two years. As it stands, I'll get the message "You've struck **** dust!" and I know, "Welp! There goes THAT dwarf." *waits 10 seconds for the inevitable near-instant suffocation*