The HFS is the only reason, but it's a really, really, really good reason. The kind we have is Evil Deadly Murdering Dwarfivorous !!Hidden Fun Stuff!!, the most dangerous of the three possible HFS's. Please don't forget that !!Hidden Fun Stuff!! can fly, can't be hurt by water or magma, and cannot be blocked by any dwarf-constructed object except a solid wall or solid floor.
You probably want to train up a weaponsmith or armorer before using the adamantine anyway (there's a legendary blacksmith, but it's more efficient to have a mason make statues out of raw adamantine - hugely more bang for your buck).
If you want to mine the adamantine, I suggest digging around at the bottom level to find the pits. Draft a peasant as a miner and make a room way at the bottom with bed, food, drink etc. you assign to that peasant, then wall it up (with actual walls, not doors or floodgates!) so there's no path from the room to the outside world. Then just have that peasant mine non-adamantine squares until he finds the pits and gets eaten by HFS. The advantage is that once you reveal any one square, the entire pit becomes visible, and you know that any adamantine not connected to that area is safe to mine out.
As long as you make sure not to release any !!&!!'s into the fortress, you can mine out as much as you want - just remember to leave some ore for everyone else to play with, too!