Rough crystals are a gem that can appear anywhere, and rough gems outside of raw glass cannot be brought from caravans, and I assume raiding won't net any either. This means you can get them only by digging, either at your embark, or a second embark.
I once optimized an 1x1x~14 stone z embark for mostly maximum rock crystals, and
maxed out around 50. But even on 100ish mineral scarcity, the median gem amount seems to be about five times less.
An above-ground fortress....Lets imagine something small, like a depot (stairs under depot via deconstruction trick) with double-thick walls and just one archery outpost:
▒▒▒▒▒▒▒▒▒▒▒
▒,,,,,,,,,▒
▒,▒▒▒▒▒▒▒,▒
▒,▒DDDDD▒,▒
▒,▒DDDDD▒,▒
▒,▒DDDDD▒,▒
▒,▒DDDDD▒,▒
▒,▒DDDDD▒,▒
▒,▒▒BBB▒▒,▒
▒,,▒BBB▒X,▒
▒▒▒▒BBB▒▒▒▒
z1
xxxxxxxxxxx
x+++++++++x
x+xxxxxxx+x
x+x+++++x+x
x+x+++++x+x
x+x+++++x+x
x+x+++++x+x
x+x+++++x+x
x+xx+++╬╬╬x
x++x...╬>╬x
xxxx...╬╬╬x
That's XXFortressXX is going to cost you at least 125 blocks of crystal glass, assuming you use only one with bridge and depot materials.
I.e. it'd take between 35 and 178 stone embark tile z-levels, even when you max out minerals and try.
That's a lot of mining. With correct exploratory mining pattern, you can minimize it. Rock crystals seem to have at least two adjacent crystals in a clusters on a quick glance, so the cheapest would be diagonal ramps with gaps, assuming you have a good macro for it and don't waste much on startpoint - call it 1/16 of tiles being designated to dig. Not trivial, but can be done.
That's still a lot of tiles; 144 per z-level. So between 5040 and 25632 tiles.
For scale, it'd yield you between 3 and 17 legendary miners, and could nearly fit 42-212 11x11 rooms.
For a XXFortressXX.
Slade experiments back in DF2012 era showed few interesting behaviours:
One was that if you have misaligned map block, and paved it with a material, then caused a cave-in, you could get a layer based on the top visible material (so, what you paved it with). Normal stone materials, same materials, but using exotic things like wood one could yield gems.
Not every fortress can cave in bottomless pits, even then.
But with a right wood or soap material, and if it still works, you could get <2292 additional tiles of rock crystals per embark tile. (If you manage to figure out crystal glass, even better, but the memory values make me think that might be not possible.)
Since a grand above-ground fortress uses blocks in five digits, this might be only way to get required amount of crystal glass quickly enough.
You'll still probably have to defeat hell several times before you can even begin construction.
Both of those method have problems, however. So I'd want to embark on lot of AI fortresses for a look first - specifically, looking whether they built their fortress using rock crystals or something, since ores are valid candidate. I'm doubtful whether that'd work, but if gemstone or glass walls can happen it might take you looking at 150ish? AI fortresses to find one that matches a given material. If this works, this would yield you enough to build a small keep.
Oh, and good luck with getting 80 moods for crystal glass beds. Even with save-scumming, going to take many in-game decades.
Overall, crystal glass might very well be most player-expensive material in game, even beyond hidden fun stuff.