Unfortunately I don't have much room to dig from above in order to arrange a cave in. Though I'm not that familiar with cave in mechanics so maybe there is a way and I just don't see it. The layout is like this:
|...............................................
|................................XX............
======..............................=D=..........
B=====..............................=M=..........
======..............................= =..........
where: B -- ballista, D -- artifact door, M -- the forgotten beast, = -- fortification, | -- wall, X -- up/down stair.
Part of the problem is that those stairs are actually part of the central stairway for my fort (oops!). The fortification hallway the FB is in extends much further and normally everything dies before even getting close to the stairwell, so it hasn't been a problem before. I had to seal off the stairwell from 2 z up so the dust would stop drafting into my dining hall, which is only about 5 z up from this level.
Another oddity is that while most dwarves that get dusted become paralyzed and die within a day, there are actually two dwarves on this level that have been paralyzed for several years. They are currently out of sight of the FB, so it's not constantly blasting dust right now, but I know that as soon as I open this level some dwarves will try and rescue the two and end up getting many more people killed. This complicates the death-from-above idea because at least a few dwarves would need access the z level directly above this one, which still has working stairs to this level. Last time I opened it up briefly to try and build a door, 4 dwarves lost their lives because they wanted to enjoy the cool breeze of the cavern, even though everything on that level is forbidden.
I guess I'll just keep training my marksdwarves on it for now, maybe another 20,000 bolts will do the trick!