(OP here)
Rainseeker: It absolutely would work with custom tilesets. I have testing it with Mayday (although the "empty tile" detection doesn't work properly, it's just a hack to get around the fact that I don't have access to the data)
CobaltKobold: if implemented properly, I don't think that this effect would affect the frame rate significantly. Ultimately, you are drawing exactly the same # of tiles as before (though the tiles drawn might be from different levels). OpenGL makes the alpha fast. of course, more tiles that are "active" might have to be drawn, on average.
You could make it just display terrain from different levels, too, which would have essentially no impact.
Untelligent: I like the transition effect, but would be much more interested in just seeing the main idea implemented.
PTTG: sorry, I just wanted a simple screencap to show off the transitions. If I did this type of thing more than once, I'd consider buying the program
All: it definitely wouldn't be trivial to implement this in the game. I looked at the display bindings provided to the opengl speedup build guys, and it seems to be outputting tiles/sprites rather than game data (so no info about other levels could be used). I'm not sure how hard it would be to change the layer above that to provide more/different data. If Toady is interested, I'm certainly willing to put in time to try to make it work.