I've been thinking of "wants to explore sky" as a fantasy character motivation recently.
Also, this post is also kind of fragmented. Also I really like the word also.
And having cool stuff in the sky would indeed be pretty cool, but I don't like the idea of being able to mundanely reach into the heavens with any old fort. I'd prefer the groundling creatures to stay groundlings - keeps the sky as a mystical wondrous blue expanse stretching into forever and such. I would keep sky-exploration as something requiring high-level magic, or cooperation from a powerful megabeast or something. Entirely mundane ways, like Dwarven airplanes or a GIANT Spire, would be fun, too, though, and features like a Yggdrasil-kinda tree or the beanstalks would also be rad.
Also, I don't really like the idea of having celestials require a platform or stuff like that - I want it to feel like the sky, not a bunch of stuff taken from back on the ground, except somehow in the sky. ...Though, I would not be opposed to floating mountains, castles, etc. Someone already mentioned floating rock and metals, and that's cool too - floating rocks could be the foundation for the construction of a flying castle. It'd be nice for it to be sort of HFS-level, too. The only thing about having random chunks of crap flying around the sky is that... well, the sky is empty, and I don't know if I want the sky to suddenly be FLYING ROCKS EVERYWHERE. Even if they're so high up or somehow camouflaged so no one ever noticed them, it'd still be really weird.
IMO there should be two flavors of skystuff: Stuff for groundbound guys, and stuff for aerial/flying/floating/whatever critters. Floating mountains, the homes of Sky Giants (like D&D or Warhammer Fantasy, I guess?) for the former, and maybe homes with flying tunnels instead of floors and stairs, carved out of a floating rock cluster or something for flying creatures.
The sky IS kind of a hostile environment, what with the whole it-doesn't-prevent-falling thing, so it being empty like an ocean would be appropriate, anyway. Maybe occasionally some kind of sky-only thing would fly by (digression: I remember reading about some kind of bird that never stops flying - even the eggs just hatch in mid-air, with the chick flying as soon as it's born in some D&D Monster Manual or something), or maybe it's solely a place for flying creatures and wizards.
I can't think anymore. Here, have this giant post.