I really like making fabulous libraries, with lots of scholars and as many books as possible. I'll even trade for masterfully-written works from traders (one of the very few types of masterful item they deign to bring, interestingly). A few small pig tail fields (plus surface fibers, if you can get 'em) is enough to provide plenty of paper if you have experienced farmers. I try to encourage litereacy as much as possible among the dwarves. Oh, and I fill the libraries not only with bookcases (of course) but also statuary and other decorations that dwarves enjoy.
I do set up taverns...eventually, once word of the library has gotten out. I want most of my visitors to be scholars, not every crappy touring band on the continent (dabbling Oteb player, knows no other instruments, doesn't have an Oteb).
Temples...eh, I usually can't be bothered. Once a fort's pretty mature, maybe I'll set one up if any dwarf I like really needs it, but there are far better uses for their labor, particularly since the temples don't really bring in interesting visitors. Last time I set one up, it doubled as the Live Target Military Training Ground where goblins and such would be released (weaponless, of course) and left for the military to dispatch. There's just a certain je ne sais quoi about a hall of statues of dwarven deities, all covered in blood.