In addition to everything in the FPS article, try to keep your fortress space compact. If dwarves have to trek halfway across even a 4x4 world on a regular basis, you're going to take a hit. Take the extra time that first year to found your fortress where you want to build.
This is the only reason that I use reveal.exe. The wiki utilities page can tell you how to get and use it. Very useful when I want to embark on an underground river or magma pipe, but don't want to spend years trying to find it, especially when it could turn out that I need to move my entire fortress to increase FPS.
Don't make huge rooms unless you use d-o to create high traffic lanes.
If you want an underground river, dam it up right away and don't let the cave creatures breed out of control. Killing them off and damming the river helps immensely. (To dam an underground river, collapse a ceiling of the exact right size into the water.)
If you want magma, turn off temperature. Sure, it's cool to toast enemies with it, but if you're having FPS problems you shouldn't be pumping it much anyway. You can still use it to process goblin gear. Just save, turn on temperature, reload for a couple minutes, then save and turn temperature back off.
Animals are fine to keep caged as long as you're not breeding them. 1x1 pits let them keep breeding, but whenever you butcher you'll probably let them out inadvertently. Try putting the pit in a room with your butcher's workshop. Let the butcher go in to do his work, and lock the door behind him. That will help him work faster and not have to go chasing animals halfway across the map.
When you do exploratory mining, arrange the shafts so you can wall them off later. That way you don't have dwarves and pets pathing down there uselessly.
Force caravans to enter the map at a single, nearby place. That prevents them from pathing from the opposite map edge.
I don't find that high cliffs cause that many problems. Nor chasms, once you clear out the creature that live there.
HFS related:
Don't open up the demon pits until you're ready to handle them, if your map has adamantine. The "sneaking" mode that they start in eats cpu cycles in a big way.
There's more, certainly, that you can do. Most of it won't have as big an effect as turning off the temperature, using (at max) a 4x4 embark tile, keeping pathfinding short, or keeping the population low.
Edited to add more from my personal experience. Amalgam beat me to the wiki article.
Another Edit:
If you don't mind having vermin and keep you dwarves generally happy in other ways, don't bring cats. If dwarves bring male cats, fine. If they bring female cats, arrange an accident for the feline friend. It's worth it for framerate.