I keep my population cap right around 40, both to avoid the nobles and to improve my performance. When I let it get above 50 or so, I just wind up dumping immigrants into the army anyway, and before you know it I have 80 shiftless champions wandering around tearing each others' pants off and choking marmots.
After I get settled in to where I can handle raids, have an economy that can buy me metal bars and silk cloth, and start really bending my efforts away from sustenance and toward achieving something is my favorite time, which usually comes around year 2 or 3. Whether I'm putting together an elite strike force to take on the denizens of the HFS or building a novel water feature or engineering some kind of auto-goblin-chopper for the front drawbridge or just trying to make an artificial volcano over my exposed magma pipe, nothing's more exciting then the first stages of a new challenge. Later on I get frustrated and quit, of course.
And my second-place favorite is when bad things happen and you wind up with three or four survivors, who must rebuild from the ground up. I've enjoyed it twice, once due to HFS and once due to a truly amazing tantrum spiral that culminated in my godly hammerdwarf with her artifact steel hammer deciding that everything had to die, and smashing workshops, furniture and neighbors alike. It took some quick thinking and quicker building, but finally I got her locked away with my bait migrant, and she starved. Four new migrants, with no friends (thank Armok) survived the ordeal and managed to keep their morale just north of crazytown long enough to start stacking the bodies outside, restoring the bridges and furniture, and getting the place shined up in time for the next wave of nervous immigrants. Good times. It's like re-embarking, except all the mining's already been done and there are corpses everywhere.