I think this page can give you a clear and orderly idea of the different phases of the project so far:
http://dwarffortresswiki.org/index.php/History_of_Dwarf_FortressAs you will see, from the first version 0.21.93.19a (2006-08-08) the world was designed in 2D, the mountains for example were nothing more than "walls".
And a year later, the world turned 3D. You can analyze the different turning points in detail.
Of course, I see that nothing is mentioned of what version 0.47.01 added, which was the addition of villains and their corruption plots (the ability to win allies through blackmail, threats or promises to achieve their purposes), investigation methods of justice such as interrogation, the expansion of religions around the world, the power to create a group of adventurers from the beginning (before you could only handle one) and assign each of them pets or mounts of all kind.
When I was new to DF it seemed to me that its development was a bit chaotic. Often Toady would stop working on something to leave it "half way" and continue working on something else.
But over time I realized that he tends first to design all the possible elements of the world (trees, dragons, minerals, rivers, books ...) at a very basic level, forming a "simple" whole. Once this is done, he goes back to redesigning the various elements adding a new layer of complexity (like moving mountains from 2D to 3D) and adds some new things when these can be accommodated on a solid enough existing foundation. For example, economics will not be introduced until the psychology of dwarves has evolved enough for them to handle it intelligently (you are not going to lend your money to a bunch of crazy monkeys, haha).
It also makes sure that in each version the experience feels as alive and complete as possible. And if for this he has to add random and / or "scripted" elements, he will do it. But what he really wants is to make everything that happens or exists be due to a reason, that everything is cause-consequence. This is the case with merchant caravans, from time to time they simply appear because of your dwarven fortress. But in the future the caravans are expected to come from a city and actually travel a path to get to you (which could lead to bandits waiting to raid them, etc.). Over the years he is making everything feel more organic. Sometimes I enjoy discovering things more in legends mode than playing, knowing for example that the forgoten beast that attacked me existed 100 years before I started playing and things like that.