I think Muz is on to something here. It seems kinda like reverse capitalism.
Rather than producing something other people want and them giving you money for it (what you want), Toady seems to be simply producing something he wants and the consumers happen to pay him. So Toady just does what needs to be done because he is doing for himself, and we get awesome out of it.
Now, as for the development process itself, that is hard to get a bead on unless you are in the game industry, which I am not. BUT I do get to watch the people in the game design major do...whatever the hell it is they think they are doing. And it isn't pretty sitting over here on the computer science side of things. Basically, they have no idea. The teachers, who are purportedly from the game industry, also have no idea. No one knows how to make a game; I don't know how to make a game. Literally the basic plan is this:
1) Pick an idea out of friggin' nowhere
2) Write an elaborate and hefty document that details everything you want but give no one any idea on how to implement it, and certainly give no thought to the notion that it has to be implemented
3) Make game
Because of this, if you will notice, the credits for a game are usually one-sided: you've got a staff of thirty 'artsy' creative types making art, levels, etc., and a small nucleus of only a few programmers doing the actual engine writing part. NOW, I am not saying that art doesn't take a long time and a lot of people, I am the art head for a game project. But it is not a good way to design a game when the largest portion of the staff doesn't know Ruby from Python. Hell, it gets worse when the two sides don't talk to each other.
Why? There seems to be an artificial divide in the game dev world between the expressively savvy and the technically savvy. Why do so many programmers dislike UIs? Because it is expressive; I know when I am in programming mode, I am totally like "why can't they just friggin' use the command prompt for everything?" Same thing for the artsy team: the programming is a technical mire of complex inter-relationships that express nothing, they only allow things to be expressed.
The sad part is that the two sides really don't know how close they are to each other. Your programmer can write the skeletal animation code that makes a mesh move like real muscles and the rag doll physics to make the body move like it was real, whereas your 3d artist can make that mesh look and move well because he knows how muscles work. In the end, it is really the same thing and the two have a lot of common ground, but they don't talk about it.
Toady gets around all that BS simply by having only himself and no bloody publishing company telling him what he is doing. The creative team and the implementation team are sitting nice and cozy in one head. Well, plus ThreeToe, but they're brothers. Whatever magic is going on in that pair's craniums is beyond any class anyone can take on game development, because it can't be taught. They didn't seem to start with much of a big plan, only the idea "let's make a game, the kinda game we like to play." It is like Tolkien and C.S. Lewis's thing about "writing books we like to read because no one else writes them." I am not trying to draw comparisons on awesome here, only methodology.
Hell, the first thing they tell you in game design is, basically, pick a genre. Both Tolkien and C.S. Lewis and Toady and ThreeToe took that very notion of genre and said "fuck you." Is DF an RPG or a city sim? Is LotR a fantasy novel, or an epic saga*?
The really interesting thing is that somehow by ignoring the presentation of the game, Toady has made the presentation the forefront of development. Freeing the game from the framework of a genre or an interface idea or a specific 'game mechanic' lets the game be whatever it needs to be as it evolves. That is the only lesson I can gleam from bay12 that can be written down on paper.
*I am talking literature here, not praise.