War produces additional temporary need, as does a plague. In both cases, increased need, tabulates to increased rate of technological development. On this we seem to agree.
Even "Peaceful" (cough) tech development is really driven by needs of other sorts, be they petty (like in the space race, where ICBMs shot at the moon were "making up" for damaged self image.) or practical (We need to stay competitive to remain relevant on the world market.)
Necessity really is the mother of invention.
I am not so sure I like the idea of "progress bar" type research though. I don't have a truely better proposal, but I dislike it, for its over use as a mechanic.
I would rather see a top end put on technology, with a modified LOG function for "total knowledge" required for acquisition, all done behind the scenes.
Here's an idea to float:
Theoretically, the more pressing the need for something is, the more people in a civ will rush to fill it. The more people working on solving that need, the more ideas are created, and shared within that civ. The more ideas there are being circulated (even old ones), the greater the probability of a breakthrough. Setting limits for "total number of people in that vocation" before unlocking tech, with the exception of importing that knowledge as an override (books, immigrants, and to a lesser extent, exposure to finished goods.), would roughly approximate this.
So, you end up with a curved graph, starting off pretty flat on the left, but growing steeper as it goes right. (not TRUE LOG though, or else top tier techs would NEVER be attainable! Just "LOG-Like.")
Example of a LOG curve
Treating a book as being equivalent to a craftsman within that skill tree (as a proxy repository of an idea), would make having lots of books on the subject equal to having lots of craftsmen, without the logistical overheads, making it more possible to attain, and would allow knowledge to be directly traded between civs. This means civs would need to make books to retain current knowledge, so that craftsman can focus on 'improved' knowledge. This prevents a civ from relapsing on tech, if all the craftsmen are lost through some event.
Why a LOG curve?
Ok, Let's look at it this way:
I have an embark that I started at year 1, on an island. I wont be getting *ANY* visitors, ANY time soon!
I have 7 dwarves, and I assign 2 of them as combo masons + miners. That is 2 people, theoretically making ideas, and sharing them. The chance of discovering a cure for a disease by chance is not 50,50. It's closer to 1,99999 than it is to 50,50. If we extrapolate that to developing a specific technique, we have a 2 out of (some increasingly large chance) of discovering the technology as an idea, or adapting an existing idea into a new one. Our dwarves are posing as surrogates for ideas, so we (wrongly, but imposing more realistic metrics makes this computationally burdensome) assume that each of them only has 1 idea about that tech inside their little pixellated brains. The first dwarf, at the start of the log curve, holds the idea that if you swing the pick at the stone wall, you can knock off chunks. The second one holds the idea that if you swing it a slightly different way, it knocks off a little bit more that is useful. Etc. The more dwarves you have that are competent in that skill, the more "ideas" you have in circulation, and the more profound the technology available. When you throw in books, which allow a dwarf to write down their idea, so that if they die, you dont lose it-- (capped at 1 book per subject per dwarf, to avoid runaway breaking of the system), and you can effectively double that immediately. The first dwarf writes his idea down, the second dwarf writes his idea down, and now both dwarves can hold new ideas, drawn from the first ones. Kinda like a mood, in that it only happens once, per competent or better skill. Once they write the book, they just serve as a repository for that knowledge, and as a manufacturing cog.
The dwarves at the site procreate, and I assign the child the same labor as my 2 previous dwarves. He reaches competent, and gets his first idea. We now have a value of 5. He book-moods, and now we are at 6. Because we arent getting migrants, our population growth rate is very slow, since we only have 7 starting dwarves (meaning at most 3 pairs, if genders are equally distributed), and it takes 12 years for them to grow up and have children of their own.
Compare with a site that is on the mainland, and is accessible.
We start with the same 7, and select the same 2 in the first year, and get to a tech idea count of 4 quickly. Migrants come, and some are ALREADY! competent at this skill! Their ideas immediately add to the pool, and then they bookmood, raising it higher.
We now have considerably more than 7 adult dwarves, and the rate of population growth is likewise considerably higher. As such, the rate that "New ideas" come into the fortress's tech tree increases exponentially faster than does the isolated island embark. THAT is why we use a LOG curve.
The faster the civ's population reaches competent, writes their ideas down, popps off, and get replaced with fresh recruits, the faster the civ's "Technological pulse rate" will be. This does not yet factor in the distribution of the population doing certain tasks though.
It makes no sense for a fortress to have 199 miners, and 1 farmer. Not unless their economy is explicitly tied to selling raw stone anyway. It makes more sense to have about 5 to 6 miners, 2 to 4 farmers, and lots and lots of other mixed tradesworkers, based on what is available, and what must be traded for. This is why books are really necessary, and invaluable. They essentially allow you to operate as if you have considerably more dwarves on site than you actually do, with no reasonable cap to the number of books you could theoretically store on site. The city of Alexandria was what it was, because it was a massive nexus of stored knowledge, in the form of books, which attracted minds containing new ideas. In an era where people generally believed that the heart was responsible for consciousness, Galen was demonstrating how the brain was actually implicated, through his experiments and exchange of knowledge in Alexandria. That's how big an impact books are.
So,
Let's say you would really like to have steel weapons, because you have iron ore, magma, and boatloads of flux on your embark, but steel is not obvious as a process. You have a few choices...
1) WAAAAY disproportionately pump metalworking until the dwarves learn about steel. (dwarves can be proficient in may skills, afterall. It's just a matter of investing the time and resources to build them up.)
2) Train, bookmood, then slaughter dwarves to get books built up, and rely on migrants to get fresh minds at a reasonable pace to progress, and risk being labeled a deathtrap.
3) Buy copied books from the caravan, at a very high price, and build up a library.
To avoid game breaking consequences, we prevent making copies of books on site in fortress mode, but we might be able to allow that if the player foolishly sells an original book at market. (If we allowed mass copying of books, and used books as a surrogate for dwarves containing skill knowledge, then we allow a runaway process where copied knowledge == new knowledge. This wont do at all, so we forbid copying of existing books on site. Either that, or we use some combination of data flags on copied book items, to prevent multiple copies of the same book to count toward the score-- making the copying and selling of books into a new trade.) This is kludgy, and artificial-- and needs good ideas. Please contribute.
The RNG should weight the distributions of craftspeople in generated civs, based on intrinsic racial need modifiers, and site location scarcity levels. Craftspeople density should only swing above "meeting need", where trade is required to get sorely needed materials or goods. (like dwarves needing wood for beds, but living on desolate mountains) Overproduction is just as bad as underproduction, unless it meets a need tangentially. (Rabbits only birth litters of 12 or more kits, because lots of them die. Animals with lower mortality rates birth fewer young, like humans. Overproduction is bad! You only overproduce when you expect to lose much of the produce, and gain something else by that loss.) Determining how to do that, is not something I have really thought hard on. I'd love to see some ideas.