Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[NW_Kohaku]

To start from the broadest objective, and work inward, the purpose of this thread is to create a well-formatted, organized proposal to solve a tricky problem in what Dwarf Fortress currently lacks as a game, and how I believe I have found a way to correct these problems. 

Note: For the sake of your sanity and eyesight, I reccomend cranking page zoom up to at least 150%, if not 200%, to make the text a little easier to read, and leaning back a bit and relaxing.  Smoke 'em if you got 'em, this is a long series of posts.

Oh crap! This thing hits the character cap for several posts?!  Screw that, give me the tl;dr version!

Just click HERE for the tl;dr version!  Please comment on the pictures!  Tell me what is confusing, what you don't like, and what you think is easy to understand!

The original thread can be found here, however, the reason that this thread had to be created is that the arguments I am going to outline start to really take off after about 20 pages of text, and continue for about 25 pages after that.  (The major idea that this whole thread revolves around starts on page 31, however.) People quite understandably did not want to have to slog through the entire evolution of the idea when it took place over essentially a hundred pages worth of text just as a means of entry to the argument. 

I also want to say that, although what I am advocating for is "my idea" or "my vision" or whatever pretentious label might be slapped on it by myself or others, I want to recognize that there are dozens of people whose ideas I pondered over and built upon in creating this.  Not just the likes of Silverionmox or Draco18s, who came up with the basis of this idea, but also the many people I argued with, some of which quite heatedly, and in threads in different forums.  I remember that at least one person (I'm sorry, I can't recall who, exactly because I'd have to track down what thread had that particular argument to find out,) had felt that I was not listening to the arguments that were arising around what I was advocating.  This is not only not true, it was actually the arguments against what I was proposing that did the most to build its later, better-thought-out forms.  Argument, even heated argument, is a powerful force for creation, as there is no better way to find the weaknesses of your ideas than to expose it to your harshest critics, and see what portions of your ideas stand on their own, and what gets holes punched into it.  Unfortunately, this person felt I was ignoring him or her because I was pointing him/her back to a previous argument and response session, as I felt it was a topic already covered, and that I had already responded to his/her arguments.  This is why this reboot is necessary, to make clear what I am (and am not) arguing for. 


The Problem:

Spoiler (click to show/hide)

First, I need to explain why I felt the need to expand the scope of the solution to be greater than the original problem.  This was because, in part, I recognized a problem of greater scope, and saw that farming was the perfect way to tackle the problem. 

Basically, I noted a few months ago that there were essentially only two ways to really play Dwarf Fortress, and this was to go full-scale into the military aspects (which meant modding in tougher and tougher monsters, or restricting your means of dealing with monsters until the game still posed a challenge militarily), or to go into exploring the game as a physics sandbox, and create massive super-cities like Undergrotto, or else to build water-based computers out of gears, pressure plates, water, and logic gates.  Why do we only play these two ways?  Because just "playing the game" was not sufficient.  You see, the game is "hard" at first, but this is only because people don't understand what they are doing because there is no tutorial, and smash into the "learning cliff".  Once you've scaled it, Dwarf Fortress is fairly easy.  You just need to set up what basically amounts to a wall between you and the nasties outside, and then have a farm, still, beds, and whatever other amenities you deign build for your dwarves.  Once you have cleared this basic hurdle to immediate survival, the game throws very little at you as a growing challenge.

Now, I have already done a suggestion that deals with this problem from the perspective of trying to make late-game content more challenging by making dwarven society more complex the larger your fortress becomes through introducing social stratification and a need for luxuries to appease a higher class of dwarves.  This, however, takes on the same problem from a different perspective.

Here's why this problem needs to be solved from the perspective of "Improved Farming":  At first, the suggestion was to take up more space, time, and dwarves per unit of food that you need to put into generating food.  Early on in the Improved Farming thread (referred to from here on as IF), Footkerchief hit on the core of the problem with this solution, when he said that the real problem was that farming was a "free stuff button".  You see, as long as farming was nothing more than zoning land, and designating dwarves to throw seeds at the zone and eventually pick up all the stuff that grew out of that land, then we aren't really changing anything by requiring more dwarves or more time to do the work.  If you require more land to produce the same amount of food, the player just zones more land (and eventually adds more dwarves to the farming labor to manage the sheer acreage).  If you require more labor activities for dwarves, then you just wind up throwing more dwarves at the problem.  If you require a longer growing time, you just double the  farmland for every time you double the growing time, and wind up with the same amount of food per unit time.  Either way, all this really changes is what percentage of your fortress's population is going to be farming at any given time, and doesn't do much to actually change gameplay or make it more interesting.

Then began the process of trying to add more to the process of farming, such as adding nutrients that must be managed, so as to make the game more realistic, and then the outcry began because many people could not see this as anything but micromanagement.  They didn't want more complex farming, and they didn't want to think about farming at all.  Cries of "if you like farming, just play Harvest Moon" arose, and they basically didn't see anything wrong with farming being simple, because if you only play DF for the bloodshed, why would you care about farming?

What we need to do is not to just change the way in which the player is asked to solve the problem, especially by just making what the player does have to become more complex, or to front-load everything onto a player so that we "add another hour of setup time when you start a fort."  We need a change in philosophy.  We need to reconstruct this portion of the game to make the player tackle the entire problem from a different perspective, one that should hopefully be more rewarding, and hence, more fun, not merely "making farming harder" or "making farming more complex". 

Physiocracy:

Spoiler (click to show/hide)

This is obviously a reference to the philosophical movement, the Physiocrats (without going into too much detail, these were the first economists, who theorized that all wealth was derived from land, and that therefore, land was wealth, and eventually gave birth to lassaiz-faire economics), but I'm only using it fairly loosely, here, to help illustrate my point. 

Basically, I am arguing that the game needs to recognize itself as Physiocratic in nature.  Like with an RTS game, or like with farmers in real life, players need to handle their environments with some respect for the idea that they cannot just have everything for free, that their resources are limited, and therefore need to be spent wisely to return the best profit.  And, like with the Physiocrats, your primary resources in the game can essentially be seen in terms of the three basics of dwarfiness: Soil, Stone, and Dwarves.  In order to make a game with serious choice and decision-making involved, all these things must have some sort of finite limit on their potential. 

Now then, I'm throwing around a lot of abstract terms, so let me bring this down to earth a little to illustrate what all this means. 

First, I want to go back to the whole "free stuff button" problem.

Currently, stone is effectively free stuff - you almost always have far more of it than you can store in stockpiles or get rid of, and therefore, you will use it without even considering it a cost - it's free!  Anything you can make from stone, you make it from stone, because stone is free - free rock mugs, free statues, free tables, free chairs, free doors!

Currently, wood is limited in supply, but still fairly cheap - you can make a huge variety of things out of wood, but really, you tend to stick to barrels and bins and beds, and maybe some charcoal, but not too much.  Wood is something you can't just consume completely without any care in the world.  You have to put at least some thought into conserving it for the things you really do need, because you only have so much of it in any given year.

Currently, however, steel and to a lesser extent, bronze are relatively difficult to make.  You need fuel, metal that you have to find and are in somewhat limited (although still fairly abundant) quantities, and you're prone to supply shortages.  You can't just build steel chairs for all your dwarves, you have to really consider what your priorities are for steel. (And also note that steel and wood are both inifinite, they're just limited by the amount you can harvest per unit time.)

Of course, finally, farmable biological materials are free stuff.  This means food, cloth, dye, and basically anything else you can get away with growing on a farm.  Just throw seeds at soil, designate a few farmers, and watch the limitless resources pile up.  I even run into more problems accidentally designating too much farm, resulting in food that I can never store or even harvest properly.  (It does keep my children and nobles occupied harvesting it all, though...) I produce clothing, and use it as my primary export fairly frequently when playing, if only because I like to play at economics, even though the economic model of this game is broken, and the clothing mills are functionally expandable to an almost unlimited degree.  I can basically buy an entire caravan with one bin's worth of clothing, but wind up just giving away five or six of them, just to reduce my load of exportable materials. 

Frankly, these are the ONLY concerns about resources/materials that you really have in Dwarf Fortress.

So yes, you can add a few more jobs onto the processing part, or a little more land onto the farming part, but it doesn't change the fact that it's still basically like glass when you have magma and sand or like stone - it's free stuff, you'll never run out of it, so there's no reason to ever consider it.  Food is a default, automatic, easy gimmie, and will continue to be even if you have to up the number of dwarves working - most players just wind up killing off dwarves that they wind up have idling because they can't find any jobs for them, anyway.  Simply assigning a larger proportion of their labor to farming is no difficulty, and if their entire supply chain is automatic and thought-free, then the problem will always remain.

This is why we need to move to a new philosophy on resources, so that there is a real decision for the player to make, and for there to really be something there to manage.  We need to make materials have some sort of meaningful cost. 

The best way to do this is to make farming in-game more like real-life farming...



A biology teacher I had in college once described to my class that farming was essentially just a man-made ecosystem.  You harness nature so as to produce the materials that humans find valuable are created as efficiently as possible.  However, it is still an ecosystem, if less of a self-contained one, and ecosystems are ruled by Conservation of Mass and Energy.

Ecosystems generally contain the same mass that they have contained for thousands or even millions of years (excepting obvious eroding ecosystems, like rivers), the same chemical components and atoms generally occupy the same soil, turn into plants, get eaten by animals, are excreted back into the same soil again and again and again.  Ecosystems, meanwhile, pass energy through the ecosystem through this same mass that is always conserved - sunlight is converted into usable chemical energy by plants, and as various animals and decomposers pass the energy on through consuming the previous holders of chemical energy, that energy is eventually lost into waste heat.

Farms, again, are just ways of converting soil nutrients into materials that contain chemical energy (food), which can then be used (eaten, or used as cloth or other biological products) by the farming species.  We then burn through the energy in that matter that was once soil, and eventually, it is reduced back to a state where it is ready to be turned back into soil by decomposer organisms (urine, feces, and decaying corpses).

The problem with farms is that we remove that mass from the ground, and then we don't always replace it.  Food is removed from the area where it is grown, and needs to be replaced in the form of fertilizer.  All the mass that is taken out of the ground must be replaced.  This means that you need to manage your farmlands and lands in general from a perspective of having to balance out the ecology of your land, making sure you replace anything you take out of the ground.  That is to say, very roughly speaking, you have to recycle the same "mass" over and over again.

To a certain extent, I know that I'm going to get hit with a "dirty tree-hugging hippy" (or at least "undwarfy" or "elfish") stigma for talking in terms of ecology, but there is a huge difference between being a "dirty tree-hugging hippy", and being a farmer who cares about making sure that the soil will be capable of providing him with food come the next year.  It's not for love of every living thing on God's Green Earth that a farmer plants a tree to act as a windbreak to prevent soil erosion from taking his soil away.  It's because his soil is his most precious resource, and the basis of his livelihood, and as such, it is something he has to build up and protect at all costs. 

This is the way that I want to change the paradigm by which farming is viewed: You have a limited amount of useful mass, and as you use up nutrients in the soil, you need to replace them somehow.  You have a finite amount of ways to do this, but this is still a renewable resource.  Your job as a player is to manage your resources as well as you can.  You need to keep your crops watered, which requires access to water, obviously, but also water in greater quantities the more that you grow crops that depend upon water.  For many locations that you might inhabit, there is a finite amount of water that will enter the map per year, and you have only that much water to meter out to all your agricultural projects and whatever water-drinking your dwarves or livestock will need to do.  Assuming we have feces and urine, this will be a major way to "recycle" your soil nutrients - if you eat your food, and it passes back on out the other end of a dwarf, then you can compost that into more usable soil, but this shouldn't be entirely perfect, and there will always be goods that you trade to the outside world that will represent a net escape of nutrients from your soil.  You will need to use some methods of adding a little extra nutrients into your soil.  Dead bodies, amusingly enough, are a great additional source of nutrients.  Protect your dwarfy environment: compost those goblin corpses!  Still, you can probably only kill so many things for the purposes of replenishing farms.  I'll go into all of these different things in much more detail in their own sections later on.

To go back to the "three dwarfy resources" of Soil, Stone, and Dwarves, this is about making soil a depletable, yet renewable, resource.  Stone is already a depletable resource, if a very plentiful one.  Dwarves need to be made more rare and valuable as a resource, but that's what my other thread on social structure is for.  The idea is that if your soil is not something you can take for granted, you will then have to be forced to actually budget what you do with it.  Similar in effect to an RTS game, if you can build just about any military unit you want, but you are limited to only x amount of minerals and y amount of vespene gas per unit time, then you have to pick what units you build with that limited amount of resources per unit of time.  It's not a matter of "making farming too much micromanagement", it's a matter of "making farming only give you so much, so you have to prioritize what you want out of it". 

The Solution (guidelines):

Spoiler (click to show/hide)

I'll start off by saying that those who were arguing against some of the previous suggestions in the IF thread were correct in that we should avoid needless micromanagement, especially with anything that can be considered extremely routine.  My general philosophy regarding gaming is that if you can easily reduce some sort of routine function down to the sort of "If ____, then ____" type of statement that you can build a script to play that portion of the game for you, then it's probably best if that part of the game actually was played for you. 

The basics of the system are that we introduce a soil nutrient mechanic.  This means that soil will have some hidden numbers associated with it that indicates how much of various nutrients (the NPK "macronutrients"), water, acidity, and potentially other numbers.  These numbers will be adjusted up and down as crops are planted and grow, and generally speaking, you will need to reset the numbers in the soil to certain levels before you can grow the same crop again.  This can be achieved either by planting another crop or set of crops that will push the numbers back to where they came from, or it can be accomplished by adding something, like adding fertilizer or water, through the actions of a farming dwarf. 

This will, however, not be how a player interacts with their farm in general.  This sort of routine "push the add fertilizer button to not die" mechanic should be handled by the dwarves and AI routines, with some guidance from the player.  There needs to be an interface that allows the player to set guidelines on what sort of fertilizers the dwarves can use, and how often it can be used, or some sort of general upper limit per year of fertilizer use to allow players to manage how much of your finite materials can be spent per area. 

Because different plants need different nutrients, and because some plants can actually restore certain nutrients to the soil, this should have an effect of at least heavily encouraging, if not demanding, two-cycle, three-cycle, or even four-cycle crop rotations.  In general, real medieval farmers, even if they had no idea how or why anything worked, had a very clear understanding that maintaining their soil fertility was of utmost importance, and had developed over the course of generations very strict guidelines for how to treat them.  As time went on, societies went from simpler two-cycle fallow-planting cycles to more advanced cycles that allowed the farms to grow nitrogen-fixing plants that would both renew the soil, but also provide an edible or sellable crop while preparing the soil for the next planting of their prime cash crop. 

Players should plan out a cycle of crops on several fields, set into the farming interface (which I will describe later) the plan that they have for their fields set on repeat for the next two to four years, and then be able to let the dwarven farm overseer by-and-large handle the rest of the day-to-day work.

With that said, unlike most of what is already in Dwarf Fortress, this should not be entirely front-loaded.  In fact, I've spent a considerable amount of time working out how to make farming no longer a concern only for the very first few minutes of gameplay, and then a total non-issue beyond the initial setup.  In fact, as a goal of this whole project, farming should be either easy or entirely optional for the early parts of the game or when you have few dwarves.

The entire point of this suggestion is to make players have to make decisions about how best to manipulate the ecosystem around them so as to gather all the biological materials that they need for their fortress.  Farms are, as I said before, man-made (well, dwarf-made) ecosystems.  They are demonstrations of the knowledge, power, and mastery dwarves have over their surroundings.  For a small number of dwarves, simply hunting and gathering should be sufficient.  For a small number of dwarves, crop rotations and major farming operations should not be necessary.  As part of what I suggest, Plump Helmets, the classic dwarven early farming staple, should require nothing more than a log and water and some time alone to grow.  This means that you are functionally doing is just converting what was already growing in the above- (or under-) ground soil naturally as a tree (or tree-like mushroom) into the ready-made farm for your classic plump helmets.  Logs are instant farm material, just add water.  The problem is that as your fortress grows, it will become more and more prohibitive to cut down trees for every single plump helmet you grow.  Likewise, you can't feed a major fortress on just hunting or herbalism, as the soil should only provide for so many random plants, and only so many wild animals should be willing to come within dwarven lands.  The larger your fortress becomes, the more you have to stretch the basic resources of the soil, and the more elaborate your systems for doing so must become. 

To reiterate this point, this means that starting farms for plump helmet growth should be a fairly simple matter of "plump helmet farms need one log and water to build".  Water, which I will go into more detail on later, should not require the sort of mussing around with that it takes now, but instead be available by bucket-brigade dumping, provided you have access to water and a bucket.  Logs are already fairly easily understood and commonly available.  Hopefully, the micromanagement of having to re-designate trees for clearing, and instead setting a zone-based forestry system can be implimented, and I suggested that already, and will do so again in a special "lumber" section of this rant.  This means that making early farms should not require any kind of special setup or non-intuitive setups that punish "teh noobs".  It should be fairly straightforward, yet at the same time, inefficient, and not sustainable for larger, long-term settlements.

As players make larger fortresses, they will start to run into the need to fuel their agriculture with other forms of fertilizer.  Trees don't grow on trees, you know.  In fact, growing trees by farm is a potential goal of this, and growing trees plus crops to eat by feeding them the trees you previously grew makes for a rather pointless exercise as long as this game is serious about keeping track of conservation of mass.  Hopefully, the soil of the entire map will be tracked, which means that if you plunder the natural resources heavily enough, you can deal lasting damage to the landscape if you are rapacious enough in your resource consumption.  (Which, I am sure, will suddenly make a prime goal of many players to make the most blasted Hellscape out of their controlled territory.)  This means that, as the amount of resources that players wish to draw out of their soil increases, the degree of elaboration in the systems must rise to make the greatest possible use of their soil.  As with real-life farms, unchecked natural growth will be curtailed, and dwarves will exert more direct control over their landscape, so as to ensure that soil nutrients are being used as efficiently as possible.

For example, manual bucket-brigade watering systems will eventually need to be replaced with a serious irrigation system.  Again, some crops will take more water than others, and some embark sites will have more water to start with than others.  There should only be a finite amount of water that enters a map per year or so, which means that we will need to rethink the way that water is added to the map in this game, but I'll cover that in its own section.  For now, let us just say that there is a finite amount of water, and the more your farms expand, the more water you will need to draw into your fortress, which may eventually become impossible in certain dry embarks, forcing strict conservation of water, or in other, wetter embarks, will at least force more sophisticated methods for intaking large quantities of water per year to your farms.  (Don't forget this means that your waterworks will become a potential vulnerability to be defended.)  There is also the problem of the size of your farms - small farms are a small amount of land to cover by bucket brigading water from a well to the desired farm plot.  The larger the farm becomes, the more sense it makes to develop a real irrigation system.

Another problem you will need to handle is fertilization.  Crop rotations are nice, and will probably be implimented fairly soon to conserve on fertilizer, but crop rotations alone are insufficient.  You need regular fertilizers.  This means manure and other biological fertilizers, including compost made of some of the various dead things your fortress will inevitably pile up.  Fish bones are a prime source of soil nutrients.  Grind them up, and throw them into the soil.  No part of the buffalo goes to waste.  You can also grind up certain choice stones, like apatite or phosphorite, and use them as an inorganic fertilizer, which is actually what we do in modern real-life farming.  The problem with this is that it is non-sustainable (both in-game and in real life) as there is only a finite amount of this stuff lying around, and you can eventually run out of it.  I'll go into this in much more detail in another section, however.

Ultimately, other problems may come up when you do start seriously remaking the entire landscape in your own (twisted) image, especially considering as later seiges will probably force you to completely seal off as best you can your fortress.  Pests are a major problem in single-cash-crop farming.  The more you remake an environment to serve as a place for only a certain kind of crop to thrive, the more inviting a target you make your farms for pests, whether they are insects, fungi, bacterial, or otherwise in nature.  This can serve as a functional limit on the amount that a single type of crop might be relied upon, lest you invite disaster.  This means you need to build into your system a greater variety of crops with a greater variety of needs as you expand your farms. 

Depending on how many improvements to the game that we can put into the game in the package of "Improved Farming", we may even have more complex implications, as well.  We could, for example, have an air quality feature, where fortresses that try to seal themselves off from the world completely will suddenly have problems keeping enough oxygen inside their fortress for their people to breathe.  Oxygen-producing underground plants would then be a solution to this new problem, if you don't want to punch more chimneys into your fort, but they will need to be fed with nutrients, as well, and edible mushrooms that are more easily grown underground may take up oxygen, rather than produce it, and oxygen-producers may not be edible, while marginally-edible crops may split the difference by producing some oxygen, but at a cost of not being very good crops.  Thus, a choice is forced on the player as to how to balance the conflicting goals of wanting to eat, wanting to breathe, and wanting to not open your fortress up to the surface where they will get picked off by attackers. 

When speaking of choices, there is also the choice of what, exactly, you want to grow in your farms, as we can expand what can be grown on farms.  One thing I want to do is make livestock eat food (which means either grazing or growing feed for animals rather than dwarves) and drink water, as well as drop manure (let them graze on fields that "fallow" with some sort of grass, and they will spread that fertilizer/manure across the fields for your dwarves).  If we can grow trees on farms, or even fruit orchards, then you have choices to be made when talking about balancing nutrients to go to growing your own supply of wood (for building or for fuel or even chemicals from potash), fruit from orchards, food crops, meat from livestock, fiber materials (for making rope, clothes, etc.), dyes, and basically everything that you use in the game besides that which comes from stone or ore.  This is to say that it accomplishes what I said back at the beginning of this post, where I talked about Soil, Stone, and Dwarves as the three main resources by consolidating everything biological down to being purchased from a single resource, soil nutrients, which you must manage and make meaningful decisions based around ensuring that you will have enough soil nutrients being used to produce all the biological materials your dwarves will be using.

To recap this whole section, I do not want to "make building a farm more complicated".  I want to make building a farm more simple, actually.  I just want to make expanding a farm more complicated, and, for that matter, make it necessary to expand from just your single starting 5x10 farm that we now assume is the only thing we will ever need.

Given how some people have responded in the past to this idea, and how some people seem to just like not having any real systems for needing any real social support structure for the war efforts they produce, a choice that some people may seem to prefer may be to simply build a "lean, mean machine", where they have very few dwarves, and survive on little infrastructure or need for complex farms, and maybe just trade for food.  That's fine, too, so long as you keep your fortress fairly small, and military-oriented.  In fact, I want to discuss this more in the "Working with other suggestions" section.

Ecological Diversity and Underground Ecology:

Spoiler (click to show/hide)

One other major goal, which I feel should be part of this "first post" argument, is that plants should become far more diverse.  Not just in terms of a "farm mod" where there becomes a truly massive chart of plants that can be grown, but having serious ecological diversity, where tropical regions have tropical plants, and arctic regions have arctic plants.  Basically, when a player embarks in a different biome, they should be able to expect an entirely different set of potential crops to be available to them, so that every fort feels a little bit different.  These plants should not just be aesthetically different, either, but have substantial differences in their growth times, water and nutrient needs, and sensitivity to climate and potential pest problems. 

Underground, the ecology may not become all that much more diverse than its already bizzare ecosystem already is, but there is something that needs to be seriously addressed in the matter of how the ecosystem works to begin with.  So far, plants (or rather, mushrooms or whatever they are) just grow any time there is soil.  There will need to be some sort of source of energy for the underground biomes.  Yes, "magic" is sort of an answer, but there needs to be some serious rules for what makes the magic operate, and why and how the plant-like life can harness it to grow.

In fact, having a rule system for what makes whatever magical system grows the things that grow underground might even make for a dual-biome system, where there might be underground climates that change based upon abundance or scarcity or even different kinds of whatever magical energy source forms the basis for underground lifeforms.

Choices

Spoiler (click to show/hide)

There is more than just the problem of the game being "too easy", it's also a problem of the game being "too shallow".  DF has complexity, but not very much depth.  I like to compare it to the creatures in 40d - they had plenty of stats on them, but really, all that mattered in a creature's stats was their size.  Right now, there's very little plant diversity because there's only one or two metrics you can really even judge a plant on - the rate at which you get food, and the value of whatever other use you might be able to get out of that plant.  Nobody wants hide root because it's just a silver barb or a dimple cup, but worse.

We can make the game much more deep and involved than it currently is, but it requires that we actually add into the game the metrics by which we can actually differentiate one plant from another.

I'm going to post this link to Extra Credits' monologue on choice, who make several very good arguments about game design worth watching.  The key to setting up a challenging or difficult choice is that you need to make the player want to further at least two mutually exclusive goals at the same time.  If all your choices are based upon just one metric (having enough food), then you can't make a player stop and choose, it's just a matter of solving the math problem of "what course of action will get me the most food for the least effort?"  There's nothing much in that decision tree that gives a player some sort of conflicting desire to do anything else, except maybe some players' desire to spend as little time thinking about a farming system as is absolutely mandatory. 

In order to set up conflict in the choices, I have involved several ways in which the player's innate desire (get as much valuable stuff as possible) are in conflict with other desires the player has. (The fear of an environmental backlash from overexploitation of the land, long-term degradation of the land versus short-term boosts in productivity, the threat of pests vs. the potential to grow greater numbers of more valuable crops, the choice between different forms of biological resources if trees are put on farms to compete with food crops, as are very tedious to grow but powerful alchemical ingredients when I get around to my next suggestion on alchemy, and of course the simple amount of player time and dwarf labor it takes to design and manage systems of ever-increasing complexity.) 

Do you not believe that these are sufficient conflicting desires?  I am always open to more suggestions on how to set up a conflict of player choice, but at the same time, you need to make sure it actually does this in a way that sets up a conflict of choice, and is not just a way to try to make the game "harder" without really adding to the depth of choice.  If all you have is one calculation to make on what is the best way to get enough seeds to sustain your farm, then once you have made that calculation, and make it every time, then the game is just as "easy" as it was before you made it "harder" - the problem is solved, and there is nothing else to think about.

The rest of this is to be continued over the next several posts, due to the limit of a mere 40,000 characters.  I'm going to leave several of these posts blank for placeholders to fill in later on.  I know it's a bit of the poor organization I'm trying to avoid to do this, but I have literally worked on this post so far all afternoon, and kind of just want to get something put up on the board at this point, and will backfill the details later.  For now, I want the general idea up.

[NW_Kohaku]

Urine and Feces  (or more specifically, decomposition and rot):

Spoiler (click to show/hide)

Before going into the mechanics of the suggestion, I want to go ahead and make a special argument for what is one of the more controversial issues that I want to incorporate into this suggestion.

There are two directions I want to argue, one to Toady, and the other to basically the forum at large, and I want to start with a quote from Toady.

From a Reddit article:

I like fertilizer, animal tracking and sewers. I dislike potty breaks. This is an example of realism that I think has a lot of potential for trouble. Potty breaks in adventure mode might be realistic, but there are immersion issues there. He he he, I mean in the sense of the player being kicked out of their groove. The other kind of immersion wouldn't be so bad, because sewers are common adventure environments. In dwarf mode, dwarves already take a lot of time out for self-maintenance, and this would be a more senseless kind, compared to something like eating.

So then, as the suggestion (directed at Toady), I think the best way to handle self-maintainance would be to deal with "potty breaks" by rolling it in with drinking.  You probably wouldn't even need to have a separate counter, just make dwarves need to go pee as soon as they've finished drinking.  I think I once did a calculation on how much booze dwarves would need to drink proportionately, considering how much they drink at once, and the answer was something like 55 gallons of booze in one chug.  So yeah, after pulling that stunt, they have to hit the latrine. 

A positive/negative aspect of this would be that sewers would be made somewhat more simple by the fact that you could very easily control where dwarves will need their latrines by just controling the booze stockpiles, and putting your major restroom facilities right next to it. 

The next logical step would be to make the elimination of solid waste be related to eating solid foods.  Since we don't have proper control over making dwarves eat in the dining halls we built just for that purpose, then toilet spacing might need to be based around where you have tables, and making sure that things like prisons have access to the plumbing, which could make for some interesting escape attempts, really.

there's a problem with the way that DF iterates many of the counters in the game, in general. 

That is, apparently, almost everything in the game that iterates iterates every single frame. 

This is a big problem for FPS because it means that every single frame of the game requires iterating through the entire creature list (living and dead) and iterating things like hunger and thirst and the like. 

Also, because the game stores hunger and thirst in unsigned shorts, the exact number of frames it takes to starve or get hungry is based arbitrarily upon the size of the container.  (The maximum hunger value is the maximum value of an unsigned short - 65535.)

Instead of having iterations of every stat for every creature every frame, we could instead start working off of a "scheduler" system for many of these iterations.

For example, water only gets checked for movement every (IIRC) 11 frames, and only 1 in 4 tiles actually move (in order) during that time.  This keeps the game from having to calculate every single water tile's movement every frame, and cuts the FPS loss from fluids to 1/44 of what it would be. 

This same principle can be applied to hunger and thirst, as well as bathroom breaks:

Each unit has a unit ID, and a place in the vector of creatures. 

You can set up a schedule so that a creature's hunger and thirst are iterated every 1000 frames.  (This is an arbitrary number, and can be whatever makes sense.) That is, in the first frame, every creature with a unit ID ending in *000 adds to its current hunger and thirst levels and checks for things like hair growth and the like.  Then, the next frame, every creature with a unit ID ending in *001 adds to its current hunger and thirst levels.

You don't have to just add 1.  You don't have to just add 1,000, either.  You can add 647 if you want to, because you don't have to deal with whole integers for whole frames, anymore. 

This means that you can set creature metabolism modifiers so that creatures like elephants don't starve so much. 

You can then also make this number based upon attributes (so larger, stronger, faster creatures burn more calories and need to eat more often, while small, lethargic people keep their calories) and break down the need for all meals to instantly reset hunger to 0 in dwarves, so that dwarves with large appetites will take stacks of food with them when eating. 

By doing this, you can actually make dwarves take breaks less often than they already do, since you can adjust the rate of those meters filling up because you are no longer bound to the one-frame-one-iteration standard.

AND it saves FPS!

The only cost is that, depending on the size of the scheduler period, and the size of the iteration to hunger/thirst/sleepiness, there may be sudden large jumps in the hunger/thirst/sleepiness values, so keeping a reasonably small period (that is, keeping it to something small like going for a full loop every 1000 frames rather than 100,000 frames) has to be balanced against making a situation where a dwarf has only a single scheduler period to eat or die. 


Beyond that, there's combined breaks - when a dwarf goes on break, instead of going on break to do a single thing, they can make a combined break, where, if they're thirsty at all, they will also take a drinking break during the same break as their eating break.  They can even grab food and drink in the same trip now that we have code for carrying multiple stacks of items going to the same location. 

This leads to a simple solution for where to find time for bathroom breaks: Just make them go when they're done eating or drinking.  It can also simplify where you need to put the bathrooms: right next to the dining halls. 

That way, you don't have any actual extra number of breaks dwarves go on, and if you make the actual time (number of turns) spent during the process of eating or drinking shorter, then you can even make the time spent on a given break not even significantly larger than it already is, provided you have an efficient fortress layout.

One of the key things in dealing with this, however, if we are to make a system of balanced inputs and outputs into the ecosystem, then we are possibly going to need some sort of nutrition system or the like to be able to, essentially, measure the nutrient value of a character's poop.  (Yes, I feel silly just for typing that, but this is serious.)   Basically, if we are going to have dwarves drinking booze but possibly also just water, then some villainous leaders might try to game the system by horrifically forcing their dwarves to drink plain water, just to get some bonus nitrogen out of their urine, assuming that the entire urine system makes all urine uniformly nitrogen-rich and of uniform quantities. 

This means that we might need to have some sort of nutrition system in place to track how many nutrients are being put into a dwarf's body, and how much are ready to be expelled, (a growing dwarf, whether because they are a child, adding on muscle, or just becoming fat should soak up nutrients they are not going to excrete to add to their mass) and a method of tracking the nutrient content of a sewer's worth of biological waste.  To a certain extent, nutrient value might just be tracked by invisibly segregating different nutrient numbers in a given tile's clump of poo (72 units of nitrogen, 120 units of phosphorous, 31 units of potassium, etc.), but just saying "pile of poo" with no visible direct measurement (except maybe raw mass) of what nutrient content that pile actually has. 



Now for the forum-goers at large, one of the most recurring things I've seen in the urine/feces arguments seems to be an insistance that "it would be gross, and would add nothing to the game".  Even when people talk about designing sewer systems, I've seen people respond immediately with how it's not adding anything to the game. 

Well, I hope I've made a case for conservation of matter thus far, but I'll try and go back into it...

You just can't have a very realistic farm without having a limited supply of fertilizer to spend on it, and feces and urine is by far the best way to manage this.  Making those two your primary sources of fertilizer means that your fertilizer production is tied directly to how many animals you have living in your fortress, which means how much food they consume, which means how much food is being eaten.  This links food production and food consumption to fertilizer production, which is needed, in turn, for food production, creating a balanced chain that repeats itself.  Growing fabrics for clothing can be turned around with some sort of "compost pile" building that can turn tattered clothes (and other biologicals) into fertilizer of like value to what you might get out of the waste produced when creatures eat food. 

Perhaps the best way to describe why this is important is to talk about the "ecosystem" argument again in more detail.  What we have currently in DF is a system where matter springs into existence from nothing, and then dissapears into nothing if we ever manage to get rid of it (but more likely, it just stays in our fortress forever).  What we really should be striving for, if we are to have a truly realistic experience, is to have what some ecology sciency peoples call "Cradle to Cradle Technology".  That is, anything that is biological ultimately comes from the soil.  Raw materials (like food) are extracted from the soil, and become useful materials (like animals, dwarves, clothes, etc.) that eventually reach the end of their useful life (or lives, period, for dwarves and animals) and we must then begin the process of rot and decay, so that the soil is replinished with the matter that it gave up to create a new set of raw materials.  There is no creation or destruction, just change from one form, one link in the chain, to the next.

Yes, sure, this means we'll have someone throwing poo, and some people will be able to point at our game and say, "Hey, someone who plays that game throws poo! Only monkeys play that game!"  The thing is, however, that if we do this, we will be able to respond to those people, "Our game accurately models the life cycle of bioindustrial products.  Plus you can kill things with magma.  Your game only lets you kill things with swords, and you don't even have a serious farming system.  Who would play such simplistic childrens' games?"

Forestry, Wild Soil, Soil Erosion, and Ecological Damage

Spoiler (click to show/hide)

Right, so now, it's time to talk about the chance to be a real tree-hugging hippy, or, much, much more likely, the chance to utterly rape the earth.

In essence, this idea was spawned from following the idea of having tracked soil nutrient levels in the wild to its fullest logical conclusions, so I think it would be best to start with the part that is most related to the rest of farming, first.

Tracking the "wild soil", the soil that is not a part of your farms as a large (possibly embark tile-sized) chunk of dirt, the way that I'm pretty sure things like vermin are partially treated, would serve the most immediate purpose of preventing someone from simply constructing a farm, depleting the soil, and then reconstructing it over again.  Farms that are deconstructed would have their "zone" simply turn from a farm zone into a "wild" sub-zone, which would then potentially be averaged in with the rest of that wild zone.  If it were placed in an elevation or layer that does not already have a wild soil set of stats (such as farming on a muddied stone layer unlrelated to a cavern layer, or a muddied artificial floor on the surface), it would create one.

It can, however, be used to represent something much more interesting with a little bit more modeling.

For example, if wild soil is going to have its own stats and plants growing on it, why not simply have the broad wild soil on the surface consume the current mechanics for how wild plants grow already?  We have soil nutrients, a selection of wild plants that may grow, including grasses, shrubs, and trees, and some sort of mechanic for measuring how much water will be dropped on the soil in a given year that adjusts grass growth rates already.

This creates something that can potentially make forestry and herbalism a little more interesting.  Currently, trees regrow eternally, as do shrubs.  If we let dwarves completely strip the land bare, we are consuming the soil nutrients from the surface just as surely as if we had been reckless with our farms.  If we continually strip out resources from the wild soil, it will become barren and lifeless, or at least, not suitable for growing certain kinds of plants, just like how overtaxing farm soil will do so.  The biome might functionally have to shift from a forest to a grassland as trees become no longer supportable temporarily, and grasslands might slip into deserts. (This can be reversed by leaving plenty of dead goblins to rot out in the fields, where they will turn into nutrients for the soil to support new wild flora.)

In fact, we could even start thinking of the wild soil as just being "farms" where the plants that are growing are outside of player control.  In fact, if we go that far, we can even start to think of farms as what they are in real life - a zone where we have designated only one type of crop to be produced, and whose seeds we plant, but which is just a slice of nature we are trying to artificially control, and where wild plants can try to grow if we don't keep it weeded.  Potentially, even beneficial shrubs might grow in a farm that is poorly maintained.

In order to do this, we need to generate a list of off-map "virtual seeds" for all of the plants, which work similarly to the way that vermin work currently - they are simply a resevoir of a certain number of potential plants that can start growing somewhere on the map when conditions for that plant are favorable.  Any plant that survives long enough to hit a flowering stage will add virtual seeds to the world.  These virtual seeds have no fixed location, being a sort of Shrodenger's Gun, and can be revealed to be anywhere whenever they are needed to appear.  Completely eliminating every seed in a location can potentially cause a local extinction of that kind of flora.

Balancing this, there needs to be some sort of drift of new seeds into an area.  Every visitor, whether a migrant or trader, can have some odds of accidentally carrying a wild seed with them from where they came from or some land they traversed to get to where you are.  Odds for this need to be stacked in favor of more durable seeds - it's hard to accidentally bring a peach pit with you without realizing, but a prickly burr will more likely cling to your pant leg.  Wild animals should also carry seeds like this from nearby locations, and a certain number of wind-borne seeds should blow in from lands outside your rapacious grasp.

Likewise, massive ecological damage, such as Boatmurdering your entire surface with magma, should cause the overall number of seeds to plummet, making local extinctions much more likely.  There needs to be a die-off for seeds every set period of time (the New Year, being as it is right at the change of Winter sounds as good as any) to make seeds that cannot turn into viable sprouts will die off eventually, with ecological trauma, whether it is a cold snap, prolonged drought, mountaintop removal or stripmining project, paving the surface or building castles on top of natural terrain, or magma flooding should have an ability to really decimate seed populations.

Although fauna are a different subject to be covered in a separate section, fauna should not want to visit any land that has no plant life for them to eat (or does not attract grazing fauna for them to eat if they are predators), and as such, hunting wild animals should also be impacted.

(I suddenly feel the need to create sub-sections, but whatever...)

The seeds that will actually grow are based upon the soil conditions in the area that they are growing.  The game should basically draw up a chart based on the soil, and see how well the seeds' ideal growing conditions match up with the actual soil conditions of each soil zone, with some possible random skewing to one direction or the other to represent small patches of fertility or infertility in some areas.  Seeds from the top candidates are selected by a random roll against this weighted list to start growing as a sprout.  This check does not have to occur very frequently at all, maybe once a game month, and can just have the start of the sprouting period randomly offset a number of days.

Sprouts should obviously have an impact on the soil quality that they grow from - wild soil zones, in fact, may need to have larger integer nutrient tracking numbers than I was already thinking of if it is to contain roughly 2304 tiles worth of soil's data.  (A 16-bit number still only has an average of 28 values per tile.  Hopefully, the soil will be less volatile than farms, of course, but still, 16 bits of data is a good baseline.)  Sprouts should gradually grow into their full blooming states without any need for dwarven intervention, although certain environmental factors should of course be a factor.  Trampling should obviously have negative effects, but also drought, pests (yes, they affect wild plants, too), herbivorous fauna, and potentially a sudden scarcity of a given nutrient they rely upon can cut down on growth speed. 

Boatmurdering your soil should, ironically enough, actually help give your soil a giant bump up in soil nutrients, even if it kills off the sprouts and seeds.  Any survivors, or any seeds you re-introduce or are introduced to your lands from outside sources will find the soil to have plenty of nutrients, however.

This, however, brings me around to one of the next points I wanted to talk about - soil erosion.

I remember reading something about how grasslands are actually a form of artificial biome created by large herbivorous herd creatures for their own convenience.  Obvoiusly, this is not a conscious choice, but it is the impact of their presence.  You see, most temperate areas have basically two default settings - forest and desert.  Forests have plenty of water and soil nutrients, while deserts have little water and soil nutrients.  Forests are created when an area with water but no soil nutrients supports lichen and moss for a while to break down the rocks in the area, and create some thin soil.  This eventually gives you some room for grass to start growing in the thin soil, and help break down the rocks, and build up the soil even further, while the roots of the grass prevent the soil from washing away.  Gradually, when there is enough soil to support them, shrubs, and finally trees start to move in, and these giant plants take advantage of their size to strangle all other competition by spreading their leaves, and leaving smaller plants like grass to starve in the shade.  Eventually, with time, rich enough soil and enough rainfall gives you rainforests, where titanic trees compete to grow taller, and strangle all their competition. 

This all depends on the soil being able to retain its nutrients, however.  If the soil ever loses its nutrients, you have to go all the way back to lichen and start over again, and this process takes thousands of years.  And the soil loses its nutrients whenever it rains, or water covers the area (something I will cover more in the Water Management section), and there are no plant roots to prevent the soil from just washing away down the river.

Cattle or other herbivorous herd animals trample and kill anything bigger than shrubs, however, and keep their land grassy.  They also poo, and add soil nutrients in that way.  Now then, grasses can survive in very dry climates, and basically just wait out droughts in desert-like conditions for the time to spring back to life.  So can succulents, like cactuses.  Grasses also cling to their soil nutrients, keeping the soil rich.  Grazing animals can prevent a wet area from gradually building up to a forest, but they can also keep a dry area as a grassland when it would otherwise decline into a desert because of the lack of water.  Cattle revitalize the soil with their droppings, and the grasses they help to grow keeps the soil nutrients in the area. 

The destruction of wild grazing animals in dry areas causes what is called "Desertification", where grassland gives way to desert.

Now, this natural process can be interesting to understand, but it should also be part of a very compelling ecological model for Dwarf Fortress - we can have our dwarves potentially causing the desertification of the land they plunder for all its resources.  (Or, theoretically, we could build up the soil, and create a forest where once there was nothing but sparse, dry shrubs, but seriously, what are you, an elf?!)

To do this, we need a model for soil erosion, where soil is degraded by rainfall or water drainage (see the Water Management section), and where grass or sprouts or other plants will protect the soil they reside upon, and tiles adjacent.  Tiles not adjacent to the plantlife will start to suffer a form of "soil nutrient damage".  Rainwater is more damaging than flooding from another source, but the hows and whys of this are for another section.  Potentially, a large enough patch of grassless soil will need to be severed from the general embark tile-wide zone, to create a new zone with the damaged soil that will not grow anything in that particular area.  This can potentially happen if you have a trail of particularly heavily trampled soil, such as near a fortress enterance.  It will eventually become dead, desert soil with no water-soluable nutrients (I.E. nitrogen and phosphorous) at all.

In total, we may eventually be able to truly have visitors capable of judging the environmental impact of your fortress from a great distance - if, suddenly, there is a ring of black, smoldering death around your fort where vibrant forest surrounds it, elves need not even count your tree kill ratio to know you are their most horrifying nightmare come to life.  If your magic green thumb can coax to life even the most barren of lands with vibrant biodiversity, then everyone will call you a hippy elf and properly shun you for your eco-friendly crap. 

Either way, when we talk about Dwarf Fortress for its ability to simulate something complex or detailed, what better way is there to do this than to simulate the utter god-like mastery over their environments that Dwarves hold by letting them turn lush greenery into permanent deserts as part of a simulation from the lowliest grass blade to the whole of a chunk of a continent?

[NW_Kohaku]

NPK soil nutrients

Spoiler (click to show/hide)

Basically, the root of the system, this is the most basic method by which we can make crops more interesting and dynamic.  To sum the progress of the thread up, while I was initially in favor of a very simple system, I was successfully convinced of the need for a more robust system to track soil nutrients.  Generally, I'm working with a 6-variable model right now, although there's things I am wavering on expanding into making more variables to include.  These variables are Water, N, P, K, pH, and biomass.

NPK stands for Nitrogen, Phosphorous, and Potassium, which are the three "Macronutrients", the most important atomic components in the soil.  Photosynthesis converts CO₂ and H₂O into simple sugars, like C₆H₁₂O₆.  Most of the building blocks of life are made of some combination of CHON or CHONP, so NPK are important components, but air and water are far more heavily used, while NPK are used in that specific order of prevalance.  With that said, they are all critical, it just requires lesser and lesser amounts by mass.  Air is not being measured in any of these systems so far, although it is certainly an exciting future pathway for the game to explore, especially if we are to have oxygen-producing subterranean plants.  Water is functionally the biggest "super nutrient" in the game, then, followed by NPK.

As they are arranged in order of how much is used by volume, they are also arranged in terms of how water-soluable they are, and by extention, how easily plants can gain access to them.  Water obviously flows easily through soil, nitrogen is very soluable, while potassium is very likely to bind to the clays in the soil, and is not very water soluable.  Water soluability affects how much runoff occurs IRL, although that may or may not need to be modeled.  Water soluability also affects how available a nutrient is to the plant's roots.  If something is water soluable, it flows through the soil, and the roots can easily be there to suck up the nutrients, while less soluable K requires the roots to dig right next to where the K ions are to suck them up.

This ultimately means that Water and Nitrogen are values that will fluctuate more wildly, while Potassium is very stable.  Water is something you will constantly need to replinish (although, honestly thinking about it, in a sealed cavern, you could probably find ways to reclaim the water).  Replacing nitrogen is the main point of crop rotations, and the soil can be depleted of nitrogen within just a year or two of planting "heavy feeder" plants.  Phosphorous takes a middle road on this, while Potassium is something that you can probably go for a few years without worrying about. 

Something about nitrogen, however, is that while you need plenty of it, it is also the easiest nutrient to replace, whether by planting nitrogen-fixing crops, like peanuts, or just by using urea, the primary component of urine, to make ammonia, which is how the overwhelming of nitrogen fertilizer is applied to the world's crops. 

Nitrogen fixation occurs because certain root-dwelling microorganisms of plants like clover, peanuts, and beans are capable of converting the stable N₂ in the atmosphere into a usable form for living beings.  Crop rotation is based around the planting of nitrogen fixating plants to restore nitrogen to the soil between the growth of nitrogen-heavy plants like wheat, rice, corn, or other major starch foods.

Phosphorous and potassium, however, are not so easily replaced from the functionally infinite well of atmospheric gasses, and generally, the best, most renewable means of replacing them is to just use dead or decaying materials to replace what you have taken out of the soil.

It is, however, entirely possible to mine for mineral fertilizers, such as apatite stones for use as a phosphorous fertilizer.

This would mean that expanding farms beyond the "easy soil layers", and "making your own soil" by muddying the stone would probably become a long-term project.  You would need to start with crushing some mineral fertilizers down to gravel, keeping them muddy, and trying to grow "rockbreaker" plants (or rather, moss and lichen) to break the fertilizer stones down into a useful soil.  (Of course, letting logs rot into soil by growing mushrooms also works to a degree.)

To a certain degree, it's not really worth retyping all this text, and reposting it is just a waste of character limits, so I'm just going to punch in a link to the post on how we could represent NPK to the player, this post on fertilizers (in fact, page 32 in general outlines the basis of this), and this link to a series of posts modeling how this could be raw-ified. 

There are still two more variables that I have to really go into, however, and those are soil pH and biomass.

Soil pH (acidity) is another very long-term variable.  In fact, it should take about a year to adjust a soil's pH a single whole number one way or the other.  Soil acidity is altered by the process of decomposition, so places like forests where plenty of rotting leaves fall will have more acidic soil, while deserts will have more basic soil, and different plants are adapted to different soil types.  Generally, you want very slightly acidic soil, however. 

Fertilizers, especially ammonia and urea, can alter soil acidity, while plants themselves don't generally do much to do so.  The general point of soil acidity, however, is that it's very slow and labor-intensive to change it, so it's probably better to just try to avoid having a soil acidity problem in the first place, or grow your plants to just suit local soil acidity rather than alter soil to be perfect for the crops you want. 

Biomass, meanwhile, exists largely for the purposes of growing funguses, which cannot photosynthesize, and as such, need something to decay to grow and gain chemical energy from.  Biomass is a measure of the available carbohydrates in the soil, generally from "dead stuff" in the soil.  At low values, it represents depleted/eroded soil, where few plants besides ground cover.  At medium values, it represents increasingly rich humus, suitable for sustaining most crop plants.  At very high values, it can represent an outright sludge of decaying material that may invite bacteria causing soil toxicity, but which is perfectly suitable for "crop" fungus. In cases of "mundane" fungus, biomass may be essentially the only nutrient worth tracking besides water.

Speaking of extra stats that may be trackable, soil drainage is another potential variable that I didn't talk about in a serious manner, but could potentially be a useful means of differentiating certain plants from other ones.  Plants such as grapes, for example, require plenty of drainage, and drainage is very difficult and expensive to build into your soil.  It basically requires manually creating raised planting beds, or artificially adding a slope to the landscape to create drainage.  Both of these are extremely labor-intensive operations, and represent massive allocations of time and effort to make certain types of crops more viable for farming.  (That is, if you want the best wines, you have to put in the effort to manually terraform your landscape for that purpose, and that purpose alone.)

Now then, as I say in one of those posts, I think that what we need to do is track all these numbers in terms of 0 to 255, at a minimum, as a means of having a serious way of nuancing the differences between one season's soil and another.  Potassium, for example, only goes up or down two or three points in some of the proposed plants I made up, so that there is only a very subtle change in soil fertility unless you ignore it for long periods.

Water Management

Spoiler (click to show/hide)

Ultimately, if we are going to be serious about farming, water has to be a major part of this.

There are a few separate issues, all of which fall generally under "water", which range from watering the crops and irrigation to water availability to the mineral content of bodies of water and their effects on soil erosion, as well as general weather for modeling droughts or other significant hazards to farming.

Irrigation Methods:

Before I start, I should reiterate something from the NPK system, since I believe I glossed over this point, and let it be covered by a post I linked back to (which not all of you will necessarily follow): Water is a separate integer value that represents the moistness of the soil.  Actually flooding the tile with water (as in having a 1/7 water depth or more) will cause the water value of that tile to instantly max out, which may be harmful to many crops which are not suited for actual flooding.  Part of the system I have outlined is that you can kill plants with too much of something, including making their soil too swampy, although there are some exceptions that cannot be drowned.

First, and most simply, you can just plain put your fields outside.  Above-ground farms with no ceiling above them should be capable of just soaking up rain in any sufficiently humid climate.  A slightly more elegant weather system will probably be required, but I'll focus on that in another sub-section.  For now, I believe that the best way to model keeping crops watered through rainfall would be to declare the rain that falls when the game declares "it is raining" to be the especially heavy rains, while some unannounced invisible light showers can be simulated behind the scenes to keep the overall rainfall suitable for simply having open-air fields with no irrigation methods.  Basically, while showers occur, the game invisibly waters the fields for you.

Next, and least efficiently, we should be able to "bucket brigade" our water.  To a certain extent, this will be the "default" watering method of any underground farming method, and would heavily encourage building your reservoirs as close to the farm plots as possible.  This should obviously become very inefficient as farms grow larger, and would probably require multiple reservoirs near every set of farm plots to compensate, but by that time, you should really have moved on to better irrigation methods.

Beyond that, there is the "controlled flood" method.  Suitable for things like initial muddying or just before planting, flooding the soil obviously gets things wet in a hurry, but as I said earlier, it should be deadly to most plants that are already planted.  Some crops (like something modeled upon celery or rice) may be able to survive simply being planted in standing water.  (In fact, the reason why you grow rice in standing water is not because rice needs it, but because rice is one of only a few plants that survive that sort of treatment, and it makes an effective weedkiller.)  Generally, though, this should be discouraged, even if it is the most familiar method, currently.

Finally, we get to methods of true irrigation, and there was some contention regarding how best to do this.  In the interest of giving the idea a fair hearing, Silverionmox was a proponent of simply having visible water trenches dug alongside farm plots filled with certain levels of water that simply supplied water through the sides of the trench.  His main argument for this was that it would involve systems already part of how we deal with water, and he believed it was not a good idea to mix means of transporting water.

On the other hand, I was a proponent of a separate system of sending water to farm tiles.  There are two possible ways of doing this.  The more basic method would be to have farm tiles "improved" by having special furrows for letting water through, or a special "water trench" tile that only carries smaller amounts of water than are normally displayed in the game, aside from possibly in buckets. 

The "dwarfier" response to the need for irrigation, which I favor the most, would be to involve some sort of plumbing system with small pipes (not the kind you can walk through, which are slated to be introduced, but a smaller one just for water), which just have regularly punched holes in them to act as a "drip feed system". 

Gravity or pump-driven water pressure would allow either one of these irrigation systems to send water down its track, and we could abstract it to automatically evenly distribute the water.  The advantage of this sort of system is that if it is a system where the numbers are not based upon the already in-game physics system, it would be much easier for the dwarven AI to be able to understand, and hence control, and hence automate the entire irrigation process.  You can simply make each copper sprinkler pipe connect to a designated sluice or other control system, and dwarves could simply stand on top of that sluice's tile for as long as they need the sluice open, and it will close off the water supply as soon as they are done manually keeping it open, preventing any messy problems with irrigation systems being left open to flood your fort. 

Even more advanced irrigation systems may involve letting fertilizers become added into the reservoirs or sluices, so that fertilizer is distributed at the same time as water is distributed to the farm zones. Ammonia/nitrogen fertilizers are highly soluble, so this is an easy way to keep nitrogen levels at a constant rate throughout the year.  Simply mixing a small amount of ammonia into the water as it heads through the sluice to the plants creates the "manure tea" that allows for a trickle of fertilizer along with the stream of water.

To point something out, each farming zone would likely require their own independent irrigation systems, as different crops consume water at different rates, and you can kill plants by flooding them with too much water, meaning giving every crop equal amounts of water can be problematic.

Another thing that is useful about transporting water through "small pipe" methods is that modeling it should be significantly less of a burden on the CPU than the current random water sloshing of the physics engine - a small pipe that just has a "funnels x water per tick" function will not be as processor-intensive, which will be important, since we're going to be talking about using much more water, when water is already a significant drain on processor power.


Water Availability:

As I have discussed at the beginning of this entire proposal, one of the ways in which I want to make players face a serious choice about how to manage their resources and food is to make the resources you can use to make growing food be finite, yet renewable.  While I have previously stated a goal of this to be trying to truly respect Conservation of Matter, water may well be a serious exception to this rule, as things like condensation rates and tracking of humidity levels may be too complex and finicky models even for Dwarf Fortress's standards.  Instead, water needs to be constantly introduced to an embark site, yet these rates of water input need to be better calibrated than they currently are. 

I want to also point out at this time, as I will probably have to point out in many areas, that the Volume and Mass suggestion contains a proposal to rework the way that objects, and especially water are tracked, which would make the tracking of water supplies easier overall than they would otherwise be.

Water currently is introduced to maps through what can only be called infinite water generation points.  Edges of maps where water starts out, whether it is a small brook, a tiny underground lake, an aquifer, or an entire ocean, they all produce water at the same rate (on a per generation tile basis), and they all have infinite quantities of water.  This is perhaps most amusing in cavern lakes, as if you embark on an entire 16x16 embark, the lake will be entirely self-contained, and no water will regenerate in the lakes in the caverns.  Embarking in the exact same location as a 15x16 or anything smaller, however, will mean that, if any part of the lake is cut off, the water flow into the rest of the cavern lake will be infinitely renewed, even if only a single tile was cut off, just as if it were an entire ocean.

What needs to happen is that this game needs to start tracking water input sources, and setting finite limits on how much water will be introduced per water source.  The ocean can pretty much remain the way it is, as there is no real need to quantify how much water is in the ocean, we can pretty much assume it's infinite, but everything else needs to be a depletable source.

*** LATE EDIT: As of 2015, California, for example, is going through consecutive years of extreme or "exceptional" drought.  Over 80% of California's water consumption is in agriculture.  Much of the blame is being placed upon the fact that Californians chose very water-intensive crops, such as alfalfa (used to feed cattle) or almonds (used for almond milk) when their water tables were high. 

Again, as a form of engineering challenge, working with limited amounts of available water per year can be a serious consideration for players, setting a "roof" on what sustainable crops they can produce, especially if they have to balance growing food crops, trees, cloth, and feed/pasture grass for livestock.  Dry areas may force players into hard choices about priorities, limiting population, trading for food and cloth, or encourage extreme engineering to manage their water supply.

Ironically, in grasslands too dry to be arable, grass-fed cattle are actually somewhat more suitable as a source of food compared to more water-intensive plants, although these tend to require wide-ranging semi-nomadic cattle drives to keep fed.  Embarks on tundra, at least, will no longer be a trivial event of just eating nothing but meat and eggs from poultry farms.

Periodic drought as a "disaster" (possibly with embarks having warnings of how severe a drought can be) where depletable water sources do not refill as normal for one or more years in a row (little/no rainfall and little/no cavern lake, brook, or aquifer refilling due to lack of rainfall/snowpack) to add severe challenges to farming management.

Similarly, with some work, a "flood" might occur. River tiles have a natural set of entry tiles that spawn water continuously.  Adding additional tiles a z-level above them that spawn water periodically during flood season or a flood event could create for a more dynamic water source, especially if the seasonal floods, like the Nile floods, (or old cave floods,) had different soil nutrients in its mud (contaminants). 

Something similar with magma could represent volcanic eruptions.  Raising a magma pipe up in terms of where magma will spawn to for a season can produce a Fun surprise for dwarves.  (Especially if potentially triggered by excessive magma dumping, like abyss creatures were in 2d...) If there were walls in the way, preventing full magma expansion (and it wasn't being drained off elsewhere...) it could create local pyroclastic events.  Receding magma flows could leave behind either new igneous walls (like mafic gabbro or basalt if there was plenty of expansion area, or felsic granite and andersite if there was pyroclastic activity) and leave magma-flooding contaminants/nutrients in the soil.

/EDIT ***

Brooks, especially, should have very finite amounts of water inflow.  We might actually want to go from the current method of having a 7/7 water tile covered by a "floor" that allows walking over a "shallow" brook to simply having a 1/7 or 2/7 stream inside of a riverbed, especially since most of these riverbeds are already recessed a single z level and have ramps on them, anyway.  All creatures can "wade" through a 1/7 or 2/7 water supply, anyway.  Water could then be added to brooks at a smaller rate.  Rivers or other larger water bodies can still remain the way they are, but have better throttling over their water inputs, although river width should make a fairly major impact, already.

Every source of water (besides the ocean) needs to have some sort of throttle on the amount of water it can provide - rivers and brooks need to have some sort of counter that tells the game how much water will be flowing through that body of water.  This should not be static.  There are two primary sources of water for rivers and brooks, and they are rainfall and snowpack. 

Rainfall should periodically flood brooks and rivers with excess water that exceeds the soil's ability to retain the soil, leading to brooks that occasionally fill up with greater amounts of water, while they may occasionally run dry at times in drier climates.  Rainfall will require a separate climate tracker for "upstream" weather patterns, although this may not necessarily need to be more complex than the on-map weather tracking, nor does it need to be checked as often.  An arbitrary upstream spot should be selected, and have its weather rolled to see when brooks will flood from upstream sources, provided the source of a brook is far enough away from the map (sometimes brooks start on the embark site, meaning the weather on-map is the source of the brook).  This can mean that it may be dry in the embark, but it rained on the mountain that feeds the brook that runs through your map, so you can at least draw water from the brooks.

Snowpack is a far more regular and predictable source of water, which is why it is relied upon by farmers to supply water to their fields far more than rainfall is.  Snowpack involves the snow that falls upon the windward sides of mountains gradually melting through the Spring and Summer months, and being replenished through the Winter.  This naturally leads to higher water levels in brooks during the warmer seasons, and less (possibly none) in the colder seasons.

Such periodic water supplies mean wise farmers ensure they build reservoirs or dams to ensure they have supplies of water even during the dry periods. 

Another common surface water source is the murky pool, which I will cover in conjunction with on-map rainfall.  Currently, rainfall is simulated in a fairly granular way by having "raindrops" that drop entire 1/7 water units.  If they fall in a murky pool, they become regular surface water supplies.  If they fall anywhere else, they basically just dissapear, although they "wash off" any contaminant they touch, and douse dwarves caught in it in water. 

Rather than this, I think a slightly better way of handling this concept would be to involve the use of the previously mentioned system for tracking "wild soil" zones, as well as farm zones.  Rainfall gradually just adds to the water variable of that soil zone any time it is raining.  If the water variable become maxed out, then an overflow occurs, and every chunk of overflow water will produce a 1/7 water unit to be produced.  If there is a murky pool in or adjacent to the wild soil zone, the 1/7 water unit will be spawned at one open murky pool tile (since it is designed to emulate the low-lying land where water pools in rainstorms).  If there are no murky pool tiles open (such as by being built over or already hitting 7/7), then the water "floods", and is produced at a random spot in the soil zone.  This means that completely saturated soil will start behaving similar to the way that rainfall behaves now, except with every tile possibly spawning 1/7 water units. 

This means that areas on the surface with no soil will probably have to get hit by 1/7 water units, as well, as paved stones and mountains with exposed stone and no soil will just immediately "flood".  This can, however, get very processor-taxing, so a method of abstracting this process in a less processor-intensive manner may be advisable. 

This may also open up a desire for storm grates or sloped roads or slanted roofing (although this may be simulated with better recognition of ramps) that you could use to somehow direct rainfall, not only to make sure it doesn't flood undesirable areas, but also so that you can divert more rainfall into water resevoirs.  Of course, this can depend slightly upon the sheer ability of the game to even have enough an ability to display some of these things to the player.

Speaking of which, rainbarrels should be buildable, which would use these same mechanics to gradually have water fill them.  Combined with the perhaps more industrial scale of a completely paved surface with a storm grate down to a resevoir should give enterprising dwarves the ability to ensure that water is not wasted on silly surface soil where it can evaporate without their aiblity to exploit it to its fullest potential.

A final method of water supply would be the underground water supplies and lakes.  Lakes, underground lakes, and aquifers are all bodies of water that do not rely so much upon a "flow rate" as it relies upon having a massive quantity of stored water, with a generally fairly slight refilling rate.  This would mean that these lake-type water supplies can be depleted much faster than they will be capable of naturally refilling, which can mean that aquifers or underground lakes can theoretically be drained completely, with only a tiny trickle of incoming water to replinish it.  Countering this, however, pumping water into an aquifer or lake should replinish it, so just pumping water from an aquifer into an aquifer using power supplied by the water flow of the water from the aquifer will still be a perpetual motion machine of sorts. 

The actual way this would appear to the player should actually be somewhat similar to how it appears now, with water continuously arriving from off-map, up until the point where you completely exhaust the reserved water of the lake or aquifer, at which point the z-level of the water table either drops, or if it is at the lowest possible point, it simply does not produce more water except at the rate at which the resevoir refills, which may be glacially slow.


Soil Erosion:

This is one of those things where the concept is simple, but the implimentation may be extremely difficult, but it may be required as a means of achieving a truly excellent model for environmental factors, especially of the kind I was talking about back in the Forestry section. 

Water currently "muddies" stone or soil where water flows, allowing those areas to become fertile.  In a limited sense, this is relatively realistic, as water tends to dissolve soil nutrients, and then carry those nutrients with them as they flow, depositing some of them as they are carried by the water across a surface. 

To be a truly compelling model, however, water needs to be a bit more complex, and this may involve breaking the current water system to an extent to replace it with something a little more complex, more similar, again, to what I had suggested in Volume and Mass. 

Water, in order to act the way that I envision, needs to also have a soil nutrient tracking system.  This obviously shouldn't happen across every tile of water, but rather, across water bodies.  Contiguous water bodies need to have what amounts to their averaged-out dissolved soil nutrient content. 

When bodies of water meet, they need to average themselves out. 

When bodies of water flow over any surface, they create the "muddied" surface, or more accurately to the new method I am suggesting, create or add to a wild soil zone.  The newly created wild soil from mud should have soil nutrient levels matching that of the water, representing the deposited silt, while the water has its dissolved soil levels reduced to represent the loss of that silt.  The gain and loss of dissolved nutrient levels should, of course, be averaged by the number of units of water contained in the body of water, meaning large bodies of water will have barely noticeable changes in their nutrient content as they flow.

When bodies of water flow over already existing soil that does not have plants growing on top of the soil, soil erosion occurs, gradually taking nutrients from the soil, and putting it into the water body, increasing the water body's dissolved soil nutrient content, while decreasing the soil's retained soil nutrient levels.  This should occur much more quickly for the water-soluable nutrients of Nitrogen (10 times the rate of Potassium) and Phosphorous (3 times the rate of Potassium) more than it should for Potassium, the least soluable nutrient.  The amount of nutrients that water sucks up depends upon the volume of water compared to the surface area of soil it is leeching nutrients from.

Generally, different water sources should have different nutrient levels.  Murky pools should leech some nutrients from the surrounding land when it is being filled with rainwater, and should generally have fairly decent amounts of nutrients in them.  Rainfall itself, however, should have almost no nutrients, and only possible pollutants, although that is a matter for another section.  Brooks and rivers should contain nutrients brought by the gradual erosion of the mountains where they originate, but generally, brooks should be fairly pure, while only the widest, slowest rivers should carry massive amounts of silt from upstream, and the general length and area of the watershed of a river should be used to determine the dissolved mineral content of rivers.  Aquifers and lakes should generally pool a decent amount of dissolved nutrients, although water entering an aquifer is generally filtered by the soil, so that it has a lower saturation point.

Ultimately, this means that letting water flow over your fields is a bad idea, as you want them nutrient rich.  Having standing water that has time to settle, and which you can dump fertilizers into constantly, can allow farming of some types of crops like rice, since you can just bring the water itself up to saturation point, and make sure that there isn't too much water to dilute your overall amount of fertilizer.

Water added to soil that is less than the soil's saturation point does not produce soil erosion, so careful control of irrigation in underground chambers should not need to fear soil erosion, so long as nothing goes wrong... 

Soil erosion on the surface, however, after dwarves have properly trampled all the plants, can allow for the gradual change of an area of the map from forest or grassland into desert or muddy swamp where nothing lives.


Reverse Soil Erosion (Floodplains)

As an extrapolation upon floodplains, if a river actually has more nutrients in it than the soil it runs over, then it will wind up adding to soil fertility.  This will actually create soil if there is none, which basically replaces the current "muddy" system of irrigation. 

The quality of nutrients that are deposited, however, is dependent upon the source of the water.  A clear mountain stream probably has few nutrients in it.  A broad river that has emptied a continent, like the Nile or the Amazon river, especially when close to its delta, will be a slowly drifting, nutrient-clogged body of more mud than water. 

These major riverways are then useful for floodplain agriculture, where "reverse soil erosion" lets you purposefully flood your fields every year to replenish your fields the easy way.

Just as a game balance technique, however, there needs to be some limit to how effective this is, or everyone will want to embark on the Nile for free farm goodies.  Either the reverse soil erosion technique doesn't produce very good soil, and hence crops, compared to a better-managed system, or the amount of water per year that you can get from such a river needs to be fairly small.  The idea of draining a river (even a fairly slow-moving one) like the Nile or even the Mississippi is pretty absurd, however, so the first option would have to be the best, and make reverse soil erosion just not particularly attractive, even in extreme cases like a Nile (which is very rare in-game, already).

Since rivers are based upon their upstream flow rates, very large rivers like a Nile should have the sort of massive changes in water level that wind up creating a seasonal flooding - the water flow may drop the river's depth to where some tiles are accessible when they weren't before, and you can farm the unflooded tiles, simulating the Nile river delta... but beware years that lack a flood, and even more, the years with too much of a flood.


Magma Flooding

Magma, and the ashes and such that it leaves behind, are a tremendous source of nutrients, fresh from the core of the earth.  They can be fairly toxic and plagued with heavy metals, but magma should also be a prime means of very rapidly fertilizing a new soil, or creating one entirely.  Magma could potentially use methods similar or identical to the water flooding code to cause dramatic changes to the soil, or even leave a form of "magma soil" behind when it "evaporates" if there was no soil there before. 

Magma flooding can be a fast and easy way to replinish soil, although the heavy metal toxicity would be problematic for most plants if anything more than a light dusting took place, or water wasn't used afterwards to try to flush the heavy metals out.


Salt

Salt is another major issue when dealing with water.  Salt water is currently handled by a simple flag that indicates salinity in the same basic method of map data storage that water already currently uses.

This might have to change somewhat, although to an extent, for the purposes of map data, the binary flag for whether or not water is salinated enough for the purposes of being "contaminated" for drinking might be just fine as-is.  What we will probably need, however, is a new variable for salinity of both water and of soil.  (This starts off my even more expanded set of variables to cover "pollution".) 

Salt water from places like oceans should obviously have a high starting salinity, while fresh water has a low to negligible starting salinity.  These waters, separate from their sources, might be mixed, with the salinity diluted, but this is mainly for the purposes of soil salinity.

While almost any living organism needs salt, almost any organism will die if exposed to too much salt, as osmosis (through control of salt) is the primary way in which cells maintain proper hydration.  This means plants in salty soil die of thirst, and salting the soil can ensure the death of most plantlife (and subsequently, animal life).  Only plants called halophytes are particularly adapted to salty water, and these are generally not crop plants, and would make poor ones, anyway, as they grow slower, but there's always making new plants up to fit the niche.

Irrigating soil with salty water, or some other means of accidental soil salinization (including even regular irrigation, see [url-http://en.wikipedia.org/wiki/Soil_salination]the Wikipedia article part on irrigation[/url]) will increase salt "pollution" of the soil, and make farming significantly more difficult until the soil can be leeched of the salt.

Some of the previously mentioned halophytes are actually best used for this - saltbushes will concentrate the salt in their leaves, so that if the saltbushes are harvested, the salt is removed from the soil, and can be taken elsewhere for disposal or even eating, as apparently, people do eat these things. 


Weather:

Generally, the weather model currently in DF is fairly good for what we need it for.  It will need to be expanded to fit some of the changes I have talked about, however.

There should come wet and dry years, as well as merely wet or dry seasons.  Very stormy years, and possibly even things of rare, catastrophic nature, such as the eruption of the Krakatoa volcano, which caused "The Year Without a Summer" in North America and Europe, can liven up the battle against the elements themselves.  Droughts and floods, however, should generally be the main worries of farming dwarves.

Rain can also be more abstracted than it is currently.  Rain as it is now might be better made as slightly more rare than it currently is, and instead have "showers" that are more frequent, but with less obvious gameplay effects.  Showers produce small increases in wild soil water levels that allow wild plants or unirrigated farms to go the periods between the major rainstorms. 

[NW_Kohaku]

Pests, Wildlife, and Invasive Species

Spoiler (click to show/hide)

This was actually what I got the most excited about back in the last iteration of this thread over time, but not so many other people seemed to think as I did.

The basics of this system is what I already mentioned, pests are either vermin or invisible microorganisms that plague your crops occasionally.  I will explain that in this section, but I will also go into some expansions upon my original idea inspired by some of my more recent thoughts on how to better model a simulated ecosystem.  To this end, it will be a section on more than just vermin, and be about how animals interact with drastic artificial changes in their environment in general.

Pests: 

As mentioned earlier, pests exist to provide a balance against always growing the same crop by making crops vulnerable to being attacked by an invasion of pests.  If a fortress is reliant upon only one crop, and that crop is attacked by a disease that wipes out most of that species in one year, then you can face a total disaster as your entire harvest disappears before your eyes.  If you have a balance of 8 or so crops, then one is wiped out by a sudden pest invasion, you can probably try to conserve or purchase from outsiders to make up for the difference, but otherwise be fine. 

To really discuss how to change them, though, let me go over what we already have, first:  Vermin and major fauna already exist as a pool of potential creatures that live in the biome.  These creatures are spawned gradually on the map as other vermin or fauna are removed (by being killed, walking off the edge of the map, or just being replaced, if a vermin), so that there are only a certain number on the map at any given time.  Vermin are pretty ruthlessly slaughtered by cats, which hunt all vermin, whether a bug, a turtle, or a fairy, by using their special vermindar^TM that can detect the scuttling of a beetle from anywhere in the tri-county area.

Now then, this sets up what we can do with pests: Pests can be something like a specific kind of bug, like a bowl weevil that attacks cotton, or a microorganism like a disease or a fungus that you cannot even see as a vermin, and are much more restrained in your methods of combating them.  Regular fauna, like rabbits or wild herd herbivores that get into your fields somehow can also act as a kind of "big pest", but so long as we make them grazers, and let them have a desire to eat your best crops, that can probably become a fairly easy thing to model.

This means that either the plant itself has to designate what its natural enemies are in the raws, or the pests will have their specialties listed in their own raws.  The same can be said of the countermeasures against the pests - not only the poisons they are vulnerable to, but the pests should have some classifying tokens, such as flying insects having something like a [PEST:FLYING_INSECT] token that designates what their own natural predators are.

The actual arrival of pests depends on whether they were there to start with or not (in the biome), and how much of a given crop you are growing.  The closer they always were, and the more food you offer them, then the more likely they are to attack.

In order to model these pests properly, however, we must simulate many more pests on the field of play at once than we currently do.  This basically means we're going to need to abstract the system, and say that there are more insects in the field than we can actually see on the map at any one point in time.  It also means that cats can't just be our feline-shaped vermin-seeking missile system that completely sterilizes the whole fortress of any lifeform smaller than a gopher.

This means that pests might be tracked, instead, as a sort of amorphous multi-tile blob constituted of a very large number of insects, rather than a single creature, and not like a "swarm", but as an area that is merely "infected" with that kind of insect.  Parasitic fungi might not even appear on the map in any way at all, instead only appearing if you were to inspect the soil.  Some individual creatures may spawn for cats to zero in on, but this is something that requires a more specialized hoard of combatants to fight.

Counter-Pests:
Fighting the war against pests, dwarves can have access to their own armies of abstracted pest-hunting minions.  Spiders could become your vanguard of insect-hunters, jumping spiders pouncing upon crawling insects, while web-spinning spiders defend against aerial assault.  Against the stealthy threat of disease, an expansive counter-intelligence network of molds and fungi in the soil wait, ready to pounce upon their favorite meals - other diseases or fungi.

As pests arrive in your fortress, it should help push up the population of whatever creatures eat those pests, just as the arrival of certain plants invite the arrival of those pests in the first place.  Dwarves can attempt to culture or breed these vermin, and place them in their fields to protect against crop-attacking pests.  They can also gradually migrate into fields that are under serious threat by their favored prey.

Spiders can be fairly generalized - spiders generally just specialize on whether they hunt flying insects (by setting webs) or crawling insects (by pouncing or using trap-doors).  Fungi have to be more specialized, targeting only specific pests.  There would have to be a wide variety of soil fungi to protect against the wide array of diseases and parasitic fungi in the soil. 

Companion crops are another major way to protect against pests - certain crops have a natural ability to repel insects.  For example, peanuts are vulnerable to aphid attack.  Marigold flowers can be planted because their smell attracts the natural predators of aphids, and because of that, aphids also have an innate fear of the scent of a marigold, as well.  Rice farmers also use a type of lily called azolla when their rice has grown enough for the flooding of the paddie, as it covers the surface of the water, choking out any encroaching weed that might compete with the rice.  See this Wikipedia list for examples.

Likewise, certain hornets lay their young as parasites in the bodies of caterpillars.  They eat their way out of the caterpillar, killing it in the process.  Some bushes have the power to release a pheromone that calls those hornets to come defend them from caterpillars when they are attacked - a protective ring of these types of bushes can keep dangerous leaf-eating caterpillars away from your crops by sending in hoards of hornets.

Dwarves with access to alchemical solutions may also try to dump poisons on their fields, in the hopes that it will do more damage to the insects killing their crops than it does to themselves when they eat the crops.  There may be some drawbacks to this course of action, however, as pesticides tend to kill far more than just the vermin they are trying to eliminate.  Pesticides are also notable for killing off the other counter-pest creatures, like beneficial spiders that hunt the pests.  You can stop the advance of pests for one year with pesticides, but if you kill off your counter-pests, then you are leaving yourself more vulnerable to the next wave if you can't kill them all with your pesticides.

Counter-Counter-Pests:
Bugs Bunny: "Of course you know, this means war."

Counter-pests should also have some weaknesses.  When there are spikes in plant populations, their parasitic predators will soon spike in population, and this allows for a spike in the population of the creatures that eat those pests.  This opens the door for the next thing up the food chain, or the diseases or parasitic fungi that attack your spidery allies to also spike in population, or small birds or insectivorous bats will swoop in, until the whole food chain hits an equilibrium.

Population modeling
As with the "virtual seeds" in the soil of the wilds around your fortress, the populations of all these vermin in the vermin wars need to be calculated off-board, and generally fairly invisibly to the player.  The player should basically only know of how the war is going by looking at his fields, and seeing a report that says how much damage the enemy has inflicted upon his crops, and what enemies are attacking.  The player should also only get a readout of what kind of counter-pests they have, and a vague sense of their numbers, as with a "numerous jumping spiders" and "few orb spiders". 

Every few growdur ticks, the foodchain is allowed to work - every creature is given its chance to attack its prey, and reduce the prey species' numbers, while at the same time, the access to food gives their own species a boost to their population (geometrically).  A certain number of prey creatures needs to be eaten in order to maintain the population, or starvation will actually decrease the predator population.  If it is a crop-targeting pest, then it instead deals damage to the overall "harvest" value of the crop, lowering the total number of harvestable crops.

If pests can go without being preyed upon, then their populations should start to truly explode, especially if they are bacterial or the like.  This can cause the sort of "Potato blight" scenario.  A dense population of crops unprotected by whatever predators evolved to kill the pests will allow the blight to utterly wipe out their prey crops, especially if it was an imported crop, when the pest's natural enemies have not yet emigrated to combat the growth of the pest. 

Also hopefully, it might be possible to create something akin to the "swarm of locusts" as a disaster that might occasionally strike your fortress.

Non-Vermin Fauna:
The game should also keep track of the number of plants and vermin on the field because it will attract or repel the herbivores or carnivores that feed upon the plants and vermin, as well.  Although an elk or a wolf is tracked individually, they should be attracted whenever their favored types of food are in abundance in an area, and this means that if you want to hunt specific prey, you might want to start ensuring that the types of plants or vermin that attract specific animals are in abundance in your region.  If you want to hunt unicorns, for example, and they are fond of a specific herb to graze upon, then growing some near the edge of your fortress as bait would lure them closer in.

Naturally, wild animals should now start eating either the grass and shrubs or the vermin or possibly the occasional megafauna attack on regular creatures.  This may require having more "herds" of animals on the map at the same time, so that predators can actually arrive on the map at the same time that at least a few kinds of herbivores are also on the map. 

Creatures like wild antelope or rabbits or elephants should also seek to eat any sort of plantlife they find, and hunt after more preferred crops.  If that means that your fields, and their nice dense rows of tastier vegetables are a juicier target than the open fields of mostly grass, then they'd prefer to get into your fields, if they can see any chance.  The same can also be said of grazing livestock animals like cows, which would love nothing more than to have a buffet in your tomato patch when you've been trying to get them to eat all that clover, instead.

"Pest" Plants:
Weeds are just plants whose seeds are in the soil, which are better suited to the soil or faster-growing than your desired crop, and which can crowd out the crops you were trying to grow.  Weeds might actually be another crop you have been planting in the past, such as clover to let fields fallow, when you are trying to grow corn, instead.  Weeds can be targeted by purposefully hitting those weeds with the pests that they are vulnerable to, or using some form of "weed killer" from alchemy, but the most selective form of weed killer which has the least impact on the land itself is a dwarf just uprooting the weed, and replanting the seeds of his chosen crop.

Invasive Species:
Because of the way that the system is set up, pests that are "locked out" of a farm initially, if you grow the crops in a greenhouse environment, would have unhindered growth until pests somehow found a way in.  The more that a population becomes abundant, however, the more likely it becomes that some creature will adapt or migrate to take advantage of the unchecked food source.  An initial wave of pests might seem unstoppable, only being countered by counter-pests once you can get them, breed them, and unleash them upon their foes.  Worse, it's kind of hard to have spiders ready and waiting on "guard mode" in your farms, because they need to have a prey species to survive upon or else they will starve and die off.  Basically, you can only launch your defenses after the invasion has already started.  It might be better to just not even try to sterilize your fields.

Likewise, "invasive weeds" might initially have no natural predators in the wild, and overtake the fields... until something either adapts to take it down, or some species is introduced that feeds upon those weeds.

Environmental Factors on Pests
Winters kill surface pests, especially insects, and especially during freezes.  Doing major tilling or movement of the soil or just digging through the soil should also kill plenty of insect pests.  Floods might bring in new microorganisms, as well, inviting new diseases or fungi that kill those diseases.

Earthworms, nematodes, and other beneficial vermin
Certain creatures in the soil could be of benefit to the soil and the crops in general.  Sterilized soil might lack these beneficial vermin, which may aerate the soil (adding to the drainage score), or help biodegrade excess biomass, or even clean up toxins in the soil.

Pollinators
Another type of beneficial vermin creature are the bee, the butterfly, the moth, the fly, the bird, and the bat.  Since many plants are incapable of fruiting without a pollination vector, and wind is not always the best means of pollination, bees and other pollinating insects and fliers are critical to growing food.  For underground farming, especially, bat-based pollination may be necessary. Underground, wind-based pollination is likely to go only one or two directions, if at all, based upon the cavern's physical layout, while water-based pollination is going to be torturously slow.  Having a movable bat colony that you could use to pollinate your crops could be a means of having a controlled farming system.

Dwarven control
To an extent, much of this warfare takes place in the abstract, invisibly, beyond the control of the dwarves, themselves, however they still need to have at least some management over the warfare of these vermin, as they play a major part in the success of their farms.

For starters, it should be possible for you to be able to see the vermin in the soil and then be able to mark some sort of vermin for capture by the appropriate dwarves from the menu where you look at the soil, and can see the list of vermin in that soil.

Once in captivity, you can release these vermin in areas of your choosing.  If you have no insects in your underground fields yet, for example, you might set up an airlocked fallow field for the purposes of breeding spiders to protect your fields.  You can bring in pests like flies or ants to serve as a food supply for the spiders to prey upon to build up their numbers.  In that way, you have a stable of spiders to hunt down any sudden invasion of pests to your fields that were formerly vermin-free.

On population models:
These are some resources on how these sorts of models can actually look:
http://en.wikipedia.org/wiki/Lotka-Volterra_equation
http://en.wikipedia.org/wiki/Population_modeling

Fertilizers

Spoiler (click to show/hide)

Let me get one thing out of the way, first off, to help people grasp the important part of this concept:

[size=12]Fertilizers must be expensive[/size]

I don't mean "expensive to buy from a caravan", I mean that they are rare, precious, and difficult to obtain, and worth conserving because of the difficulty of obtaining them.

Think Dune for a moment.  In Dune, water was so scarce and precious that you not only had to wear a special suit to preserve as much water as possible, but when someone died, a special burial practice was performed to preserve all the water they could, and when nobles threw water out on the floor, peasants would soak it up with rags to keep as much of the wasted precious resource for themselves as they possibly could. 

Fertilizers, likewise, need to be something that is really, really easy to waste, but for someone who needs every edge they can get, worth going through every possible trick to reclaim as much of that resource as they possibly can.

This means poo, by the way.  I'm talking about getting poo up to being a resource people cannot afford to waste by merely throwing it out and ignoring it.  And yes, this is a serious subject.

If we are going to make this a game where advanced farming is based around farming in crop rotations, and the "gaps" between the fertility of the field after one crop is done with the field, and the place where the field needs to be for the next crop to be able to grow are bridged by adding fertilizers (what I am proposing), then the key game balance issue upon which this entire system becomes either a brilliantly crafted challenge for new and advanced players alike, or a miserable joke where every problem can be solved by simply slapping the button for adding more of your nearly-infinite resources to the problem comes down to one major question:

How precious is your poo?

Yes, I'm serious, quit asking!

Making fertilizers that come from biological waste your primary source of fertilizer serves the important balancing purpose of giving you a set ready amount of fertilizers that you can use based upon the number of dwarves and livestock that you are actually feeding with your plants. 

Here's another important aspect to remember with using fertilizers, however: They rarely have one direct effect for using them.  There are about a dozen different variables for soil for a reason.  Everything you do to the soil will have an impact on more than one variable, not all of which are good.  This means overreliance upon any single resource will come back to bite you later on when you've managed to turn your soil toxic by trying to just apply the same fertilizers in absurd quantities to just force your soil to keep growing a single crop over and over.  To that end, I'm adding in some little guideline + and - symbols next to some of the fertilizers so you can keep track of what sorts of things this might have an impact on.

Urea - Untreated urine contains mostly water and a chemical called Urea.  As a nitrate, it is useful both as a source of Nitrogen in the soil, and as an ingredient for the concoction of explosives, including Saltpetre, one of the key components of gunpowder.  Urea is broken down by common bacteria that live in most soils into ammonia, which most plants use as their nitrogen source.  Spreading untreated urine on your fields replinishes the most nitrogen for your buck (as well as phosphates and potassium to boot), but can have unintended consequences as it may raise the total soil bacteria population up to dangerous levels, and kill sensitive plants with biological toxins, add to soil salinity, is typically acid, and generally "burn" plants (biological toxins).  Some plants may be resistant to this burning, however, so weed-type plants are perfectly fine with you just going to town on them.  It is possible to dilute urine in water significantly enough that untreated urine can simply be applied directly to the fields.  A special function on the irrigation pumps may make this a decent idea. (Soil: N +++++, P ++, K ++++, BioM +, pH --, Na ++, BTox +++)

Lant/Ammonia - Functionally, urine that has been allowed to age and to have bacteria process and break down the urine outside of the soil itself.  This is the primary source of ammonia in the Middle Ages and Ancient times, and was used quite extensively as a cleaning agent until the invention of the Haber-Bosch process in the 20th century.  (Yes, treated urine is used as a cleaning agent.)  Ammonia is generally known to be a weak base, as opposed to urine, which is generally a weak acid.  Both these would be impure forms of urea and ammonia, but the general rule should apply.  (Soil: N +++, P+, K+++, pH +)

This means we will need to have some sort of "Gong Farmer" job, and chamberpots, although "sewers" are pretty popular with the crowd that wants biological waste in the game.  If we use sewers, collecting all the useful urine, and separating it out from the common water and solid waste may be more difficult, and may require dwarves whose most unfortunate job it is to actually strain out the useful urine from the useless other waste that goes in the sewers. 

In some communities where Lant was positively necessary for the industry of the area, however, it should be noted that there were communal requirements that you actually save your urine, and keep it separate from any other waste so that as much of it could be harvested as possible.  How, exactly, such an edict could be put in place over your subjects, and what its social ramifications are might require further discussion, however.

Manure and Night Soil - Solid waste provides another form of fertilizer, although, as the stuff the body couldn't use, is not quite as useful as a direct injection of nitrogen as urea is, although it would work well as a means of building up the humus layer.  Night soil is the manure of humans (or in our case, humanoids), and can put potential parasites and pathogens into the soil above and beyond the problems of regular manure.  Human waste also carries heavy metals, which can gradually accrue heavy metal pollution in the soil.  Applying manure or night soil directly to the fields can "burn" plants, which cannot take the increased bacterial activity and biological toxicity.  Composting the manure or night soil, however, leads to a significant drop (up to half) in the nitrogen content of the night soil, while at the same time, mitigating many of the downsides. (Soil: N ++++, P +, K +++, BioM ++, pH --, Humus ++, BTox ++++, MTox +(humanoid only))  May also add pests in the form of parasites and diseases the host animal passed out.

Compost - Night soil and manure left to compost outside the fields allows many of the biological processes that decompose those materials to take place away from the plants, where the toxic chemical byproducts of the funguses and worms and bacteria do not harm the more delicate of the crops you grow.  Composting can be done by a vat filled with fecal matter with worms to go at it and the like, but we might also try "composting by farm", and grow some kind of mold by tossing waste on it.  This would mean growing a type of "crop" purely on manure whose sole function is to generate a "fruit" of fertile soil for collection.  Composting can also take place with dead organic matter of all sorts, including clothing, wood, rotting food, dead creatures, etc. (Soil: N++, P+, K++, BioM ++, pH -, Humus ++, (Humanoid only - BTox +, MTox +))

Wierd has supplied a good source for far more detailed how-to-bake-your-own fertilizer, which includes data on specific nutrients that can be gleaned from specific composted biomass.

Bone Powder - I'll grind your bones to make the wheat to make my bread.  Bones are a good source of Phosphorus, compared to the other biological waste, and so grinding bones will get you some of that phosphorus that other sources are generally lacking in.  They're also less toxic than some of these other options, in general.  Phosphorus also is a key nutrient in getting a larger fruit from a plant, and you generally can't overdose on phosphorus the way that many other nutrients do, so bones become much more valuable as fertilizer than anything but a mood ingredient if this suggestion gets through. (Soil: N+, P ++++, K ++, pH +)

Dead trees - Many real-life mushroom farmers simply water logs and seed them with spores of mushrooms to let the mushrooms grow, or grow them in beds of sawdust while inside of limestone caves.  I've said multiple times that we could make the "easy start-up crop" of the game just be a plump helmet growing on a wet log.  They obviously have plenty of nutrients, but they really don't add them to the soil until decomposers are done with them, so it's mostly just decomposer fuel. (Soil: BioM +++++++++) The decomposers should add other nutrients, raise humus, drop pH, and raise soil biological toxicity.

Potash - Dead trees burned up so that their potassium content is more readily accessable.  It's not as ultimately "nutritious" for the soil as letting a dead tree decay, and harvesting the mushrooms that grow on it, but if you need potassium in a hurry, then it works. Generally, it's kind of a waste of a tree. (Soil: K ++++++, pH +, Na ++)

Slash and Burn - The quick and dirty way to turn a forest into a farm.  Watch the elves scream in horror!  Basically, this combines dead trees and potash to a degree, since it burns part and buries part of the tree to get some quick fertility into the soil.  The problem is that you can only really do this once, plus it generally causes some massive soil erosion problems a couple years down the road. (Soil: N++, P ++, K +++++, BioM ++++, Humus +, pH -, Na +, BTox ++)

Green Manure - Created by either simply tilling the husks of crops, grasses, or other plant matter into the soil, or by burning them to ashes, and tilling that into the soil.  Basically, it's dead trees or slash and burn, but with old crops or grasses.  Clover is typically grown without being "harvested", you just kill it and let it decompose so that you turn the clover itself into fertilizer when it's done acting as a nitrogen fixer.  Depending on whether you burn it or not, it should be the same as either the dead tree or slash and burn, but with less biomass or nutrients for less mass to return to the soil.

Saltpeter - You can make this by combining urine, manure, and straw or other dead plant matter, and leaving to "ferment" for a few weeks.  Or you can mine it.  It's a key component of gunpowder, but also a prime fertilizer.  Yes, gunpowder is made by fermenting decaying plant matter with human biological waste.  (Soil: N +++, K +++++, Na +)

Dead creatures - Similarly to the "dead trees" method, simply letting corpses and remains rot on your fields may be a functional means of both disposing of and benefitting from general "waste" materials.  Special types of mushrooms may prefer or demand growing on a corpse (while plants generally will not), so we might have something like "blood caps" that only grow from horribly mutilated bodies.  You could even farm them by simply having a trapdoor that opens up, drops goblin invaders 20 z-levels, and then splatters their pieces across the field.  "Goblin! Apply directly to the farm plot.  Goblin! Apply directly to the farm plot." Not too much different from dead trees, though, really.  (Soil: BioM ++++++, BTox ++)  Rots on its own, more toxic and less nutritious than trees.

Guano - You can mine guano, the droppings of seabird that eat sealife which are rich in phosphorus.  Places like Chile had peaks of guano islands created by thousands or millions of years of bird droppings on the island that were mined for phosphates as fertilizers.  England and Germany actually fought a war over access to mountains of bird crap just before World War 1 because they were so valuable in feeding the populace of Europe.  We'd have to have some kind of special geological formation to have a "guano" "stone" layer, though. (Soil: N +++, P +++++, K +++, Na +, pH -)

Apatite - A sedimentary stone that occurs in real life at the bottom of dried lake beds, but not in DF right now.  Maybe it could be something you find in limestone plus things limestone metamorphs into, like marble.  Apatite requires processing that would require coke/charcoal and sand to turn into a phosphorus gas that can be turned into a phosphorus salt.  The way it is made in modern day real life is with an electrical arc furnace, but theoretically, some kind of metal pressure cooker on top of a magma vent should be able to reproduce the pressure and heat.  It requires heavy-duty equipment and a air-tight set of feeding and capturing chambers that can withstand high temperature and pressure, plus sound understanding of the chemisty going on. May simply be declarable "outside the range of dwarven tech", though.  (Soil: P +++++++, pH +)

Sylvite - A salt that occurs in Rock Salt formations.  Valuable source for mining modern Potassium sources.  It's also a salt, though, so it will cause some big problems with salinity if used extensively.  Plus, like most of the "mining for fertilizer" solutions, it's just not sustainable - you have to keep finding or importing more of it.  (Soil: K +++++, Na ++++)

Flux stones - For liming the soil, which raises soil pH.  Technically limestone is traditional, but chalk, marble, and other flux-y materials are useful, as well.  Needs to be turned into a salt to really get into the soil, though, and it generally requires some breaking down to get back to Humus.  (Soil: pH +++++, Na ++, Humus --)

Sulfur/Brimstone - Will acidify soil if you need to.  Also a salt.  (Soil: pH ----, Na ++++, Humus --)

Peat Moss - It's like mining compost out of a bog, but it's fairly acidic.  Add with flux stones to nullify the acid.  (Soil: N++, P+, K++, BioM +++, pH ---, Humus ++++)

Pollutants

Spoiler (click to show/hide)

This kind of became its own topic, but it should be a fairly short one.

Pollutants are an important part of the system, since they can penalize an over-reliance upon any given form of agriculture or fertilizer when used to an extreme degree.  When talking about pollutants, it's important to make clear how difficult it should be to remove pollution.  Some pollution types need to be harder to deal with than others, but pollution in general needs to have negative impacts that are at least notable or they lose all their meaning.  Having a form of crop that absorbs pollution is neat and should be included, but shouldn't also be a major valuable crop in other regards, or else the idea that something like soil salinity is a form of pollution and a real problem really fade away.  A salt shrub or grass that is only really useful for grazing livestock is fine.  Some kind of mushroom that grows in toxic waste dumps and is perfectly edible or useful as an industrial product while returning the soil to pristine form kills the whole notion that heavy metal pollution is something you should avoid doing to your fields.

There are a handful of pollution types I have settled upon: Soil acidity, salinity, biological toxins, and heavy metal toxins.

Soil pH and salinity are fairly well covered in other sections, so I don't want to rehash myself on those accounts.

Biological toxicity is a relatively easy-to-handle form of pollution compared to the three others.  Biological toxicity occurs from most forms of organic fertilizer, like rotting plant and animal matter or fecal matter.  It represents the toxicity of the by-products of the decomposer bacteria and fungi, which can be very harmful to many forms of general crop plants in anything more than small quantities, and can lead to infections just working on the soil for the farmers.  (How'd you like to work a field with rotting goblin corpses being tilled into the soil?)  Biological toxicity should be relatively fast and simple to deal with, since "decomposer crops" can just directly turn biomass into regular soil nutrients and clean up biological toxicity within about a few months to a year, depending upon how dense the pollution really is. 

*** LATE EDIT: Winner suggested toxicity could be represented as pests that grow under certain conditions, which inherently hinder growth or cause disease for plants.  This could be a more overt method of showing the problems of poorly-handled fertilizer, and reduce the number of variables tracked, but may also add an additional layer of complexity in making plants have different reactions. 

Using companion crop planting, it should even be possible to plant some decomposer fungi in some of the farm slots that are growing a normal crop, to cut down on the biological toxicity, so long as the biological toxicity isn't at problematic levels already.  Such companion plantings would require a small dip in amount of land in direct food production in order to keep the land generally in food production without having to stop regular food production cycles to put decomposers in as its own separate crop cycle.

Heavy metals are another kettle of fish, entirely.  While salinity is a problem that requires salt leeching, and soil pH can make different crops potentially viable, heavy metal poisoning (mercury, lead, arsenic, cadmium, chromium, nickel, uranium, polonium, etc.) of the soil can be the worst kind of problem.  Aside from accumulating in the people who eat the plants as well as the plants, themselves, heavy metals can be extremely problematic in that they are extremely difficult to get out of the soil once they start to accumulate.  Few bacteria can do the job, although certain forms of plants can do so (such as willow, apparently).  Any plant or fungus that can be grown in highly toxic soil should probably take a very long time to gradually chip away at the problem, and generally be very slow-growing and of little economic value.

A sufficiently toxic patch of land from heavy metal toxins may just be so polluted you'd rather just move your farm elsewhere, rather than wait through the process of reclaiming that land.

Heavy metals are often a problem in real life thanks to the water runoff of mining operations, or else metal smelting operations.  How much it will be possible that toxic metal can be carried of in rain or groundwater that are contaminated from contact with open mining projects or furnaces is up to Toady how realistic he wants his simulation to be.  Some of the means of fertilization already have ways to accrue heavy metal toxicity, however. 

Also on the topic of runoff, fertilizer runoff can create water pollution, which I'll put into the "farming water" section.

Alternative Farming and Living Resources

Spoiler (click to show/hide)

Section in its own post due to character limits in posts

Livestock

Spoiler (click to show/hide)

Section in its own post due to character limits in posts

Cross-Culture Agriculture

Spoiler (click to show/hide)

Section in its own post due to character limits in posts

Plant Growth and Farming Skill

Spoiler (click to show/hide)

Section in its own post due to character limits in posts

Advanced NPK, Crop Rotation, and Polycultures

Spoiler (click to show/hide)

Section in its own post due to character limits in posts

Xenosynthesis

Spoiler (click to show/hide)

Section in its own post due to character limits in posts

Nutrition Models

Spoiler (click to show/hide)

Section in its own post due to character limits in posts

Food Webs

Spoiler (click to show/hide)

Section in its own post due to character limits in posts

Animal House

Spoiler (click to show/hide)

Section in its own post due to character limits in posts

Farming Water

Spoiler (click to show/hide)

Section in its own post due to character limits in posts

[NW_Kohaku]

The tl;dr Version:

Spoiler: The problems this tries to solve (click to show/hide)

The purpose of this thread is to solve several problems with the current game:

• Problem 1: Farming now is, generally, easy and boring.
  
  • Farming needs to be more interesting, with more crops that are different.
  • Current crops (especially above-ground) are nearly indistinguishable in how they are grown.
  • Farming should include more physical types of farms and farming techniques.
  • Different climates and biomes make no difference in how you farm or what you farm.
• Problem 2: Farming is currently a front-loaded challenge
  
  • We have a "learning cliff" currently - you have to know everything to get anywhere in the game.
  • Once you know everything, the game is too easy, all problems have been solved.
  • The goal should be to make something simple to start with, so any random new player survives their first year, but progressively more difficult as time goes on, challenging those who have mastered the basics
  • It should be possible for players to get by without knowing terribly many tricks, but with rewards for the player who truly masters the system, in much the same way as dwarfputing.
• Problem 3: Deep without micromanagement or pointless complexity
  
  • All automatable things should be automated.
  • Anything that is a simple "what's the quickest way to make the most food" math problem is a calculation, not a choice, and that is automatable.
  • Once all automatable functions are out of the way, high-level management choices should be made as a matter of a strategy, based upon comparing difficult-to-compare-mathematically things, such as short-term payoffs vs. long time setting up but better long-term payoffs.
  • The game needs to do the basic math calculations itself, but leave the grand strategy to players.
  • To keep players engaged, occasional challenges should arise to bring player attention back to the farms, but not in a micromanagement way.
• Problem 4: Creating a Medieval Fantasy Ecosystem Simulation
  
  • While abstracted, the game should leave players with the sense that this is an actual ecosystem, where the changes they make impact their world.
  • This can mean clear-cutting a forest leading to soil erosion that means no more "free trees", for example.
  • This can also empower players to purposefully alter the ecosystem to better suit their needs if they have a mastery of the concept.
  • If the game models real-world systems, even if abstracted, it creates learning opportunities through play.

The reason this thread is long is because it is arguing for why most simple solutions do not actually address all of these problems, and why a marginally more in-depth solution is needed.  Specifically:

Spoiler: The solutions proposed (click to show/hide)

• Solution 1: Multiple soil quality variables
  
  • Multiple variables for soil quality (instead of just a simple "fertility") mean that different methods of fertilization can be used without being a simple calculation. ("What gives me the most bang for my buck?") You cannot rely upon a single fertilizer type, you have to use several.
  • If you have differing amounts of different kinds of fertilizers, crops that consume different nutrients become more or less attractive from a strategic perspective - depending on certain types of crops means depending on keeping the flow of certain fertilizers moving.  This is a strategic, non-micromanagement choice, whereas a single fertility value simply requires automating the re-fertilization.
  • Some soil quality variables can be easy to change (like how recently plants have been watered) but others much more difficult (like if the soil is highly acidic), and some may be specific to the embark site (no growing tropical plants in an arctic tundra).  This means different embarks are going to require different crop types and farming methods that support them.
  • Several of these soil variables, especially the nearly impossible to change ones, may add variables to the game, but not terribly much complexity, as players will have few reasons to actually worry about soil acidity or salinity once you have already picked a plant suited for your soil.
  • Pollution variables can exist as a means of making players choose between short-term gains and long-term damage to their fields
• Solution 2: Pests
  
  • Pest creatures, from fungal infections to caterpillars eating leaves to locust swarms, can exist as a "disaster" form of macro-problem for players to deal with, keeping farming challenging and slightly unpredictable.
  • They can be controlled through both limiting the number of crops of a certain type (reducing the exposure to a single crippling pest infestation) and using natural pesticides - the predators of those pest creatures, or fragrant herbs that repel bugs.  (Birds, bats, and spiders eat insects, many herbs use pheromones to repel insects or attract predators of pests like wasps that lay their eggs inside caterpillar bodies.)
  • Since pests are not seen on-map, the relative population numbers of pests can be tabulated invisibly and infrequently, saving on processor power and not appearing complex to players.
  • Solutions to pests can be easy to understand, if not necessarily easy to implement - bat or bird nests near crops with birds/bats in them will keep pests down.  You then need to acquire and care for those creatures, however.
  • Since pests, especially diseases, are attracted to large, single-crop farms, it is a problem more for players who keep using simple farming methods for years, so it doesn't overwhelm players on their first few years, but challenges players in established forts to diversify.
• Solution 3: Different methods of obtaining Fertilizers
  
  • Crop rotations with crops like beans that are nitrogen fixators is a common and simple solution to one type of fertility deficiency, but not all.
  • Decomposing organic waste (including corpses, grinding down bones, and manure) into fertilizer is the most sustainable and reliable method.
  • Some fertilizers can be mined or traded for, but they are of finite or unreliable supply.  Minable Sulfur can be used to change soil acidity, and make some soil suitable for different crops long-term, however.
  • Exotic fertilizers, like magma, can be used, but require more elaborate engineering, and may cause soil pollution that limit what crops are viable
• Solution 4: Water Management
  
  • Water is consumed by farms.  This means you need to constantly bring more water into your fortress.
  • Water generation from off-map or aquifers would be slowed down, to make there be limits on how much water you can use.  This would mean desert embarks are tougher.
  • More elaborate systems are more efficient but take more time to set up and reward player engineering skills with better farm yields.  Simpler versions like bucket-watering are faster to set up, but consume more labor.  Newbie methods are not "wrong", and don't lead to dying, but there is strong incentive to learn.
• Solution 5: Polycultures - plants affect other plants
  
  • Plants can be planted with/near other plants, such as the fragrant herbs that repel some types of pests, or ground covers that reduce water consumption.
  • Plants like beans that add nitrogen to the soil can be planted with some crops instead of just in a cycle - supplying more nitrogen even as the crop grows.
  • It is an advanced farming method for advanced farmers, but not necessary for basic farming - an "advanced level" for people who master the system.
• Solution 6: More farmable materials
  
  • If you can farm trees for wood, plants for cloth, flammable oils, medicinal herbs, and many other resources besides just food (basically, anything that isn't mined can be farmed), then there is a reason to constantly upgrade farms - you want more than just basic survival on food, you want more stuff.  This lets the complexity of a farm for food become easy, but having to sustain more farms for more crops for more uses means that you are encouraged to farm more complex methods to gain greater yields for non-essential products.
• Solution 7: Nature plays by the same rules (but abstracted more)
  
  • Non-farm soil tiles have a generalized region that shares the same fertility and pollution values as farms.  This means that when grass/shrubs/trees grow, it takes nutrients from the soil.  When they burn or are decomposed, it returns nutrients to the soil, but if you harvest from the land without returning, then the soil loses fertility just like a farm.
  • Trees are therefore not an infinite resource anymore - you may have to farm them, yourself to get greater yields.
  • Wild animal and livestock populations play by pest rules, to a degree - you have control over what is attracted to your land, and livestock can get parasites.
  • You can eventually turn a land into a place where nothing grows or functionally change its biome if you work hard enough at it.  Garden of Eden or Hellscape, choice is yours.

"But what do I actually do as a Player?"

Spoiler (click to show/hide)

So, rather than an overview of what each button does, let's instead go over what you would want to do when you are first setting up a fort.

"I just embarked, I want to farm plump helmets, since those are traditionally fast and easy to farm."
OK, so dig yourself some nice open space in your new fortress for your farm to actually occupy.  Once that's done, you want to open up your land management menu, and get to designating the actual farming zone.

You'd open the menu with "l" and need to start designating the farm with "w" to bring up the second tab, the farm designation window.  From here, zone out your farming plot.  There's no real need for "building" a farming plot in this system, it's just designated as the land that your dwarves will work when they are trying to grow something. 

To actually get them working, open up the scheduler window by hitting "e" to activate that tab.  We will assume you actually brought plump helmet spawn, and haven't eaten them all or something silly at this point.  So, what you need to do is create a new farming block, for a good three or four months, and then scroll over, and select the plump helmet seeds.  In all odds, if you just dug out the place for this farm plot, you will need to add to the biomass and water levels of this plot to make it suitable to the plump helmets' growth.  Plump helmets can survive with little existing nutrients and even soil of any kind, but grow on rotting material like logs, so you can simply add several logs to the farm plot, and keep them wet to both grow plump helmets, and also build up the soil nutrients, paving the way for future underground crops to be planted in the same area.  To do this, you need to add some permissables - use "z" to start adding permissables of providing water by bucket and logs to grow those plump helmets on.  Use the + key to keep adding the number of logs you will permit the dwarves to apply to the ground until the ♠ symbol turns green.  Make sure the ≈ symbol is green as well by adding to the number of permissable buckets of water to apply to the soil, and use the * key to add ten bucketfuls at a time, or - until you hit a negative number, which should make an ∞ symbol appear, designating infinite uses of that (presumably plentiful) resource.

This means you have set up a schedule for that farm to be used for plump helmet farming for several months, and have given permission for the dwarves to use the resources they need to run that farm.  Dwarves will determine whether they need all the materials they are permitted to use or not at the time, but permissables give you a chance to throttle how much they are allowed to use if you have some other priorities for those resources.  The logs you are using, for example, are potentially useful in other industry, and you wouldn't want to throw all your logs at just farming if there are other uses you have for those logs.

Speaking of which, you probably don't have enough logs to sustain that farm forever.  You need to designate some area for clear-cutting to generate the wood you need for your farms and future industry.  The land management menu has replaced the old designation menu for the purposes of designating trees for cutting down.  Now, you designate a zone where trees grow (similar to the plot designation), and then set the area to be continually chopped down for lumber.  You do this with the "a" Forestry menu.  You can also set it for herbalism at the same time to gather some easy food to tide you over until your farms are working at full capacity, and get you the seeds for aboveground farming.  Herbalism, in fact, is a skill with a simultaniously more defined and less defined role now that farms and wild soil are made to be fairly similar.  Herbalism easily supports small numbers of dwarves that can manage to keep their harvesting areas safe from skulking goblin ambushers, but at the same time, plundering too many plants from the soil without returning nutrients can gradually lead to a more barren landscape with less plants overall.

Also, while you don't need to flood your fields anymore, if you have bucket brigades watering your fields, then the closer you can get a replinishable basin of water to your farms, the less distance the dwarves have to ferry water by bucket from the basin to the fields.  Maybe later, you can improve this system by building irrigation pipes connected to a pump or sluice that will take the "bucket brigade" out of the picture entirely.


Now let's assume you're ready for aboveground farming with some of those seeds you gathered.  You've walled in an area, and also managed to cut down or gather all the trees and shrubs in the area, and need to have a denser, more sustainable food source.  Now you can start really building your first self-sustainable farm system, one which doesn't require outside resources aside from the manure the living creatures in your fortress generates.  To do this, zone another farm in this aboveground area. 

Why do you need to go through this trouble?  Well, plump helmets require a constant supply of more trees cut down to serve as their biomass.  This means that eventually, you're going to be needing hundreds of trees cut down each year to just keep your plump helmet farms going, not even counting the rest of your industry.  That can get unsustainable fast.

Now, you're going to need to schedule a more complex system than you did before with the plump helmets.  First, make a block to harvest some staple crop in.  Let's assume it's wheat, and wheat takes 8 months to grow, starting from Slate/mid-Spring to Timber/late-Fall.  Wheat is a "heavy feeder", and needs plenty of nutrients in the soil to grow.  We'll assume that the soil just happened to already be mostly ready for the purposes of this demonstration, but you might need to spread several layers of fertilizer or grow some preliminary crop to prepare the soil for this wheat.  Growing wheat will probably require permissables set throughout the year for several applications of manure, water, and it will probably be a good idea to give permission to till the soil before the wheat is planted (this kills weeds). 

Now, because wheat is a heavy feeder, it will severely deplete the soil's nutrients, so we need to have a system in place that lets soil fertility climb back up, and it needs to do this without claiming hundreds of trees per year that you may not have on hand to keep feeding your agriculture industry. This means crop rotations.  You can first try to at least maximize your growing season, however, and try to grow something small and light-feeding, like onions, as a winter crop.  Light feeders won't consume as many of your resources, and may grow faster than some heavy feeders, but generally aren't as economically valuable or capable of feeding as many people as your heavy feeding staple crops usually will.  They will, however, give you a chance to really use up the last of your soil fertility while you go ahead and start adding more manure and biomass to allow soil bacteria a chance to break down your fertlizers into usable nutrients while still giving you something of economic value from your fields.

Now you need to schedule what you do for the next year, and we want this year to be based around the idea of replinishing the soil for the wheat the next year.  This means it's going to be a two-year crop rotation cycle, which has the advantage of simplicity, although it isn't as efficient as more advanced crop rotation cycles. 

Instead of merely "fallowing" the field, which lets weeds take over, we will instead schedule clover, which can serve as fodder for livestock.  Clover is a soil replinisher, it actually adds some nutrients to the soil as it grows, although it still does take up some other nutrients.  During the time that clover grows, however, we can send in our cattle and livestock as grazing herds, and let those cattle leave their own fresh fertilizer behind on the fields as they eat - giving us back some of those nutrients that the clover wasn't replinishing all by itself.  We can grow clover for part of this "set-aside year", and another replinishing crop that produces an edible crop, like peanuts or soy beans, during their particular season.  During all of this time, we keep on adding manure, helping push up the soil fertility in anticipation of the heavy feeders to come next year.

At the end of this second year, we can then set the scheduler to be on "repeat".  The wheat will then be slated to be grown the next year, and before it is planted, the soil will be tilled, fertilizers will be added, the wheat will be sewn, and wheat will be grown, watered, and harvested, all on auto-pilot by the dwarves.  A single scheduling session at the start of the first year is all you need to do to set this up.

Of course, that doesn't mean that farms will run perfectly forever - pests may strike, fertilizers may run low, droughts may occur, but generally, you can leave the day-to-day operation of the farm in the hands of the farmer dwarves, while you can focus on other aspects of the game.


This doesn't mean we have to stop at the simple, however, as the point of this is to allow players to enjoy far more variety in their fields, and to have more complex and nuanced systems, without actually forcing the player to perform any more complex a calculation than they actually want to perform.

We could also try to have a farm plot that grows alternative or more complex sets of crops - for example, orchards of fruit trees that require bees for their pollenation, and which must be spaced apart from one another to have space to grow, with grasses or even legumes growing in the spaces between to both maximize land use and restore nutrients to the soil.  So long as cattle only eat the grasses, and not the trees or their fruits, their fertilizers will be applied directly to the soil the orchards are using without any significant use of labor.  If the orchards are attacked by pests that destroy your fruit harvests, you can even try growing flowers that repell the enemy vermin, or attract insects that prey upon the vermin that are eating your fruits.

This requires use of the scheduler to create a polyculture - growing more than one plant on a single farm plot.  Using the scheduler, you need to add more than one type of seed to be planted, and the ratio of the seeds that you will plant. 

This would take care, however, as the soil conditions would need to be suitable for all those plants at the same time - so plants must be selected that require very similar growing conditions, but which won't actually compete with one another over space.

Meanwhile, another player may try their hand at a fishery and a chinampa, creating a floating "raft" to place their farms upon over the fisheries, so that fish and crops can be grown in a single location. 

Pretty Pictures:

Spoiler: Plot Viewer Window (click to show/hide)

I get a little carried away naming things, sometimes.

Spoiler: Plot Zoner Window (click to show/hide)

Spoiler: Scheduler Window (click to show/hide)

Spoiler: Improvements Window (click to show/hide)

The green thingy is a glass irrigation pump.  The brass irrigation plumbing has been built, and the dwarf is smiling about it, I guess.

[NW_Kohaku]

Fortress Interface (the how it could operate part)

Spoiler (click to show/hide)

Now for the truly difficult part: The part where players are capable of approaching a complex system, yet being able to understand how to control their environment in as intuitive a manner as possible.  I'm going to try to break this up into different sections based upon the menu you use to access the information.

Direct Look

This actually brings me to my first little puzzle - how do you "look" at soil?  There's 4 different keys to "look" at something, 'k' is used for finding general objects, 'v' usually is used for units, 'q' is used for building operation, and 't' is used for components inside of constructed objects.  If something is just going to be the "soil", and not a "built farm", then keeping the standard method of viewing and controlling the soil as the 'q' button seems a little odd.

In viewing the soil, just as an object, I think I settle on the 'k' key, and having a special screen just for looking at the soil when you hit 'enter' at it as an object that sits on top of the floor.  Alternately, there might be some function of the farming interface that lets you look at specific soil, but I think that putting more information into the 'k' key, which is your primary "I want to look at objects" button, should be a good means of trying to find information on the soil without having to go into too many layers of screens in another interface.  (Unless there is some reason it would be very difficult to code putting different types of information in the k-look->enter menu, it would be better for intuitive information-gathering to make it k-look based.)

Direct looking at soil should give a list of all the vermin contained in the soil, as well as what seeds may be germinating at the time.  Detailed viewing of the soil should give a list of all relevant factors about the soil that the player is capable of knowing about the soil.  This includes everything in this mockup post

Farming/Land Overseer Menu

The degree of high-level control the player will need to exert over the farmers with these changes mandates a whole interface dedicated to the task of land management, of which farming management is a major part.  To that end, I suggest taking up the 'l' binding, since fortress mode doesn't use that except in certain build contexts, and making this key call up the land overseer interface.

From this interface, everything you really need to know or do regarding farming (except control of individual farmers and the engineering of new locations to farm) and other tasks related to agriculture and land use are viewable and controllable.  This is split into several different tabs which should have their own keybindings for ease of control, similar to the military screen, although I would recommend just making the tabs ordered as "q", "w", "e", "r", "t", etc, so that they are organized based upon spatial reasoning than upon "u" being the most important letter in "supplies". 

I recommend the following tabs:
Farm Plot Viewer
Farm Plot Zoner
Farm Plot Scheduler
Farm Plot History
Farm Plot Improvements
Herbalism/Forestry Zoner
Ranch Manager
Biological Encyclopedia
Food Web Chart
Wild Land Data

Some sort of memory of what farm plot you were looking at should be involved, allowing you to switch to the History of that plot from the Viewer.

Farm Plot Viewer

• The 'default' view, the farm plot view is for taking a quick look at all your farm plots, and their status.
• This spawns a list of your currently zoned farm plots, from which you can choose which plot to zoom in on.
• There are sub-windows that let you view different aspects of the farm:
  
  • What plants are growing (and how full they are growing and how damaged the crop has been) (also, what irrigation method is being used)
  • Nutrient levels in the soil (see the first image) (Water, N, P, K, CEC/Drainage, and Humus level)
  • List of pests/vermin in the soil that farmers can see.  Diseases like bacteria and fungal infections can only be seen if they are attacking one of the plants in the area.  Vermin are color-coded by how they are enemies of your current crops (red), kill the pests of your current crop (green), kill your counter-pests (yellow), and are irrelevant to current pest battles (gray).  "Zooming in" on pests to get a detailed view (going to the encyclopedia?), including a rough estimate of their numbers in that one plot.  Either an icon or just a phrase or number could also be placed in the sub-window before zooming in to give an idea of numbers.
  • Pollutants are listed in another window, similar to nutrients, but for what doesn't fit on the first page (pH, salinity, biological toxins, and mineral toxins)
  • Magic energy fields' (or just Good/Evil and Savage if we aren't implementing that yet) influence can be viewed, if we are capable of directly seeing which spheres have power in the area of the farm plot.  This can be as simple as listing all the spheres active, and a simple statement from "negligible" to "overwhelming" influence in the area.
  • Climate of the field can be displayed.  This would give soil temperatures, how much rainfall can be expected, whether the area freezes in winter, and other information relating to the environment that affect plants.

Critical Issue: Nutrient Display Method:
Soil fertility displaying NPK soil stats are fairly difficult to describe in absolute terms without talking about soil chemistry in a game-breaking manner.  Talking about soil in terms of how the leaves are healthy (or magically foreseeing that they will be healthy) is a way of side-stepping this issue, but it becomes relative to the crop you are using now, and essentially impossible to foresee how another crop will react to the same soil.

If you see soil only relative to the current crop, you will need to ask the game to tell you how it treats a different crop, requiring you to actually pick that crop, and ask whether that crop will like this soil or not.  This gets compounded when you are talking about trying to choose multiple crops in a row - you will need to foresee where the soil's fertility will be after another crop is done using it.  This can be done with a good interface, which will let you forecast soil conditions several plantings in advance - queuing up multiple plantings, tallying the soil treatments that would take place, and then giving a list of what is then suitable for the soil at that time.  In this way, the soil is only ever described in terms relative to crops, but it will require some pretty tricky coding, and will have much potential bugginess.  This will require that at every step in the process of planning out multiple plantings into the future, the game will have to figure out how many applications of fertilizer (within a player-set limit to how much of each fertilizer may be spent per unit time) will be used to make soil conditions ideal for the next crop in line.  The player will probably want to punch through enough crops to make the soil once again ready for whichever crop started the cycle, creating a crop rotation schedule.  (This means that soil conditions after the last block, assuming that all the fertilizer available in the first block could be used upon the soil, would allow for the soil to be in very close to the same condition as the soil started the scheduling on.)  Considering that there may be three or so crops per year in a crop cycle of up to four or five years, this may mean generating a system that can go up to 15 plantings and 5 years into the future. 

For putting all the heavy lifting of this system on the computer and the code, however, this means that players can schedule crop rotation systems without ever having to consult wikis, raws, or spreadsheet calculation tables.  You can even introduce completely procedural plants without ever revealing the true stats of each crop to the player, and let the interface do the work of calculating how to present the information and how the crop rotation cycles can be built.  I believe this method of building the system would take a significant amount of time coding and bug fixing on top of everything else I have suggested, but would wind up giving the best possible experience for the playerbase once the system is in place, and would keep all but the most ardent haters of domestic production from refusing to farm because they would not like the interface.

The alternative to this is displaying soil fertility stats in absolute terms.   This would mean that they would be given the absolute value of the integers in the soil, and soil integer names, and absolute values of what each plant needs, and how much they alter the soil.  Much of the lifting is done by the player as they are forced to sit down with a spreadsheet or pen and paper, and try to plot out the exact sequence of actions that will manage to make a sustainable cycle.  This is much easier on the coder and the game, but a much greater challenge to the player, and I don't think very many players are going to want to really explore farming if this it is implemented in this way.  A few repeatable systems will probably be put up on the wiki, and few players will likely experiment beyond that point.  The overwhelming number of players would likely not want to use the farming system any more than they are absolutely forced to if this is how the system gets implemented

There are, of course, shades of gray between these two, but the first of those solutions is the ideal way to bring this system to the players.  All I can say is, "We love you Toady, but please bite the bullet, and make the complex, micromanagement-reducing interface for us."  I assume that the first of those solutions is used for the rest of this section.

Farm Plot Designator

• Allows players to designate new farming zones
• I would base this off a combination of how burrows currently allow you to color-code, and how farm plots (or the placement of floors or roads) are designated currently, just to make things easy
• An "undesignate plot" option would also need to exist.
• A "rename" as well, since it's more logical to be here.

Farm Scheduler
This is the meat of the system, where much of the actual work takes place.

It will unfortunately be much more crowded than some of the other tabs.  Several full-screen windows may be necessary.

For starters, there will need to be a farm plot selection screen, just so that it will not take up part of the screen for the rest of the menu selections.  Alternately, it just carries over what plot you were looking at in the Viewer section, so that you skip that part.

Scheduling involves several steps:

You must first build scheduling blocks, typically the length of time you want a crop to be cultivated.  This might generally be best to do on a per-month basis.  Every crop would take at least a couple months to cultivate, if not 7 or 8, but players may want extra months to ensure that dwarves have time to plant the seeds.

After this, the player has to select the type of crop that will be the primary crop.

All of the seeds that the player has access to is listed in a screen like the embark screen (where adding a new item to be taken with you takes you to a new categorical selection screen where you pick the specific item you want to add to your list), with color-coded icons designating how well they will handle the current soil.  Each icon's shape should be a different aspect of the soil, and the colors of the icons of course represent how well the seeds will do in soil with those qualities.  (Remembering that this may be a projected quality of the soil if this is not the first scheduling block in the schedule.  If, due to a shortage of labor or fertilizers, the quality does not reach its expected levels, the player will need to have a warning flash up.  Some sort of option over how intrusive the warning will be may be necessary, because a pause game and recenter warning may be desired if people are depending upon these crop rotation cycles working for years uninterrupted to survive.)

Every farm plot should also have a permitted amount of every kind of fertilizer to put into the fields per block.  Certain routine actions may be included in this "fertilizer" field, as well, such as activating or deactivating manual weeding in the field, including killing off (or chopping down, if a tree) any .  The application of chemicals to the soil would also fall under this category. Trying to seed the soil with some beneficial vermin could also count as a routine activity, which you could schedule in this field, as well.  How or if the soil is allowed to be tilled falls into this category, as well.  Because there are myriad different things that can be added into this list, and in different amounts, it may be best to build this section similarly to the embark screen's ability to pick specific things to bring with you on embark: Adding to the list brings up a menu where you select category, then specific item, then amount.

To create polycultures, so that, for example, azolla can be grown in the same field that already has rice growing in it, or other companion planting methods, another feature needs to be in place to allow for both selecting the additional seeds, as well as for setting a ratio of seeds to be planted.  This can use the same keys as the permissions for fertilizers and other routine actions - a button to add new types of crops in as a companion crop (from an embark screen type of second menu), and then using the + and - keys to adjust ratios of seeds to be planted, the way that you would adjust how many uses of fertilizer you give permission for using.

Some special types of farming require exceptions.  Orchards, for example, involve planting a tree, and harvesting its fruits or nuts for as long as the tree lives. 

Ranching also needs exceptions - you need to be able to set when a plot is going to be growing a cover crop like grass that is permissible for ranch hands to allow livestock to graze the plots of land.  (So that in a crop rotation cycle where the land is left to grow clover when it fallows, you can let cows graze the clover, and leave their droppings in the field.)  Herds should be given access to different farm plots during different scheduled stages in crop rotations in this screen.  Herds are designated in the Ranching screen.

An even more major exception would need to be created for aquatic farming methods - having options to flood fields and grow algae requires that the farming system be capable of recognizing flooded fields and set up the ability to farm from chinampas or farm seaweed or cultivate fisheries in conjunction with farms.  These could be handled much like dry farms, but with a recognition of what can be grown with flooded fields.

Since these are complex, time-consuming loops that players must build, having an import and export option (to a .txt file the way that embark profiles work) would help the process, letting players simply use their favorite crop rotation schedules over and over again if they can set up the initial conditions to start the loop in the first place.  In the case of a four-year crop rotation system, the player might export one of his field's schedule, and import it on the other three fields three times starting on different years, so that four different plots are running the same schedule, staggered out a year at a time.

Khym Chanur adds that the crop rotation methods of a civ should become civ knowledge that players can import and start with. 

Khym also would like for the game, if possible, to calculate how much fertilizer would be required at that point in a farm, and recommend the appropriate amount of fertilizer permissions.

Farm History
Displays the past actions taken upon the farm plot.  Something perhaps looking a little like Legends Mode. 

This will show what crops have been grown in the past in the plot, what initial conditions of the soil were like then, what pests attacked, and how fiercely, and how many seeds were planted, and how much of the crop actually survived to harvesting.  It should also give a tally of how many dwarf hours of labor was put into that farm plot for that period (in per-month format, perhaps), not counting transit times.  It should also show the relative effects that planting season had upon the soil when the crop was harvested.

The purpose of this is to give players a better sense of the impact of their previous decisions, and to be able to see what worked and what failed.  To that end, it might be nice to keep the history of some plots that are deleted if they actually have at least a year's worth of history.

Farm Improvements
This gives the player a chance to designate the less routine types of work that are done to a field.

This includes the watering method of the farm plot.  Basic farming involves either rain watering or bucket-brigade watering of a farm plot.  Improving the irrigation of the land can involve making minor channels in the field itself, or using a series of copper or brass tubes as a "sprinkler" system by punching holes into them.  This would involve having a special irrigation pump built next to the farm, so that dwarves could control the pump and adjust how much water is allowed into the farm without direct player input, automating the whole process.  The brass tubes combined with an advanced irrigation pump should also allow for dwarves to carry fertilizer and apply fertilizer to the fields at the same time as they irrigate it by putting the fertilizer into a hopper on the advanced irrigation pump, saving some labor time by not forcing them to spread the fertilizer manually and water the soil, as well.  As a safety feature of the device, the pump should only be active for as long as the dwarf is actually manning the pump.  I say brass, but I would also accept "lead" or "copper" or maybe "bronze" or even some flavor of "gold" or "silver" if you've got nothing better to do with the stuff, but not "iron" or "steel" (which rust).  Using lead might actually lead to some heavy metal pollution in the soil, as well, so hey, that might be something Fun to sneak in there, right?  Adding in an advanced irrigation system would require building those special pumps next to the farm plot, or with a pipe section built to the farm plot, with no forks in the pipes off of the farm plots allowed.  (Each pump needs exactly one field to water.)  Making the upgrade to advanced irrigation piping takes having a sufficient number of piping pieces and application of a significant amount of labor.  Where the pieces actually go should be left to the dwarves themselves, so as to insure that the game can assume that a dwarf operating the special irrigation-only pump will consistently water every tile evenly, and that there is no overlapping watering.

Adding logs to a field, as with the example of farming plump helmets by growing them on logs, could be considered a one-time event, and be put in improvements.

The ability to designate specific plants for weeding or cutting down (if the plot is not set for automatic weeding) would also  have to fall into this category.

A means of purposefully adding counter-pests or even pests to your fields should also be possible in this menu.

Likewise, adding gravel of certain kinds of stone as a slow-release nutrient would be an improvement of the soil.

Any other one-shot action I didn't think of would go here.

Forestry Screen
In this screen, the player will have the ability to set zoning for the purposes of collecting resources from wild/uncared for soil locations.  This can actually just be a farming plot that you let grow wild, or a designated wild soil zone.  You can set these areas for cutting down any trees that grow to maturity, harvesting bushes with herbalism, and possibly also designating fishing zones, if "farming" also bleeds over into fishing.  This will allow players to no longer be forced to manually designate what trees to cut down - you can set which areas are to be cut down.  (Potentially, an advanced scheduling screen function might let you designate trees to be cut down at certain times of year.)  Options can also be included for other means of dealing with the wilds, as well, including slash-and-burn of the plants in the region for the purposes of slash-and-burn agriculture.

Ranching Screen
This screen exists for the purpose of organizing "herds" of animals.  Herds are for grouping animals with like dietary needs so that they can be managed by dwarves through areas that feature silage or grazing grass.  Herds need to be scheduled through the areas that they graze to manage overgrazing of fields, and to ensure that you don't leave a cow in the corn field where it can eat your best crops.

Biology Encyclopedia
This should form a searchable database of every plant, vermin, and creature that the player has encountered.  Including the best relative stats that can be displayed to the player involving their growing needs (if a plant), their combat potential (if a creature), their place in the Food Web, and their economic uses (like what an average version of that creature slaughters for).

Food Web
A companion to the Encyclopedia, this screen's purpose is to give a visual aid to the player in identifying the complex relationships between the various pests and vermin and their predators and prey.  When a plant, pest, or creature is selected, that creature is put at the center of the screen with its possible prey listed to the left, and possible predators listed to the right.  The player may scroll through the lists on both sides, and re-center upon any of those lifeforms, and also skip back over to the Encyclopedia entry of the lifeform they are looking at.

Wild Land
This screen combines the plot viewer functionalities with a free-roaming cursor, so that you can explore what areas in the wild  are most ready for your development, and also to give you some idea of some of the more complex actions taking place outside the immediate control of your farms.  It would give a sense, for example, of the magic energy fields that are shifting in the caverns outside your immediate zone of control (but which you have revealed). 

This section is not considered complete.  Suggestions are welcome and encouraged.

Adventurer Interface:

Spoiler (click to show/hide)

This section is its own post due to post character limits

Common concerns and questions:

"But how much FPS cost will this have?  Temperature is already a big drain on FPS, and this will require more calculations on tiles!"

Spoiler (click to show/hide)

Note that temperature slowdown occurs because every tile is calculated for heat.  If you have a 4x4 embark (each embark tile being 48x48 regular tiles) with 200 z-levels, then you have 4 * 48 * 4 * 48 * 200 = 7,372,800 tiles in an embark.  Cycled through every temperature check. 

Now then, the soil system I am proposing makes soil a "floor covering", and in that same area, would be 147,496 tiles to check (and I further proposed conglomerating embark tiles worth of wild soil together at that, which would significantly reduce this number down to about 16 plus however many plots the player makes, which is not altogether dissimilar to what you just said in "tracked by layer", anyway).  These are performed on growdur cycles, which occur every 100 normal frames, further reducing the number of calculations taking place by two orders of magnitude.

Many of these calculations are not terribly complex, at that, and this is thanks in large part to keeping much of the system as an abstracted system outside the sight of the player.  You can fudge around nutrient values between soil tiles near one another if the player never can measure the exact amount of phosphorous that last tuft of grass took from that one tile. 

This suggestion's relation to other suggestions:

Spoiler (click to show/hide)

Part of what makes this suggestion a giant, sprawling mess that requires this reorganization is that I am also thinking in terms of how to rebuild several other aspects of the game, with regards to making a more involved experience on the part of the player.  Some part of what makes me appear to type this so breathlessly is that I am constantly thinking of the myriad other ways that the game might be improved, and how to manage the ramifications of one idea with another.  (If I spent half as much time typing as I do pacing and thinking about things, I would be even more verbose than I am now!) This means that some of what I am arguing for relies upon some other suggestions also being implimented at some point to work to its fullest potential. 

For starters, I think the entire "all the challenge of the game is in the first two years" aspect of the game should be overhauled, and that's why there is the Class Warfare suggestion exists.  This would not only make dwarves themselves a more precious resource, but also create a whole class of new items that you would need to supply your dwarves with as your fortress advances, making play more complex as the game progresses from being a mere outpost to a true city.

To make this really make sense, and not simply be a chore heaped upon the player, however, a certain precondition has to be met.  You have to be able to declare what your fortress will be at some point in the game, and be able to say "No, don't send more dwarves, I want to stay at this stage, where it's still simple". 

Dwarven Imperialism would help with this to a great degree.  I might just go back and make another, more sweeping suggestion at some point, but what relates most directly to what I am talking about now, what I want to focus on is having a way to declare what your fortress's purpose is.  If we can declare what purpose we want our fortress to have in the overarching story of our entire nation, then not only will we feel as players that the world outside our little fort is more important and has a place in the world other than being your own personal sandbox and shield against the waves of attacking goblins, elves, zombies, or whatever.  It also means that players can focus in on the one aspect of the game that we really want to play, and as such, the people that argume that "I don't want to have to farm" can have their preferred playstyle in the form of a military fort that might have its food supplied to it, provided it also trains raw recruits, holds a strategic pass, and sends soldiers out to the king or his generals whenever they are called upon to do so, and face more severe combat challenges to occupy them instead of facing the challenges of trying to manage more complex societies.  Players building cities for the purpose of trade or claiming and settling new lands, however, would need to worry far more about how to build a working society with larger populations, especially if we are dealing with rules from Class Warfare.  To a certain point, "build the largest city you can manage to maintain" might become the goal.

Bromance of the Dwarven Kingdoms is a spiritual successor to that post.  It involves numerous diplomatic and military suggestions for focus on a fort's external challenges, as opposed to this thread's internal challenges.

Ultimately, I bring this up because I think that at a certain point, this game is going to have problems with its player base simply wanting the game to be different things for different players, and the best way to do that is to actually make the game BE different things for different players. 

The Stranger is a thread about having travellers, which can potentially be a part of the entire caravan arc.  I'm getting a bit ahead of myself in mentioning this one, but having travellers that visit your fortress can be used as part of a system for trading biological materials from very distant lands, and also having a system of "regional specialties" basically depends upon having this sort of complex ecological diversity.  We can also advertise a certain regional specialty from our own fortress, and the like.

A very fun and thoughtful thread I have read while writing this is the Buzzin Beard thread which covers uses for bees and honey.  If dwarves can grow flowers for bees to grow specific honey types, it opens up new doors to bee use, and multiple aspects of industry and production can be tied together.

Another major suggestion that would impact this one would be one of the several suggestions relating to making mining more difficult, such as  Realistic Mining. This would make expansion in general more of a serious issue for a fortress, and make caverns more useful as pre-excavated areas, and also make stone more precious as a resource, the way that we need to make soil and dwarves also more precious, as you would have less metal and stone automatically at your disposal, and excavating an entire mountain would be a more serious project than merely designating an entire mountain, and waiting a week for your dwarves to finish their Mountaintop Removal mining project.  For the ultimate extension of this idea, there is Volume and Mass as a suggestion for an overhaul of the entire way in which you have to remove all the stone from a tile to make it accessable, rather than vaporizing whole walls.

With regards to the caverns system, I have had a couple of threads involving magic and the Gods, represented in Physical Gods and this thread's own spin-off thread, Xenosynthesis.

For further suggestions on magic, Contract Magic.

Alchemical Material Property Tokens is the long-delayed Alchemy thread I reference a few times.  It is meant to work hand-in-glove with procedural plants and crops to make more interesting types of farming possible.

Compare animals and observed animal traits suggests how to add some interface features that allow players to have more insight into ranching animals without forcing players to rely on memory hacks for information.

Categorical or Algorithmic Exotic Taming relates to the cultural knowledge aspects of the game, and as such, could be used in conjunction with farming, as well, giving crops taxa and knowledge about specific crop types could become civ knowledge.

Adventurer Personality Profiling.  Doesn't really relate, but since nobody reads this thread, I can just use it as a personal index of all my other threads for faster searching to reach back to everything I've written in the past.

Optional Starting Dwarf Numbers and Immigrant Control reflects an argument for severely cutting immigration and making labor more valuable.

Raw-editable Procedural Worldgen Cities for players to make their own types of cities.

Geologic Structures are a collection of real-life geology phenomenons to add with the 3d ore veins.

Potential and Affinity ties diet to potential growth rates.

[Max White]

Oh god, wall of text, It blinds me! Ah well, it must be read!
Wait, you mean to say in the origenal thread, people went so far as to say 'Just go play harvest moon if you want more farming'? For shame! Do people not know that complexity and depth are the heart of DF? Rest assured, there is nothing 'undwarfy' about being up for a greater challenge. It is in the spirt of the dwarf to face things larger then itself, as so many things are.

Well all in all a very compelling thread, and sad to say I agree with just about all points I have read. Boring, arn't I? Although on the topic of numbers and FPS, making a value for every squair to keep track of minerals has FPS lag potential. Instead, I propose that each farm plot have it's own value. You can't farm off a farm plot, so values for that land would be irrelevant.

Reading your thread, I had thoughts of a water flood chamber, to irrigate my farms, hooked up to a pressure plate at the bottem of a lake that dries up and floods from season to season, therefor making a repeater that goes on a yearly rotation. Truely, this is a dwarfs suggestion.

[Spamyouhou]

I have found the different ideas you've posted so far very compelling (Yes, I did read the entirety of your post !).
I usually play DF in short bursts and can't force myself to play the game once I have passed the point where my fortress is set up , and making farming a more important and challenging part of a large fortress (instead of just being a temporary annoyance at the beginning) would definitely renew my interest in DF.

In my opinion, what's really important when talking about improving farming is making it more complex without turning it into a huge micromanagement fest. That's why an intuitive and simple yet powerful interface to deal with the repetitive stuff (somewhat like the military interface even though it can be improved too) is needed. It should be easy to automate harvesting, planting, fertilizing etc so that the overseer can focus on decision making and planning a long term strategy.

Also, I believe there is also a need for animals to have bigger impact on the environment. Right now, it's possible to have 100+ animals (not counting the wild ones) running around without any effect on your fortress (except for lag, of course). They should affect the running of a fort both in a negative (they require a lot of food) and a positive way (they get rid of pests and weeds, produce some material for fertilizer, and maybe are the only source of meat if few caravans are able to reach your fort).

Anyway, I am really eager to see what Toady is going to do when he decides to take care of the `Farming improvement` part on the dev page.

[Kattaroten]

I like all of your ideas, except the one about the research menu... Research should only be incorporated in the game if it is in a way that fits the dwarf fortress theme better then a tech tree. Because i mean, who likes tech trees?

[NW_Kohaku]

It's always nice to see responses, especially for something I spent an entire day writing, just because it can always be intimidating.  That I actually get positive responses is better still.  Thank you guys for taking the time to read all this (and hopefully the parts I'm going to attach on in subsequent updates).

Anyway, I had problems in the last iteration of this discussion partly because people were not seeing "the whole picture", as it were, but arguing against individual parts because I was involved in long arguments about those specific parts, and people thought I was only proposing a single piece, without seeing the whole machine, as it were.  And trying to point someone to a disorganized 46-page thread riddled with what basically amounts to the same argument going back and forth didn't get too many takers willing to read that much, so it basically wound up being the same people who had actually been reading the thread all along versus everyone else. 

Worse, it was like I was in a catch 22 - if I talked about parts being made more complex, then I was "drowning players in complexity", if I talked about having parts of it automated, then I was "making a pointless mechanic if players don't use it, anyway". 

So, basically, I want to front-load the really pie-in-the-sky abstracts in this new argument.  The details down in the mud are what I'm trying to build down to.  The last thread basically worked in reverse of this method, starting with discussions on how to make interesting or realistic mechanics, and then trying to put an overarching theme or idea on top of that to make the pieces make sense in the context of one another, while growing in complexity.

Anyway, I'm worried I didn't reserve enough space for everything, even with five consecutive posts.  My first post was split into two because it goes over the "mere" 40,000 character (11 pages in a text editor) limit for one post.

I'm thinking maybe to just use that "Implimentation" list tree as an actual table of contents, and linking the headers to individual posts with certain explanations if I really need more tl;dr space for some of this.

Anyway, up next, I wanted to post some "frequently argued rebuttals", then go into the arguments for certain things like maybe urine and feces, before going into the real nuts and bolts, so that only someone who has slogged through most of the abstract argument gets to the details.

Although on the topic of numbers and FPS, making a value for every squair to keep track of minerals has FPS lag potential. Instead, I propose that each farm plot have it's own value. You can't farm off a farm plot, so values for that land would be irrelevant.

Having every tile track individual soil stats technically wouldn't cause FPS lag unless it actually managed to take up all your standard memory, and forced the computer to use Virtual RAM, it would instead just take up a lot more memory.  FPS lag is caused by pathfinding - if a dwarf wants to path to a stone to use in some process, the game currently calls upon a chart of every stone in the entire embark, and measures distance from dwarf to material for every single stone in the embark.  That takes up FPS.  If it's just a number for measuring what will grow there, then it shouldn't have many FPS problems.

That said, literally putting in a soil quality measurement of every tile on the map would blow up your computer's memory, no question.  Instead, I think that something that tracks "wild soil" by embark tile-shaped zones and possibly elevation (differentiating surface from cavern, and layers of cavern from one another) where only tiles that have some sort of "soil" or "muddy" contaminant on their floor would be a more memory-efficient means of tracking things.  Every tile is just averaged into the overall larger pool of nutrients that is shared across a large region of the map.

This could potentially carry over into farm tiles, as well, where an entire "farm plot" of as many as 100 tiles (but probably more like 10) can use the same data, but this would need some sort of sanity checking to make sure that you aren't just dumping fertilizer and water only on the one closest tile, and ignoring the others.

If we are talking about a farm plot of 10 tiles that has a nitrogen level of 100 out of 255, and it needs to be at least 120 at all times for a plant, and a single unit of ammonia adds 20 nitrogen per tile, and you can have a bucket with up to 10 ammonia, then you can carry 200 points of nitrogen in a single bucket, and it will be divided by 10 tiles, for a total of +20 nitrogen to the whole farm plot's nitrogen score.

I like all of your ideas, except the one about the research menu... Research should only be incorporated in the game if it is in a way that fits the dwarf fortress theme better then a tech tree. Because i mean, who likes tech trees?

It's something I have some mixed feelings on, to be honest, but I think it can be rigged into a more interesting system if we can mix it with some procedural development. 

I'm well aware that players aren't likely to even be playing a game long enough to have serious development of new varieties of selectively bred crops on their own.  What I'm arguing for is that the game procedurally develops new selectively bred "superior versions" of some crop or another, based upon the history of the world.  This means that humans might have special forms of more productive wheat for trade than the wild wheat you might get from herbalism, and these would be different every time. 

Honestly, I'm not big on tech trees in general, myself, but Andeerz was really selling them a few months ago, as part of a way to have procedural growth of different races, and means of having trading of technologies and ideas to make one generated game become substantially different from another game.  It wouldn't be something that you could really build through very much in the span of time that players are making their forts, but it would be something that made worlds different more than just in terms of geology and a giant record in the "history" section of who killed what.


(And even my response is tl;dr... maybe this is getting to be some sort of clinical problem?)

[FGK dwarf]

A very well-thought-out post, in my opinion [I should probably say I've not read the original thread]. Certainly farming needs a big overhaul. Some thoughts:

That said, literally putting in a soil quality measurement of every tile on the map would blow up your computer's memory, no question.  Instead, I think that something that tracks "wild soil" by embark tile-shaped zones and possibly elevation (differentiating surface from cavern, and layers of cavern from one another) where only tiles that have some sort of "soil" or "muddy" contaminant on their floor would be a more memory-efficient means of tracking things.  Every tile is just averaged into the overall larger pool of nutrients that is shared across a large region of the map.

Soil quality need only be tracked for tiles that actually have soil, which won't be many (basically the surface, subsurface soil layers, and the caverns--farming on stone shouldn't be possible without moving soil from elsewhere. A 4x4 embark with 3 caverns, surface and say 4 subsurface soil layers would have (3+1+4)x4x4x48x48 = 300,000 tiles. If you want 4 bytes of data for each tile (water, nitrogen, phosporous and potassium) that's only an extra megabyte of RAM.

Maybe soil acidity could be introduced too? It could be changed by the application of lime.

Subterranean ecosystems could semi-plausibly be powered by:
1) decaying organic debris, whether brought in by your dwarves or washed in through the underground river system--this would make caves with no water pretty sterile, which is quite realistic.
2) Chemotrophs or lithotrophs--organisms that build organic matter out of inorganic (see http://en.wikipedia.org/wiki/Chemotroph and http://en.wikipedia.org/wiki/Lithotroph). These might require special conditions such as the presence of particular rocks or hot vents (maybe proximity to magma?)

I like the idea of ramping up farming complexity as population grows. The early Plump Helmet Logs would be the dwarven equivalent of slash-and-burn agriculture, which can support a small number of people for a few years before the soil becomes depleted and they have to move on.

[aka010101]

Okay, The wall of text, as overwhelming as it was to read, has some truly awesome ideas, but one of the things that really grabbed me was one of the parts of kohaku's reply.

I'm well aware that players aren't likely to even be playing a game long enough to have serious development of new varieties of selectively bred crops on their own.  What I'm arguing for is that the game procedurally develops new selectively bred "superior versions" of some crop or another, based upon the history of the world.  This means that humans might have special forms of more productive wheat for trade than the wild wheat you might get from herbalism, and these would be different every time.

While i really like the idea of selectively breeding crops, i think it should be a bit more complex than that. Instead of having a single 'better' type of crop, should be different traits you can give it, such as how hardy it is, how fast it grows, and so on. This could also tie into the raws of the plants themselves, having certian traits for growth rate, nutritional needs, ect.
My other thought on this is while procedural advancement is fine for the AI civilizations, when it's a player controlled civ, i think you should have a bit more control, such as ordering your farmers to breed, say.. maybe a faster growing strain of pig tails for example. Then perhaps randomize it, so you may end up with a strain that's just faster growing, or maybe faster growing but a bit more adapted to your exact ecosystem, making it harder to export. Or perhaps you get something else, as accidental discoveries happen occasionally. But, as the player, i feel you should have some say in the direction of reasearch , but perhaps that's an argument for another thread.

[FGK dwarf]

...
While i really like the idea of selectively breeding crops, i think it should be a bit more complex than that. Instead of having a single 'better' type of crop, should be different traits you can give it, such as how hardy it is, how fast it grows, and so on. This could also tie into the raws of the plants themselves, having certian traits for growth rate, nutritional needs, ect.
My other thought on this is while procedural advancement is fine for the AI civilizations, when it's a player controlled civ, i think you should have a bit more control, such as ordering your farmers to breed, say.. maybe a faster growing strain of pig tails for example. Then perhaps randomize it, so you may end up with a strain that's just faster growing, or maybe faster growing but a bit more adapted to your exact ecosystem, making it harder to export. Or perhaps you get something else, as accidental discoveries happen occasionally. But, as the player, i feel you should have some say in the direction of reasearch , but perhaps that's an argument for another thread.

Well, actually this might work quite well without player input, just by natural selection. Plants that grow quicker will get harvested more frequently; plants better suited to dry conditions will produce more seeds when you try to grow them in an arid biome. The player will get more seeds out of these than the standard varieties available at embark, so they will in time come to dominate the pool of seeds available for planting.

(This assumes the plant traits can be passed on genetically, which I believe is already possible for animals...?)

If you wanted more player input, you might actively set up a farm that's slightly under-watered, and segregate the seeds from that to get successively more drought hardy crops. Or you might set aside all seeds that were harvested early in the season (might be difficult to manage though) to pick out the fast growers.

I think I'd prefer something like that than a general order to your planters/herbalists/whatever the plant-breeder profession will be to "breed for better X". It would be more fiddly to manage, but would really be a small perk, so if you don't want to bother with it then it can be ignored. If you had a "breed for better X" order, it would be trivial to just tell the Dorfs to do it and reap the benefit for no real effort.

[Uristocrat]

I occurs to me that a lot of effort could be saved (and two long-standing requests could be commingled) by having dwarves take their potty breaks on site, in the fields.

Could also let us compost all those useless body parts that are hard to get rid of.

[jseah]

And trying to point someone to a disorganized 46-page thread riddled with what basically amounts to the same argument going back and forth didn't get too many takers willing to read that much, so it basically wound up being the same people who had actually been reading the thread all along versus everyone else. 

I read the whole thread.  Took me AGES.  Witnessed the enlightenment, almost. 

It's understandable why not many people wished to read the whole thing. 

Subscribed.  Can you bump this whenever you make a substantial update?