Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[Ungweliante]

I've become convinced that plantgrowth is - by far - the greatest source of lag.

Some observations:

1.0) I have a fortress with ~200 dwarves. The fortress plays at around ~20 FPS. I have several ( 8 ) very big rooms, which I plan to use as tree farms. The uppermost of these rooms has a soil floor, and plants are growing there.
1.1) Having the rooms increases lag a lot. Yet, flooding the rooms (with constantly flowing water) decreases lag, by 10 - 15 FPS.
1.2) Unflooding all the rooms - so that they now have mud - increases lag so that it falls to ~3 FPS, from ~30.

1.3) If I abandon the fortress, and re-embark with ~7 dwarves, lag is unaffected. Still ~3 FPS.

2) Further, I'm able to do a 16x16 square embark with fairly good FPS, if I use DFHACK to paint the surface area as stone floor.

In my logic, this eliminates flowing water, large rooms without soil / mud, and creature pathfinding as major sources of lag.
A possible explanation why people report the amount of dwarves - and the related creature pathfinding - as a major lag source, is that when the amount of dwarves increases, new bedrooms are dug for them. And often these are situated in soil layers, thus increasing plant growth.

Opinions?

[Graknorke]

I don't know, I don't think that available growing surface area increases during the life of a normal fort much since most of it will be in the stone, but FPS will still decrease massively.

[Rendell]

I think the constant tracking/Checking of the masses amount of rotting clothes, corpses and food not stored in barrels/bins can cause a serious amount of lag, as the game has to monitor the current state of the those items to determine when they move to the next level of decay.   
 

[Larix]

How big exactly are those "very big" rooms? Plant growth is a continuous process that's _probably_ checked very frequently - maybe every step, maybe less frequently, but surely several times a day by observation. It makes sense that this takes up processor time and thus potentially FPS, but unless the greenery areas aren't staggeringly huge, the total plant life FPS hit seems to be small to negligible. I've never opened up more than about 100x100 underground pastures (a bit more than a full screen, perhaps half as much as there's surface area on a 4x4) in a single fort, so probably never got near the threshold where plant growth matters. And i've had my fair share of FPS woes.

Observation also shows clearly that creatures _do_ constitute a significant FPS hit - FPS will observably decrease when invaders are on the map (sometimes to the point that you can tell there are ambushes around just because FPS are consistently down five points), and when caravans are visiting. I've also seen significant FPS decreases for flowing water, but results there are quite variable, and for mechanically opening/closing buildings.

So, no, the other FPS killers aren't elminated at all - in many forts, total growth-check area doesn't significantly increase over the fort's lifetime; your observations just _add_ another potential issue that has been overlooked so far. A drop from 30 to three makes me suspect there's something else going on there, though; possibly going over some kind of threshold where the computer has to page out main memory, reducing all memory-requiring activity to a fraction of normal speed.

[Ungweliante]

How big exactly are those "very big" rooms?

I have de-divided the rooms, so there are 2 rooms of 93x44 (=4092) squares, 1 irregular room of approximately 93x36 / 2 (=1674) squares due to aquifer wall blockage, and 1 very big room of 93x44 (=4092) + 65x124 (=8060) + 24x67 (=1608) + 25x63 (=1575) = 15335 squares.

The lag drop to ~3 FPS happens when I open up the rooms for traffic. But only if they have plantgrowth. When the rooms were dug, there was lag during stone-hauling, but nothing like ~3 FPS.

Observation also shows clearly that creatures _do_ constitute a significant FPS hit

But if creatures are a significant FPS hit, how is it possible that the game stays at the same ~3 FPS, even though the amount of dwarves goes from ~210 to 7 due to abandoning and reclaiming?

[doublestrafe]

Are the rooms dry, or are they still full of 1/7 water tiles?

[Di]

Not that quest for eliminating all plantlife for a slight chance of increasing fps wouldn't be a respectable endeavour but that hardly could be the long longed-for cure.

For example I couldn't agree with this one:

when the amount of dwarves increases, new bedrooms are dug for them. And often these are situated in soil layers, thus increasing plant growth.

One hardly deserves a noble title of overseer if he allows himself to call a hole rid with moss and fungus a bedroom for a dwarf.
While the sentence above may be argued I'd like to note that, as a consequence of following that maxima, I rarely dig more than a handful of dirt (Approximately 400-500 tiles) and still get a steady decrease of fps.

Actually, the exact cause might be quite different for different players. I for example wouldn't call pathfinding the worst offender, unless your dwarves have to pass through a complicated maze to get food and then go back to the dinning room, the pathfinding impact should be pretty small. Even large rooms are bad only when dwarves are rushing there en masse.
Fluid movement cost is laughable unless you're flooding the caverns.
Turning off the heat calculations never got me more than a couple of frames back.

Recently, I've run a fort with 30 citizens increasing their numbers only via babies, and there wasn't lot of digging there either as fort was aboveground. Yet over the course of years game speed decreased to its stop. So if anything is worthy of title 'the most fundamental cause of slowdown', it'd be the spirits of the dead and socks, both of which are unlikely to be discarded during the abandon\reclaim.

[slothen]

Its not plant growth.

[Quietust]

Fluid movement cost is laughable unless you're flooding the caverns.

On the contrary, if you have fluids frequently changing depth between 4 and 5 in areas where dwarves can walk, it can cause significant lag as it constantly recalculates the path tags for most of the map (that is, the information that allows the game to determine whether a dwarf in tile A is capable of walking to tile B). For the sasme reason, contraptions which frequently open/close doors/hatches/grates/bars/bridges/floodgates can also cause significant lag while they're running.

[itg]

I've also found that pressurized fluids, in particular, can cause massive fps loss. I imagine this is because every frame the fluid has to do some potentially lengthy pathfinding to find an acceptable output square.

[Miuramir]

The lag drop to ~3 FPS happens when I open up the rooms for traffic. But only if they have plantgrowth. When the rooms were dug, there was lag during stone-hauling, but nothing like ~3 FPS.

The above is potentially interesting.  We know that DF has a respectable amount of optimization built into the pathing; dwarves don't just do A* every time.  IIRC there was a comment from Toady that at least one level involved some sort of abstracted room / region path caching. 

Ideas:

* It's possible that areas that might have plant growth are effectively treated as "uncached", because a tree might pop up at any time. 

* It's possible that your giant rooms have exceeded the size of some internal room size variable, and can't be cached because they are too big. 

* It's possible that *trying* to cache the giant rooms is causing DF to exceed some system cache size, and is effectively forcing it to do slower reads from main memory, reducing overall performance. 

* It's possible, although rather less likely, that interaction with the "dwarves track mud" and "dwarves trample plants" checks is slowing things down drastically, perhaps in conjunction with the above. 

Can you experiment with putting a High Traffic route, possibly with a road or otherwise cleaned off, on as optimal as practical path between your various doors, and marking the rest of the giant rooms Restricted?  If you've got such huge rooms, you might need to up the "cost" for Restricted in your config file, as the default may not be enough to force desired behavior on such large scales. 

[Krawdad]

I haven't noticed anything special about soil layers causing FPS drop, but planting crops definitely causes a significant drop for me. I have one fairly large plump helmet farm that only gets planted in autumn and another plot for millable plants for 3 seasons. As soon as autumn rolls around I see the FPS drop from ~100 to ~60, but its temporary enough that it doesn't both me too much. I also have a lot of exposed magma, so turning temperature on drops the FPS from ~100 to ~40.

Also, it wouldn't help you for this savegame but I would recommend trying either the Accelerated or Accelerated Modest DF mods. When I played vanilla DF my longest fort was maybe 15 years old with ~250 inhabitants before I gave up because it was stuck around 20 FPS. Now with Accelerated Modest my current fort is 34 years old with ~250 dwarves, ~400 animals (things got out of hand), the magma sea exposed, caverns exposed and open, a flowing river on the surface, many large stockpiles, and even an above-ground arena and I still get around 100 FPS normally.

Every few years I'll run a 'clean map all' command, 'cleanowned scattered X' to dump all the tattered clothes strewn everywhere and mass dump all the goblinite laying around then 'autodump destroy' all of it, but it's still fairly minimal oversight.

[Rum]

It sounds like the water flooding those rooms changes the pathfinding which iirc is the largest source of lag.  The dwarves have to path over every single one of those tiles in your plant growth rooms, except when you have more than 2/7 of water in them which makes them unpathable except to swimmers. 

[GavJ]

There's a pretty obvious confound here in that when you flood a room, it is no longer available for pathing considerations, either...

To test this properly, you need to have the rooms entirely cut off from any possible pathing, at all time points:
1) Measure FPS.
2) Dig the rooms out.
3) Measure FPS with open to pathing.
4) Seal them fully with walls and closed floodgates too from the water source, everything.
5) Measure FPS.
6) Flood the rooms. The reseal completely all entrances of any sort!
7) Measure FPS while still flooded.
Wait for the water to all evaporate.
9) Measure FPS.
10) Open to pathing to test trampling hypothesis.
11) Measure FPS.
11) Wait until a lot of stuff is grown already fully.
12) Measure FPS.
13) Open it now to pathing again to act as control against test of trampling hypothesis.
14) Measure FPS.

With all those numbers, we should eb able to evaluate most/all of the theories suggested so far.

[TruePikachu]

DaVinci, happen to have access to any call profiling tools? Might really help the research as well, when combined with your disassembly.