Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[malvado]

What's the best way to clean up a fortress once you hit the fps death?
I used to make some nice forts back a few versions that didnt have a lot of trouble playing them again, this time though I see pretty huge fps loss when visiting some of my 10 - 20 year old fortresses as an adventurer and would like some advise to what one should do to have some nice fps when visiting these forts (or playing them again in Fortress mode).

*Is it really necesarry to Atom smash every single rock?
*Should population of fortress be kept down as well?

All advise is really appreciated.

[Geltor]

the one and only major culprit of lag is the population, both in fortress and adv mode. while theres not much you can do if you want to keep your population intact, the only thing i can think of is slaughtering every non-resident creature and retiring. usually when you retire a bunch of local creatures become "outcasts" and wander about your fort, this includes wagons and traders as well.

[vjmdhzgr]

Traffic designations can supposedly help a lot. I don't know too much about how to make them, but there's a lot of guides out there.

[Sanctume]

What about blocking off large mined areas / quarries that will no longer be needed to be explored, i.e. adding to the pathing logic?

[Robsoie]

For adventure mode, a site population is a big source of horrible laggy gameplay, they're going to be constantly talking with each other (and only Toady knows what other calcutations , line of sight, stress, etc... they're spamming every turns that eat the performance) ,

Now i wonder if pathfinding is the same problem in adventure mode than it is in fortress, usually when you go on sites in adventure mode, the huge majority of the NPC aren't moving around, only a very few do that.
So unless Toady forgot to turn off the pathfinding calculation spam for the immobile NPC, it should be less important.

Too bad when you're playing fortress mode you can't yet send whole groups of dwarves to go strike the earth somewhere else to create new AI fortresses away, this would help in keeping your population in control if you didn't setup lower pop cap (the default pop cap that is at 200 and max at 200 is way too high if you hope to avoid fps death in your fortress and in adventure mode) without having to get them dead instead.

I don't know if cutting every single tree on your map can be helpful before retiring, or if similarly to fortress mode, they constantly regrow everywhere in adventure mode.

[pikachu17]

having a fast adventurer will also help, but mostly it's just population of the place, which is the same reason dark fortresses are incredibly laggy.

[Hesperid]

the one and only major culprit of lag is the population, both in fortress and adv mode. while theres not much you can do if you want to keep your population intact, the only thing i can think of is slaughtering every non-resident creature and retiring. usually when you retire a bunch of local creatures become "outcasts" and wander about your fort, this includes wagons and traders as well.

Completely disagreed. Ruling out flows, cave-in generators, mechanical megaprojects and other such (typically intentional) fps drainers, the two major sources of calculation drain are population and items. Of the two, item stacks are a considerably larger FPS killer because of how numerous they are, and they are numerous because people keep getting told this fairytale that it's all about the pathfinding.

The pathfinding is much less of a toll than people advertise, especially in modern versions because of path caching and other shortcuts.

But items... most fortresses accumulate gigantic amounts of stone, logs, people love keeping a surplus of raw materials like bars and blocks, and a surplus of clothes so nobody runs around naked, and massive stockpiles of meat (because animals overpopulate faster than their meat gets eaten), and prepared food stacks and drinks. If you're using dfhack and set it to produce more food when you have fewer than a certain "amount", then that gradually blows up the stack count because eating will slowly result in a massive number of food stacks with just 1 food in each.

I took it upon myself to destroy all unneeded items in my fortress. Extra meat? Destroyed it. All leathers and piles of cloth that I kept around? Gone. Gems and bars, it hurt but they had to go. All those logs, stones, everything but what I immediately needed had to go. Next I programmed a utility for dfhack to destroy all the remains creatures were leaving in the caverns, and all the unused webs, invader battle arrows that were lying around and being melted too slowly. I went from about 20k item stacks down first to only 9k, and with 80 population in my decades and decades old fortress, I shot back up to the 100 FPS I'd recorded back when I first got that population. I went crazy with it and slimmed down to 6,3 k and I've got scripts running to keep me there.

And it is god damn smooth sailing, brother. The years go by and my FPS is stable. My debugger is recording NO increase in computation time spent, except when caravans pass by or invasions arrive, but I've verified that after those events I go back to my baseline.

And I don't even have traffic areas or pathfinding design or any of that "old school" trickery. My guys wander the open areas to different miniforts that I've constructed for the separate species, and it is not killing FPS at all.

[Robsoie]

Thanks for mentionning the items, despite i use quantum stockpile i didn't thought it would be that huge before doing the experience myself :

i haven't got my FPS display, i only judge the game by how fast it's playing.
And on my 6 years old current fort, i have 100 dwarves (max cap i setup) on my 2x2 embark, and it's fortunately very playable i don't have any slowdown yet (due to the 2x2 embark and 100 max popi guess) but it's noticably and obviously not running as fast as it used to be in the 1st year.

So i decided to do the test, i pressed d - b - d and set my quantum stockpile for dump , then with dfhack i typed
autodump destroy
5000 items were destroyed (wow so much and i didn't even built much craft).

After i unpaused, the difference in speed is -very- noticable, that's quite impressive for me to see 100 dwarves running around that fast, as fast as what i could see on the 1st year when there was not much things on the map.

Good i had made a copy of that save as for the experience i destroyed all the weapons and armor i had forged on this

This is eye opener actually, i always thought the pathfinding was the biggest culprit in performance (as shown by a dwarf getting stuck on a tree or a wall and getting a mood), but from that experience, it's looking like the items accumulation through the fortress years are the big player there.

Considering how very fast items are accumulating in a fort life (by example i got 2000 mussel shells on that small map in less than a year !) even when you're not having much industry running, this item huge impact on performance is definitively something Toady should take a serious look to considering everyone will have to face the fps death problem if they play a "normal" (as in not making sacrifice of gameplay part to run without much performance hit) fortress more than only a few years.

edit : turned on the FPS display to check
before the eradication of all the stockpiled item in my quantum stockpile fps are going between 85 and 90 (as i said due to small 2x2 and only 100 pop, it's very playable) , after autodump destroy them all fps = 100 (probably because i'm at the cap there, as i didn't edited)
Though i use GPS=15 so it may (or not) have an impact, didn't tested much around that.
But when it comes to the dwarves movement , there is a definitive difference in speed.

I'll have to check my older saves, i had a much longer lived fort (more than 25 years), with a larger embark and more than 150 dwarves that was running slow, the experience would be more interesting with destroying all items.

[gchristopher]

I took it upon myself to destroy all unneeded items in my fortress. Extra meat? Destroyed it. All leathers and piles of cloth that I kept around? Gone. Gems and bars, it hurt but they had to go. All those logs, stones, everything but what I immediately needed had to go. Next I programmed a utility for dfhack to destroy all the remains creatures were leaving in the caverns, and all the unused webs, invader battle arrows that were lying around and being melted too slowly. I went from about 20k item stacks down first to only 9k, and with 80 population in my decades and decades old fortress, I shot back up to the 100 FPS I'd recorded back when I first got that population. I went crazy with it and slimmed down to 6,3 k and I've got scripts running to keep me there.

Most of those can be resolved with simple atom smashing.

Could you please share code/tips/techniques for cleaning out cavern detritus, especially cave spider webs? My current fort attempt has several thousand webs. Spent arrows and other litter are also a concern.

Another concerns I've seen voiced is litter that's fallen from trees.

[Robsoie]

To clean the cave spider webs, press d - b -d and select the whole zone to make every item marked for dump, then with dfhack type
autodump destroy

To clean the broken arrow, in dfhack type (that may also clean the spider webs without having to do the dump -> autodump destroy)
clean map


I tested the complete item removal on my previous save that had a 25+ years long lived fort +150 dwarves+ larger embark in which i had the fps down to 29, then after destroying the 12000 items i had in my quantum stockpile + every useless stuff in every cavern  , the fps went up to ... 35, surprising me that it didn't do much than that, but that's still noticable.
Though much less than i expected, so the problem is still having other factors too.

[Hesperid]

Most of those can be resolved with simple atom smashing.

Could you please share code/tips/techniques for cleaning out cavern detritus, especially cave spider webs? My current fort attempt has several thousand webs. Spent arrows and other litter are also a concern.

Another concerns I've seen voiced is litter that's fallen from trees.

You got it, pal. It's in the spoiler of this post.

You can do robsole's autodump and destroy-here, that's how I did my first experiments as well.

But spider-webs grow in caverns at a huge rate, in few days you'll have hundreds again. And trees grow and need to be cut down all the time, resulting in logs. Here is a script that destroys cave spider webs, spent arrows, subterranean carcasses and logs lying around. Each of those can be toggled off separately with the booleans at the top, and you can define what Z-level you want to consider "subterranean". For me, this script gets run every 14 days.

repeat -name mrclean -time 14 -timeUnits days -command [ mrclean ]


mrclean.lua

Spoiler (click to show/hide)

Code: [Select]

-- Mr. Clean - a script by bay12 forums user Hesperid

-- Settings

-- Destroy subterranean cave spider webs? (does not destroy GCS or FB webs because they have utility)
local clean_cave_spider_silk = true
-- - || -  corpses and body parts lying around underground
local clean_remains = true
-- Scattered ammunition anywhere on the map
local clean_ammo = true
-- Scattered logs anywhere on the map (keep this false if you want to have logs stockpiles)
local clean_logs = true

-- This setting defines what is considered subterranean for clearing out webs and corpses
local remains_beneath_z_level = 35

-- If 'true' then the script does not print lines of text for destroyed items (you probably want this)
local quiet = true




-- Program follows

local ic = 0
for kk,item in pairs(df.global.world.items.all) do
local liq_Type = tostring(item._type)
if (liq_Type == '<type: item_threadst>') then
if (tonumber(item.pos.z) < remains_beneath_z_level) then

if (clean_cave_spider_silk and string.find(tostring(dfhack.matinfo.decode(item.mat_type, item.mat_index)), 'CREATURE:SPIDER_CAVE:SILK') ~= nil) then
if (item.flags.on_ground) then
item.flags.hidden = true
item.flags.garbage_collect = true
if not quiet then print("Destroyed subter web") end
end
end


end


elseif (liq_Type == '<type: item_remainsst>') then
if (tonumber(item.pos.z) < remains_beneath_z_level) then
if (clean_remains and item.flags.on_ground) then
item.flags.hidden = true
item.flags.garbage_collect = true
if not quiet then print("Destroyed subter remains") end
end
end


elseif (liq_Type == '<type: item_corpsepiecest>') then
if (tonumber(item.pos.z) < remains_beneath_z_level) then
if (clean_remains and item.flags.on_ground) then
item.flags.hidden = true
item.flags.garbage_collect = true
if not quiet then print("Destroyed subter remains") end
end
end


elseif (liq_Type == '<type: item_corpsest>') then
if (tonumber(item.pos.z) < remains_beneath_z_level) then
if (clean_remains and item.flags.on_ground) then
item.flags.hidden = true
item.flags.garbage_collect = true
if not quiet then print("Destroyed subter remains") end
end
end


elseif (liq_Type == '<type: item_ammost>') then
if (clean_ammo and item.flags.on_ground) then
if (tonumber(item.stack_size) == 1) then
item.flags.hidden = true
item.flags.garbage_collect = true
if not quiet then print("Destroyed scattered ammo") end
end
end


elseif (liq_Type == '<type: item_woodst>') then
if (clean_logs and item.flags.on_ground and not item.flags.construction) then
if (tonumber(item.stack_size) == 1) then
item.flags.hidden = true
item.flags.garbage_collect = true
if not quiet then print("Destroyed scattered logs") end
end
end
end

ic = ic + 1
end

print ("Total item indexes " .. ic )


[Robsoie]

thanks for the script, will be useful to me.

[Hesperid]

No problem!

If you're building an above-ground fortress, don't be tempted to tweak this script to destroy the building materials in your constructions. It does not appear to increase FPS at all, even if you have a ton of constructions. But what it will do is destroy the value of your constructed rooms.

[gchristopher]

You got it, pal. It's in the spoiler of this post.

You can do robsole's autodump and destroy-here, that's how I did my first experiments as well.

But spider-webs grow in caverns at a huge rate, in few days you'll have hundreds again. And trees grow and need to be cut down all the time, resulting in logs. Here is a script that destroys cave spider webs, spent arrows, subterranean carcasses and logs lying around. Each of those can be toggled off separately with the booleans at the top, and you can define what Z-level you want to consider "subterranean". For me, this script gets run every 14 days.

repeat -name mrclean -time 14 -timeUnits days -command [ mrclean ]


mrclean.lua

Spoiler (click to show/hide)

Code: [Select]

-- Mr. Clean - a script by bay12 forums user Hesperid

-- Settings

-- Destroy subterranean cave spider webs? (does not destroy GCS or FB webs because they have utility)
local clean_cave_spider_silk = true
-- - || -  corpses and body parts lying around underground
local clean_remains = true
-- Scattered ammunition anywhere on the map
local clean_ammo = true
-- Scattered logs anywhere on the map (keep this false if you want to have logs stockpiles)
local clean_logs = true

-- This setting defines what is considered subterranean for clearing out webs and corpses
local remains_beneath_z_level = 35

-- If 'true' then the script does not print lines of text for destroyed items (you probably want this)
local quiet = true




-- Program follows

local ic = 0
for kk,item in pairs(df.global.world.items.all) do
local liq_Type = tostring(item._type)
if (liq_Type == '<type: item_threadst>') then
if (tonumber(item.pos.z) < remains_beneath_z_level) then

if (clean_cave_spider_silk and string.find(tostring(dfhack.matinfo.decode(item.mat_type, item.mat_index)), 'CREATURE:SPIDER_CAVE:SILK') ~= nil) then
if (item.flags.on_ground) then
item.flags.hidden = true
item.flags.garbage_collect = true
if not quiet then print("Destroyed subter web") end
end
end


end


elseif (liq_Type == '<type: item_remainsst>') then
if (tonumber(item.pos.z) < remains_beneath_z_level) then
if (clean_remains and item.flags.on_ground) then
item.flags.hidden = true
item.flags.garbage_collect = true
if not quiet then print("Destroyed subter remains") end
end
end


elseif (liq_Type == '<type: item_corpsepiecest>') then
if (tonumber(item.pos.z) < remains_beneath_z_level) then
if (clean_remains and item.flags.on_ground) then
item.flags.hidden = true
item.flags.garbage_collect = true
if not quiet then print("Destroyed subter remains") end
end
end


elseif (liq_Type == '<type: item_corpsest>') then
if (tonumber(item.pos.z) < remains_beneath_z_level) then
if (clean_remains and item.flags.on_ground) then
item.flags.hidden = true
item.flags.garbage_collect = true
if not quiet then print("Destroyed subter remains") end
end
end


elseif (liq_Type == '<type: item_ammost>') then
if (clean_ammo and item.flags.on_ground) then
if (tonumber(item.stack_size) == 1) then
item.flags.hidden = true
item.flags.garbage_collect = true
if not quiet then print("Destroyed scattered ammo") end
end
end


elseif (liq_Type == '<type: item_woodst>') then
if (clean_logs and item.flags.on_ground and not item.flags.construction) then
if (tonumber(item.stack_size) == 1) then
item.flags.hidden = true
item.flags.garbage_collect = true
if not quiet then print("Destroyed scattered logs") end
end
end
end

ic = ic + 1
end

print ("Total item indexes " .. ic )


That's really interesting. So just flagging an item as hidden and for garbage_collect does it. Thanks!

I was figuring to try making cave spiders extinct by setting their populations to 0.

It crashed for me on 42.0.4, because some items in df.global.world.items.all do not have a mat_type. Adding a check for whether the table contains the key before referencing it fixed it.

[Hesperid]

It only reads mat_type for items that are thread. Just out of curiosity, what was the item that you had that was thread but didn't have a material?

edit:

I was figuring to try making cave spiders extinct by setting their populations to 0.

Not bad, I'd also prefer to solve the problem from that end. The game creating webs all the time and then having them garbage collected immediately does waste a lot of processor time that could be better spent. I was tempted to binary hack the routine that spawns cave spider webs, but then, that's not a very portable solution.