Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[Ten_Tacles]

Each person who is doing a "listen to story" or whatever is checking that the person they are listening to is their family member or vice versa, so it gets exacerbated massively by having highly populated, tightly-packed taverns (which I did in this profile). The search looks through every historical figure, in a binary search, i.e. it's O(logn), but if you take into account how many units do it it's sorta O(mlogn), which can indeed grow pretty fast.

Now, I have no idea about the inner workings of DF, but checking every historical figure seems to be a bit overkill?
A dwarf will never meet their grandfather in your fort if their grandfather is not in the fort.

[Kat]

it gets exacerbated massively by having highly populated, tightly-packed taverns

so basically every tavern, unless citizens are heavily restricted in their movements via burrows ?

[Martin]

older generations of intel core are also worse than newer ones, believe it or not, and the best CPU on the market for playing dwarf fortress is probably an 8-core AMD CPU, and, as of next month, a 16-core CPU with 32 threads... because it has 144MB of L2+L3 cache.

The most important thing for a CPU is single-core performance. The most important thing for single-core performance, at least for a game like DF, is cache size. Look at older Core cache sizes compared to newer ones.

This is why we need an ARM native Mac version. The M2 should hands down be the fastest machine to run on.

[Putnam]

So what would that mean for a multi generational fort like Archcrystal? Does it only do a check for family members that are still alive? In the relationships window it will only show up to grandparents I think, although you can select and see their grandparents from that menu.

Each person who is doing a "listen to story" or whatever is checking that the person they are listening to is their family member or vice versa, so it gets exacerbated massively by having highly populated, tightly-packed taverns (which I did in this profile). The search looks through every historical figure, in a binary search, i.e. it's O(logn), but if you take into account how many units do it it's sorta O(mlogn), which can indeed grow pretty fast.

Am I right in assuming then that the more historical figures there are (ie, the more history that has been generated) the bigger impact this has?

Yes, but logarithmically, i.e. if it takes 1 microsecond at one hist fig it'll take 10 microseconds at 1000 and 20 microseconds at 1,000,000

Each person who is doing a "listen to story" or whatever is checking that the person they are listening to is their family member or vice versa, so it gets exacerbated massively by having highly populated, tightly-packed taverns (which I did in this profile). The search looks through every historical figure, in a binary search, i.e. it's O(logn), but if you take into account how many units do it it's sorta O(mlogn), which can indeed grow pretty fast.

Now, I have no idea about the inner workings of DF, but checking every historical figure seems to be a bit overkill?
A dwarf will never meet their grandfather in your fort if their grandfather is not in the fort.

Might want to look up what a "binary search" is

[ab9rf]

A dwarf will never meet their grandfather in your fort if their grandfather is not in the fort.

How else are they supposed to know if their grandfather is in the fort?

[ayy1337]

A dwarf will never meet their grandfather in your fort if their grandfather is not in the fort.

How else are they supposed to know if their grandfather is in the fort?

You can do O(1) existence lookups, like say a hash table of creatures in the fort

[Miuramir]

older generations of intel core are also worse than newer ones, believe it or not, and the best CPU on the market for playing dwarf fortress is probably an 8-core AMD CPU, and, as of next month, a 16-core CPU with 32 threads... because it has 144MB of L2+L3 cache.

This might not be entirely true. The cpu cores in the upcoming 12 and 16 core 3D v-cache CPUs from AMD are split over 2 dies, and the increased chunk of cache is actually only present on one of those dies. Also critically for single threaded apps only cores on the die without the extra chunk of cache can boost up the max advertised frequency. Granted the faster cores can still access the big cache on the other die, but that's only after it's already checked it's own cache and at the penalty of increased latency. This higher cache latency might negate the clock frequency benefit to the point where the 8 core is just as fast as the 12 & 16 core in single threaded cache bound apps like DF. I'm also wondering if our OS's thread manager is going to be aware of these split-cache chips and will be smart enough to schedule threads onto the most appropriate cores. I guess we will have to wait until somebody has run benchmarks with these CPUs to see.

If I understand correctly the Milan-X (third+ gen AMD Epye with 3D cache) has up to 768MB of L3 cache.  I've been saying for some time now that what we really want is a CPU with enough on-die cache to fit an entire save file on the die; and this might be getting close for the first time.  (How much space is a 1x1 micro-fort these days?)  Something like the Epyc 7373X, which has the highest *steady state* clock rate of the 768 MB L3 chips, would only set you back around $4k. 

AMD is advertising it for "computational fluid dynamics (CFD), finite element analysis (FEA), electronic design automation (EDA), and structural analysis"; note that these scientific and engineering workloads are governed by many of the same sorts of many-to-many calculations that DF does. 

There's a newer 4th generation 9000 series "Genoa" which have 12-channel DDR5-4800 which should help with main RAM <> socket transfer, but I'm not currently seeing one with more than 384MB L3. 

Folks asking about specialized chips for DF... the good news is that if you're doing something like simulating the stress and heating on a spacecraft frame during reentry, you've got a very similar sort of workload pattern to DF, and those people have a lot more money than we do.  The bad news is that technology still isn't quite up to what DF really needs, and we're looking at chips in the $4k to $12 range just for the CPU. 

[YashaAstora]

A dwarf will never meet their grandfather in your fort if their grandfather is not in the fort.

How else are they supposed to know if their grandfather is in the fort?

Shouldn't it be possible for the game to compile a list of who's in the fort, and then have dwarves only check that list? Hell, couldn't it also compile a separate list of everyone in a tavern when a performance starts, and then have the dwarves in the tavern only check that second? list

[Miuramir]

#4 is the game's software rescale function. 4.3%, 7.440s

Would implementing a "shader cache" sort of arrangement help here?  Is this something you expect the SDL2 update to improve anyway? 

[Untrustedlife]

So what would that mean for a multi generational fort like Archcrystal? Does it only do a check for family members that are still alive? In the relationships window it will only show up to grandparents I think, although you can select and see their grandparents from that menu.

Each person who is doing a "listen to story" or whatever is checking that the person they are listening to is their family member or vice versa, so it gets exacerbated massively by having highly populated, tightly-packed taverns (which I did in this profile). The search looks through every historical figure, in a binary search, i.e. it's O(logn), but if you take into account how many units do it it's sorta O(mlogn), which can indeed grow pretty fast.

Last time I had FPS issues however they were fixed by walling off the cavern. Unless something major changed with DF 50 vs 47 walling off the cavern made the game go from 4fps to 45... That is pretty significant change and that cant really be explained with what your encountering. Maybe there is some kind of edge cases where path finding does indeed consume more CPU then what your seeing here. I would probably gather some save files from community members that have FPS death and check this again.

There are multiple known edge cases. There was probably something stuck in some trees in the caverns that wanted to get into your fort or out onto the surface, and walling it off made the caverns completely cut off, thus making it stop trying to do that. Being stuck in trees makes things pathfind every tick or so, which is indeed extremely slow, but most of the time stuff just isn't pathfinding that much.

I did another 3-minute profile on my 260-citizen fort, this time with the knowledge of the source. Percents are going to overlap some. Specific seconds are not.

The slowest thing was line-of-sight checks. Due to the fact that it was only counting the checks in the function itself, this was a total of 14.7% of CPU time. However, if you remove one specific thing from line-of-sight checks (more on that later), it's 8%, and thus faster than the next thing. 13.181s in the function itself.

Great to see numbers here, thanks!

I know 14.7% doesn't point to any massive optimization potential, but -- is the line-of-sight implementation more amenable to a concurrent/thread-pool approach than the pathfinding?  I would not expect LOS to use global mutable state, but I wouldn't have expected that for pathfinding either, so what do I know.

Technically yeah, could make a bunch of futures that fill out the line-of-sight info for each individual unit before they're used for the individual units, but the biggest intervention right now is probably just optimizing the "get glowtile" function lol

Hey there! Software engineer here. Have you considered just caching a pointer to or ID of or whatever data you access often of the dwarves family members inside whatever object represents the dwarf (assuming each hist fig has an ID or teh data you are accessing could be saved there) , so that you dont have to search through the hist figs every time? Do you think that would have a noticeable performance improvement?

Eg a dwarf might have a list of family member names etc, or their ids directly on them. That is updated the first time you check or whatever.

You dont need to include it in the save data, as you would simply cache it the first time you check after the game is loaded. And not check afterwards.

Also, the entire LOS system almost seems pointless. For LOS in adventure mode you could just do something seperate.

[A_Curious_Cat]

#4 is the game's software rescale function. 4.3%, 7.440s

Would implementing a "shader cache" sort of arrangement help here?  Is this something you expect the SDL2 update to improve anyway? 

An SDL2 update would be very appreciated.  The Linux distribution I’m currently using doesn’t support the old SDL…

[Putnam]

SDL2 update is mostly done, I didn't do terribly much in the way of optimization besides the inherent optimization you get from moving to the GPU; however, you can just turn off the rescale function, so at least that's going on.

There's already a texture cache for rescaled stuff, it's just invalidated rather often. A bit less often with SDL2, though.

[jecowa]

Will a fort on a 5-year-old world run faster than a fort on 500-year-old world because there are less historical figures to check for familiar relations?

[Putnam]

Yes, but negligibly, the main issue here is activities (storytelling, poetry etc.) with dozens or even over a hundred units involved

[jecowa]

#4 is the game's software rescale function. 4.3%, 7.440s

Does playing the game at 100% scale help a lot with lowering this number? Then only 8x12 interface/text tiles would still need scaling.