Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[smjjames]

It should be all under one bug report though rather than multiple individual reports.

[smjjames]

Hm, might or might not be connected to the fort mode lag problems, but I hit an issue where I was having heavy lag despite walking in the middle of nowhere.

http://www.bay12games.com/dwarves/mantisbt/view.php?id=8393

[StagnantSoul]

Just did some personal !!science!! to check out a few theories, thousands of logs have no major effect. I added in 50000 logs, and lost 8 fps. 50000. Most forts get into 2-3000.

[SimRobert2001]

Where did you add them? Items that share a space take up less computing power than items spread through the fort. It is definite evidence that items DO take up computing power, however.

[StagnantSoul]

Oh. I added them in one spot far away from my fort.

[SimRobert2001]

People have reported that quantum dumping all of their items into one square helps their FPS. I'd be curious to see what happens when those 50k worth of trees are spread out over an entire map.

[GavJ]

Where did you add them? Items that share a space take up less computing power than items spread through the fort. It is definite evidence that items DO take up computing power, however.

it's probably RAM more so. I'm guessing what happens is that Toady uses the path algorithm appropriately to not double check multiple items of the same sort in the same spot, so it gets faster for that reason when you make piles. BUT he still keeps them as individual items in memory (logs don't dynamically become log stacks etc.), so having 50,000 things may start to reassign vectors more often and stuff like that that uses up extra FPS for memory shuffling and bookkeeping and such.  This would generally be much more extreme with items that have descriptions and quality like crafts, than logs though.

In a sense, there's no solution for that the way things are now. But in reality, do we really need that 40,175th auto-generated description of a bracelet, and does it really need to be tracked consistently?  Maybe after 100 of something exists in your fort with descriptions, it just starts stacking them generically without (still by quality though). Or something.

[Dirst]

Where did you add them? Items that share a space take up less computing power than items spread through the fort. It is definite evidence that items DO take up computing power, however.

it's probably RAM more so. I'm guessing what happens is that Toady uses the path algorithm appropriately to not double check multiple items of the same sort in the same spot, so it gets faster for that reason when you make piles. BUT he still keeps them as individual items in memory (logs don't dynamically become log stacks etc.), so having 50,000 things may start to reassign vectors more often and stuff like that that uses up extra FPS for memory shuffling and bookkeeping and such.  This would generally be much more extreme with items that have descriptions and quality like crafts, than logs though.

In a sense, there's no solution for that the way things are now. But in reality, do we really need that 40,175th auto-generated description of a bracelet, and does it really need to be tracked consistently?  Maybe after 100 of something exists in your fort with descriptions, it just starts stacking them generically without (still by quality though). Or something.

I thought artworks weren't actually defined until you looked at them, which allowed for hilarity like engravings of artifacts that didn't yet exist when the engraving was made.

Though in general what you are describing is the "pocket lint problem" in that DF tries to track the world in too much detail for sustained use.  This is completely different than the living world issue, because the outside world is still at highly abstract worldgen levels of detail anyway.

My gut feeling is that the FPS issues are from taking rickety code apart, fiddling with some of the innards, stuffing it all back in, and releasing it before any optimization was done.  I, for one, appreciate the early access.

[GavJ]

Lots of items has always created lag, and lots of items in quantum piles has always created somewhat less lag than just lots of items, but not removing all of it.  That part being discussed, however large or small a contribution it may be to current lag, is not a new version issue.

[Putnam]

Where did you add them? Items that share a space take up less computing power than items spread through the fort. It is definite evidence that items DO take up computing power, however.

it's probably RAM more so. I'm guessing what happens is that Toady uses the path algorithm appropriately to not double check multiple items of the same sort in the same spot, so it gets faster for that reason when you make piles. BUT he still keeps them as individual items in memory (logs don't dynamically become log stacks etc.), so having 50,000 things may start to reassign vectors more often and stuff like that that uses up extra FPS for memory shuffling and bookkeeping and such.  This would generally be much more extreme with items that have descriptions and quality like crafts, than logs though.

In a sense, there's no solution for that the way things are now. But in reality, do we really need that 40,175th auto-generated description of a bracelet, and does it really need to be tracked consistently?  Maybe after 100 of something exists in your fort with descriptions, it just starts stacking them generically without (still by quality though). Or something.

Looks like descriptions and quality aren't really taken into account.

[GavJ]

Looks like descriptions and quality aren't really taken into account.

What does quoting an enum have to do with anything? All I'm saying is that items with qualities and decriptions have to have that information stored somewhere. Which means you need to have things like big arrays of individual objects with those properties, instead of things like "a stack of 4,172 logs" as a single C++ object. This undeniably takes up way more RAM, which can indirectly affect FPS. And more directly, many objects requires resizing vectors at times when stack objects wouldn't, causing more reassignment in general which will affect CPU.

Somebody said above that descriptions aren't generates until you look at something. If true, then that helps a lot. But it only partially helps. Quality is tracked all along, and basic "description" in terms of sub-type (like emu leather versus mule leather etc.) is tracked all along, causing similar issues. I.e., you might have a stack of emu leather that gets stored efficiently, but usually you'd still also have like 15 other kinds of leather, thus making it only very partially efficient in small stacks. At some point, small enough stacks might not even help, because stack management might cause more lag then they give you.

[Footkerchief]

What does quoting an enum have to do with anything? All I'm saying is that items with qualities and decriptions have to have that information stored somewhere. Which means you need to have things like big arrays of individual objects with those properties, instead of things like "a stack of 4,172 logs" as a single C++ object. This undeniably takes up way more RAM, which can indirectly affect FPS. And more directly, many objects requires resizing vectors at times when stack objects wouldn't, causing more reassignment in general which will affect CPU.

Somebody said above that descriptions aren't generates until you look at something. If true, then that helps a lot. But it only partially helps. Quality is tracked all along, and basic "description" in terms of sub-type (like emu leather versus mule leather etc.) is tracked all along, causing similar issues. I.e., you might have a stack of emu leather that gets stored efficiently, but usually you'd still also have like 15 other kinds of leather, thus making it only very partially efficient in small stacks. At some point, small enough stacks might not even help, because stack management might cause more lag then they give you.

This kind of bitpicking is the essence of premature optimization.  All that to "indirectly" improve FPS?  As always: don't trust your intuition, trust the profiler.

[Uronym]

Remember the fundamental rules of program optimization:

1. Don't.
2. Don't... yet. (for experts only)
3. Since you won't listen, at least use a profiler.

[GavJ]

This kind of bitpicking is the essence of premature optimization.  All that to "indirectly" improve FPS?  As always: don't trust your intuition, trust the profiler.

1) I didn't say anything about bitpicking. I said making stacks of like objects, and also making more objects alike by getting rid of 573 types of leather and so on (or wood types) when most people don't care (or implement a system for indicating special "artifacts" where you do personally care for nostalgia, and those alone).  That could be a 100-fold reduction of items in game, easily. That's pretty macro level stuff. Nary a bitwise operation was mentioned or implied.

2) Premature optimization is optimizing before you know the bottlenecks and/or before you even have working code. Toady has working code already. And people have complained and profiled sources of specific bottlenecks already. Item count is one of them. Masterwork has in fact already even gone in and reduced the redundant types of leather and has already demonstrated that this speeds up code. It's not even a theoretical fix... (my suggestions do go beyond that, and are partially theoretical, but I don't have the code to profile it myself, it is assumed Toady would do that yes).

3) I don't think you can apply normal concepts of "premature optimization" to dwarf fortress, anyway, even if it might normally be considered slightly premature before a whole program is finished. Because there are already thousands of people playing it every day. Unlike a normal program that exists only on the programmer's computer when halfway done (and maybe a few special testers), this is already in the public, and there is a reasonable expectation of doing macro / major types of optimization now. Not at the end. Because by the time it's finished, it's entirely possible that 70% or something of all the games ever played will already have been played! It would be dumb/kinda obnoxious to optimize only at the end and abandon the people playing it for the 25 years beforehand... Also impacts donations, of course.

[guessingo]

Maybe we will get lucky and cpu clock speeds will increase again...