Bay 12 Games Forum

Please login or register.

Login with username, password and session length
Advanced search  

Author Topic: [40d] Massive lag when calculating in stocks screen.  (Read 986 times)

Shurikane

  • Bay Watcher
    • View Profile
    • http://www.shurikane.com
[40d] Massive lag when calculating in stocks screen.
« on: September 19, 2009, 07:59:11 pm »

Well, poor ol' me decided to use the stocks screen to find out the composition of my mountain audit.

I have exactly 548,114 units of stone.

I'm 2.5 hours in and the game is still calculating.

Oh, and it's taking 1.25 GBs of RAM.

Not even Java is that slow.  Just how long is it supposed to take just to find out the number of entities that correspond to the stone category?!


UPDATE! The calculation has finished.  It went on for roughly 4 hours.
« Last Edit: September 19, 2009, 09:39:55 pm by Shurikane »
Logged

Draco18s

  • Bay Watcher
    • View Profile
Re: [40d] Massive lag when calculating in stocks screen.
« Reply #1 on: September 19, 2009, 08:33:15 pm »

Yeah.  We're not sure what exactly it's doing, but it does a lot of it.  20,000 is "too much" for most people.
Logged

Gertack

  • Bay Watcher
    • View Profile
Re: [40d] Massive lag when calculating in stocks screen.
« Reply #2 on: September 20, 2009, 12:44:56 am »

I suspect it asks, "how many granite are there?", followed by "how many talc are there?", and so on and so on, traversing the entire list of stones (or worse, all items) for every type of stone there is: O(n^2)

So with 548,114 stone that works out to roughly, assuming no "c", 300,428,956,996 calculations, which means Shuri's computer handled about 20,863,122 stone checks per second.

Easiest way to count: sort the list first.  Can also use a hash table.

But yeah, it's waaaaaaay slow.

(On a related note, my 0 FPS fort would love the same sort of attention to game scaling outside the stocks screen.)
Logged

Nadaka

  • Bay Watcher
    • View Profile
    • http://www.nadaka.us
Re: [40d] Massive lag when calculating in stocks screen.
« Reply #3 on: December 17, 2009, 12:54:41 am »

sorry, sort then iterate is a horrible way to count stones.

You are looking at an omega(NlogN) and omega(N) operation.

Using K buckets, one for each type of item (you don't even need a real bucket, just an integer counter), you can count all types of items in omega(NK) time.

If you do use real buckets as the items container with insertion at item creation, you can reduce that to an average omega(N/K) or even omega(1) if the bucket manages its count dynamically.
Logged
Take me out to the black, tell them I ain't comin' back...
I don't care cause I'm still free, you can't take the sky from me...

I turned myself into a monster, to fight against the monsters of the world.

profit

  • Bay Watcher
  • Finely Crafted Engravings... Or it didn't happen.
    • View Profile
Re: [40d] Massive lag when calculating in stocks screen.
« Reply #4 on: December 17, 2009, 07:39:58 am »

I created a mod to handle this problem... By melting it all down....
Logged
Mods and the best utilities for dwarf fortress
Community Mods and utilities thread.

Derakon

  • Bay Watcher
    • View Profile
Re: [40d] Massive lag when calculating in stocks screen.
« Reply #5 on: December 17, 2009, 04:57:38 pm »

Part of the issue is that the game has to check each stone to see if it's used for a building, forbidden, designated to be dumped, etc.

Profit: Shurikane said he was performing an audit. He wants the stone around; he just wants to know exactly how much of it he has. Making it go away doesn't solve anything.
Logged
Jetblade - an open-source Metroid/Castlevania game with procedurally-generated levels

numerobis

  • Bay Watcher
    • View Profile
Re: [40d] Massive lag when calculating in stocks screen.
« Reply #6 on: December 18, 2009, 03:47:55 pm »

Part of the issue is that the game has to check each stone to see if it's used for a building, forbidden, designated to be dumped, etc.
Half a million is pretty tiny for anything near-linear-time, so something wonky is up.  Even O(nm) is fast when m isn't very big (hundreds of types, say).  So that's not what's going on.
Logged

Shaostoul

  • Bay Watcher
  • Expanding your universe.
    • View Profile
    • Shaostoul Patreon
Re: [40d] Massive lag when calculating in stocks screen.
« Reply #7 on: December 20, 2009, 07:18:14 am »

Maybe it's just not handled as well as it could? I've noticed that any amount of "stone" (large = longer times) causes a slight pause to a long wait.

Stone could be substituted by say, crafts.
Logged
I mod games and educate others how to do so as well, if you'd like to learn join my Discord and you can join a bunch of like minded individuals. (Presently modding Space Engineers and No Man's Sky.)

Looking into modding DF? This forum guide & wiki guide may still be a good start!