Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[Kogut]

Why the new account?

To reset list of watched threads ( http://www.bay12forums.com/smf/index.php?action=unreadreplies )

[Xen0n]

Why the new account?

To reset list of watched threads ( http://www.bay12forums.com/smf/index.php?action=unreadreplies )

Ha!  I was actually just asking myself how I coul go about doing that last week, and I guess you've given me an answer!  Thanks!

[ghostwoods]

I've been worried by the reports that open space causes FPS loss, even when it's not being pathed through. If it's inevitable, it's inevitable, but I wanted to see if pattern played a part.

HYPOTHESIS: There may be a different in FPS impact of open space, depending on layout.

METHODOLOGY: Set up a fort with food-, sleep- and booze-proof Dwarf miners in a flat place with a low cloud ceiling and no enemy civs. Wall them in and prepare a suite of craftshops on the first stone layer. Fork this fort. In one fork, dig out a 50x50 square of tiles. In the other fork, dig out a 2500-long snaking passageway. Collect all stone in a quantum pile, in the same spot near the craftshops. Set the craftshops running on the same task, and compare FPS. Once out of stone, wall off the 2500-tile gap, wait until the first caravan has come and gone, and compare FPS whilst everyone idles at a surface-side statue.

NOTES: Used autodump to collect the stone in a pile, then reclaimed it and slapped a 1-square stockpile over it.  After the crafting, dug out a bit more stone around the craftshops until there was enough there to wall off the stairs. (Totally forgot I'd brought some wood for that purpose). The two migrant waves weren't quite the same size -- 11 adults and 1 child in the square fork; 6 adults, 2 horses, 2 cats and a child in the snake fork.

RESULTS

All results are +/- 15ish FPS. Fort was saved and DF fully quitted between each of the measurements

Code: [Select]

                                               SQUARE             SNAKE

DURING CRAFTING                                390                470
IDLING BEFORE CARAVAN                          370                445
DURING CARAVAN                                 335                410
AFTER CARAVAN                                  350                425     

CONCLUSION

Holy hell. The Snake layout is a consistent 20% faster than the Square layout right across the board. And that's off just 2500 squares. I've used twice that just mining marble to trade on one layer of my current fort.

I suggest a new strategy, Artoo. TUNNEL EVERYTHING!

[Xenos]

I will try some testing once my new computer parts arrive and I build my new baby.  Since I am busting out an AMD 3100 MHz octocore processor I can run multiple tests concurrently and boost efficiency!    (I only bought 8GB RAM since I wanted to save a little money but I can easily buy another stick soon to run more than 2-3 tests at once.)

Let me know what you want me to test more specifically. 

[Xen0n]

I've been worried by the reports that open space causes FPS loss, even when it's not being pathed through. If it's inevitable, it's inevitable, but I wanted to see if pattern played a part.

HYPOTHESIS: There may be a different in FPS impact of open space, depending on layout.

METHODOLOGY: Set up a fort with food-, sleep- and booze-proof Dwarf miners in a flat place with a low cloud ceiling and no enemy civs. Wall them in and prepare a suite of craftshops on the first stone layer. Fork this fort. In one fork, dig out a 50x50 square of tiles. In the other fork, dig out a 2500-long snaking passageway. Collect all stone in a quantum pile, in the same spot near the craftshops. Set the craftshops running on the same task, and compare FPS. Once out of stone, wall off the 2500-tile gap, wait until the first caravan has come and gone, and compare FPS whilst everyone idles at a surface-side statue.

NOTES: Used autodump to collect the stone in a pile, then reclaimed it and slapped a 1-square stockpile over it.  After the crafting, dug out a bit more stone around the craftshops until there was enough there to wall off the stairs. (Totally forgot I'd brought some wood for that purpose). The two migrant waves weren't quite the same size -- 11 adults and 1 child in the square fork; 6 adults, 2 horses, 2 cats and a child in the snake fork.

RESULTS

All results are +/- 15ish FPS. Fort was saved and DF fully quitted between each of the measurements

Code: [Select]

                                               SQUARE             SNAKE

DURING CRAFTING                                390                470
IDLING BEFORE CARAVAN                          370                445
DURING CARAVAN                                 335                410
AFTER CARAVAN                                  350                425     

CONCLUSION

Holy hell. The Snake layout is a consistent 20% faster than the Square layout right across the board. And that's off just 2500 squares. I've used twice that just mining marble to trade on one layer of my current fort.

I suggest a new strategy, Artoo. TUNNEL EVERYTHING!

Spoiler: Oh wow!  That's some impressive SCIENCE, and potentially gamechanging, design-wise. (click to show/hide)

So I'm understanding correctly, in both cases the 2500 tiles of empty space were totally walled off and inaccessible, the only difference was the shape?  The snaking passageway was one single curving hallway, or a branching maze?  And both of these were on a single z-level?

Makes me curious what the difference between a 50 x 50 x 50 cube of open air vs. 125000 tiles of passages (one 1 or across several z-levels),

Spoiler: but I do not have the chops to set something like that up :P (click to show/hide)

[Bulwersator]

I think that this may be explained by different migrant waves. To eliminate this problem in my test I walled entire map using drawbridges or waited for hardcoded waves and then lowered migrant/child cap to 0.

[ghostwoods]

Xen0n: Thanks Both spaces were on a single z-layer, and the strip was just one long twisty passage, not a maze.

You could do a 2744-cubed open space by channeling a 14x14 block down 14 levels, but in order to avoid !!FUN!! you'd probably need to assign it as one strip of 14 squares at a time. You _really_ wouldn't want to have to do that 50x50x50! A similar space in one strip could be done with staircases in a 27x27x14 block (you don't need to leave a blank layer between z-levels, as mining leaves a floor).

Bulwersator: It's possible, but the first figures recorded are from the six weeks before any migrants arrived, and they show almost exactly the same discrepancy.

[Urist Da Vinci]

I've been worried by the reports that open space causes FPS loss, even when it's not being pathed through. If it's inevitable, it's inevitable, but I wanted to see if pattern played a part.

HYPOTHESIS: There may be a different in FPS impact of open space, depending on layout.

...

CONCLUSION

Holy hell. The Snake layout is a consistent 20% faster than the Square layout right across the board. And that's off just 2500 squares. I've used twice that just mining marble to trade on one layer of my current fort.

I suggest a new strategy, Artoo. TUNNEL EVERYTHING!

I'd like to verify that the FPS loss isn't because the space IS being pathed through. I'd suggest repeating the 50x50 open space test when the map is painted with a large high traffic zone. I get the feeling that the results will be equal to the tunnel method.

[expwnent]

Better yet, put it past a forbidden door.

[Xen0n]

I've been worried by the reports that open space causes FPS loss, even when it's not being pathed through. If it's inevitable, it's inevitable, but I wanted to see if pattern played a part.

HYPOTHESIS: There may be a different in FPS impact of open space, depending on layout.

...

CONCLUSION

Holy hell. The Snake layout is a consistent 20% faster than the Square layout right across the board. And that's off just 2500 squares. I've used twice that just mining marble to trade on one layer of my current fort.

I suggest a new strategy, Artoo. TUNNEL EVERYTHING!

I'd like to verify that the FPS loss isn't because the space IS being pathed through. I'd suggest repeating the 50x50 open space test when the map is painted with a large high traffic zone. I get the feeling that the results will be equal to the tunnel method.

I was actually about to suggest the same thing, or editing the 'Normal' traffic values to =1.  I'm also curious about a control fort, with no spaces mined out, but maybe an equivalent amount of stone hacked in to keep the # of items + crafting consistent, just to see how big of a hit mining out a space like this is compared to not mining at all.

[ghostwoods]

Wouldn't it be a bug if pathing was being calculated through solid stone walls? The 2500-square empty spaces were completely walled off after the initial construction phase, in both cases. Xen0n, I'm not that much of a hacker. I'll be interested to hear what you find out.

[Xen0n]

Wouldn't it be a bug if pathing was being calculated through solid stone walls? The 2500-square empty spaces were completely walled off after the initial construction phase, in both cases. Xen0n, I'm not that much of a hacker. I'll be interested to hear what you find out.

Haha, I was hoping somebody would know if doing something like was even possible.  As much as I'd like to try something like this out, I spend so much time doing science IRL that I barely have time to log a couple hours into DF per week

[WJLIII3]

I am suspending my atom-smashing test for now to check on the contraction theory, instead.

I am going to try to have a million stones on my map at a single point in time before eliminating them all.

I am apparently up to roughly 100,000 stones on the map at once, but it appears as though the sheer numbers have completely overwhelmed my bookkeeper, or else that bug is back, because I'm down to lowest possible precision again. 

FPS has plummeted way down to 30 from "merely" having 100,000 stones on the map.

No bug, I shouldn't think. You create and destroy 240 units of stone every tick. The bookkeeper is trying to keep track of them, which is completely impossible.

[Xob Ludosmbax]

Testing effects of open space on FPS using dfhack. 

The goal is to increase understand of long term FPS loss with the hope of discovering how to decrease or eliminate it.

Summary of Conclusions:

Pathfinding is a lot more significant than open space with regards to FPS.  Clearing a large section of the map only has about a 12% effect on FPS in and of itself.  Pathfinding through that space while unable to reach the destination had about 5 times more effect on FPS.  I didn't count the number of flying clowns.  Visible but non-open  tiles do not seem to have a significant effect. 

Also, we could probably get better results by timing how long it takes to finish the season rather than a subjective evaluation of the FPS meter, then timing the second season, as paths and such would be cached by then. 

Further research questions:

Spoiler (click to show/hide)

* Redo the experiments measuring elapsed real-time per season rather than looking at the FPS meter.
* Is dfhack tiletypes and manual in-game digging equivalent in terms of FPS?
* If the clowns could reach the dwarfs, would that reduce the FPS penalty?
* Large amounts of accessible/inaccessible floor (as opposed to open air) would be interesting to look further into, as the results were strange.  Ghostwoods's experiment was in this vein. 
* How much would it affect FPS if the dwarfs were able to reach the empty space (or floor), then it was blocked off by various methods (wall, floor, bridge, door).
* (Tangental) Is clown release deterministic?
* (Tangental) Get someone to put up a series of saves for a fort suffering from long term FPS loss, starting from embark, so that a comparative analysis can be done.  What year has the largest FPS loss?  What changed?  In years that have little FPS loss, what changed so we can rule it out?

Baseline parameters:

Spoiler (click to show/hide)

FPS cap set to 10k
All tests were started with a fresh copy of the fork, and terminated at the end of the first spring.
All setup was done before the first unpause after loading.  No changes were made after that.
In most tests, FPS has an initial stable value and a second stable value after a few weeks game time.  The final stable value is the one reported. 
It isn't part of the experiment, but DF is running in a VM.

Test 0: Save with no changes
FPS seems stableish at 400.

Test 1: 129 z-levels of open air (with clown introduced measurement error)

Spoiler (click to show/hide)

Go to level -3, set hotkey F3
K, move curser to NW corner, then move SE 1 tile.
liquids
range
x: 92, y: 92, z: 131, depth: 0
<enter>
q

tiletypes
range
x: 92, y: 92,z: 131
paint mat stone
paint sh wall
paint hidden 0
<enter>

Move cursor 1 more tile SE, and up 1 zlevel
range
x: 90, y: 90,z: 129
paint mat air
paint sh empty
<enter>

(Oops, the circus came to town, water fills a few individual squares.)

FPS seems to be at a stable 200, sometimes dropping to 160-180, went all the way up to 250.

Test 2: 3 z-levels of open air (with clown introduced measurement error)

Spoiler (click to show/hide)

Go to level -3, set hotkey F3
K, move curser to NW corner, then move SE 1 tile.
liquids
range
x: 92, y: 92, z: 5, depth: 0
<enter>
q

tiletypes
range
x: 92, y: 92,z: 5
paint mat stone
paint sh wall
paint hidden 0
<enter>

Move cursor 1 more tile SE, and up 1 zlevel
range
x: 90, y: 90,z: 3
paint mat air
paint sh empty
<enter>

(Oops, the circus came to town, water fills a few individual squares.)

FPS eventually stabilised in the 300-320 range, with spikes to 360.

Conclusions for tests 1 & 2:
We can conclude that ~1M tiles of inaccessible empty space drops FPS by ~68%, or put differently, every 15k visible tiles drops FPS by ~1% (assuming it's linear).

Oops, I made a mistake.  Flying clowns need to pathfind through all that space.  That invalidates the results as pathfinding and open space have became conflated.

It should be noted that if no path to the destination is available  A* will do a complete and exhaustive search of every tile accessible to the unit, which is a huge performance hit.  This is pretty much worst case for A* pathfinding. 

Let's try this again without clowns.

Test 3: 128 z-levels of visible open air.

Spoiler (click to show/hide)

Go to level 2, set hotkey F3
K, move curser to NW corner, then move SE 1 tile.
liquids
range
x: 92, y: 92, z: 130, depth: 0
<enter>
q

tiletypes
range
x: 92, y: 92,z: 130
paint mat stone
paint sh wall
paint hidden 0
paint special 0
<enter>

Move cursor 1 more tile SE, and up 1 zlevel
range
x: 90, y: 90,z: 128
paint mat air
paint sh empty
<enter>

FPS eventually stabilised in the 340-360 range, with spikes to 400. 

Test 4: 3 levels of visible open air

Spoiler (click to show/hide)

Go to level 2, set hotkey F3
K, move curser to NW corner, then move SE 1 tile.

liquids
range
x: 92, y: 92, z: 5, depth: 0
<enter>
q

tiletypes
range
x: 92, y: 92,z: 5
paint mat stone
paint sh wall
paint hidden 0
paint special 0
<enter>

Move cursor 1 more tile SE, and up 1 zlevel
range
x: 90, y: 90,z: 3
paint mat air
paint sh empty
<enter>

FPS eventually stabilises at 400, spiking to 420 and dropping to 360.

Conclusion for Tests 3 & 4:

We can conclude that ~1M tiles of inaccessible empty space drops FPS by ~12%, or put differently, every 83k visible tiles drops FPS by ~1% (assuming it's linear, which it probably isn't).

Further, we can attempt to unconflate pathfinding from the first run through.  It seems ~84.6% of the FPS drop in the first run through can be attributed to pathfinding.  Therefore FPS reduced by pathfinding through 98k empty tles is about 58%.  Put another way, every 98k visible tiles reduced FPS by 1% due to pathfinding.  (same linear disclaimer as above.)

-- Additional experiments --

Test 5: 128 z-levels of visible stone wall.

Spoiler (click to show/hide)

Go to level 2, set hotkey F3
K, move curser to NW corner, then move SE 1 tile.

liquids
range
x: 92, y: 92, z: 130, depth: 0
<enter>
q

tiletypes
range
x: 92, y: 92,z: 5
paint mat stone
paint sh wall
paint hidden 0
paint special 0
<enter>

FPS eventually stabilises at 380-400.

Conclusion for test 5:
Visible, unopen space had no noticable effect on FPS. 

Test 6: 128 z-levels of floor

Spoiler (click to show/hide)

Go to level 2, set hotkey F3
K, move curser to NW corner, then move SE 1 tile.
liquids
range
x: 92, y: 92, z: 130, depth: 0
<enter>
q

tiletypes
range
x: 92, y: 92,z: 130
paint mat stone
paint sh wall
paint hidden 0
paint special 0
<enter>

Move cursor 1 more tile SE, and up 1 zlevel
range
x: 90, y: 90,z: 128
paint sh floor
<enter>

FPS seems a lot more variable, fluctuating from at least several seconds of stability at many values between 140 and 400.  If I had to pick a number, I would say 200, but I definitely did not get a good reading. 

No conclusions drawn due to poor measurements

Test 7: 128 z levels of visible air with access via stairs

Spoiler (click to show/hide)

Go to level 2, set hotkey F3
K, move curser to NW corner, then move SE 1 tile.
liquids
range
x: 92, y: 92, z: 130, depth: 0
<enter>
q

tiletypes
range
x: 92, y: 92,z: 130
paint mat stone
paint sh wall
paint hidden 0
paint special 0
<enter>

Move cursor 1 more tile SE, and up 1 zlevel
range
x: 90, y: 90,z: 128
paint mat air
paint sh empty
<enter>

Move cursor to NW corner of level 131, and then move SE by 2
range
x: 1, y: 1,z: 20
paint mat stone
paint sh stair_updown

F3 and up a level, so the same stuff is on-screen as in other tests while we're checking FPS

FPS: 340-360  It seemed to take longer to reach the stable point than test 3, but that wasn't measured.

[expwnent]

Great work! I'll stop worrying so much about non-pathable open space.