Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[DuckBoy2]

You need more controls on your atom smashing experiment, anything with repeated toggling that causes pathing calculations to fire is going to be hell on your fps. 

On a 4x4 zone, I can drop my fps from whatever its capped to down to 96 with a single lever toggled bridge, or 88 with a row of 10 hatches, even if no one is doing anything requiring pathing. 

Ive got a science fort set up right now for doing automatic chutes and drop pods, and I can drop my fps from 100 to ~20 with the flick of a lever that flips 9 hatches, dropping my meeting hall onto floating pressure plates that toggle floodgates to drop my dwarves 10 stories onto a pile of ducks.  And I can do about the same just flipping the hatches and floodgates.

[bombzero]

4x4 zone, single lever toggled bridge,no pathing. 

so... you locked your dwarves in a room with a lever controlling a drawbridge that would smash them all.... awesome..

[GalenEvil]

This may or may not be relevant to your post, but it seems appropriate to at least post this in a !!SCIENCE!! thread instead of making my own since this is probably not viable !!SCIENCE!!...

Assumptions:
*   FPS is reduced by a non-sorted vector list of items (your assumption if I am reading this correctly)
*   FPS is reduced by the A* or A* equivalent pathing algorithm used by this game

My Theorems:
*   Creation and subsequent destruction of like-typed items (as in your experiment) will not create any appreciable FPS loss
*   Creation and subsequent destruction of variable numbers of non-like-typed items may create FPS loss (IE. create 10 bins, create 9 mugs, destroy 3 bins, create 10 bins, destroy 3 mugs, create 9 mugs, destroy 15 bins, destroy 10 mugs, create....etc) such that entire sets of item-creation jobs are not being immediately destroyed, and could possibly create empty vector indices within the lists as it is not guaranteed that an entire stack of a created good are destroyed entirely, which could, and logically would, free up the memory location if done haphazardly.
*   Corollary to the above: or at least a simplification of it. Each job set of a specific non-unique item could be creating a list set in the item vector list. If the above holds true then destroying only part of the list would not necessarily free up the memory associated with said list set to allow that set to be used for something else. Iff, (if and only if) the entirety of the item-set is destroyed the memory is freed.

Repercussions for this experiment:
*   With the terms of the experiment you, the player, are destroying objects immediately after creation, thus possibly voiding the memory set soon after its creation. This may be the cause of inconsistent results with your initial hypothesis.
*   Forcing the objects to being completely unique, by way of encrusting with gems or by other means may not alleviate the problem if the objects are being treated as unique within the lists, and the encrusting of said item(s) is only modifying the data that the object pointer is pointing to instead of creating an entirely new item.
*   As only a single type of item is being created possibly only a single list of entries is being created, and as such the number of iterations the engine needs to go through to access a particular object's entry is severely limited. Different types of items would be needed in order to create a "realistic" array of objects for the engine to search through to find a single item.
*   There may or may not be a separation between the following informations within the item data: Creator of item, location created (the object id # of the workshop that created the item), the quality of the created item, and any alterations made to the created item (such as encrustings, sewn on images, etc). More !!SCIENCE!! would be needed to figure this one out and if it has any effect on the item lists.

My general thoughts and musings that dont specifically have anything to do with this experiment:
*   A* pathfinding is pretty rediculous sometimes and doesn't always net the "best" result, especially if it is being checked every frame (leading to FPS death via creatures that are trapped and want to go elsewhere or otherwise extremely complicated path possibilities...)
*   This is a really good experiment, though I believe that much more !!SCIENCE!! will be needed to come to an accurate conclusion...perhaps a grant of some sort should be set up for research into this !!SCIENCE!! ? Lol
*   I may or may not be speaking entirely out of my rump right now as I do have a little bit of C++ experience as well as Java, and am in the process of creating a game engine to support a few of my game ideas....probably closer to the "may" than "may not" since everything is still in the ver 0.0000001a stages.

Hope that some things spark interesting musings within anyone who reads this, but not expecting it. Looking forward to seeing the conclusion to your !!SCIENCE!!

GalenEvil

[Kogut]

I wonder how much various user science bits have helped Toady over the years.

Recently somebody found what caused adamantine spires, hell on embark, collapsing caverns in the sky and general world corruption.

Wh... what was it?

http://www.bay12games.com/dwarves/mantisbt/view.php?id=5077

0005077: Browsing world gen map causes corrupted feature files

Spoiler: Also described as... (includes "3/4 of the map is a big hole going straight to hell") (click to show/hide)

has duplicate    0002463   resolved   Footkerchief    Reproducible crash on worldgen
has duplicate    0002432   resolved   Footkerchief    Worldgen crash while Placing Civilizations: Impoverished Word Selector
has duplicate    0002713   resolved   Footkerchief    Massive aboveground adamantine spire, dwarves ignore designations, sometimes crashes
has duplicate    0005343   resolved   Footkerchief    3/4 of the map is a big hole going straight to hell.
has duplicate    0005367   resolved   Footkerchief    cavern cavein on world gen/starting as an adventurer.
has duplicate    0005136   resolved   Footkerchief    Adventure mode crashes immediately after starting
has duplicate    0005187   resolved   Footkerchief    Embarking in a specific area causes a non recoverable lock up
has duplicate    0005401   resolved   Footkerchief    Crash on attempt to enter Dwarf Mode
has duplicate    0005043   resolved   Footkerchief    Human city inside of sand cavern
has duplicate    0004375   resolved   Footkerchief    Dwarven Town is in Hell. Huge lag makes game unplayable.
has duplicate    0005423   resolved   Footkerchief    Immediate crash upon selecting world and going to the site finder screen
has duplicate    0005425   resolved   Footkerchief    regions beyond region 1 not selectable and other worldgen weirdness
has duplicate    0005430   resolved   Footkerchief    two more necromancy bugs
has duplicate    0003831   resolved   Footkerchief    Start a new fort, dwarves all idle. No work at all.
has duplicate    0005306   resolved   Footkerchief    A new embark site has lava in the sky, cavern collapsed, game hangs up
has duplicate    0004163   resolved   Footkerchief    World gen doing crazy things with cavern placement.
has duplicate    0002268   resolved   Footkerchief    Cavern Lake Placement Error - Crossed Layers
has duplicate    0000051   resolved   Footkerchief    Cavern collapse at embark
has duplicate    0004833   resolved   Footkerchief    When embarking, crashes when memory use reaches 2GB.
has duplicate    0003356   resolved   Footkerchief    Embarked on a mountain of Slade, with Eerie Pits immediately revealed.
has duplicate    0003775   resolved   Footkerchief    World gen problems
has duplicate    0003101   resolved   Footkerchief    Square pit all the way down
has duplicate    0004840   resolved   Knight Otu    giant square hole on embark to hfs
has duplicate    0005268   resolved   Footkerchief    Crash on Embark
has duplicate    0005323   resolved   Footkerchief    Entering Fast Travel Causes Crash
has duplicate    0003884   resolved   Footkerchief    Consistent adventurer travel mode crash
has duplicate    0004385   resolved   Footkerchief    The Underworld has Collapsed.

http://www.bay12games.com/dwarves/mantisbt/view.php?id=5077#c20165

Lightning4: "I've discovered this bug exists as long as feature files have gone into the /current/ folder while browsing the map."

[Di]

Hey, nice science you have here.
I'd suggest you deleting all the creature raws except creature_standard (and deleting everyone out of it except dwarves) for a clear result. This will also prevent traders bugs from hindering you in your search for other bugs, since traders won't come without trade animals.
By the way, neither amount of raws, nor the duration of worldgen have noticeable effect on fps. (This can be verified by using the same seeds. I was testing that alongside with buying 150 pet dwarves on the embark in order to bring fps into standard range.)

[Sus]

This community is by far the best alpha-testing group granite-spiked socks can buy.

Agreement.
*Gives NW_Kohaku his ¤hematite sock¤  for great ‼SCIENCE‼*

[Sphalerite]

When doing this type of dedicated science fortress, I usually wall off the map edge with raising bridges.  While that does take a year or two to do even with a 2x2 embark, once you've done it you won't be bothered by immigrants, traders, invaders, wild animals, diplomats, or anything else spawning on the map edge.

[Quietust]

I wonder how much various user science bits have helped Toady over the years.

Recently somebody found what caused adamantine spires, hell on embark, collapsing caverns in the sky and general world corruption.

Wh... what was it?

It was failing to clean up intermediate files after aborting an embark and then incorporating them into the next world you generated. The files in question were map feature data, so the result was the game placing map features (cavern layers, the magma sea, the Underworld, adamantine spires, deep pits and magma pools, etc.) with totally invalid data and breaking the landscape.

[NW_Kohaku]

You need more controls on your atom smashing experiment, anything with repeated toggling that causes pathing calculations to fire is going to be hell on your fps. 

Actually, my controls for the atom smashing experiment will be to have a run where I start from the same save where I've set up the atom smasher, keep it running by flipping the lever, and just don't make or smash goblets.

The point of the experiment is to measure whether or not atom-smashed objects still have an impact on lag (and therefore, if atom-smashing objects is actually as good for your lag as people think it is), not whether or not drawbridges are bad for lag (which they clearly are, but that's not the experiment). 

I can say, however, that after going to sleep letting this thing run, what with my setup requiring that I manually (well, by macro) designate all those goblets in the workshops for dumping, simply having large numbers of objects on the map has a clear impact on FPS. 

10,000 goblets on the map - runs at 120 FPS. 
30,000 goblets on the map - runs at 60 FPS.

I've been trying to manage to keep this at around 10,000 goblets on the map while I was watching it, so as to keep a decent perspective on the FPS as it rises or declines, but I will finish my experiment off with no goblets left un-smashed so as to compare it to the control.  And basically, the control could just be the fork save itself, as I don't have any noticable FPS decay as of yet.

What I'm finding may just be that my whole assumption on how this lag was being caused was wrong, and the only thing really causing lag this whole time was the animals.  Although I still want to test a large caravan save, an item-improvement save, and some others.

Again, if someone wants to help set up some sample data by running this stuff themselves, this is really brain-dead stuff to run (basically, you only have to get rid of the trader, but you just leave it running on its own for spring-autumn and post-trader to winter.) It just takes basically 4 hours of simulation running time to actually accrue the 10 years of sample data.  Which means that it takes the better part of my day per experiment I have to run if I'm only running off my own computer.

Someone helping out would make this go much faster.

[kaijyuu]

Dunno how relevant this might be, but my FPS drops usually happen when my metalsmithing gets into full swing. I usually make 20-30 magma smelters and let them have at it for a few years. Digging out the veins doesn't cause it, but once dwarves start running all over the map, making tons of bars, my FPS drops (and never returns, even if I cancel all the jobs). Notably, this causes no increase in total items; once I get steel production going, it actually causes a drop.


Could be jobs somehow causing FPS death, maybe items in buildings (as my bar stockpiles are always overflowing).




Really though, my suspicion is the cause being units. Dead units probably still eat up CPU time, though not to the degree of alive ones. I wonder if Toady sticks the dead ones in a separate vector when cycling through each unit's behavior; we know they still exist (to some degree) in memory due to the dead/missing list.

[NW_Kohaku]

Dunno how relevant this might be, but my FPS drops usually happen when my metalsmithing gets into full swing. I usually make 20-30 magma smelters and let them have at it for a few years. Digging out the veins doesn't cause it, but once dwarves start running all over the map, making tons of bars, my FPS drops (and never returns, even if I cancel all the jobs). Notably, this causes no increase in total items; once I get steel production going, it actually causes a drop.


Could be jobs somehow causing FPS death, maybe items in buildings (as my bar stockpiles are always overflowing).

Could you describe your metalsmithing process?

As in, do you use coal or charcoal from cutting down aboveground/underground trees, and do you set up special farms for those? 

Do you use magma forges? 

What types of items do you make?  (Steel only, or do you make non-steel crafts?  Do you only make weapons or do you make craft items for sale, as well?)

Do you trade your metal items away to caravans?

There are different potential points for FPS loss for different ways in which you operate your industry...

[kaijyuu]

Certainly.


Magma smelters/forges entirely.
Don't do strip mining; I do exploratory mining. Emptied passages/veins are sealed off for pathing purposes.
20-30 magma smelters, as I said before.
Never really any big stockpiles, so most bars stay in smelters.
I make a ton of bolts and shields to level up my weaponsmith/armorsmith.
Bins to level up my blacksmith.
Bins/weapons/armor are emptied almost immediately so there's no buildup there.
Steel is usually done with coal/lignite. These do produce a net gain of total items, I know, but not by any huge amount (few thousand with a ton of coal). Never really make tree farms or anything.
Metal items are usually kept, though I melt down the non-masterwork stuff. I trade away stone crafts (usually mechanisms).


The funny thing is, I don't usually start this up until I have a significant amount mined out. It's mostly to give my idlers and miners something to do. So as far as total items goes, this isn't what boosts the item count on my maps usually (that'd be farming). I have 250 FPS forts allll the way until my industry's set up, then, I'm down to 140 and steadily sinking.

[NW_Kohaku]

Magma smelters/forges entirely.
Don't do strip mining; I do exploratory mining. Emptied passages/veins are sealed off for pathing purposes.
20-30 magma smelters, as I said before.

There are two major things that stand out - first, you need to know what the total number of items are in your fortress.  There are checks (like temperature) that take place per item, so having large numbers of total items will cause major problems.  (Although stack items like 25 bone bolts will probably be counted as just one.)

The other is your magma forge.  How do you supply the magma to your magma forge?  Unless you set your forges up directly on top of the magma sea, you might be introducing some liquid motion, which brings up the fluid mechanics that are real FPS drains.  Magma motion also brings up more temperature checks that can be a serious strain on FPS.

[kaijyuu]

Always the magma sea, pipes, or volcanoes. I literally never have built a single pump in my entire DF career.

And temperature's always turned off for me, by the way.




Might be coincidence and something else, I dunno. I do try to limit the number of items I produce, and am constantly destroying stuff (like stone). FPS still drops. My bets are still on units; buildup of vermin is possible, along with dead/gone units still being counted somehow.

[rooth]

Posting to watch and a small question;

Wouldn't it be more efficient to use DF hack's ''autodump destroy'' instead of atomsmashing? It, as far as I know, essentially does the same but gives you much faster work results.

some other science on the subject;

I've done a ''huge focus on fps'' fort on 34.05 on a 3x3 embark and currently have +/-150 dwarves, first cavern acces on a deep map, about 200 animals in cages, 200 on the loose breeding bounce between 135 and 140[my cap] fps after 17 gameyears. The ''one lever that ruines pathing'' hypothesis is one i can fully get behind, it's the only method I could so far use to get my fps to drop below 70 yet, it jumping back to 140 when the ''pathing cooled down'' [for example; forest titan appears, use alarm to get people in side, close drawbridges, release them from alert, restabilizes after a short while]

I'm really glad Toady is working on clothing as I used to have 30 fps forts after 20 years and the biggest change in fps for me was getting rid of clothes, that literally was the difference between 20 and 80 fps on old forts. All the other tricks like efficient building, conserving itemload, caging pets,turning off temp etc can get me to my cap but the clothing fix did not change my game in any way, while losing temp or havign to create your own magma forges with Dfhack kinda removes some of the game's FUN aspects 

I've even considered lowering my fps cap as now that my 60 children are growing up, I'm aiming to work out some narratives and for some reason I could not get the same connection to my dwarves on 140 fps as I used to have on my older laggy forts; everything just flew by !