Bay 12 Games Forum

Please login or register.

Login with username, password and session length
Advanced search  

Author Topic: Advanced World Gen "finalizing sites..." crash  (Read 1808 times)

tnuhnivad

  • Bay Watcher
  • <dorf> has been possessed!
    • View Profile
Advanced World Gen "finalizing sites..." crash
« on: February 01, 2018, 02:06:25 am »

I'm attempting to make a very large world with little ocean area to maximize playable area, very long history 800+ specfically for generating lots of dragons to reach full size, large amount of megabeast and many semi's. My param's are as follows...


[WORLD_GEN]
   [TITLE:TEST GEN]
   [DIM:257:257]
   [EMBARK_POINTS:1504]
   [END_YEAR:2]
   [BEAST_END_YEAR:300:80]
   [REVEAL_ALL_HISTORY:1]
   [CULL_HISTORICAL_FIGURES:0]
   [ELEVATION:35:400:3200:3200]
   [RAINFALL:4:100:400:400]
   [TEMPERATURE:25:75:400:400]
   [DRAINAGE:0:100:400:400]
   [VOLCANISM:0:100:400:400]
   [SAVAGERY:0:100:400:400]
   [ELEVATION_FREQUENCY:2:1:65:50:35:25]
   [RAIN_FREQUENCY:2:1:25:50:50:1]
   [DRAINAGE_FREQUENCY:2:50:40:20:10:1]
   [TEMPERATURE_FREQUENCY:1:1:1:1:1:1]
   [SAVAGERY_FREQUENCY:2:50:35:25:15:5]
   [VOLCANISM_FREQUENCY:2:1:1:1:1:50]
   [POLE:NORTH_AND_SOUTH]
   [MINERAL_SCARCITY:4500]
   [MEGABEAST_CAP:15000]
   [SEMIMEGABEAST_CAP:5000]
   [TITAN_NUMBER:225]
   [TITAN_ATTACK_TRIGGER:0:0:100000]
   [DEMON_NUMBER:69]
   [NIGHT_TROLL_NUMBER:69]
   [BOGEYMAN_NUMBER:69]
   [VAMPIRE_NUMBER:69]
   [WEREBEAST_NUMBER:69]
   [SECRET_NUMBER:200]
   [REGIONAL_INTERACTION_NUMBER:69]
   [DISTURBANCE_INTERACTION_NUMBER:69]
   [EVIL_CLOUD_NUMBER:69]
   [EVIL_RAIN_NUMBER:69]
   [GENERATE_DIVINE_MATERIALS:1]
   [GOOD_SQ_COUNTS:0:0:0]
   [EVIL_SQ_COUNTS:100:500:2500]
   [PEAK_NUMBER_MIN:0]
   [PARTIAL_OCEAN_EDGE_MIN:0]
   [COMPLETE_OCEAN_EDGE_MIN:0]
   [VOLCANO_MIN:0]
   [REGION_COUNTS:SWAMP:0:0:0]
   [REGION_COUNTS:DESERT:0:0:0]
   [REGION_COUNTS:FOREST:6000:0:0]
   [REGION_COUNTS:MOUNTAINS:0:0:0]
   [REGION_COUNTS:OCEAN:5000:0:0]
   [REGION_COUNTS:GLACIER:0:0:0]
   [REGION_COUNTS:TUNDRA:0:0:0]
   [REGION_COUNTS:GRASSLAND:2000:0:0]
   [REGION_COUNTS:HILLS:2000:0:0]
   [EROSION_CYCLE_COUNT:500]
   [RIVER_MINS:350:350]
   [PERIODICALLY_ERODE_EXTREMES:1]
   [OROGRAPHIC_PRECIPITATION:1]
   [SUBREGION_MAX:5000]
   [CAVERN_LAYER_COUNT:3]
   [CAVERN_LAYER_OPENNESS_MIN:75]
   [CAVERN_LAYER_OPENNESS_MAX:100]
   [CAVERN_LAYER_PASSAGE_DENSITY_MIN:0]
   [CAVERN_LAYER_PASSAGE_DENSITY_MAX:25]
   [CAVERN_LAYER_WATER_MIN:50]
   [CAVERN_LAYER_WATER_MAX:100]
   [HAVE_BOTTOM_LAYER_1:1]
   [HAVE_BOTTOM_LAYER_2:1]
   [LEVELS_ABOVE_GROUND:15]
   [LEVELS_ABOVE_LAYER_1:15]
   [LEVELS_ABOVE_LAYER_2:5]
   [LEVELS_ABOVE_LAYER_3:5]
   [LEVELS_ABOVE_LAYER_4:1]
   [LEVELS_ABOVE_LAYER_5:2]
   [LEVELS_AT_BOTTOM:1]
   [CAVE_MIN_SIZE:50]
   [CAVE_MAX_SIZE:150]
   [MOUNTAIN_CAVE_MIN:69]
   [NON_MOUNTAIN_CAVE_MIN:20]
   [ALL_CAVES_VISIBLE:1]
   [SHOW_EMBARK_TUNNEL:2]
   [TOTAL_CIV_NUMBER:15]
   [TOTAL_CIV_POPULATION:15000]
   [SITE_CAP:1500]
   [PLAYABLE_CIVILIZATION_REQUIRED:1]
   [ELEVATION_RANGES:0:0:0]
   [RAIN_RANGES:0:0:0]
   [DRAINAGE_RANGES:0:0:0]
   [SAVAGERY_RANGES:0:0:0]
   [VOLCANISM_RANGES:0:0:0]

I've tested extensively altering one thing at a time to make sure it works then add in another change I want and so on so I don't go to generate such a long history to run into issues. Everything was working just fine until I started cranking up the megabeast, semis, and titans. at which point the world gen will make it all the way to "finalizing sites..." at which point the application doesn't necessarily crash but hangs endlessly. Can anyone pin point or help me understand why this is occurring? I've included a screen shot below.

https://i.imgur.com/zRuaRqo.png
Logged

Shonai_Dweller

  • Bay Watcher
    • View Profile
Re: Advanced World Gen "finalizing sites..." crash
« Reply #1 on: February 01, 2018, 02:58:57 am »

Are you using 32 bit or 64 bit DF?
Logged

soulsource

  • Bay Watcher
    • View Profile
Re: Advanced World Gen "finalizing sites..." crash
« Reply #2 on: February 01, 2018, 03:02:27 am »

Just a shot in the dark, but how does system resource usage look like when the application hangs (especially RAM)?
Might be that df runs out of physical memory (certainly possible for very large worlds), and either starts swapping (what will slow it down to a near halt), or just have allocations fail (if memory is allocated as non-pageable), what could, depending on how (if) those errors are handled, lead to all kinds of crazy behaviour.

Also, as Shonai_Dweller said, if you're using a 32bit build, you're very likely at the address space limit (which is 4 GB for 32bit applications on 64bit Linux, and less on all other platforms - I don't remember the numbers by heart).

If you're certain that the game is far from using all your resources, it might be worth reporting this as a bug on the official bug tracker, as it seems reproducible.
Logged
Quote from: Porkins Windu
Really, DF and Metal are practically the same. Drunkenness, Death, Depression, Depravity, Despondence, Demons, Dementia, Darkness. And thats just the D's!

tnuhnivad

  • Bay Watcher
  • <dorf> has been possessed!
    • View Profile
Re: Advanced World Gen "finalizing sites..." crash
« Reply #3 on: February 01, 2018, 10:18:58 am »

Just a shot in the dark, but how does system resource usage look like when the application hangs (especially RAM)?
Might be that df runs out of physical memory (certainly possible for very large worlds), and either starts swapping (what will slow it down to a near halt), or just have allocations fail (if memory is allocated as non-pageable), what could, depending on how (if) those errors are handled, lead to all kinds of crazy behaviour.

Also, as Shonai_Dweller said, if you're using a 32bit build, you're very likely at the address space limit (which is 4 GB for 32bit applications on 64bit Linux, and less on all other platforms - I don't remember the numbers by heart).

If you're certain that the game is far from using all your resources, it might be worth reporting this as a bug on the official bug tracker, as it seems reproducible.

Well, I'm using 44.05 which I thought was 64 bit. I glanced at the task manager processes when it hanged and it is up there in usage. As to be expected but I have a pretty good machine with 8gb of ram, a pretty beast i5, etc. When I downloaded DF 44.05 I just clicked " windows download" Is 64bit 44.05 something I have to specifically find because I thought the game went to 64 bit standard after a certain version?
Logged

tnuhnivad

  • Bay Watcher
  • <dorf> has been possessed!
    • View Profile
Re: Advanced World Gen "finalizing sites..." crash
« Reply #4 on: February 01, 2018, 10:26:29 am »

Just a shot in the dark, but how does system resource usage look like when the application hangs (especially RAM)?
Might be that df runs out of physical memory (certainly possible for very large worlds), and either starts swapping (what will slow it down to a near halt), or just have allocations fail (if memory is allocated as non-pageable), what could, depending on how (if) those errors are handled, lead to all kinds of crazy behaviour.

Also, as Shonai_Dweller said, if you're using a 32bit build, you're very likely at the address space limit (which is 4 GB for 32bit applications on 64bit Linux, and less on all other platforms - I don't remember the numbers by heart).

If you're certain that the game is far from using all your resources, it might be worth reporting this as a bug on the official bug tracker, as it seems reproducible.

I checked the downloads on bay 12 and sure enough there are two options if you go to all versions. Will download the for sure 64 bit and try again.
Logged

tnuhnivad

  • Bay Watcher
  • <dorf> has been possessed!
    • View Profile
Re: Advanced World Gen "finalizing sites..." crash
« Reply #5 on: February 01, 2018, 10:39:35 am »

Just a shot in the dark, but how does system resource usage look like when the application hangs (especially RAM)?
Might be that df runs out of physical memory (certainly possible for very large worlds), and either starts swapping (what will slow it down to a near halt), or just have allocations fail (if memory is allocated as non-pageable), what could, depending on how (if) those errors are handled, lead to all kinds of crazy behaviour.

Also, as Shonai_Dweller said, if you're using a 32bit build, you're very likely at the address space limit (which is 4 GB for 32bit applications on 64bit Linux, and less on all other platforms - I don't remember the numbers by heart).

If you're certain that the game is far from using all your resources, it might be worth reporting this as a bug on the official bug tracker, as it seems reproducible.


Tried again with the 64 bit version if I wasn't using it before and the same parameters, again, hangs at finalizing sites. Using about 1.3 gb of RAM.
« Last Edit: February 01, 2018, 03:38:54 pm by tnuhnivad »
Logged

Shonai_Dweller

  • Bay Watcher
    • View Profile
Re: Advanced World Gen "finalizing sites..." crash
« Reply #6 on: February 01, 2018, 04:00:08 pm »

Yeah, that crashes (hangs at Finalizing Sites) reliably for me too (vanilla 44.05).

Reducing maximum beast sites to 1500 and 500 respectively works though. It hangs for a while on Finalizing, then completes.

Your 5000 semis works ok too with a lower megabeast number. It's just megabeast max of 15000 that seems to be causing the freeze. Which is weird as it's just a max number. It shouldn't have to think about anything at the end of worldgen. Definitely one for the bug tracker.
« Last Edit: February 01, 2018, 04:19:38 pm by Shonai_Dweller »
Logged

daggaz

  • Bay Watcher
    • View Profile
Re: Advanced World Gen "finalizing sites..." crash
« Reply #7 on: February 01, 2018, 05:02:16 pm »

You should try 14999 megabeasts. 
Logged

Shonai_Dweller

  • Bay Watcher
    • View Profile
Re: Advanced World Gen "finalizing sites..." crash
« Reply #8 on: February 01, 2018, 05:45:19 pm »

You should try 14999 megabeasts.
Why, what happens then?
Logged

tnuhnivad

  • Bay Watcher
  • <dorf> has been possessed!
    • View Profile
Re: Advanced World Gen "finalizing sites..." crash
« Reply #9 on: February 01, 2018, 11:05:59 pm »

Yeah, that crashes (hangs at Finalizing Sites) reliably for me too (vanilla 44.05).

Reducing maximum beast sites to 1500 and 500 respectively works though. It hangs for a while on Finalizing, then completes.

Your 5000 semis works ok too with a lower megabeast number. It's just megabeast max of 15000 that seems to be causing the freeze. Which is weird as it's just a max number. It shouldn't have to think about anything at the end of worldgen. Definitely one for the bug tracker.

Yea I ended up reducing the overall megabeast caves to 1000. I did report it to the bug tracker tho. In order to achieve my original plans I ended up editing megabeast raws. Adding a few things to make them a little harder to kill, upped frequency/pop tokens, etc. Now my issue is trying to find the right numbers so civs can survive the length of time I'm looking for.
Logged

Shonai_Dweller

  • Bay Watcher
    • View Profile
Re: Advanced World Gen "finalizing sites..." crash
« Reply #10 on: February 02, 2018, 01:32:03 am »

It's funny it crashes at the end though, isn't it? I mean, it's already made the world, placed up to 15000 megabeast lairs (presumably a lot less due to lack of space), had them roaming about throughout history only to then hang on Finalizing Sites. What's it doing?
Logged

soulsource

  • Bay Watcher
    • View Profile
Re: Advanced World Gen "finalizing sites..." crash
« Reply #11 on: February 02, 2018, 01:40:58 am »

To find out what the game is doing, one would probably need a debugging build and source code access.
Logged
Quote from: Porkins Windu
Really, DF and Metal are practically the same. Drunkenness, Death, Depression, Depravity, Despondence, Demons, Dementia, Darkness. And thats just the D's!

daggaz

  • Bay Watcher
    • View Profile
Re: Advanced World Gen "finalizing sites..." crash
« Reply #12 on: February 02, 2018, 02:18:39 am »

You should try 14999 megabeasts.
Why, what happens then?

Dont know, that's the point.  If you have a max-range issue, a common first test is to set the counter down by one, to see if it is specifically that value or just any high value that is causing the crash.  There are a lot of simple typos or mistakes a human can make that put the math off for just that value, for example replacing > with =. 
Logged

wierd

  • Bay Watcher
  • I like to eat small children.
    • View Profile
Re: Advanced World Gen "finalizing sites..." crash
« Reply #13 on: February 02, 2018, 03:23:44 am »

The good old "off by one" error...

Logged