Bay 12 Games Forum

Please login or register.

Login with username, password and session length
Advanced search  
Pages: 1 2 3 [4] 5 6 ... 8

Author Topic: DFHack script: showbiomes  (Read 26975 times)

feelotraveller

  • Bay Watcher
  • (y-sqrt{|x|})^2+x^2=1
    • View Profile
Re: DFHack script: showbiomes
« Reply #45 on: October 04, 2016, 03:46:56 am »

Oh, sorry for being a bit obtuse.  The ones I have seen remind me of candy spires.  (Probably just the 16x16 thing at work.)

[Edit: I'll keep my eyes open for various anomalies.  Not sure that I will get through a great number of cases, though.

Is it worth including a link to this thread on the information page?  And getting it included in the default DFHack (and docs)?  Guess that's assuming nothing crops up over the next couple of days.]
« Last Edit: October 04, 2016, 04:00:25 am by feelotraveller »
Logged

PatrikLundell

  • Bay Watcher
    • View Profile
Re: DFHack script: showbiomes
« Reply #46 on: October 04, 2016, 04:02:27 am »

Ah, I see. All air biomes I've seen are uniform within a 16*16 block and often most or all of the air blocks have the same biome (which I think is the intended behavior), which is why the shear detection only probes one of those points for each Z level. I suspect the candy thingies also uses 16*16 blocks, while magma channels use the whole embark tile block (9*9 16*16 block).

I've tried to find something that identifies air tiles as being air (rather than just any empty space) as well as some logic to the air biome tropicality determination, but haven't found anything that fits with existing evidence (and that incorrect air division line is a good test subject, since it can be used to shoot down quite a few theories).
Logged

feelotraveller

  • Bay Watcher
  • (y-sqrt{|x|})^2+x^2=1
    • View Profile
Re: DFHack script: showbiomes
« Reply #47 on: October 04, 2016, 04:12:19 am »

The embark which really made me think that had 5 air shears and 4 of them swapped in and out of a 32x32 block for the 15 odd air levels including sometimes/often no shear being present in part or parts of the block.  The shear detected list was veerrrry long.   :)  Moving up and down z-levels was like looking at candy spires with reveal turned on with added bonus of colour/texture variation.  The 5th shear was steady all the way up in the opposite 32x32 corner (2 embark tiles diagonal), but even then it just looked like a dream spire, to me...  Unfortunately I threw the embark/worldgen away as I was iterating my worldgen at the time.

[Edit: the swapping shears were in the top left corner of the embark (3x3) and the non-swapping one in the bottom right.]
« Last Edit: October 04, 2016, 04:19:23 am by feelotraveller »
Logged

PatrikLundell

  • Bay Watcher
    • View Profile
Re: DFHack script: showbiomes
« Reply #48 on: October 04, 2016, 05:19:07 am »

I'm not sure any potential additional users would get much out of a reference to this thread. As it currently stands, they'd likely gotten the script from here in the first place, and I'm not sure the discussion would be very enlightening (although possibly amusing or embarrassing depending on your sense of humor, to follow the fumbling about).
When it comes to official DFHack inclusion I don't know what kind of process they use to add things, and the script should stabilize first regardless. If the script is considered to be of sufficient use I wouldn't oppose it, though.

I agree shearing detection seems to be more common in the top left tile than in others, but they do occur elsewhere as well.
Logged

feelotraveller

  • Bay Watcher
  • (y-sqrt{|x|})^2+x^2=1
    • View Profile
Re: DFHack script: showbiomes
« Reply #49 on: October 04, 2016, 06:23:03 am »

Yeah, I guess I was thinking two steps ahead with the link; if it gets included in DFHack it would be good to have a place for reporting oddities.  (And more I was just dreaming, you know, about those hordes of testers who would supply the data to refine the script... me wants to be lazy and get back to coddling beardlings... you too no doubt.  :D)  In the meantime I notice you don't have a signature... go on you know you want to.

Banter aside I wanted to post another case of biome division (I think?) to see if that helps.  It goes all the way to the roof, is not reported as a shear, is a tropical/temperate division and at a guess has one of the ground level biomes involved.  Should rule out latitude at least if it is the same phenomenon.  The embark also exhibits a shear not detected by the shear detector but shown in the DFHack window as 'shear only'.  And yes it exists graphically in showbiomes.  I have not done any farm plot testing...

Spoiler (click to show/hide)

Beyond that a couple of points I have noted: tropical rivers (and rightly so by creature populations) can, and do, exist in entirely temperate embarks; and more relevant to the script I have seen murky pools crossing the (underlying and coloured) boundaries of biomes (so the murky pool straddles two biomes) - is this right?  What happens if the biomes are one tropcial, one temperate (have not seen this yet but got me thinking).  Okay more hot air but if I say enough something I say will turn out to be useful.  ::)
« Last Edit: October 04, 2016, 06:31:01 am by feelotraveller »
Logged

PatrikLundell

  • Bay Watcher
    • View Profile
Re: DFHack script: showbiomes
« Reply #50 on: October 04, 2016, 06:57:35 am »

I'd rather do without signatures, images, etc (but don't mind if others do otherwise).

Shear detection is not fool proof, as it detects only the ones where an out of bounds offset happens to appear on the tile probed. I'm probably going to do away with shear detection completely, and go for biomes only. Non ground biomes can and do appear without the offsets being wrong.

I'll look at the tropical/temperate line. Although I doubt it will help me find the rules used for the air, it might provide another test case to squash incorrect theories with.

Rivers and pools are not actually biomes as such, but appear as tile features within their home biome. I currently operate according to the assumption they use the lake logic for determining tropicality and the global map tile pointed at by the home biome as the reference location for that determination. This means they should be able to change tropicality on biome boundaries, although I haven't seen it (yet). I suspect it would be a bit difficult to verify or refute the tropicality determination of a split river, though, as you'd need to observe tropicality dependent fish in both parts. It can be noted that ragundo's code doesn't cover rivers and pools, as these are on the wrong level, so I'm operating on what I think is a reasonable assumption. Furthermore, the tropicality determination of lakes is different from most (or possibly all) other biomes, so I'd expect to see tropical rivers split in temperate biomes, as I think lakes are tropical further from the equator than most other biomes.
Anyway, an embark with a split river (either actually split but displayed as the same throughout or the reverse) would be useful, as it would show the assumption was wrong.
And your thinking and questions are quite valid. In addition to that, you sometimes realize things when you try to explain them to others, as you need to think them through to formulate an answer.

Edit: I generated the world, but it didn't turn out exactly the same. However, the embark location looked the same overall (same brook curve and overall biomes) but different savagery/evilness. The temperate/tropical savanna line is present in the sky though, but at a different location. Farm plotting shows both sides are actually temperate (again). Something that's probably significant is that the temperate/tropical lines follow embark tile boundaries, not the smaller 16*16 tile ones.
« Last Edit: October 04, 2016, 08:42:50 am by PatrikLundell »
Logged

Fleeting Frames

  • Bay Watcher
  • Spooky cart at distance
    • View Profile
Re: DFHack script: showbiomes
« Reply #51 on: October 04, 2016, 08:49:57 am »

I need more error cases to investigate to get further.
Here's one embark DL [1,7 MB] that reports joyous tropical grassland on the west third of the first z in air in 1.9(located like last one here, where there actually are all tropical swamps (which should be all untamed in this world).

2z above embark, from north to south west edge, the discrepancy is reported two tropical grasslands and tropical swamps VS plotted tropical grassland, temperate grassland and swamp (bit odd, since I'd expect temperate to be further north than tropical.)

@feelotraveller: Geese, the rivers are not quite same as lakes. (though yeah I expect they'd be determined same way).

feelotraveller

  • Bay Watcher
  • (y-sqrt{|x|})^2+x^2=1
    • View Profile
Re: DFHack script: showbiomes
« Reply #52 on: October 04, 2016, 08:58:37 am »

To be honest, whatever murky pools do it will not practically amount to much.  Rivers, yes, I'll take a look to see if what I can find.

The shears and air biomes are more of a curiosity to me; although I can see how they could potentially be useful in certain circumstances.  There are an awful lot of shears detected in (or near) the top left corner.  Makes me think something strange is going on, either with the script, its upstream sources, or df itself.  Would the top left corner happen to be a 0,0 reference or the first element in a matrix (feelotraveller gestures vaguely at poorly understood things)?

So far as the script accurately charts the ground biomes I'm a happy embarker.  Unfortunately I don't have the patience to test a bunch of sites in detail but it so far nothing in 1.9 looks off in this regard.

@Fleeting Frames
Nitpicking much?
Logged

PatrikLundell

  • Bay Watcher
    • View Profile
Re: DFHack script: showbiomes
« Reply #53 on: October 04, 2016, 09:50:38 am »

Given the pay you get for testing the script, I get great bang for the bucks ;)

The preference for off biomes to appear in the top left corner is odd. However, the boundary effects ought to be based on embark tile boundaries coinciding with world map boundaries (i.e. at indices 0 and 15 both vertically and horizontally. However, since we have no real idea of what these thingies are and why they're there, we can't really draw much of any conclusions.

The air biomes can be exploited in at least two ways:
- Access to plants not otherwise available.
- Fleeting Frames has shown that you can likewise get access to creatures not otherwise available by providing edge access for a floor in that biome (I think it was water buffalos wandering into an air swamp biome.

Earlier experiments I've made indicate you won't get herbs above ground level, for unknown reasons, even if you provide soil (I cast obsidian and shaved off one level to get soil). If you use DFHackery to provide soil both at the biome and the level below the area should support trees, but given that herbs won't grow on legally generated soil above ground level (but did on a soil covered floor at the ground level), I think you won't get trees either without DFHackery. At the time of those experiments I thought the air was a uniform biome. I'd certainly would like to know what causes air biomes to support farm plots but not wild growths beyond 'grass' (if nothing else, that might help distinguish an air biome from a ground one).

You may note I've moved away from the shear terminology and instead use "air biome". The reason is that the air tends to have a biome different from the ground even if there's no shear present.
Logged

feelotraveller

  • Bay Watcher
  • (y-sqrt{|x|})^2+x^2=1
    • View Profile
Re: DFHack script: showbiomes
« Reply #54 on: October 04, 2016, 11:10:13 am »

I reckon we get better value for money with the scripting you do.   8)

Yes, farming and animals, but practically only at a later stage of fortress development, and probably only if they are different from those accessible easily on other levels.  (If you are a dedicated windy like myself there might also be access to wind unavailable at ground level, which is what drew me to Probe long before I mentioned it to Fleeting Frames.)  The reward has to be worth the effort, after all.   :)  And yes I'm sure there are exceptions even here but for me, practically, it will never happen since I tend abandon 90+% of my forts (for reasons) soon after the first caravan.  And in the few which make it longer I'm usually pretty determined with pushing on with whatever project/feature drew me to that fort anyways.

Back to the script: Rivers.  Okay I got bored part the way through but maybe someone has additional tools/patience that they can bring to bear, or I'll do some more in a couple of days time.  I found this which shows the boundary as showbiomes currently displays it:



Unfortunately it does not accurately capture the boundary - although it does get pretty close.  There were only tropical (aquatic) creatures in that river according to the (u)nits list.  Here's a screenshot of the embark location (from the last worldgen I posted):



Embarking 3 embark tiles south (i.,e, bottom of that embark zone) also results in tropical but no temperate aquatic life while the river is displayed as entirely temperate.  (I'm leaving out the creatures which appear in both lists in all cases.)  But going south one region tile and embarking at top of that embark zone gives temperate aquatic life.  So my first take was that the boundary is 5 embark tiles too close to the tropics.

However I thought to check with the next river over.  So I tried embarking 7 region tiles west and one south at the top of the embark zone with river in the centre tile.  All tropical creatures: so things are not so simple.  I then embarked 3 embark tiles south, where the river splits in two and got all temperate aqautic life.  Then came a bunch of inconclusive tests where I embarked multiple times in between the two latter embarks trying to find the exact line, going a bit east or west while still including the river for variation.   Then I tried to replicate the initial tests from this region tile by embarking at the top, then 3 tiles down and all tests were inconclusive generating only creatures which inhabit both biomes, such as carp.  (The real frustrating part was being unable to verify, or alternatively contradict, the initial tests in case there is some random element involved.)

So my very provisional tentative understandings (aka needs more testing).  Tropical rivers extend slightly more into the temperate zone than showbiomes currently portrays, approximately 1/3 to 1/2 an embark tile.  Suggested but needs more data points to rule randomness or contribution of other factors such as temperature etc.  However currently it seems as if this boundary is not a straight one but wanders somewhat, for whatever reason.  Tentatively again, but no cases of temperate and tropical aquatic life sharing the same (on embark map) river and a decent amount of cases (presumably where overlap of biomes occur) where neither specific biome types are present suggests the inability to get both tropical specific and temperate specific creatures in the same embark contained river - furthermore that mixed rivers will only ever contain creatures common to both temperate and tropical versions.  Unfortunately both latter hypothesises are ones that are easy to prove wrong but can only have weight added by (exhaustive...) further testing.

Futurenotes: my testing was with 3x3 embarks but different elements could be tested better with smaller (does 1x1 still need a mod?) embarks - finding the dividing line, or larger embarks (16x16 possible?) for investigating what happens with rivers that contain overlapping biomes.

Logged

PatrikLundell

  • Bay Watcher
    • View Profile
Re: DFHack script: showbiomes
« Reply #55 on: October 04, 2016, 02:30:34 pm »

@Fleeting Frames: Hm, this is mighty interesting. Since you didn't specify the embark size, I embarked with a 3*3 and and then a 1*1, and neither matched your description. I then embarked with a new 3*3 (since 1*1 was the second one), and it didn't match either, and, furthermore, didn't match my recollection of what the first attempt looked like, so I carefully noted down what they were, and made yet another embark, and this time it definitely doesn't match the notes. In all cases the air background biome is reported as Tropical Freshwater Swamp (Untamed Wilds), but the anomalies appear in different places and are of different biomes, although all of them ones present at the ground level. When I say I embarked again I mean I scrubbed the folder and extracted the download again to reembark.

The anomalies being random further reinforces the idea that they are bugs, and it makes it rather difficult to replicate problems without actual saves.
The only good thing here is that this attempt displays 3 anomalies at the level above the embark, 2 of them tropical grassland and one of them temperate grassland. Plotting them shows the temperate grassland and one of the tropical ones as tropical, while the other tropical grassland comes up as swamp, which begs for more investigation.
My current assumption is that tropical/temperate determination for air biomes is unreliable, so I'm not surprised about that. The swamp thing, though, is odd. I expect I have somehow screwed up.

I'll get on to the river thing as well. Tropicality determination is based on a combination of latitude and parameters. Currently everything that's >= 200 (when remapped to a single North pole 259 height world) is tropical, while there's a zone from 171 -199 that may be determined as tropical depending on parameters. These parameters vary per biome, but include such things as rainfall, vegetation (I haven't seen any case where these have had different values), and latitude (where higher latitude values allows for wetter biomes to be classified a tropical. Latitude is determined in world map tiles, and each 2 m embark tile ultimately (it's a two step process) points to its own world map tile or one of the 8 neighbors. Thus an embark may have tiles referring to the latitude of its own latitude or the world map tiles above or below them.
For zero pole worlds the latitude boundaries are 85 and 75 degrees instead. Ah, yes, in some cases there's a check for world size, so only the two larger sizes can actually match the "maybe tropical" case (tropical dry broadleaf...).
Logged

Fleeting Frames

  • Bay Watcher
  • Spooky cart at distance
    • View Profile
Re: DFHack script: showbiomes
« Reply #56 on: October 04, 2016, 02:42:21 pm »

Earlier experiments I've made indicate you won't get herbs above ground level, for unknown reasons, even if you provide soil (I cast obsidian and shaved off one level to get soil). If you use DFHackery to provide soil both at the biome and the level below the area should support trees, but given that herbs won't grow on legally generated soil above ground level (but did on a soil covered floor at the ground level), I think you won't get trees either without DFHackery. At the time of those experiments I thought the air was a uniform biome. I'd certainly would like to know what causes air biomes to support farm plots but not wild growths beyond 'grass' (if nothing else, that might help distinguish an air biome from a ground one).
Well, turning back on the original topic it could be that it is not considered traversable floor on embark - but this is about the only idea I have....

An idea I can test *tests*

Also an inherent demonstration of air biome being good when ground under it is evil (reported joyous temperate grass but is good tropical grass)

Though I suppose obsidian casting above ground level is still good to get access to clay or sand on a fortress whose soil layers don't have either.

@feelotraveller: Any excuse for pun ^^



Hm, but I did specify the 1x1 embark. Image link from first page

But different air biomes on same embarks of same location is very interesting!

Also, btw, recall me getting cucumber from over 1 regional map tile away? I'm not sure 9 biomes is the limit.
« Last Edit: October 04, 2016, 02:48:21 pm by Fleeting Frames »
Logged

PatrikLundell

  • Bay Watcher
    • View Profile
Re: DFHack script: showbiomes
« Reply #57 on: October 04, 2016, 03:51:19 pm »

I tried more farm plots on my reported Grassland, and they all indicate Swamp. I guess I have to cover the whole tile and join it up with the one diagonally down which have matching plotting and reporting.

I took a break and looked at feelotraveler's river instead (with a 6*3 embark). All of it was reported as temperate except a little piece in the far north, following a biome boundary. I then tried a slight change of the logic: instead of having pools and rivers use whatever latitude reference the biomes they're located on/in, I used the local (i.e. embark) world tile instead, and it all turned to tropical (possibly just shifting problems elsewhere, of course). However, that results in a prediction that can be falsified by observation: an embark would never be capable of hosting both tropical and temperate river and pool critters simultaneously (ocean is a completely different thing: if there are distinct arctic, tropical, and temperate ocean creatures you should be able to get both on a whacky embark).

The 9 biome limit is valid only if DF plays by ragundo's rules (and those I've found and later found in DFHack code). A tile references an embark tile, and embark tiles reference world tiles. However, individual 2 m tiles have individual biome parameters that appear to all be identical within a biome blob's boundary. However, there is nothing to enforce that, so instead of having a region id that points to a grassland biome and have grassland parameters, it's possible to have e.g. desert parameters and a desert region_id, which is what the air only biomes do. I've actually wondered what would happen if you hacked that: would it change e.g. farm crops available, or was that determination made earlier? Would an old farm plot keep its crop set, or would it change?

Interesting that a reclaim would enable herb growth, but it might somehow be tied to the path pre calculation that I think is made anew on a reclaim (and possibly also on a resume), as speculated. And another case of temperate/tropical determination failing in the air...

I think obsidian casting results can be determined ahead of time by using Probe, outside of bugged areas, of course.

Embark size: Yes, I saw the embark was 1*1 originally, but I didn't know if it still applied.

Anyway, I've updated to version 1.10. The changes are mostly cosmetic:
- Tropicality determination of pools and rivers using the embark tile rather than referenced tile of host biome (real change)
- Moved the biome list from the DFHack window to the help window.
- Reworked the help text (including feelotraveler's suggested thread link)
- Removed the shear indicator as almost every embark has an air base biome that's not present on the surface.
Logged

feelotraveller

  • Bay Watcher
  • (y-sqrt{|x|})^2+x^2=1
    • View Profile
Re: DFHack script: showbiomes
« Reply #58 on: October 04, 2016, 04:30:56 pm »

Quick report: have an embark with 2 biomes both forested joyous wilds: one temperate broadleaf, one tropical moist broadleaf.  Shows correctly in game window but the DFHack report has them both as tropical moist broadleaf (in cyan).  Can probably supply screenie/save later if needed but guessing it may well just be a typo sort of thing.

Will have another close look at rivers in a couple of days, under the new setup.
Logged

Fleeting Frames

  • Bay Watcher
  • Spooky cart at distance
    • View Profile
Re: DFHack script: showbiomes
« Reply #59 on: October 04, 2016, 10:03:30 pm »

Quote
Interesting that a reclaim would enable herb growth, but it might somehow be tied to the path pre calculation that I think is made anew on a reclaim (and possibly also on a resume), as speculated. And another case of temperate/tropical determination failing in the air...
Interesting, yeah. Decided to test saving and unsaving and then unretire too, with just digging and no tiletype messing this time, because last time reclaim turned to out to be overkill for animals.

But no shrubs. Not even after doing a dirt road onto grass after unretire and then leaving DF running for few years thanks to a fruit gathering zone automating food :P

Then I put soil walls underneath with tiletypes, without any save or restart or unretire.

Shrubs popped up very quickly, also got saplings.

Conclusion: It is not necessarily (just) the reclaim, it is having soil wall underneath for even shrubs.

Also tested another theory I had from long ago for caving in natural obsidian wall above the embark, whether it'd transmute it into soil wall. It didn't.

Shame, I hoped air biomes to be even more useful. Though they're already not that far off from land biomes.
Pages: 1 2 3 [4] 5 6 ... 8