Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[PatrikLundell]

@feelotraveller: Incorrect surface biome determination is definitely of interest, and shouldn't be a typo, so I'd like to take a look at the save (if the save is after embark a screenshot isn't needed as the save itself should contain all that's needed). Incorrect air biome temperate/tropical determination is currently not of interest, as it's know it happens, but I don't have any idea of how to correct it.

@Fleeting Frames: My understanding of cave-in transmutation is that that only happens for soil caving in, not rock, but I did consider the same experiment.
When I did my experiment with obsidian soil creation I cast obsidian both on top of my recessed courtyard (i.e. on the roof replacing the former ground), as well as two level high obsidian blocks. in all of those cases I channeled away the top level of obsidian, resulting in a soil level both on top of my roof and on top of an obsidian platform (the exception was a two tile high pillar on the ocean shore, where I did get the rock type the shore was made of rather than soil). Grass grew on all the soil surfaces, but herbs only appeared on the roof (very sparse, about the same as you get on the surface when digging out the first soil level.
Thus, it seems surface herbs are dependent on the layer below being a soil type (although it can make do with the soil being dug away, but poorly), just as you say.
The only reason I'd want to get herbs from an air biome is to get at their seeds for my farm plots, but you can still hope for the human caravans to provide that. Fruit trees are, of course, another resource you can't get in any non DFHack way, and it's a pity air biomes won't support them without DFHackery (and if you resort to that you can just plunk down a sapling on the regular surface).

[Roses]

I have been following this thread for a bit and just had a couple questions.

So if I am understanding the purpose correctly you run the script post embark when you are at your fortress site and it tells you the biomes of your site correct? Operating under that assumption my questions are;

1. What are the current issues that are unresolved when determining biome information?
2. Are you reading each map tile (or block) directly or accessing that information from somewhere else? (What I mean by this is could you feasibly get the biome information for the entire world map, or are you limited to just the fortress map?)
3. I know I had more questions but they escape me at the moment, I am sure I will remember them later...

[Warmist]

<...>

huh i tried that for a looong time and nothing spawned on soil + soil wall. Maybe my biome was strange or sth. Or soil was bad type. Was really hoping for a hanging gardens :<

[PatrikLundell]

@Roses: Happy to hear there's someone else interested!
1. The logic for determining biomes provided by ragundo seems to be working out in this context as well, with a number of caveats:
    a. The logic doesn't cover rivers and pools, as those aren't actually biomes. It's still unknown if the latest tropicality determination approach is correct for those.
    b. The air above an embark is often of a biome different from those present on the ground. The tropicality determination logic for those is still unknown, as applying ragundo's logic result in tropical/temperate lines where there aren't any according to the farm plot biome determination method. It doesn't help that I don't have any good way to determine if a biome is an air one or a ground one, so I still wouldn't know when to apply the "other" logic...
    c. In addition to the regular air biome above an embark there are often strange 16*16 blocks of other biomes "randomly" appearing at various levels. These may not be the same and may not appear in the same location even when embarking at exactly the same location from copies of the same world generation. They are problematic: Not only do they suffer from the tropicality determination issue the "normal" air biome has, but I have at least one case where farm plotting provides a different result from the biome determination logic (this is the case I'm looking at now). It seems to be correct most of the time, though, so I don't understand where things sometimes can go wrong but not always.
    d. If any surface biomes are incorrectly determined I'm interested in investigating why, since I currently think the correct logic is in place (and hence, failures to produce the correct result should be bugs in the implementation rather than the underlying logic).

2. I'm essentially using the df.global.world data structure, sometimes directly and sometimes via DFHack provided operations, but they're "just" a more convenient way to access the same data. Much of the access logic has been copied from Probe. The detailed 2m tile information is present only for the embark (and, I think, previous embarks), but is not generated for the rest of the world, and so is not available there. The information on the region level (the 16*16 tiles within each world map tile) is available at all time, and that's the info presented in the pre embark situation. My current understanding is that the biome properties themselves are static and tied to world map tiles. Each world map tile biome can then bleed into its region tiles, as well as into the 8 neighboring world map tiles (and their region tiles), resulting in a structure where each region tile has a "main" biome reference to the world map tile that is the biome source. When an embark is generated, the process is repeated at the level below with each individual 2 m tile referencing one of 9 region tiles, that then in turn referece one of 9 world map tiles. A caveat is that my current understanding might not be correct...
I suspect DF uses the same logic for both adventure mode and fortress mode embark level info. If that's the case it might be possible to create a script/plugin that caused the focus to move over the entire world in adventure mode and collect the data while the focus is in each tile (I think adventure mode then discards this data when the area is departed). All of that is pure speculation, though.

3. You know where this thread is should you manage to recapture your questions...

@Fleeting Frames: As per the rambling above, it shouldn't be possible to have more than 9 biomes in an embark without breaking the logic. You'd have to somehow manage to reference a world map tile that's further away than one step (which is what Probe does on bugged offset data). My speculations further above about hacking the biome are thus incorrect. You might, however, be able to hack a tile to refer to any of the 9 reachable biomes (still with unknown effects, of course).

Edit: Forget the garbage struck out. Each 16*16 block within the embark tile has a reference to a world map coordinate where it resides, but there is nothing, I think, for it to refer to any world map tile.

Edit2: I think I've understood why the main "air" biome frequently differs from the surface ones. The normal air blocks contain a uniform offset value of 0. However, that offset means "the biome belonging to the world map tile to the NW of the one I (i.e. the block) am referencing (which should be the world map coordinate the block resides in)", which is likely a bug in DF (the value should probably be 4 = no shifting away). Similarly, the recovery code when that offset is out of bounds sets the reference to the current tile, not the one to the NW (the DFHack C code leaves the source coordinate unchanged rather than shifting the X and Y indices). The case I'm grappling with at the moment is an out of bounds case that should result in the main biome of the embark world map tile, but nevertheless ends up what's probably the biome to the NW (there's no way to check except farm plotting).

Edit 3: After some additional though, I think you can theoretically have 18 biomes in an embark, although I doubt there's any combination that can achieve it. 9 biomes can reside in the embark world tile and the eight surrounding it, but tropicality is not part of the biome itself, so in principle both versions could manifest themselves if the parameters controlling tropicality are fine tuned with the embark.

[Fleeting Frames]

On the hanging gardens: That gave me an idea, and it is weird: I kept getting shrubs after digging away the soil layer underneath done by tiletypes.

Hm, which gave me a hypothesis to falsify: It is not as much the lack of soil layer but the presence of stone layer underneath.

*builds a floor platform*

*casts obsidian on top*

*digs away*

*retires*

*unretires*

No~pe. Hypothesis falsified.

Also, further report of airshears not getting biomes correct - report of all being joyous and untamed, then get a message of "a cloud of evil dust has drifted nearby" on highest z-level's SE shear. Good thing didn't eat my embark team, can provide save before retire if desired. Got ones from NE and NW corners on said z-level too after unretire.

Finally, I've certainly gotten tropical conifer shears from SE, so either I misunderstood your explanation or there's more to it. On the flip side for 9 biomes max argument, I've failed to get any shears from 3+ regional tiles away for reasonably sure (34 embark tests, though that is far less than required for 5 biomes in 4 squares or less), though.

[feelotraveller]

Incorrect surface biome determination is definitely of interest, and shouldn't be a typo, so I'd like to take a look at the save (if the save is after embark a screenshot isn't needed as the save itself should contain all that's needed).

I have another case of it from today.  But let me be clear, in case I was not yesterday.  showbiomes in the main window correctly identifies the biome.  However the report in the DFHack terminal is incorrect.  In the case I have just come from the biome is tropical savannah, clear S on the map but the terminal output is temperate savannah.  That is why I thought it might be a typo/the terminal script calling the wrong string.  I can provide a save tomorrow, easily of this second case, (and can dig out the one from the original report probably but with some effort) but thought you might check first, since it seems clear to me that the problem is not with the original script logic but with the output written to the terminal.

[PatrikLundell]

Yes please, I'd like to look at that failed identification as well.

I think you misunderstood my explanation, if I understand your description correctly. The NW reference is for the background air biome. Shears within that background will usually match the surface one (possibly with opposite tropicality) as a result of error handling of bugged offsets. However, if the offset is wrong but within range, it should still be able to refer to any direction.

The lookup for biome data is:
- block: Data element describing a 16*16 block of 2m2 tiles at a given Z level.
- des = block.designation [tileX][tileY], where tileX and tileY are the 0-15 X and Y indices within that 16*16 block.
- offset = block.region_offset [des.biome], where des.biome contains the 0-8 values shown in the biome offset view. Offset SHOULD be in the range 0-9 and map to the different directions, and is used as offsets to the world map coordinates of the block (which should refer to the embark world map block, unless bugged, and I've yet to see it being incorrect). The result is a world map coordinate reference which should be embark world map coordinates +/- at most 1 in both the X and Y coordinates, i.e. the world map coordinates of the embark and the 8 surrounding tiles.
- biome = df.global.world.world_data.region_map [x_adjusted][y_adjusted]. Biome contains the biome parameters such as temperature, salinity, etc, including savagery and evilness.
This logic is taken from Probe and matches other existing code. The overflow handing that Probe lacks is taken from other existing code.

In addition to that, there's also a further process to set the tropicality of the biome where one of the surrounding embark level tiles (or itself) is referenced, and that embark level tile then references a world map tile whose location provides the latitude for the tropicality determination.
This logic is mostly based on ragundo's code, but the first step of going via a neighboring embark tile is mine, and may thus be incorrect.

The mess above makes my head hurt...

How and why shears occur is currently unknown, so saying what it can and can not reference cannot really be done. The logic above allows for any of the surrounding wold map tiles, but the biome bleeding that's assumed to generate the biomes firstly at the embark tile level and then at the fortress tile level (2 m2 tile) should require the biome to exist as the main biome at a neighboring embark tile. It can be noted that this bleeding logic is my idea of how it seems to work at the surface, so it can certainly be wrong.

Anyway, I finally managed to figure out what __OFSUB__ mess was supposed to do and matched that with the macro I've found on the net. It turned out my guess on what the code actually did in the end was correct, i.e. a straight check vs 200. The __OFSUB__ stuff is there to handle cases that never appear in this code, namely a subtraction becoming an addition as a result of the terms having opposite signs, and then if that addition results in a register overflow and a wrap around (as in the name OverFlow on SUBtraction...).

@feelotraveler: Unfortunately, the biome summary list can be incorrect because the info collected is not fully sufficient as tropicality is missing. There are two sources of error I've identified:
1. A biome can appear on the embark in both its tropical and its temperate form, but the collection only notes the biome object, not the complete biome type determination.
2. Since I don't know the logic for air biomes they can be wrong. Also, if I was to enhance data collection for tropicality, air biomes being the first ones to be detected can mess things up tropicality wise.
Thus, I think this issue is not of interest, as it's known, but I don't yet know how to handle it. It doesn't help that I have no sure way of identifying an air block tile as being air rather than "ground".

The terminal output string generation cannot match the wrong string, because it uses the provided language operation to generate the string from the enumeration type value itself. I can easily stuff things up on the help page and the display pages, but this uses a raw script language functionality.

[feelotraveller]

Ah, okay... I was not aware of the 'provisionality' of the terminal output.

Would there be a way for remove the reference to tropicality from the terminal output to avoid ongoing confusion?

[Fleeting Frames]

Here's the evil airshear DL shown as joyous: 2,1 MB (I'm amused that they're so small it'd be not unreasonable to share them as 64-bit encoded text string.

And yeah, that explanation made sense, my bad (got it wrong initially) ^^;

[feelotraveller]

Think I might have spotted an edge case...

Embark has tropical savannah that df reports (embark screen) as Untamed Wilds.  Showbiomes reports savagery as 66 in DFHack window.  Shown as Light Green, for Wilderness, in main window.  Can supply save if needed.

(This was a reembark after a quick test and I noted that although there were several air shears over several levels in the previous embark there were none present this time around only the 'base' air biome.  I believe you are already aware of this phenomenon but first time I had seen it.)

[PatrikLundell]

@feelotraveller: I'm trying to get biome identifications to be correct, rather than to reduce detail to hide the uncertainty, but that might be a fallback option if I fail.

Incorrect savagery: Interesting find. The script uses the logic 0-33, 34-66, 67-100 for the grouping of savagery and evilness. When trying to look that up on the wiki I fail to get any numbers for the groups. Does anyone have a reference to a specification of where the boundaries lie?
There's one check I'd like to see performed on that save, either by you or by me if you provide the save, and that's to check whether there are whip vines or highwood (including saplings) within that biome, since those are present in savage biomes only. If they are present it's a clear indication the biome is in fact savage, while an absence is inconclusive. Farm plotting can't be used since plots can grow suitable crops regardless of the local evil/savage rating.

If "reembark" means embarking afresh in a copy of the world the shear mutability is known. If it's a reclaim or resume control that would be new info, though, as I have no info on those cases.

@Fleeting Frames: Thanks for the save, I'll look at it to see if some nuggets of wisdom can be gleaned from it. Uuencoding a few million characters is a bit much, though, and wouldn't fit in a post.

Getting confused by my walls of text is probably fairly easy

Anyway, I've been bitching about inability to determine where a location is an air biome or not. However, the info is already displayed by the biome offset view, so it's a matter of checking whether the 256 offsets in a tile are all zero or not. Since doing that individual for each 2 m tile would likely make the script even slower, I need to reorganize the code to check blocks only once, rather than 256 times.

Edit: The save examination provided a lot of interesting info. I've farm plotted the save on all level except the top (which can't be reached), and there's a fairly poor match between what we get and what the script produces. What I've seen indicates the following:
- I've yet to see the base air biome being wrong. Thus, I'll assume that is correct when there are no anomalous offsets for the time being.
- All the biomes present are present in the surrounding (or current) tile (to the degree a match can be established). There's terrifying dry broadleaf in several surrounding tiles, and there are a fair number of dry broadleaf tiles in the air. This biome is not reported for any of the 16*16 tiles visible at pre embark for the area, and so must have jumped in, somehow. One interesting case is the detection of a temperate savanna tile in the air, while the surrounding tiles contain only tropical, and the embark 16*16 none at all. Thus, it has jumped in and changed tropicality. My assumption for the time being is that anomalies can take on any of the 9 nearby world map biomes with a possible tropicality change. It's also evident the out of bounds recovery mechanism is flawed: it likely seemed to work simply because the fallback is a likely outcome from the one of the nine selection, in particular since its basic type may be present at multiple world map coordinates. As a result, I'll revert back to flagging anomalies as unknown, and will try provide a list of what they can be instead.
- It's still unknown if non zero offsets while still within bounds actually correspond to the biome usage of the offset would yield. I'll assume that for the time being, but it needs to be tested further. However, if all air biomes are shown as either the air base one, a within bounds one, or unknown, it should be reasonably easy to target the within bounds ones for a check.

[feelotraveller]

No highwood or whip vine on the embark.  (even though there is a 100 savagery biome also present...)

It was an embark on a fresh copy of the world.

Quick search reveals nothing on internet. 

Okay, lets test- opens up worldgen.txt and changes to
[SAVAGERY:66:66:51:51] = endless rejections (actually stopped around 50... but that's enough I reckon)
[SAVAGERY:65:65:51:51] = world with all wilderness (had good/evil zeroed)
Right, well while we are here 
[SAVAGERY:33:33:51:51] = all wilderness
[SAVAGERY:32:32:51:51] = all calm

tl;dr
cutoffs are 0-32, 33-65, 66-100
feel free to confirm

[feelotraveller]

Okay River tests look good.  (Spent a few hours...)     Yay, for Patrik.

Every embark had a river that was either all temperate or all tropical.  No aquatic life from the wrong biome was observed.  Predominantly tested the final embark tile of temperate and first embark tile of tropical.

Over several different worlds of different sizes (and poles) I was unable to get a river that was a mix of temperate and tropical on the same region tile.  (=the boundary was always between region tiles)  So the problem of mixed rivers does not arise since every embark is contained by the region tile.  If anyone has a world (don't know if certain no poles worlds might produce this, for instance) that this occurs on I will take a look.

[PatrikLundell]

Thanks for all the dedicated testing, feelotraveller!

I've been restructuring the script and trying to sort out why it doesn't work afterwards, but think it's now back together, finally. I've changed the savagery and evilness boundaries to those measured (and assume both use the same boundary).
In addition to that, I've dumped the attempt to determine biomes when the offsets are out of bounds, instead printing red "@" signs (resulting in red dwarves with Phoebus and no TwbT). The symbol and color can be changed, of course, but '*' is already used for glacier.
Since the restructuring includes detection of air blocks I've made an attempt to get air biomes to be determined using the latitude of the map tile the biome hails from, as opposed to the standard which is to use the map tile it came from last (can bleed into another tile and then into the own tile from there). I've previously tried the embark's tile, but that resulted in the wrong tropicality in at least one case. This approach has been successful on at least one embark (Fleeting Frames' latest save: a useful one, that!), where the air contains a basic biome of tropical swamp and a few additional ones of tropical and temperate grassland (both of those present at ground level as well) which is encouraging (that embark has a dry broadlead one as well, but those tiles are out of bounds ones I can't identify code wise).
The next step will be to try to identify which biomes you might get as anomalous/shear biomes. That might be useful as an indication of what you can encounter, if it turns out to be correct.

Edit:
OK, I've pushed out version 1.11, which prints potential biomes as well. As you can see from the DFHack window output, you can get "duplicates", as their parameters can differ without their biome type changing.

Edit2:
I've pushed out version 1.12. The changes are that all biomes encountered are collected (the previous version would miss the second tropicality version of a biome in the list when present), the help page presentation of encountered and potential biomes are filtered down to one of each biome + savagery/evilness classification, and an updated help text.

Edit 3:
I've pushed out version 1.13. This version has added the functionality of panning the view.

Edit 4:
I've pushed out version 2.0. Reorganized the key help and fixed a display bug with shifting the view upwards.

[feelotraveller]

It is nice to be able to move around the screen. 

Haven't looked at 2.0 yet but one slight annoyance with 1.13 and earlier is that the zoom level is different from what I get in game.  This impacts when trying to look at a particular location of the map, say a murky pool or the confluence point of several biomes.  I've had to resort to tile counting several times...