Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[kiiranaux]

Tonight I did a deep dive into how SELECT really functions, and what I need to know to use it properly. I came to two conclusions:

• We need a tool that removes a tag/token that the game has already loaded from the vanilla raws. Something similar to how the [CV_REMOVE_TAG:] token works (e.g. [REMOVE_TAG:EDIBLE_COOKED] instead of cutting the entire dwarven wine segment out of the plump helmet raw, just to redefine it with a token edited out). Without this tool, mod incompatibility is going to swiftly become an issue. I've seen other games solve that issue by mod authors including something in their description that tells what is being changed so players know how to solve incompat problems. One little tool and we won't have to do that here!
• The SELECT tool requires understanding of the hierarchy of how raws are recognized by the game's code. However, since knowing this hierarchy has never before been important when modding previous versions, there's nothing on the wiki that explains or defines it. We should start treating that hierarchy as important enough to make it very clear on the wiki for v50. I'd start doing this myself, but I don't know wiki markup or what this hierarchy actually looks like. Usually you can infer it from indentation, but I've seen examples where that falls apart.

Re #2: Something critical I've learned while editing Highfantasy is that the game seems to prefer CUT_ tags to take place in a file named the same as the vanilla file hosting the object to be CUT. This is insane, and probably relates to "understanding of the hierarchy of how raws are recognized by the game's code." I was getting insidious list-shift errors until I made this change, for item_weapon specifically.

[kiiranaux]

Here's another sneaky one. With the change to initial trade caravans not using wagons, if a civilization uses a custom pack animal via ALWAYS_PACK, you must manually add a CAPACITY to the raws of that creature itself. Capacity defaults to null leading to empty caravans.

[Bihlbo]

Re #2: Something critical I've learned while editing Highfantasy is that the game seems to prefer CUT_ tags to take place in a file named the same as the vanilla file hosting the object to be CUT. This is insane, and probably relates to "understanding of the hierarchy of how raws are recognized by the game's code." I was getting insidious list-shift errors until I made this change, for item_weapon specifically.

Just to be clear, you're saying that if you have two files, one named item_weapon.txt and another item_weapon_nukes.txt, and both of them cut the same object to replace it with something else, the game chooses the replacement from item_weapon.txt over the other?
Just asking, because on my first reading I assumed something else, and to that I'd say that my file item_weapon_personal_tweaks.txt does a fine job of successfully cutting and replacing an object. I have a feeling you wouldn't disagree, because you probably have had the same experience. It would mystify me if you hadn't.
Or maybe you mean the first part of the filename needs to be the same? Like, I can't do CUT_ITEM_WEAPON:ITEM_WEAPON_WHIP in the file called backyard_bbq_recipes_by_grandpa.txt?

Here's another sneaky one. With the change to initial trade caravans not using wagons, if a civilization uses a custom pack animal via ALWAYS_PACK, you must manually add a CAPACITY to the raws of that creature itself. Capacity defaults to null leading to empty caravans.

You should update the wiki with that sneaky tidbit. Sounds valuable to know.

[kiiranaux]

Or maybe you mean the first part of the filename needs to be the same? Like, I can't do CUT_ITEM_WEAPON:ITEM_WEAPON_WHIP in the file called backyard_bbq_recipes_by_grandpa.txt? Just asking, because on my first reading I assumed something else, and to that I'd say that my file item_weapon_personal_tweaks.txt does a fine job of successfully cutting and replacing an object. I have a feeling you wouldn't disagree, because you probably have had the same experience. It would mystify me if you hadn't.

Actually, I do mean the "something else". If we're having different experiences there must be more variables in play ! :O Here's my situation.

Spoiler (click to show/hide)

Mod contains two files. Item_weapon is an edited copy of the vanilla file item_weapon. Every melee weapon is modified slightly in damage values, and, the items BOW CROSSBOW BLOWGUN are moved to a second file, Item_weapon_ranged. Item_weapon contains CUT for all vanilla melee weapons (before redefining them as new objects with the vanilla names again), and the syntax is correct. Item_weapon_ranged contains CUT for BOW CROSSBOW BLOWGUN (before redefining them as new objects with the vanilla names again), and the syntax is correct. This situation confused the game engine and causes list shift errors*

Deleting item_weapon_ranged and moving both CUT BOW CROSSBOW BLOWGUN and the redefinition of BOW CROSSBOW BLOWGUN back to item_weapon? No more list shift errors.

*An example of a list shift error weapons would be, for example, you're embarking, prepare carefully (same would happen on "play now"), and instead of a pickaxe you are given a 2 handed sword. Then you click digging implements and the list includes crossbows. So you click to add one of those crossbows and it puts a flail in your inventory. Etc.

List shift errors are nasty and are always caused by the game reading duplicate object entries. I've seen list-shift errors for creatures cause an entity to have its creature be "Acorn Flies" or "Pandas." I'm all but certain Kruggsmash ran into a list shift error during his video series playthrough of the mod Primal a couple years ago too.

You should update the wiki with that sneaky tidbit. Sounds valuable to know.

I should update the wiki with multiple bits from this thread. Here's another one that's absolutely blowing my mind.

The plant token [CLUSTERSIZE:#] has been changed internally. Now, a plant's cluster size must be greater than or equal to 1 + the number of instances of [STOCKPILE_PLANT_GROWTH] in its raws, or else a plant WILL NOT be gatherable by citizens.
For example, the item Cabbage has both its default state, and one instance of the tag [STOCKPILE_PLANT_GROWTH], 1+1=2. If CLUSTERSIZE is set to 1, citizens cannot gather cabbage (they will try and fail). If cluster size is set to 2 or high, citizens can gather cabbage.

[Bihlbo]

Actually, I do mean the "something else". If we're having different experiences there must be more variables in play ! :O Here's my situation.

Thanks for the explanation. Very interesting. Along with the mods I've downloaded, I'm running a mod that I just made for personal use. Every file for that mod is named the vanilla filename, with "_personal" appended to the end. No list shift errors like you describe, or at least I think that's the case. In my current game my dwarves like sleek boots and jagged caps, and some migrants arrived wearing silk low boots... maybe that's just an oddball culture or maybe that's an error, because I didn't add anything that would cause it.

[Shonai_Dweller]

Here's another sneaky one. With the change to initial trade caravans not using wagons, if a civilization uses a custom pack animal via ALWAYS_PACK, you must manually add a CAPACITY to the raws of that creature itself. Capacity defaults to null leading to empty caravans.

Thanks! Will need that. Can't do without the annual hobbit caravan and their alligator mounts. Dwarves need their unicorn cheese.

[Vanzetti]

I just have to say, based on this thread: I love how Dwarf Fortress is both extensively moddable, yes the modding format is so arcane that you people sound like you are researching an alien spaceship.

[Eric Blank]

Honestly, if it was a real scripting language like LUA I never would have gotten into it. I just don't have the head for actual coding, but the df raw format makes sense.

[D_E]

Out of curiosity, does anyone know if SLAIN_SPEECH is a creature-level token or a caste-level token?

[Bihlbo]

Here's the latest from Area 51. Still having a problem that looks like list shift errors (still don't know if that's what it is). I had one item file that added a couple of things, and also cut shoes to redefine them with a couple of additional tags (it doesn't work to simply select the item name and add the BARRED tag; I had to cut the whole thing and redefine it). In the game, everything was fine, so I saved it after they made some bone shoes. But after reloading the save, all the shoes showed up without an icon, and though the material stayed the same, they were all now "jagged" high boots. That tells me the game is pulling divine equipment adjectives and applying them to bone shoes, but it's also shifting shoes to high boots.

So, I changed the setup. I copied item_shoes.txt and pared the innards out to just the shoes I was editing and a CUT_ order. I made item_personal.txt go barefoot (no shoes in there now) and went in to test again with a new world. Same thing happened.

Time to test with a brand new mod. I'm going to see if this tictac can manipulate gravity. Maybe my mod is too complicated, and something somewhere else is causing the problem. New mod makes trousers craftable with bone. The item_pants.txt file selects pants and adds the BARRED tag (which says "this can be made of bone"). New world. Embark screen has no bone trou. Craftsdwarf workshop has no reaction for bone trou. Exit out, back to edit. Now the mod's item_pants.txt file cuts pants and adds them again, with the tag. No bone trou in embark, but in the craftsdwarf workshop, in the bone menu, I have the option to make bone trousers. Butcher the yak, make some pants (leather too, for comparison). Save to title menu, reload the save. Now the leather and the bone pants are jagged leather/bone greaves instead of mundane trousers. Glad it's consistent, but I wish I knew what I was doing wrong. At least I know it's not the same filename problem kiiranaux was having.

Create new world, this time with generation of divine materials turned off. Now bone trousers cannot be made, and on reload the fresh leather trousers did not change. So, contrary to how modding used to work and how I understood it to work, adding a tag to an item borks it. "I have improved my skill, this might be bliss."

Change to the test mod: it now adds a new item and doesn't touch existing trousers. PANTS2 is a paste of the trousers entry, but with the BARRED tag replacing LEATHER. New entity file makes PANTS2 common for dwarves. The trousers2 legwear is available in embark, but only in cloth (because it still has the SOFT token). Some dwarves are wearing a pair, in cloth. "Make bone trousers2" shows up in the bone category in crafts 'shop (without needing to make a custom reaction for it, mind you), and it takes 3 bones to make a pair. Made leather trou too, for comparison, but trousers2 aren't craftable in the leatherworker 'shop. Save to title screen, load the save, and the bone trousers2 have not been changed to a divine product.

Conclusions: You cannot add a tag to a clothing item by doing a SELECT_ITEM_X followed by NEW_TAG. You can CUT_ a clothing item and remake it with an additional tag, but on loading, the game gets very confused and thinks it is That Which Should Not Be, making all copies of the item you cut and redefine into a divine model, without any visuals attached. So, don't make a new clothing item with the same name as an item in the vanilla raws, even after you cut out that item.

Guesses: The game sees an item, and then it sees what materials can be used to make it, then fits it in to the standard crafting menus and makes the icon for the item match the material's color. But, if the game has to do this a second time when loading the game, and your version of the item doesn't match vanilla raw information for that item, then it treats your version of the item as though an angel made it.

[Putnam]

Tonight I did a deep dive into how SELECT really functions, and what I need to know to use it properly. I came to two conclusions:

• We need a tool that removes a tag/token that the game has already loaded from the vanilla raws. Something similar to how the [CV_REMOVE_TAG:] token works (e.g. [REMOVE_TAG:EDIBLE_COOKED] instead of cutting the entire dwarven wine segment out of the plump helmet raw, just to redefine it with a token edited out). Without this tool, mod incompatibility is going to swiftly become an issue. I've seen other games solve that issue by mod authors including something in their description that tells what is being changed so players know how to solve incompat problems. One little tool and we won't have to do that here!
• The SELECT tool requires understanding of the hierarchy of how raws are recognized by the game's code. However, since knowing this hierarchy has never before been important when modding previous versions, there's nothing on the wiki that explains or defines it. We should start treating that hierarchy as important enough to make it very clear on the wiki for v50. I'd start doing this myself, but I don't know wiki markup or what this hierarchy actually looks like. Usually you can infer it from indentation, but I've seen examples where that falls apart.

Re #2: Something critical I've learned while editing Highfantasy is that the game seems to prefer CUT_ tags to take place in a file named the same as the vanilla file hosting the object to be CUT. This is insane, and probably relates to "understanding of the hierarchy of how raws are recognized by the game's code." I was getting insidious list-shift errors until I made this change, for item_weapon specifically.

Absolutely not, what? I haven't had this problem or anything even resembling it. I suspect you're doing something really weird.

The hierarchy is new in v50 and is exactly as you see in-game: mods are loaded in order, top to bottom. You can change the load order on save load. There are mod info tokens allowing you to force certain mods to be loaded before or after others. Raws within mods are loaded in alphabetical order. Nothing more to it than that.

Also, the problems with bone trousers you're getting are explained on the wiki page:

Craftsmen can make this item from bones. Randomly generated outfits don't include bone armor.

The problems with reloading the world causing off-by-one errors or whatever is probably due to CUT_X being done in the wrong order and thus duplicated raws (what y'all are calling "list shifting", it's been a known thing for years lol) happening, which is odd. Perhaps you're putting your mod before the vanilla items in the load order, which would cause obvious problems.

[Bihlbo]

The hierarchy is new in v50 and is exactly as you see in-game: mods are loaded in order, top to bottom. You can change the load order on save load. There are mod info tokens allowing you to force certain mods to be loaded before or after others. Raws within mods are loaded in alphabetical order. Nothing more to it than that.

When I was referring to the hierarchy, I had my macro lens on. Not file hierarchy, but hierarchy of tokens within one entry in the raw. For instance:

Code: [Select]

[PLANT:MUSHROOM_HELMET_PLUMP] --- Top level
[NAME:plump helmet][NAME_PLURAL:plump helmets][ADJ:plump helmet] --- 1st level subordinate to the top level
[USE_MATERIAL_TEMPLATE:STRUCTURAL:STRUCTURAL_PLANT_TEMPLATE] --- probably also 1st level sub
[MATERIAL_VALUE:2] --- 2nd level, subordinate to 1st level
At least, I'm guessing that's true. If I wanted to change the MATERIAL_VALUE here, I'd need to know that it's "within" that material template. I'm just inferring that it is, since I can't find anything that explains the hierarchy. The indentation helps, but there are a lot of raws without any indentation, or with spacing that may or may not be indicative of something.

Also, the problems with bone trousers you're getting are explained on the wiki page:

Craftsmen can make this item from bones. Randomly generated outfits don't include bone armor.

Sorry that I wasn't clear. I'm not at all having a problem with bone armor not showing up in randomly generated outfits. Yes, I mentioned that it wasn't showing up, but just as a detail to confirm that the random outfit aspect is working as expected. The problem I'm having with bone trousers is that the game turns them into jagged bone greaves (or some combination of adjective + not the object it should be) when a save is loaded.

The problems with reloading the world causing off-by-one errors or whatever is probably due to CUT_X being done in the wrong order and thus duplicated raws (what y'all are calling "list shifting", it's been a known thing for years lol) happening, which is odd. Perhaps you're putting your mod before the vanilla items in the load order, which would cause obvious problems.

Obvious enough that even I figured out I should always load my mod last.

Here's something I used for testing. Feel free to fire it up, save, and load to see what happens to the caps:

Code: [Select]

item_helm

[OBJECT:ITEM]

[CUT_ITEM_HELM:ITEM_HELM_CAP]

[ITEM_HELM:ITEM_HELM_CAP]
[NAME:cap:caps]
[METAL_ARMOR_LEVELS]
[LAYER:OVER]
[COVERAGE:50]
[LAYER_SIZE:10]
[LAYER_PERMIT:15]
[MATERIAL_SIZE:1]
[SOFT]
[LEATHER]
[BARRED] - this has been added
[SCALED] - this has been added
[HARD]
[METAL]
[STRUCTURAL_ELASTICITY_WOVEN_THREAD]
[SHAPED]
It actually works far better than what I expected, because it adds the option to make bone/shell caps to the appropriate menus in the appropriate workshop, without anything else required. I'm totally mystified as to why this causes such problems after the dwarves make the item and the game is loaded from a save.

What I've done to test this:

• First, I tried doing a SELECT_ITEM and then adding the two tokens. Didn't work at all, so switched to this CUT-and-replace method instead.
• New world. Activated this mod, no other mods. Last in load order.
• Build a butcher, craftdwarf 'shop, leather 'shop. Butcher a yak.
• Make a leather cap, and make a bone cap. Save and exit to main menu while the caps are easy for me to find again, in the workshops.
• Load the game, check the workshops. Each cap, both the bone and the leather cap, has been changed to a [adjective] + bone/leather + non-cap item (e.g. "rounded yak leather helm").

It's consistent and repeatable. Either I've done something wrong (not sure how, the item_helm.txt file is the only file in the objects folder), or I've discovered an unexpected behavior that is worth documenting.

[Putnam]

Just noticed the wiki had this added at some point:



Hope nobody's trying to do this, because it won't work and whoever added it should probably make sure the things they're claiming are what you "have to do" even work at all before adding them to the wiki.

To be specific: SELECT_ and CUT_ work no matter where they're put. Starting a file with

Code: [Select]

patch_ will simply make it not be loaded. The edit that added that claimed that I was the one providing the info, but I never said anything like that, eek.

Code: [Select]

[CUT_ITEM_HELM:ITEM_HELM_CAP]


It's just CUT_ITEM, you're duplicating raws by doing this, check your errorlog. If you were using SELECT_ITEM_HELM, that wouldn't work either, it's just SELECT_ITEM for all item types.

[Bihlbo]

Code: [Select]

[CUT_ITEM_HELM:ITEM_HELM_CAP]


It's just CUT_ITEM, you're duplicating raws by doing this, check your errorlog. If you were using SELECT_ITEM_HELM, that wouldn't work either, it's just SELECT_ITEM for all item types.

Really? I seem to have gathered that the way cut and select worked was by adding that to the beginning of the entry you want to affect. For example, the post above which doesn't work (shoulda been a hint, I suppose). To me, it looked like [ENTITY:MOUNTAIN] gets selected by adding SELECT_ between the bracket and ENTITY. Or, [INORGANIC:ILMENITE] gets cut by adding CUT_ the same way. I suppose this means the [CUT_USE_MATERIAL_TEMPLATE:STRUCTURAL:STRUCTURAL_PLANT_TEMPLATE] I've been using means nothing when read by the game. I thought it was a universal tool that could select and cut any particular thing, but I guess it's more like all the rest of the tokens: a list of tokens that are specifically defined, and limited to only what shows up on that list.

Thank you very much for making it clear that those examples brought me to the wrong conclusion. I'll make a clear glass statue in your honor.

What errorlog? How do I access this? I've heard of it, but assumed that it involved some 3rd-party software that I don't have installed.

[Bihlbo]

I forgot to mention the following token:

[LOG_CURRENT_ENTRY]

It works for all object types that SELECT and CUT work with, and prints the entire entry that contains it into a file. Just in case you want to know exactly what SELECT_CREATURE is doing.

Does this produce an error log? How is the token used? Does it go before or after the file's OBJECT token? Where is the file? I'm guessing it would be in the mod folder, but maybe it's in the installed mods folder version of the mod, or maybe in its own special corner somewhere. edit: found it!