Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[RantingRodent]

What does it do?
Updating your mod for each release is a pain, right? Well hopefully not any more. I kept trying to solve the time problem with complex utilities, but this time I'm building something super simple. RawPatcher to the rescue. The utility goes through the DF raws and modifies the values that were changed in your Mod raws, and adds new entries where none previously existed.

The tool will attempt to intelligently merge raw objects according to this logic:

1. The first token in the tag is treated as the name of that "property".
2. If the property doesn't exist on the source object, it is added to the end.
3. If an exact match for the property is found in the source object, no changes are made.
4. If one instance of the property exists in the original and mod objects, the mod property replaces the original property.
5. If multiple instances of the property exist in either location, the mod property is added to the end of the object.

You can also force behaviour by using the following tag formats:

[ADD_TAG:METAL_ORE:IRON:100] will add the tag [METAL_ORE:IRON:100] to the end of the object, regardless of what is already there. If a tag is found which matches this exactly, it will not be added.

REMOVE_TAG removes a single property that best matches the provided tag. An error results if multiple tags ties for best match.
[REMOVE_TAG:METAL_ORE] will remove [METAL_ORE:IRON:100] if it is the only METAL_ORE on the object. There will be an error if it also has [METAL_ORE:LEAD:100]
[REMOVE_TAG:METAL_ORE:IRON] would remove [METAL_ORE:IRON:100] if both tags existed.

[REMOVE_ALL:METAL_ORE] will remove all METAL_ORE tags on the object. Only accepts a single parameter.

DFFD link: http://dffd.wimbli.com/file.php?id=3353

Installation
1. Install Adobe AIR
2. Double-click the AIR file
Should work on PC, Mac and Linux. Contact me if it doesn't, please

Usage
1. Create "mod raw" files containing ONLY your changes. (see below for example)
2. Point the utility to your mod files and a clean copy of the df raws
3. Point the utility to an output folder
4. Click "Do It"

Notes
- files in the source folder with no counterpart in the mod folder will be copied directly
- files in the mod folder with no counterpart in the source folder will be copied directly
- the order of objects in the raws don't matter. The tool is smart enough to match up tags to objects
- the tool is not smart enough to match things up well with hierarchical sub-tags at this point, so some of the more complex raws may not work well
- you can use this tool to combine several mods together by using the output from the first merge as the base for the next merge
- all comment text is completely stripped during this process

Roadmap
I'll eventually put up a poll to see what I should prioritize, but this is my personal order for now.
1. Eliminate the restrictions on sub-folders are raw file types
2. Save last selected folders
3. Generate a mod folder describing the difference between two sets of raws
4. Select multiple mods to apply at once, detect conflicts
5. Single file merge
6. Rule-based changes
7. Add support for init files
8. Retain comments

Example
In the DF Raw (inorganic_stone_mineral.txt)

Spoiler (click to show/hide)

inorganic_stone_mineral
[OBJECT:INORGANIC]

[INORGANIC:HEMATITE]
[USE_MATERIAL_TEMPLATE:STONE_TEMPLATE]
[STATE_NAME_ADJ:ALL_SOLID:hematite][DISPLAY_COLOR:4:7:0][TILE:156]
[ENVIRONMENT:SEDIMENTARY:VEIN:100]
[ENVIRONMENT:IGNEOUS_EXTRUSIVE:VEIN:100]
[ITEM_SYMBOL:'*']
[METAL_ORE:IRON:100]
[SOLID_DENSITY:5100]
[MATERIAL_VALUE:8]
[IS_STONE]
[MELTING_POINT:12736]

In the mod Raw (inorganic_stone_mineral.txt)

Spoiler (click to show/hide)

[INORGANIC:HEMATITE]
[TILE:'S']
[REMOVE_ALL:ENVIRONMENT]
[ENVIRONMENT:IGNEOUS_INTRUSIVE:VEIN:100]
[REMOVE_TAG:MATERIAL_VALUE]

In the merged Raw (inorganic_stone_mineral.txt)
inorganic_stone_mineral

Spoiler (click to show/hide)

[OBJECT:INORGANIC]

[INORGANIC:HEMATITE]
   [USE_MATERIAL_TEMPLATE:STONE_TEMPLATE]
   [STATE_NAME_ADJ:ALL_SOLID:hematite]
   [DISPLAY_COLOR:4:7:0]
   [TILE:'S']
   [ENVIRONMENT:IGNEOUS_INTRUSIVE:VEIN:100]
   [ITEM_SYMBOL:'*']
   [METAL_ORE:IRON:100]
   [SOLID_DENSITY:5100]
   [IS_STONE]
   [MELTING_POINT:12736]

[Dral]

Sounds spiffy.

Is there currently (and if not, do you plan to add) a way to allow mods to delete tags from existing objects? Also, a way to delete entire existing objects?

[RantingRodent]

Ah, yeah, I forgot to add that to my list. I would like to add that as well. Basically just by adding a token like so:

[REMOVE_TAG:AUTUMN] would remove the [AUTUMN] tag.
[REMOVE_OBJECT:WEED_RAT] would remove the [PLANT:WEED_RAT] object

Another feature I didn't mention above was that I would like to allow changing only some parameters from a list and leave the rest alone. My thought was this type of syntax:

[SOMETAG:...:...:...:42:42:...:...]

[Artanis00]

This is a good concept, but it strikes me that writing mods as essentially deltas is rather foolhardy. You'd have to have the originals open as reference while you manually write changes you intend to make to them in a separate file, and once that is done, you'd not be able to tell by looking at that file what state it changes from nor what to (and neither would a program). Even with the base raws, you'd not know what it changes to without this tool.

Not to mention that if your mod is broken by the new version, you'll be left editing deltas against new raws to fix it. This may be exceptionally difficult--especially if the new version removed an object you were editing.

tl;dr: this idea will have you specifying transformations to raws, rather than changing the raws themselves, probably with few benefits.

I think it'd be better to just Diff the modded raws against the originals, which is what I assume most do now. With that, you have normal raw format files to work with before and after migrating to the new version, rather than deltas (which are managed on an as-needed basis by existing tools).

Spoiler: alternate solution (click to show/hide)

If I may, simply version controlling the DF install (with Git, for example), would give you the ability to do just this rather easily (i.e.: no writing diffs manually). A sample flow would be to make a branch for the mod (vanilla install in the master branch, the mod in my-mods-name) and make all mod changes in my-mods-name. Then, when a new DF releases, you switch to master, wipe out the contents of the branch, unpack the new version into the repository and commit it into the master branch (you just replaced the master branch's version with the new version), and rebase the mod branch on master (replay the branch's changes on top of the updated master branch).

[RantingRodent]

you'd not be able to tell by looking at that file what state it changes from nor what to (and neither would a program). Even with the base raws, you'd not know what it changes to without this tool.

I'm not sure what you mean by this. You know what it changes to; it changes to what you put in the file.

[Artanis00]

you'd not be able to tell by looking at that file what state it changes from nor what to (and neither would a program). Even with the base raws, you'd not know what it changes to without this tool.

I'm not sure what you mean by this. You know what it changes to; it changes to what you put in the file.

Code: [Select]

[SOMETAG:...:...:...:42:42:...:...]

Say you've written that in the delta.

From that line, all I know is that SOMETAG parameters number 4 and 5 are going to be 42. I don't know what the final result will be without finding SOMETAG in the original raws and looking at parameters 1, 2, 3, 6, and 7.

Or you could run this tool and look at the final raw...

In programming, there is a cycle known as "edit, compile, test". Right now, without this tool, DF modding follows something similar: edit raws, start up DF, test changes. Because this tool has you edit deltas for raws, you've added an additional loop in that cycle: (edit compile test) compile test. Essentially, edit raw deltas, compile to raw, check raw, good?break:continue, start DF, test.

Or you could edit the raws and Diff them to the next version, which would do this:

Code: [Select]

-[SOMETAG:1:2:3:4:5:6:7]
+[SOMETAG:1:2:3:42:42:6:7]

I didn't write that, but the new version, and it tells me what the original is and what the final line is. And I could run it backwards if I needed to.

[RantingRodent]

Well yes, for that one feature the point is valid. I thought there were more tags with a fixed number of parameters over 2-3, but on further investigation that's not the case (I only really do graphics modding, so my knowledge of the rest of the raws is limited). That was just an afterthought idea, though, not at all central to the utility.

To be clear, that isn't implemented right now and was pretty near the bottom of my priority list. Right now you enter whole tags and that's it.

[calrogman]

Here's the proper way of doing it.

Code: [Select]

wget http://bay12games.com/dwarves/df_31_16_linux.tar.bz2
tar -xf df_31_16_linux.tar.bz2
cp -R df_linux/raw/objects df_linux/raw/objects_orig
#write mod
cd df_linux/raw
diff -r -u objects_orig objects > mod.diff
Then I can stick your .diff in df_linux/raw and run

Code: [Select]

patch < mod.diff, gen a world and play your mod.  If I decide I don't like the mod I can run

Code: [Select]

patch -R < mod.diff to get back to a clean raw folder.  If it was up to me all mods for this game would be distributed as diffs.  It helps me keep my raw folder clean, apply and remove mods as I wish and it discourages messing with my init files and graphics.

The magical thing about using diffs in this manner is that it's also totally cross platform.  Every OS in the history of the universe has a port of patch.

[RantingRodent]

In programming, there is a cycle known as "edit, compile, test". Right now, without this tool, DF modding follows something similar: edit raws, start up DF, test changes. Because this tool has you edit deltas for raws, you've added an additional loop in that cycle: (edit compile test) compile test. Essentially, edit raw deltas, compile to raw, check raw, good?break:continue, start DF, test.

Ah, forgot to comment on this. This isn't a tool for initial development. It's a tool for applying your mod once it's developed. At this point, the "edit" step should be done with for good. When a new version of DF comes out, it's very unlikely that you should actually have cause to edit the raws again to bring your mod up to the new version. Instead your cycle starts with testing, and in most cases will end with testing. If you aren't adding a new feature to your mod, why should you even be cracking the raws open?

I had considered using a simple diff tool, but I've been developing software for a little under a decade now, and I've been burned many times by simple diff tools being too dumb for a particular situation. If I have to go over the diff results with a fine-toothed comb, it defeats the purpose; to avoid expanding on my 60-80 hour work weeks. I'd rather build a domain-specific tool which I can actually rely on to do the job without me reviewing the output in detail. I don't have confidence that a simple diff tool could handle taking a mod and applying it to subsequent DF versions without generating a fresh diff for each subsequent version.

[i2amroy]

So just for clarification as what functions this has exactly seem to be spread out over two or three posts, this tool currently:
1) Replaces tags if they already exists
2) Adds tags if they don't already exist
3) Currently can't remove tags but will be able to soon

Just making sure I have this right as it sounds like a useful tool for me (especially since it works on mac), at least for making all of those tiny adjustments that I have accrued in my time playing dwarf fortress.

[RantingRodent]

Yeah, that covers it.

[i2amroy]

Ok, cool. I'll download it right away!

[RantingRodent]

Great. Let me know how it turns out. I'm on a PC, so I haven't been able to actually test it on a Mac. I'd like to know for sure that it works properly.

[i2amroy]

Just got a chance to test it and it works great! The only problem that I ran into in my testing was when I tried to add a new caste to a creature (dwarves in this case). The Raw Patcher will sometimes overwrite duplicate tags (a tag that shows up more than once, such as [CASTE:XXX]) if you tell it to add a new one in (in this case I ended up with no [CASTE:FEMALE]). It worked great for everything else though, and even properly edited some of the longer tags. This is a great tool for people like me who are just trying to add our own little things to our versions of dwarf fortress.

P.S. You do want to make sure that you don't set your end folder to one of your starting mod and base folders but to a different folder instead though, or it will just overwrite your set end folder's files with the ones from the other folder without combining the two first.

[RantingRodent]

I forgot about that. I still need to figure out a good approach for tags which occur multiple times. At the very least, I can force it to search for the best match when it's replacing. That much is easy enough. An explicit ADD_TAG should cover other cases. I haven't got the time tonight, but I'll try to get a version out this weekend with ADD_TAG and REMOVE_TAG, as well as the object marker mappings for the raws where it doesn't match.