Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[YAHG]

Check if there's an existing one and rename the new one to (x)a or something.

I think it's reasonable to assign some responsibility to the person using the tool.  Maybe two mods were trying to adjust the same object (two attempts at turning fanciful CENTAURs into actual creatures), maybe one is trying to tweak the other's changes (fiddling with the changes made by a graphics pack), or two different people happen to have come up with the same idea for a name (a HEARTH building, a CAVEMAN entity, etc.).  It would be nigh impossible for the tool to figure out which case is which.  Bonus points if the user can assign prefixes in that third case without re-writing the source files.

A lot of this is over my head. But would some sort of standardized commenting signature help in this case?

Say like if you son is YAHG you do

#YAHG-8-12-2014-1:01AM

Or something like that, your program could prolly do a time stamp pretty easy.

[thistleknot]

Looks like Putnam is all up for this btw.

that's a great idea.  Comments actually don't need to be in a specific format ahead of time, but what can be done.   

Is [things like] broken [token's

and

###Comments

or even

comments

that are in the raws.

Can be stored in their own separate non token data object.

Maybe I can just load them into their own memory object just like any other token, but upon output.  I flag it for non token/tag output.

In the case of broken tags/tokens (sometimes they are not broken but merely disabled).  We would merely rewrite the tag/token/comment as we read it.  With broken token and all.

We could append in the text file #PossibleBrokenTag, maybe as a new line even; maybe default behavior is to count the # of broken token counts and output it to a new window each instance in patch file format, but go ahead and modify the file anyways:

Yes, that would be wonderful.  It would be like a broken token log file.

As to possibly duplicate object types, or the lack of matching token matches within an object.

A % of pre, post, and combined token matches could be displayed to show how accurate the object mapping was.

Red flag ones would be token matches of less than 50% I would say.  A log file of token matches could be produced to let a player know if anything questionable was merged.  So in the case of tigerman, or two mods who have the same name for two different creatures.

A 0% means the object didn't exist beforehand until after the mod.

Spoiler: i guess one separate issue would be, 3 way merges (click to show/hide)

... If two mods literally set separate values for the same token/tag.

Say vanilla clutch:size was 2:3:4

and civforge was 3:4:5

and random person a's personally modded changes were 4:5:6

well.  In that case a manually resolved conflict would be thrown.  However, I don't think this issue is a real concern, as a modder would be implementing one mod at a time over another mod with this patch setup.  However...  3 way merges should be allowable.

[Warmist]

Currently my biggest wish for mod-manager is some sort of state tracking. I.e. i don't like how it detects if mod is already merged in.

Second in wish list is this. Namely some smarter way of adding-removing everything and being raw aware. Imho there is no silver bullet: even multi-million programming industry did not crack this problem and there are so called "merge conflicts" that need to be sorted out by hand.

One drawback i have almost no experience with raw moding. But even making my simple mods (mostly as showcase for various plugins and adding some flavor to df) i experienced one big problem: which entities should get reactions/buildings? What happens if some other mod adds entities (especially after you added reactions)? What happens if you are using BIG mod (e.g. masterwork and all the entities are unknown)?

I think we need a "raw manipulating library" i.e. that a script could ask for various things (e.g. all creatures that have X in their token) and manipulate them. Then both procedural generation of raws and various merge operations (also mod installation which might even depend on existing raws in some strange ways) would use same library.

[thistleknot]

when you guys say Raw Aware.

Do you mean understanding the data structure of the entire raw entity?

And...I don't know how dfhack mods things in raws.  Either it's a token, or it's a comment with the setup I just described 2 posts up.

What happens if some other mod adds entities (especially after you added reactions)?

This happened with 40_06.  When I was editing trees back to basic wood types, I had to go through new trees.

This app would not go to the extent of extending the edit to those new entities.

I think we need a "raw manipulating library" i.e. that a script could ask for various things (e.g. all creatures that have X in their token) and manipulate them. Then both procedural generation of raws and various merge operations (also mod installation which might even depend on existing raws in some strange ways) would use same library.

I think that can be accomplished with simple regular expression and batch file editing.

I've seen it recommended for creature natural discipline fix on reddit.  They used a regular expression to append a new token to the creature entity.

http://www.reddit.com/r/dwarffortress/comments/2d0z1e/devlog_made_sparring_people_use_their_weapons/cjl69ei

The best I could do... Is maybe log the # of matching tokens before and after a change for things like this.  Then a modder can see what else needs to be edited afterwards.

[Dirst]

Currently my biggest wish for mod-manager is some sort of state tracking. I.e. i don't like how it detects if mod is already merged in.

It's conceivable to derive a "fingerprint" from a merge file to determine if it is already merged in or not, but a far simpler solution would be to tag a specific file with comments, maybe something like creature_standard since that will always be present.  It should include the mod's name, version number, and a timestamp of when it was added.

Second in wish list is this. Namely some smarter way of adding-removing everything and being raw aware. Imho there is no silver bullet: even multi-million programming industry did not crack this problem and there are so called "merge conflicts" that need to be sorted out by hand.

The only way to do this reliably is to hope that the player hasn't made any changes by hand, then remove mods in reverse order until you've taken out the target, then re-apply the ones that were supposed to stay in.

A tool to identify the player's hand-made changes and diff them into a "mod" would be very clever, but it'd require a backup copy of all the raws before the player messed with them, and it would still break if the hand-made changes involved the mod to be removed.  If it works, the hand-made tweaks would just be the most recent mod to be peeled off and re-applied in the removal process.

One drawback i have almost no experience with raw moding. But even making my simple mods (mostly as showcase for various plugins and adding some flavor to df) i experienced one big problem: which entities should get reactions/buildings? What happens if some other mod adds entities (especially after you added reactions)? What happens if you are using BIG mod (e.g. masterwork and all the entities are unknown)?

Inserting stuff into entity_default.txt is one of the biggest arguments in favor of a merge utility, since simply overwriting the file is almost certainly the wrong answer.  For an early version, it's fine that only entities foreseen by the modder get new buildings and reactions, but see below.

I think we need a "raw manipulating library" i.e. that a script could ask for various things (e.g. all creatures that have X in their token) and manipulate them. Then both procedural generation of raws and various merge operations (also mod installation which might even depend on existing raws in some strange ways) would use same library.

If we go with some minor scripting capabilities, it's possible to put in a condition such as "add [NATURAL_SKILL:DISCIPLINE:1] to every CREATURE with a [LARGE_PREDATOR] tag" or "add [PERMITTED_BUILDING:MAGMA_KITCHEN] to every ENTITY with a [DEFAULT_SITE_TYPE:CAVE_DETAILED] tag".  The issue becomes removing those mods.  If the tag to be added was already there, it will be removed if the mod is removed unless we get into complicated tagging of tags with comments.

[salithus]

I think we need a "raw manipulating library" i.e. that a script could ask for various things (e.g. all creatures that have X in their token) and manipulate them. Then both procedural generation of raws and various merge operations (also mod installation which might even depend on existing raws in some strange ways) would use same library.

If we go with some minor scripting capabilities, it's possible to put in a condition such as "add [NATURAL_SKILL:DISCIPLINE:1] to every CREATURE with a [LARGE_PREDATOR] tag" or "add [PERMITTED_BUILDING:MAGMA_KITCHEN] to every ENTITY with a [DEFAULT_SITE_TYPE:CAVE_DETAILED] tag".  The issue becomes removing those mods.  If the tag to be added was already there, it will be removed if the mod is removed unless we get into complicated tagging of tags with comments.

I'm really naive when it comes to this stuff, but it seems like the easiest thing would be to use scripting to add tags, then take a diff and use that to remove the tags (instead of reversing the script itself).

[thistleknot]

Github and deriving hopefully universal patch files was my main concern.

These other tidbits of left over code would be best handled w scripts.

I've done some work w otherodding tools that allowed for batch operations on the "raws" like ctp2edit. That's why I was thinking to log raw tokens that matched what was changed but weren't due to not being targeted in the patch file.

[Putnam]

Currently my biggest wish for mod-manager is some sort of state tracking. I.e. i don't like how it detects if mod is already merged in.

Second in wish list is this. Namely some smarter way of adding-removing everything and being raw aware. Imho there is no silver bullet: even multi-million programming industry did not crack this problem and there are so called "merge conflicts" that need to be sorted out by hand.

One drawback i have almost no experience with raw moding. But even making my simple mods (mostly as showcase for various plugins and adding some flavor to df) i experienced one big problem: which entities should get reactions/buildings? What happens if some other mod adds entities (especially after you added reactions)? What happens if you are using BIG mod (e.g. masterwork and all the entities are unknown)?

I think we need a "raw manipulating library" i.e. that a script could ask for various things (e.g. all creatures that have X in their token) and manipulate them. Then both procedural generation of raws and various merge operations (also mod installation which might even depend on existing raws in some strange ways) would use same library.

The main issue with merge conflicts is that such things affect two of the same file; in Dwarf Fortress, you're merging things that (at the very least ought to be) different files. It would not be difficult to simply add an identifier to the end of every file from a merged mod and ignore or (again) add an identifier for any duplicate raws.

[salithus]

Are you guys planning to have it look at an existing, previously untouched-by-the-manager, install and determine what mods are currently in use? If so, hats off to you - that seems like a big undertaking. I was thinking instead you'd have some sort of "undo" queue to track what mods were installed, and god help whoever wants to use it with an already-modded install.

[thistleknot]

I was not trying to implement a version tracking system for mods.  No.  Merely a way to merge 2 or 3 mods into one single mod.

Nor was I trying to dissect a mod and figure out what's applied.  That is something that could be attempted, but... not by me.  If anyone wants to take this project elsewhere when I'm done with it, so be it.

My intended goal is something akin to rawexplorer and mod manager; but primarily for merging two mods together by focusing on objects and merging them by tracking their raw tags differences.

Version tracking can be done using a separate tool like github or git.

Update/Note:
Btw,

I got the program up to a state where I can load two text files, and derive a patch in a new output window that can be saved.

The three things I needed to learn to do [for the most part]

open file a, file b  & compare.
output diff to window b

Next, I hope to apply a patch file.

Then... I'm going to deal with a modified patch file by counting tag's... and reading entity objects.

So it will be a long arduous journey, but I think knowing how to work with different forms was my biggest hurdle, soon it should be about parsing tags and objects, and then singling out those objects to new windows.

[Dirst]

The main issue with merge conflicts is that such things affect two of the same file; in Dwarf Fortress, you're merging things that (at the very least ought to be) different files. It would not be difficult to simply add an identifier to the end of every file from a merged mod and ignore or (again) add an identifier for any duplicate raws.

Unfortunately there is only one place to edit an entity to enable buildings and reactions.  Changing anything that appears in vanilla also invites potential conflicts (for example, a mod alongside a graphics pack).

A [ENTITY:MOUNTAIN:CONTINUED] tag would be nice, but would take a huge effort on Toady's part.

[palu]

Creature variations for other things would be even better.

[Sean Mirrsen]

And here I was thinking nobody would need it if I remade ModBase.

I still have the source code around... somewhere... might take a look at it and update it to the current DF version. Easier said than done, but at least it'd be starting off a working algorithm...

[thistleknot]

ModBase?

I'm not sure if I've heard of that tool.

You know, now that I think about it.  I may have glanced at it.  However, I don't think I ever really got into it.  It sounds almost exactly what I was hoping to derive with these types of patch files [I was going to combine the concept with git to achieve an overall version control system that would allow me to create my own custom mods].  However... I didn't/wasn't planning on the specific ability to deselect objects to be applied.  It was either an all or nothing, with the modder reserving the ability to manually edit the patch files after they are derived if they wish to achieve specific configurations of a mod.

Update:

It sounds like your tool is somewhat raw aware?

  You can change the order of the mods in the list (this influences mods that change the same things), and you can disable them entirely, or disable their individual parts from the Components list.

[PeridexisErrant]

I'm working on defining a standard format for mod loaders and I'd love feedback (and adoption!). 

This is up the more advanced handle-odd-merge-cases end, but it would be great to see everyone working from the same basic format.  Aside from anything else, it would probably mean that anyone could fork and add better logic to the interesting parts rather than having to reinvent the wheel.