Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[thistleknot]

so...

I'm thinking on this idea.

There is a way to COMBINE patch files.  If we combined them, we could apply them once vs retrying every iteration with a new patch.  Maybe even allow a player what patches/mods they want to test together, find the ones that are incompatible with each other, and show the player how they could use a 3rd party tool like diffmerge or tortoisegitmerge and show them the two patch files and the base folder if they so choose to do the merge themselves.

HEAD Patch file info:

tortoisegitmerge will throw a error about the two [patches] not being able to be applied neatly WITHOUT PROPER HEAD INFO?  So yeah.  I'm not 100% sure, but the Head info is in the top of each file, but I believe the paths in my output are relative to MY GIT INSTALLATION.  So... if we derived patches off of working actual folders... the head might actually exist on the disk, the head being the base files I believe (ex 40_09 ascii).  So the conflicts may/may not be resolved automatically with a merge?

I'm going to try some tests.

Anyways. 

As to applying grx patches last, 100% agree on that.

Also...

[King Mir]

I wrote up a simple python function that tests if merging is possible. I test it by passing strings, making it diff on individual letters, but if you pass it a list of strings, it will diff that as lines.

Code: [Select]

import difflib

def can_merge_seq (mod_text, vanilla_text, into_text):
    van_mod_match = difflib.SequenceMatcher(None, vanilla_text, mod_text)   
    van_gen_match = difflib.SequenceMatcher(None, vanilla_text, into_text)   
   
    van_mod_seqs = van_mod_match.get_matching_blocks()
    van_gen_seqs = van_gen_match.get_matching_blocks()
   
    cur_v = 0
    while cur_v < len(vanilla_text) :
        (i1,j1,n1) = van_mod_seqs[0]
        (i2,j2,n2) = van_gen_seqs[0]
        if i1 > cur_v and i2 > cur_v:
            return False
        if i1 + n1 - cur_v < i2 + n2  - cur_v:
            cur_v = i1+n1
            van_mod_seqs.pop(0)
        else:
            cur_v = i2 + n2
            van_gen_seqs.pop(0)       
    return True

print can_merge_seq ('anything at all','vanilla','vanilla')
print can_merge_seq ('oooh','vanilla','vanilla')
print can_merge_seq ('vanil','vanilla','nilla')
print can_merge_seq ('van','vanilla','la')
print can_merge_seq ('van','vanilla','la')
print can_merge_seq ('vani','vanilla','lla')
print can_merge_seq ('vonilla','vanilla','banana')
print can_merge_seq ('vannilla','vanilla','banana')
print can_merge_seq ('vonilla','vanilla','venilla')


[Pidgeot]

Changes to d_init.txt and init.txt may need to be treated specially. Maybe just let graphics modify them, then post-process the files with user settings in a script.

Ideally the graphics packs will only change the fields they actually need to change, making this a non-issue.

The way I'm handling it in my launcher is to just read specific fields from the d_init.txt and init.txt files, and only overwrite those. For a full-blown patch system, the easiest way is probably to do just make a full patch and then filter out any extraneous changes to those files (or filter them out during patch creation).

[thistleknot]

As to what Button was saying about the merge conflict:

http://www.bay12forums.com/smf/index.php?topic=142295.msg5586461#msg5586461

"Oh, those should be able to merge well enough together one after the other. I mean, from a logic perspective. Obviously merge tools tend to be... a little simplistic."

The fact that we could have just "added those lines on top of each other" means that could be an option to present the user.

I think the reason that happened is because one mod added 1 line, and the other mod added say 5 lines to the same spot.  Both mods/patches read the original location of that FILE AS BLANK.  Since we are only ever merging two mods at a time, since VANILLA is always our base.  If both lines expected a blank spot (can be read from patch file), and they found something there (in the case of merge conflicts).  It's safe to assume that we CAN ADD BOTH PATCHES, ONE ON TOP OF THE OTHER.

Actually, you know... now that I think about it.  Our base files don't have blank spots.  So what's happening is... their is a token mismatch.  At that point we can introduce logic on how to deal with token mismatches.  It's probably one mod realizing another mod wrote to that spot.

[Dirst]

As to what Button was saying about the merge conflict:

http://www.bay12forums.com/smf/index.php?topic=142295.msg5586461#msg5586461

"Oh, those should be able to merge well enough together one after the other. I mean, from a logic perspective. Obviously merge tools tend to be... a little simplistic."

The fact that we could have just "added those lines on top of each other" means that could be an option to present the user.

I think the reason that happened is because one mod added 1 line, and the other mod added say 5 lines to the same spot.  Both mods/patches read the original location of that FILE AS BLANK.  Since we are only ever merging two mods at a time, since VANILLA is always our base.  If both lines expected a blank spot (can be read from patch file), and they found something there (in the case of merge conflicts).  It's safe to assume that we CAN ADD BOTH PATCHES, ONE ON TOP OF THE OTHER.

Actually, you know... now that I think about it.  Our base files don't have blank spots.  So what's happening is... their is a token mismatch.  At that point we can introduce logic on how to deal with token mismatches.  It's probably one mod realizing another mod wrote to that spot.

This is the problem I was trying to fix a bunch of posts ago by doing two passes.  Slip in bookmarks where each change is supposed to occur secure in the knowledge that no lines will actually move, then perform the actual changes at those bookmarks which could move things around quite a bit.

This still doesn't do a good job with re-ordering things.  To get that, you'd probably have to nuke the vanilla file completely and replace it using a different filename.  The launcher would detect that one mod's changes got deleted by another, but absent some advanced raw-aware logic it just isn't possible to merge the two.  That advanced logic comes... later.

[King Mir]

As to what Button was saying about the merge conflict:

http://www.bay12forums.com/smf/index.php?topic=142295.msg5586461#msg5586461

"Oh, those should be able to merge well enough together one after the other. I mean, from a logic perspective. Obviously merge tools tend to be... a little simplistic."

The fact that we could have just "added those lines on top of each other" means that could be an option to present the user.

I think the reason that happened is because one mod added 1 line, and the other mod added say 5 lines to the same spot.  Both mods/patches read the original location of that FILE AS BLANK.  Since we are only ever merging two mods at a time, since VANILLA is always our base.  If both lines expected a blank spot (can be read from patch file), and they found something there (in the case of merge conflicts).  It's safe to assume that we CAN ADD BOTH PATCHES, ONE ON TOP OF THE OTHER.

Actually, you know... now that I think about it.  Our base files don't have blank spots.  So what's happening is... their is a token mismatch.  At that point we can introduce logic on how to deal with token mismatches.  It's probably one mod realizing another mod wrote to that spot.

The problem is, sometimes additions in the same spot don't conflict, sometimes they do. Two mods adding a creature in the same spot are probably ok. Two mods adding a the same tag to the same creature might cause problems. A Simple solution is to not allow adding to the same spot.

[King Mir]

This still doesn't do a good job with re-ordering things.  To get that, you'd probably have to nuke the vanilla file completely and replace it using a different filename.  The launcher would detect that one mod's changes got deleted by another, but absent some advanced raw-aware logic it just isn't possible to merge the two.  That advanced logic comes... later.

We've been talking about this in the abstract, but is there a test case of two mods that we'd like to be able to merge with one doing complex reordering, and the other adding or removing stuff? At the very least we need to make sure that we properly detect that the mods can't be naively merged.

[Button]

All right, I've got a little pack of minor mods with potentially interesting interactions uploaded now. http://dffd.wimbli.com/file.php?id=9443 .

These features coexist happily in my raws, but may be a little challenging to merge together automatically, so I hope you find them useful .

[Dirst]

This still doesn't do a good job with re-ordering things.  To get that, you'd probably have to nuke the vanilla file completely and replace it using a different filename.  The launcher would detect that one mod's changes got deleted by another, but absent some advanced raw-aware logic it just isn't possible to merge the two.  That advanced logic comes... later.

We've been talking about this in the abstract, but is there a test case of two mods that we'd like to be able to merge with one doing complex reordering, and the other adding or removing stuff? At the very least we need to make sure that we properly detect that the mods can't be naively merged.

Spacefox did a lot of tidying up inside the gem raws in DF2012, but since the vanilla raws are now tidier I'm not seeing the same kind of re-organizing.  Still could use the 2012 versions a test case.

[thistleknot]

"Two mods adding a the same tag to the same creature might cause problems"
Couldn't this type of conflict be caught by reading the two patch files and being like, "hey, BOTH OF THESE are adding the same token in the same contextual match".

Of course it might be a little simplistic.

Two mods might be incorporating modest mod fixes but reordered the tokens!  Aghast!

That would still result in duplicates.

To address that situation, advanced mod merging would have to be implemented and detect object individual token +/- changes

update on interdiff combining patches
btw, I'm not having much luck merging patch files.  I think I have them "merged" but can never get them to apply correctly.  I know it can be done with github somehow and auto merge conflicts.  However, trying to get something like that done outside of github in diff patch gnu tools, idk.

I'm going to write up a stackexchange problem specifically targetting github users and seeing if there's an equivalent way of doing the same thing at the command line.

[King Mir]

All right, I've got a little pack of minor mods with potentially interesting interactions uploaded now. http://dffd.wimbli.com/file.php?id=9443 .

These features coexist happily in my raws, but may be a little challenging to merge together automatically, so I hope you find them useful .

Thanks, looking at it now. Already clear to me that two mods that add the same file cannot be trivially merged.

[thistleknot]

This still doesn't do a good job with re-ordering things.  To get that, you'd probably have to nuke the vanilla file completely and replace it using a different filename.  The launcher would detect that one mod's changes got deleted by another, but absent some advanced raw-aware logic it just isn't possible to merge the two.  That advanced logic comes... later.

We've been talking about this in the abstract, but is there a test case of two mods that we'd like to be able to merge with one doing complex reordering, and the other adding or removing stuff? At the very least we need to make sure that we properly detect that the mods can't be naively merged.

One solution to this problem (and is what I asked RawExplorer mod author a while back if he could incorporate).

Was object tag-id alphabetizing.

However, comments kind of create havoc with that.  However, if WE DID ALL THIS ON OUR END.  We could alphabetize our flattened raws based on some parse token-id trick.

The great thing about it is, the FIRST <token> is our object:type, so immediately, we can determine what we should be alphabetizing on.

[King Mir]

"Two mods adding a the same tag to the same creature might cause problems"
Couldn't this type of conflict be caught by reading the two patch files and being like, "hey, BOTH OF THESE are adding the same token in the same contextual match".

Of course it might be a little simplistic.

Two mods might be incorporating modest mod fixes but reordered the tokens!  Aghast!

That would still result in duplicates.

To address that situation, advanced mod merging would have to be implemented and detect object individual token +/- changes

If two mods add the same content in the same spot, we may be able to detect that and allow it. In the general case there's the possibility that they are changing the same token in incompatible ways.

EDIT: Button's mods modify c_variation_default.txt by adding the same content in two mods. So that's a test case for that.

[King Mir]

This still doesn't do a good job with re-ordering things.  To get that, you'd probably have to nuke the vanilla file completely and replace it using a different filename.  The launcher would detect that one mod's changes got deleted by another, but absent some advanced raw-aware logic it just isn't possible to merge the two.  That advanced logic comes... later.

We've been talking about this in the abstract, but is there a test case of two mods that we'd like to be able to merge with one doing complex reordering, and the other adding or removing stuff? At the very least we need to make sure that we properly detect that the mods can't be naively merged.

One solution to this problem (and is what I asked RawExplorer mod author a while back if he could incorporate).

Was object tag-id alphabetizing.

However, comments kind of create havoc with that.  However, if WE DID ALL THIS ON OUR END.  We could alphabetize our flattened raws based on some parse token-id trick.

The great thing about it is, the FIRST <token> is our object:type, so immediately, we can determine what we should be alphabetizing on.

You'd have to figure out top level tokens for that. That's not in PeridexisErrant's version 1 objectives. It also means you have to track two kinds of changes separately: those that add/remove/reorder top level tokens, and those that modify existing entities.

[King Mir]

All right, I've got a little pack of minor mods with potentially interesting interactions uploaded now. http://dffd.wimbli.com/file.php?id=9443 .

These features coexist happily in my raws, but may be a little challenging to merge together automatically, so I hope you find them useful .

So as an update on this here's what I found:
1)the two larger mods modify creature_bug_slug_new.txt in incompatible ways.
2)the two larger mods modify c_variation_default.txt with the same changes, which should be ok.
3)two mods add body_snail.txt, which cannot be allowed