Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[Sowelu]

I'm taking a look at it. No real experience doing this, but I'm somewhat familiar with how X-COM did it, and it looks very similar. I'll let you know if I find out anything.

EDIT: Got the .cat files unpacked. but can't make any sense of the IMC files. They're apparently mostly battle sprite animations. You probably already knew all that though. Oh well. Sorry. I might try a few more crazy things if I can think of any. But for now I"m tapped out.

Well, here's my understanding.  1) They use a very limited palette, so surely less than a byte, considering that it uses custom palettes for each person.  3 bits seems low, but 4 bits seems like enough.  2) I think someone on the mailing list said they looked like they were RLE.  In any case, they're a format where changing one bit can change big chunks of it.  3) Each .imc file has a lot of data; walk and attack cycles in each direction.  But that was probably obvious.

Many thanks for taking a crack at it!  I'll see what I can engineer as well, but this is known to be HARD.  I think someone went as far as trying to muck with the bits and bobs in machine code to see what was going on, and that looked fruitful but didn't last forever.  Apparently there's another file somewhere with animation data that is loaded at the same time as the .imc files at start of combat; it is possible-to-likely that all header information is embedded in that and not in the .imc's.  I've got a sneaky suspicion that each byte might be 4 bits for color and 4 bits for run-length.  Which still leaves a question:  Is the internal data stored as one spritesheet that the animation file knows how to blit from, or is it stored as its own weird catalog?

The cluebook has a lot of information in terms of battle mechanics, for what it's worth. I'm not sure how the skill rolls (and similar things) are done, but that MIGHT be reverse-engineerable, which of course is not a word (e.g. increase skills/stats to find out what's relevant for a task, and how the chance of success scales marginally at each point).

Combat rolls are pretty well documented, except (I think) for the actual to-hit formula that happens after skill modifiers for adjacent allies/enemies etc.  As for skill and stat rolls in the CYOA cards, in my experience they are linear from A to B; something easy might be linear from 85% to 115% based on your Speak Common (so your odds with speechcraft 0 is 85%, but it hits 100% at Speak Common 50) while something quite hard might be -30% to 70% based on a skill + a stat.  Okay, I don't know ALL the details especially when stats come into it, but that's how I'd do it.  I'm thinking that something like Cheat Engine could be useful here; hook when it reads your Speak Latin and then see what it does from there (or shove other values in there on the fly).

[sluissa]

Well, here's my understanding.  1) They use a very limited palette, so surely less than a byte, considering that it uses custom palettes for each person.  3 bits seems low, but 4 bits seems like enough.  2) I think someone on the mailing list said they looked like they were RLE.  In any case, they're a format where changing one bit can change big chunks of it.  3) Each .imc file has a lot of data; walk and attack cycles in each direction.  But that was probably obvious.

Many thanks for taking a crack at it!  I'll see what I can engineer as well, but this is known to be HARD.  I think someone went as far as trying to muck with the bits and bobs in machine code to see what was going on, and that looked fruitful but didn't last forever.  Apparently there's another file somewhere with animation data that is loaded at the same time as the .imc files at start of combat; it is possible-to-likely that all header information is embedded in that and not in the .imc's.  I've got a sneaky suspicion that each byte might be 4 bits for color and 4 bits for run-length.  Which still leaves a question:  Is the internal data stored as one spritesheet that the animation file knows how to blit from, or is it stored as its own weird catalog?

According to the mailing list, one of the original artists for the game chimed in and said that the animation sprites were all just in a grid and the programming to run the animation was elsewhere. I'm guessing the IMC files are nothing but sprites. But I could be wrong there.

I really keep wanting to think it's PCX format somehow. That would fit with RLE and also that they used that format in the tactical sections of X-COM as well. If it were as simple as that though, I'm sure someone else would have figured it out.

[G-Flex]

As for skill and stat rolls in the CYOA cards, in my experience they are linear from A to B; something easy might be linear from 85% to 115% based on your Speak Common (so your odds with speechcraft 0 is 85%, but it hits 100% at Speak Common 50) while something quite hard might be -30% to 70% based on a skill + a stat.  Okay, I don't know ALL the details especially when stats come into it, but that's how I'd do it.  I'm thinking that something like Cheat Engine could be useful here; hook when it reads your Speak Latin and then see what it does from there (or shove other values in there on the fly).

The way I see it, if we don't mind changing how that works a bit: It's 2010, so we can afford to do a little bit of math. I'd prefer a more continuous algorithm, based on a few input variables (skill ratings, difficulty of task, how much skill matters for the task, etc.) leading to a probability curve that's smooth and winds up asymptotically approaching 1. That's just my math nerd side talking, but hey, it could be interesting.

[Sowelu]

According to the mailing list, one of the original artists for the game chimed in and said that the animation sprites were all just in a grid and the programming to run the animation was elsewhere. I'm guessing the IMC files are nothing but sprites. But I could be wrong there.

Well then!  I'll dive right in once I'm back from work and start tweaking some files.  I've never decompressed a catalog before but it doesn't sound too painful.  "Set the 1s bit of every byte in the middle half of the file to ON" or "Set every fourth byte in that area to ON" both sound pretty good to me for starters.

It'll be nice to see our alchemist friend's sprite with a custom multicolor hat.  Not possible in vanilla I think.  A lot of the default sprite appearances aren't possible to get with the color selector.

The way I see it, if we don't mind changing how that works a bit: It's 2010, so we can afford to do a little bit of math. I'd prefer a more continuous algorithm, based on a few input variables (skill ratings, difficulty of task, how much skill matters for the task, etc.) leading to a probability curve that's smooth and winds up asymptotically approaching 1. That's just my math nerd side talking, but hey, it could be interesting.

Well, I figure that many cards will have XML entries describing the algorithm, with a bunch of basic ones allowed (while being able to hardcode more complicated formulas).  So, linear algorithms, exponential ones, log ones...all kinds, at the whim of the card writer.  Many of them need to *hit* 1, though, not just approach it.  A guy with INT 44, SpLt 80, RdWr 80, Alch 80 should not get laughed out of university, and a group with virtue of 60 should not be turned away from monasteries for being insufficiently virtuous.  Four guys with agility and strength 15 wearing heavy armor with no rope are not making it up to that third floor castle window, period.  Of course, any given choice ought to have multiple alternate paths.  I guess you'd have one XML tag for a selection, then an ordered list of tags inside it that define what algorithms to calculate, and where to go/what to do if that roll triggers.  Partial failure is good, so you don't end up alternating between "You drop your sword and look like a total fool" and "You whirl your sword around and look so badass that the bandits run in terror".  A middle ground might still be a failure, but perhaps it would let you choose a different approach.  Then again I guess it might be sufficient to fail as normal with the same bad consequences, but not be so schizophrenic about describing your abilities.

...Then again, the ability to always fail in bad conditions and sometimes pull off that 1% chance is part of making the game what it is, so I can't complain overmuch.  It keeps the game interesting, instead of letting it slide into safety and boredom.  Throwing the player into "Oh crap now what do I do" situations sometimes without being too unreasonable, or sometimes just letting them fail out of something completely ("Well damn I guess I can't get into that evil lord's tower at all no matter what I do right now"), is important as long as it's reasonable.

There's a lot of content that players won't see in a serious game.  It sure would be nice if there was a reason to risk jail.  ...Oh!  That's one big change I'd like to make.  The fine for being out late at night.  You come back from a big two-raubritter score and a fat pocketbook and see "Knowing that the usual fine for this offense is 20 florins, 8 groschen and 7 pfennigs, you decide to..." is not very impressive.  Aside from breaking & entering for quests, what's a good excuse to chuck the players in jail?  Revolutionary wars to take place in is one thing.  Corrupt officials?  Oh hey, now there's a plot device.  After you've started having dreams and seeing gargoyles, covens or other people start talking to cities and influencing people to try and ruin your rep.  Having an evil wealthy merchant, evil local lord, or (god forbid!) evil bishop would certainly spice things up:  Deal with it quietly or find a way to discredit them and you're good; otherwise you might need to stay out of town or risk unjust arrest.

[G-Flex]

If there end up being actual political factions of some form, jail is a fairly obvious device. As it stands, the game really doesn't have any notion of doing something that one town won't like, but that another will; it's all "upright society vs. everyone else", more or less. Of course, that's part of the charm; it's this really naively idealized version of 15th-century society. I think a little political intrigue could be incorporated without spoiling that, of course.

[sluissa]

Sowelu: The mailing list has a file repository on it that has a decatting program on it.

Files > merle > tools > decat.exe

I can't promise this actually works correctly, but if I stick a cat file in a get a bunch of seemingly properly named files out of it.

[Sowelu]

Ah right!  Thanks.  I'll work with that.

So what other bad guys do we want?  So far we've got:
- Street scum
- Low-class highwaymen
- Higher-class highwaymen like alchemists and corrupt priests
- Robber barons
- Demon-worshipping villages
- Some undead in a tomb or two
- Demons, including that awesome one with the bridge
- Crooked local lords in towers
- Covens of witches
- The Wild Hunt
- Dragons
- Gargoyles
- Mine denizens like gnomes and vulcans
- Wild creatures like wolves and giant spiders
- Baphomet's lackeys

What else can we add to fit the setting?

[G-Flex]

Are undead particularly fitting? I honestly don't know much about medieval Germany. I mean, I'm sure they had SOME sort of undead, just not which, or under what circumstances.

[Cthulhu]

You forgot Schrats.

[olemars]

Sowelu: The mailing list has a file repository on it that has a decatting program on it.

Files > merle > tools > decat.exe

I can't promise this actually works correctly, but if I stick a cat file in a get a bunch of seemingly properly named files out of it.

There's a tool for it?

Oh well, was fun figuring out the catalog format and extracting the files manually. Not that it's a terribly complicated one. There are separate imc files for attack and walk animations by the way.

I've been looking at a couple of the IMC files in hex.  There does seem to be a bit of header info in each one. The first 32 bytes doesn't seem to change a lot within a character class, the next 8 changes little.

Additionally there are only a few blocks of FFFF in each file, all near the start. Linebreaking on it reveals a pattern:

Code: [Select]

1F 81 05 00 02 00 01 FE FF E8 E3 FA FE 04 00 17 00 10 F0 FF C3 03 00 03 02 F2 FD FE 01 FF FF
00 7C E9 FF EF FF 01 25 00 14 FC 78 E2 16 00 13 00 0F FC 15 00 19 84 23 D2 BE 06 00 07 FE 40 1C B8 FE 03 F8
10 41 7E F8 C4 13 01 21 02 1A 03 E0 25 01 30 02 FF FF
2D 03 27 00 37 01 42 02 3D 03 3C 00 4A 01 50 02 FF FF
50 03 4E 00 5D 01 64 02 60 03 64 00 72 01 75 02 FF FF
73 03 76 00 85 01 86 02 87 03 8D 00 9A 01 99 02 FF FF
99 03 9F 00 AC 01 AB 02 A9 03 B4 00 C1 01 BE 02 FF FF
BB 03 C5 00 D2 01 D0 02 CC 03 DA 00 E7 01 E2 02 FF E1
DE 03 EB 00 F9 01 F4 02 F0 03 6C 0E 02 30 F9 06 FA 04 14Notice the alternating columns of 1, 2, 3 and 0. Extracting the columns show that the following byte increases for each occurence.

Code: [Select]

1 21? 2 1A 3 E0? 0 37
1 30 2 2D 3 27 0 4A
1 42 2 3D 3 3C 0 5D
1 50 2 50 3 4E 0 72
1 64 2 60 3 64 0 85
1 75 2 73 3 76 0 9A
1 86 2 87 3 8D 0 AC
1 99 2 99 3 9F 0 C1
1 AB 2 A9 3 B4 0 D2
1 BE 2 BB 3 C5 0 E7
1 D0 2 CC 3 DA 0 F9
1 E2 2 E1? 3 EB
1 F4 2 F0 3 6C?


I'm thinking 0..3 are frames in an animation. No idea what the other number means.

That's as far as I've gotten yet, which isn't very. Knowing the dimensions and the pallette would help tremendously though, since then it would be easier to experiment with drawing to pixmaps.

[Sowelu]

A screenshot on MobyGames makes it look like Gunther walking with a hammer is 16x32.  I might have miscounted (still at work) but it seems like a good number to me.  I think that everyone's sprite is only 16 colors as well.

So, Olemars:  Your last block has 50 bytes.  If it was raw pixel data, that would be 100 pixels which is way too small for 16x32 (512 pixels).  Also when I look at it there's only so much you can RLE.  The byte increases make zero sense to me.

I suspect that it should be read in three-nibble chunks.  Instead of "1 30 2 2D 3 27 0 4A", read it as "13 0 22 D 32 7 04 A".  Or "1 3 0 2 2 D 3 2 7 0 4 A" which isn't much to go on, but hey, more columns.

The world map did something interesting.  Something like...There were 256 tiles.  Each map entry was only 4 bits long.  Those 4 bits were the low 4 bits of the tile index, while the upper 4 bits were taken (1 each) from the tile's four neighbors' lower bits.  I think that the coders were not shy about using nibbles.

EDIT to remove stupid stuff:  So that header is the animation table.  If you take "3 27", "3 3C", "3 4E", "3 64", and you copy that 27 over the other values, one of your character's walk animations will be sliding across the ground while holding the same frame.

The more I poke around the file, the more confused I get.  I was screwing with Hans' file (the alchemist), and I wound up with a picture of Gunther (the hero)!  Are we missing something really big here?  I mean, it doesn't make sense, but.  I guess it's likely that I corrupted a pointer somewhere, and that different hero pictures are stored back-to-back.  In any case I think that data is stored in 16x16 chunks.  Hmm...  Oh yeah, and no longer convinced that it's only 16 colors either.  I got Hans to leave a trail of palette-shifting fire effects.  Anyway.  Leaning less towards recognizable RLE and more towards homebrew compression.  I think this means something like decompiling.  Bleh.

Although this is still worth staring at:  The alphabet-frequency attack is looking interesting.  Turns out the hex editor HxD can generate frequency plots.  Here's what it looks like, for our four heroes attacking with bows, in order from skinniest to fattest:

Spoiler (click to show/hide)

Gretchen


Hans


Gunther


Ebhard

Also notice the file sizes:
4,918 F60CBBW.IMC    Gretchen
5,524 A00CBBW.IMC    Hans
6,038 F01CBBW.IMC    Gunther
4,952 C00CBBW.IMC    Ebhard

Ebhard the cleric has wide expanses of the same color, so I'd expect that he compresses well with RLE.  Gretchen has lots of empty space, being thin, so I'd expect the same from her.

There's a couple spikes in the middle that grow as you move down the images; those are 0xAA and 0x55.  Those spikes exist in most player character IMCs...but are completely missing in the dying ones.  0x7A is also popular everywhere.  The first two IMCs have a big block of interesting stuff on the right starting at 0xE8, which is common in other files too; the second two see it starting at 0xE0.

Most stuff is well-distributed, including (frustratingly) 0xAA and 0x55.  There is an interesting clustering effect starting at 0xE? though, at least on Ebhard.  You get long areas with no 0xE0, or no 0xE8 or F0 or F8, then suddenly they're 10% of the file for a several tens of bytes.  It's pretty subtle still, and I haven't seen a great pattern to it...

But, changing any byte, nay, any bit at all in any byte whatsoever causes a LOT of damage to the image.  (The damage is limited to the region of the file that you modified, so, you'll only corrupt one frame of one animation often.)  So what the eff.

[traz]

I have started a blog, and posted a few ideas about a Darklands remaking project. I'm a coder, so my thoughts are mainly about the technical-side. Please visit my blog if you are interested: http://darkagesrpg.blogspot.com

[olemars]

Still looking at the sprite animations, and I'm definitely onto something. And it does indeed look like there is some RLE going on, and... something else.

Will report back soonish if/when I get anywhere useful.

[olemars]

Guess that counts as "anywhere useful". I've got the basics of the file compression down pat, as well as the frame index, and the individual sprite encoding.

Haven't figured out the palette data yet, nor the animation frame count/order, although the latter isn't terribly important. There's also a good bit of header values and other metadata I haven't found any use for yet either.

Palette data is the trickiest part. There are several palette files in the folder, but I don't know which chunks I need in them. Plus at least some of the sprites have some of their colors determined at runtime. Now I'm just creating a generic 256 color palette, the result of which is visibile in the picture. That's technically a sprite for the regular city gate guard (E02)

[G-Flex]

I have to wonder if even the developers of the game understood their character graphics code. After all, they offered a reward to anyone who could even give them hints for reproducing that weird bug that changed the color palettes on your characters.