Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[olemars]

I think the programmer in charge of this bit of the game was a follower of the "job security through code obscurity" school of software development philosophy. The compression is completely homebrew. I'm pretty sure it's hand-tailored in asm. The battle code will also open and read the same chunks from some files 5-6 times in a row for no apparent reason.

[Sowelu]

Olemars, you're my best friend ever.  And that is one of the biggest boons to all Darklands remake efforts in probably the last six years.  Could you toss some of that code my way?

I'm thinking the best action from here might be to manually figure out the palette for each sprite, because that stuff is just plain tricky.  But ripping all that stuff will be totally great.

How consistent are the sprite sizes and hotspots?

Oh yeah, and TOTALLY post that stuff on the Darklands yahoo group.  You will be loved for all time.

[olemars]

I'm organizing my data and making a nice little app for reliably decompressing and extracting the sprites. I'll also write up a description of the algorithms in use.

I'm not sure what you mean by hotspots but sprites can be any size really (within the bounds of reason anyway). The one i posted is 14x30, while another in the same file is 19x30. Some of the attack sprites are fairly tall, up to 16x50 or something, especially the "overhead chop" attack. I don't think they're even the same size within the same animation sequence, although it's mostly just one pixel width/height plus or minus.

For the human walk animation files there are 72 sprites, which means 9 frames per direction, or 8 frames + 1 standing still. That also means there are several thousand of these little sprites in total. Pixel artists must be the most patient people on earth.

[Sowelu]

OH yeah.  Patience is the single biggest virtue of any artist, not just a pixel one.  They are probably the second most patient people on earth, right after the English guards with the tall fuzzy hats.

Hotspots is the term I've heard used to describe a constant point relative to the world.  For example, "the floor directly under the center of the actor".  The inverse of the offset you use to draw it to the screen.  If your sprite has a large backswing or a large thrust, then "the center of the sprite" is meaningless.  There's probably two bytes for the hotspot data, I would guess; if I was creating the format, I'd expect one of them to be close to 1/2 the width of the frame, and the other to be either a constant very low number (if measured from the bottom) or just slightly below the height of the sprite (if measured from the top).

Any ideas where I should look for the combat background images?  IE floors, walls etc.  Hopefully the format will be a lot simpler...

[olemars]

Don't have access to the files now (at work), but probably IMAPS.CAT or BC (which is also a cat file). I think those contain only pic files.

[Sowelu]

They're not PIC files to my understanding.  PIC files start with "M0" or "X0", and none of the stuff in either of those does.

IMAPS.CAT contains a bunch of stuff numbered like .302 and stuff, which each start with 0xA1.
BC contains stuff named like .FFC, .FLC, .FRC, .NWC, .WWC, and .NFC.  They have a much less standard format.  (all just at a glance)

[olemars]

Right. More formats to dissect
There are some other "hidden" catalogs too; LCASTLE and a few files called DARKLANDS.something. The game really has a lot of assets.

EDIT: The files in BC and IMAPS.CAT are all compressed with the same algorithm as the IMC files. Hopefully they'll make more sense after decompression.

[sluissa]

Amazing work, Olemars. I don't know how much time you actually put into this, but I might just rank "dissecting homebrew compression techniques by hand." up there somewhere near Artist and Fuzzy hat guy.

[Sowelu]

Quadko from the Darklands mailing list has given permission to "take, use, adapt, extend, subvert, copy wholesale, and
build upon" his stuff.  Considering that he's written a "playable" overworld map and town card system, this is pretty awesome.

...At some point, I want to write an importer.  There's some tools out there that will read your saved games and print out stuff like location, quests, skills etc.  Might as well import old saved games into the remake.  How cool is that:  "Try out your old party against an arbitrary collection of critters, to test the battle system".  It would be even cooler if we could write back out to the same format (sans remake-specific values) just for yuks, but.....no.

Hey, I wonder if they used that same compression format for the music?  I think people were having a hard time ripping the Darklands music at one point.

[traz]

please do not reimplement te green-hair bug.   

[sluissa]

Hey, I wonder if they used that same compression format for the music?  I think people were having a hard time ripping the Darklands music at one point.

I'm not sure why it would be easier to try to rip the music out of the game rather than just play the game and have something record the sound as you're playing it. There isn't that much music in the game and it's all rather repetitive.

[olemars]

Hey, I wonder if they used that same compression format for the music?  I think people were having a hard time ripping the Darklands music at one point.

If the .PAN files are the music files, then yeah it looks like it's probably the same compression. They all end with 0x00 0xF0 0x00, which acts as an EOF marker of sorts for the algorithm (it's also the only marker available to sort of identify these files, which is a bit painful, even the headers are compressed...). Guess I'll have to expand the scope for my file tool a little.

On a related note, I think I have a working implementation of the algorithm now. My testfile matches 100% with one I dumped from the game's memory.

[Sowelu]

Hey, I wonder if they used that same compression format for the music?  I think people were having a hard time ripping the Darklands music at one point.

I'm not sure why it would be easier to try to rip the music out of the game rather than just play the game and have something record the sound as you're playing it. There isn't that much music in the game and it's all rather repetitive.

Okay, that's a good point.  As long as it grabs the midi, and not the wave output.  ...Okay, hell, even the wave output would be nice, but why do that when you can intercept the original without too much extra trouble?

My combat guys chase each other around.  I am hoping that by the end of this week I can post a crummy playable combat sim.

[olemars]

Here's a WIP version of the file tool, with at least some basic functionality. I figured it would be a nice opportunity to brush up on my Model/View cred, so it has a tree view of the files you open and a preview for sprites (at 5x zoom). It only supports reading CAT files, decompressing IMC files and extracting animation sprites so far.

You open CAT files through the file menu, and right clicking on a file in the tree view gets you some export options. "Export all children" dumps all the subcontents of a file into an equivalent folder hierarchy. If you do this on a CAT, it could mean a lot of files since each IMC file can have up to 70 sprites and there's 50-70 IMC files in each CAT.

Download link.

Only dependency is Qt, which is bundled. And of course the Visual Studio 2008 C++ runtimes. If you get errors about "Side By Side" "R6024", "the application is configured incorrectly" or something similarly useless, you're probably missing the runtimes. Just google "MSVC 2008 SP1 Redist".

As for stability, it gets the "works for me" stamp of approval. I'm so confident in my own programming abilities* that there's hardly any error checking/handling, which I feel is in the spirit of the original Darklands developers. Use at your own risk (there isn't really a lot that can go wrong though).

I'll put it on github or something eventually, I just want to add support for more file types first, palette editing and maybe sprinkle some comments on the code. Might as well add PIC viewing functionality too since the decode algorithm is available anyway.

Side note: At least in my version of the game there seems to be a single corrupted animation file, M40WKWH.IMC, which apparently is the "wild hunter", or this fella:

It's a giant squirrel with antlers and a superman costume, right? Would be interesting to know if that file is corrupt for someone else too. You'll get an error message if it is. It could be a hickup in the decompression algorithm, but in that case this is the only file it breaks on. I can probably salvage most of the file though with some hex editing, it's just a frame or two at the end that's missing.



*lazy

[Sowelu]

Toooottally awesome.  Thanks!  I'll put this stuff to good use...