Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

Me

16 (22.9%)

Me!

54 (77.1%)

Total Members Voted: 69

[Halfling]

News: Got a pm from sackhead - removing CAN_LEARN from mamaphints instead to fix these issues is okay by him. Should add natural skills to make them skilled instead.

[WillowLuman]

I'd say Skilled Fighter, Competent axeman, spearman, and swordsman, plus Proficient misc object user.

[vyznev]

A sort of a conceptual question that occurred to me is, what should the scope of this project be?  What I mean, in concrete terms, is, what parts of Dwarf Fortress are we replacing?

We started out by wiping the raw/objects folder and recreating it from scratch, but with the addition of the troll speech file, we've now expanded into data/speech.  Also, while looking at what else there is under the data folder, I figured out a nifty new (at least to me, might be old news to others) modding trick:



More pragmatically, should we amend the installation instructions?  "Delete your raw/objects folder and unzip this file in its place." doesn't really work anymore if we start messing with the data folder too.

[Halfling]

Scope - replace anything we feel like replacing eventually, so long as it works and is cool. For some definition of works and cool. So long as it contributes to the experience, but that's up for the turn holder and community to evaluate. And preferably doesn't require external utilities. That's how I see it.

That window frame looks a lot like aurorae. Now, I'm not sure we should be editing those into anything specific for a given race, but - what's the correct command to open said files?

Considering multiple races are playable and do everything, now that I think of it, replacing fort mode dialog screens with something more neutral would be desirable.

[WillowLuman]

Yes, that would be good.

AFAIK, we're just replacing raws and adding some new data files, possibly editing them. We want to make something that people can install if they only have a fresh copy of DF, with a little file-moving. No external programs or hacks, just a new raw folder and unzip/replace the data folder.

[WillowLuman]

Incidentally, how did you edit that file?

[Halfling]

The files are in zlib-compressed blocks. There's some ancient, unacceptable (obscure, third-party binary without source) windows utilities that I shouldn't hence help propagate for editing them. Since he appears to be using Unity Ubuntu I assume he has a better way to access them.

[WillowLuman]

I never figured out how to use that utility putnam made. Damn. I guess vyznev can do it for us.

[Halfling]

Anyone can use zlib, it's free, usable on anything such as a phone, and there's a working code example on the site (assuming that's correct information that that is the algorithm). There's ample documentation too. Just needs some delving into that I personally didn't find time for on how to uncompress these files. That's why I'm a little miffed at the creators of the original so far to be unnamed utilities. They could have just told us how to do it instead of distributing a stupid chunk of useless binary. I mean how is that advantageous, unless you're looking to make some kind of name and get downloads for your file?

Ah well... but since he can do it and is interested, so much the better. Pm'd about it too.

[WillowLuman]

"provide for your people" would do nicely, I think.

"Dig that dirt!" doesn't quite have the dramatic appeal of "Strike the Earth!", and neither match that not all play styles in this mod revolve around digging. Though, even for aboveground constructions, some "Striking the earth" is involved, i.e. groundbreaking, so perhaps it still works.

"A new chapter of your history" or "your people's history" could be what we're looking for.

[Putnam]

I never figured out how to use that utility putnam made. Damn. I guess vyznev can do it for us.

I didn't make it, I just spam it everywhere whenever people ask and use it for a couple of my mods.

[vyznev]

Considering multiple races are playable and do everything, now that I think of it, replacing fort mode dialog screens with something more neutral would be desirable.

Yeah, the screenshot above was really just a test to see if it worked at all. We really should come up with something that works for any playable race. (It would also be nice to know what substitution strings work; with luck, we might be able to get, say, the name of the race inserted into the text.)

Incidentally, how did you edit that file?

I made a pair of Perl scripts to convert them to plain text files and back:

Spoiler: Perl scripts (click to show/hide)

Code: (df_announce_decode.pl) [Select]

#!/usr/bin/perl
use strict;
use warnings;
use Compress::Zlib qw'uncompress $gzerrno';

undef $/; # slurp whole files
FILE: while (<>) {
    my @chunks = unpack "(V/a)*", $_;
    warn "*** found ".@chunks." compressed chunks in $ARGV\n";
    CHUNK: for my $i (1 .. @chunks) {
my $data = uncompress $chunks[$i-1];
if (not defined $data) {
    warn "*** uncompressing chunk $i failed: $gzerrno\n";
    next CHUNK;
}
(my $n, $data) = unpack "Va*", $data;
warn "*** found $n strings in chunk $i\n";
my $j = 1;
while (length $data) {
    (my $m1, my $m2, $data) = unpack "Vva*", $data;
    if ($m1 != $m2) {
warn "*** length mismatch for string $j: $m1 != $m2\n";
next CHUNK;
    }
    elsif ($m1 > length $data) {
warn "*** string $j should have $m1 bytes, but only ".length($data)." remain in chunk!\n";
    }
    (my $str, $data) = unpack "a[$m1]a*", $data;
    s/\\/\\\\/g, s/\r/\\r/g, s/\n/\\n/g, s/\t/\\t/g for $str;
    s/([^ -~])/sprintf "\\x%02X", ord $1/eg for $str;
    print "$str\n";
    $j++;
}
    }
    print "\n";
}


Code: (df_announce_encode.pl) [Select]

#!/usr/bin/perl -w
use strict;
use warnings;
use Compress::Zlib 'compress';

# read input: 1 line -> 1 packed string, skip blank lines
my $n = my @lines = grep /\S/, <>;
chomp @lines;

# decode \t, \r, \n, \\ and \xNN escapes in input
my %map = (t => "\t", r => "\r", n => "\n", "\\" => "\\");
s/\\([trn\\]|x([^0-9A-F]{2}))/$map{$1} || chr hex $2/egi for @lines;

# prepend length to strings (twice, for some strange reason)
$_ = pack "Vva*", length, length, $_ for @lines;

# concatenate strings and prepend count, then compress
my $packed = pack "V(a*)*", $n, @lines;
my $zip = compress $packed;

# prepend length of compressed data and print
print pack("V", length $zip), $zip;
Usage example:

Code: [Select]

perl df_announce_decode.pl df_linux/data/save/fortressintro > /tmp/fortressintro.txt
gedit /tmp/fortressintro.txt
perl df_announce_encode.pl /tmp/fortressintro.txt > df_linux/data/save/fortressintro
The decode script can read multiple files (and will separate them in the output by a blank line). The encoding script just does one file, although it wouldn't be hard to change that.  Note that the decoding script, in particular, is quite a bit more complex than it really needs to be, because I wasn't sure about the format and wanted to make sure it could handle any surprises.

That's why I'm a little miffed at the creators of the original so far to be unnamed utilities. They could have just told us how to do it instead of distributing a stupid chunk of useless binary.

Agreed. So here's the meat of it:

• Each file contains a 4-byte little-endian length tag, followed by a zlib (RFC 1950) DEFLATE stream.
• The decompressed stream starts with a 4-byte count, followed by the indicated number of text records.
• Each text record starts with two little-endian length tags, first a 4-byte one and then a 2-byte one. Both contain the same value. Don't ask me why. These length tags are followed by the indicated number of bytes of text.
• Each file seems to contain at least three text records, sometimes more. The first record just repeats the name of the file, the second contains the title, and the remaining records contain the body of the announcement. There doesn't seem to be any obvious logic in how the body is split into records; I suspect the splitting points may simply correspond to manual line breaks in whatever text file these were originally generated from, and my scripts behave accordingly (i.e. they map each record to one line, and vice versa).

For example, here's the decoded version of data/announcement/fortressintro from vanilla DF, with one line per record:

Code: [Select]

fortressintro
[TITLE]A Dwarven Outpost[/TITLE]
[P]You have arrived.  After a journey from the Mountainhomes into the
forbidding wilderness beyond, your harsh trek has finally
ended.  Your party of seven is to make an
outpost for the glory of all of [VAR:NATIVENAME:GLOBAL:YOURCIV].
[P]There are almost no supplies left, but with stout labor
comes sustenance.  Whether by bolt, plow or hook, provide
for your dwarves.  You are expecting a supply caravan just before
winter entombs you, but it is Spring now.
Enough time to delve secure lodgings, ere the [VAR:NAME_PLURAL:GLOBAL:LOCAL_LARGE_PREDATOR] get hungry.
A new chapter of dwarven history begins here at this place, [VAR:NATIVENAME:GLOBAL:YOURFORT],
"[VAR:TRANSLATEDNAME:GLOBAL:YOURFORT]".  Strike the earth!

Things to test:

• Can there be more than one DEFLATE stream per file? The presence of the length tag would allow it, but I suspect it may just be there to simplify the input code.
• What happens if the first record doesn't match the file name?
• Does the title need to be a separate record, or can it share a record with the body? What happens if you have more than one title? No title?
• What tags can we use in the body? The decoded text above shows a few, but are there more?

Ps. I just tested, and it seems the Perl scripts above can decode all the files under data/announcement, data/dipscript and data/help folders. The data/index file is also in the same format, but the strings in it have been further obfuscated somehow, and I haven't fully figured out the method yet.  (Subtracting the first byte of each string from 255, the second from 254, etc. seems to be enough for the first few bytes, but then it breaks down.)

Edit: The subtraction trick works, except that after the fifth byte, you loop back to 255 again.  In Perl code:

Code: [Select]

my @str = unpack "C*", $str;
for my $k (0 .. $#str) {
    $str[$k] = 255 - ($k % 5) - $str[$k];
}
$str = pack "C*", @str;

And here's the decoded index file:

Spoiler (click to show/hide)

Code: [Select]

index
1~data\help\icons
2~data\announcement\diplomatrebuffed
3~data\announcement\merchantintro
4~data\announcement\semiend
5~data\announcement\invasion_elves
6~data\announcement\hastyking
7~data\announcement\kingarrival
8~data\announcement\invasion_goblin
9~data\announcement\invasion_human
10~data\announcement\merchantexit
11~data\announcement\fortressintro
12~data\help\main
13~ELVES_FIRSTCONTACT
14~HUMAN_STANDARD
15~DWARF_LIAISON
16~HUMAN_TRADE
17~ELVES_STANDARD
18~v0.34.11
19~
20~Programmed by Tarn Adams
21~Designed by Tarn and Zach Adams
22~Visit Bay 12 Games
23~www.bay12games.com
24~Dwarf Fortress
25~Adventurer
26~data\announcement\end2
27~data\announcement\end3
28~data\announcement\end4
29~data\save\permits
30~data\help\a_main
31~Legends
32~Reclaim Fortress
33~data\help\r_main
34~data\initial_movies\bay12games.cmv
35~data\initial_movies\toadyone.cmv
36~data\initial_movies\dwarf_fortress.cmv
37~data\movies\last_record.cmv
38~Really?  You're playing like a peasant.
39~Really?  I thought you should quit.
40~Slaves to Armok:  God of Blood
41~Chapter II: Dwarf Fortress
42~Dwarf Fortress
43~Copyright (C) 2002-2012 by Tarn Adams
44~Welcome to the alpha of Dwarf Fortress.
45~As there has been some time between releases, instability is to be expected.
46~Report crashes, hangs, lags, bugs and general disappointment at the forums.
47~They are at our website, bay12games.com.  Check there for updates.
48~You can also find an older yet more stable version of the game there.
49~As of June 2012, you can get help at the fan-created dwarffortresswiki.org.
50~Please make use of and contribute to this valuable resource.
51~If you enjoy the game, please consider supporting Bay 12 Games.
52~There is more information at our web site and in the readme file.


I'm not sure how all those lines relate to each other, but I do see some modding potential here.

[Vgray]

Should someone fix the entities so that, for example, Formic sword-users are "Swordformics" instead of Swordsmen?

[vyznev]

Should someone fix the entities so that, for example, Formic sword-users are "Swordformics" instead of Swordsmen?

Swordsbugs?

[WillowLuman]

snip

Try decoding the invasion announcements and the caravan ones, see which variable refers to race names.

Should someone fix the entities so that, for example, Formic sword-users are "Swordformics" instead of Swordsmen?

That's in the creature files. And I don't think it's really necessary. Linguistically, "man" just means "person" or "male person" in many languages. As seen in many ancient languages, when people were even more seperate, they didn't think of people from elsewhere as the same "species", and so had other names for them, like "men who wear their beards this way" or "men whose language sounds like 'barbararbarbar to us' ", but still called them men.

So, it's conceivable that a fantasy culture might see the same difference between themselves and a nation of foreign humans and a nation of foreign lizardmen.

But point is, the roots of words like "man" or "human" are as descriptions of people. Not wanting to dictate to STL here, but I'm guessing the Formic word for themselves would more likely translate to "person" than "bug". Or at least, more like "nation" than "nest".

Terry Pratchett puts it better than me. He has this story called "Only you can save Mankind", where some aliens ask this guy to save their race, but the universal translator renders their name for themselves as "mankind" and their name for their planet as "earth", because those are our words for "all our people" and "our planet"