Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[kutulu]

Ok, I think I'm down to the last missing item, but it's proving to be difficult -- the translation vectors.

So far, here is what I have found.  It has been consistent across every instance of df I've run, so anyone else who can dump or examine memory that can verify these would be greatly appreciated.  These are all 40d11, so they probably won't work on earlier versions:

In all cases, .text starts at 0x08048000, .bss at 0x08808000.

Code: [Select]

DwarvenRaceIndex is at 0x092a7b84, value = 0xa6
RaceVector.Start is at 0x093016b0
CreatureVector.Start is at 0x092bee50
LanguageVector.Start is at 0x09301770

As was posted earlier, the value at vector.start is a pointer to first entry in the vector.  (For the curious, the vector length appears to be at Vector.Start - 4).  Since these are vectors of objects, that location is itself a pointer to the object in question.  So far, in each case the first 4 bytes of the object is a std:string: Race name, Creature First name, or Language word.

The value in the field is a pointer to a null-terminated char array.  The length of the array is at offset -12 from that, and the capacity is at offset -8.

For the creatures, so far I have found the following fields:

Code: [Select]

[0]: FirstName (std:string)
[+4]: Custom Nickname (std:string)
[+38]: Custom Profession (std:string)
[+44]: Race Index (int) -- for dwarves = 0xa6 (from DwarvenRaceIndex).

I assume the last name starts at or near offset +8 but the last names are actually english words read from the dwarf translation table in the translation vector, and I haven't found that yet.  I'm still trying to work out the structure of the translation objects and word tables, which don't match up with what the Win32 C# code is doing to read them in.

For those playing along at home, though: the first translation in the vector should be the DWARF one, and the first words in the DWARF language table are kulet ("abbey"), alak ("ace"), and bidok ("act").

More to come...

[CautionToTheWind]

I must confess i visit this thread every day looking for a download link! I greatly appreciate that you are working on this and am planning a great DF marathon when this project succeeds.

[AxelDominatoR]

kutulu: I just compared the addresses with my memory dumps.
They all fits perfectly, so I can confirm it working here.
If I have some free time this evening I will try playing a bit with the translation vector.

[0x517A5D]

Even better news: the addresses also line up with my static analysis (disassembly).

In the Windows version, the translations vector is almost right after the language vector.  There is one unidentified vector between them.  So, as GCC++ vectors are three pointers long, try the translations vector at 0x09301788.

And, IIRC, it is a vector of vectors, 4 members, 1 for each language.

Also what screwed me up about creature/race vectors is that in the raws, a race declaration is started with CREATURE.

Other address of possible interest (40d11_linux):

• 0x09301668  Matgloss vector
• 0x09301698  Body parts vector
• 0x093016BC  Item type vector
• 0x09301764  Entity vector
• 0x09305B14  Descriptor vector (colors and shapes)

Those all found by looking at the raws OBJECT parsing code.

[0x517A5D]

A bad thought: Linux DM is NOT going to be able to set custom profession names.  That's because doing so would require allocating memory via DF's string memory management mechanism.  Allocating memory in DF from outside is not enough, because the memory manager won't know how to handle that memory when it's time to free it.  So can't do it.  Unless someone can figure out how to call into string::reserve or similar from DF's context.

Windows DM can do this because MSVC++ has special handling for strings of 0 to 15 characters -- it overloads them into the string object itself instead of having a pointer to them.

Modifying custom profession names should be possible, with the restriction that the new name needs to be the same length or shorter than the old.

[kutulu]

A bad thought: Linux DM is NOT going to be able to set custom profession names.  That's because doing so would require allocating memory via DF's string memory management mechanism.  Allocating memory in DF from outside is not enough, because the memory manager won't know how to handle that memory when it's time to free it.  So can't do it.  Unless someone can figure out how to call into string::reserve or similar from DF's context.

Windows DM can do this because MSVC++ has special handling for strings of 0 to 15 characters -- it overloads them into the string object itself instead of having a pointer to them.

Modifying custom profession names should be possible, with the restriction that the new name needs to be the same length or shorter than the old.

So in VC++ even if the string isn't set, there are 24 or so bytes allocated to the object?  In my gcc memory dumps all of the unset strings are set to the same null-pointer value, but the Win32 code seems to assume it can just write data into the custom nickname/profession spots at will.  I wonder what happens if I try to set a custom profession name longer than 16 characters...

--K

[codewright]

I'm happy there is such a high interest in making DF work on Linux. Regarding setting string properties: I guess i can be happy that the windows version has always 16 chars reserved. You are currently not able to extend this cap from within DM, because of the same reason (memory allocation). However, if you already gave a longer name in DF, DM will be able to use that. So it's not that great either, but it's enough for most cases, i guess.

Did anyone play around with compiling DM with Mono? I wonder if the 3.5 features make troubles, including LINQ.

[0x517A5D]

So in VC++ even if the string isn't set, there are 24 or so bytes allocated to the object?

Yeah.  I didn't actually open my disassembly to verify it, but I think a std string object is 28 bytes.

[kutulu]

I'm happy there is such a high interest in making DF work on Linux. Regarding setting string properties: I guess i can be happy that the windows version has always 16 chars reserved. You are currently not able to extend this cap from within DM, because of the same reason (memory allocation). However, if you already gave a longer name in DF, DM will be able to use that. So it's not that great either, but it's enough for most cases, i guess.

Did anyone play around with compiling DM with Mono? I wonder if the 3.5 features make troubles, including LINQ.

Yes, it compiles and runs just fine under Mono.  Mono doesn't support WPF or LINQ yet, but DwarfManager doesn't appear to use either of those things.  All I had to do was remove the project reference to System.Windows.Presentation.dll and it builds fine.

Of course I couldn't do much with it, since I get an EntryPointNotFound error the minute it tries to load anything from kernel32.dll.  What I'm using right now for testing is just a patched version of your latest from svn, with a Linux specific MemoryAccess class derived from the Win32 one, but I haven't gotten to the point where I can maniuplate data from the UI.

[codewright]

I'm happy there is such a high interest in making DF work on Linux. Regarding setting string properties: I guess i can be happy that the windows version has always 16 chars reserved. You are currently not able to extend this cap from within DM, because of the same reason (memory allocation). However, if you already gave a longer name in DF, DM will be able to use that. So it's not that great either, but it's enough for most cases, i guess.

Did anyone play around with compiling DM with Mono? I wonder if the 3.5 features make troubles, including LINQ.

Yes, it compiles and runs just fine under Mono.  Mono doesn't support WPF or LINQ yet, but DwarfManager doesn't appear to use either of those things.  All I had to do was remove the project reference to System.Windows.Presentation.dll and it builds fine.

Of course I couldn't do much with it, since I get an EntryPointNotFound error the minute it tries to load anything from kernel32.dll.  What I'm using right now for testing is just a patched version of your latest from svn, with a Linux specific MemoryAccess class derived from the Win32 one, but I haven't gotten to the point where I can maniuplate data from the UI.

That sounds like good news. It really doesn't use WPF, guess that ref is a left-over from some testing, but it does use LINQ on objects (no SQL or other things). So I guess Mono can handle that rather easy part of LINQ.

Keep up the nice work!

[kutulu]

More progress -- I've found the translation vectors.  This was the last major stumbling block to reading an entire dwarf, now I just need to confirm that the labors and skills are where I think they are.  Hopefully by next week I will have a utility that can read dwarves from the running dwarfort process for anyone who wants to test it.

For reference, here is what I have (all the offset numbers are in hex so "address + 20" is 32 bytes past address.)

Code: [Select]

DwarvenRaceIndex: At 0x092a7b84 (bss + a72b84)
RaceVector: At 0x093016b0 (.bss + af96b0)
LanguageVector: At 0x09301770 (.bss + af9770)
CreatureVector: At 0x092bee50 (.bss + ab6e50)
TranslationVector: At 0x09301788 (.bss + af9788)

Vectors look like:

Code: [Select]

  [address - 4] Length
  [address    ] Start
  [address + 4] End

Strings look like:

Code: [Select]

  [address - c] Length
  [address - 8] Capacity
  [address    ] Characters

The Dwarf object, so far:

Code: [Select]

  [address     ] FirstName (string)
  [address + 4 ] Nickname (string)
  [address + 8 ] First index for LastName
  [address + 20] Last index for LastName
  [address + 3c] Custom Profession (string)
  [address + 44] Race Index

The indexes for the lastname are indexes into the word table for the dwarf language translation.

Translations look like:

Code: [Select]

  [address     ] Language Name (string)
  [address + 1c] Address of Word Table
  [address + 20] Address of End of Word Table

The value I have listed as "address of end of word table" always seems to appear in offset 20 and 24 in the translation, and it points to the entry one past the last actual word table entry.  Possibly some kind of stack smashing protection or something obvious I'm just not getting, but otherwise, the word table is just an array of strings.

Now that I can read most of the dwarf out of memory, I'm going to work on writing back custom nicknames and professions.  Since gcc doesn't give me a buffer to work with unless I set one inside DF, I also need to add some kind of sanity check to prevent the user from trying to write a profession name back over top of the null pointer.

Stay tuned...

[Gertack]

Given sufficient black magic, you can allocate memory in the tracee.  Think manual esp manipulation and forcing the eip to emulate a function call.  Rather more complicated given they're C++ objects, however.  For example, GDB can do function calls in the debugged program's context.

[Kiberkiller]

Excuse me but
1) i really want a DF program (that helps me arrange labors) for Mac OS X
2) i've hear that linux stuff should work on Mac
i'm not really into programming (scripting doesn't count) so i wonder if it's possible to make mac version of this?

[AxelDominatoR]

Mono runs under Mac OS X so, after finding the right addresses for the mac DF binary, I think it should be possible to run Dwarf Manager on it.

[CautionToTheWind]

*cheers*