Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[kutulu]

Hi,

Does anyone of a DwarfManager-like utility that runs on Linux?  Before I start working on one of my own I'd like to at least make sure I'm not reinventing the wheel...

--K

[DaveT]

I asked precisely the same thing a few days ago and it was suggested that I try and get it working with mono. Haven't got around to doing so yet but I took from that there isn't a native linux version anywhere yet.

[kutulu]

Well, you may be interested to know that I'm actually making good progress towards getting DwarfManager working in mono.  I've got the program detecting that its running on Linux and switching to /proc and ptrace(2) for memory access.  Not having a working IDE is killing me but once I can confirm that the memory layout of data is the same in the binary I should be set.

--Mike

[CautionToTheWind]

How is this going?

[AxelDominatoR]

I'm really interested in the linux porting, too. I don't have much free time, but eventually I can help, if needed ( I'm a programmer ).

[kutulu]

It's going ok, though it had to take a back seat this week to $DAYJOB, I should be back to working on it in a few days.  I've hit a bit of a snag: I can successfully attach to the dwartfort process and read it's memory space as I was hoping.  But a quick, brute-force scan through the memory space didn't turn up anything that looked remotely like dwarf names or professions (and obviously, the existing memory offsets used on Windows aren't helpful on Linux).  So it's just gonna take some grunt work to find the data in memory.

I also need to get in contact with the original author, to see if it's possible to get this into a branch in his svn repo for easier distribution.  If not, once I get something working I will try to put it up somewhere for testing.

--K

[AxelDominatoR]

About a month ago I did a little research on windows vs linux DF memory layout and it's really different.
I don't know how you're actually reading the process memory, but are you considering segments? Dynamic data should be in .bss segment.

[0x517A5D]

I have located the dwarven race index, the main creature vector, and the language vector in a disassembly of 40d11.

But I don't know how the actual process gets loaded into memory.

My disassembler maps the .bss segment starting at 0x08807500, is that constant or does it ever get mapped differently?  If it does change, is there a good way to determine the start of the .bss segment on a running program?

I'm uncomfortable using absolute memory locations, as that already bit me in Windows.  I would be much happier to use offsets within a program segment.

(Also, I don't currently have Linux installed on a machine powerful enough to run DF, so I can't do actual testing, just static analysis.)

[AxelDominatoR]

Sure, there's a way to correctly identify the .bss segment. You must read the virtual file:

Code: [Select]

/proc/<pidnumber>/mapswhere <pidnumber> is the process id of the dwarf fortress executable.
There you will find the address boundaries pertaining all memory areas mapped by the process.

The .bss is the third line.

You can get more info here: http://www.makelinux.net/ldd3/chp-15-sect-1.shtml. Scroll down to the "15.1.6. Virtual Memory Areas" chapter.

[kutulu]

Thanks for that link, it makes me feel better that I'm doing the right thing.  I have found the .bss segment in the maps file, and I'm having no problems reading data from it.  My problem is that I dumped the entire thing to a file and didn't find a single recognizable string anywhere.  Of course I'm using a .NET managed FileStream to read from /proc/pid/mem so its possible my reads are just messed up.  Monodevelop doesn't seem to currently have an integrated debugger, so I'll have to resort to mdb (blech).

--Mike

[AxelDominatoR]

I'll see if tomorrow I will be able to finish a little utility I started programming a few weeks ago in C. We could compare memory core dumps, then. ( I, too, was able to read the .bss, but didn't found nothing useful in it. I dedicated little time on it, however... )

[0x517A5D]

Okay, offsets in .bss seems to me like the way to go.

In 40d11-linux:

• DwarvenRaceIndex is at .bss+0xAA0684 and is a int16 (or uint16)
• CreatureVector is at .bss+0xAB7950
• LanguageVector may be at .bss+0xAFA270
• TranslationsVector may be at .bss+0xAFA288

Vectors have a different layout.  In MS x86 C++, they are:

• void * unused
• void * Start of the pointer array
• void * End of the pointer array
• void * End of allocated memory

In GCC x86 C++ they appear to be:

• void * Start of the pointer array
• void * End of the pointer array
• void * End of allocated memory

Strings are also very likely to have a different layout.  I haven't dug into that.

Most of the further work needs to be done on live data, which I can't do until I get Linux installed in a virtual machine, so... over to you guys.

[AxelDominatoR]

I managed to put together a really simple DF memory dumper, so now I can browse the live DF memory map with an hex editor.

It seems that 0x517A5D address list does not fit, however :/

I'm actually recalculating things to see if it's my fault.

[0x517A5D]

Okay, that is disappointing.

Well then.  Do this.  Search the .text segment for this sequence:

Code: [Select]

0f bf 15 ? ? ? ? 66 85 d2 78 18 8b 0d ? ? ? ? a1 ? ? ? ? 29 c8
The question marks represent any byte.

There are two occurances of this sequence.  Both work the same.

The four bytes in the first set of question marks is the address of the DwarvenRaceIndex.  The second group is the address of the CreatureVector.Start.  And the third is the address of CreatureVector.End, which is 4 bytes above CreatureVector.Start.

[kutulu]

Thanks for that link, it makes me feel better that I'm doing the right thing.  I have found the .bss segment in the maps file, and I'm having no problems reading data from it.  My problem is that I dumped the entire thing to a file and didn't find a single recognizable string anywhere.  Of course I'm using a .NET managed FileStream to read from /proc/pid/mem so its possible my reads are just messed up.  Monodevelop doesn't seem to currently have an integrated debugger, so I'll have to resort to mdb (blech).

Hooray for progress.

Good news: MonoDevelop clearly does have an integrated debugger, I'm just blind.

Better news: I was able to find the CreatureVector and LanguageVector just by grep'ing a memory dump looking for the words I knew would be there.

Bad news: The CreatureVector, at least, was in an area of memory above 0x7fffffff (in fact, the .bss segment went way up into the 0xf0000000's).  This is bad news because, currently, DwarfManager uses signed integers for all of its address variables.  This is why it never found anything during a "guess layout" scan -- the brute-force for() loop over the entire memory area was wrapping around into negative numbers long before it got there.  And since .NET rarely uses "uint" for any of its library methods, and doesn't provide a way to get an unsigned int out of it's "void *" interop structure, this will require plastering typecasts and/or changing parameter types in lots of code I was hoping not to touch.  But at least I know what to change...

--Mike