Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[peterix]

Trying to compile from the latest tarball () on Linux gives:

Code: [Select]

Scanning dependencies of target dfveinlook
[ 87%] Building CXX object tools/supported/CMakeFiles/dfveinlook.dir/veinlook.cpp.o
/home/matt/temp/peterix-dfhack-da2fb1c/tools/supported/veinlook.cpp: In function ‘void puttile(int, int, int, int)’:
/home/matt/temp/peterix-dfhack-da2fb1c/tools/supported/veinlook.cpp:103: error: ‘mvwaddwstr’ was not declared in this scope

Commenting out all instances of mvwaddwstr() made it compile.

More curses weirdness. Good thing I'm removing that bag of fail in the new dfhack... Anyway, this is a wide-character ncurses function. You need that library (and its -dev packages where applicable). Real problem here is that every distro packages these librariesslightly differently. Any solution to this problem will depend on what packages you have installed and how weird is ncurses in your distro.

I'm trying to figure out how to use dfincremental to determine:

1) Where and how the game stores info on wild beehives and ant colonies.
2) Where and how the game stores info on site-wide population counts (like how many turtles are left to fish before extinction)

Any advice?

Yes, first and foremost, get some decent tool for viewing process memory. Edb will do (evan's debugger), and will let you edit memory contents too. It's a bit sparse feature-wise compared to the windows equivalents, but it works... unlike most other things.

1) Should be quite manageable with the coord search, followed by backpointer search on the address(es) found. I already found those a few times, but never added the support into dfhack, mostly because anthills were just a flavor thing... might be interesting with the bees though

2) This is a bit tougher. The tool doesn't have number search for unknown values with relative increments/decrements between searches - only exact values. Shouldn't be too hard to add some kind of naive algorithm doing this, but the memory requirements with the current design (or non-design?) might be steep.
The information is also probably stored in some very non-obvious data structure (see Maps and the things that hang off the world-data address - mainly geology and local/global features. Some of it might overlap, and if you can backpointer search to it, you can save some effort.).

About the tool in general:
At first you are asked which memory regions to search. This is entered in a '##-##' format. For example '1-4' would search in regions 1-4. Make sure you include DF's static data and the heaps(mostly unnamed regions with read/write access that don't belong to any library). Omitting the DF code can remove some false positives when doing backpointer scan.

Then, pick what you want to do. Just about everything should work on linux, so that's fine. Most of the searches will ask for variable sizes, and increments between addresses to check. Most will also let you search, adjust things in DF, search again for new values, etc. (incremental search).

The backpointer scan starts with an address you give it and walks back from it, searching for pointers in the whole memory. It follows the path of those pointers... endlessly. It can be a bit slow, so make sure you build in Release mode. It also loves to go into endless goose chases across the whole DF memory. Unfortunately, I didnt include any way to stop it while it's running, so you'll have to CTRL+C out of the whole thing. Sometimes you'll have to combine it with a plain old number value search to get all the pointers to an address.

The coord search uses the position of DF's cursor to search for addresses, where the same position is stored. You can't use it to find the cursor, but placing the cursor over something you want to find, you can repeat the search incrementally. Works best with moving things obviously... and make sure there's nobody standing over the anthill or something.


If you have any further questions, check out the IRC channel. I'm around most of the time

[Khym Chanur]

2) This is a bit tougher. The tool doesn't have number search for unknown values with relative increments/decrements between searches - only exact values. Shouldn't be too hard to add some kind of naive algorithm doing this, but the memory requirements with the current design (or non-design?) might be steep.

I suppose I could, for example, for deer make it have [POPULATION_NUMBER:250:250] and when 3 deer show up search for 247.

About the tool in general:
At first you are asked which memory regions to search. This is entered in a '##-##' format. For example '1-4' would search in regions 1-4. Make sure you include DF's static data and the heaps(mostly unnamed regions with read/write access that don't belong to any library). Omitting the DF code can remove some false positives when doing backpointer scan.

Sometimes between libraries there's a memory segment which doesn't belong to libraries.  Is it just the memory up to the first library that's important?

The coord search uses the position of DF's cursor to search for addresses, where the same position is stored. You can't use it to find the cursor, but placing the cursor over something you want to find, you can repeat the search incrementally. Works best with moving things obviously... and make sure there's nobody standing over the anthill or something.

I tried that and got something like this:

Spoiler (click to show/hide)

[code]>>13
Select coord size (2,4 bytes) default=4
>>
search ready - position the DF cursor and hit enter when ready
>>
Searching for: 49:48:80
bytestream 12/16 bytes
31 0 0 0 30 0 0 0 50 0 0 0

Found single coord!
0x14d0fd74
>>
Searching for: 49:48:80
bytestream 12/16 bytes
31 0 0 0 30 0 0 0 50 0 0 0

pread failed: can't read 186036224 bytes at addres 179658752
errno: 5
Range 0xab56000 - 0x15cc1000 not readable.
search ready - position the DF cursor and hit enter when ready
[/quote]

So, first, there's the error.  Second, if there hadn't been an error, would the second time I hit enter have searched on from 0x14d0fd74 to try to find a second match?[/code]

[peterix]

Sometimes between libraries there's a memory segment which doesn't belong to libraries.  Is it just the memory up to the first library that's important?

Yes. It does belong to the library. Only it's the non-static memory bits allocated by the library AFAIK. Most of them can be safely ignored. libgraphics could be a very different deal, but it's unrelated to what you search for

I tried that and got something like this:

Spoiler (click to show/hide)

[code]pread failed: can't read 186036224 bytes at addres 179658752
errno: 5
Range 0xab56000 - 0x15cc1000 not readable.
search ready - position the DF cursor and hit enter when ready
[/quote]

So, first, there's the error.  Second, if there hadn't been an error, would the second time I hit enter have searched on from 0x14d0fd74 to try to find a second match?[/code]

This kind of search finds all the matches. The incremental part is related to incrementally reducing the search space, not walking some list of values one by one.

The error comes from a different source. My guess is, that you hit the linux limit of only one debugger attached to a process. When you do a search with the tool, it attaches, does the search and detaches again to accomodate running other tools. Edb isn't that nice about it and will just hog the process. Same with gdb and just about any other linux debugger.

Sometimes I get this urge to go into the kernel and rewrite the debugging API from scratch, and then make my own, actually *GOOD* debugger... The current API is riddled with terrible corner cases, bad design (Seriously, where's the threading support?) and black magic (It's something nobody seems to want to touch.) :<

[Khym Chanur]

About dfincremental's coord search, it always gives the exact same address, which I'm guessing is the game's memory location of the cursor.  If so, what use is the coord search?

The error comes from a different source. My guess is, that you hit the linux limit of only one debugger attached to a process.

Don't think it's that, since the only debugging process I have running is dfincremental.  If I'm doing a coord search, the error always happens just once per dfincremental session, usually on the first lookup, sometimes on a second lookup; after that, no errors.

[peterix]

About dfincremental's coord search, it always gives the exact same address, which I'm guessing is the game's memory location of the cursor.  If so, what use is the coord search?

The error comes from a different source. My guess is, that you hit the linux limit of only one debugger attached to a process.

Don't think it's that, since the only debugging process I have running is dfincremental.  If I'm doing a coord search, the error always happens just once per dfincremental session, usually on the first lookup, sometimes on a second lookup; after that, no errors.

Hmm. Can you pastebin / post the list of all the memory regions you get? This seems quite strange.

[Khym Chanur]

Pastebin of memory regions

[peterix]

Hmm... So, the thing that failed to read would be the heap. That's probably why you can't find anything. Looks like a bug, I'll have to investigate.

[Khym Chanur]

I tried adding access to the value, wall-tile and mined-out-tile for inorganic materials, but get t_maglossInorganic to compile properly was too much trouble.  So, for 0.31.25 Linux, here's the offsets (so my snooping around isn't a total waste):

* Value: offset 380, double word
* Wall tile: offset 524, single byte
* Mined out tile: offset 542, single byte

Offsets are decimal, not hexidecimal.


I noticed when looking in DF's memory via edb that the memory for the inorganic materials has junk data in it, I'm guessing because DF doesn't zero out newly allocated memory.  Is there any way to fix that, to make the real memory values clearer?

[Khym Chanur]

Okay, I've found the memory locations of some wild colonies, and doing a backpointers search on each pointer gives nearly the same address, so it looks like there's a vector of pointers to colonies.  However, if I give a colony pointer to the "pointer vector by address of an object" function, I get 584,357 results, which doesn't sound right.  If I give an address is what I think is the middle of the pointer vector to "vector by address in its array", it gives thousands of results.

So, how do I use dfincremental to find the address of the vector?

[Khym Chanur]

Also, if I make code to read and change wild colonies, should it be lumped with the creatures code, or should I put it in a new module?

[peterix]

Well, there should be a pointer to the start of that array of pointers. This is actually at the place where the vector itself is. So, just use the backpointer scan to find the beginning of that array. There's a chance that the pointer it jumps to next will be the vector, but to be safe, you can just feed the start of the array into the basic number search and get more pointers, if there are more pointers

And no, the creature stuff is tangled enough already I think. Better put it in a separate file.

[Aerval]

I am just playing a bit with dfliquids and used it to somehow "paint" a dwarven city in df with help of an ahk-script.

Spoiler (click to show/hide)

Could it be possible for dfliquids to paint other stones than obsidian or maybe at least somethings like ramps or fortifications (like in arena mode)?

[Rumrusher]

Could Dfhack take the designs of a players fort and map it to the other city sites designs so that if you visit a dark fortress the place will be model after a fort you work on before?
Though I guess first we need to know what tile spawns creatures first so we could paint in those tiles.

[Rose]

Aerval, obsidian ramps and fortifications are possible, but not other materials.

[danaris]

Could it be possible for dfliquids to paint other stones than obsidian or maybe at least somethings like ramps or fortifications (like in arena mode)?

Is there a FAQ somewhere that this could be added to?

Because it seems like every other week or so someone comes in and asks this question, and the answer is always the same...