Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[peterix]

I honestly wonder why they were path finding, though.  If you can walk through everything, can't you just travel in a straight line?

I don't know, this is just my guess Maybe to make them behave more like actual dwarves?

[Arekis]

Hey folks.  A DFHack newbie here.  I'm considering writing a little util for trading.  So, I'm trying to figure out some stuff with the item vector and I'm running into...  complexity.  From what I understand, the item definition is not complete.  Most of what I want is there, but there are two things missing: marked for trade and the value of an item.  Any advice on getting started/further along?  I seem to recall there were some memory tests someone did a while ago, but I can't find those pages.

On a marginally related topic, the latest version that seems to work with the items_vector definition is 31.16.  I get an exception telling me it is invalid when I run it with later ones (18, 21) or not set (14).  Does this mean that the items_vector value in the memory.xml is not correct for the other versions or am I just doing something wrong?

[peterix]

Hey folks.  A DFHack newbie here.  I'm considering writing a little util for trading.  So, I'm trying to figure out some stuff with the item vector and I'm running into...  complexity.  From what I understand, the item definition is not complete.  Most of what I want is there, but there are two things missing: marked for trade and the value of an item.  Any advice on getting started/further along?  I seem to recall there were some memory tests someone did a while ago, but I can't find those pages.

That's quite a task! See, item value depends on the raw values of the base item type and the value of the material. We have neither. They would have to be found I guess... unless there's also a value field inside the item objects too.

On a marginally related topic, the latest version that seems to work with the items_vector definition is 31.16.  I get an exception telling me it is invalid when I run it with later ones (18, 21) or not set (14).  Does this mean that the items_vector value in the memory.xml is not correct for the other versions or am I just doing something wrong?

No, you aren't doing anything wrong. Offsets marked as invalid are just... not valid. The items vectors should be easy enough to find and I can add those. I'm not entirely sure about the other values... they are offsets into the virtual method table of the item classes. The Items module reads their code and evaluates it to some extent. This isn't necessarily good or bad, but for it to be really useful, it would need to be able to interpret assembly much better, or we need a way to call the code directly (there are people working on this).

The 'not set' comes from version 31.13, where Toady changed the compiler. The 31.16 was the first stable version after that.

[Arekis]

Hey folks.  A DFHack newbie here.  I'm considering writing a little util for trading.  So, I'm trying to figure out some stuff with the item vector and I'm running into...  complexity.  From what I understand, the item definition is not complete.  Most of what I want is there, but there are two things missing: marked for trade and the value of an item.  Any advice on getting started/further along?  I seem to recall there were some memory tests someone did a while ago, but I can't find those pages.

That's quite a task! See, item value depends on the raw values of the base item type and the value of the material. We have neither. They would have to be found I guess... unless there's also a value field inside the item objects too.

That's what I was naively hoping for: to scrape through the data and find a value field.  For the "trading" toggle I was going to compare what the item data looks like before and after toggling that mark, that is assuming the flag is stored with the item.  If it's in another vector somewhere...

[Quietust]

How to determine an item's value:

1. Determine the item's base value. The item type is returned by a function call. Use the item type to determine the base value (in most cases, it's 10).
1a. If the item is a type of weapon, armor, trap component, or tool, then you'll need to figure out how the game determines the item value. Good luck with that.
2. Determine the item's material value. The item's material is also returned by a function call. Once you have the material (and know whether it's a builtin, an inorganic, a creature, or a plant), then you can fetch the material object itself and grab the value from the appropriate offset. I'm not certain exactly where all of those material objects are located, but inorganics are known and builtins can be handled easily enough with a lookup table.
3. Determine the item's quality multiplier. The item's quality is also returned by a function call.
4. Multiply the results from steps 1-3 together to get the item's value.
5. Locate the item's decoration vector. I don't recall, but this is probably also returned by a function call. Iterate across all entries within it and repeat steps 2-3 to find the value of each decoration, then add them to the item's value.

For values returned by function calls, there are several ways of getting at the value:
1. Read the function pointers out of the object's VTABLE and examine the code they point at, then interpret the code to determine what they'll return. DFHack has "accessors" which do this, though I don't know if they still work with the current version.
2. Use the item's VTABLE pointer to determine its type, then use hardcoded lookup tables to get the relevant values/offsets for each item type. This would tie the utility to a very specific version of Dwarf Fortress on a single platform.
3. Inject code into Dwarf Fortress (as a separate thread) which calls the subroutine with the appropriate parameters and places the result at a predetermined address. I've actually used this method myself in a tool I wrote for Dwarf Fortress v0.23.130.23a (in the old 2D versions, designating items for Melting involved adding them to a vector rather than setting an item flag).

[LordBadger]

I feel like this could be simple, but I just can't figure it out. I guess for around the last two versions of Dfhack any tool that asks Dwarf fortress to preform an action generates an error:

Spoiler (click to show/hide)

AppName: dfreveal.exe    AppVer: 0.0.0.0    ModName: msvcr100.dll
ModVer: 10.0.30319.1    Offset: 0008d635

To some extent it's the same error every time, with tools that effect the game as it plays. The only tool that seems to work is Dfprospector, though I think that may be 'cause it does not wright anything into the game. So that's it, if the question has already been asked just give me a link. Also The readme has been read, C++ reinstalled, and the forums have been searched... In my frustration I may have overlooked something. Great set of tools though, and the addition of Deramp is just ducky, in fact it's the main reason I want to get everything working.

So I clean the house, have a soda and decide to trouble shoot some more. The issue I'm having has nothing to do with the tools... Something that should have been extremely clear to me from the error (Egg on my face) this is more of an issue my computer and C++ are having. I uninstall C++ (This is the version from the read me) and I still get the same errors making it look as if it had never been installed in the first place. Anyway my apologies, question retracted.

[danaris]

What does "memory object not set: type offset key /string/MSVC/buffer" mean?
I get it whenever I run any of the tools. I'm running this on Wine with Ubuntu 10.10.

This means something is very broken. None of the recognized windows DF versions have this offset /unset/. What exactly is running inside wine? DF AND the tools? If so, what was the last dfhack version that worked for you?

I get the same thing under Wine on Mac OS X.  There's also a line above it that reads

Code: [Select]

fixme:heap:HeapSetInformation 0x0 1 0x0 0
I have DF running under Wine, works just fine as usual, and this happens when I try to run any of the dfhack tools under Wine as well (at least, I presume any of them; I haven't gone through and tried each one ). 

Last version of dfhack that I tried to use was 0.5.1, as that was the version needed for DF 0.31.18, which was the last version of it I was running.  My experiences with it were mixed, but generally acceptable.  Specifically (in case you care):

- dfvdig worked pretty consistently, but would occasionally crash out
- dfprospector never gave me any troubles
- dfliquids worked about as often as dfvdig
- dfcleanmap and dfreveal never worked properly for me under Wine, so I just fired up my Windows virtual machine and ran them there.

...I would note that dfreveal would sometimes reveal a single world-tile's worth of area (so, 48x48 tiles, I think...didn't count) in the farthest northwest corner, at the bottommost z-levels, before it crashed.  Once it did that for about 4 z-levels up from the bottom before it crashed.

If there's any other information I can gather for you to troubleshoot this, I'd be more than glad to try

[peterix]

How to determine an item's value:

There simply has to be a value cached somewhere. It can't do all this for every item every time the fortress value needs to be recalculated.

For values returned by function calls, there are several ways of getting at the value:
1. Read the function pointers out of the object's VTABLE and examine the code they point at, then interpret the code to determine what they'll return. DFHack has "accessors" which do this, though I don't know if they still work with the current version.
2. Use the item's VTABLE pointer to determine its type, then use hardcoded lookup tables to get the relevant values/offsets for each item type. This would tie the utility to a very specific version of Dwarf Fortress on a single platform.
3. Inject code into Dwarf Fortress (as a separate thread) which calls the subroutine with the appropriate parameters and places the result at a predetermined address. I've actually used this method myself in a tool I wrote for Dwarf Fortress v0.23.130.23a (in the old 2D versions, designating items for Melting involved adding them to a vector rather than setting an item flag).

All valid approaches. I like 2. in particular because it allows me to automate things and be lazy.
3. I would never do in a separate thread. That's just asking for trouble. If you go to the lengths of injecting code, do it properly and make it synchronized. DF has no idea your thread is there and you will get race conditions.

Anyway, I'll ask everyone involved just one thing: be lazy and think of the maintenance costs in the long run

[Quietust]

3. I would never do in a separate thread. That's just asking for trouble. If you go to the lengths of injecting code, do it properly and make it synchronized. DF has no idea your thread is there and you will get race conditions.

What method of synchronization would you recommend for such an old version of DF?

Some of my tools do become unstable at times, but it's mainly the ones that have to repeatedly probe the game's memory for updates - one of them periodically examines every item in the fortress and forbids them if they're carried by an invader (auto-forbid death drops is a marvelous feature I simply couldn't live without), and another repeatedly queries the "active screen" to see if I'm either looking at an item (on the screen that would show the item value, so I can [f]orbid/[c]hasm/[m]elt the item) or in the detailed view of the Stocks screen (so I can toggle forbid on the selected item just like chasm/melt), and the latter only seems to crash if it happens to query at the exact moment the screen changes, never when toggling Melt (which is the only part that actually injects code - querying the Melt flag just means iterating across the Melt vector and comparing item IDs).

[peterix]

What method of synchronization would you recommend for such an old version of DF?

Hook some function that's called in a safe place. A system call, something. Set up something like a breakpoint if everything else fails. Something between two updates of the DF world. I haven't really messed around with the old versions, but this can be done comfortably with the new SDL versions. There's a 'pointless call' to SDL_NumJoysticks just for this purpose. I did use it in the SHM (shared memory) version of dfhack, but I scrapped that due to unfixable bugs. I still want to use it to synchronize properly though.

@all the wine dfhack users:
I'll try to cobble together a wine-friendly release

[danaris]

@all the wine dfhack users:
I'll try to cobble together a wine-friendly release

Thanks!

[peterix]

@all the wine dfhack users:
I'll try to cobble together a wine-friendly release

Thanks!

OK. Here goes: https://github.com/downloads/peterix/dfhack/dfhack-test-wine-x86.zip

This is basically what 0.5.9 will be, except for some extra tweaks. If it works for you, I'll release both this and the regular MS version. It's built with GCC on Windows, so it includes all the bits and pieces of the C/C++ libraries inside, instead of relying on the Microsoft stuff. Give it a try  

[danaris]

OK. Here goes: https://github.com/downloads/peterix/dfhack/dfhack-test-wine-x86.zip

This is basically what 0.5.9 will be, except for some extra tweaks. If it works for you, I'll release both this and the regular MS version. It's built with GCC on Windows, so it includes all the bits and pieces of the C/C++ libraries inside, instead of relying on the Microsoft stuff. Give it a try

Same problem   

Well, not exactly the same.  It still says "memory object not set: type offset key /string/MSVC/buffer", but the line about "HeapSetInformation" is gone.  This is with dfattachtest, dfvdig, and dfprospector.

If you want to build a copy with some debug output turned on, I'll be happy to report the output from any given tool.

I should note that I have the current version of DwarfTherapist running in Wine connected to DF and working happily.

[peterix]

Same problem   

Ok. This time I reinstalled the compiler (I had some very old version) and built with debug symbols turned on. It *should* be possible to use winedbg as a debugger and get some backtraces... maybe.

EDIT:
OK. This one works for me under wine. It's very slow, but works:
https://github.com/downloads/peterix/dfhack/dfhack-test3-wine-x86.zip

It seems a bit strange still. I got a "memory object not set: type offset key /string/MSVC/buffer" right after trying to use winedbg... all I can say is that wine is a pile of hacks. One note about using multiple tools at the same time on linux (no idea about osx though): DON'T. The underlying platform can't do it, wine will only give you a pile of errors.

EDIT 2:
OK. I'll delay the release until this is confirmed at least semi-working. Btw, the 'suspend' part of DFHack doesn't work when you run it inside wine for whatever reason. This is probably why things were crashing randomly before... I never tested DFHack under wine and I'm not going to support it beyond a 'semi-working' state. Piling hacks on top of hacks is a bad idea.

[danaris]

Frankly, so long as dfprospector and dfvdig work consistently, I'll be happy.  If dfreveal and dfcleanmap would work even 1/3 of the time as well, that would be brilliant.

For anything more, I can always fire up my Windows virtual machine and run it in there.

And if there's anything I can do in the way of testing on OS X to help you out, I'll be more than glad to