Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[lethosor]

OS X build (0.40.12-r1)

I'm not able to issue any commands in this version. It says "resetting textures" and gives notices if I resize the window or zoom in, but no commands are working, and hotkeys appear to not be working either.

That's strange - it sounds like you're not using DF 0.40.12 (or 0.40.11). If that's not the case, have you made any patches to dwarfort.exe?

[mifki]

OS X build (0.40.12-r1)

I'm not able to issue any commands in this version. It says "resetting textures" and gives notices if I resize the window or zoom in, but no commands are working, and hotkeys appear to not be working either.

That's strange - it sounds like you're not using DF 0.40.12 (or 0.40.11). If that's not the case, have you made any patches to dwarfort.exe?

Or just ran `df` script instead of `dfhack`.

[Max™]

I know one of the first times I tried to get dfhack working I had to manually allow the dfhack script to be executed, maybe it was just defaulting to the df due to a similar problem?

[ohgoditburns]

OS X build (0.40.12-r1)

I'm not able to issue any commands in this version. It says "resetting textures" and gives notices if I resize the window or zoom in, but no commands are working, and hotkeys appear to not be working either.

That's strange - it sounds like you're not using DF 0.40.12 (or 0.40.11). If that's not the case, have you made any patches to dwarfort.exe?

Welp, that was a silly mistake. I accidentally unpacked dfhack for 0.40.11 into my df 0.40.12 folder. It does work for me after all. Sorry bout that!

[expwnent]

Anyone want to take on the challenge of adding script triggers for the following to modtools? I have visions of modding in various things, from a whoopee cushion all the way to a dragonfire trap.

• lever pull
• pressure plate activation
• gear/shaft/roller power status change
• door status change
• bridge/hatch/floor bars/vertical bars/floodgate status change
• trap status change
• minecart friction (from stop) applied; minecart dump activation

Note: lever pull can be already be done through the job complete event, but I included it in the list for the purposes of a consistent interface.

That sort of thing is easy enough to do. I'll add it to the list.

[Quietust]

depending on a end-user configured version string to determine compatibility isn't giving much assurance of actual compatibility.

There is a version string, but it isn't "end-user configured" - it's set inside the project makefiles (by the DFHack developers).

And I'd say "acceptable and expected" is not quite the same as "accepted because they're not given any other options". Like I've already mentioned repeatedly, I appreciate that putting in that layer of abstraction now that should have been there all along to prevent the issue you described is a significant undertaking. This doesn't change the fact that the state of DF is very different now than it was 6 months ago when it had been 18 months since the last update and that dependence on DFHack, especially in starter packs and the like, is significant enough to warrant finding better ways to do things.

Long ago, there was an "abstraction layer" with that level of detail. It was called "memory.xml", and it contained individual addresses and offsets for the small handful of variables that DFHack commands wanted to use. It was also very messy to deal with.

When we switched to df-structures and loading into DF itself (instead of acting like a debugger and using ReadProcessMemory/WriteProcessMemory), we did away with that because the new system was much simpler to code for - rather than having to call special functions inside DFHack to fetch an object of a particular type (from one of the specific lists it knew about at the time) and then lookup the offset of each individual field within the object it wanted to access, it could offload all of that to the C++ compiler.

And like I already said, Lua scripts do have the "layer of abstraction" you're so insistent on having - you can access fields by name, and as long as those field names don't change then you can run the script in whatever version of DF you want. Hell, I've run some DFHack Lua scripts written for 0.34.11 in 0.28.181.40d and even 0.23.130.23a, and they worked just fine as long as they weren't trying to access things that didn't exist.

If you think you can do better, then by all means do so.

[Roses]

I'll add it to the list.

You should probably trade mark that.

Long ago, there was an "abstraction layer" with that level of detail. It was called "memory.xml", and it contained individual addresses and offsets for the small handful of variables that DFHack commands wanted to use. It was also very messy to deal with.

Man, I remember when DFHack used memory.xml, so few people used it because there were only a handful of things, and hardly any modders had it in their mods. The df-structures method is so much nicer. I shudder to think of those days.

[salithus]

depending on a end-user configured version string to determine compatibility isn't giving much assurance of actual compatibility.

There is a version string, but it isn't "end-user configured" - it's set inside the project makefiles (by the DFHack developers).

I'm an end user - I set it to get DFHack to recognize mifki's build of the TwbT plugin, which is what brought up this whole discussion in the first place. If I want to compile just a plugin, I can set that version string to whatever I feel like, and DFHack will think the DLL was compiled for that version of DFHack, regardless of what is actually happening in the source.

And I'd say "acceptable and expected" is not quite the same as "accepted because they're not given any other options". Like I've already mentioned repeatedly, I appreciate that putting in that layer of abstraction now that should have been there all along to prevent the issue you described is a significant undertaking. This doesn't change the fact that the state of DF is very different now than it was 6 months ago when it had been 18 months since the last update and that dependence on DFHack, especially in starter packs and the like, is significant enough to warrant finding better ways to do things.

Long ago, there was an "abstraction layer" with that level of detail. It was called "symbols.xml", and it contained individual addresses and offsets for the small handful of variables that DFHack commands wanted to use. It was also very messy to deal with.

When we switched to df-structures and loading into DF itself (instead of acting like a debugger and using ReadProcessMemory/WriteProcessMemory), we did away with that because the new system was much simpler to code for - rather than having to call special functions inside DFHack to fetch an object of a particular type (from one of the specific lists it knew about at the time) and then lookup the offset of each individual field within the object it wanted to access, it could offload all of that to the C++ compiler.

And like I already said, Lua scripts do have the "layer of abstraction" you're so insistent on having - you can access fields by name, and as long as those field names don't change then you can run the script in whatever version of DF you want. Hell, I've run some DFHack Lua scripts written for 0.34.11 in 0.28.181.40d and even 0.23.130.23a, and they worked just fine as long as they weren't trying to access things that didn't exist.

If you think you can do better, then by all means do so.

That doesn't make any sense in the context of the discussion so far. My whole reason for thinking that there's got to be a better way of doing things is that I used:
DFHack source from .40.11r1
symbols.xml for DF .40.12
DF .40.12

And everything worked fine. If symbols.xml isn't that abstraction layer anymore, why was updating it enough to get everything working? Why was it needed at all? Those are the questions that kickstarted this whole thing. Waiting on the DFHack team to put together for a release when there are bug fixes or breaking changes makes sense. Waiting on them when all that was required to get DF .40.12 up and running, which has a critical fix for dwarves getting stuck unconscious, doesn't make sense if the only two things required were updating symbols.xml and making DFHack and the plugins all think they had been changed to work with .40.12.

As a result, an update to DF .04.12 for the Starter Pack PE maintains could not be released until the DFHack developers put together a release. For whatever reason, this took 6 days to do. Considering that the DF update from .10->.11 took 7 days and Toady's notes say a .13 release could be today or tomorrow, continuing this cycle makes starter packs effectively worthless. We have gone from a system where, 6 months ago, the starter pack was constantly at the latest version of DF to now, where it never can be.

 I don't particularly care how we get that gap closed, what's important to that me is we figure out a way to do it. Yours is the first post since this conversation began that wasn't completely "well we've always done it this way so get over it" and actually started discussing why the roadblocks exist and what it would take to overcome them. With that in mind, unless the DFHack team intends to completely obsolete non-Lua plugins, that's not an effective approach as things like the Starter Pack depend on a ton of existing plugins - TwbT, Falconne's jaunts, etc.

It seems to me that if just updating symbols.xml was enough to get things moving from .40.11 to .40.12, there's got to be some way to eliminate the wait on the DFHack team to do an r1 release if the only thing that is required for an update is symbols.xml.

[ag]

The main reason symbols.xml is still loaded at runtime is that the values in it cannot ever carry over to a new release and must be found one way or another anew. Keeping the file loaded at runtime allows adding more values to it without recompiling as we work on it.

However the bulk of the knowledge about DF is all the rest of xml files, and they are in effect compiled into both dfhack core and any plugins. If anything at all other than symbols.xml changes everything must be recompiled.

On the other hand if symbols.xml is truly the only thing that changed like with 0.40.12, then you can just get a new version of it and drop into the old dfhack, and it will work just fine.

[Rumrusher]

Wait isn't the whole convo started from the idea of separating dfhack from DF?
Which long time ago was how dfhack originally was? most of the problems with dfhack back then was it needed to hook into the game and that tick off anti-virus software pretty bad.
Like most of the scripts/plugins had to run 'look and poke' functions to get anything done. we couldn't even touch graphics of DF back then either since we kinda looking through on the outside.
like now the same problems still exist and kinda never go away, also there's no per tick effects or scripts since well dfhack had to stop dwarf fortress and scan the whole game to run those addons.
and running stuff like that tank the fps. so if you're crazy enough you could do what AG said.

[lethosor]

depending on a end-user configured version string to determine compatibility isn't giving much assurance of actual compatibility.

There is a version string, but it isn't "end-user configured" - it's set inside the project makefiles (by the DFHack developers).

I'm an end user - I set it to get DFHack to recognize mifki's build of the TwbT plugin, which is what brought up this whole discussion in the first place. If I want to compile just a plugin, I can set that version string to whatever I feel like, and DFHack will think the DLL was compiled for that version of DFHack, regardless of what is actually happening in the source.

If you set the DFHack version when compiling DFHack, that's not what Quietust is referring to by "end user". If you want a plugin to be compatible with a specific version of DFHack, it's safer to compile DFHack and that plugin with an accurate version (or ask the plugin's author to build for that DFHack version), rather than changing the DFHack version to match a plugin. The version check is in place to prevent problems resulting from incompatibilities between DFHack versions - it may have been safe to bypass in this case (by changing the DFHack version), but that doesn't guarantee that it will be safe in the future.

...

That doesn't make any sense in the context of the discussion so far. My whole reason for thinking that there's got to be a better way of doing things is that I used:
DFHack source from .40.11r1
symbols.xml for DF .40.12
DF .40.12

And everything worked fine. If symbols.xml isn't that abstraction layer anymore, why was updating it enough to get everything working? Why was it needed at all? Those are the questions that kickstarted this whole thing. Waiting on the DFHack team to put together for a release when there are bug fixes or breaking changes makes sense. Waiting on them when all that was required to get DF .40.12 up and running, which has a critical fix for dwarves getting stuck unconscious, doesn't make sense if the only two things required were updating symbols.xml and making DFHack and the plugins all think they had been changed to work with .40.12.

When you built your version of "0.40.12-r1", there hadn't been any DFHack changes made to the main repo since 0.40.11-r1 - the only changes were made to df-structures. Expwnent merged in around a dozen pull requests an hour or so before making an official release. This brings up another point: there are now multiple "0.40.12-r1" builds floating around that have different behaviors and features. Among other things, "hack-wish" no longer crashes in certain cases, "search" has been updated for 0.40.12, and workflow no longer produces repeating error messages. However, since builds built before these changes identify themselves as "0.40.12-r1", there's no easy way to tell which build of "0.40.12-r1" that specific bug reports apply to, for example.

As a result, an update to DF .04.12 for the Starter Pack PE maintains could not be released until the DFHack developers put together a release. For whatever reason, this took 6 days to do. Considering that the DF update from .10->.11 took 7 days and Toady's notes say a .13 release could be today or tomorrow, continuing this cycle makes starter packs effectively worthless. We have gone from a system where, 6 months ago, the starter pack was constantly at the latest version of DF to now, where it never can be.

Why exactly do starter packs "require" DFHack? If playing the latest version is necessary, it's entirely possible to play the latest version without DFHack (or any other utilities, for that matter) for a few days. An alternative is to build DFHack independently, as Fricy did for his pack (and Beautato attempted to). (Admittedly, doing so brings up DFHack versioning issues again, but my point is that the delays in updating some packs are primarily due to the authors of those packs choosing not to update until certain utilities are updated.)

I don't particularly care how we get that gap closed, what's important to that me is we figure out a way to do it. Yours is the first post since this conversation began that wasn't completely "well we've always done it this way so get over it" and actually started discussing why the roadblocks exist and what it would take to overcome them. With that in mind, unless the DFHack team intends to completely obsolete non-Lua plugins, that's not an effective approach as things like the Starter Pack depend on a ton of existing plugins - TwbT, Falconne's jaunts, etc.

From what I've heard on IRC, Expwnent plans to release 0.40.13-r1 within a day of df-structures being updated. (If you take a look at the old DFHack thread, updates were often available a day or so after a release, except for major/new-feature releases).
I'm not sure what you mean by obsoleting non-Lua plugins - Lua plugins don't necessarily work with new DFHack releases without modification (they don't need to be recompiled, but that leads to problems being encountered at runtime instead).

[expwnent]

I try to keep such predictions to the irc channel so people don't get mad if I fail to live up to them. I hope to get it done that fast but something could always go wrong.

[khearn]

I want it NOWWWWW!!!

You PROMISED!!!!!

Waaaaah!!!

[salithus]

that doesn't guarantee that it will be safe in the future.

When you built your version of "0.40.12-r1", there hadn't been any DFHack changes made to the main repo since 0.40.11-r1 - the only changes were made to df-structures. Expwnent merged in around a dozen pull requests an hour or so before making an official release. This brings up another point: there are now multiple "0.40.12-r1" builds floating around that have different behaviors and features. Among other things, "hack-wish" no longer crashes in certain cases, "search" has been updated for 0.40.12, and workflow no longer produces repeating error messages. However, since builds built before these changes identify themselves as "0.40.12-r1", there's no easy way to tell which build of "0.40.12-r1" that specific bug reports apply to, for example.

My point exactly: This is why basing compatibility on the version string specified during compilation does nothing. At the time I compiled DFHack and posted a zip for other users:

A) All I did was follow the instructions posted by others on how to compile DFHack for Windows.
B) I had no idea that it should have been marked r0.
C) Even after knowing it should have been marked r0, there were other plugins (in this case TwbT) that I didn't know how to recompile separately that were marked for r1.

Why exactly do starter packs "require" DFHack? If playing the latest version is necessary, it's entirely possible to play the latest version without DFHack (or any other utilities, for that matter) for a few days. An alternative is to build DFHack independently, as Fricy did for his pack (and Beautato attempted to). (Admittedly, doing so brings up DFHack versioning issues again, but my point is that the delays in updating some packs are primarily due to the authors of those packs choosing not to update until certain utilities are updated.)

It's asinine to think that starter packs, which are specifically geared towards new/inexperienced DF players, aren't dependent on the functionality that DFHack and commonly used plugins bring. Saying it's entirely up to the pack author's choice is like blaming the restaurant industry for people getting hungry.

From what I've heard on IRC, Expwnent plans to release 0.40.13-r1 within a day of df-structures being updated. (If you take a look at the old DFHack thread, updates were often available a day or so after a release, except for major/new-feature releases).
I'm not sure what you mean by obsoleting non-Lua plugins - Lua plugins don't necessarily work with new DFHack releases without modification (they don't need to be recompiled, but that leads to problems being encountered at runtime instead).

What I mean is that, per Quietust's posts, Lua scripts are currently the only way to have a layer of abstraction in that would have prevented needing DFHack plugins recompiled between .40.11 and .40.12. Unless you plan to make that the only option for plugins (which we all realize is not the right way to fix this gap), we need to consider other options.

Also, can we please quit approaching this with the "it won't fix this every time" mentality? I know that there are going to be times between versions that compatibility will break. My point is that it is not every release as evidenced by being able to use .40.11-r1 source with .40.12 symbols, so there is clearly an opportunity to improve the process.

[peterix]

Then I urge you to GO and IMPROVE the process yourself.