Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[Artanis00]

Well, rejoice! Before, I was getting in sideways and telling people to disable their rabid antivirus programs and poorly designed OS protections. Now I'll be distributing binaries that don't have to 'break in', so to speak. This is a massive improvement. And it's damn fast in comparison with other alternatives.

I can see how this could be used by some nasty person to do bad things, sure... but they could do it before and they'll be able to do it no matter how I change DFHack. The key thing is to *NOT* download binaries from someone you don't trust. I give you source code of everything I release. Read it. Compile it. You don't need the binaries, those are just for convenience.

Oh, I trust you. It's just all these other people.

I am well aware that malicious stuff could be distributed before, since we've been posting tools for a while here in various forms, just keep in mind that since DFHack is changing to a plugin system, it'll be easier and more common to grab executable binaries from (hopefully) the forum or DFFD and have DF|DFHack (an already trusted executable) run them. Like you say, ideally people will avoid downloading and running untrusted code, but only a cursory glance at the history of computer security will show that that is not going to happen.

I am also aware that a plugin in an interpreted language can be malicious, but at least it has to be distributed by source code, and is thus easily examined, and some quick searching suggests it shouldn't be too hard to sandbox the interpreter (PyPy, for example).

I guess I'm trying to say, while you are building this, think about how it might also be secured.

I am not writing stonesense in python.

Hell.

No.

I wasn't saying exclusive. Just, don't stop plugins support at C/C++. Clearly, graphics intensive applications need a language with more and better graphics support. Python can do it, but I certainly agree with your sentiment on that!

On the other hand, something like Therapist could probably be handled in Python or another interpreted language with little trouble.

[peterix]

I wasn't saying exclusive. Just, don't stop plugins support at C/C++. Clearly, graphics intensive applications need a language with more and better graphics support. Python can do it, but I certainly agree with your sentiment on that!

Well, the plan is to create an external API based on the Google Protocol Buffers library. This allows describing the interface itself and then generates the required code for various languages. The data transport would have to be handled using some (local) IPC method. Then all you'd need is implement the IPC bits, let the compiler generate the protocol bits for you and be 'done' with the thing. Less maintenance for everyone is the key here. The performance could be on par with the old dfhack... maybe a bit better in some cases.

The DFHack parts won't live inside a plugin. They are tightly integrated with DF and at the same time, I'll be slowly reducing them to simple headers which don't really make sense in a plugin. What's left is the core parts - SDL wrappers, support code for the plugin system, the terminal window thing and the offset file parser. I'm thinking about turning the parser into a header generator - make it create the basic structures for you.

[doomchild]

I wasn't saying exclusive. Just, don't stop plugins support at C/C++. Clearly, graphics intensive applications need a language with more and better graphics support. Python can do it, but I certainly agree with your sentiment on that!

Well, the plan is to create an external API based on the Google Protocol Buffers library. This allows describing the interface itself and then generates the required code for various languages. The data transport would have to be handled using some (local) IPC method. Then all you'd need is implement the IPC bits, let the compiler generate the protocol bits for you and be 'done' with the thing. Less maintenance for everyone is the key here. The performance could be on par with the old dfhack... maybe a bit better in some cases.

I'm definitely looking forward to this.

[Root Infinity]

Why does it say "Cannot get map info!" when I try to use DFImmolate?

More info:

• DF 0.31.25 Windows (XP)
• Loo(k)ing at the (palm) tree I want to remove

[peterix]

Why does it say "Cannot get map info!" when I try to use DFImmolate?

More info:

• DF 0.31.25 Windows (XP)
• Loo(k)ing at the (palm) tree I want to remove

OK. The attach part apparently worked, but it failed while trying to init the map. Do other tools work for you? Do you have some form of antivirus active?

[0x517A5D]

Ok, this is a test release (codename Rainbow Kittens!). It's the base DF 31.25 with DFHack added to it. Just run the exe as you would normally, you'll get the DFHack console next to the normal DF window. There's no support for plugins or external tools yet and the only (real) tool that works is reveal. Anyway, enjoy. This is a taste of things to come

So you're not injecting a DLL using CreateRemoteThread() ?  Nor something sneaky like SetWindowsHookEx().

You're actually replacing one of the DLLs that DF loads, using DF's normal calls into that DLL to do your hooks, and then thunking chaining the call to the real DLL ?

And, I suspect, you're using the SDL.DLL because DF calls it every frame, and DF is not in the middle of simulation when that call is made.

I would not have thought of this.

Hmm.  Then every DFHack tool will be compiled into the special DLL ?  Or will there be an interprocess communication API using, say, shared memory?

[Root Infinity]

Why does it say "Cannot get map info!" when I try to use DFImmolate?

More info:

• DF 0.31.25 Windows (XP)
• Loo(k)ing at the (palm) tree I want to remove

OK. The attach part apparently worked, but it failed while trying to init the map. Do other tools work for you? Do you have some form of antivirus active?

All the other tools I've tried work fine, and AVG, which reported activity, but I have an exception for the folder I have DFHack in.

[peterix]

So you're not injecting a DLL using CreateRemoteThread() ?  Nor something sneaky like SetWindowsHookEx().

Hmm.  Then every DFHack tool will be compiled into the special DLL ?  Or will there be an interprocess communication API using, say, shared memory?

That's not cross-platform. I'd rather avoid anything like this, because it might be locked down, or otherwise limited.

The tools will be plugins, I already ported some of them.

And yes, something like that. Shared memory is similar enough between Windows and Linux... and I assume OSX has it too. At least someone tried to port DFHack to OSX using that.

I already experimented with replacing SDL and making DF talk over shared memory. It didn't end well, because DFHack was still outside DF. It normally reads tiny variables from big, sparse structures and this is sort-of OK while accessing the memory using the OS debugging APIs. It's not so good when you use a proper IPC lock together with synchronous, one at a time calls over shared memory. So I used this custom spinlock, which made it about 50x - 100x faster. On a dual core processor. While nothing else was running. And it only worked with Intel chips.

I scrapped that and this is kind of a reboot, with proper design It will take some time to implement the SHM API, because this time, I'd like to get it right.

[jaxad0127]

Just to note, it will be possible to make DFHack accessible from remote systems as wall as the local computer, but you'll have to work to make that possible (no easy security holes here!).

I am also aware that a plugin in an interpreted language can be malicious, but at least it has to be distributed by source code, and is thus easily examined, and some quick searching suggests it shouldn't be too hard to sandbox the interpreter (PyPy, for example).

Python, at least, can be distributed in a compiled form too.

[Artanis00]

Just to note, it will be possible to make DFHack accessible from remote systems as wall as the local computer, but you'll have to work to make that possible (no easy security holes here!).

I am also aware that a plugin in an interpreted language can be malicious, but at least it has to be distributed by source code, and is thus easily examined, and some quick searching suggests it shouldn't be too hard to sandbox the interpreter (PyPy, for example).

Python, at least, can be distributed in a compiled form too.

I'm not in a position to double check at the moment, but I believe there is a switch that disables loading bytecode that doesn't have a corresponding source file.

[edit: nothing presents itself. It may be that no-one has thought this might be useful yet.]

[peterix]

All the other tools I've tried work fine, and AVG, which reported activity, but I have an exception for the folder I have DFHack in.

OK. I looked at it and I see no obvious error. The tool works too... So, no idea. I'll put together a build of prospector for the new plugin-based DFHack, for you to try. This should rule out interference from the OS.

[Root Infinity]

All the other tools I've tried work fine, and AVG, which reported activity, but I have an exception for the folder I have DFHack in.

OK. I looked at it and I see no obvious error. The tool works too... So, no idea. I'll put together a build of prospector for the new plugin-based DFHack, for you to try. This should rule out interference from the OS.

It gets weirder... It's Windows XP (family computer), and it works fine when run from an Admin command prompt, but not when it's run from Windows Explorer as Admin. Huh...

At least I found a way to get it to work... Thank you for your time, Peterix!

[Quietust]

Maybe there's some current-working-directory oddness or lack of environment variables messing it up?

[peterix]

Maybe there's some current-working-directory oddness or lack of environment variables messing it up?

Well, must be something weird with Windows. When all the other tools work as they should, and prospector being able to read the PE header from DF's memory (so the base offset must be good), but not the map pointer... it's really weird. Microsoft is good at keeping the APIs quite stable, but there are differences. Must be one of them. I unfortunately can't test with XP. My XP VM is unrecoverably broken, I lost the disk image and MSDNAA won't let me redownload it.

[Khym Chanur]

Trying to compile from the latest tarball () on Linux gives:

Code: [Select]

Scanning dependencies of target dfveinlook
[ 87%] Building CXX object tools/supported/CMakeFiles/dfveinlook.dir/veinlook.cpp.o
/home/matt/temp/peterix-dfhack-da2fb1c/tools/supported/veinlook.cpp: In function ‘void puttile(int, int, int, int)’:
/home/matt/temp/peterix-dfhack-da2fb1c/tools/supported/veinlook.cpp:103: error: ‘mvwaddwstr’ was not declared in this scope

Commenting out all instances of mvwaddwstr() made it compile.


I'm trying to figure out how to use dfincremental to determine:

1) Where and how the game stores info on wild beehives and ant colonies.
2) Where and how the game stores info on site-wide population counts (like how many turtles are left to fish before extinction)

Any advice?