Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[Rose]

Well, peterix is currently working on a way to efficiently pass data between DFhack and other programs.

Your best bet would be to hang out in #dfhack on freenode.

[Mithaldu]

A-HA, now that is promising new. Thank you! Will be there as soon as i can again.

[peterix]

peterix, do you think there's any chance of compiling the connector library with gcc or connecting to DF without needing to compile that massive thing? I'm pondering how to hook this up with Perl, but all these close ties to C++ and MSVC have seemingly made that impossible.

Also, on the very very tiny chance that that might provide some pointers: Are the python bindings i saw traces of actually functional still?

Well. I pretty much scrapped the idea of hacking into DF sideways from an another process. DFHack is loaded into DF and impersonates the SDL library. As long as you can pull data over some channel from one process to another, it's OK. I want to use google protobufs together with zmq to create a stable versioned API independent from all the black magic in DFHack. There's even code written by other people implementing something like this:
http://pastebin.com/hNPwfGSS
http://pastebin.com/Fdyvs9MU
http://pastebin.com/gvS81H8P

So, yeah. The library in DF isn't going away and pretty much *is* DFHack - it's the server part (or what will become the server part). In it, I have to use the same compiler, the same version of the STL and the same allocator for things to NOT blow up horribly. Stuff that will talk to it over some form of IPC obviously won't be limited in such a way. Actually, it might be possible to make your own protocol plugin and optimize what you send around or pre-process stuff on the DFHack side where you have full memory access.

The python stuff is currently dead - it was written for the previous design.

[Mithaldu]

I'm not even talking about the server stuff. That's perfectly fine and could be in Befunge for all it impacts me. I was talking about the kind of stuff that is loaded in by Overseer and the likes.

Also, yes, i know the bit about wanting to use a DF-loaded dll as a go-between. If i remember right you had that idea while we poked around with lifevis.

Protobuf, not sure if i like how it looks like, kind of a slightly less verbose XML. My instinct says JSON would be nicer and likely faster, but i guess that would require real world benchmarking and i might be misunderstanding. ZeroMQ is interesting, though not very Windows-friendly. Would be an excuse to sit down and make it work with Perl.

Two questions: Why not have it open a network port? What's good enough for databases should work here too, no?
Also: Would an implementation of the DFhack server accepting input via the process' STDIN and sending back output via STDOUT/STDERR be hard for you to do? I think that would be one of the most simple things, where any wrapping process just starts DF and ties up the filehandles.

Thanks for the note on the python stuff, won't waste time looking then.

Another question though: Can you explain just how Overseer actually DOES talk to the dfhack server at the moment?

[peterix]

I'm not even talking about the server stuff. That's perfectly fine and could be in Befunge for all it impacts me. I was talking about the kind of stuff that is loaded in by Overseer and the likes.

They are still using the old dfhack.

Protobuf, not sure if i like how it looks like, kind of a slightly less verbose XML. My instinct says JSON would be nicer and likely faster, but i guess that would require real world benchmarking and i might be misunderstanding.

More like serialization of object data into binary messages. There should be no text involved unless it's a string. The message descriptions are translated into C/C++/Java/Python/Perl/whatever code with a compiler.

ZeroMQ is interesting, though not very Windows-friendly. Would be an excuse to sit down and make it work with Perl.

Two questions: Why not have it open a network port? What's good enough for databases should work here too, no?

zmq uses a network port and I won't have to deal with it manually... at least that's the idea.

Also: Would an implementation of the DFhack server accepting input via the process' STDIN and sending back output via STDOUT/STDERR be hard for you to do? I think that would be one of the most simple things, where any wrapping process just starts DF and ties up the filehandles.

That would completely break DFHack on linux and DF in TEXT mode. Creating named pipes or some such IPC stuffs is certainly possible, but STDIN/STDOUT are quite crowded already A nice thing about the current DFHack is the fact that you can extend it with plugins - all it needs is spawning a thread and handling events instead of just implementing single commands. DF could be running any number of servers.

Another question though: Can you explain just how Overseer actually DOES talk to the dfhack server at the moment?

It doesn't. It's still using the old DFHack lib. I rewrote pretty much everything since then and the way to talk to external apps is pretty much missing - I have some bits and pieces of code though.
See:
https://github.com/peterix/dfhack/branches

'master' branch is current - the SDL impersonating stuff
'legacy' is the last version of old dfhack. I'll keep it around and possibly even update the offset file if it's still compatible with the next DF version... it won't be getting any new features though.

[Box]

Having a bit of a rough time so far using your very nice mod, here.  It's saved my plenty of times already, but I'm having some issues with my embark I'm trying to resolve.

Prospect tells me I have about 1800 tiles of "candy" (that's the new spoiler-free term, I think?) but using reveal I can't spot nearly that much myself.  I would guess I have under 200, less than 150 being safe to extract.  So I tried using tiletypes to paint myself some using the "HFS" material to artificially extend the tubes of candy, but it doesn't seem to work.  Stuff like soil and obsidian works fine, but I can't get anything else to.

Am I doing something wrong or are my files messed up or is this just a semi-functional tool?  I saw the earlier posts regarding it, but not much on its actual functionality.

[Quietust]

The tiletypes tool has some rather annoying limitations, as I discovered yesterday - if the tile you are painting onto contains a "varying" tile (e.g. floor/river), then you cannot paint a non-varying tile (i.e. anything else) onto it unless it happens to contain the "main" variant of said tile (for floors, a normal apostrophe).

[Mithaldu]

They are still using the old dfhack.

Well dang, that explains a lot! Thanks for the link, even though in that case there's not much of a point in poking at it anymore.

More like serialization of object data into binary messages.

Alright, i misunderstood it then. Just hope unpacking it won't be too slow in pure perl.

zmq uses a network port and I won't have to deal with it manually... at least that's the idea.

That makes sense, no real point in making your own protocol if there's one already.

That would completely break DFHack on linux and DF in TEXT mode. Creating named pipes or some such IPC stuffs is certainly possible, but STDIN/STDOUT are quite crowded already

Aw.

Guess i'll have to learn how to use named pipes sometime then.

A nice thing about the current DFHack is the fact that you can extend it with plugins

I see how that's nice, but in all honesty, my C knowledge extends about as far as that it comes after B.

I think it'll be easier for me to keep poking at the Perl-ZeroMQ maintainers to see if i can help them make things work on windows.

Thanks again for the answers. It means i won't be able to start implementing right now, but it helped me find some other goals to work on.

Oh, one last thing: Any plans on when the zmq stuff might be done on your side?

[peterix]

Oh, one last thing: Any plans on when the zmq stuff might be done on your side?

Done... I don't know. It might never be really 'done', just like DF. I'd like to have something experimental and !!fun!! after the next weekend though.

[Rumrusher]

speaking about !!fun!! I was checking out the updates to Dfusion and notice some new stuff I can't get direct access to through notepad(try to patch it my self but kinda broke on me) I was wondering if there going to be a DFhack r7?

[Mithaldu]

Oh, one last thing: Any plans on when the zmq stuff might be done on your side?

Done... I don't know. It might never be really 'done', just like DF. I'd like to have something experimental and !!fun!! after the next weekend though.

That's a lot earlier than expected.

I might have to try and see what the hell happens if DF runs on one machine and the 3d monitor on another.

[tryrar]

I have a small problem...I had used reveal to properly dig my magma pump stack, and decided to take a short break, so I saved and closed. Now Reveal is stuck on! What do I do?!

[Rose]

I recall there being an unreveal function.

[tryrar]

ummm...I tried that already, it says nothing to unreveal

[peterix]

ummm...I tried that already, it says nothing to unreveal

Point the 'k' cursor at a floor that should be visible, run 'revflood'. You can use the same to hide stuff after re-walling.