Bay 12 Games Forum

Please login or register.

Login with username, password and session length
Advanced search  
Pages: 1 [2] 3 4

Author Topic: Dwarf Explorer - A graphical dfhack plugin viewer for DF Structures  (Read 24157 times)

Clément

  • Bay Watcher
    • View Profile
Re: Dwarf Explorer - A graphical dfhack plugin viewer for DF Structures
« Reply #15 on: July 08, 2019, 04:12:25 pm »

ragundo, use windeployqt to copy Qt dlls, it will include all the required Qt plugins (platform, image formats, ...). You will find it in the bin directory of the Qt SDK you used to build the dfhack plugins.
Logged

ragundo

  • Bay Watcher
    • View Profile
Re: Dwarf Explorer - A graphical dfhack plugin viewer for DF Structures
« Reply #16 on: July 08, 2019, 05:12:33 pm »

Yes. I know about windeployqt, but the help page says that it's for deploying exe files, not dll ones.


I think that only qwindows.dll is needed to run the plugin in other systems (in my system Qt is not listed in the PATH variable and it works by deploying only the Qt5**.dlls).

I'm researching about the possibilty of copy qwindows.dll in the Dwarf Fortress directory by using a   qt.conf file. I wouldn't like to add a platforms subdirectory to store qwindows.dll
Logged

ragundo

  • Bay Watcher
    • View Profile
Re: Dwarf Explorer - A graphical dfhack plugin viewer for DF Structures
« Reply #17 on: July 08, 2019, 05:25:37 pm »

Hmmm

Do you have visual studio 2017 runtime installed?
Qt is compiled against that runtime.

Another thing that can happens is that your path finds another Qt dll instead of the ones I supply (5.13.0).
Dwarf Therapist uses Qt 5.6

I'm on mobile now but I'll try later with a windows vm

Well, I updated the download file with qwindows.dll. It will go to a platforms subdirectory in the Dwarf Fortress directory as recommended by the Qt docs.
Could you try again to launch the plugin?. My local VMs are all 32 bits so pretty useless.

Thanks
Logged

risusinf

  • Bay Watcher
    • View Profile
Re: Dwarf Explorer - A graphical dfhack plugin viewer for DF Structures
« Reply #18 on: July 08, 2019, 10:52:43 pm »

Looks quite handy, wonder it this will be any of help for debugging efforts. Thank you for your work. Looking forward to the linux version.
Logged

PatrikLundell

  • Bay Watcher
    • View Profile
Re: Dwarf Explorer - A graphical dfhack plugin viewer for DF Structures
« Reply #19 on: July 09, 2019, 04:02:46 am »

Yes, that did the trick.

- It's a nice improvement to provide directories in the zip file. That makes "installation" that little bit easier (and harder to screw up).
- gui/gm-editor suppresses display of overloaded pointers because of the danger involved in following the incorrect interpretation. Is it possible to at least provide a warning in this situation (looking at the df.global.world.incidents.all
  • .unk_v42_1 pointers, where the pointer interpretation is controlled by the subtype field, but there's no way I know of to express the usage of this discriminant in the DF structures syntax, so it's provided only as a comment).

- Would it be possible to allow the user to invoke dwarfexplorer with a path parameter such you can do with gui/gm-editor? I'm thinking of things like referring to element 700000 of the hist fig vector, for instance, where you don't really want to manually scroll past a bazillion pages to find that element.
- Presentation of the comments can be very useful, especially if that inspires the writing of more comments. It might be useful to provide a popup for this field when hovered over to allow you to see the full comment (See e.g. df.global.world.world_data.feature_map).

Just some initial feedback after a quick look.

Edit: It's annoying that you have to start from scratch digging your way down into the structures every time you resume/suspend. I think it would be useful if the former view was remembered such that it could be recreated to the extent possible (and this goes for windows displaying sections as well: when suspended again, it would be useful to be able to "refresh" them, with the possible result of them becoming empty).
« Last Edit: July 09, 2019, 04:10:30 am by PatrikLundell »
Logged

ragundo

  • Bay Watcher
    • View Profile
Re: Dwarf Explorer - A graphical dfhack plugin viewer for DF Structures
« Reply #20 on: July 09, 2019, 05:57:17 am »

Yes, that did the trick.

- It's a nice improvement to provide directories in the zip file. That makes "installation" that little bit easier (and harder to screw up).
- gui/gm-editor suppresses display of overloaded pointers because of the danger involved in following the incorrect interpretation. Is it possible to at least provide a warning in this situation (looking at the df.global.world.incidents.all
  • .unk_v42_1 pointers, where the pointer interpretation is controlled by the subtype field, but there's no way I know of to express the usage of this discriminant in the DF structures syntax, so it's provided only as a comment).

- Would it be possible to allow the user to invoke dwarfexplorer with a path parameter such you can do with gui/gm-editor? I'm thinking of things like referring to element 700000 of the hist fig vector, for instance, where you don't really want to manually scroll past a bazillion pages to find that element.
- Presentation of the comments can be very useful, especially if that inspires the writing of more comments. It might be useful to provide a popup for this field when hovered over to allow you to see the full comment (See e.g. df.global.world.world_data.feature_map).

Just some initial feedback after a quick look.

Edit: It's annoying that you have to start from scratch digging your way down into the structures every time you resume/suspend. I think it would be useful if the former view was remembered such that it could be recreated to the extent possible (and this goes for windows displaying sections as well: when suspended again, it would be useful to be able to "refresh" them, with the possible result of them becoming empty).

I'm glad that you make it run. Many thanks for the time that you spend trying to make it work.!!

I've read some interesting articles about Qt deploying and now I understand why the plugin always works in the development machine the way I released originally and not in the client one. That will help me for the linux deployment in a few weeks.

About your comments:

Overloaded pointers. Yes I agree. This is something similar to some unions, where there's another field called type that could be uses to identify the union type. I'll see what I can do about that.

Invoking Dwarf Explorer with a path is the following thing I plan to do. It has its complications because the tree loading is lazy, and data only gets added to the tree as you expand the branch, but I'll manage to do it.

The tooltip with the comment data is a great idea. Noted!

And yes, the need to dig down the tree everytime you pause and resume is very annoying and the thing that I don't like at all of this release. This is related to the laziness loading of the tree. For now, I open the structure in another window and resume Dwarf Fortress. The window changes to the OUTDATED state and keeps a copy of the original data. When you pause again, I navigate to the same structure and comparte the old data with the new data. Not perfect I know.

Anyway, this is a 1.0 release and there's a lot of things to improve. I plan also for the next version to be able to follow refers-to attributes. That would improve the data navigation a lot.

Again, thanks for reviewing the plugin and don't hesitate to ask any feature that you would like to be implemente. This plugin was done for people like you.
Logged

PatrikLundell

  • Bay Watcher
    • View Profile
Re: Dwarf Explorer - A graphical dfhack plugin viewer for DF Structures
« Reply #21 on: July 09, 2019, 08:37:11 am »

Given the original description, it was clear it was going to be testing of an alpha or beta version, with a fair bit of development still being planned, and I have no problem with trying it and trying to provide some feedback. However, I'm not a great tester.

Overloading: Yes, the high level language concept of discriminants (as it's called in Ada) to determine how overloaded data should be interpreted isn't supported by High Level Assembly (a.k.a. C(++)) unfortunately. XML does have the capacity to describe it though, but most users of "XML" seem to ignore the built in features and use home brewed attributes instead, nor are XML schemas used to any extent to describe the legal syntax (that could then be validated to catch mistakes).

I believe it's necessary for the tool to load only the data that's to be displayed or starting would take forever and use up a huge chunk of memory, but there's a difference between laziness and complete amnesia... Remembering where you were is a kind of implicit expansion (with checks to prune references that do no longer exist). I agree this is a lot easier with the logic of gui/gm-editor, since you'd only need to keep track of a single expansion path, but it should definitely be doable with a reasonable effort.
The logic needed is probably similar to the one needed to open a given path, but with some additional complications, so I'd probably tackle the simpler problem first.

The ability to use an old and a new copy of some data and compare them, ideally with highlighted differences, would probably be very useful for field identification, as you'd be able to see what your in-game change resulted in (at that location) at a glance. Windows could possibly start outdated on a reload, with options to refresh or compare.

I agree the ability to follow references would be quite useful, but it can be noted that a lot of references are indicated only through field names (such as "hfid"). However, if there's a tool that can make intelligent use of such references, the drive to update the XML description with those attributes ought to be enhanced...
Logged

Fleeting Frames

  • Bay Watcher
  • Spooky cart at distance
    • View Profile
Re: Dwarf Explorer - A graphical dfhack plugin viewer for DF Structures
« Reply #22 on: July 09, 2019, 10:57:56 am »

Regarding compare, another thing one may find useful is just recursively comparing differences between two structures - did this in a small (unposted) script myself for identification of flag meanings.

PatrikLundell

  • Bay Watcher
    • View Profile
Re: Dwarf Explorer - A graphical dfhack plugin viewer for DF Structures
« Reply #23 on: July 09, 2019, 12:27:55 pm »

Recursion can be dangerous, as there are circular structures in DF (in particular linked lists, where the "previous" link points back to the previous element). However, comparison of the "same" view of the data at two different vector indices of the same vector could indeed be useful.

Edit: Some additional feedback:
It turns out comments using the "comment" attribute on compounds aren't picked up by dwarfexplorer (and they end up twice in the generated .h file (https://github.com/DFHack/dfhack/issues/1446, but that's a different issue).
« Last Edit: July 16, 2019, 05:29:08 am by PatrikLundell »
Logged

Ianflow

  • Bay Watcher
    • View Profile
Re: Dwarf Explorer - A graphical dfhack plugin viewer for DF Structures
« Reply #24 on: July 17, 2019, 03:45:46 pm »

I'm honestly not sure what the intended usage of this (which if I understand right is just a way to directly view the stored data of DF?) and applications are, but I'm not that far into modding.
This sounds really cool, and vaguely reminds me of when I attempted to get into Cheat Engine, but I assume this might be a debugger tool?
Logged
And thus, "The running of the goblins" became an annual tradition and the first dwarven contraceptive.
There are no moghoppers. We have always been allies of Oceania, and at war with Eastasia.

PatrikLundell

  • Bay Watcher
    • View Profile
Re: Dwarf Explorer - A graphical dfhack plugin viewer for DF Structures
« Reply #25 on: July 17, 2019, 04:51:55 pm »

I'm honestly not sure what the intended usage of this (which if I understand right is just a way to directly view the stored data of DF?) and applications are, but I'm not that far into modding.
This sounds really cool, and vaguely reminds me of when I attempted to get into Cheat Engine, but I assume this might be a debugger tool?
Similar to gui/gm-editor, but with a different take on it, so one would be better for some situations and the other of others (once it's 'finished').
The use would be for modding things, looking up things to know what to write into scripts (getting paths, fields, and types right. You can use the XML files for that as well, but a tool is often more convenient than finding the right file), investigations for structure identification (when DFHack has mapped structures, but their usage hasn't been identified), etc.
Logged

ragundo

  • Bay Watcher
    • View Profile
Re: Dwarf Explorer - A graphical dfhack plugin viewer for DF Structures
« Reply #26 on: July 18, 2019, 04:17:30 pm »

I'm honestly not sure what the intended usage of this (which if I understand right is just a way to directly view the stored data of DF?) and applications are, but I'm not that far into modding.
This sounds really cool, and vaguely reminds me of when I attempted to get into Cheat Engine, but I assume this might be a debugger tool?

As Patrick says, is like gm-editor on steroids. This is not a debugger, is for understanding how DF manages its data. You can use gm-editor for that, but I like more a tree structure as you don't miss the "locality" of your data as it happens with gm-editor. If you want to understand how the DF structures work I think that this is the ideal tool.

Also features like automatic translation of coordinates and language_names, be able to see the field comment and browse the memory make everything more snappy. Very useful also for undestanding lua scripts and C++ plugins

Things that will be added in future versions:
  • Tooltips for comments
  • Open a structure by typing its name.
  • Search for fields
  • Open refers-to attributes in new window
  • Show the xml data that defines a structure
  • Define new structures on the fly
  • Compare changes of a structure as time goes by
  • Changing data in the memory viewer
  • Changing data in the tree
Logged

ragundo

  • Bay Watcher
    • View Profile
Re: Dwarf Explorer - A graphical dfhack plugin viewer for DF Structures
« Reply #27 on: July 26, 2019, 03:21:21 pm »

Linux release available!!!

https://github.com/ragundo/DwarfExplorer/releases

If you uses the Linux LNP, you need to download the version compiled for GCC4.8.

Installation:
  • Install Qt using your package manager.
    In Ubuntu:sudo apt-get install qt5-default
  • Unpack the file with the plugin in the /hack/plugins folder of your DF installation
  • Launch dfhack
  • Load your world
  • type enable qapplication in the console
  • type dwarfexplorer in the console
  • Enjoy ;)
« Last Edit: July 26, 2019, 03:23:59 pm by ragundo »
Logged

strainer

  • Bay Watcher
  • Goatherd
    • View Profile
Re: Dwarf Explorer - A graphical dfhack plugin viewer for DF Structures
« Reply #28 on: July 28, 2019, 06:32:51 pm »

Looks like a very useful development tool already. I spent a while peeking through structures with lua commands. It should be much easier to figure out anon references with this.
Logged
Klok the Kloker !

Clément

  • Bay Watcher
    • View Profile
Re: Dwarf Explorer - A graphical dfhack plugin viewer for DF Structures
« Reply #29 on: July 29, 2019, 08:37:38 am »

In Ubuntu:sudo apt-get install qt5-default
qt5-default may be too much (it downloads development files), libqt5widgets5 should be enough for Ubuntu/Debian or whatever install libQt5Widgets.so.5 for your distribution.
Logged
Pages: 1 [2] 3 4