Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[Max™]

God damn I am sharp as a brick sometimes.

So, ccmake specifically has an option to pick which arch you are building for, I looked at it and saw it said 32, but didn't realize I should change it to 64 until after getting linux successfully built and seeing "hack/libdfhack.so wrong ELFCLASS" pop up.

Double edit: it works! Still missing certain addresses (df.global.ui_advmode among them) but hey!

Hmm... haven't been looking recently, but when 64-bit was first released, I thought I saw reports of much better performance on larger embarks or world histories.  I thought that would likely expand to an older fortress' accumulated junk.

That was me. I tested the same exact world-gen/seed/everything except on 32 and 64 bit.

I ran them both for three minutes and tracked info about their progress.

Code: [Select]

32 bit started: 11:21:30, hit stop 11:24:30, finished 11:24:45 on year 184.
32 bit pops:
>56349 Dwarves
>30427 Humans
>48514 Elves
>11572 Goblins
>3913 Kobolds
>Total: 150775

64 bit started: 11:27:30, hit stop 11:30:30, finished 11:30:36 on year 186.
64 bit pops:
>59593 Dwarves
>33106 Humans
>42689 Elves
>12330 Goblins
>10884 Kobolds
>Total: 158602
Then I did two runs for 300 years and checked how long it took them.

Code: [Select]

test64 start: 10:38:30, stop: 10:54:01, finished: 10:54:34 (year 300), 16:04 total worldgen
64 bit pops:
>60553 Dwarves
>32485 Humans
>51199 Elves
>36119 Goblins
>5738 Kobolds
>Total: 186094

test32 start: 10:56:30, target: 11:12:01 (year 286),stop: 11:13:24, finish: 11:14:05 (year 300), 18:05 total worldgen
32 bit pops:
>54795 Dwarves
>28097 Humans
>45384 Elves
>27181 Goblins
>8280 Kobolds
>Total: 163737So in summary, 64 bit world-gen reaches the same year earlier, despite having significantly higher populations (23k+ in the 300 year test), responds faster to the stop signal, and takes longer to slow down as badly as the 32 bit world-gen.

This isn't directly applicable to fort mode, but world-gen is one of the most extreme examples of lag you usually encounter outside of like century old megaproject forts or 10k+ goblins and 100k trolls/beak dogs in a dark fort while you collapse it in adventurer mode, and it was easier to quantify the difference due to the bitness change.

[lethosor]

Anyway, I personally think some alpha version for osx/win64 could be released now. Or are there are any specific (missing/broken) things that holding off the release?

Mainly offsets (different ones on different platforms, but I think all of them are missing debug flags, and 64-bit Linux and Windows are missing more). It's not a deal-breaker for an alpha release, probably, as long as people realize what's missing.

[Max™]

Yeah, the unit all/active/bad lists aren't populated yet, ui_advmode isn't known, cur_season, created_item_* globals, various others.

[jecowa]

That was me. I tested the same exact world-gen/seed/everything except on 32 and 64 bit.

Max™, your tests were done with the 32-bit and 64-bit builds of Dwarf Fortress v0.43.05, right? You weren't using Dwarf Fortress v0.43.03 as the 32-bit version, were you?

I'm not interested in doing any 32bit work (my opinion is that we should drop 32bit support to make our lives easier).uld be released now. Or are there are any specific (missing/broken) things that holding off the release?

Yeah, keeping DFHack up-to-date seems like enough work as it is without having to support double versions. The legacy version of Dwarf Fortress isn't supported by DFHack either, so there's precedence for not supporting everything. I'm kind of surprised that new machines are still being shipped with 32-bit Windows, though. Someone had trouble getting Dwarf Fortress running on his Windows 8.1 tablet, and it turned out he needed the 32-bit version of Dwarf Fortress.

[Max™]

It was the same versions, 43.04/43.04 for both and I've had similar results but didn't do exactly the same tests with 43.05.


Hmmm, I was able to get edb running with df and poked around trying to figure out what the hell all this stuff is.

Code: [Select]

[Analyzer] determining function types...
[Analyzer] complete
[Analyzer] elapsed: 830 ms
[Analyzer] region unchanged, using previous analysis
[Analyzer] elapsed: 2 ms
[Analyzer] region unchanged, using previous analysis
[Analyzer] elapsed: 1 ms
[Analyzer] identifying executable headers...
[Analyzer] adding entry points to the list...
[Analyzer] found entry point: 0x408ef6
[Analyzer] attempting to add 'main' to the list...
No main symbol found, calculated it to be  "00000000004050c0"  using heuristic
[Analyzer] attempting to add functions with symbols to the list...
[Analyzer] attempting to add marked functions to the list...
[Analyzer] attempting to collect functions with fuzzy analysis...I couldn't find 0x408ef6 in https://github.com/DFHack/df-structures/blob/master/symbols.xml but I have no idea if that means anything unfortunately, just going by the stuff that I could figure out, I tried to point it at something in the stack which mentioned the keybind for move cursor left, and trace that back to the start of the function a couple times after hitting said key but it froze up when I was trying to analyze the parent I think.

[lethosor]

We will not drop 32-bit support. It's really, really, really useful for memory research. Yes, once 64-bit support is entirely stable, 32-bit support will probably be a lower priority, and might not be complete in later releases, but taking it out is just asking for trouble. Example: we ran into difficult-to-diagnose issues in both the DFHack core and structures because people decided to start developing right away with 64-bit DF only.

Yeah, the unit all/active/bad lists aren't populated yet, ui_advmode isn't known, cur_season, created_item_* globals, various others.

On what build and OS? Those are part of world, so if they're empty and shouldn't be, that's an issue with world's address or layout.

The legacy version of Dwarf Fortress isn't supported by DFHack either, so there's precedence for not supporting everything.

Yeah, but that's not the same thing at all. The legacy version does not use SDL, which DFHack requires in order to hook in. Also, the legacy and SDL versions are otherwise identical besides the rendering engine they use, which isn't true of the 32/64-bit difference.

Edit: That is, barring weird hardware issues, people should be able to run the SDL and legacy builds on the same machine+OS, which isn't true of the 32 and 64-bit builds.

[Max™]

Oh, sorry, Arch Linux, df 43.05 dfhack dev branch.

Code: [Select]

Plugin isoworldremote is missing required globals: cur_season
Plugin strangemood is missing required globals: created_item_count, created_item_type, created_item_subtype, created_item_mattype, created_item_matindex
Plugin sort is missing required globals: ui_building_assign_type, ui_building_assign_is_marked, ui_building_assign_units, ui_building_assign_items
Plugin search is missing required globals: ui_building_assign_units, ui_look_list
Plugin rendermax is missing required globals: cur_year_tick
Plugin zone is missing required globals: cur_year_tick, ui_building_assign_type, ui_building_assign_is_marked, ui_building_assign_units, ui_building_assign_items
Plugin jobutils is missing required globals: job_next_id
Plugin workflow is missing required globals: job_next_id
Could not activate tweak fast-heat (fast_heat_hook::updateTempFromMap)
Could not activate tweak fast-heat (fast_heat_hook::updateTemperature)
Could not activate tweak fast-heat (fast_heat_hook::adjustTemperature)
Cannot enable plugin: search
multilevel is not a recognized command.
DFHack is ready. Have a nice day!
DFHack version 0.43.05-alpha0 (development build 0.43.03-r1-160-g714ba1a)
Type in '?' or 'help' for general help, 'ls' to see all commands.
[DFHack]
# gui/gm-editor df.global.ui_advmode
[string "expression"]:1: Cannot read field global.ui_advmode: global address not known.
stack traceback:
        [C]: in metamethod '__index'
        [string "expression"]:1: in main chunk
        (...tail calls...)
        /home/thefunk/.df/hack/scripts/gui/gm-editor.lua:535: in local 'f'
        ./hack/lua/dfhack.lua:562: in function 'dfhack.run_script_with_env'
        (...tail calls...)
[DFHack]#
I could pull up gui/gm-editor df.ui_advmode but not df.global.ui_advmode, was it moved above the global table or something?

[lethosor]

df.ui_advmode is the type, df.global.ui_advmode is the instance.

Strange, I'm pretty sure world.units was working for me on OS X (both builds), which should be the same layout-wise as Linux.

[Max™]

Just posted the output, and world.units itself is there, but none of the entries were populated, but it might have been my error, I just checked and cleaned everything out to make sure it was all fresh from the dev branch and whatnot and now units is populated right, but the ui_advmode is still not known.

[lethosor]

Okay, glad to hear units looks okay. ui_advmode is missing because nobody has located it.

[Max™]

Yar, also, what is that animal hospitals thing? I can't find it anywhere to kill the message.

[lethosor]

dwarfvet? What is "the" message?

[Max™]

Every time you enter or leave travel mode: 'Clearing all animal hospitals' pops up, yeah, that fixed it.

[PeridexisErrant]

Another announcement of sorts:

It turns out Ruby 1.8 (which we use) doesn't actually compile for 64-bit Windows, at least not without techniques that nobody has investigated. Ruby 2+ would compile, but I don't know if it would work, so I'm waiting on jjyg for that.

Besides that, I can't think of any major blockers for a pre-release (there are some for a stable release). Do note that if we did put up a build now, no ruby scripts would work at all in the 64-bit Windows build. (I suppose now would be a good time to convert "multicmd" to a built-in command, before that becomes an issue.)

Hmm.  Just to float the idea, what are the arguments for and against dropping Ruby support?

Keep Ruby:  seriously, it's fine.  Why break what works?  People actually do use it!

Drop Ruby:

• No API documentation at all
• Relatively poor structures API (I think?  Hard to tell when there's no docs...)
• Standardising on Lua would free up developer resources
• Fewer scripts - there are 190 .lua and 38 .rb files in the dfhack repo; 147 to 28 in the scripts repo.
• Fixing and maintaining Ruby support might be a similar amount of work to dropping it and a one-off porting effort; especially if fixing it involves a useful degree of documentation.
• It makes tooling more difficult (number of languages installed, cases for build systems, docs support, etc) for developers and deters new contributors.

To be clear, I'm not proposing that we should drop Ruby support because it's run into a problem here - it's because Ruby has been a second-class scripting language in DFHack for some time now, the total absence of documentation is annoying me, and it makes supporting other things (eg unifying in-console help) more difficult.

[Warmist]

Another announcement of sorts:

It turns out Ruby 1.8 (which we use) doesn't actually compile for 64-bit Windows, at least not without techniques that nobody has investigated. Ruby 2+ would compile, but I don't know if it would work, so I'm waiting on jjyg for that.

Besides that, I can't think of any major blockers for a pre-release (there are some for a stable release). Do note that if we did put up a build now, no ruby scripts would work at all in the 64-bit Windows build. (I suppose now would be a good time to convert "multicmd" to a built-in command, before that becomes an issue.)

Hmm.  Just to float the idea, what are the arguments for and against dropping Ruby support?

Keep Ruby:  seriously, it's fine.  Why break what works?  People actually do use it!

Drop Ruby:

• No API documentation at all
• Relatively poor structures API (I think?  Hard to tell when there's no docs...)
• Standardising on Lua would free up developer resources
• Fewer scripts - there are 190 .lua and 38 .rb files in the dfhack repo; 147 to 28 in the scripts repo.
• Fixing and maintaining Ruby support might be a similar amount of work to dropping it and a one-off porting effort; especially if fixing it involves a useful degree of documentation.
• It makes tooling more difficult (number of languages installed, cases for build systems, docs support, etc) for developers and deters new contributors.

To be clear, I'm not proposing that we should drop Ruby support because it's run into a problem here - it's because Ruby has been a second-class scripting language in DFHack for some time now, the total absence of documentation is annoying me, and it makes supporting other things (eg unifying in-console help) more difficult.

The biggest issue (at least as i understand it) is that the wrapper for lua is mostly automatically generated from layouts. While the ruby wrapper is handcrafted to some extent. Ideal case would be extending lua wrapper generation to support other languages.
That being said - never liked ruby (well except those few experimentation weeks long time ago, but we all were young... )