Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[Bumber]

Hmm.  Just to float the idea, what are the arguments for and against dropping Ruby support?

Is it Ruby support that's keeping us back on VS2010 for Windows? Edit: Or is the compile page just out of date with respect to 64-bit DF?

I've been meaning to try my hand at compiling DFHack (and possibly finding offsets,) but I don't really want to deal with any issues installing VS2010 over VS2013.

[mifki]

Hmm.  Just to float the idea, what are the arguments for and against dropping Ruby support?

Is it Ruby support that's keeping us back on VS2010 for Windows? Edit: Or is the compile page just out of date with respect to 64-bit DF?

I've been meaning to try my hand at compiling DFHack (and possibly finding offsets,) but I don't really want to deal with any issues installing VS2010 over VS2013.

43.05 requires VS2015 - not 2010 and not 2013. It's not related to Ruby support. The problem with Ruby is that the code contains some assembly language parts which are not supported by VS compiler on 64bit.

[Bumber]

The problem with Ruby is that the code contains some assembly language parts which are not supported by VS compiler on 64bit.

The only assembly code I see in plugins/ruby is in ruby.cpp:

Code: [Select]

#ifdef WIN32
__declspec(naked) static int raw_vcall(void *that, void *fptr, unsigned long a0,
        unsigned long a1, unsigned long a2, unsigned long a3, unsigned long a4, unsigned long a5)
{
    // __thiscall requires that the callee cleans up the stack
    // here we dont know how many arguments it will take, so
    // we simply fix esp across the funcall
    __asm {
        push ebp
        mov ebp, esp

        push a5
        push a4
        push a3
        push a2
        push a1
        push a0

        mov ecx, that

        call fptr

        mov esp, ebp
        pop ebp
        ret
    }
}
Which (I think) we would just add:

Code: [Select]

#else
#ifdef WIN64
__declspec(naked) static int raw_vcall(void *that, void *fptr, unsigned long a0,
        unsigned long a1, unsigned long a2, unsigned long a3, unsigned long a4, unsigned long a5)
{
    // __thiscall requires that the callee cleans up the stack
    // here we dont know how many arguments it will take, so
    // we simply fix rsp across the funcall
    __asm {
        push rbp
        mov rbp, rsp

        push a5
        push a4
        push a3
        push a2
        push a1
        push a0

        mov rcx, that

        call fptr

        mov rsp, rbp
        pop rbp
        ret
    }
}Although, I'm not really sure what this function "raw_vcall" is doing, or why the function at fptr wants a pointer to "that" in the CX register (conventionally the 4th parameter, after DI, SI, DX.)
That means we'd be calling fptr(void *that, void *fptr, unsigned long a0, void *that, ...) if fptr were a C function (which I'm now doubting.)
Should the 64-bit version be using unsigned long long ints? Are long ints upgraded automatically on the 64-bit compiler?

[mifki]

Which (I think) we would just add:

MSVC compiler does not support __asm when compiling for 64bit.

[Bumber]

Which (I think) we would just add:

MSVC compiler does not support __asm when compiling for 64bit.

So you'd have to assemble separately and use extern, if that's even possible with DF?

Or could you use these: https://msdn.microsoft.com/en-us/library/hh977022.aspx?

[mifki]

Which (I think) we would just add:

MSVC compiler does not support __asm when compiling for 64bit.

So you'd have to assemble separately and use extern, if that's even possible with DF?

Or could you use these: https://msdn.microsoft.com/en-us/library/hh977022.aspx?

Yep, it's not a real inline assembly, but just a naked fn, so should be possible to move it to external source.

However, I just noticed lethosor mentioned compiling Ruby itself 1.8 vs 2 for 64bit, I don't know what problems there.

[lethosor]

The ruby plugin should compile on 64-bit Windows since this change: https://github.com/DFHack/dfhack/commit/9da2dcb8a26f7d953ea73a0c0e85ed370f900c8e

Bumber: you're obviously looking at the master branch, while you should be looking at the develop branch. Specifically, the existing "#else" clause should work on Win64, according to Qartar.

The problem is that Ruby itself (before version 2) did not support 64-bit Windows at all. Even if the ruby plugin works, the interpreter will not, so the plugin is useless until we can get Ruby 2 working (possibly not too hard) or Ruby 1.8 working (which could be impossible).

Hmm.  Just to float the idea, what are the arguments for and against dropping Ruby support?

Keep Ruby:  seriously, it's fine.  Why break what works?  People actually do use it!

Drop Ruby:

• No API documentation at all
• Relatively poor structures API (I think?  Hard to tell when there's no docs...)

The structures API is just as good as Lua's (otherwise it would be pretty useless). The way it's implemented makes it somewhat slower when you try to do certain things, from what I remember from BenLubar's projects, but that's not something that would be explained in detail in the API docs anyway.

• Standardising on Lua would free up developer resources

Yes, once we rewrite everything in Ruby, which is not a small task.

• Fewer scripts - there are 190 .lua and 38 .rb files in the dfhack repo; 147 to 28 in the scripts repo.

I got 42 to 11 on the develop branch (https://github.com/DFHack/dfhack/find/develop). It's worth noting that those are all libraries, so all that means is that the parts of the Lua API written in Lua are more comprehensive and/or split into more files.

Yes, Ruby scripts outnumber Lua scripts, but C++ plugin files (244) outnumber both of them combined.

• Fixing and maintaining Ruby support might be a similar amount of work to dropping it and a one-off porting effort; especially if fixing it involves a useful degree of documentation.

Ruby 2 shouldn't be too much harder to get working. It just requires enough familiarity with the Ruby interpreter to fix whatever issues come up, which I don't have.

• It makes tooling more difficult (number of languages installed, cases for build systems, docs support, etc) for developers and deters new contributors.

Developers do not have to install Ruby, or Lua, at all. Both of those interpreters are bundled with DFHack.

[lethosor]

Well, I dropped in the system Ruby 2 library on OS X, made a change from https://github.com/DFHack/dfhack/issues/271#issuecomment-152553877, and "multicmd" works now. "exterminate" crashes on the title screen in 64-bit DF, though, but not in 32-bit DF, which is a bit promising.

[PeridexisErrant]

Cool, thanks for the detailed response.  Let's keep Ruby!

(you *do* need Ruby and Lua to run the complete test suite, for syntax checking)

[gchristopher]

FWIW, Toady had a pretty friendly response to a question that (as a non-contributor to DFHack or Therapist) was really none of my business.

Would you consider making the lives of the DFHack and other tool creators easier somehow? It seems like if Dwarf Fortress had an init or command line option where it would print out the memory address/offsets of all the key structures, then the community would be on much easier footing to quickly catch up to DF releases. That wouldn't expose any proprietary code or be nearly the effort of implementing a full mod API, since all you'd need to add is print statements for requested structure memory addresses, right?

There have been conversations along these lines over the years, though it has been a while on this particular matter.  Utility people can and do PM me.  I really don't recall where it was left last time it came up, or if it was decided that finding memory addresses was the fun part?

[mifki]

FWIW, Toady had a pretty friendly response to a question that (as a non-contributor to DFHack or Therapist) was really none of my business.

Would you consider making the lives of the DFHack and other tool creators easier somehow? It seems like if Dwarf Fortress had an init or command line option where it would print out the memory address/offsets of all the key structures, then the community would be on much easier footing to quickly catch up to DF releases. That wouldn't expose any proprietary code or be nearly the effort of implementing a full mod API, since all you'd need to add is print statements for requested structure memory addresses, right?

There have been conversations along these lines over the years, though it has been a while on this particular matter.  Utility people can and do PM me.  I really don't recall where it was left last time it came up, or if it was decided that finding memory addresses was the fun part?

That's interesting. I asked Toady some questions via PM, including one like this, some time ago, but at some point he didn't answer and I decided that reminding again was a bad idea. I will reconsider.

[milo christiansen]

"finding memory addresses was the fun part"? What planet does he live on?

In other news: I have discovered that "dfhack.gui.makeAnnouncement" (and a few related functions) do not work. The problem is that no matter what you pass for the position argument it will be the wrong type. I have implemented this function completely in Lua as a workaround, so a fix is no big deal for me, but others may want to use this function someday... Anyway, I'm off to file a bug report so this won't be forgotten.

For reference, this is my replacement:

Code: [Select]

-- dfhack.gui.makeAnnouncement does not work. The problem lies in assigning the position, no matter what you pass
-- for the position it fails to work.
--dfhack.gui.makeAnnouncement(df.announcement_type.CANCEL_JOB, df.announcement_flags.D_DISPLAY, unit.pos, msg, COLOR_LIGHTRED)

-- The following is the above function reimplemented (more-or-less faithfully) in Lua

local msg = strings.replace(hook.msg, "%name", dfhack.TranslateName(dfhack.units.getVisibleName(unit)))
dfhack.gui.writeToGamelog(msg)

local continued = false
while msg ~= "" do
local new_rep = df.report:new()

new_rep.type = df.announcement_type.CANCEL_JOB

-- new_rep.pos = unit.pos will fail with an error. This is why dfhack.gui.makeAnnouncement fails.
new_rep.pos.x = unit.pos.x
new_rep.pos.y = unit.pos.y
new_rep.pos.z = unit.pos.z

new_rep.color = COLOR_LIGHTRED
new_rep.year = df.global.cur_year
new_rep.time = df.global.cur_year_tick

new_rep.flags.continuation = continued
new_rep.flags.announcement = true

local nmsg = ""
if #msg > 73 then
nmsg = string.sub(msg, 1, 73)
msg = string.sub(msg, 74)
else
nmsg = msg
msg = ""
end
new_rep.text = nmsg
continued = true

new_rep.id = df.global.world.status.next_report_id
df.global.world.status.next_report_id = df.global.world.status.next_report_id + 1

df.global.world.status.reports:insert('#', new_rep)
df.global.world.status.announcements:insert('#', new_rep)
df.global.world.status.display_timer = 2000
end


[jecowa]

"finding memory addresses was the fun part"? What planet does he live on?

Maybe he meant that it's more fun for him.

[Dirst]

"finding memory addresses was the fun part"? What planet does he live on?

Maybe he meant that it's more fun for him.

The statement makes a lot more sense when you put the ‼ symbols on the fun.

[milo christiansen]

I have long believed that if Toady would just release the header files for DF the world would be a much nicer place...

The statement makes a lot more sense when you put the ‼ symbols on the fun.

I knew something was missing