Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

Faster, no (unknown) problems

Faster, problematic

Same speed, no (unknown) problems

Same speed, problematic

Slower, no other (unknown) problems

Slower, problematic

Doesn't work at all

[Lord Nightmare]

Re: 64-bit compilation, near as I can tell compiling it in 64-bit mode would take no more than throwing a switch in the makefile.

The main problem is toady's lack of an installed 64-bit linux system. It should be simple enough to get around, given that his *hardware* is already 64-bit, but that's something to look at later. It'd be no more than a few percent speedup, anyway.

MAME at least gained a pretty respectable 10% or so speedup for some systems when compiling in 64bit. I don't know how applicable that is to the way data is arranged in DF tho.

LN

[CobaltKobold]

When you say that Adventure Mode is to be broken until DF2010, do you plan to release a final 40d## with the Adventure mode fix (if it is something that can be fixed on your end of the programming.) Even though there will be the new DF out?  So people don't have to give up any adventurers they managed to keep alive in 40d16?

Can't you just move your adventurers back to 40d16 (or 40d11 or 40d)?

Do mind that d16's still got the "death lockup", so go back farther.

Put me down for "sharp edges".

[Aachen]

Haven't noticed this mentioned yet (though it's quite likely I'm just inattentive) but on loading a file, I get a one- or two-second screen image that appears to be from the penultimate screen save.

[Andir]

I know this is somewhat off topic, but...

As far as I can tell, compiling in 64 bit would actually slow down a program. It would require all pointers to be twice as large, and likely many integers in the code would also be widened to 64 bits. So unless there are performance gains due the ability to use more than 2GBs of RAM inside the process, you will just have a larger memory footprint, and increased pressure on the caches.

Running on a 64bit OS on the other hand can help as the extra memory space could mean less swapping memory to external storage.

That's not really how it works....  64-bit processors are meant to run 64-bit operations, having them deal with 32-bit ops can slow it down slightly.  So moving to a native 64-bit compiler/compile can actually speed things up but it's really a crab shoot.  You're more likely to see a speed increase than you are a decrease.  The processor is still processing the same amount of instructions per cycle with 64 or 32 bit data, but most 64-bit processors have to do special instructions to handle 32-bit data so there's that overhead removed.

The application memory footprint is only as big as the developer makes it.  I've heard your argument before when people talk about making integers use 64-bit addressing and while true on it's double use of RAM, it doesn't make it slower per say.  If the developer uses proper structures the size of the application moving from 32-bit to 64-bit is not going to eat up more memory.  This is mainly dealing with higher level languages like .NET where they have the possibility of having an Int64 data type and automatically forcing every old application using "integer" to use this new Int64 instead of Int32 which is the default integer type now.  Any application that uses "integer" as a generic would then take more space than if the developer had used Int32.  (I used "int" here, but the same applies for other data types as well.)  I'm assuming Toady is using a fairly recent compiler that interprets integers as 32-bit values, short int is 16-bit.  I doubt he's specifically assigning them specialty types like __int32, __int64, int32_t, int64_t or 'long long int' that are included in some compilers so it really depends on the compiler and most of them are sticking to 32-bit integers.

Pointer sizes are really irrelevant in terms of speed as well.

Edit:  Here's an article that seems to give more detail on why 64-bit native apps can (in most cases) be faster: http://www.pcstats.com/articleview.cfm?articleid=1665&page=1

It points out how math intensive applications benefit the most from the 64-bit registers, but it shows a general improvement for most everything else.

[Baughn]

Andiir is basically right. DF is written in C++, and C++ (or C, for that matter) on 64-bit intel computers use the same size types as on 32-bit. The  only difference is that pointers are 64-bit, and the 64-bit long long uses native instructions instead of emulation.

However, DF also does a few bulk-data operations (memcpy, memset, etc.), which definitely *are* faster on 64-bit. So overall, I'd expect a slight improvement.

EDIT: Oh, and 32-bit integer operations are not slower than 64-bit operations on 64-bit processors. At least, not intel's 64-bit processors. They know perfectly well that a lot of applications use them, so the CPUs are still tuned for those.

In fact, due to using less cache space and memory bandwidth, they can be significantly faster.

[madman]

On 64 bit Linux (using gcc):

sizeof(char)=1
sizeof(short)=2
sizeof(int)=4
sizeof(long)=8
sizeof(long long)=8
sizeof(pointer)=8

so the sizes for int downwards are the same as for 32 bit, I'm not sure about longs on 32 bit.

I went to a seminar on this topic a few years ago, and the general result of moving for 32 to 64 bit seemed to be:

* Pointers get larger -> more memory -> more pressure on cache -> slightly slower
* More registers (only available in 64 bit mode) -> significantly faster
So overall things tend to be faster, as the register starvation of 32-bit mode gets avoided.

[As an aside, on PowerPC, moving from 32 to 64 bit made things slightly slower, as they get the larger pointers with no increase in registers].

My reasons for wanting 64 bit are only marginally to do with speed anyway - I'm more bothered about having to install lots of 32 bit libraries that the package system won't maintain. This could make them a security issue, and I'd really like to be able to get rid of them.

The only potential showstoppers I see for 64 bit are:
* If there's lot of coercion between ints and pointers. This would make 64 bit unlikely to ever be possible,
* Toady doesn't want to maintain more targets.

[bombcar]

The other show-stopper being the doubling of versions from the current three to six as you make a 64 bit version for each OS.

[Baughn]

It is highly unlikely there would be a 64-bit version for windows, nor that there would be one for OS X so long as it starts up with a 32-bit kernel by default.

Besides that, having a 64-bit linux version would just mean overhauling the build script to build both variants. It'd double compilation time, though, which is already pretty high.. if Toady had a separate linux computer that'd be a non-issue, but he's dual-booting his windows machine as it is now. It might mean not building a 32-bit version at all. How many people still use 32-bit Linux, anyway?

Oh, and DF contains absolutely zero pointer<->integer conversions, but quite a few longs. I've seen no patterns of code that would fail if those longs get larger, but it'd probably be a good idea to replace them with ints anyway.

[CobaltKobold]

How many people still use 32-bit Linux, anyway?

*raises claw*

[madman]

Dropping 32 bit support would be a bad move. The are people who still use 32 bit (often for compatibility reasons with things like Dwarf Fortress? chicken and egg problem). 32 bit DF can be got to run on 64 bit Linux (with some tweaking and grumbling), 64 bit programs can not be got to run on 32 bit systems (AFAIK).

I wouldn't think compilation time for an extra target is too big of a deal - how often would he need to compile, except near a release? Testing time for another target would surely be a much bigger hit.

[shadow_slicer]

I went to a seminar on this topic a few years ago, and the general result of moving for 32 to 64 bit seemed to be:

* Pointers get larger -> more memory -> more pressure on cache -> slightly slower
* More registers (only available in 64 bit mode) -> significantly faster
So overall things tend to be faster, as the register starvation of 32-bit mode gets avoided.

That's basically correct. The extra registers available in 64 bit mode can really make a large impact on computations. When I was working on the path-finding stuff I was seeing a 5x improvement in pathfinding just by dropping the -m32 compilation flag. Obviously DF will not see such a large improvement, but something like 10%-20% would be quite reasonable.

[madman]

That's basically correct. The extra registers available in 64 bit mode can really make a large impact on computations. When I was working on the path-finding stuff I was seeing a 5x improvement in pathfinding just by dropping the -m32 compilation flag. Obviously DF will not see such a large improvement, but something like 10%-20% would be quite reasonable.

shadow_slicer, was it you that added the "64bit native DF" paragraph to the eternal suggestions at http://www.bay12games.com/forum/eternal_voting.php ? Can I suggest you add something about ease of installation on 64-bit systems (not having to install a difficult to maintain 32 bit subsystem)? For me, that is the compelling reason to want a 64 bit version, not the speed increase.

I'm _very_ surprised at the 5x improvement in the pathfinding - that's off the charts compared to any improvement that I've ever seen (although I don't bother with 32 bit at all anymore). It must be something where the x86 register spill really hurts.

[Malicus]

Put me down for "sharp edges", as well.  ...granted, if both are available, then it makes the whole thing moot.

[shadow_slicer]

That's basically correct. The extra registers available in 64 bit mode can really make a large impact on computations. When I was working on the path-finding stuff I was seeing a 5x improvement in pathfinding just by dropping the -m32 compilation flag. Obviously DF will not see such a large improvement, but something like 10%-20% would be quite reasonable.

shadow_slicer, was it you that added the "64bit native DF" paragraph to the eternal suggestions at http://www.bay12games.com/forum/eternal_voting.php ? Can I suggest you add something about ease of installation on 64-bit systems (not having to install a difficult to maintain 32 bit subsystem)? For me, that is the compelling reason to want a 64 bit version, not the speed increase.

Done

I'm _very_ surprised at the 5x improvement in the pathfinding - that's off the charts compared to any improvement that I've ever seen (although I don't bother with 32 bit at all anymore). It must be something where the x86 register spill really hurts.

I was also very surprised, and at first thought something had gone horribly wrong. My initial version always used 32-bit mode, but then when we were integrating it with Khazad, I noticed that it ran around 5 times faster in Khazad than by itself . I initially thought I had messed something up, but it passed all of my unit tests. I went through all the differences between the two and then compared the compilation options. I was finally able to replicate the speed increase by compiling my version in 64 bit mode. It was actually very frustrating  .

[quinnr]

I use 32 bit linux -.-