Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[Lozzymandias]

Are there any long term plans to support multi threading in dwarf fortress, or any other measures to share computational load between cores and/or the GPU. It seems to me that FPS death is probably the biggest obstacle to long-term enjoyment.

[Daniel the Finlander]

--better question would be, can human or elf civ adventurers turn up in sieges? That would be Fun. Populate the nearby forest retreat with elephant man demi-gods then start a fortress mass-producing wood crafts for elf merchants.

Remember to drench your questions in limegreen.

Are there any long term plans to support multi threading in dwarf fortress, or any other measures to share computational load between cores and/or the GPU. It seems to me that FPS death is probably the biggest obstacle to long-term enjoyment.

Yes, but it will be very, very difficult to make multi-threading work. The game will be 64-bit in the near future, so that should help a bit.

[Tygroux]

So I tend to play high-casualties forts where corpses tend to be hard to recover, and slabbing is sometimes difficult.
Is there any plan to add a toggle to fort-mode ghosts? What are their short and long-term purpose?

[TheFlame52]

There already is one. Look in the options.

[PatrikLundell]

@Daniel the Finlander: I don't think 64 bit will speed anything up. If anything it should slow things down very marginally, since pointers are twice as large, and thus use up twice as much memory. The main reason to use 64 bits is that it's capable of addressing huge amounts of memory (using current standards), thus doing away with the 2 GB memory access restriction. Actually using more memory will slow things down since more time will be spent shuffling data between the hard disk and memory, and between memory and the processor (through the caches).

I agree retro fitting threading into a huge non threaded program is a daunting task, but it will probably have to be done at some point. I'd start with stuff that have a minimal data overlap with the rest of the program, with world events outside the fortress for fortress mode being a candidate (nothing happening outside should ideally affect anything inside except when someone from the outside brings the information into it, and vice versa). Implementing that might require two copies of the outside world state, with one being the "truth" and the other being what's known locally and thus being available for engravings etc (not separating data means it has to be protected against concurrent access instead, or you can get nasty hard to track down bugs when data is read at the same time as it's written/updated). The upside of that is that it could enable DF to deliberately twist the local knowledge on synchronization, if desired (i.e. diplomats, traders, and spies feeding lies and propaganda to the fortress, "fog of war" might obscure some knowledge about distant parts of the world, etc).

[Max™]

Being able to load more than 2 GB and have it accessible from speedy ram shouldn't be slower than shuffling 2 GB of data back and forth from the hard drive is.

[King Mir]

Being able to load more than 2 GB and have it accessible from speedy ram shouldn't be slower than shuffling 2 GB of data back and forth from the hard drive is.

AFAIK, DF doesn't use the file system to shuffle things back and forth, except when saving and loading. So no gain there.

It really is a trade off between larger embarks on 64 bit and a slight performance advantage on 32 bit.

[Max™]

Being able to load more than 2 GB and have it accessible from speedy ram shouldn't be slower than shuffling 2 GB of data back and forth from the hard drive is.

AFAIK, DF doesn't use the file system to shuffle things back and forth, except when saving and loading. So no gain there.

It really is a trade off between larger embarks on 64 bit and a slight performance advantage on 32 bit.

Maybe not in fortress mode, but adventure mode you can definitely see the difference when you are roaming around on foot and you cross into an empty hunk of desert vs a chunk of densely populated town with an insurrection taking place. If that was all sitting in ram already it wouldn't take as long, and indeed it wouldn't have the same pause at each new embark tile where the game pulls it up and unloads the tiles which are totally out of sight.

[PatrikLundell]

I don't play adventure mode, so I don't have any first hand knowledge of it. However, if DF loads and offloads tiles in adventure mode it would do that regardless of whether the code is compiled as 32 or 64 bit. Using 64 bit would make it possible to change the DF code to load everything into (virtual) memory instead however, resulting in a longer initial load time but avoiding the program loading/offloading data to/from the disk during the game. If the physical memory is insufficient (including the OS deciding DF shouldn't get more), you'd still get page fault loading/offloading as the OS swaps memory to/from disk during play.

Just to clarify: I think a move to 64 bits is a correct one and needed eventually regardless. However, it's just an enabler that doesn't achieve anything beyond larger embarks (that are faster to die the FPS death) on its own; additional work is needed to use the potential provided.

[Dirst]

I don't play adventure mode, so I don't have any first hand knowledge of it. However, if DF loads and offloads tiles in adventure mode it would do that regardless of whether the code is compiled as 32 or 64 bit. Using 64 bit would make it possible to change the DF code to load everything into (virtual) memory instead however, resulting in a longer initial load time but avoiding the program loading/offloading data to/from the disk during the game. If the physical memory is insufficient (including the OS deciding DF shouldn't get more), you'd still get page fault loading/offloading as the OS swaps memory to/from disk during play.

Just to clarify: I think a move to 64 bits is a correct one and needed eventually regardless. However, it's just an enabler that doesn't achieve anything beyond larger embarks (that are faster to die the FPS death) on its own; additional work is needed to use the potential provided.

Pre-fetching can be done more intelligently than you're suggesting, and virtual memory is somewhat faster than file I/O.  If the physical RAM is available you'll see an orders-of-magnitude improvement.

64-bit also adds some future-proofing before 32-bit support gets crufty.

[vomov]

What with the possibility of 64-bit, and we've already got SDL and legacy versions, I be curious:
Will we get a boatload of different versions, or a 'catch-all' download that figures out what we need?
It's not like the DF zip is huge or anything

[Knight Otu]

What with the possibility of 64-bit, and we've already got SDL and legacy versions, I be curious:
Will we get a boatload of different versions, or a 'catch-all' download that figures out what we need?
It's not like the DF zip is huge or anything

Separate downloads, in all likelihood.

Have you decided how you will distribute the 64-bit builds (once they're ready)? Will it be a single ZIP per OS, containing both 32 and 64-bit executables, or one ZIP for OS/bitness combination?

I'll probably have a bunch of separate zips, but the list is getting long the way it is formatted now.  But that's the way I'm leaning.  We'll also enter a sort of annoying save compat territory -- even if I manage compat between the bitnesses, 64-bit saves will often be too large for 32 bit DF to load.  I imagine there could be "false" bug reports over this, but I'm not sure how to handle it without spending a zillion years coding calculations estimating memory sizes before load or something.

[Dirst]

Separate downloads, in all likelihood.

Have you decided how you will distribute the 64-bit builds (once they're ready)? Will it be a single ZIP per OS, containing both 32 and 64-bit executables, or one ZIP for OS/bitness combination?

I'll probably have a bunch of separate zips, but the list is getting long the way it is formatted now.  But that's the way I'm leaning.  We'll also enter a sort of annoying save compat territory -- even if I manage compat between the bitnesses, 64-bit saves will often be too large for 32 bit DF to load.  I imagine there could be "false" bug reports over this, but I'm not sure how to handle it without spending a zillion years coding calculations estimating memory sizes before load or something.

I think it would be easier for the saving client to write something in the file about how much memory was needed (or predicted to be needed) to handle that save.

[Thief^]

Or just a "saved from 64-bit" flag that makes the 32-bit version pop up a warning that it might crash when it tries to load it?

[PatrikLundell]

A good way to handle out-of-memory problems is to always check the pointer returned on memory allocation before using it rather than having the program go belly up trying to use a null pointer, and instead terminate gracefully with an appropriate message when it happens. I'd probably search the code base for the allocation operations used and add that handling to each place (or preferably to those operations themselves, if they're not primitives/library calls). That wouldn't just take care of the 32 bit load of 64 bit save case (given all other compatibility things were handled already), but would also deal with the 32 bit bug reports of "my 6*6 embark suddenly crashes shortly after loading the save and continuing".
If you just wanted to handle the load game case, only the memory allocation calls used there would need to be checked before use, but I'd do it throughout.