Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[Zarathustra30]

So, my extra large, thousand-year world finally finished, and I was able to embark on a 16x16 site that took about 20 minutes to load. There were about 6 minor cave-ins upon embark, plus some other crazy vertical shafts, all localized to the western side. I will post the world as soon as I re-embark to see if it does it again.

Edit: here's the world: http://dffd.bay12games.com/file.php?id=12192

I did a 16x16 embark on the volcano in the northeastern continent. The second time I embarked, the same thing happened. I'm not sure if this is a problem, because the game is playable. It's just weird.

Edit2: I forgot to mention, I am on Win 10 Pro.

Edit3: Embarked on a 16x2 along the same area and the problem didn't reproduce.

[Toady One]

Brought to you from the ancient past:
------------
Linux:
dpkg --license: 1.14.20ubuntu1 (i386)
lsb_release -a: Ubuntu 8.10, codename intrepid

Mac:
sw_vers -productVersion: 10.5.6
system_profiler SPHardwareDataType:

Model Name: Mac mini
Model Identifier: Macmini2,1
Processor Name: Intel Core 2 Duo
Processor Speed: 1.83 GHz
Number Of Processors: 1
Total Number Of Cores: 2
L2 Cache: 2 MB
Memory: 1 GB
Bus Speed: 667 MHz
Boot ROM Version: MM21.009A.B00
SMC Version: 1.19f2

------------
Is there hope?

[mifki]

Brought to you from the ancient past:
------------
Linux:
dpkg --license: 1.14.20ubuntu1 (i386)
lsb_release -a: Ubuntu 8.10, codename intrepid

Mac:
sw_vers -productVersion: 10.5.6
system_profiler SPHardwareDataType:

Model Name: Mac mini
Model Identifier: Macmini2,1
Processor Name: Intel Core 2 Duo
Processor Speed: 1.83 GHz
Number Of Processors: 1
Total Number Of Cores: 2
L2 Cache: 2 MB
Memory: 1 GB
Bus Speed: 667 MHz
Boot ROM Version: MM21.009A.B00
SMC Version: 1.19f2

------------
Is there hope?

Yep, Ubuntu is ancient. I'd recommend just reinstalling it with something newer (maybe not latest, to avoid problems with compatibility for players on older systems). Like 15.10 or 14.04 is fine. If you're not using Linux for anything else, that should not be difficult to reinstall it + packages required to build DF.

Mac is fine, I'd upgrade it to at least 10.6, but that's not necessary. However, I need to check about 64-bit support in GCC. EDIT: What "gcc -v" says?

The main thing you need to decide is whether you want to use ancient GCC 4.5 or something newer (which may be easier to install on newer systems).

[Quietust]

Ubuntu 10 or 11 ought work just fine, and they're both available in 64-bit (which is generally a prerequisite for x86/x64 multiarch support - building 32-bit binaries on a 64-bit system is easy, but building 64-bit binaries on a 32-bit system is a lot more work).

Tried to benchmark the 32 bit test versus the 64 bit test in world generation, on pre-history generation the 64 bit version was much faster with identical results, but even though the seeds were identical, their histories differed by a lot and I ended the test there, there though the 32 bit version didn't seem as left behind as earlier, but since the results differed I would take that with a few grains of salt.

Do the 32-bit and 64-bit versions, within themselves, consistently generate the same world with the same seeds? The fact that 32-bit and 64-bit generated different worlds would be meaningless if they themselves weren't able to generate the same world twice.

Yeah, I don't know that we'll ever have the different versions generating the same world.  Compiler optimizations have shifted around the order of calls before in a way that I can't easily control.

Optimization should never affect the actual behavior of the code - they may reorder individual instructions in order to minimize the impact of memory/cache latency, but the actual operations should always be the same. If 32-bit and 64-bit code behave differently, then it's due to a bug in the code itself (possibly from data types being different sizes, though that isn't a problem on Windows like it is on Linux/Mac).

[Toady One]

Ubuntu 10 or 11 ought work just fine, and they're both available in 64-bit (which is generally a prerequisite for x86/x64 multiarch support - building 32-bit binaries on a 64-bit system is easy, but building 64-bit binaries on a 32-bit system is a lot more work).

I don't currently know how to decide between 10 all the way up to 15.  Would using 14.04/15.10 cut anybody out?

Yeah, I don't know that we'll ever have the different versions generating the same world.  Compiler optimizations have shifted around the order of calls before in a way that I can't easily control.

Optimization should never affect the actual behavior of the code - they may reorder individual instructions in order to minimize the impact of memory/cache latency, but the actual operations should always be the same. If 32-bit and 64-bit code behave differently, then it's due to a bug in the code itself (possibly from data types being different sizes, though that isn't a problem on Windows like it is on Linux/Mac).

I remember some nightmare years ago that seemed to come down to some issue with what MSVC 6 was doing vs what I was getting elsewhere, but it has been long enough that I don't recall any details.  Maybe that came down to a type issue too.  In either case, I doubt I'll be able to clean up the code to the point where we can get reproducible worlds between OSs/bits.

[lethosor]

I don't think using a newer OS would cut anyone out. When it comes to Linux, I might recommend using an LTS version of Ubuntu so you have longer before support for it is officially dropped, but that's probably up to you. Using a newer GCC might break compatibility for some people, but that should be relatively easy to fix by distributing the newer libstdc++ that comes with it. (Back in 2010, GCC 4.5 was fairly new, which is probably why you had to add libstdc++ then.)

Regarding OS X - I didn't know you were running 10.5, wow. It looks like that model should be 64-bit, though, according to specs I found online, so try passing -m64 to GCC and see what happens. ("uname -p" actually returns "i386" on my machine too, which is definitely not a 32-bit one.) If that doesn't work, I think newer GCC versions would work there, and some people seem to have had success with it.

Anyway, if you do end up having to rebuild compiler(s), my suggestion is to go with a newer GCC version (since GCC 4.5 is pretty unsupported) and the same version on both OSes that use it. Although if you can make the existing ones work, that's probably fine too.

[Quietust]

I just generated the same exact world twice in a row with the 64-bit Windows build, and history turned out differently each time - I used the seeds ABCDEFGHIJKLMNOPQRST + UVWXYZabcdefghijklmn + opqrstuvwxyz01234567 + 89ABCDEFGHIJKLMNOPQR on a MEDIUM REGION with a 100 year history, and while both worlds had the same geography and initial historical figures and sites/structures, history itself turned out completely differently. The problem may simply be that history is not deterministic, and the bug tracker confirms that this has been a problem since at least version 0.40.

[Random_Dragon]

Did you have rain shadows on? :V

[mifki]

Anyway, if you do end up having to rebuild compiler(s), my suggestion is to go with a newer GCC version (since GCC 4.5 is pretty unsupported) and the same version on both OSes that use it. Although if you can make the existing ones work, that's probably fine too.

No need to build GCC manually, use Homebrew on OS X http://brew.sh, it supports 10.5, and standard packages on Linux.

[lethosor]

https://github.com/Homebrew/brew/blob/master/share/doc/homebrew/Installation.md recommends OS X 10.9 or higher. (But if it works, yeah, it's probably easier. If not, I might be able to provide a GCC build that works on 10.5.)

[Random_Dragon]

I forget the exact name of the setting, but it's the one that's been known for a long time to introduce variations in map layout with the same seed, which could affect the resulting history.

[Thundercraft]

64 bit has nothing to do with speeding up the game or multithreading.

Perhaps speeding up the game was not the main goal for developing a 64-bit DF. But your statement is misleading. A 64-bit app has the potential to be significantly faster than a 32-bit counterpart because it allows programming tricks to do things faster. This includes reading and writing 64-bits of data in a single instruction and letting different threads manipulate a shared variable at the same time. And, yes, a 64-bit app has the potential to take better advantage of multithreading and multiple cores than a 32-bit app.

The development of a 64-bit branch of any software project brings an expectation that it will be superior to the 32-bit version in some way. It should be more secure, more stable (i.e., less crashes due to running out of memory), run faster, or perform better in some way. Otherwise, what's the point? The move to 64-bits (not to mention maintaining separate 32-bit and 64-bit versions) is extra work.

In the case of games, especially, a move to 64-bits is almost always done for either better performance or access to more than 4 GB of memory. Usually both.

The fear was that using 64 bit would actually slow down the game. Which thankfully doesn't seem to be the case.

If the 64-bit version was significantly slower than the 32-bit version, I doubt many players would use it. Again, what would be the point?

From 34.11 to 42.xx, there have been reports of a huge drop in FPS and FPS Death is now much more common. And, as more and more features get added to DF, this situation is only going to get worse.

Just giving pathfinding a separate thread, alone, should see a huge boost in performance. It's been more-or-less proven that pathfinding in DF is the single biggest performance hit.

...It would require a lot of the features to be rewritten, like pathfinding...

Pathfinding would have be modified, sure. But I seriously doubt it would have to be completely rewritten merely to migrate it to a separate thread.

[lethosor]

A 64-bit app has the potential to be significantly faster than a 32-bit counterpart because it allows programming tricks to do things faster. This includes reading and writing 64-bits of data in a single instruction

Well, yes, but pointers (i.e. memory addresses) are 64 bits in a 64-bit program, so the speed of working with pointers shouldn't change. Other optimizations (e.g. working with two 32-bit integers at the same time) aren't nearly as common.

and letting different threads manipulate a shared variable at the same time. And, yes, a 64-bit app has the potential to take better advantage of multithreading and multiple cores than a 32-bit app.

That's not true at all. There's no difference between what 32-bit and 64-bit programs can do with threads, or CPU cores.

The development of a 64-bit branch of any software project brings an expectation that it will be superior to the 32-bit version in some way. It should be more secure, more stable (i.e., less crashes due to running out of memory), run faster, or perform better in some way. Otherwise, what's the point? The move to 64-bits (not to mention maintaining separate 32-bit and 64-bit versions) is extra work.

Well, the main motivation here was to stop crashes from DF hitting the memory limit, which have gradually become more common with large worlds in newer versions.

In the case of games, especially, a move to 64-bits is almost always done for either better performance or access to more than 4 GB of memory. Usually both.

The performance difference is pretty small (as a couple people have pointed out already).

Just giving pathfinding a separate thread, alone, should see a huge boost in performance. It's been more-or-less proven that pathfinding in DF is the single biggest performance hit.

...It would require a lot of the features to be rewritten, like pathfinding...

Pathfinding would have be modified, sure. But I seriously doubt it would have to be completely rewritten merely to migrate it to a separate thread.

It's not that simple. I don't know how much would have to be rewritten, but what people don't always realize is that pathfinding needs to be in sync with the rest of the events happening in-game, so it's unlikely that a single pathfinding thread and another "other processing" thread would both be running at maximum speed. Sure, a pathfinding thread could compute the paths of all dwarves in the fortress for the next 5 years while the game is paused or not much is going on, but if a dwarf's job changes or a map change blocks some of those paths, those paths will need to be recalculated. (There's also a lot of inter-thread communication that would need to be added, which doesn't exist at all currently and is tricky to get right.)

[Max™]

Ubuntu 10 or 11 ought work just fine, and they're both available in 64-bit (which is generally a prerequisite for x86/x64 multiarch support - building 32-bit binaries on a 64-bit system is easy, but building 64-bit binaries on a 32-bit system is a lot more work).

I don't currently know how to decide between 10 all the way up to 15.  Would using 14.04/15.10 cut anybody out?

14.04 is a LTS release so it'll be easier to keep going, the newest one is 16.04 (they're literally the year+april or october for ubuntu, oct 2015 was 15.10, etc) and you should be able to update the relevant compiler packages for a while.

[KillzEmAllGod]

snip

Well this is Toady we're talking about and trying to get him to multithreading won't happen just by telling him that its so much better, he already knows that he just needs to learn how to do it. The other matter is that it might take too much time just to get multithreading to work, hes pretty much said that he wouldn't want to stop making the game just to spend 2 years rewriting it so its multithreaded. Did hear that in the suggestion that just moving to 64 bit could end up with pathfinding being 5 times faster, though that might have been before running and jumping, its also early in so there might be more gains to come.

Anyway I just want magic and economy, this was rather quick for being 64 bit though I'm sure there's more to be done before the next part of the game he works on.
Toady did such a good job with 43 and the extra features of armor and weapon damage.