Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[Willfor]

I thought 31.x was the military arc.  At least, the first few releases of it.  That's where the new military screen, schedules, alerts, etc. came from.

That was all preparatory work for the coming military changes. The framework had to be set in place, and it would have been a major save-breaking feature, so he threw it in while he was already doing a lot of save breaking features. Now he has the framework, it just needs expanding upon.

[Footkerchief]

One has to wonder why he called the military arc finished when there are so many problems with ranged soldiers.  That's supposed to be EA's thing, not Toady's.

There's never been any such thing as a "military arc" and Toady has never called anything like that finished.  Where did you get that idea?  He's well aware of the equipment/marksdwarf bugs, and will probably continue fixing them after the next feature-oriented release.

I thought 31.x was the military arc.  At least, the first few releases of it.  That's where the new military screen, schedules, alerts, etc. came from.

0.31.01 included a military overhaul, but as stated above, that's just groundwork for upcoming military stuff (a.k.a. the Army Arc).

[Makbeth]

I see.  Sorry, I was under the impression that the military stuff we saw in the first few 31.x releases was all the attention that the military was going to get for a while, now that the focus has been on adventuring and caravans.

[Hiiri]

I don't think anyone's mentioned this yet; soap making is still broken, lye stays in the bucket when it needs to be in a barrel. 

[Sutremaine]

Lye will get moved to a barrel eventually so long as the barrel is in a lye stockpile. I'm not sure if the stockpile needs to be lye-only, but that's the way I always set it up.

[Hiiri]

Happened to me too once or twice (out of a hundred?) too. Maybe I'm just doing it wrong, but it really seems a bit too random to be a viable workaround.

[Spectre Incarnate]

Hi, I asked my brother what he thought of the multi-threading conversation. He is a very talented computer scientist and codemonkey, the co-creator for a platformer/RPG called "Grimmy Gold Edition", and has designed many useful tools for World of Warcraft. He has allowed me to quote him.

Spectre: I was curious what you thought of this comment on the Dwarf Fortress boards.

And since CPU manufacturers are now focusing more on multithreading than individual core performance, I suppose we can't even hope that it will run that much better on future computers.  Oh well.

Tanaalethan: Exactly my sentiments.
Spectre: Your rig was able to run a 16x16 grid though, when most people who have decent gaming rigs can't play with anything past 5x5.
Tanaalethan: I did lag a little bit though. I'd probably be fine at around 12x12.
Spectre: That's still awesome and you know it.
Tanaalethan: I also turbo at 5.2GHz, though. Most computers are 3.2-3.4 at most.
Spectre: *major tower envy*...Right, but we're talking future computers and yours is pretty future in comparison to a standard comp.
Tanaalethan: Future computers are not getting faster, they're getting clustered. When they talk "multithreading" they're talking about the multi-core CPUs.
Spectre: So each processor works on something different, but not any faster?
Tanaalethan: Exactly.
Spectre: Well, suck. Haha
Tanaalethan: Most applications nowadays have a multithreading function, meaning it can make use of all the cores, making the application split between open threads. This makes the task go faster as each core is working on a different aspect. Say, maybe you need a big image rendered. The application would say "Ok, you have 6 cores. I'm gonna split this into 2 rows, 3 columns. Each core is going to work on a single cell." or maybe "each core is going to work on its own aspect. This one's set on lighting, this one for the normalizing, these two are on the pre-render, these two on the colorizing, then after that [is finished], two are on post-process, and the other four are free."
Spectre: And since Toady isn't planning on multi-threading DF because he isn't sure it'll make things go faster, we're pretty screwed either way?
Tanaalethan: Actually.. if he added multithreading, DF would get exponentially faster. Just a simple dual core multithread would make any computer there that has to stop at 5x5... able to run a 10x10 or more. Basically, as it stands, DF is limited by CPU speed. If he added Multithreading, DF would be limited by system memory. He would also be able to add a lot more functions, since a computer like mine could run a 25x25-30x30 [if it supported that size], and it caps out at 16x16 (good size anyways) he could use the extra processing space to do other stuff.

Spectre: What about this person's comment?

Toady isn't avoiding a multithreaded approach based on principle.  It simply isn't clear that the potential performance gains from a naïve attempt at parallelism would win out against the inherent failures of parallel computing.  Namely, the memory accesses of pathfinding are dense, and there are potential gains with regards to caching which disappear once threading is implemented (unless you're really super clever about it).  My point here is that if toady were to throw together some slap-dash multithreading for pathfinding, it would quite possibly end up being slower than a single-threaded approach.

Tanaalethan: caching doesn't go out the window when you multithread

Sorry he couldn't elaborate any further, we both had errands to run at that point. If you guys have any questions for him, let me know.

[Makbeth]

Wow, thanks for that.

But something seemed odd in that conversation.  I was under the impression that embark size wasn't limited by processing power, but by memory.  Embark crashes always seemed to be happening when the DF process was using a certain amount of memory, the value of which was constant regardless of where the embark was or what was in it.  The rule seems to be to stay below that limit if you want DF to not crash, which for most world gens means somewhere below 16x16.  There's been a couple times when I've gotten my worlds to load in 16x16, but since they have 40 extra layers between levels 3 and 4, that doesn't happen often.

I've been led to believe that this constant value is the memory limit for 32 bit operating systems.

Something else I've been wondering though:  Photoshop seems to exceed this limit and keep running just fine when I open my 7 GB world topography map.  Why can some programs exceed the limit and others not?  Would Toady be able to code DF someday so it can continue to run with huge embarks?

[Spectre Incarnate]

I'm going out to dinner, so I'll give him that info after I get back. No guarantees that he'll answer, he doesn't like to talk to me for very long.. I am his boring sister after all. 

[Thief^]

Something else I've been wondering though:  Photoshop seems to exceed this limit and keep running just fine when I open my 7 GB world topography map.  Why can some programs exceed the limit and others not?  Would Toady be able to code DF someday so it can continue to run with huge embarks?

Either AWE (the software equivalent of the hardware feature PAE), which allows a program to manually page ram in and out of its 2GB virtual memory space, allowing it to "access" more than 2GB, or it simply uses a file on disk.

It's a real PITA to program for either way, building a 64-bit executable of DF would be far easier.

[Khift]

Other things to add to the OP:
- Baron appointment bug; you have to put off meeting with the diplomat until after the caravan has already left the map in order to get the prompt to promote a Baron.
- Whips cutting through steel armor like cheese.
- FBs / Titans / ampersands with undirected dust attacks capable of instagibbing dwarves and even killing themselves. (Due to using cave-in code.)
- Sun Berries not spawning in good areas.
- Overly aggressive farm animals. People are reporting having five camels in a max size pasture and still getting nonstop fighting.
- No job cancellations when trying install a new beehive, so if you have two beekeepers if both try to split the same hive one will end up incapable of completing the job indefinitely.

All relatively minor, admittedly, but still present. If I had to decide which was the most important I'd say the whip issue is because those things are pure murder.

[Makbeth]

Something else I've been wondering though:  Photoshop seems to exceed this limit and keep running just fine when I open my 7 GB world topography map.  Why can some programs exceed the limit and others not?  Would Toady be able to code DF someday so it can continue to run with huge embarks?

Either AWE (the software equivalent of the hardware feature PAE), which allows a program to manually page ram in and out of its 2GB virtual memory space, allowing it to "access" more than 2GB, or it simply uses a file on disk.

It's a real PITA to program for either way, building a 64-bit executable of DF would be far easier.

Ah.  Is that what the "scratch disks" are for then?

Pity.  That ceiling is inconveniently low, not just for DF, but for my rendering software. 

[Spectre Incarnate]

Okay, we're back. My stomach didn't appreciate going out to dinner yesterday for some reason. 

But something seemed odd in that conversation.  I was under the impression that embark size wasn't limited by processing power, but by memory.  Embark crashes always seemed to be happening when the DF process was using a certain amount of memory, the value of which was constant regardless of where the embark was or what was in it.  The rule seems to be to stay below that limit if you want DF to not crash, which for most world gens means somewhere below 16x16.  There's been a couple times when I've gotten my worlds to load in 16x16, but since they have 40 extra layers between levels 3 and 4, that doesn't happen often. I've been led to believe that this constant value is the memory limit for 32 bit operating systems."

Tanaalethan: He's saying a 16x16 map will cap out the memory limit, and the lag is from lack of memory. Memory limit for 32-bit operating systems is 3.24 GB. I still have the 16x16 saved, so I'll check it's memory usage.

Process: DwarfFortress.exe *32
CPU Usage: 16.666% Total | 100% Core 1 | 0% Core 2 | 0% Core 3 | 0% Core 4 | 0% Core 5 | 0% Core 6
CPU Usage Uptime: 98.678%
Physical Memory Usage: 1,511,216 k (1.44GB )

At 16x16, it's capping out the CPU, but it's going nowhere near the 32 bit mem limit. It's using (currently) 1.44GB out of 3.24GB (if there's a software cap of 32bit max), or out of 8GB. Granted, this is on a hexacore 5.2GHz [overclocked], 8GB RAM, and 4.2GB video RAM. Just be demographic, I can see most people playing this game as having older computers.
Spectre: Right, they do, but the question is if DF is limited by memory even if a CPU is larger or multi-core? For future computers in the next couple of years or so?
Tanaalethan: Multithreading is the future. All CPU manufacturers are done making single core processors. 3.4 is the fastest they'll go without overclocking. All that's gonna happen, is you'll see 3.2-3.4GHz processors with more cores.
Spectre: Is the ability to embark without crashing on a higher zone size caused by CPU or mem? Or is it both?
Tanaalethan: Well, if the computer runs out of memory, it'll crash. But if the processor can't keep up, the process is going to freeze.
Spectre: So it's a combination. And if we had multi-threading, those of us with higher memory would be able to use it more efficiently with more cores?
Tanaalethan: Very much so.
Spectre: I'm right?!
Tanaalethan: Yes
*Spectre bounces*
Tanaalethan: As it stands, [without multithreading] most computers regardless of memory will have to stop at I'd say 6x6, 7x7 at most with an i7.
Spectre: Yeah, [Husband] and I use 5-7.
Tanaalethan:  I hit 100% CPU usage at 10x10. The lag just gets higher the higher grid I go above 10x10. If [Toady] had multithreading, and removed the hardcoded memory limit, you could go as large as you wanted. [With my rig] I'd be able to run a 40x40 map with 0 lag, and still able to run apps int he background with Win7 Aero at 1900x1080p. Screen resolution does matter. Not much, but it does change it a little.
Tanaalethan: That is, of course.. if the memory is == to the mapsize^2. Same for CPU. From what I've noticed, it's not, and my previous calculations put my computer at able to run a 96x96.
Spectre: What does memory is == to the mapsize^2 mean?  I'm not great with math.
Tanaalethan: Mapsize^2 means "Mapsize Squared" If a 16x16 meant 1.44GB, it would be 0.005625GB per 1x1 space
Spectre: o_o please say you used a calculator lol
Tanaalethan: yes I have a calculator hotkey on my keyboard
Spectre: hahaha, scared me for a second there
Tanaalethan: LOL
Tanaalethan: The best way to find the memory per space would be to make 3 maps. 1x1, 1x2, and a 2x2, then cull their sizes for the Natural Memory Space. (the memory for non-map data). Do the same for processor usage.

[Narmio]

On the subject of memory, anyone trying to do usage calculations between different maps would need to factor in the total number of Z levels - different maps seem to have different depths.  There's also the question of whether revealed vs unrevealed spaces affect memory usage - obviously any critters down there that haven't been spawned yet won't be using memory, but will the space itself? 

Additionally, a clarification for the aid of anyone non-technical reading this:  A 64-bit executable and a multi-threaded executable are different things.  The former concerns some technical details of variables within the system and means that the total amount of addressable memory is much, much higher.  The latter is about doing different, unrelated calculations simultaneously. You can be neither, one or the other, or both.  DF is currently neither. Both would be a crazy amount of work to change.

[Kogut]

Now you can ask him about problems connected with multithreading (or maybe wikipedia link will be enough: http://en.wikipedia.org/wiki/Dining_philosophers_problem)