Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[thompson]

My hope is Toady writes the new map system with multithreading in mind. All he really needs to do is multithread the CPU bottlenecks and leave everything else in a single thread. Could be useful for fluid dynamics, for instance. Anything that requires a ton of memory access won't work though.

Multithreading would also make for more interesting bugs...

[Su]

if by "interesting" you mean "next to impossible to solve without huge expenditure of time and effort" you're right on the money.

multithreading is very, very complicated, despite what you might think. the majority of modern games only use it for rendering [which df already does!], and df is vastly more complicated than most games.
it's simply not worth it right now to try implementing something that may or may not grant a performance benefit of unknown amount at the expense of breaking determinism and making wrapping one's head around the code significantly harder.

I hope the map rewrite is more compact too though

[thompson]

Hi Su,

I am well aware of the complexities of multithreading. I had my tongue firmly planted in my cheek when I joked about the new bug potential.

For a game as complex as DF, it would be best ONLY to multithread processes that have few or infrequent interactions with other processes. A price setting algorithm for the economy could be an example (assuming prices only need to be updated infrequently, so you can periodically synchronise the economy thread with the main thread - you need to design specifically for this though). There's a reason I didn't suggest pathing as a multithreading candidate.

[jonesmz]

Hopefully this isn't considered a thread necro, since it's only 5 pages deep on the suggestion forums.

Anyway, I am a professional C++ programmer who deals with multi-threaded code basically 40+ hours a week.

Seriously, it's not anywhere near as hard as people in the DF community claim.

Is it going to happen? Who knows. That's the Whim of the Toady One.

Would it be difficult? No more so than a variety of other meaningful software engineering problems. Which doesn't mean it would be easy.

Should it happen? Personally, I believe that splitting out a few dedicated and cpu intensive tasks into their own thread(s) would improve the game. My rough guess is that pathfinding wouldn't be a bad choice, or if not that temperature / water stuff. But without studying the code itself I'd never be able to make more than a guess.

I just wish people would stop saying things along the same sentiment of "Omg no one should ever even consider multithreading, it's so difficult, no not even difficult it's impossible!!!".

[Starver]

(Better to have necroed/initiated a thread (NPI) dedicated to the actual suggestion of mutli-threading. Yes, this drifted over to that, but if it's the best place you found when going looking for this then you maybe should have started a proper one. IMO. Three months of bump isn't at all bad, in general, though, given the policy in this area. Positively fresh!)


Now, you might say in response to this...

Toady said the game needs optimizing way more than multi-threading.

The main issue is that DF relies on memory access a whole lot. Multi-threading helps with parallel calculations, but it won't read data any faster, especially since it has to be transferred to another core.

...that one way to optimise would be to multi-thread some (more) things, like Thompson said to Su.

We don't know really, though, how much the sole-Dev of this solely-Deved project is already dealing with the consequences of his own brand of spaghetti-code. Maybe there's a lot of tidying-up going on, all across the board, as various modes of the game are touched on for other refinements, and maybe at some point he feels he could safely offload another entire new mechanic to another thread. But I'm sure you know what it's like to deal with legacy code from your own past. And then there's how/if to Kill Your Darlings, in a coding sense, without totally ruining what you already had.

Don't forget that any man-hours wasted (either prior ones negated or new ones sent up a false alley) in this project are actual-hours in real-time, and can't be assumed to be amortised across teams of programmers under a Rapid/Agile/Scrum framework, or whatever the current atmosphere suggests, with barely a hint on the daily timesheets.

Who knows what pitfalls lie in the heart of the code? The Toady knows!

[Shonai_Dweller]

An odd place to post this, there are dedicated threads on multithreading. I'm sure there's at least one in the suggestions forum somewhere.

What you have to start with is what Toady has actually said on the matter otherwise these threads always collapse into a mass of "Toady hates optimizaton" and "How could someone with 30 years programming experience ever possibly multi-thread anything if he never studied a university course on it", etc.

The main points that I can recall are:

1) In his side-projects (which people often forget as he never releases them, instead features end up in DF if they're good enough) he has experimented ("a little") with multi-threading. So it's not a complete unknown, unthinkable concept for him.

2) He has a list of "things which need optimizing" and there's quite a lot, apparently. He'll do these before thinking about multi-threading.

because...

3) No matter how easy/difficult it is, it's going to take a lot of time. And there's no real guaranteed result of the kind that fans want to see (better fps as opposed to realistic things like "ability to browse mythgen output during history generation instead of your whole computer being locked up"). At least not compared with carrying out all the other "easier" optimizations.

Anyway, that's just based on some comments he's made. Now feel free to argue back and forth of the possible merits (although, as you say, without seeing the code, who knows?).

[Bumber]

Anyway, I am a professional C++ programmer who deals with multi-threaded code basically 40+ hours a week.

Seriously, it's not anywhere near as hard as people in the DF community claim.

How often does that involve taking single-threaded code in an existing project and rewriting it? With DF complex as it is, we'd have to worry about deadlocks and race conditions on top of all of the existing bugs.

Even if Toady could pull it off, it might turn out that the parts that can be multi-threaded aren't the biggest bottlenecks. Anything involving historical figures, for instance, is going to involve reading from RAM, which grinds things to a halt. Instead, it might be better to find ways to optimize caching in the existing code, rather than waste numerous months on rewrites.