Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[Nadaka]

Profit: You don't sound like you do much multi-threaded programming. Spawning a new thread takes resources, as does every context switch. Spawning one thread per task for hundreds or thousands of tasks is rarely the best solution even for tasks that do not require a lock.

[profit]

Profit: You don't sound like you do much multi-threaded programming. Spawning a new thread takes resources, as does every context switch. Spawning one thread per task for hundreds or thousands of tasks is rarely the best solution even for tasks that do not require a lock.

Of course each one takes resources and thread build up and tear down takes processor time, however, just because it is not the best use of resources yet in the windows PC inside of the X86 architecture world does not mean it does not work well elsewhere.

Inside of CUDA every single task often DOES have its own "thread".  with tens of thousands of them spawned and torn down constantly, because of nearly zero overhead because it was designed from the ground up to handle it.

Just because a Pentium under windows xp does not handle context switching smoothly does not mean that it is never handled well anywhere.

but... yes for x86 type program it would probably be best currently to make a number of  worker threads that match the number of cores lock them to the core and dispatch requests round robbin and then as they complete each task... But.. I was kinda thinking more future than just a couple cores.

[Nadaka]

It has nothing to do with x86, or windows xp. Short of large cluster computing and  expensive video cards in the last couple years; no CPU, operating system or programming language has been able to handle that kind of parallelization "well" for relatively trivial tasks. The vast majority of real world applications (where performance is a concern) do not and can not work like that.

[profit]

It has nothing to do with x86, or windows xp. Short of large cluster computing and  expensive video cards in the last couple years; no CPU, operating system or programming language has been able to handle that kind of parallelization "well" for relatively trivial tasks. The vast majority of real world applications (where performance is a concern) do not and can not work like that.

But, some do.

CUDA does it exceptionally well in protein folding simulations, and parallelized physics calculations next to shader calculations.

It handles trivial "threads" well and had very little over head when dealing with them. *Technically they are no longer threads they are operations as they exist only as long as it takes to do the one thing they are in existence to do.

FUSION does it well, as well.

Just because yesterdays applications don't work like that does not mean tomorrows will not.  *Although I admit probably most will not need to since most applications wait for humans way more than they spend doing any "work"

And dwarf fortress is closer to needing a cluster level solution than nearly any other game I have ever seen due to its sheer scale and detail.

[sproingie]

You can always go to town with Khazad and prove it's a workable model.  Pathfind a thousand dorfs at 50FPS and I'll be satisfied.  This is otherwise just another "all you have to do is rewrite everything from scratch" sentiment.

Theoretically, it could work.  But Toady would rather work on a game than a High Performance Computing engine.

[profit]

You can always go to town with Khazad and prove it's a workable model.  Pathfind a thousand dorfs at 50FPS and I'll be satisfied.  This is otherwise just another "all you have to do is rewrite everything from scratch" sentiment.

Theoretically, it could work.  But Toady would rather work on a game than a High Performance Computing engine.

Actually I offered.  I even offered to sign a binding NDA and become bonded...  Mr Adams said he would rather not let anyone in any capacity see anything that goes on under the hood of the game.

I am just arguing this point for the Theoretical aspect anymore toady doesn't want to learn how to thread, he wont let someone else do it... But since people said there is nothing to be gained here... and they are completely wrong and I have nothing better to do with my time than prove they are, I am here.

Here you can see 16,384 characters path finding here at 60 FPS here though -> http://download-developer.amd.com/GPU/videos/Froblins.H.264.HD.136MB.mov

So by your statement you now are satisfied.

[sproingie]

I'm not debating whether the model is workable: I'm perfectly satisfied that it *can* be done, and the Sushi Engine is proof of that.  I remain unsatisfied that anyone will actually implement this in a form that's actually portable back to DF.  So far I just see a lot of grumbling that DF isn't an ATI tech demo.

[assimilateur]

So far I just see a lot of grumbling that DF isn't an ATI tech demo.

It might seem like that, but I assure you, what you see is something else. You're seeing criticism of a game's dev who prefers to pile on feature upon feature, seemingly disregarding the elephant in the room which is unbearable performance for a great deal, or perhaps even the majority, of users. Said performance issues are only going to get worse if the dev persists in his priorities.

[Rowanas]

But computers will keep getting better and since people can quite happily play with a fair number of things going on anyway, why stop to upgrade a feature that would go unused in the majority of cases?

[Nadaka]

But computers will keep getting better and since people can quite happily play with a fair number of things going on anyway, why stop to upgrade a feature that would go unused in the majority of cases?

The theoretical limit for silicon is about 10 Ghz. Right now I think the highest experimental clock was 6.something Ghz. Typical CPU's have been hovering around 3Ghz for a long time. For the foreseeable future, most computer improvements will come mostly from using more processors instead of faster processors. I believe it could already be to late for dwarf fortress to shift to major gains through parallelism, and with each added feature it will be harder to make that switch.

[Shades]

Here you can see 16,384 characters path finding here at 60 FPS here though -> http://download-developer.amd.com/GPU/videos/Froblins.H.264.HD.136MB.mov

So by your statement you now are satisfied.

He said dwarves and I assume in the df world, the path finding in that demo is trivial from what I can see. As a technical demo it's impressive as a display of performance it's not.

[assimilateur]

But computers will keep getting better and since people can quite happily play with a fair number of things going on anyway, why stop to upgrade a feature that would go unused in the majority of cases?

Wait a second, did you seriously call improved framerate a "feature that would go unused in the majority of cases"?

[profit]

Here you can see 16,384 characters path finding here at 60 FPS here though -> http://download-developer.amd.com/GPU/videos/Froblins.H.264.HD.136MB.mov

So by your statement you now are satisfied.

He said dwarves and I assume in the df world, the path finding in that demo is trivial from what I can see. As a technical demo it's impressive as a display of performance it's not.

So... It uses pathfinding, jobfinding, fluid simulations, realtime obstacle avoidance, atmospheric modeling, the characters, eat, sleep, mine, react and have nearly every characteristic that dwarves can have except their propensity to run blindly into trouble and you say that it is not a display of performance...

Nearly every calculation that grinds the simulation to a halt in dwarf fortress is handled flawlessly in an admittedly specialized  parallel environment.  So it is highly relevant...

So... What other worthless theory can I kill today?

Lets see...

Cant be done... *Ooops can be!*
Will take too much memory *Ooops it doesnt!*
No one has done anything like it *Ooops they have!*
It only is for cluster computing or rendering *While it can be It is not ONLY for that*
Threads MUST be very limited in number *I conceded a little on this one but for the most part they don't have to be*
It will only increase speed by 2% *Nope, looks like it could increase speed by a factor based on the number of cores available if done right.*
We just have to wait for faster processors *Oops they haven't really gotten faster in the last 8 years!*
Alls I have seen is a tech demo that was not relevant *Oops it, really is because all the behaviors that dwarves have are mirrored in it. Its not a tech demo of an airplane simulator its one that uses the same basic mechanics as dwarf fortress just in a finer grained (and probably more static) world.

Anyone else want to try their luck with an ignorant statement?!

[The-Moon]

As much as everyone wants to defend toady and DF, that will not help the fact that toady will only be able to take DF so far until one day hes not going to be able to work on it no more without converting to multi-core processing.

You cannot continue to pile, feature apon feature into a game and expect one single processor to handle it all. Or expect and hope single core processors are going to get better.

I got a top of the line model of a computer which can barely run DF even after editing the Init file extensively and turning off most if not all features.

Toady has now currently added in a lot of new stuff during this past year. All these new features are however limited to a single fortress map.

What happens when toady starts making it so that the rest of the world is active as your playing a fortress?

You can barely run a fortress now at a decent speed, then hes going to add on top of that, a fully active world?

I do not think so. Not without multi-core processing.

You can all defend toady and the game, which i do as well defend him. However we all have to face the cold hard truth that DF will eventually die or come to a halt before reaching its full potential without muti core coding. At some point toady is going to have to bite the bullet and start thinking about MCP coding. You cant have a game like this and limit it to a single processor.

[profit]

But computers will keep getting better and since people can quite happily play with a fair number of things going on anyway, why stop to upgrade a feature that would go unused in the majority of cases?

This was answered earlier in the thread, but here I will review since you apparently came late to the party and chose not to read up till now...
  Computers have been getting better but CPU's have not been getting faster.  They have remained about the same speed for the past 8 years.  They have simply been getting more efficient and getting a higher number of cores.  This allows them to do much more and be technically faster, but if an application like dwarf fortress is single threaded they do not get hardly any faster.. They do benefit a little if something like a virus scanner is running as it can run on a separate core and not slow down the games thread.

As an example someone is playing on a 2002 laptop and they have as good of performance as someone with a 2005 machine...
the 2005 machine has more cores, and way more performance, but it's spare performance is useless because dwarf fortress' single thread design cannot take advantage of it.

Ironically if you just want to play dwarf fortress an old 90 dollar 3.8 GHz Pentium 4 chip overclocked a bit will net you better performance than a 959 dollar i7 chip.