Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[Blacken]

Ah geez man you really, really need to chill out. No one is mad at you for shooting down ideas. Like I said: YOUR CONTRIBUTION IS CONSIDERABLE. But your tone is constantly offensive and you constantly go off-topic over your personal crusade.

Let's look at other ways to do it. I will present the idea to Toady of we have something useful, if we can't come up with anything then that's fine. Is it impossible to outsource processes much the way you outsource labor? When DF normally queries for closest resources every time the dwarves perform a job, causing massive processor delays, can't we have it query an updating separate database? We would basically be rewriting a small part of the program separately without the need to change the very thing you keep saying it is impossible to change.

Toady's cooperation is necessary. Rather than assuming we can't get that cooperation, let's just consider whether it is possible that we might be able to do something if we can. It's not going to kill us to consider the hypotheticals here in this thread.

For the last time, if you dont' want to play ball, then leave. This isn't the place for your crusade against your invisible enemies. If you want to continue to contribute without the sarcasm and off-topic defensiveness, then it would be nice to have your knowledge.

Damn you're stubborn about it.

You seem to be placing some kind of intent in my actions that is not there. I am pointing out things that are, to someone with experience and skills in this area, obvious. You are getting upset at this. That is something for you to work out, sir, not me. There is no crusade. There is me telling you what works, and me telling you what doesn't work, and you stuffing your fingers in your ears and going "LA LA LA BUT WHAT ABOUT IF WE HAD THIS STUFF THAT DOESN'T EXIST?". I am making a point of being about as polite as is possible here when I say that yeah, you could do it if you have all this stuff you don't actually have. But that doesn't make for any better discussion than "no, it cannot be done given the constraints of reality," now, does it?

Given cooperation from Toady on whatever necessary level, is it even possible to create an interface/utility that would do what I've suggested?

See what I mean? "If not for all the roadblocks in the real world, could something be done?"

You're asking the wrong question, because if not for all those roadblocks--if Toady was interested in cooperating to the degree necessary to do anything significant--it could be done just by modifying the source code. There! Done! End of problem! But in the real world, the source code is not available and as such modifying object code to parallelize application functionality (you can use whatever phrasing you want for it, it boils down to this) is a nontrivial task.

Basically, you're asking for the source, which Toady has repeatedly said he's not releasing. That's more the problem than any understanding of modern CPUs.

Though there is one way around that. If you can create an algorithm/equasion that can find Prime Numbers then you won't need the Source Code.

How does this relate? Why is prime-finding relevant to the discussion?

[Neonivek]

How does this relate? Why is prime-finding relevant to the discussion?

No

Alright I am joking about that No

It is rather Relevant but mostly I wanted to wait until someone understood the reference.

I am making a point of being about as polite as is possible

 

No seriously this is my response

Honestly Blacken, you cannot hold that to be true hense: "CAPS SPEAKING"

Though I do like the fact that your at least trying, but I question how sincere (hmm can't spell that word) you are.

[Footkerchief]

You're asking the wrong question, because if not for all those roadblocks--if Toady was interested in cooperating to the degree necessary to do anything significant--it could be done just by modifying the source code. There! Done! End of problem! But in the real world, the source code is not available [...]

It's not impossible that other sections of the source will get opened up, though.  The way people got the I/O code opened up was by demonstrating that they'd be able to make good use of it to improve DF's performance and expand the player base.

[bombcar]

Basically, you're asking for the source, which Toady has repeatedly said he's not releasing. That's more the problem than any understanding of modern CPUs.

The last thing I want to do is hijack the Adams brothers' dream. It's also obvious that I couldn't do any of the required programming myself. But I would like to discuss the possibilities. From what's been said, we can't do it ourselves. Given cooperation from Toady on whatever necessary level, is it even possible to create an interface/utility that would do what I've suggested?

Basically, it becomes a tautology. If Toady makes it multi-threaded, it will be multi-threaded!

The only place I can see him being able to give another portion (as he did with the OpenGL/SDL) is to define a "dwarf controlling" interface so that programs like Dwarf Therapist can more easily connect to the options available. It's a big task otherwise - literally. Even preparing the source code to be able to utilize multiple processors will make this arc look like a short release.

It's more likely that the idea of "burrows" is evolved more and more until you effectively have separate fortresses in one big fortress; and then you can begin to think about putting them on separate cores.

I'm assuming the "finding primes" is a reference to NP-complete problems. They're problems that sound easy to a human, but have been proven to be impossible for a computer to perfectly solve.

[Blacken]

The thing is that if Toady were to knock off a chunk of the code (say, pathfinding) and pass it to the community for improvements like he did with the OpenGL code, then it would have to be made roughly independent of the rest of the program by him to such a degree that a lot of the work involved in multithreading it would probably already be done.

That would only hold for an asynchronous approach, right?  I'm not sure, but I suspect the pathfinding people had a synchronous approach in mind, so that they wouldn't have worry about the map changing while a path request is handled (although I guess that isn't a huge problem from a dwarves-aren't-omniscient POV).  I suppose I should go ask them.

As you say, the omniscient approach is much more easily made asynchronous (but still has to do weird and potentially ugly updates as far as if something that is known tries to update). To parallelize an omniscient-view pather, it would probably need to be synchronized.

(I'd love to see an ignorant pather, though. Fortress defense? "Don't let them see you!")

Has anyone brought up how switching the game to multithreading wouldn't have as much of an impact as it would seem?

Depends on what you could multithread. Water physics is probably not really parallelizable given its reliance on stateful data, but pathing almost certainly is (and given certain degenerate cases, would be a pretty big chunk to actually deal with). Given an omniscient pather like the current one, it should be relatively easy. Pathing already locks important stuff so other dwarves can't get to it, it seems like it would be a simple task to parallelize give the source code.

You're asking the wrong question, because if not for all those roadblocks--if Toady was interested in cooperating to the degree necessary to do anything significant--it could be done just by modifying the source code. There! Done! End of problem! But in the real world, the source code is not available [...]

It's not impossible that other sections of the source will get opened up, though.  The way people got the I/O code opened up was by demonstrating that they'd be able to make good use of it.

Certainly true, but this is pretty core functionality. I have a hard time seeing such a case evolving, in such a way that it can be modularized so Toady doesn't have to give away DF-code to get it to work.

[G-Flex]

The thing about pathfinding is that we can't be 100% sure what the algorithm will actually need to do.

Odds are it'll need to handle all kinds of weird things, like creatures flying (or not), being magma-immune (or not), skill and personality checks for things like climbing or jumping over gaps, and other stupid things.

[Neonivek]

Well that certainly isn't the answer to my question. So it goes without saying that we are once again working under the premise that Multithreading will have dramatic improvements over lag.

Though I have no idea how much multithreading can improve the speed, I do know that it would need to have an overall improvement greater then 10 to "Fix" the game.

[G-Flex]

"Greater than 10"? Ten what?

If pathfinding can be successfully parallelized, then hell, that's probably good enough on its own, considering that it's probably responsible for the majority of CPU time (or at least the plurality) eaten up by the program, and given how even computers today have four cores available to them.

[MiamiBryce]

From what I understand, right now DF's current format only enables it to run on a single processor, failing to utilize the power of the dual- or quad-core processors most of our computers currently have. A good emulator (something along the lines of DosBOX) could unlock DF's potential.

Your understanding is flawed and I don't think I saw it corrected in any of the posts, though I admit there was a lot of tl;dr because I tire of these same old arguments easily.  Any modern operating system will distribute the workload of DF among all cores of your processor.  I have a quad core processor.  When I run DF, all four of my cores spike to somewhere between 25 and 30 percent usage.

The issue would be that because DF is not multi-threaded there is overhead in distributing the process to multiple processors so it may not run any faster when distributed - it possibly might even run SLOWER because of the overhead of marshaling.  And lets not even get started on the multi-threaded, everything that can be said has been said and we can go reread those flaming threads if we want to.

[G-Flex]

From what I understand, right now DF's current format only enables it to run on a single processor, failing to utilize the power of the dual- or quad-core processors most of our computers currently have. A good emulator (something along the lines of DosBOX) could unlock DF's potential.

Your understanding is flawed and I don't think I saw it corrected in any of the posts, though I admit there was a lot of tl;dr because I tire of these same old arguments easily.  Any modern operating system will distribute the workload of DF among all cores of your processor.  I have a quad core processor.  When I run DF, all four of my cores spike to somewhere between 25 and 30 percent usage.

I think you're misunderstanding something.

Your computer is not running DF on all four cores at once. It's bouncing it AROUND the cores, so that 1/4 of the time it's on one core, 1/4 of the time it's on another, etc.

Big, big difference.

[Blacken]

From what I understand, right now DF's current format only enables it to run on a single processor, failing to utilize the power of the dual- or quad-core processors most of our computers currently have. A good emulator (something along the lines of DosBOX) could unlock DF's potential.

Your understanding is flawed and I don't think I saw it corrected in any of the posts, though I admit there was a lot of tl;dr because I tire of these same old arguments easily.  Any modern operating system will distribute the workload of DF among all cores of your processor.  I have a quad core processor.  When I run DF, all four of my cores spike to somewhere between 25 and 30 percent usage.

I think you're misunderstanding something.

Your computer is not running DF on all four cores at once. It's bouncing it AROUND the cores, so that 1/4 of the time it's on one core, 1/4 of the time it's on another, etc.

Big, big difference.

This would be why affining it to a single CPU often sees (sometimes significant) perf improvements.

[bombcar]

From what I understand, right now DF's current format only enables it to run on a single processor, failing to utilize the power of the dual- or quad-core processors most of our computers currently have. A good emulator (something along the lines of DosBOX) could unlock DF's potential.

Your understanding is flawed and I don't think I saw it corrected in any of the posts, though I admit there was a lot of tl;dr because I tire of these same old arguments easily.  Any modern operating system will distribute the workload of DF among all cores of your processor.  I have a quad core processor.  When I run DF, all four of my cores spike to somewhere between 25 and 30 percent usage.

Then your OS is playing silly buggers with DF. Since DF is single-threaded, you should see one core go to 100% usage (assuming no FPS limit), and the others basically idle.

[nil]

This would be why affining it to a single CPU often sees (sometimes significant) perf improvements.

This is sort of off topic (although given that the topic is apparently irredeemably flawed I guess that's all right) but are there any good utilities out there for facilitating or automating core affinity?

[The Architect]

So... finding prime numbers sounds like it would have two fundamental stumbling blocks:

1) The basic code utilized by computers, having at its most basic level only a yes/no or odd/even, in other words opposite or mathematically speaking 0/1, +/- aspect is very limiting in this instance.

I realize this is the basis of computing, so please don't jump my case. I'm just pointing out that the way our most basic computer system exists disqualifies it to a large degree for this kind of work.

2) The infinite nature of the range covered by prime numbers, and of course the increasing difficulties as the problem involves larger numbers.

You could create a database, but otherwise the calculations required each time your range increased would become increasingly difficult. If you process enough volume you could come up with various principles and basic algorithms (many of which are already known) to speed the process, but you still need a considerable database to avoid actually recalculating each time or at least to simplify the process by storing your known formulae.

[Neonivek]

The other reason why forcing DF to a single core also leads to better performance is because at the same time it also removes the CPU limits for Dwarf Fortress. (So instead of 30% it can go up to 100%)