Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[janglur]

So I want to try playing with some emulators or virtual machines to create a virtual system for just DF.  I love departmentalization and software segregation, so I want to install, say, WinXP on a virtual machine that has only one VCPU, but uses all 3 of my real CPU cores to aggregately outperform what would be possible for Dwarf Fortress if I did this natively.

Anyone have any ideas how to accomplish this and on what VM-host?

[Tellemurius]

so you want 3 cores emulating one core? the problem is the speed will be the same you won't get any big boost

[janglur]

So i've been told, but not explained *why*.  As it stands, CPUs clearly aren't planning to improve single-thread performance anymore, just adding infinitely more and more cores.  Which considering my entire computer has, maybe, four multi-thread applications... and that I almost never even run them anyway... this is just a massive waste of power, even on my aging Phenom II.

There HAS to be a way to put two cores to work for DF and make a worthwhile gain.  There HAS to.  Otherwise, what the hell have we been doing with technology this past decade?!

[Tellemurius]

the problem is the synchronization between the cores, they would be sharing workload but they need to be on time to have correct data to spit out. The program would need to separate the workload into threads to see multicore bonus.

[ChairmanPoo]

There HAS to be a way to put two cores to work for DF and make a worthwhile gain.  There HAS to

THere already is. DF is not really single-threaded, IIRC some stuff can actually run on a second core (no benefits beyond that, though).

At least I read that in a thread in the upper forums a while ago.

[dree12]

There HAS to be a way to put two cores to work for DF and make a worthwhile gain.  There HAS to

THere already is. DF is not really single-threaded, IIRC some stuff can actually run on a second core (no benefits beyond that, though).

At least I read that in a thread in the upper forums a while ago.

You can dedicate up to 13 cores to DF and a marginal speed boost will result. This is 1 for the calculation, 1 for most interface, 1 for graphics, and a couple others that will be mostly idle.

[Lectorog]

There HAS to be a way to put two cores to work for DF and make a worthwhile gain.  There HAS to

THere already is. DF is not really single-threaded, IIRC some stuff can actually run on a second core (no benefits beyond that, though).

At least I read that in a thread in the upper forums a while ago.

The process in question is too small to have any noticeable impact on gameplay. A multi-core processor cannot actually increase your DF performance in this regard. (IIRC)

[janglur]

Lectorog is correct.  Whether DF is on one, two, or all three cores, performance doesn't improve.  In fact, each additional dedicated* core makes it lose FPS at increasingly significant intervals.

* = Dedicated, as in it's the ONLY program AT ALL allowed to access the core.

[Pnx]

The trouble is multi-core processing works by having one core delegate out processes to the other cores. But the processes have to be built in a way that makes this possible to do efficiently.
Having ten people all work on the same job doesn't necessarily mean the job will get done faster. They have to be organized, and the organization takes work, which eats up processor cycles, which means the job gets done slower because the processor that was sorting it all out could have just done the damn job itself in all the time it took to get those lazy new employees to do it.
Unless the processes are built to be delegable, which isn't always possible.

As a side note this really belongs in DF General discussion.

[janglur]

Why?  It's not strictly for DF.  I have many, many single-threaded apps that need the same boost.  Minecraft, Minecraft's Server, this fractal compression tool i'm using that's marvelous but unbeleivably slow, etc.

[Starver]

Was apparently ninjaed even before I started this, and now six more replies, but I've started so I'll finished, and my answer (could do with being reduced a bit) is still more or less going to be:


Running a single process on a single core, runs instructions in order.  e.g. Instruction 1, instruction 2, instruction 3, etc.

Running multiple processes on a single core needs each process queue to be time-shared and dealt with a bit at a time, e.g. Instruction A1, Instruction B1, Instruction C1, Instruction A2, Instruction B2, Instruction C2.

If you have multiple cores, you can (ideally, perhaps) have one running Instruction A1, 2, 3, etc while another is running Instruction B1, 2, 3, etc.  Faster than a single core can possibly cycle round.

But if you have just one process, are you going to put Instruction 1, 3, 5 on CoreA and Instruction 2, 4, 6 on CoreB?  (Or whatever combo you need when you've got CoreC, D, E, F, etc.)  Well, it would shift a set of non-branching instructions requiring absolutely no immediate reference to the results of the previous command (currently still being operated on the other core, in your effort to save time).  But as soon as either complication occurs (and others, left out for simplicity) then you're going to have an overhead of "shouldn't have run that, forget it" or just having to NOOP while you're waiting to see what you're now accessing.  Never mind that you're probably sharing a bus to the shared cache and another, slower one, to the main memory (and, onwards, to any disk storage, whether virtual memory or actual file data).



Which is not to say that you couldn't get something to work that way, by deliberately coding so that result dependencies aren't needed until later, and decision-branches are known well in advance, and working nicely on a clump of data that can reside in the respective non-shared cache for periods, before synchronising back to the shared areas.  But then you're effectively programming for parallel processing...  Taking bare (single-threading) code and pushing it through the system is going to have varying, but probably not fortunate, results.


Massive multi-processing is an interesting thing.  The kind the guys who get 100+ games consoles and put them in racks to handle subdivisions of a program.  But, even then, there's an amount of speculative processing.  Being done on the assumption that another calculation says this was Ok (grabbing separating blocks of data, running them in isolation, then recombining and re-splitting accordingly).


As I haven't worked (at any appreciably low level) with this sort of stuff for a decade, there's probably new thoughts (as well as details I've been forced to leave out/smudge), but... TL;DR; Single-core don't work that way.


(Although I'm sure that we're supposed to have the graphics on a separate thread from the physics, more or less, in current DF.  And if you have a good multi-core system (and Windows and the firmware, combined, does its job in scheduling all processes in a path of least resistance), you might at least get your multi-core speed-up on multi-core because one core gets the one half of DF, another core gets the other half of DF and all the rest of your 126 cores[1] is handling all the AV scanning, Blue-Ray ripping, torrent-streaming, encryption-breaking, additional seventy separately initiated Worldgens starting and (above all!) process scheduling, and not slowing your main game down at all...  Right? )

[1] On a speculatively futuristic system, here...

[Levi]

This is why I'm skeptical that I'll ever need more than four cores for pretty much anything.  Frankly even four seems to be too many. 

[MorleyDev]

The main problem is taking true advantage of multiple cores requires a major shift in both software design and programming paradigm.

We're used to an easy-to-understand way of programming where typing
functionA();
functionB();
functionC();
will cause them to execute in order when the previous one finishes.

But to truly use multithreaded and to effectively scale to n cores, you have to switch to something which looks more like:
schedule(functionA);
schedule(functionB);
schedule(function C when function A finishes);
join(functionB);

You have to design and think in terms where a function isn't quaranteed to be finished as soon as it's called. The whole industry is slowly having to make this shift, and video games are at the forefront of it in a lot of respects due to the need to squeeze as much out of the CPU as they can.

Frankly even four seems to be too many.

They said the same thing about having more than 16kb of RAM at one point.

[Levi]

Frankly even four seems to be too many.

They said the same thing about having more than 16kb of RAM at one point.

True, but RAM requirements I know will go up.  I'm not entirely convinced required number of CPU's wil.  Like you said, its quite a bit harder to program things to work in parallel. 

[MorleyDev]

Well at the end of the day we're gonna hit a wall in terms of individual core speed, moore's law has it's limits. Either we get quantum computers working, or multiple cores is going to become all we got in terms of CPU speed growth. Easier to keep loads of weaker cores cool since you've got an increased surface area to cool, as opposed to the "double the transistors" approach which we know has it's limits because eventually cores would get so hot as to rival the sun xD

My current laptop has four cores, and compared to the double core processor I had on my old laptop I can see a difference. The general leg-work of the OS is being divided between them, which means even single-core programs run a lot faster simply by virtue of having more of the CPU cycles to play with. The gain isn't linear (you can't just double the cores to double the speed) but it's not negligible.

Another way things could go is to start creating dedicated Processing Units for different aspects of programs. Taking the idea of a GPU and extending it to everything else (like the already existing PPU). The main job of the CPU would mainly become pushing blocks of instructions onto those processing units that are designed to be able to handle them efficiently at the expense of everything else.

But that would take ages to be introduced since it'd be a huge shift in the entire gosh-darn industry...