Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[NW_Kohaku]

So yeah - 64bit Windows 7, intel core i7 2667m, 4 gigs DDR3 1333 MHz ram - both are running steadily 5 minutes into the test and hovering between 55,000 FPS and 53,000 FPS. I can see no difference between them.

Wow, I know my computer's old, but damn.  I'm only getting 600 FPS at the best.

[Hotaru]

As I said, this Asus ux31 I have is really, really fast for DF for some reason. In the world I genned for "Salore Flowerpetals'" adventure, where the earth is about 20 z-levels deep, I can run a 16x16 embark at playable speed levels unless there is a river there. It also will gen a large world to the year 1000 in 30 minutes.

[bombzero]

As I said, this Asus ux31 I have is really, really fast for DF for some reason. In the world I genned for "Salore Flowerpetals'" adventure, where the earth is about 20 z-levels deep, I can run a 16x16 embark at playable speed levels unless there is a river there. It also will gen a large world to the year 1000 in 30 minutes.

As i mentioned, it may not be there in the morning 

anyways, im assuming you have a very fast processor?
fun fact, i bought a 400$ computer and a 70$ graphics card and i can play almost every single game on mid-high graphics... just need a better processor and ill have a qualified gaming computer.

speaking of processors, NW based on your observations do you think multi-threading would help or hurt in the short and long term of DF development?
just a slight question, not trying to derail the thread on a multi-threading talk.

also small idea for your experiment, test whether items traded TO caravan affect fps, if you are correct about the fps issues then the mass dump of stone goods on a caravan is a fps killer and common in long term forts, if trade=bad then this is even more urgent for Toady to fix seeing that this is the caravan arc of development.

[Hotaru]

As i mentioned, it may not be there in the morning 

anyways, im assuming you have a very fast processor?

Oh, but it's morning already and I'm a night owl in the weekends 

The processor is Intel i7 2677m. I don't think it's that fast. But I can recommend this laptop, it's really great all round. Best computer I've ever had, and I've owned computers like 15 years.

[kaijyuu]

I thought my computer was decent and I only get 8000 fps in the normal arena map with 0 units

[bombzero]

how much did it cost? im not exactly rich so i prefer building my own stuff, more for less, and always exactly as you want it, just have to get the few thousand ill need for my ultimate gaming computer that will be able to run DF at THOUSANDS OF FPS!!!!!!!! and less importantly any other game released by any company for the next few years...

[NW_Kohaku]

speaking of processors, NW based on your observations do you think multi-threading would help or hurt in the short and long term of DF development?
just a slight question, not trying to derail the thread on a multi-threading talk.

I haven't done all that much with multi-threading, so I can't say whether the benefits or the costs are under or overstated when I see these things, however, I do know that some things are typically on different threads in many programs.  Mouse and keyboard input, for example, can be put on its own thread so that when the game is going through some serious calculations, like long worldgen periods between years, the game can still recognize requests to pause the process.  Graphic output can also be put on its own thread fairly easily.  Pathfinding (concurrently in the same frame) can be "embarrassingly parallel" as someone put it in the suggestions forum a while ago, as traversing the connectivity grid is something that can be done without having any interference from the other pathfinders (unless you were trying to pathfind to the point where another character was pathfinding, but there aren't any such events in the game, already) provided you separate out the claiming of jobs (so that two different haulers don't try to pathfind to the same sock to haul away concurrently). It might be entirely possible to do something like splitting temperature checks up into multiple threads where you do something like cut up the map into quarters and go through every item in those quarters in individual threads, since temperature checks shouldn't spill over across tiles, although that might depend on how temperature checks actually iterate.

also small idea for your experiment, test whether items traded TO caravan affect fps, if you are correct about the fps issues then the mass dump of stone goods on a caravan is a fps killer and common in long term forts, if trade=bad then this is even more urgent for Toady to fix seeing that this is the caravan arc of development.

I'll try to get to that.  I still need to do magma-dumping, huge caravans, and then huge tradeaways to caravans, as well as obsidian-casting items, sock destruction, and ludicrous animal gibbing. Maybe something else I've forgotten.  Too many things to check...


*Ding*! I finished baking the Medium Contraction Experiment.  I have it set to [SPEED:0] dwarves, so maybe that'll help tamp down some of the noise, or maybe not?  I just have to do the quick control run and I'll try to see if I get more notable results on this one.

[NW_Kohaku]

Again, I think I'm getting a small, but present result. 

I seem to get 310 FPS (again, I put in SPEED:0 dwarves this time) with the medium contraction control, and 280 FPS with the medium contraction end state.

The saves on DFFD: http://dffd.wimbli.com/file.php?id=5913
Mediafire: http://www.mediafire.com/?d1zmnrvg39gey8c


EDIT:
Actually, going back to the control, I'm getting closer to 400 FPS now, so it represents a much larger drop than I at-first thought. 

[bombzero]

I haven't done all that much with multi-threading, so I can't say whether the benefits or the costs are under or overstated when I see these things, however, I do know that some things are typically on different threads in many programs.  Mouse and keyboard input, for example, can be put on its own thread so that when the game is going through some serious calculations, like long worldgen periods between years, the game can still recognize requests to pause the process.  Graphic output can also be put on its own thread fairly easily.  Pathfinding (concurrently in the same frame) can be "embarrassingly parallel" as someone put it in the suggestions forum a while ago, as traversing the connectivity grid is something that can be done without having any interference from the other pathfinders (unless you were trying to pathfind to the point where another character was pathfinding, but there aren't any such events in the game, already) provided you separate out the claiming of jobs (so that two different haulers don't try to pathfind to the same sock to haul away concurrently). It might be entirely possible to do something like splitting temperature checks up into multiple threads where you do something like cut up the map into quarters and go through every item in those quarters in individual threads, since temperature checks shouldn't spill over across tiles, although that might depend on how temperature checks actually iterate.

well my thought was that fluid calculations and temperature checks are interdependent on eachother, a dwarfs body needs to check its temperature as well as its medical condition, water can interrupt pathfinding, dwarfs need to know where other dwarfs will be in a few frames to pathfind effectively, overall a lot of things in DF are directly connected with checks that must be preformed before certain things can happen, my main concern is you would get threads waiting on threads.

seperating keyboard/mouse from everything else would be nice, figuring when sieges, caravans and migrants show up and with what could be its own thread, but if too many things are split up issues could develop worse then what it solves, DF is a LOT more complex then most games, and has MANY more calculations to make every second.

[wierd]

@multithreading... i have a 64 bit system, you may have one, but what about people with 32 bit? switching to multi-threading would strain 32 bit processors. Also, switching the entire game code would take forever, and introduce a slew of bugs. So long term gain, yes. but with the cost of a portion of the fanbase who have older computers, and a few years of development time, which would drive off another portion of the fanbase.

mostly the calculations need to be optimized, which Toady is trying very hard at, and what NW's experiment could potentially help with.

Also, it is better to improve a single thread process' algorithms then it is to make that process multi-threaded.
having 2,4,6, or even 20 cores with 1 thread a piece wont improve the framerate of the game if the same code issues are dragging it down.

tl;dr: would screw the last few 32 bit users, wouldn't help till the code bugs are fixed, would introduce many new bugs.

@Hataru, im breaking into your house and stealing your computer, just thought id let you know in advance.

Actually, multithreading and 64bit address space are different beasties. Multithreading means you divide the program into different subprocesses, instead of a single enormous one. This let's the program make use of multiple cpus and/or multiple cores.

The 64bit advantage is "huge" registers inside each core, and access to a huge memory space.

Again, multithreading is not tied to 64bit arch, you just need to have more than one cpu, or have a cpu that at least does intel's hyperthreadding. 

Also, I indeed did not presume that toady doesn't know about multithreading. I presumed he is avoiding it because concurrency is a royal PITA, caused by a jackhammer without lube.  (At least compared to a simple single thread process structure.)  However, given the stagnation of cpu speed in the marketplace over the past decade, and the market trend for more cores instead, along with the expected feature set of this game, toady is going to have to bite the bullet and switch to multithtreading someday. (I can't see the war and caravan arc releases operating without it. Not at playable fps.) The sooner he starts building his MT framework, the less blood he will pass rectally later while trying to add features.

For the time being, I was suggesting simply creating "dummy" threads on other cores, so he could get access to their registers, because doing so would greatly speed up processing of the primary thread under certain types of processing conditions. (Accumulators, flags, pointer arrays, etc could be stored in the vacant cores' registers, even though the process is still single threaded otherwise. The access to the other core registers is cheap compared to a ram access. Other things that might help are dummy operations on the other cores to get data into the shared cpu cache while the main core is busy as a kind of prefetching. This way the main thread makes more efficient use of the cpu instruction cache, reducing the need to hit ram.)

Those things re not full blown multithreading, but would allow a multicore cpu to benefit playing DF.

[NW_Kohaku]

Yeah, when I get the control up to the same seasonal time-frame as the test case, I have FPS around 450 (summer is a fast season) ranging from 350 to 500, compared to 260 only rarely spiking up into 300 on the test case.  I think I am getting reliably 25% framerate drops from having done this test case. 

[bombzero]

ahhh dummy threads, good thinking. i remember helping make a mod for a open-source single thread game along those lines, one of my first actual coding experiences.

anyways, essentially the plan would be to move more stuff away from RAM and instead into the registry? hmm... temperature, pathing, item indexing, and combat would be priorities i imagine, some of the other stuff is perfectly fine operating on the current system, but everything higher frequency could run off the registry, without hurting 32 bit users.


anyways on topic: NW is it possible that its one particular set of pointers not being deleted when atom smashing? the fact that you get 'some' fps improvement but not total improvement seems to suggest that.

[wierd]

A cpu has readable/writable "slots" in it called registers.

Take a simple addition instruction. It has 3 registers. Operand1, operand2, and output.

Assembly programs can void accessing ram entirely in many cases by PUSHing the output from the last operation into another register for the next instruction. 

The idea here is that we put frequenly accessed data and pointer values inside the registers of unused cores.  No instructions are being done with those registers, so they are sitting there doing nothing.

Fetching data/pushing data from/to these outside registers is faster than waiting for ram, because the registers are almost directly connected to each other internally inside the cpu die. The latency is much lower, so the core doing the heavy lifting waits less while it gets the data it needs.

The cache prefetching basically does a similar operation to what the main thread will do, so that the cpu's branch prediction will store the instruction and its output in the cpu cache.  The next time the instruction is called (on any core, because they share the same cache) it can grab data from there instead of doing the actual process cycle, or fetching from ram, depending on the instruction.

The goal is to reduce the number of waitstates in the main thread using clever tricks.

[NW_Kohaku]

anyways on topic: NW is it possible that its one particular set of pointers not being deleted when atom smashing? the fact that you get 'some' fps improvement but not total improvement seems to suggest that.

I'm not atom smashing right now, I'm using a reaction.

I set up the reaction so that the deletion of 100 stones creates 1 goblet as a by-product.  For the control, I created an equal number of goblets just straight-out.

The theory behind the contraction test is that the arrays are never sorted, so that if I created a huge number of items, and then left a small number of items still in those arrays, but spread out across a large swath of these arrays that the arrays would never be cleaned up - you'd wind up with huge numbers of arrays that were empty except for a single item in them. 

Since I'm getting FPS decay for trying this, it seems like the theory worked.  The contraction left a lot of mostly-empty arrays behind that are never sorted, and chew up some FPS being iterated through. 


Again, if I could get some outside corroboration on these saves:
The saves on DFFD: http://dffd.wimbli.com/file.php?id=5913
Mediafire: http://www.mediafire.com/?d1zmnrvg39gey8c

I find a notable drop in FPS performance, although I'm not sure how notable it might be on Hotaru's damn supercomputer. 

This might be the evidence I was looking for, and a process of players destroying large numbers of objects while keeping a few objects in their stockpiles forever is definitely the sort of thing that occurs in long-term fortresses that might lead to gradual FPS decay.

[Mego]

Posting to watch and help. I'll start running those experiments either Saturday or Sunday.

Specs for reference:

Toshiba Satellite A215-S4747
Windows Home Premium x64, SP1
AMD Turion 64 x2 Mobile Technology TL-56 1.80 GHz
ATI Radeon X1200 Graphics Card
2.50 GB DDR2 RAM