Bay 12 Games Forum

Please login or register.

Login with username, password and session length
Advanced search  
Pages: 1 ... 4 5 [6] 7 8 ... 12

Author Topic: Multi-threading?  (Read 23775 times)

KaelGotDwarves

  • Bay Watcher
  • [CREATURE:FIRE_ELF]
    • View Profile
Re: Multi-threading?
« Reply #75 on: December 17, 2009, 04:23:24 pm »

People are obviously playing the game, which isn't to say that it wouldn't be nice if DF ran faster.
I think he didn't mean literally unplayable, but if a game runs at 20-30 fps on most modern or semi-modern rigs, then it's a disaster.

unpaused (rubberbands quite a bit, can be higher)

paused (max fps eliminated, rubberbands a lot, can be higher)

This was research I did on a previous thread testing fps performance in base DF, that is, no init performance boosts. Granted my computer is better than most, but some people would do well to choose a smaller map, limit the amount of dwarves, and tweak their init file. There's running water and magma on the map as well. To note, 600+ fps is unplayable, because DF cycles way too fast and you can't tell what's going on and the season's already changed.

I run the rather large Halltraded succession game from the DFFD at 120ish fps standard. This goes higher if dwarves are doing nothing and drops if there's a siege or ambush. There are hundreds of creatures involved.

Does fps slow down with old fortresses with 500k rock and if you cheat to embark with 30+ wagons, 300+ dwarves, have a catplosion... yes it does.

Toady does not have a computer like mine, in fact, it's closer to lower-mid range. Be secure with the knowledge that if DF were truly "unplayable" he would know.

I guess the point I'm trying to make keeping in mind that other forums like SA and facepunch say Bay12 doesn't tolerate criticism of DF or Toady- Well, as much as all of you we wish DF ran faster so we can embark on 20x20 spaces and build epic city-state fortresses of 1000+ dwarves. The d# series is a triumph of the community working with Toady to address the shortfalls of the openGL display.

But keeping in mind constructive criticism and achievable results -sticking to my first posts in this thread - why don't you change your embark habits and init if the game is running slow? http://dwarffortresswiki.net/index.php/Maximizing_framerate Instead of just throwing out examples of FUSION and CUDA, why not attempt to work with the 'pathfinder project', those that have had the experience of working with Toady? I'm rather astounded by the amount of people that keep suggesting something as inviable as multi-threading at this moment, or what appears to be the snobby elitism of a few programmers either demanding that this free game paid by donations should be open-source or the code is horribly unoptimized without:
1. having seen the code.
2. programming anything nearly as complex as DF

When I've read parts of the threads on SA, 4chan, Penny Arcade, Facepunch (probably more I'm forgetting) and someone invariably posts with that attitude and I'm struck by how concern-trollish it is. ::)
« Last Edit: December 17, 2009, 04:27:03 pm by KaelGotDwarves »
Logged

assimilateur

  • Bay Watcher
    • View Profile
Re: Multi-threading?
« Reply #76 on: December 17, 2009, 04:41:01 pm »

Does fps slow down with old fortresses with 500k rock and if you cheat to embark with 30+ wagons, 300+ dwarves, have a catplosion... yes it does.

Does fps slow down with 2- or 3-year fortresses, with 10k rock and if you don't cheat, have no cats, and embark on a 4x4 site... yes it does. Not for you perhaps, but for a great number of people.

Toady does not have a computer like mine, in fact, it's closer to lower-mid range. Be secure with the knowledge that if DF were truly "unplayable" he would know.

One lower-mid range computer is not like the other. On mine, it's bordering on the unplayable (I said bordering, because I'm ultimately able to put up with 20-25 fps, but that will change if it goes any lower); if it runs well on his, then I guess I can congratulate him.


You see, I can understand that this game runs as slow as it does. It's a very complicated one-man project. What I do have a problem is with people sugarcoating this fact. I mean, you made it sound as though it were possible for everyone to get reasonable speeds, while being able to enjoy DF (I mean, I probably would get better FPS if I embarked on a 2x2 wasteland with no water or magma, and never produced more than a couple of beds and basic food, but why would I want to play such a game?), when clearly the performance of this game is ridiculous on most machines.
Logged

KaelGotDwarves

  • Bay Watcher
  • [CREATURE:FIRE_ELF]
    • View Profile
Re: Multi-threading?
« Reply #77 on: December 17, 2009, 04:49:03 pm »

If you post a copy of your init file in spoiler codes here, I'd be happy to take the time to go through it and see where we can help your fps. I used to play DF on my laptop which rumbled at 15 fps on large forts, so I feel your pain.

assimilateur

  • Bay Watcher
    • View Profile
Re: Multi-threading?
« Reply #78 on: December 17, 2009, 05:13:53 pm »

I appreciate the offer. You might be able to figure something out, but I'm not holding my breath. Two things:

1. I've experimented with different print modes. It seems to run a bit better with what I've got there, namely partial print:5 with single buffer (IIRC I had experienced flickering without said single buffer enabled).

2. I know weather and temperature are supposed to be resource hogs, but the last time my fort slumped to around 20 fps, disabling them made little difference. In the case of temperature, I had a very choppy or stuttering experience where it would rubberband between like 20 and 50 fps constantly, being even more irritating that a constantly low framerate.


Spoiler (click to show/hide)
Logged

KaelGotDwarves

  • Bay Watcher
  • [CREATURE:FIRE_ELF]
    • View Profile
Re: Multi-threading?
« Reply #79 on: December 17, 2009, 05:37:12 pm »

Spoiler (click to show/hide)

Try this, the next thing would be to lower the partial print number or make it higher if there's flicker, or to adjust the buffer.
« Last Edit: December 17, 2009, 05:39:06 pm by KaelGotDwarves »
Logged

assimilateur

  • Bay Watcher
    • View Profile
Re: Multi-threading?
« Reply #80 on: December 17, 2009, 05:45:13 pm »

I'll see how that goes, thanks. It'll probably take me till tomorrow or even later to make a new fort with any kinda population, I'm just experimenting with world gens right now.

As I said earlier, I'm sorry but I don't really expect much from your adjustments. No temp and weather didn't do much for me earlier, neither did different print modes. I only hoped that you'd come up with something I hadn't thought about, and so far I didn't see anything like that in the new init.

EDIT: When changing it to realtime or high priority earlier, I was having increased lag in the interface when paused with little or no improvement in framerates when unpaused.
« Last Edit: December 17, 2009, 05:46:51 pm by assimilateur »
Logged

Canis786

  • Escaped Lunatic
    • View Profile
Re: Multi-threading?
« Reply #81 on: December 17, 2009, 06:27:28 pm »

I did my masters thesis on parallel processing and as of right now it is highly dependent on the project. It is very difficult to multi-thread some applications due to dependencies on previous tasks. In fact doing so will most likely decrease the speed of an application. Take this as an example,

Job A requires Job B
Job C requires Job A

Both B and C require A to be completed. This can have some parallel processing but there is no easy way to add this into an application. Also by doing so there are increased chances of bugs. Job B and C should not access the same data at all. Doing so could mean sync bugs popping up which are incredibly difficult to find as they are essentially random. (Due to how OS's handle mutli-threading).

There are simple tasks that can be threaded such as for loops that don't require previous input. As I don't know how the code works for DF I can only give a few examples but I believe the modified A* algorithm could be one such area. Instead of calculating the path of 10 dwarves one after the other you could thread the loop to calculate 5 dwarves and 5 dwarves. Which gives you a max speed benefit of 2x. I do not believe each dwarf gets a path calculated every time step as that would be such a waste of processing power. Most likely pathing is queue based so entities that need paths are added to a queue and every x time step some are given a path. In this case multi-threading wouldn't give much benefit (if any) and it would require a major rework of the back end.

Another would be temperature. I believe temp would be a good threaded area as it could be well separated from the main code. This means the number of variables shared between threads would be drastically reduced making the task much easier. Since temp is a huge hog of CPU cycles throwing it on another thread would be a viable solution as long as all temp information is double buffered. The reason being the temp at a location might change by 1 unit and while that memory address is being written, the other thread would read it and get some strange piece of data. This would have an extremely small chance of happening but it would be incredible strange to have a dwarf burst into flames because the main processing thread read the temp of the cell he was on as 10000 while the memory data was being changed. (though it would be comical).

However, as has been stated we don't have ANY clue what the code for DF looks like. Because of that everything I have stated is an assumption of how I would program such an application as of this moment. The actual implementation could be TOTALLY different. That said with programming anything is possible and it all depends on time and the amount of complexity you are willing to deal with.
Logged

Gertack

  • Bay Watcher
    • View Profile
Re: Multi-threading?
« Reply #82 on: December 17, 2009, 06:43:38 pm »

I'm rather astounded by the amount of people that keep suggesting something as inviable as multi-threading at this moment, or what appears to be the snobby elitism of a few programmers either demanding that this free game paid by donations should be open-source or the code is horribly unoptimized without:
1. having seen the code.
2. programming anything nearly as complex as DF

If you don't consider 4 hours to enumerate 500,000 objects "horribly unoptimized" even without the code, then you, sir, should be anointed sainthood for your patience.

People keep suggesting multi-threading because they see DF only using one CPU on their dual/quad-core machine and they CARE about the game because it is Fun and they want to see it get better, so they suggest what seems "obvious" to make it faster.  Algorithmic complexity is a far more subtle problem and happens more often on the old, large forts that I doubt receive as much testing from Toady.  At that point in the fortress people tend to think 200 dwarfs = pathfinding awful.  Is it? Maybe. If the pathfinder runs a lot of "impossible" path checks then that would be a good part to fix because it would be awful on large embarks, but I tend to blame the job system for the overall slowdown in fortresses.

I play a fort that gets 0 FPS. Why? It's still fun.  Would it be better it if was faster? Yes!
Logged

assimilateur

  • Bay Watcher
    • View Profile
Re: Multi-threading?
« Reply #83 on: December 17, 2009, 07:11:49 pm »

If you don't consider 4 hours to enumerate 500,000 objects "horribly unoptimized" even without the code, then you, sir, should be anointed sainthood for your patience.

People keep suggesting multi-threading because they see DF only using one CPU on their dual/quad-core machine and they CARE about the game because it is Fun and they want to see it get better, so they suggest what seems "obvious" to make it faster.

Amen.
Logged

Terratoch

  • Bay Watcher
    • View Profile
Re: Multi-threading?
« Reply #84 on: December 17, 2009, 09:21:56 pm »

Holy sh*t. Didn't mean to start a flame war. I was just asking, sorry.
Logged
The only thing better than dining on a fine goblin is using said goblins tallow for soap to fix up my soldiers.

profit

  • Bay Watcher
  • Finely Crafted Engravings... Or it didn't happen.
    • View Profile
Re: Multi-threading?
« Reply #85 on: December 18, 2009, 04:07:49 pm »

Holy sh*t. Didn't mean to start a flame war. I was just asking, sorry.
It's ok

It is not a flame war just contentious issue... No one has said they are gonna kill anyone yet or anything.

You're all good, just use search next time =)


On the subject matter, there was someone on here using a MAC who had 120FPS in a succession fort...

Can we get some other FPS numbers from the Apple camp? 

I am curious if there is a massive difference between builds...  or is the person who posted those values exaggerating significantly and adding the general noise surrounding things. I find it unlikely, but sometimes one optimization of something that is used 4 million times a second can make a massive difference.

If there was a massive difference... I am so freaking tired of FPS:1 I would consider buying a mac to play DF on if I could have 100FPS in my forts...

« Last Edit: December 19, 2009, 03:10:04 pm by profit »
Logged
Mods and the best utilities for dwarf fortress
Community Mods and utilities thread.

cooz

  • Bay Watcher
    • View Profile
Re: Multi-threading?
« Reply #86 on: December 19, 2009, 07:04:42 pm »

Any reason why OpenMP couldn't be use? Adding it to C++ code is as simple as adding few OpenMP Directives (no need to rewrite whole code creating client-server, emulators, etc...) and each for loop with independent elements could be divided to processor cores with almost zero effort.
Logged

Sunday

  • Bay Watcher
    • View Profile
Re: Multi-threading?
« Reply #87 on: December 19, 2009, 08:12:27 pm »


It's ok

It is not a flame war just contentious issue... No one has said they are gonna kill anyone yet or anything.

Oh yeah?  We'll soon fix that.


I will kill everyone!!!

OK, back to your fancy shmancy programmer-talk.
Logged

eerr

  • Bay Watcher
    • View Profile
Re: Multi-threading?
« Reply #88 on: December 19, 2009, 10:50:57 pm »

multi-thread what?
In-game lag comes from a few actions that all depend on the same information.

This includes Pathfinding, temprature, water-mechanics, map modifications and screen drawing(or so I've heard).

Unfortunately, the lag is always with just one of these processes at a time.
Only  with screen-drawing vs everything else could you afford multi-threading.
Anything else is a minimal gain, where you have one massive processor hog and another process which is 2% of it's size.

I know measurements of multithreading power are underconstruction by people more experienced than I, but multithreading 100%  of the game into  a 100% process and a 2% process is hardly any progress at all.
and that is ignoring the problem of syncronizing pathfinding with map-edits!

Well.. Since it is all done inside of a frame anyhow... You could break the temprature subroutine off into 100 threads or so...  They dont have to modify anything until the end of the frame.

The fluid simulation could also be broken off into discrete threads, probably one for every tile with a fluid simulation flag on it.

Every pathfinding job could each have its own thread...

At the end of the frame when it is time for an update, they all sequentially add their data to the relevant parts of memory.

It would not be easy, but just because they all depend on the same information does not mean they are all changing it at the same time. 

It is possible to do it... and if you had bothered to look at the video and read the messages stated earlier in this thread you would know IT HAS ALREADY BEEN DONE by someone else. 

Therefore.... Your statement is completely incorrect with no redeeming and useful information.   It has been done. It is possible. It did not just give 2% it gave at least a several Thousand percent increase.


The nature of Temprature is that it doesn't need to update changes very fast, with the except of stuff moving like water, magma and stone. Changes in temprature can happen at 1/16 the rate (and 16x the power?) rate of water is now and nobody will every notice. Then bear in mind that temprature doesn't require pathfinding like water does.


Exactly how will you make 100 threads of moving fluid?? Threads are not cheap to create or destroy, nor can you run more threads than the number of processors available.
Far better to make 1-5 threads and allow those threads to handle seperate bodies of water, or individually numbered tile.
I suppose that might actually work, but there is some syncronization overhead necessary.

Depending on the implementation, It might or might not be faster with just 2 processors, and would require some really impressive threading magic.



"Every pathfinding job could each have its own thread..."
The pathfinding jobs only cause lag when they start colliding with each other. Only one creature can stand up on a tile at a time! The problem is how DF handles these conditions. All paths that conflict are dumped into memory. This will invariably occur with two paths at a time, and can cause syncronization-issues. issues that are very problematic no matter how you attack them, short of revamping the whole movement system.


I know multi-threading is possible, I just don't want Tarn to destroy DF to make it support multithreading. Just because other people built the engine with multithreading from the bottom up doesn't mean it takes Tarn a mere four months to write in Multithreading.

Logged

Silverionmox

  • Bay Watcher
    • View Profile
Re: Multi-threading?
« Reply #89 on: December 20, 2009, 08:15:34 am »

Does fps slow down with old fortresses with 500k rock and if you cheat to embark with 30+ wagons, 300+ dwarves, have a catplosion... yes it does.
Does fps slow down with 2- or 3-year fortresses, with 10k rock and if you don't cheat, have no cats, and embark on a 4x4 site... yes it does. Not for you perhaps, but for a great number of people.
For the record, rocks have nothing to do with pathfinding speeds... the corridors they came out of do. Workaround: place a door to seal off a large mining area that you're not currently exploiting, and lock it.
Logged
Dwarf Fortress cured my savescumming.
Pages: 1 ... 4 5 [6] 7 8 ... 12