Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[dreiche2]

well, apart from the fact that I don't think that these kinds of calculations are currently much of a resource hog, these kinds of threads would still need to talk to each other, and thus be synchronized in some way.

From my limited knowledge I think it looks like going for calculation intensive for-loops where the iterated tasks do not need to talk to each other, only "report in" to the main program after they are done, is most promising. But yeah, I don't really know how it actually works code-wise, I look forward to looking into it some time soon...

[Draco18s]

Threads and processes differ from one operating system to another but, in general, a thread is contained inside a process and different threads in the same process share some resources while different processes do not.

I.E. a multi-threaded single process application can't run on two cores.  The distinction is subtle, I think, but yes, before an application can run on multiple cores it needs multiple threads.

Uh... did you mean "it needs multiple processes"? Because I think otherwise your statement contradicts itself...

Oops, yes.  It was 3am.

[numerobis]

There's two kinds of threads: user-mode and kernel-mode.  Kernel-mode threads, also known as light-weight processes, are how you think about something that actually runs.  User-mode threads get mapped to kernel-mode threads when they get on the CPU, or they can just chill out and be your memory of some thread of execution that you'd like to run whenever there's CPU time for it.  Sort of like that bill on your desk that you'll pay "when you get around to it" as compared to the bill in your hands that you're actively writing a cheque for.  Back in the dark ages, some libraries could have many user-mode threads and many kernel-mode threads (Solaris), some could do any number of user-mode but only one kernel-mode (for lack of kernel support in Linux at the time), and others had exactly one user-mode thread per kernel-mode thread (Win32).  Presumably, progress has been made in the ten years since I last actually cared.

There are other ways of thinking about multiprocessing also, it doesn't just have to be threads -- in fact, I'd argue that threads are often the worst possible way to parallelize: too complicated and error-prone, so you end up paying a lot in synchronization and you spend lots of time bug-fixing.  In all cases there's some kind of kernel-mode object to put a program on the CPU.

In terms of return on investment, it's usually much easier to get a factor of two by making your sequential code run faster than by parallelizing it to run on 4 cores.  Said method of making things faster may make it harder to parallelize later.

[katana0182]

I apologize for the forum thread necromancery. I understand the whole concurrency issue, and understand that MP thread synchronization is difficult and annoying... I'm trying to figure it out in VB.net right now for some work-related stuff...

But...shouldn't it be possible to split discreet, non-realtime tasks off from the main thread and have them return later on, within a given time-frame?

For example...
1. Main thread decides that temperature overlay for map is outdated...
2. Main thread decides that since [SMP:ON] it will fire up a thread to update temperatures; said thread must return within 10 gameplay frames....
3. Main thread instantiates subsidiary thread (foo), and passes it necessary data.
4. Main thread continues for 10 gameplay frames, while thread (foo) processes temperature update of map.
5. After 10 frames, thread (foo) is queried to see if it has completed results. If it has, and is not locked, thread (foo) returns the results to main thread, and is destroyed. If thread (foo) is locked or has not returned results, it is signalled to stop, and main thread processes temperature update of map on next frame using its own resources.

A pattern like this of making the main thread robust enough to do the task of the subsidiary thread in the event that the subsidiary thread fails to return data should be able to resolve concurrency issues, although it probably isn't an efficient use of resources...but it might be a way to start with MP? What do others think?

[dreiche2]

Well, that's a possible idea, the question is how much work and risk does it take to realize this in practice... and well, I played around with boost.thread in the last days, and I've got a basic system implemented as you describe it. Nothing DF specific, but basically just a small generic producer-consumer setup (see wikipedia).

For an example, I had the main thread simply wait for the child threads to deliver after n frames have passed, not stopping them as you describe.

All I have to do is write it up so I can post it here, but my problem is that my free week draws to an end. I hope I can do it till next weekend the latest. I can't say myself if it would be worth to adapt the thing for DF, but in any case it might be interesting to anyone curios about the threading issue in general...

[Vicomt]

I was having a wonder about DF's pathfinding last night as well.....

as I understand it atm, DF has a single pathfinding dataset which all the dorfs use to pathfind. Now, considering that different dorfs with different jobs don't normally need to pathfind to the entire map, would it be feasible (if a little memory hungry) to allocate each dorf with their own pathfinding dataset, which could be tweaked (i.e. have various sections of map removed/added) based on their currently active jobs and preferences?

f.ex. Urist McMiner (with only mining active) only needs to know where he is and where the mining zones are), whilst Urist McFarmer needs to know his location, and where the farms and foodstockpiles are. Pathfinding could then use two shorter (distance) runs to

• if the dorf is NOT within the destination pathable area, path to a pathable location (depending on where the dorf is, this might be skipped)
• else (the dorf is within the pathable destination area) path to the destination (which should be faster given the now shorter physical distance from dorf to destination)

now I've never actually worked with any pathfinding code, so I don't know if there's huge holes in my logic, and I'm assuming that finding two paths with shorter physical lengths, plus the overhead of keeping each dorfs pathfinding dataset is actually more efficient that just running the single longer pathfinding operation.

comments?

[Core Xii]

For example...
1. Main thread decides that temperature overlay for map is outdated...
2. Main thread decides that since [SMP:ON] it will fire up a thread to update temperatures; said thread must return within 10 gameplay frames....
3. Main thread instantiates subsidiary thread (foo), and passes it necessary data.
4. Main thread continues for 10 gameplay frames, while thread (foo) processes temperature update of map.
5. After 10 frames, thread (foo) is queried to see if it has completed results. If it has, and is not locked, thread (foo) returns the results to main thread, and is destroyed. If thread (foo) is locked or has not returned results, it is signalled to stop, and main thread processes temperature update of map on next frame using its own resources.

Not feasible. Every item and entity and stuff has a temperature in the game. You can't seriously be suggesting that we duplicate the entire game state for the other thread? That would brutally murder memory usage. On the other hand if you didn't duplicate the data to be local for the other thread, it would have to constantly access the global data, and that kind of defeats the whole purpose. So, no, temperature calculations probably cannot be multithreaded like that.

However, it COULD be multithreaded in a sequential manner - The game processes all temperature calculations in a single run, and divides the map into blocks for threads to work on independently, etc.