So I finally got my "Zander" engine working with my experiments for playing around with concurrency. It works as follows:
At the lowest level you have tasks and a thread pool. The thread pool takes in tasks and executes them on one of
n threads.
Tasks can have children, nexts and shared nexts.
A child is pushed onto the thread pool when the task is finished executing.
A next is pushed onto the thread pool when the task, and all of the tasks children, and all of the nexts of the tasks children, and so-on down the graph, are finished executing.
A shared next is a next which has to wait for multiple tasks to fulfil the above conditions before being enqueued.
This lets you build up huge, dynamic graphs of program flow without caring about how many threads you have, the system is guaranteed to work on any number of threads > 0. You can even use nexts and shared nexts as join points so the need for explicit low-level synchronisation (mutexes and so-on) is reduced.
The game main pushes itself onto this thread pool as a root task. It creates the graphics and audio drivers, and prepares the resource cache (which can load from passed resource sources, defaulting to the file structure but this can be changed to custom sources, with a physfs-based system for loading from archives being included in the engine). Then it starts the state manager looping on the pool.
The state manager maintains a stack of states and a list of "event managers" that have two steps: dispatch, where they send out events, and poll, where they gather information about which events they need to dispatch. States maintain their own task graph for update and draw logic, which is passed to the state manager. The state manager does the following things in a series of steps every frame, with each sub-step being executed concurrently:
Step 0:
- Delete any removed managers from the system.
Step 1:
- Check if any states exist. If not, close main.
Step 2:
- Get the draw commands from each state
- Flush the inputs from the window.
- Update audio drivers if required.
- Dispatch the event managers.
Step 3:
- Update the logic for each state that can be updated by placing their logic into one long task chain (each state can be concurrent internally, but no two states will update at the same time).
- Flush the draw commands from step 2 onto the screen, updating the display.
- Poll the event managers.
The next step is to enqueue step 0 again. Due to the nature of the thread pool, this will effectively cause the system to loop again.
It's complete overkill, but still kinda nifty to play around with
Plus it makes debugging in a multithreaded environment easier, because I can instantly rule out or link the problem to timing issues and crashes by dropping
n to 1. If it still goes wrong, it's the code. If not, it's the threading. And it makes things easier to unit test and identify problems, you can just isolate chunks of the task "graph" to test.
Now I just need to actually make something with it XD Might refactor the task system out into a more std-like syntax first and release that separately...