Bay 12 Games Forum

Please login or register.

Login with username, password and session length
Advanced search  
Pages: 1 ... 3 4 [5]

Author Topic: Securing DF in case of unfortunate event?  (Read 22721 times)

GavJ

  • Bay Watcher
    • View Profile
Re: Securing DF in case of unfortunate event?
« Reply #60 on: May 08, 2014, 06:49:00 pm »

I have indeed written multithreaded code for a largescale game project (a creatively powerful 3d photoshoppy equivalent of minecraft). The project ended up being scrapped, interestingly (considering the grpahical difference to DF!), due to complete failure of me or my partner to wrap our heads around the abysmal thought-graveyard of humanity that is OpenGL. The multithreading actually worked super well.

Although it was from the beginning, which makes all the difference of course. And I also agree that regardless, DF does have a ton of other much lower hanging optimization fruit first.

Also I did it in Java, which had a bunch of super useful tools for doing things like dynamically adjusting threads on the fly for you. And in general fills in a lot of gaps efficiently for people who don't 100% understand all the engineering level things going on. This very much helped with avoiding unnecessary overhead, for instance. I don't know how good those tools are in other languages or Toady's expertise.
« Last Edit: May 08, 2014, 06:55:39 pm by GavJ »
Logged
Cauliflower Labs – Geologically realistic world generator devblog

Dwarf fortress in 50 words: You start with seven alcoholic, manic-depressive dwarves. You build a fortress in the wilderness where EVERYTHING tries to kill you, including your own dwarves. Usually, your chief imports are immigrants, beer, and optimism. Your chief exports are misery, limestone violins, forest fires, elf tallow soap, and carved kitten bone.

catpaw

  • Bay Watcher
    • View Profile
Re: Securing DF in case of unfortunate event?
« Reply #61 on: May 09, 2014, 01:53:48 am »

Its one of the uppersides of java that its much easier to write proper thread-safe code than in most other languages.

As I've repeating myself several times now, the abysmal thought-graveyard of humanity is though, where the real performance gains lay. Utiziling the GPU even for non-graphical-vector stuff, like temperature or pathing.
Logged

reality.auditor

  • Bay Watcher
    • View Profile
Re: Securing DF in case of unfortunate event?
« Reply #62 on: May 09, 2014, 07:58:53 am »

Of course you can't do *everything* in parallel, and I explicitly said you wouldn't.
My impression is you wanted that on beginning (except actual movement for some reason), but later changed goalposts, of course without admitting that maybe making everything in main loop threaded simultaneously isn't best idea in universe.

That's a very compelling argument you have there. "This is INSANE, because... because... unstated reasons!"
It is obvious to everyone here (except you, apparently).

Can you please describe in more specific detail what sort of "horrible problems" or "random crashes" would occur from calculating, say for example, pathfinding, liquid flow, and temperature completely at the same time if you wanted to?
These two things (pathfinding and temperature) theoretically should not be affected - unless some data is shared. Like, you know, dwarves avoiding too hot places. At end is larger explanation of potential problems with that.

And if you get yourself out of the mindset that EVERYTHING has to be "squared off" and fully up to date by the end of the tick, a lot more parallelism becomes possible. For example, who on earth cares whether on one tick, you do liquid flowing for a tile, and then temperature based on its new position, versus temperature based on its old position and then temperature-less flowing afterward? It will catch up in the very next tick
Nonsense. Complete, utter nonsense.

If you do not wait for flow or temperature calculations to end of tick, these calculations start to get out of sync with rest of game - and this is cumulative. Rest of game gets what is essentially unpredictable and randomly changing water/temp values based on something that may happened on this tick, last tick or who knows what, when and where. In other words, it will NOT catch up, as state of flow/temp relies, among other things, on previous state - and that will be incerasingly out of whack.
Next problem - map remodelling. Changes on map will be reflected late and inconsistently (like bridge closing and still letting some magma through like it wasn't here).
Another problem: in same tick, different "blocks" in main loop (or even different creatures/items/whatever) may get different value of water/temperature. Hell, it is possible that during handling of one creature (say temperature calcs for different layers of tissue) temperature will change in middle of calculations. This kind of thing can and will create extremely subtle and obscure buggy behaviour happening in already complicated code. I do not know how temperature calcs are done - if code writes values multiple times in same place during one run, it will get even worse.

I have name for someone that would accept this kind of risk for ANY gain.

and I don't see how that will meaningfully affect anybody's gameplay.
*insert sarcastic laugh*
Logged
Are weapons like the least lethal thing in DF?

catpaw

  • Bay Watcher
    • View Profile
Re: Securing DF in case of unfortunate event?
« Reply #63 on: May 09, 2014, 08:58:46 am »

Its quite plausible to have pathing base on the temperatures of the tick before. So pathing can be done while temperature is done in the main thread. This is of course imples to have a copy of the whole dataset tick by tick*. Just add the very first step for dwarves to not set into stuff that burns them instantly.

Personally I'm a hugh fan of immutable datastructures anyway. This makes next to other benefits multi-access much easier. Its sometimes just not the most effective to copy the whole thing just change one value... I for one just accept this possible perfomance loss.
Logged

GavJ

  • Bay Watcher
    • View Profile
Re: Securing DF in case of unfortunate event?
« Reply #64 on: May 09, 2014, 10:38:24 am »

Quote
These two things (pathfinding and temperature) theoretically should not be affected - unless some data is shared. Like, you know, dwarves avoiding too hot places. At end is larger explanation of potential problems with that.

No it doesn't matter that dwarves want to avoid hot places. Go ahead and pathfind and just don't give a crap whether there is some slim chance that the square will catch on fire that same turn you pathfind. If it does, it's no big deal. Your only penalty is a dwarf walking a little bit further on rare occasions. NOT game crashes. NOT fiery death. As long as the dwarf checks for fire in front of him before he actually TAKES his next step, he can still cancel the now-dangerous path before he actually enters the danger.

Hence my emphasis on movement being the primary thing that needs to be non-parallel, not pathfinding. Planning wher eyou might go in the future does not actually move you anywhere impossible or dangerous. Moving moves you there...

Also, imagine a path that will take, say, roughly 500 ticks to complete. And let's assume that at some point it ends up obstructed.  There is only a 1/500 chance that the thing that ends up obstructing that path will begin obstructing it in the exact same tick that the path was planned. Whereas there is a 499/500 chance that it will happen during some other tick, and that even if you calculated them in sequence, you'd still have to rely on MOVEMENT, not pathfinding, to see the dwarf to safety. So you're already obligated to check in front of you each step.

This obsession with thinking for some reason that pathfinding needs super up to date info all the time every time is the source of disagreement here.  It doesn't. It can do perfectly fine with 1 or 2 or even 5 tick old information if it has to, almost every time. And the other times, it's just inefficient, not game crashing.

Quote
If you do not wait for flow or temperature calculations to end of tick
I apologize if I was unclear. Yes, every type of calculation needs to finish before every tick ends.

What I meant was that, for example, temperature does not NEED to have up to date information as of this exact same tick from liquid flow. Even though liquid flow is a major source of temperature change, you could just use the previous tick's state and be totally fine. Maybe one random guys' crazy dwarven computer won't work anymore, or something. But the other 99.9% of everybody else who doesn't absolutely need ice to melt THIS tick instead of NEXT tick gets a faster game, and that's worth it.

The only reason why, say, flow and temp would need to be done in sequence within a tick would be if temperature MUST be accurate based on fluid flow in that same tick. I see no reason why this is the case. So yes, you still wait for each thing to finish before ending the tick, but you don't need to wait for flow to finish within the tick before doing temp. You can just work off a cache of the last tick's state.

Quote
My impression is you wanted that on beginning (except actual movement for some reason), but later changed goalposts, of course without admitting that maybe making everything in main loop threaded simultaneously isn't best idea in universe.
Nope, goalposts still the same. I didn't say everything. I said:
* Pathing
* Temp
* Liquid flow
* Combat

I may be wrong about some of that, but I stand by it for the moment.

Combat is the only one I think is a bit iffy, but I am still pretty sure it would work, as long as it has its own thread to itself, not split up into several. You go based on last tick's positions, and combat resolves before you move this tick (since movement happens at the end), which should be fine.
« Last Edit: May 09, 2014, 10:42:07 am by GavJ »
Logged
Cauliflower Labs – Geologically realistic world generator devblog

Dwarf fortress in 50 words: You start with seven alcoholic, manic-depressive dwarves. You build a fortress in the wilderness where EVERYTHING tries to kill you, including your own dwarves. Usually, your chief imports are immigrants, beer, and optimism. Your chief exports are misery, limestone violins, forest fires, elf tallow soap, and carved kitten bone.

Tawa

  • Bay Watcher
  • the first mankind all over the world
    • View Profile
Re: Securing DF in case of unfortunate event?
« Reply #65 on: May 09, 2014, 04:16:33 pm »

Lol, this thread has been mostly, if not completely, derailed.
Logged
I don't use Bay12 much anymore. PM me if you need to get in touch with me and I'll send you my Discord handle.

Dirst

  • Bay Watcher
  • [EASILY_DISTRA
    • View Profile
Re: Securing DF in case of unfortunate event?
« Reply #66 on: May 09, 2014, 04:21:13 pm »

Lol, this thread has been mostly, if not completely, derailed.
Not only derailed, it was derailed with the assistance of about fifteen impulse ramps.
Logged
Just got back, updating:
(0.42 & 0.43) The Earth Strikes Back! v2.15 - Pay attention...  It's a mine!  It's-a not yours!
(0.42 & 0.43) Appearance Tweaks v1.03 - Tease those hippies about their pointy ears.
(0.42 & 0.43) Accessibility Utility v1.04 - Console tools to navigate the map

Xazo-Tak

  • Bay Watcher
  • *Camping forever*
    • View Profile
Re: Securing DF in case of unfortunate event?
« Reply #67 on: May 10, 2014, 03:24:49 am »


Fortunately avoiding all of this is rather trivial modding - just open toadyone.txt and add [NO_AGE], [NO_EMOTIONS], [NODRINK] and [NOEAT]. Memorialize everyone on the map to avoid ghosts then atom-smash them. Turn off caravans, immigrants and invaders, and seal the fortress off from wildlife. Toady can now continue working until you experience save corruption.

Wait since when did Toady drink? I just assumed he absorbed H2O out of the air through his skin.
It's to prevent him from harming himself by trying to drink or eat.
The last time we heard of him drinking and eating, he had ended up in a bad condition.
Logged
How to have recursive Fun:
Have Fun
Reclaim fort
Destroy your main graveyard with a cave-in
Pages: 1 ... 3 4 [5]