Bay 12 Games Forum

Please login or register.

Login with username, password and session length
Advanced search  

Author Topic: Possible micro site generation technique?  (Read 1224 times)

PatrikLundell

  • Bay Watcher
    • View Profile
Possible micro site generation technique?
« on: February 18, 2016, 04:58:30 am »

If I understand it correctly, adventure mode map locations are generated by taking the unfinalized world description and then generate the specifics based on a set of RNGs tied to that location, so the location will be reconstructed the same way every time.
A way to allow permanent changes to a location without fully generating and saving it would be to continue to generate the location, and then add a post processing step where each modification is reapplied (basically "remove tree at location X, then remove tile (X, Y, Z), then ...), more or less like a macro. The modification actions performed could initially be stored in the order they were performed, but may benefit from some kind of analysis where you can collate some actions (such as "remove this block of tiles", rather than a sequence of individual digs). It should also be noted that these remove (and add gabbro wall at (X, Y, Z)....), ... actions should differ from their originals in that the repeats don't generate/require any material (no logs, boulders, etc). If an analysis and collation is done, it could also take the results cave-ins into account, to generate the post cave-in situation, rather than recreate the cave-in (which, again, shouldn't generate any material).
The obvious drawback to this approach is that extensive modifications may result in rather long location load times. An upside is that location regeneration probably is a good candidate for a parallel process that can be executed as the location is approached. There is no doubt some DF players will start mega projects in adventure mode is given the chance, though...
A further refinement would probably be to have some kind of logic to determine that a site has been so extensively modified that it needs to be changed from regeneration into a saved state, similar to a retired fortress.
Logged

Bumber

  • Bay Watcher
  • REMOVE KOBOLD
    • View Profile
Re: Possible micro site generation technique?
« Reply #1 on: February 18, 2016, 10:26:12 am »

Or it could just keep a list of items and changed tiles.
« Last Edit: February 18, 2016, 10:28:01 am by Bumber »
Logged
Reading his name would trigger it. Thinking of him would trigger it. No other circumstances would trigger it- it was strictly related to the concept of Bill Clinton entering the conscious mind.

THE xTROLL FUR SOCKx RUSE WAS A........... DISTACTION        the carp HAVE the wagon

A wizard has turned you into a wagon. This was inevitable (Y/y)?

PatrikLundell

  • Bay Watcher
    • View Profile
Re: Possible micro site generation technique?
« Reply #2 on: February 18, 2016, 12:25:09 pm »

I agree. Swap this tile for that tile generates the same result in an easier manner. There's still the issue to do the same for follow on effects, such as cave-ins and fluids, and there's also the possibility to clear replacements of tiles that are replaced again later to keep the replacement list queue size down.
Logged

Bumber

  • Bay Watcher
  • REMOVE KOBOLD
    • View Profile
Re: Possible micro site generation technique?
« Reply #3 on: February 18, 2016, 05:50:02 pm »

If a tile caves-in or fluid passes through, it's been modified. You don't need to clear replacements if they're just overwritten. I believe it's as simple as keeping an array of tiles where anything unmodified is just zero/null, and then compressing the data on disk. Lots of identical values compress very well.
Logged
Reading his name would trigger it. Thinking of him would trigger it. No other circumstances would trigger it- it was strictly related to the concept of Bill Clinton entering the conscious mind.

THE xTROLL FUR SOCKx RUSE WAS A........... DISTACTION        the carp HAVE the wagon

A wizard has turned you into a wagon. This was inevitable (Y/y)?

PatrikLundell

  • Bay Watcher
    • View Profile
Re: Possible micro site generation technique?
« Reply #4 on: February 18, 2016, 06:10:27 pm »

It's true cave-ins etc. result in modifications, but the code will still have to "realize" that. If you detect modification by scanning for modifying actions you'll miss that unless you actively update those secondary "actions". If modification is done through a common operation (rather than writing directly into the data structure, which isn't unlikely), it's easy to add a hook to it, though.
I'd just use a sorted linked list of tiles modified. That way you'll detect that you've already modified the tile and can overwrite that entry rather than add a new one.
Logged

Bumber

  • Bay Watcher
  • REMOVE KOBOLD
    • View Profile
Re: Possible micro site generation technique?
« Reply #5 on: February 18, 2016, 06:30:06 pm »

It needs to be called whenever something would modify a tile. It could get costly for fluids, but there's no real way around it unless you're okay with seeing a lake drain over and over each time you return to an area.

I'd just use a sorted linked list of tiles modified. That way you'll detect that you've already modified the tile and can overwrite that entry rather than add a new one.
I wouldn't. You're going to be iterating through that list with every single tile change.
Logged
Reading his name would trigger it. Thinking of him would trigger it. No other circumstances would trigger it- it was strictly related to the concept of Bill Clinton entering the conscious mind.

THE xTROLL FUR SOCKx RUSE WAS A........... DISTACTION        the carp HAVE the wagon

A wizard has turned you into a wagon. This was inevitable (Y/y)?

PatrikLundell

  • Bay Watcher
    • View Profile
Re: Possible micro site generation technique?
« Reply #6 on: February 19, 2016, 03:30:26 am »

Yes, I agree it needs to be called for every modified tile. The tricky part is to ensure you actually manage to round up every case where a tile modification is made. Fluids is a bit of a tricky issue, though. You don't want to update the state for every depth change happening, but you somehow need to keep track of it. A possibility is to add fluid affected tiles to a post process list. As the area is unloaded, this list is processed to determine what the long term state of those tiles should be (Drained and 7/7 depth are given possible end states. It's messier when it comes to partial depth, but I guess you could just spread the fluid evenly over the area, and add the partial bit randomly). The fluid post processing list would thus be integrated into the state list during finalization, and then be disposed of.

It's true the list would be iterated over for every modified tile, but the tiles are modified one at a time a player speed (apart from cave-in/flooding secondary effects), so the iterations would be spread out over one tick here and one there. That ought not to have any noticeable effect. You'd get that time back on loading, where you only need to iterate over the tiles actually modified, rather than every tile in the embark only to dismiss 99.9% of them as unmodified. Now, if you're going for a mega project so you've modified a few hundred thousand tiles, you might possibly get a small stutter when modifying a new tile, but I doubt it.

Edit:
It looks like Toady has already started on the modifications, so I guess he's already settled on a way to handle it. Thus, this discussion is probably overtaken by events.
« Last Edit: February 19, 2016, 03:39:43 am by PatrikLundell »
Logged

Elone

  • Bay Watcher
    • View Profile
Re: Possible micro site generation technique?
« Reply #7 on: March 01, 2016, 09:08:47 am »

Bit late to say this since as you said, Toady has already decided what to do.

But you dont have to check it once per tick per tile to mark which tile has changed. Just do it once, when the game is about to save. Take the freshly generated chunk, take the existing changes which are sitting in your RAM, and where these two sets dont match, thats your savegame. Save that to disk.
Logged
▼ It's all their fault. ▼

Bumber

  • Bay Watcher
  • REMOVE KOBOLD
    • View Profile
Re: Possible micro site generation technique?
« Reply #8 on: March 01, 2016, 09:21:35 am »

Bit late to say this since as you said, Toady has already decided what to do.

But you dont have to check it once per tick per tile to mark which tile has changed. Just do it once, when the game is about to save. Take the freshly generated chunk, take the existing changes which are sitting in your RAM, and where these two sets dont match, thats your savegame. Save that to disk.
That requires keeping a spare copy of the untouched chunk in memory to compute the difference at unload. (Or generating it a second time on unload.)
« Last Edit: March 01, 2016, 09:23:46 am by Bumber »
Logged
Reading his name would trigger it. Thinking of him would trigger it. No other circumstances would trigger it- it was strictly related to the concept of Bill Clinton entering the conscious mind.

THE xTROLL FUR SOCKx RUSE WAS A........... DISTACTION        the carp HAVE the wagon

A wizard has turned you into a wagon. This was inevitable (Y/y)?

Elone

  • Bay Watcher
    • View Profile
Re: Possible micro site generation technique?
« Reply #9 on: March 01, 2016, 09:41:46 am »

Bit late to say this since as you said, Toady has already decided what to do.

But you dont have to check it once per tick per tile to mark which tile has changed. Just do it once, when the game is about to save. Take the freshly generated chunk, take the existing changes which are sitting in your RAM, and where these two sets dont match, thats your savegame. Save that to disk.
That requires keeping a spare copy of the untouched chunk in memory to compute the difference at unload. (Or generating it a second time on unload.)

I'm aware of this, but this is how some games do it. Starbound does it like this. Starbound saves only changes, then when you come close to a chunk, the chunk is regenerated every time (and saved data added to it). And if you dont want to regenerate it, and would rather offload it to RAM, that wouldnt be much of a problem either, it would almost double the RAM usage but this game is mainly only CPU-heavy and has other resources to spare. And even then, regenerating a chunk takes... how long? Have you ever counted? How many split-urists before a chunk is generated and ready to play, which currently happens anyway for as long as AdvMode exists?
Logged
▼ It's all their fault. ▼

Bumber

  • Bay Watcher
  • REMOVE KOBOLD
    • View Profile
Re: Possible micro site generation technique?
« Reply #10 on: March 01, 2016, 09:58:19 am »

I'm aware of this, but this is how some games do it. Starbound does it like this. Starbound saves only changes, then when you come close to a chunk, the chunk is regenerated every time (and saved data added to it). And if you dont want to regenerate it, and would rather offload it to RAM, that wouldnt be much of a problem either, it would almost double the RAM usage but this game is mainly only CPU-heavy and has other resources to spare. And even then, regenerating a chunk takes... how long? Have you ever counted? How many split-urists before a chunk is generated and ready to play, which currently happens anyway for as long as AdvMode exists?
It depends on what's in the chunks. DF can get very complicated, especially in sites, and is 3D loaded down to HFS (because stuff below can be relevant.)
« Last Edit: March 01, 2016, 10:01:03 am by Bumber »
Logged
Reading his name would trigger it. Thinking of him would trigger it. No other circumstances would trigger it- it was strictly related to the concept of Bill Clinton entering the conscious mind.

THE xTROLL FUR SOCKx RUSE WAS A........... DISTACTION        the carp HAVE the wagon

A wizard has turned you into a wagon. This was inevitable (Y/y)?

PatrikLundell

  • Bay Watcher
    • View Profile
Re: Possible micro site generation technique?
« Reply #11 on: March 01, 2016, 02:37:16 pm »

You can bet that some lunatics will make adventure mode mega project modifying a huge number of tiles.
I'd rather have the cost of changes taken as they are performed, rather than having them done in bulk on save and/or reload (some things may have to be delayed to reload, such as draining fluids if you want to take the time you've been away into account properly). You don't have to check things every tick if the modifying actions/effects do the checking as part of performing the actions, which probably is best implemented by having tile modifying primitives that perform that check, and don't modify the tiles using any other means (reading is fine as long as you're single threaded, and might work while multi threaded as long as the modifying actions are atomic and the reader doesn't do anything with the tiles. Rendering should work in that case, for instance, with an inconsistent rendered state just overwritten on the next update, but otherwise having no effect).
Logged