Bay 12 Games Forum

Please login or register.

Login with username, password and session length
Advanced search  

Author Topic: Parsing world.dat  (Read 1375 times)

Riemann

  • Bay Watcher
    • View Profile
Parsing world.dat
« on: February 22, 2008, 07:02:00 pm »

It occured to me this week that in all the hours I've spent scrolling around world maps looking for start locations (even with regional_prospector) I could have just written a tool to do it. Damnit Jim, I'm a programmer not a map... looker ... guy.

Based on the info on Ricks page I've got a tool underway to decode world.dat with the intention of eventually making it able to search for locations on the world map that meet certain criteria.

So far (~3 hours into the project) it just opens the file, grabs each 20K chunk and deflates them into an output stream.

If this turns into something more useful I'll start posting source code. It's a damn big file though (that ~20 meg world.dat uncompresses to a good ~85 megs of data).

Edit: Forgot to metion, the code is entirely in C# 2.0 / .NET 3.0 with WPF as the UI.

[ February 22, 2008: Message edited by: Riemann ]

Logged

beorn080

  • Bay Watcher
    • View Profile
Re: Parsing world.dat
« Reply #1 on: February 22, 2008, 07:12:00 pm »

I know this is way down the road but If you could make it so it can deparse the data stream, you could make custom embark zones that contains chasms, above and under ground rivers, HFS, magma, and all the other goodies in one 2x3 spot and that would rock.
Logged
Ustxu Iceraped the Frigid Crystal of Slaughter was a glacier titan. It was the only one of its kind. A gigantic feathered carp composed of crystal glass. It has five mouths full of treacherous teeth, enormous clear wings, and ferocious blue eyes. Beware its icy breath! Ustxu was associated with oceans, glaciers, boats, and murder.

Riemann

  • Bay Watcher
    • View Profile
Re: Parsing world.dat
« Reply #2 on: February 22, 2008, 07:16:00 pm »

Hell, right now I just want to figure out the basic struct for map data. But if I can make it work the possiblity is open to making an actual map editor / creator.

But damn it's been 10 years since I've done this kind of thing (eg: decoding an unkown binary).

Logged

penguinofhonor

  • Bay Watcher
  • Minister of Love
    • View Profile
Re: Parsing world.dat
« Reply #3 on: February 22, 2008, 07:36:00 pm »

Would you be able to input anything you want, instead of being forced to look for maps with magma, adamantine, cave rivers, iron, flux, a chasm, trees, no aquifer, and the cure for cancer?

Because some of us don't like those kinds of maps.

Logged

Riemann

  • Bay Watcher
    • View Profile
Re: Parsing world.dat
« Reply #4 on: February 22, 2008, 08:06:00 pm »

Well, again if I can figure out this binary parsing shit, mostly what I am interested in is creating a good search UI to look for as wide a range as possible. Eg: Other than perhaps an initial test-setup it will not be hardcoded what to look for.

This is not intended to just be a searcher for "perfect" locations. More what I have in mind is: I want to try a terrifying map in X climate with some limestone.

Or whatever. I imagine it might be useful to Toady as well just to be able to search up a location that meets certain criteria for debuging / testing.

Logged

Derakon

  • Bay Watcher
    • View Profile
Re: Parsing world.dat
« Reply #5 on: April 05, 2008, 11:19:00 pm »

Bumping, as I have a similar interest. I'm on a Mac, though, so I'd probably have to write my own program to do the searching. Did you have any success in parsing the map data?
Logged
Jetblade - an open-source Metroid/Castlevania game with procedurally-generated levels

Earthquake Damage

  • Bay Watcher
    • View Profile
Re: Parsing world.dat
« Reply #6 on: April 06, 2008, 06:44:00 pm »

Name:  Riemann
Status:  Writing a utility

Are you a math/comp sci major or something?

Logged

Rick

  • Bay Watcher
  • Engineer
    • View Profile
Re: Parsing world.dat
« Reply #7 on: April 06, 2008, 06:54:00 pm »

There is a lot of data in world.dat, and making changes without knowing its structure is probably a bad idea.
Logged

Derakon

  • Bay Watcher
    • View Profile
Re: Parsing world.dat
« Reply #8 on: April 06, 2008, 08:02:00 pm »

I don't think anyone who was actually interested in making a utility was thinking of making one to modify the map data; just to read it. Like the original poster, I want to have a program (well, more likely a script of some sort) that would parse through the map data and say "there is a site with magma, an [underground] river, flux, iron ore, and skeletal puppies all within 3 squares of each other at <84>". To do that, I need:

* To be able to read the binary world.dat file
* To know where the map region data begins
* To know the structure of the map region data

I downloaded a basic hex viewer program and loaded up a pocket world.dat, but I wasn't even able to find basic things like the world's name (or, on embarking, the fortress name). I am on a Mac, but it's an Intel Mac; there shouldn't be any endianness problems.

Logged
Jetblade - an open-source Metroid/Castlevania game with procedurally-generated levels

Rick

  • Bay Watcher
  • Engineer
    • View Profile
Re: Parsing world.dat
« Reply #9 on: April 06, 2008, 08:04:00 pm »

Pretty sure my world.dat page says that the data is compressed in blocks using zlib?
Logged

Derakon

  • Bay Watcher
    • View Profile
Re: Parsing world.dat
« Reply #10 on: April 06, 2008, 08:11:00 pm »

D'oh, I missed that one line at the top. Okay, then. Guess the first thing to do is write a program to uncompress everything. Thanks, Rick.
Logged
Jetblade - an open-source Metroid/Castlevania game with procedurally-generated levels

numerobis

  • Bay Watcher
    • View Profile
Re: Parsing world.dat
« Reply #11 on: April 06, 2008, 11:15:00 pm »

quote:
Originally posted by Derakon:
<STRONG>I'm on a Mac, though, so I'd probably have to write my own program to do the searching.</STRONG>

The save files are compatible, so you shouldn't need any such.  And if you're on Intel mac, you wouldn't even need to worry about endianness.

Logged

Derakon

  • Bay Watcher
    • View Profile
Re: Parsing world.dat
« Reply #12 on: April 07, 2008, 10:31:00 am »

It's more that the OP was talking about using .NET, really.
Logged
Jetblade - an open-source Metroid/Castlevania game with procedurally-generated levels

numerobis

  • Bay Watcher
    • View Profile
Re: Parsing world.dat
« Reply #13 on: April 07, 2008, 10:59:00 am »

quote:
Originally posted by Derakon:
<STRONG>It's more that the OP was talking about using .NET, really.</STRONG>

Mono supposedly mostly works.

Logged

Rick

  • Bay Watcher
  • Engineer
    • View Profile
Re: Parsing world.dat
« Reply #14 on: April 07, 2008, 01:47:00 pm »

Mono should have no issues with file related stuff, WPF/WinForms on the other hand...
Logged