Bay 12 Games Forum

Please login or register.

Login with username, password and session length
Advanced search  

Author Topic: DF map compressor problem.  (Read 2152 times)

Arkan15

  • Bay Watcher
    • View Profile
DF map compressor problem.
« on: July 25, 2007, 10:27:00 am »

When I try to run the C version, it says that I need to download the new .NET framework 2.0.bunchofnumbers, and doesn't work.

When I try to download the java version, I get taken to a page with a bunch of java code and nothing happens.

Soo... What blatantly obvious problem am I missing and/or where can I get the .NET framework?

Logged
IP AND TEAR YOUR GUTS!
YOU ARE HUGE! THAT MEANS YOU HAVE HUGE GUTS!

X

  • Bay Watcher
    • View Profile
Re: DF map compressor problem.
« Reply #1 on: July 25, 2007, 11:24:00 am »

The fact you should just use PNG?

X

Logged

Haedrian

  • Bay Watcher
    • View Profile
Re: DF map compressor problem.
« Reply #2 on: July 25, 2007, 03:53:00 pm »

Try the windows update site...
Logged
When life gives you kittens, make biscuits

Likes llamas for their long necks

Veroule

  • Bay Watcher
    • View Profile
Re: DF map compressor problem.
« Reply #3 on: July 25, 2007, 04:02:00 pm »

Get the latest java updates from http://www.java.com/en/download/manual.jsp

.NET should been named .NOT

Logged
"Please, spare us additional torture; and just euthanise yourselves."
Delivered by Tim Curry of Clue as a parody of the lead ass from American Idol in the show Psych.

Markavian

  • Bay Watcher
  • DF Map Archive Admin
    • View Profile
    • DF Map Archive
Re: DF map compressor problem.
« Reply #4 on: July 25, 2007, 05:53:00 pm »

Arkan15, you need Java WebStart for the java link to work, which involves having Java installed like Veroule has said in their response.

You can get .NET 2.0 from Microsoft's website. Both tools work identically, so its up to you to pick the preferred software packs, both are ~20MB to install, both perform as well as each other.

If you have any other problems, give me a shout and I'll try and sort you out.

Microsoft .NET 2.0 redist/install for x86 machines: http://www.microsoft.com/downloads/details.aspx?familyid=0856EACB-4362-4B0D-8EDD-AAB15C5E04F5&displaylang=en

Regards,
- Markavian http://mkv25.net/dfma/

Logged

Arkan15

  • Bay Watcher
    • View Profile
Re: DF map compressor problem.
« Reply #5 on: July 25, 2007, 07:37:00 pm »

quote:
Originally posted by X:
<STRONG>The fact you should just use PNG?

X</STRONG>


Have you actually read about how the compressor works?

DF map images are so big that even in PNG they're several MB. The compressor encodes them using some sort of... recursive thingamajig and gets them to a few hundred KB.

quote:
Arkan15, you need Java WebStart for the java link to work, which involves having Java installed like Veroule has said in their response.
You can get .NET 2.0 from Microsoft's website. Both tools work identically, so its up to you to pick the preferred software packs, both are ~20MB to install, both perform as well as each other.

If you have any other problems, give me a shout and I'll try and sort you out.

Microsoft .NET 2.0 redist/install for x86 machines: http://www.microsoft.com/downloads/details.aspx?familyid=0856EACB-4362-4B0D-8EDD-AAB15C5E04F5&displaylang=en  

Regards,
- Markavian http://mkv25.net/dfma/  


Thanks!

Logged
IP AND TEAR YOUR GUTS!
YOU ARE HUGE! THAT MEANS YOU HAVE HUGE GUTS!

X

  • Bay Watcher
    • View Profile
Re: DF map compressor problem.
« Reply #6 on: July 25, 2007, 07:49:00 pm »

world_map-5-1050-0.png
Dimensions: 2056x3084 Type: PNG Image Size: 238kb

Only getting one order of magnitude smaller than a general purpose image format is not worth creating a proprietary, undocumented, NET/Java/Flash requiring mess.
This does not mean that the way of taking screen/mapshots in the game is not in need of improvement.

X

Logged

Arkan15

  • Bay Watcher
    • View Profile
Re: DF map compressor problem.
« Reply #7 on: July 25, 2007, 07:59:00 pm »

quote:
Originally posted by X:
<STRONG>world_map-5-1050-0.png
Dimensions: 2056x3084 Type: PNG Image Size: 238kb

Only getting one order of magnitude smaller than a general purpose image format is not worth creating a proprietary, undocumented, NET/Java/Flash requiring mess.
This does not mean that the way of taking screen/mapshots in the game is not in need of improvement.

X</STRONG>



You are aware that if you haven't mined into the mountain, the mapshot will be smaller and not have detail past that point...


right?

Logged
IP AND TEAR YOUR GUTS!
YOU ARE HUGE! THAT MEANS YOU HAVE HUGE GUTS!

puke

  • Bay Watcher
    • View Profile
Re: DF map compressor problem.
« Reply #8 on: July 25, 2007, 08:00:00 pm »

most of my PNGs are not that small.  theyre usually about 2-4 megs.  that kind of size advantage is certainly worthwhile.  i like standards as much as the next guy, but there is so much slop code out there that its nice to see someone doing something efficiently.
Logged

Leerok the Lacerta

  • Bay Watcher
  • Linuxer
    • View Profile
    • The Online Journal of Leerok the Lacerta
Re: DF map compressor problem.
« Reply #9 on: July 27, 2007, 11:20:00 pm »

I say, do try PNGOUT. It generally will reduce the size of most PNGs and is particularly effective on Dwarf Fortress maps. I think it also takes BMP as input as well. I think my maps have reached down below 200K, even when I'm at the magma.
http://advsys.net/ken/utils.htm

Makes me wonder how the Z-axis version will handle map exports.

Markavian

  • Bay Watcher
  • DF Map Archive Admin
    • View Profile
    • DF Map Archive
Re: DF map compressor problem.
« Reply #10 on: July 28, 2007, 07:30:00 pm »

Following the current discussion items:
For a map that stretches from the front of the fort out to the magma fissure, you can expect the bitmap file to be 82MB for a 12x12 tileset. (5052x5760)

Optimizing the image using Paint Shop Pro 7, brings the filesize down to 750KB.
Optimizing the image using optipng, brings the filesize down to 702KB.
Both using 256 colour palettes, 50 unique colours.

This same map, compressed identically using SL's map compresssor (.NET or Java) is 71.9KB.

The compressor works by identifying unique tiles in the image, and storing them in a tile index. This is similar to the colour palette in an image, but made up of image tiles. This bitmap data is stored using RLE. The rest of the map file is numeric indexes which reference tiles in the tile index. The resulting data is then compressed using zlib compression library.

quote:
<STRONG>Only getting one order of magnitude smaller than a general purpose image format is not worth creating a proprietary, undocumented, NET/Java/Flash requiring mess.</STRONG> -Posted by X

Well, that's not a very constructive comment, but I will continue to point out my perceived benefits of the system:

  • From the viewer's perspective, they only need Flash to view other people's map. The reduced filesize means people have to wait (at best) 10 times less to load and view a map, e.g. 798ms to download, and in total 1.2 seconds to start rendering the map. Try downloading and viewing a PNG file map; is it noticably quicker?
  • From my persepective as a website owner, that order of 10 magnitude means huge savings in filespace and bandwidth. With the success in submissions, I have been able to build a community site that many have felt confident to contribute to (much appreciated).
  • The file format is not closed, a description is available on SL's website. If I knew how, I would write a image plugin for read/write. I don't though. Even if I did, you'd have to download and install that plugin, which I'm sure everyone agrees is a bad idea. At least with the Flash viewer, everyone who has the latest version of flash (which 86% of net users do) can view the maps.
  • From the encoder's perspective, the player who wants to share their map; it is unfortunate that they are restricted to use Java or .NET to compress their maps - but at least SL has provided alternatives. Someone else did attempt to write a C++ compressor, not sure where that is now.

I am not prepared for the website to support images that are between 250KB ~1MB, potentially up to ~2MB (PNG files) for the larger tilesets. And if that were the case, we wouldn't have the map archive. Thanks to SL's (genius?) with the compressor, and my work creating the viewer/website I think the community is a better place.

RE: Exporting maps in the new version of the game. Toady has indicated he has made some improvements to the output file and naming. However, I imagine that it still only exports a single layer image. I am hoping that exporting the full fortress layers will create sequentially numbered bitmap files, and that SL can write a new compressor / variant fileformat to support multiple layers. I can then write a new version of the viewer that supports both formats.

Logged

X

  • Bay Watcher
    • View Profile
Re: DF map compressor problem.
« Reply #11 on: July 28, 2007, 10:45:00 pm »

http://en.wikipedia.org/wiki/Order_of_magnitude

& etc. with the rest of your post.

X

Logged

Shadowlord

  • Bay Watcher
    • View Profile
Re: DF map compressor problem.
« Reply #12 on: May 04, 2009, 03:25:21 am »

Although this is rather an old thread, which would make me more of a Thread Necromancer, I just spent 10 minutes converting the 17 BMPs from one of my forts which used a graphics set and a fancy graphical font (16x16) into PNGs using Paint.NET, and it compressed them from 1 GB to 9.78 MB, and it took me 10 minutes to do it, which took much longer than using the (er, my) Map Compressor. Now if I could tell paint.net to batch convert all the files that would be easier, but it still would have taken at least 5 minutes, I think. Anyway, there are programs that WOULD batch convert them, but then you have the hassle of getting them, etc etc. Onward:

PNGs: 9.78 MB.

Now, X said that an order of magnitude improvement in efficiency wasn't good enough.

So, I compared it to my fdf-map from the same images: 145 KB. Every time you have an image of a plump helmet, in every map, that image will still only be stored in the fdf-map file *once*. Especially with the multiple maps, there's just no way for conventional image compression to take advantage of the images being in multiple files because there's no way for them to analyze multiple files and identify patterns common to all of them (especially without running out of memory).

fdf-map: 145 KB.

For those, the fdf-map file is 1.4434% of the size of the PNG files, which is just about two orders of magnitude now, took less time to make, and was easier to make. The only problem would be that you need .net 2.0 or mono, and mono doesn't seem to work everywhere, and although vista supposedly comes with .net 2.0, apparently a few people have had trouble with that (I saw some forum posts by this one guy)? I've got links right to where to get it in the page(s) for downloading the compressor though.

Someday I'd like to rewrite/port the map compressor in another language which is portable and doesn't require additional stuff to be installed by the user, rewriting it as a portable library which DF can call functions in to directly write out an fdf-map file, rather than writing out BMPs. Then it could be included with DF, and users wouldn't have to do anything except hit export from inside DF, and upload their fdf-map file to the map archive. But currently that's not a priority. (I do know several other languages, including c++, but would probably end up attempting it in haskell instead)

I'd like to be doing something similar to what Toady's doing, making a good detailed free game, with modular text/tile-based graphics, and actually earning a living from the donations (eventually), rather than not making any money and not working at anything I like, and that (albeit distant) goal is the main reason the map compressor library rewrite idea is basically just an idea and nothing more at this point. (It's also the same reason I suspended pretty much all of my other projects)

If at some point in the future I've got time to devote to side projects and decent income, I'll probably revive the idea.
« Last Edit: May 04, 2009, 03:31:38 am by Shadowlord »
Logged
<Dakkan> There are human laws, and then there are laws of physics. I don't bike in the city because of the second.
Dwarf Fortress Map Archive