Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[rayanth]

I am working on a new tileset, but having trouble determining just HOW dwarf fortress changes the colors of the base chosen tileset for the different materials.

For example, in the default tileset, everything is just two colors (except in 800x600 where dwarf beards are white) so it's easier to see how colors get applied. But if you look at a more graphical tileset, like MayDay (which I love but would like to modify a bit) You find multiple shades of grays, and even some other colors like yellows on ore, and red/green on the switches to tell whether they are on or off.

These colors do get overridden as well to a certain extent (there's no recognition of yellow in Hematite ore, for example, but in other ores there is. For most grays, switches still show the red/green dot readily, but green is not visible on microcline switches. etc.). What I'm trying to figure out is the actual method by which DF uses to replace the tileset colors with the colors for the material, and maintain shading etc.

I've tried searching throughout the wiki and didn't see anything addressing this specifically, only how to change the colors FOR the materials. I've also poked around for a while on this forum and my google-fu must not be strong today, or I'm in the wrong place (If I am, please let me know). My experiments with photoshop and the base images and screencaps of the colorized versions from the game seem to indicate that hue/saturation are partly responsible, but there's no recognizable math to it, so I'm suspecting it's something else and the hue/sat just come out as a result.

Can anyone give me some insight on this?

Thanks a ton,
Rayanth

[ag]

From looking at the graphics code (it is kinda open-source in the linux version), I'd imagine it uses per-component multiply of the texture color by the foreground color of the tile, plus alpha-blend with the background color. Basically, you paint the background colors as raw polygons in opengl, then render foreground+texture polygons on top.

The 'graphics' tile rendering (i.e. those for creatures and stuff like that) also has some kind of grayscale mode, when the texture is first converted to grayscale before applying the same procedure.

[rayanth]

doing a multiply in photoshop gets me a lot closer than where i was before, but is a little too dark. i'll look into opacity settings later. i'm also now digging through the graphics source to see if i can find specifically where it all gets done....

thanks for the tip! eventually i'll get this figured out.

[MasterShizzle]

This is relevant to what I'm doing as well. I'm trying to prettify an ASCII set, but I'm having trouble understanding how DF processes grayscale vs. other things. I can add a background tint to the tiles so that ground tiles appear green etc, but how do I do things like in Mayday or Phoebus, where walls appear differently when they're engraved? Phoebus, for example, keeps the walls the same color but adds the zig-zaggy engraving details when engraving. What is it about these tiles that makes some colors appear solid at times, but appear as the foreground color other times? I've tried adding gray to those tiles, but when I run the program those tiles just appear as their regular color plus gray.

[rayanth]

I'm still doing a barebones rebuild of the graphics system to pinpoint how it really works. Unfortunately it's slow going. No complaints about the system, mind, but it's so diverse and interconnected that it's difficult to follow what's going on, so i'm doing it the other route... rebuilding it with only the core necessities in a separate project, half-pseudo-coding as i go, so i can follow things easier.

[CLA]

This is relevant to what I'm doing as well. I'm trying to prettify an ASCII set, but I'm having trouble understanding how DF processes grayscale vs. other things. I can add a background tint to the tiles so that ground tiles appear green etc, but how do I do things like in Mayday or Phoebus, where walls appear differently when they're engraved? Phoebus, for example, keeps the walls the same color but adds the zig-zaggy engraving details when engraving. What is it about these tiles that makes some colors appear solid at times, but appear as the foreground color other times? I've tried adding gray to those tiles, but when I run the program those tiles just appear as their regular color plus gray.

Let me explain:

Smoothed/constructed walls use the wall tiles with the foreground color of the material and black as background color.
Engraved walls (with varied engravings off) use the wall tiles with the foreground color of the material with another background color(I think it's always LGREY the wiki page "color" has more accurate info IIRC).
So, a smoothed Bauxite wall will be dark red with black background. If you engrave it, it will be dark red with light grey background.

Now, in your tileset, white color will be foreground, transparent (magenta in the old bmps) will be background. If you make white pixels grey, they will get the foreground color, but look darker.
If you color a pixel black, it will always stay black. If you give it another color it will stay that color (or do some crazy multiplying shit, I don't remember).
If you make something white with, say, 50% opacity, it will be colored in the foreground color, but the background color will shine through.

Again:
smooth: FG colored, BG black
engraved: FG colored, BG colored
The trick is to make a wall tile that will look partially different depending on whether it's given a black background by the game or not. We're using the fact that both, transparent and black pixels of a tile will look the same when smooth, but different when engraved.

So if you take a wall tile, make it white and for example add black squiggly lines "inside" (i.e. in between the two white lines in the curses tileset for example) of the wall, it will look just as always when smoothed, but you'll see black squiggly lines when engraved.

This obviously only works for tiles that are used for more than one symbol and get a black background assigned as one and not as the other.
I think that's the case only for minecart tracks and walls.

[rayanth]

My work in deciphering the code led me to an interesting nested loop in the renderer_2d_base class.

This loop loops through each component (RGB and sometimes A) of each pixel in the source texture, and does a complex operation merging the source pixel's colors, the foreground color(adjusted for alpha) and the background color (adjusted for the opposite of alpha or something) and then merged together. the ultimate effect.... is the exact same as 'multiply' in Photoshop.

Now, I may have said before that I did NOT get the right results in photoshop using 'multiply' -- this is because i'm a dolt. I was doing screegrabs using windows 7's Snipping Tool, and knowing that JEG is lossy compression was going with GIF to save the file then open it in photoshop. I forgot that GIF is limited in color pallette, and it was also giving me lossy color-shifting to match the real colors to the GIF pallette.

So, doing everything again but saving as PNG, I got virtually identical to the game. (only differences are where the color in the tileset is partially transparent. still working that part out)

My apologies for doubting you, ag, and my thanks for pointing me in the right direction. what you said sounded right, but my initial experiments led me astray. when I finally saw that loop in the code it all made a lot more sense. and even more sense when I realized I was losing data due to GIF, and redid my screengrabs.

I'll post the final results soon, when I get the last little bit worked out.

[CLA]

You might want to cross-pollinate a little with BradUffner, he's working on something similar with his raw explorer.

[rayanth]

Aaaaaaand I've got it.

Alright, here's the basics. Mind, all of my work has been done with the MayDay tileset, and I've reached my conclusions using the Ore image, with a Hematite coloring (because I had a huge pile of that in one game and it made it easy to screengrab)

So here goes :

Everything that gets colored has a foreground and background color. This is explained in the wiki quite well. What's not explained is how they are merged, and how any alpha values other than 100% opaque can be accounted for. To reproduce the coloration in Photoshop, Follow these steps :

1) Create a 16x16 image, and zoom in enough that things are visible. Disable the background layer (or ensure that it is transparent)
2) Create three new layers So that things are in this order : Foreground, Temporary, Color, (Background Layer)
3) To the Foreground Layer, copy the 16x16 tileset image.
4) Color the Temporary Layer 100% opaque black.
5) Merge the Foreground and Temporary Layers (ensure the Foreground layer is on TOP)  -- this replicates the fact that tiles in the code are blitted to a black surface, so the alpha values from your tile will show black through them
6) Apply the 'foreground' color (in my case 'red', or in the mayday tileset, 200/20/0) to the Color Layer
7) Set both the merged foreground/temporary layer, and the Color layer to 'multiply' mode.
Flatten Image.

And you've now replicated the values. Now, the actual *code* takes 'background color' into effect as well. In Hematite, this is 'lgray' or 190/180/180 in mayday (192/192/192 in default) which equates to 50% which effectively has No Change to the output. I have not yet toyed with any backgrounds other than lgray. It is possible there will be additional steps needed for it.

One caveat: my values in some cases are off by 1. this could be rounding error, or it could be error introduced by the fact that I'm not using the slightly skewed lgray value from Mayday in my photoshop operations. I will toy with this later. In any case, it's minor enough to not be noticable. My eye can't discern between 62/5/0 and 63/6/0


Here's the math, straight from the code :

for EACH R/G/B component of the foreground color (fg), background color (bg) and source (src) do the following :

Divide each value by 255, to make it a number from 0-1
Divide the Alpha value by 255 as well. (alpha)

resulting component = ((alpha * (src * fg)) + ((1 - alpha) * bg)) * 255

'alpha' in this case seems to always be 1 (from 255/255), because it's AFTER the texture's been blitted to a black background and merged. I'll do more testing on that, later.

<MAJOR EDIT>
In the interest of ensuring nobody coming at this later on is thoroughly confused, discussion following this post (see page 3) revealed that I was basing my data on misinformation - When I found that the only way to replicate the images I was seeing was to merge with black, I did not realize that the test point I was using actually had a black background in game. This skewed the above procedure (The Above procedure will ONLY work for tiles that are supposed to have a black background per the raws)

If I get one of those "round tuit" things I hear so much about, I will try to revise the Photoshop Recreation Method in a post later in this thread. Currently my interests are focused elsewhere.
-- Rayanth

[ag]

I'm sure you're wrong about that black thing. The correct procedure should be: 1) multiply the texture with transparency by foreground, 2) place background color behind and let regular alpha formula (fg*alpha + bg*(1-alpha)) work.

[rayanth]

it made no sense to me either, however the corners of my chosen test tile in the mayday tileset have a light tan color with 40% alpha. no matter what i did, that came out as an odd peach color using the formula. but when i merged black first, it came out dead on.

when the tileset surface is initialized, in df, it's initialized to black, and then the tiles are loaded onto it. this would indicate the alpha layers are merging with black, which is why i thought to try it. i will be trying with different color combos later on, particularly those without an lgray background (one of the browns, like copper ore, should do).

sadly, right now my vacation is over and i'm heading back to work. this will still be my active free-time project, but my free-time has been drastically reduced.

[Quietust]

Smoothed/constructed walls use the wall tiles with the foreground color of the material and black as background color.
Engraved walls (with varied engravings off) use the wall tiles with the foreground color of the material with another background color(I think it's always LGREY the wiki page "color" has more accurate info IIRC).
So, a smoothed Bauxite wall will be dark red with black background. If you engrave it, it will be dark red with light grey background.

Actually, both of those colors are specified in the raws - smoothed walls (as well as smooth floors, rough floors, stairs, ramps, boulders, etc.) use the material's BASIC_COLOR, while engraved walls use the material's TILE_COLOR (i.e. exactly the same as unmined tiles). It just so happens that all stone types specify either 0 or 7 for the TILE_COLOR's background (0 gets reinterpreted as "light gray" instead of "black").

The stock raws just use [DISPLAY_COLOR:a:b:c], which is equivalent to [BASIC_COLOR:a:c] and [TILE_COLOR:a:b:c] (and [BUILD_COLOR:b:a:X] where X is 1 if a == b and 0 otherwise), so they generally have the same color between smooth and engraved, but mods are not bound by these restrictions.

[CLA]

Actually, both of those colors are specified in the raws - smoothed walls (as well as smooth floors, rough floors, stairs, ramps, boulders, etc.) use the material's BASIC_COLOR, while engraved walls use the material's TILE_COLOR (i.e. exactly the same as unmined tiles). It just so happens that all stone types specify either 0 or 7 for the TILE_COLOR's background (0 gets reinterpreted as "light gray" instead of "black").

The stock raws just use [DISPLAY_COLOR:a:b:c], which is equivalent to [BASIC_COLOR:a:c] and [TILE_COLOR:a:b:c] (and [BUILD_COLOR:b:a:X] where X is 1 if a == b and 0 otherwise), so they generally have the same color between smooth and engraved, but mods are not bound by these restrictions.

Thanks for clarifying that.

[rayanth]

I'm sure you're wrong about that black thing. The correct procedure should be: 1) multiply the texture with transparency by foreground, 2) place background color behind and let regular alpha formula (fg*alpha + bg*(1-alpha)) work.

Part of the problem is that after the Multiply, all alpha values are 100% opaque, so the background color never comes into effect. If I understand it right, this SHOULD also mean that the 'background color' never really comes into effect anywhere the source tile was 100% opaque, because the background section of the equation will then always equal 0.

Here is my work with Hematite Ore from the Mayday tileset. I will attempt to provide all of my work with explanations. Note that the only alpha values other than 100% in this image are the corners where it looks kind of light tan-ish. the value there is 40% according to Photoshop, which equates to 102/255.

I am posting these at their actual size, which is 16x16. This is to prevent stretch-distortion from getting in the way. I did all of this work at 16x16, zoomed to full 3200% in Photoshop so I could work at the pixel level easier. Feel free to download and zoom in to look closer.

The Original  Source Tile, and the way DF Renders it from the Mayday tileset (Hematite is 'RED' which in Mayday is 200/20/0 RGB, with an LGRAY background, which in Mayday is 190/180/180)

Spoiler (click to show/hide)

If I use the method I outlined, merging to black first and then multiplying by the red value, you get (Left) Compared with that same image from DF's rendering : (Right)

Spoiler (click to show/hide)

Notice that the pixel values are virtually identical - close enough that it can be chalked up to rounding errors, most likely.

If I do a straight multiply with the source and the red color, I get :

Spoiler (click to show/hide)

-- obviously the 'background' corners are way off. the middle is not, because it was 100% opaque anyway. Further adding a LGRAY background and Merging would accomplish nothing, as all alpha values are already 100% now. Doing a Multiply with LGRAY will throw off the colors of the ore itself.

Finally, If I manually calculate the value for the corners, using the equation from the source code and plugging in the numbers into a spreadsheet, I get :
(fg = 200/20/0; bg = 190/180/180; src = 199/178/154; alpha = 102)
((alpha * (src * fg)) + ((1-alpha) * bg)) *255
R : ((0.4 * (0.780392 * 0.784314)) + ((1-0.4) * 0.745098) * 255 == 176.4314
G : ((0.4 * (0.698039 * 0.078431)) + ((1-0.4) * 0.705882) * 255 == 113.5843
B : ((0.4 * (0.603922 * 0)) + ((1-0.4) * 0.705882) * 255 == 108
Rounding, I have 176/114/108, and this is what that looks like :

Spoiler (click to show/hide)

Clearly, THAT is not correct. The only explanation that I could discern was that it was because earlier in the code the texture surface is initialized to black, then the tilemap is loaded onto that. anywhere the tilemap is transparent, it will thus merge with the black, and THEN if you do the multiply with the red color, you get precisely what DF got. (+1 on two values, but again, possibly rounding errors, and damned close enough)

In a nutshell, that's how I reached my conclusion. All of the work shown here was done several times to ensure I made no mistakes mathematically, or in transcribing the numbers to photoshop.

I still want to do the work with other colors, but I don't have the time to finish that tonight. My next shot will be with Copper Ore, as it has a different background color.

If I have still managed to make some kind of critical error somewhere, please don't hesitate to speak up. As is often the case with these types of things, once you've done it so many times, you become blind to mistakes you repeatedly make.

[rayanth]

Alright, so I lied, I had time. Instead of plugging and chugging all of the numbers, even in the spreadsheet, I just followed my outlined procedure above, for Copper Ore.
Copper Ore has a Foreground Color of BROWN (MayDay: 140/98/56) with a Background Color of RED (MayDay : 200/20/0).

My procedure : Merge source image with a Black background, then Multiply by the Foreground color... the results speak for themselves. I never once touched the background color, and it came out identical. Visually, you can't tell the difference : On the left is my representation, on the Right is DF's :



The pixel values are not PERFECT - many are off by 1. The source of this "rounding error" is in the graphics code, which is assigning a floating point math result, to a variable declared as an 8-bit Int. To my knowledge, this usually results in truncation of the decimal from the float, effectively always rounding down. My own math and photoshop's math round nearest. so any value where the floating point result was >= x.5 will be off by 1.

-- Rayanth