Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[Pidgeot]

As for the border thing, I assume that's only for the example files or - if we follow pidgeots suggestion to use it for colorization indication - temporary, because those would be visible in game.

The idea is that if you have, say, a 16x16 set of tiles, the generated image from all of this uses a grid of 17x17* - 1 pixel in each dimension is used for the border, and then the remaining 16 pixels are the actual tile graphics.

If the 16x16 tile graphics are completely blank - i.e., all pixels transparent - then we ignore that tile, because it's not specified, otherwise we cut out that 16x16 tile, assemble them all into whatever image layout the game wants.

The border is useful because it makes the separation between each tile more clear - and it also just happens that we can exploit the fact that it's there anyway to carry hidden information. For example, we wouldn't necessarily need to store the tile size explicitly - the top left (or bottom right, etc.) pixel of the image could be colored as RGB(x_size, y_size, whatever) and we could just check that when building the raws to know what we were working with. The final component could be used to mark a file format version, or you could go to RGBA to specify how *many* tiles were in each direction, etc. It's really just a way you might capture the additional metadata that needs to be applied to the raws (of course, you need to figure out *what* that metadata might be, and document how that gets mapped between the images and the raws, but the basic idea is that if you just add a single image, you can just copy the border of a cell that works similarly - or make your graphical change, update the raws after applying it, and then regenerate the grid format.

*The reason for 17x17 instead of 18x18 is that you otherwise get the appearance of a 2 pixel border between cells, but only a single pixel border on the edges. It makes things seem a bit more ordered if the borders appear uniform.

[Quiet-Sun]

The idea is that if you have, say, a 16x16 set of tiles, the generated image from all of this uses a grid of 17x17* - 1 pixel in each dimension is used for the border, and then the remaining 16 pixels are the actual tile graphics.

If the 16x16 tile graphics are completely blank - i.e., all pixels transparent - then we ignore that tile, because it's not specified, otherwise we cut out that 16x16 tile, assemble them all into whatever image layout the game wants.

The border is useful because it makes the separation between each tile more clear - and it also just happens that we can exploit the fact that it's there anyway to carry hidden information. For example, we wouldn't necessarily need to store the tile size explicitly - the top left (or bottom right, etc.) pixel of the image could be colored as RGB(x_size, y_size, whatever) and we could just check that when building the raws to know what we were working with. The final component could be used to mark a file format version, or you could go to RGBA to specify how *many* tiles were in each direction, etc. It's really just a way you might capture the additional metadata that needs to be applied to the raws (of course, you need to figure out *what* that metadata might be, and document how that gets mapped between the images and the raws, but the basic idea is that if you just add a single image, you can just copy the border of a cell that works similarly - or make your graphical change, update the raws after applying it, and then regenerate the grid format.

*The reason for 17x17 instead of 18x18 is that you otherwise get the appearance of a 2 pixel border between cells, but only a single pixel border on the edges. It makes things seem a bit more ordered if the borders appear uniform.

I was thinking that it would be best to keep the tile size the same.  The reason is that we have the walls and other items that need to be continuous.  We could take the outer pixels from the creature tile, but I have the impression that some artists use the entire tile.   What I was thinking as an alternative is to use only the corners of the tile.  4corners*4channels(RGBA)*255 gives us plenty of room to encode information.  What do you think?

I guess the flaw that I see in your current script is that it relies on preexisting definition text files. So if a HIPPO is defined twice or something is misspelled or GIANT_LION is defined as LION_GIANT or the myriad of other possibilities - how does that even work? Also, what's the point if you need a preexisting error free definition text file for the script to work in the first place, you know?

When you say it shouldn't be too difficult I feel as if you're not considering things like professions that exist in game that you can define tiles for that are not in the raws . . . how are you going to pull that off?

No pressure Quiet-Sun. ;]

I should correct myself.  It shouldn't be too difficult to do it for creatures.  Humanoids are a different beast, but I'm not going to worry about them now or I won't make any progress.   As you have said before this is not an easy task, but one that I consider tractable.  I know I'm not going to be able to pull it off in the first try, but that's not a big deal for me.

One thing that you should keep in mind is that we are both defining a standard AND standardizing existing disparaging sets.  At some point or other, something will have to be fixed by a human, but I am convinced that we can do most of the work automatically.   

What I need right now is proactive help and not sarcasm.  Identifying problems is useful and necessary, but your input would be immensely more valuable if you also try to help us find solutions.  Your help (and by you I mean everyone that understands how DF works) is critical for my success.

HERE ARE SOME THINGS THAT I NEED:

- Does anyone know where in DF's /objects/*.txt files are all the professions (or other variables that may affect tile appearance)?  I want my standardizing scripts for humanoids to be solely based on the files found in DF so that we can update things easily as the game evolve. 

- I see that some races have tons of professions (dwarves), but some seem to have a handful (worm-men).  Where in DF's /objects/*.txt can I find such a difference defined?  Is it just those that are creatures?  How many professions am I dealing with?

[CLA]

I guess the flaw that I see in your current script is that it relies on preexisting definition text files. So if a HIPPO is defined twice or something is misspelled or GIANT_LION is defined as LION_GIANT or the myriad of other possibilities - how does that even work? Also, what's the point if you need a preexisting error free definition text file for the script to work in the first place, you know?

The way I understood Quiet-Sun's efforts, there would be two things the script does eventually:

• take existing graphic sets and get information about what tile is what from the existing definition files.
• autogenerate the standard definition text files directly from the raw files. Consequently there would be no misspelling or problems with creature naming inconsistencies.
• standardize (=reorder) existing image files. based on [2]. Which is what the current script does as a proof-of-concept.

So the script would take existing creature graphics and their existing creature definition files and generate a dataset. Tiles stored separately with their definition taken from the text file.
You would get a set of tiles with a known meaning (tile 1 is creature X, tile 2 is creature Y, tile 3 is creature Z:profession A, tile 4 is creature Z:profession B, etc).

On the other end, the script would take the raws and - based on hardcoded rules - generate another dataset from it (number of creatures and their names, number of texture tokens/professions and their names, resulting dimensions of image files).
You would get a set of creatures with a known coordinate (creature X sits at 2:5, creature Y at 0:0, etc)

Finally, it would create standard creature definition text files and the new, reordered image files.

The idea is that if you have, say, a 16x16 set of tiles, the generated image from all of this uses a grid of 17x17* - 1 pixel in each dimension is used for the border, and then the remaining 16 pixels are the actual tile graphics.

If the 16x16 tile graphics are completely blank - i.e., all pixels transparent - then we ignore that tile, because it's not specified, otherwise we cut out that 16x16 tile, assemble them all into whatever image layout the game wants.
The border is useful because it makes the separation between each tile more clear -

More clear for whom? It's trivial for a computer to just count the 16 pixels, it doesn't need to 'see'. And eventually the whole process from start to finish would be automated, so nobody else would need to distinguish the tiles either.

metadata embedding

I don't see the reasoning with that. Essentially, it seems that between the first input and the final output, you suggest to store all the information in the image - I'm wondering why not store all the information in a plain text database instead? That seems easier, more compatible and more transparent.
Taking the existing tilesets, separating them, adding a border with embedded data, then taking that and removing it again to use in DF seems a bit roundabout. Why not transfer the metadata directly in the final definition file?


EDIT: Just realized, the "datasets" I'm talking about, are the creature definition files.

[Pidgeot]

I was thinking that it would be best to keep the tile size the same.  The reason is that we have the walls and other items that need to be continuous.  We could take the outer pixels from the creature tile, but I have the impression that some artists use the entire tile.   What I was thinking as an alternative is to use only the corners of the tile.  4corners*4channels(RGBA)*255 gives us plenty of room to encode information.  What do you think?

I believe it is a bad idea because it becomes impossible to use those corners for actual graphics (how would you know what is supposed to be there?) - which is already going to cause issues with some of the examples you've already shown (e.g. reindeer in Spacefox).

Even if you cheated and said you only reserved the alpha channel for this information... that's still going to make the graphics visibly different, which is going to make it difficult to properly imagine how it's going to look.

I imagine that very few people will want to manually add the metadata we're talking about - they'd much rather draw the graphics, generate some raws, modify those, then run the process in reverse. They have the *option*, sure, since it's all just pixels with defined colors, but it's a bit abstract and not something I imagine most people are up for (except perhaps if they're just modifying an existing sprite for a one-off, in which case they can copy the border as well).

For the case where things have to join up... those also have to be able to join up with *themselves*, and you can't really check that anyway in a grid format like this.

The idea is that if you have, say, a 16x16 set of tiles, the generated image from all of this uses a grid of 17x17* - 1 pixel in each dimension is used for the border, and then the remaining 16 pixels are the actual tile graphics.

If the 16x16 tile graphics are completely blank - i.e., all pixels transparent - then we ignore that tile, because it's not specified, otherwise we cut out that 16x16 tile, assemble them all into whatever image layout the game wants.
The border is useful because it makes the separation between each tile more clear -

More clear for whom? It's trivial for a computer to just count the 16 pixels, it doesn't need to 'see'. And eventually the whole process from start to finish would be automated, so nobody else would need to distinguish the tiles either.

For a person who is using the grid to figure out what they need to create. It's just as trivial for the computer to count 17 (or 18) pixels, but it's much easier for a human to make sure they're "staying within the lines" if they have lines to follow for those cases where you're doing one-off edits; otherwise it might be difficult to see you're accidentally going one pixel into another tile (which just happened to be a transparent area, so you couldn't tell easily).

It's also a tiny bit easier for the computer when it needs to figure out whether to skip a given tile (because you're looking for a tile where all pixels are transparent, as opposed to pixels where some are and some aren't).

metadata embedding

I don't see the reasoning with that. Essentially, it seems that between the first input and the final output, you suggest to store all the information in the image - I'm wondering why not store all the information in a plain text database instead? That seems easier, more compatible and more transparent.
Taking the existing tilesets, separating them, adding a border with embedded data, then taking that and removing it again to use in DF seems a bit roundabout. Why not transfer the metadata directly in the final definition file?


EDIT: Just realized, the "datasets" I'm talking about, are the creature definition files.

You can certainly use a secondary file for the metadata if you'd prefer, and it'd probably be more efficient since you'd potentially be able to avoid stitching the image together - but it's my impression that for graphics sets, the amount of customization needed is going to be quite limited post-standardization anyway. You have to analyze the image anyway to tell whether or not a given tile is present (and therefore, whether it should be pointed to in the generated raws)... so it's not much extra effort to analyze a couple extra pixels to get the metadata you need at the same time.

You could also just choose to leave all metadata in the RAWs and only use this to unify the positions for a given creature. Depends what you want to be able to do, of course... but if that's your entire goal, then it seems overkill to add all of the grid labels.

[burned]

I guess the flaw that I see in your current script is that it relies on preexisting definition text files. So if a HIPPO is defined twice or something is misspelled or GIANT_LION is defined as LION_GIANT or the myriad of other possibilities - how does that even work? Also, what's the point if you need a preexisting error free definition text file for the script to work in the first place, you know?

When you say it shouldn't be too difficult I feel as if you're not considering things like professions that exist in game that you can define tiles for that are not in the raws . . . how are you going to pull that off?

No pressure Quiet-Sun. ;]

I should correct myself.  It shouldn't be too difficult to do it for creatures.  Humanoids are a different beast, but I'm not going to worry about them now or I won't make any progress.   As you have said before this is not an easy task, but one that I consider tractable.  I know I'm not going to be able to pull it off in the first try, but that's not a big deal for me.

One thing that you should keep in mind is that we are both defining a standard AND standardizing existing disparaging sets.  At some point or other, something will have to be fixed by a human, but I am convinced that we can do most of the work automatically.   

What I need right now is proactive help and not sarcasm.  Identifying problems is useful and necessary, but your input would be immensely more valuable if you also try to help us find solutions.  Your help (and by you I mean everyone that understands how DF works) is critical for my success.

Quiet-Sun, I was not being sarcastic. I sincerely don't see the point if you need a preexisting error free definition text file.

You're right. Solutions are more helpful than just the problems. But, I do believe sharing the problems I am identifying is worth it so that solutions are sought rather then the problem overlooked. I have no idea at this point what the solution(s) would be. I am sharing it via this thread so that you and others are made aware of what needs to be addressed. Sincerely, I would love for you or someone to pull this off as previously stated. To be absolutely clear, I am rooting for you!

HERE ARE SOME THINGS THAT I NEED:

- Does anyone know where in DF's /objects/*.txt files are all the professions (or other variables that may affect tile appearance)?  I want my standardizing scripts for humanoids to be solely based on the files found in DF so that we can update things easily as the game evolve. 

No, they are not all the professions. That was one of the things I was concerned about since there are a lot of professions that are not in ANY of the /objects/*.txt files, hence my sincere question of "how are you going to pull that off?"

I do not see a way that you can cover all professions based on the /objects/*.txt files found in DF.

- I see that some races have tons of professions (dwarves), but some seem to have a handful (worm-men).  Where in DF's /objects/*.txt can I find such a difference defined?  Is it just those that are creatures?  How many professions am I dealing with?

I have a complete list I can send you, but it has everything in it - meaning, not just currently used professions, but outdated professions (in case they are implemented again), professions that are mere guesses, and professions I have not been been able to successfully verify, but they are referenced on the wiki and other definition files consistently.

On the other end, the script would take the raws and - based on hardcoded rules - generate another dataset from it (number of creatures and their names, number of texture tokens/professions and their names, resulting dimensions of image files).
You would get a set of creatures with a known coordinate (creature X sits at 2:5, creature Y at 0:0, etc)

Finally, it would create standard creature definition text files and the new, reordered image files.

Thanks CLA, but the above part is where I am lost. So instead of a preexisting definition text you would have to set up the tiles in an image to match the predefined defintion text? Also, do you mean hardcoded rules would be the solution to the professions not found in the any of the raw files? In other words, the script hardcodes all the professions somehow?

[CLA]

More clear for whom? It's trivial for a computer to just count the 16 pixels, it doesn't need to 'see'. And eventually the whole process from start to finish would be automated, so nobody else would need to distinguish the tiles either.

For a person who is using the grid to figure out what they need to create.

Oh I see now what you mean.
Photoshop (and I assume Krita or GIMP) have features to display a grid of specified size, which solves that problem elegantly. If we want to embed information in the file, I would only do it on empty tiles, and checkerboard those with two different colors. In addition, add the tile coordinate on the empty tile.

So instead of a preexisting definition text you would have to set up the tiles in an image to match the predefined defintion text?

Yes. For existing graphics, this would happen automatically based on the old definition text. For entirely new graphics, you'd have to learn the coordinates from the standard definition file.
Right now, every graphic set artist has to draw all the tiles in whatever organization they deem best, and then type out all the creature definitions manually.
With standardized creature definitions, you have to type out nothing and just draw. Downside is that you have to draw specific creatures on specific tiles. I'd still consider that a major improvement though.

This is, how I understand it, the main advantage of this whole endeavor: reduced workload for artists because the definition files are autogenerated. It can only be automated if it's standardized. And if we can take all the information we have from the raws, we never (or rarely) need to update the script.

As for professions, with "hardcoded rules" you quoted I was simply referring to the actual raw parser code. I.e. 'look for string "[CREATURE:<anything>]" and discard everything before and after' - that kind of stuff.
Regardless, you're right: If there's no way to get a correct and complete list from either the raws or DFHack trickery, I see no other solution than to hardcode it (In a separate file, ideally).


For reference, this is the ideal workflow I hope to see for graphic set authors once we have a working setup:

Spoiler (click to show/hide)

For existing graphic sets:

• run the set through the script
• done, your graphic set is now standardized.
• if you are missing creatures, the script comments out the entries
• if you want to use ADD_COLOR instead of AS_IS, you need to manually replace it

If a new DF version comes out that adds new creatures or changes names, or whatever, you simply run the script again and your graphic set is updated within seconds.

If you want to add new creatures

• ctrl+f through the definition text file, note coordinates
• draw new creature

For someone that wants to make a new graphic set from scratch

• download template and read manual on what to do
• decide on a tile size
• generate empty image files (could also be automated I guess)
• check dimension
• draw creatures

Current problems I see:
Do we have to hardcode professions?
What do we do with empty tiles? When the script can differentiate between an empty tile and a non-empty tile, it should be trivial to comment out that line. That seems to be the easiest solution.
We could instead fill the empty tiles with their default letter of the default tileset.
This would reduce the difference in the definition files, but as every graphic set needs its own definition file anyway (AS_IS vs ADD_COLOR), it would just complicate things, I think.

[Quiet-Sun]

I have a complete list I can send you, but it has everything in it - meaning, not just currently used professions, but outdated professions (in case they are implemented again), professions that are mere guesses, and professions I have not been been able to successfully verify, but they are referenced on the wiki and other definition files consistently.

Thank you Burned! this would be very helpful.   Can you please send me one without outdated professions?  We can assume your list is complete and improve it if need be.

For reference, this is the ideal workflow I hope to see for graphic set authors once we have a working setup:

Spoiler (click to show/hide)

For existing graphic sets:

• run the set through the script
• done, your graphic set is now standardized.
• if you are missing creatures, the script comments out the entries
• if you want to use ADD_COLOR instead of AS_IS, you need to manually replace it

If a new DF version comes out that adds new creatures or changes names, or whatever, you simply run the script again and your graphic set is updated within seconds.

If you want to add new creatures

• ctrl+f through the definition text file, note coordinates
• draw new creature

For someone that wants to make a new graphic set from scratch

• download template and read manual on what to do
• decide on a tile size
• generate empty image files (could also be automated I guess)
• check dimension
• draw creatures

Do we have to hardcode professions?
What do we do with empty tiles? When the script can differentiate between an empty tile and a non-empty tile, it should be trivial to comment out that line. That seems to be the easiest solution.
We could instead fill the empty tiles with their default letter of the default tileset.
This would reduce the difference in the definition files, but as every graphic set needs its own definition file anyway (AS_IS vs ADD_COLOR), it would just complicate things, I think.

That's exactly it CLA!!  That is exactly what I would like to implement.  I think I have a solution for all the problems you mention:

Do we have to hardcode professions?

I'm going to create three separate reference texture/profession files: one for creatures, one for beastmen, and one for major races (see Burned comment above).  We can then expand them or trim them as our knowledge improves (and/or DF changes).  These will determine the entries on the y axis of the matrix (what you reference as A through Z).   The creatures will come directly from the raws.  These will determine the entries of the x axis of the matrix (what you reference as 1 throuh N).

What do we do with empty tiles? When the script can differentiate between an empty tile and a non-empty tile, it should be trivial to comment out that line. That seems to be the easiest solution.

We can try both approaches.  They should not be too difficult to implement and see what works best.  Alternatively we could use a default creature template like Mayday for all empty creatures.

This would reduce the difference in the definition files, but as every graphic set needs its own definition file anyway (AS_IS vs ADD_COLOR), it would just complicate things, I think.

Actually I think I have the solution fro the AS_IS vs ADD_COLOR.  Looking at CLA's set I noticed that all his tiles are pure white.   We can use ADD_COLOR if the tile has only white, or AS_IS if not (we can make the rule more complicated if necessary, for example using histograms of the tiles).

[Quiet-Sun]

Ok Guys!  I have the standard text files for creatures now.  You can see one of them here:

Spoiler (click to show/hide)

creature_amphibians

[OBJECT:GRAPHICS]

[TILE_PAGE:CREATURE_AMPHIBIANS]
   [FILE:creature_amphibians.png]
   [TILE_DIM:X:X]
   [PAGE_DIM:2:5]

[CREATURE_GRAPHICS:TOAD]
   [DEFAULT:CREATURE_AMPHIBIANS:0:0:AS_IS:DEFAULT]
   [CHILD:CREATURE_AMPHIBIANS:0:1:AS_IS:DEFAULT]
   [ANIMATED:CREATURE_AMPHIBIANS:0:2:AS_IS:DEFAULT]
   [TRAINED_HUNTER:CREATURE_AMPHIBIANS:0:3:AS_IS:DEFAULT]
   [TRAINED_WAR:CREATURE_AMPHIBIANS:0:4:AS_IS:DEFAULT]

[CREATURE_GRAPHICS:GIANT_TOAD]
   [DEFAULT:CREATURE_AMPHIBIANS:1:0:AS_IS:DEFAULT]
   [CHILD:CREATURE_AMPHIBIANS:1:1:AS_IS:DEFAULT]
   [ANIMATED:CREATURE_AMPHIBIANS:1:2:AS_IS:DEFAULT]
   [TRAINED_HUNTER:CREATURE_AMPHIBIANS:1:3:AS_IS:DEFAULT]
   [TRAINED_WAR:CREATURE_AMPHIBIANS:1:4:AS_IS:DEFAULT]

Here is its corresponding image:



Both were generated automatically.   A quick cross-check will show you that they match.


All the rest can be downloaded here:

https://dl.dropboxusercontent.com/u/408446/DF/Standard%202.0/Creature_TXTs.zip

You will notice that for the moment the txt file has the "AS_IS" tag, also that the tile sizes are mising.   Now my next step is to open this files and modify them (with the correct AS_IS or ADD_COLOR), as I create the standard images, and we will be ready for our first ingame experiment!

[CLA]

Actually I think I have the solution fro the AS_IS vs ADD_COLOR.  Looking at CLA's set I noticed that all his tiles are pure white.   We can use ADD_COLOR if the tile has only white, or AS_IS if not (we can make the rule more complicated if necessary, for example using histograms of the tiles).

See my earlier comment:

Another thing though: How do we deal with AS_IS vs ADD_COLOR? tell the script which to use? I figure you could let the script check whether the tile only contains white/greyscale or other colors too, but there's always the possibility that someone wants ADD_COLOR even with other colors or AS_IS with no colors (for example those animated tiles, or maybe a polar bear). Replacing all instances of AS_IS with ADD_COLOR is easy enough, but if you have mixed sets, it's gonna get annoying.

Pulling the color out of the raws and coloring the png automatically is problematic too, since it depends on the color scheme.


In the end, I think the most sensible thing to do is to just keep it as ADD_COLOR as default, and have people replace it with AS_IS where they need it. It's the less used alternative anyway.

*note that I meant "keep AS_IS as default", of course.

I really think that graphic sets with ADD_COLOR are a very rare exception, and letting the script guess introduces another layer of complexity with plenty of possible issues whereas just using a text editor and replacing all instances of one for another is trivial for the few artists that want to use ADD_COLOR.

[Quiet-Sun]

Ok Guys,

Here is my first attempt at creating a functional standard graphics set.  As expected, it didn't work    The worse thing is that I'm not sure why.  Could you please look at these sets to see if there are obvious problems to you?

In this file I'm including the CLA and Spacefox sets.   I chose them because one is 16x AS_IS, and the other is 18x ADD_COLOR.  (not to mention that CLA is following this thread so hopefully he/she can find the problems easily)

https://dl.dropboxusercontent.com/u/408446/DF/Graphics%20Standardization_1st_Attempt.zip

If you have some time to check them for problems it would be great!  Thanks a lot!

I haven't touched the /raw/objects/ folders at all, only the /raw/graphics/  Could the problem be there?  other possible culprits are reversing x,y tile coordinates, having typos, messing up the headers 

[burned]

Could you please look at these sets to see if there are obvious problems to you?

Based on the structure and text files I can tell you what I see offhand.

1. The definition text file is named creature_whatever_graphics.txt, but it's defined as creature_whatever.
2. The old definition files are present which means if you correct 1. you will then have everything from Adder to Yak defined twice, each calling different image files since both old and new images are present.
3. If you get rid of the old definition files and old images, you have defined everything that is not in the set as a red tile (This includes the wagon since it's techinically a creature).

Thank you Burned! this would be very helpful. Can you please send me one without outdated professions?  We can assume your list is complete and improve it if need be.

The complete list that I've cobbled together is available right now if you want it (see below spoiler), but they are organized by professions not by used, outdated, etc. I did make some quick notes for your reference if you want to organize them differently.


Notes on notes:

"Not sure" means I feel I've seen them in game, but can't recall at the moment.
"Never verified" means I've actively tried to verify these and have never seen them in game.
I believe CHIEF_MEDICAL_DWARF works for any race, but not all professions listed work for every race.
I've noted the ones that I know are specific to a particular race, but I know there are more than I noted.
Some are defined as one thing, but appear in game as another (see elf queen and princess for example).
Law_Enforce is the fortress guard. Tax_Escort was either turned off with the tax collector or is used as the "Royal Guard" - I'm not sure and it's been awhile since I've had a king arrive.

Verification for creatures is easy, since you can spawn them in the arena. For professions you have to play adventurer or dwarf fortress to verify the tiles.

Wait . . . I'm an idiot.

*sends email*

Maybe Toady could just add professions inside the arena?

In any case, the list is below . . .

Spoiler (click to show/hide)

General
   DEFAULT
   CHILD
   BABY
   ANIMATED <<< replaced skeleton, zombie
   STANDARD <<< similar to default. I forget which one takes precedence or why there are two defaults.
   DRUNK
   CHAMPION
   THIEF
   MASTER_THIEF
   SLAVE  <<< see prisoner
   PRISONER <<< while you can define this tile, as soon as you rescue the prisoner the tile switches to the prisoner's default or profession (default, child, cheese maker, etc). I think this happens for slave as well, but I can't recall.
   SKELETON <<< Not in use
    GHOST
Miners
   MINER
Woodworkers
   WOODWORKER
   BOWYER
   CARPENTER
   WOODCUTTER
StoneWorkers
   STONEWORKER
   ENGRAVER
   MASON
   ARCHITECT
Rangers
   RANGER
   HUNTER
   ANIMAL_CARETAKER
   ANIMAL_DISSECTOR
   ANIMAL_TRAINER
   TRAPPER
Metalsmiths
   METALSMITH
   FURNACE_OPERATOR
   WEAPONSMITH
   ARMORER
   BLACKSMITH
   METALCRAFTER
Jewelers
   JEWELER
   GEM_CUTTER
   GEM_SETTER
Craftsmen
   CRAFTSMAN
   BONE_CARVER
   CLOTHIER
   GLASSMAKER
   LEATHERWORKER
   STONECRAFTER
   WEAVER
   WOODCRAFTER
   STRAND_EXTRACTOR
   GLAZER
   POTTER
   WAX_WORKER
FisheryWorkers
   FISHERY_WORKER
   FISH_CLEANER
   FISH_DISSECTOR
   FISHERMAN
Farmers
   BREWER
   BUTCHER
   CHEESE_MAKER
   COOK
   DYER
   FARMER
   HERBALIST
   LYE_MAKER
   MILKER
   MILLER
   PLANTER
   POTASH_MAKER
   SOAP_MAKER
   TANNER
   THRESHER
   WOOD_BURNER
   SHEARER
   SPINNER
   PRESSER
   BEEKEEPER
   GELDER
Engineers
   MECHANIC
   PUMP_OPERATOR
   ENGINEER
   SIEGE_ENGINEER
   SIEGE_OPERATOR
Administrators and Doctors
   ADMINISTRATOR
   TRADER
   CLERK
   DOCTOR
   DIAGNOSER
   BONE_SETTER
   SUTURER
   SURGEON
   CHIEF_MEDICAL_DWARF
Leaders - Note: For human lords and ladies don't seem possible to tile - I've tried the obvious (lord, lady, lord_consort, lady_consort, etc), but they always default to their former profession.
   MONARCH
   MONARCH_CONSORT
   QUEEN <<< I don't think this actually works. But I don't recall why I removed King, but not Queen.
   BARON
   BARON_CONSORT
   COUNT
   COUNT_CONSORT
   DUKE
   DUKE_CONSORT
   DRUID
   RANGER_CAPTAIN <<< elf
   WARLORD <<< human
   WARRIOR <<< human   
   GENERAL <<< elf - is "Princess" in game.
   LIEUTENANT
   CAPTAIN
   MILITIA_COMMANDER
   MILITIA_CAPTAIN
   ARSENAL_DWARF
   FORCED_ADMINISTRATOR
Other Nobles and Misc Jobs
   TAX_COLLECTOR <<< currently not used
   HAMMERER
   PHILOSOPHER
   ADVISOR
   ALCHEMIST
   DUNGEON_MASTER <<< currently not used
   MAYOR
   LEADER <<< elf - is "Queen" in game.
   EXPEDITION_LEADER
   CAPTAIN_OF_THE_GUARD
   SHERIFF
   MANAGER
   BOOKKEEPER
   BROKER
   HIGH_PRIEST
   PRIEST
   ACOLYTE
Visitors - Inns & Taverns
   POET
   BARD
   DANCER
   PERFORMER
   SAGE
   SCHOLAR
   MATHEMATICIAN
   HISTORIAN
   ASTRONOMER
   NATURALIST
   CHEMIST
   GEOGRAPHER
   SCRIBE
   PAPERMAKER
   BOOKBINDER
   TAVERN_KEEPER
   MONSTER_SLAYER <<< Never verified
   BEAST_HUNTER <<< Never verified
   MERCENARY <<< Never verified
   SCOUT <<< Never verified
Trade Representatives
   MERCHANTPRINCE <<< Not sure
   MERCHANTBARON <<< Not sure
   GUILDREP <<< Not sure
   MERCHANT
   DIPLOMAT
   OUTPOST_LIAISON
   SHOPKEEPER
Military Default, Law_Enforce (fortress guard), Tax_Escort was either turned off with the tax collector or is used as the "Royal Guard" - I'm not sure. And, of course Adventurer. Note: I've never tested the MASTER_ on adventurer. I'm not sure why. Heh. Also, only weapon skills seem to work with adventurer textures (e.g. you can make a Bard, but they'll use the RECRUIT tile).
   RECRUIT
   WRESTLER
   SPEARMAN
   SWORDSMAN
   AXEMAN
   HAMMERMAN
   MACEMAN
   CROSSBOWMAN
   PIKEMAN
   BOWMAN
   LASHER
   BLOWGUNMAN
   MASTER_WRESTLER
   MASTER_SPEARMAN
   MASTER_SWORDSMAN
   MASTER_AXEMAN
   MASTER_HAMMERMAN
   MASTER_MACEMAN
   MASTER_CROSSBOWMAN
   MASTER_PIKEMAN
   MASTER_BOWMAN
   MASTER_LASHER
   MASTER_BLOWGUNMAN

[Quiet-Sun]

Based on the structure and text files I can tell you what I see offhand.

1. The definition text file is named creature_whatever_graphics.txt, but it's defined as creature_whatever.
2. The old definition files are present which means if you correct 1. you will then have everything from Adder to Yak defined twice, each calling different image files since both old and new images are present.
3. If you get rid of the old definition files and old images, you have defined everything that is not in the set as a red tile (This includes the wagon since it's techinically a creature).

Thank you Burned!  I found the problem!  Graphics deffinition txts need to start with the string "graphics_"  so instead of "creature_suberranean_graphics.txt",  I had to use "graphics_creature_subterranean.txt" 

You see some of the old files, but that is because I need to keep beastmen and major races.  I made absolutely sure there were no duplicate entries.   Now I have a bug where dogs look like cows, but that is much easier to fix.  I'll upload the correct files later today, but I'm stoked to see it work!  I think we will have the standard ready very soon!

[burned]

Thank you Burned!  I found the problem!  Graphics deffinition txts need to start with the string "graphics_"  so instead of "creature_suberranean_graphics.txt",  I had to use "graphics_creature_subterranean.txt" 

You're right. I failed to catch that.

You see some of the old files, but that is because I need to keep beastmen and major races.  I made absolutely sure there were no duplicate entries.   

I see. I originally didn't understand why they were mixed; I made the mistake of thinking all the old files were present.
Why not test them without the old files to avoid errors that are not coming from your script?

Now I have a bug where dogs look like cows, but that is much easier to fix. I'll upload the correct files later today, but I'm stoked to see it work!  I think we will have the standard ready very soon!

I don't see the dog/cow error, but it's possible you mean in the files post first attempt.

I look forward to your progress.

[Rydel]

Actually I think I have the solution fro the AS_IS vs ADD_COLOR.  Looking at CLA's set I noticed that all his tiles are pure white.   We can use ADD_COLOR if the tile has only white, or AS_IS if not (we can make the rule more complicated if necessary, for example using histograms of the tiles).

A lot of ADD_COLOR graphics are grayscale rather than pure white.  I know that my Rally Ho tileset uses grayscale ADD_COLOR for the DEFAULT tile, but the rest of the the tiles are colored AS_IS

Horizontally mirroring tiles may have unintended effect for ASCII tilesets - turning b into d, p into q, etc.  This will probably be something that just needs those tiles changed by the artist, but it's something to be aware of during testing so you don't mistake it for the game grabbing the wrong tile.

STANDARD <<< similar to default. I forget which one takes precedence or why there are two defaults.

STANDARD is used when a creature has no profession.  These are the peasants, haulers, etc.
DEFAULT is used when a creature's profession doesn't have a graphic defined.

Something else to note is that a lot of tile definitions have changed rather recently and some older tilesets will use old methods which, ideally, your script should catch and correct.
For example:
[DEFAULT:MyCreature:0:0:AS_IS:ANIMATED] should now be [ANIMATED:MyCreature:0:0:AS_IS:DEFAULT]
[CREATURE:TIGER_MAN] should be [CREATURE:TIGERMAN]
I'm pretty sure Phoebus still defines zombies the old way, and any tile set that hasn't updated in the last month will use the wrong name for tiger men.  For a more complete list for the latter, I'd need to diff the raws for each version

[Quiet-Sun]

Ok Guys!  I think I have the first functional graphics sets with all creatures standardized.  Since I have three artists reading this thread, I figured I would start with theirs (I used the 42.05 versions).   I also did Spacefox 40.24, in case someone can help me look for errors.    Here are the files, please let me know of any problems (note that red tiles are supposed to be there to indicate missing tiles):

42.05:
- CLA
- Rally-Ho

40.24:
- Spacefox

A lot of ADD_COLOR graphics are grayscale rather than pure white.  I know that my Rally Ho tileset uses grayscale ADD_COLOR for the DEFAULT tile, but the rest of the the tiles are colored AS_IS

Horizontally mirroring tiles may have unintended effect for ASCII tilesets - turning b into d, p into q, etc.  This will probably be something that just needs those tiles changed by the artist, but it's something to be aware of during testing so you don't mistake it for the game grabbing the wrong tile.

Thank you and welcome Rydel!  I will then start with an AS_IS and cross check with each artist's txt files to adopt ADD_COLOR as needed.  For something like CLA's set, then it's no problem to do a mass-substitution as he/she suggests.   Regarding mirroring, I intended to verify with each artist (if active) and make a choice if not.   Certainly for ASCII we shouldn't do it.

Something else to note is that a lot of tile definitions have changed rather recently and some older tilesets will use old methods which, ideally, your script will catch and correct.
For example:
[DEFAULT:MyCreature:0:0:AS_IS:ANIMATED] should now be [ANIMATED:MyCreature:0:0:AS_IS:DEFAULT]
[CREATURE:TIGER_MAN] should be [CREATURE:TIGERMAN]
I'm pretty sure Phoebus still defines zombies the old way, and any tile set that hasn't updated in the last month will use the wrong name for tiger men.  For a more complete list for the latter, I'd need to diff the raws for each version

Thank you Rydel. I'm already using an exception file to identify discrepancies between the raws and the graphic set names.  I cross checked the raws with the  Graphics Wiki, and created an exception file that you can download here:

NAME_EXEPTIONS.txt

@Rydel and @burned, you may be interested in knowing that you seem to be misspelling the giant cats (lion, tiger, etc).  At least my script seemed to isolate this.  Check the exeptions in my file and test them in the arena just in case.

Now onwards to the beastmen!