Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

DF Duplisearch

2 (40%)

DF Creatureclasser

2 (40%)

DF CVunpack

0 (0%)

DF Creaturescale

0 (0%)

DF Biomeviewer

1 (20%)

DF Rawminer

0 (0%)

Total Members Voted: 3

[voliol]

DF Diagnosipack is a collection of modding tools for Dwarf Fortress, with a focus on diagnosing problems and getting statistics on what is found within the raws.

Tools included:

Spoiler: DF Duplisearch (click to show/hide)

DF Duplisearch checks the raws for duplicated objects. It also counts the instances of each object type (CREATURE, ENTITY, etc. etc.). Useful for merging mods, or visualizing the size of your mods compared to vanilla (and other mods).

Spoiler: DF Creatureclasser (click to show/hide)

DF Creatureclasser tell you what creature classes there are in your raws, and what they are used for. Useful for seeing which creature classes are underused, and perhaps when merging mods.
Creature classes are arbitrary tags given to creatures. They can be used by other objects to not have to specify each single creature of a large group, such as "mammals". In DF 0.47, the following 10 tokens use creature classes:

• Creature tokens -
  CREATURE_CLASS
  GOBBLE_VERMIN_CLASS
  SENSE_CREATURE_CLASS
• Interaction tokens -
  IT_AFFECTED_CLASS
  IT_IMMUNE_CLASS
• Entity tokens -
  ANIMAL_CLASS
  ANIMAL_FORBIDDEN_CLASS
  ALLOWED_CLASS
• Syndrome tokens -
  SYN_AFFECTED_CLASS
  SYN_IMMUNE_CLASS
  CE_SENSE_CREATURE_CLASS

Spoiler: DF CVunpack (click to show/hide)

DF CVunpack unpacks creature variations, so you can see how the raws for e.g. wolf men really look (spoiler: they have 7 fingers).

Spoiler: DF Creaturescale (click to show/hide)

A scale in the sense that it weighs creatures. If you allow "very detailed weights" it also outputs their tissue composition, and weights of specific body parts. This is the most complex tool in this pack, but possibly the most inaccurate as well; I do not know much about how weight is actually calculated in-game.

Spoiler: DF Biomeviewer (click to show/hide)

DF Biomeviewer allows you to view what creatures/vermin/plants can appear in each of the nearly 300 biomes (counting GOOD/EVIL/SAVAGE alignment, and cavern layers). With two viewmodes and the ability to print out the tables using Mediawiki formatting, it should be useful to both modders and mod users. Admittedly a bit laggy.

Spoiler: DF Rawminer (click to show/hide)

DF Rawminer looks through uncompressed save files and outputs ("mines") the procedurally generated raws therein, including (but not limited to) those of Forgotten Beasts, evil weather, and instruments. Spoileriffic.

Spoiler: Screenshots (click to show/hide)

DF Duplisearch:

DF Creatureclasser:

DF CVunpack:

DF Creaturescale:


DF Biomeviewer:


DF Rawminer:

There's an .exe version, and a version for Python 3. Use whichever you like. The .exe version has no dependencies, while the Python version requires Python 3.
As the .exe version reads and writes (text) files, some malware detectors may mistake it for malware.

Download .exe version (DFFD)
Download Python version (DFFD)
The tools were made (and tested) with DF 0.47 in mind, but most if not all of them should work with older versions of DF as well.

There's also a github repo: https://github.com/voliol/DF-Diagnosipack

Spoiler: Legacy download links (DFFD) (click to show/hide)

DF duplisearch 1.2.1:
For Python 3
For Python 2.7
Binary/.exe version
DF creatureclasser 1.0:
For Python 3

All these work for DF 0.44 and 0.47, and likely most earlier versions as well.

[jecowa]

I just tried it in Python 2.7.

Spoiler (click to show/hide)

$python df_duplisearch.py
{'ITEM_FOOD': 0, 'ITEM_SHIELD': 0, 'SYMBOL': 0, 'ITEM_TRAPCOMP': 0, 'ENTITY': 0, 'ITEM_PANTS': 0, 'TISSUE_TEMPLATE': 0, 'REACTION': 0, 'WORD': 0, 'BUILDING_WORKSHOP': 0, 'COLOR': 0, 'ITEM_TOOL': 0, 'CREATURE_VARIATION': 0, 'SHAPE': 0, 'ITEM_AMMO': 0, 'ITEM_GLOVES': 0, 'TRANSLATION': 0, 'BODY': 0, 'INTERACTION': 0, 'ITEM_ARMOR': 0, 'ITEM_TOY': 0, 'COLOR_PATTERN': 0, 'MATERIAL_TEMPLATE': 0, 'ITEM_HELM': 0, 'BODY_DETAIL_PLAN': 0, 'PLANT': 0, 'ITEM_WEAPON': 0, 'INORGANIC': 0, 'BODYGLOSS': 0, 'CREATURE': 0}
('reading file 1/78', 'inorganic_other.txt')
Traceback (most recent call last):
  File "df_duplisearch.py", line 67, in <module>
    rawfile = open(folder_path + rawfilenames, "r", encoding="latin-1")
TypeError: 'encoding' is an invalid keyword argument for this function

I deleted the "encoding" part and tried again:

Spoiler (click to show/hide)

$python df_duplisearch.py
{'ITEM_FOOD': 0, 'ITEM_SHIELD': 0, 'SYMBOL': 0, 'ITEM_TRAPCOMP': 0, 'ENTITY': 0, 'ITEM_PANTS': 0, 'TISSUE_TEMPLATE': 0, 'REACTION': 0, 'WORD': 0, 'BUILDING_WORKSHOP': 0, 'COLOR': 0, 'ITEM_TOOL': 0, 'CREATURE_VARIATION': 0, 'SHAPE': 0, 'ITEM_AMMO': 0, 'ITEM_GLOVES': 0, 'TRANSLATION': 0, 'BODY': 0, 'INTERACTION': 0, 'ITEM_ARMOR': 0, 'ITEM_TOY': 0, 'COLOR_PATTERN': 0, 'MATERIAL_TEMPLATE': 0, 'ITEM_HELM': 0, 'BODY_DETAIL_PLAN': 0, 'PLANT': 0, 'ITEM_WEAPON': 0, 'INORGANIC': 0, 'BODYGLOSS': 0, 'CREATURE': 0}
('reading file 1/78', 'inorganic_other.txt')
(8, 'objects found.')
('reading file 2/78', 'plant_grasses.txt')
(48, 'objects found.')
('reading file 3/78', 'creature_desert_new.txt')
(57, 'objects found.')
('reading file 4/78', 'descriptor_pattern_iris_eye.txt')
(168, 'objects found.')
('reading file 5/78', 'creature_mountain_new.txt')
(174, 'objects found.')
('reading file 6/78', 'body_default.txt')
(312, 'objects found.')
('reading file 7/78', 'item_toy.txt')
(317, 'objects found.')
('reading file 8/78', 'tissue_template_default.txt')
(355, 'objects found.')
('reading file 9/78', 'creature_insects.txt')
(376, 'objects found.')
('reading file 10/78', 'creature_small_riverlake.txt')
(387, 'objects found.')
('reading file 11/78', 'plant_standard.txt')
(439, 'objects found.')
('reading file 12/78', 'creature_riverlakepool_new.txt')
(462, 'objects found.')
('reading file 13/78', 'creature_small_ocean.txt')
(486, 'objects found.')
('reading file 14/78', 'interaction_standard.txt')
(491, 'objects found.')
('reading file 15/78', 'creature_temperate_new.txt')
(554, 'objects found.')
('reading file 16/78', 'creature_other.txt')
(563, 'objects found.')
('reading file 17/78', 'language_ELF.txt')
(564, 'objects found.')
('reading file 18/78', 'item_trapcomp.txt')
(569, 'objects found.')
('reading file 19/78', 'item_shoes.txt')
(569, 'objects found.')
('reading file 20/78', 'building_custom.txt')
(571, 'objects found.')
('reading file 21/78', 'item_shield.txt')
(573, 'objects found.')
('reading file 22/78', 'creature_next_underground.txt')
(605, 'objects found.')
('reading file 23/78', 'creature_ocean_new.txt')
(638, 'objects found.')
('reading file 24/78', 'creature_bug_slug_new.txt')
(681, 'objects found.')
('reading file 25/78', 'plant_garden.txt')
(738, 'objects found.')
('reading file 26/78', 'descriptor_pattern_pupil_eye.txt')
(849, 'objects found.')
('reading file 27/78', 'creature_large_ocean.txt')
(889, 'objects found.')
('reading file 28/78', 'c_variation_default.txt')
(921, 'objects found.')
('reading file 29/78', 'language_SYM.txt')
(1004, 'objects found.')
('reading file 30/78', 'item_food.txt')
(1007, 'objects found.')
('reading file 31/78', 'entity_default.txt')
(1013, 'objects found.')
('reading file 32/78', 'item_pants.txt')
(1022, 'objects found.')
('reading file 33/78', 'creature_small_mammals.txt')
(1033, 'objects found.')
('reading file 34/78', 'creature_equipment.txt')
(1034, 'objects found.')
('reading file 35/78', 'inorganic_stone_layer.txt')
(1059, 'objects found.')
('reading file 36/78', 'language_DWARF.txt')
(1060, 'objects found.')
('reading file 37/78', 'creature_large_tundra.txt')
(1069, 'objects found.')
('reading file 38/78', 'inorganic_stone_gem.txt')
(1196, 'objects found.')
('reading file 39/78', 'creature_birds.txt')
(1225, 'objects found.')
('reading file 40/78', 'descriptor_color_standard.txt')
(1340, 'objects found.')
('reading file 41/78', 'creature_standard.txt')
(1373, 'objects found.')
('reading file 42/78', 'item_helm.txt')
(1381, 'objects found.')
('reading file 43/78', 'plant_crops.txt')
(1415, 'objects found.')
('reading file 44/78', 'item_weapon.txt')
(1439, 'objects found.')
('reading file 45/78', 'creature_domestic.txt')
(1461, 'objects found.')
('reading file 46/78', 'plant_new_trees.txt')
(1502, 'objects found.')
('reading file 47/78', 'item_ammo.txt')
(1505, 'objects found.')
('reading file 48/78', 'creature_tundra_taiga_new.txt')
(1511, 'objects found.')
('reading file 49/78', 'language_GOBLIN.txt')
(1512, 'objects found.')
('reading file 50/78', 'material_template_default.txt')
(1582, 'objects found.')
('reading file 51/78', 'creature_large_riverlake.txt')
(1592, 'objects found.')
('reading file 52/78', 'reaction_other.txt')
(1638, 'objects found.')
('reading file 53/78', 'creature_large_tropical.txt')
(1705, 'objects found.')
('reading file 54/78', 'creature_annelids.txt')
(1707, 'objects found.')
('reading file 55/78', 'inorganic_stone_mineral.txt')
(1765, 'objects found.')
('reading file 56/78', 'creature_small_mammal_new.txt')
(1774, 'objects found.')
('reading file 57/78', 'inorganic_metal.txt')
(1800, 'objects found.')
('reading file 58/78', 'creature_amphibians.txt')
(1803, 'objects found.')
('reading file 59/78', 'creature_reptiles.txt')
(1818, 'objects found.')
('reading file 60/78', 'language_HUMAN.txt')
(1819, 'objects found.')
('reading file 61/78', 'creature_tropical_new.txt')
(1879, 'objects found.')
('reading file 62/78', 'item_armor.txt')
(1891, 'objects found.')
('reading file 63/78', 'item_tool.txt')
(1921, 'objects found.')
('reading file 64/78', 'creature_birds_new.txt')
(2005, 'objects found.')
('reading file 65/78', 'creature_large_temperate.txt')
(2053, 'objects found.')
('reading file 66/78', 'creature_large_mountain.txt')
(2061, 'objects found.')
('reading file 67/78', 'body_rcp.txt')
(2150, 'objects found.')
('reading file 68/78', 'creature_subterranean.txt')
(2183, 'objects found.')
('reading file 69/78', 'b_detail_plan_default.txt')
(2204, 'objects found.')
('reading file 70/78', 'creature_fanciful.txt')
(2207, 'objects found.')
('reading file 71/78', 'item_siegeammo.txt')
(2207, 'objects found.')
('reading file 72/78', 'language_words.txt')
(4403, 'objects found.')
('reading file 73/78', 'descriptor_pattern_special.txt')
(4421, 'objects found.')
('reading file 74/78', 'descriptor_shape_standard.txt')
(4464, 'objects found.')
('reading file 75/78', 'item_gloves.txt')
(4467, 'objects found.')
('reading file 76/78', 'inorganic_stone_soil.txt')
(4488, 'objects found.')
('reading file 77/78', 'reaction_smelter.txt')
(4511, 'objects found.')
('reading file 78/78', 'reaction_adv_carpenter.txt')
(4533, 'objects found.')
('ITEM_FOOD', 3)
('ITEM_SHIELD', 2)
('SYMBOL', 83)
('ITEM_TRAPCOMP', 5)
('ENTITY', 6)
('ITEM_PANTS', 9)
('TISSUE_TEMPLATE', 38)
('REACTION', 91)
('WORD', 2196)
('BUILDING_WORKSHOP', 2)
('COLOR', 115)
('ITEM_TOOL', 30)
('CREATURE_VARIATION', 32)
('SHAPE', 43)
('ITEM_AMMO', 3)
('ITEM_GLOVES', 3)
('TRANSLATION', 4)
('BODY', 214)
('INTERACTION', 5)
('ITEM_ARMOR', 12)
('ITEM_TOY', 5)
('COLOR_PATTERN', 240)
('MATERIAL_TEMPLATE', 70)
('ITEM_HELM',
('BODY_DETAIL_PLAN', 21)
('PLANT', 224)
('ITEM_WEAPON', 24)
('INORGANIC', 265)
('BODYGLOSS', 13)
('CREATURE', 767)
(4533, 'objects found in total.')
('Done.', 0, 'duplicated raw entries found.')

Is there anything bad about deleting the encoding="latin-1" bit?

[voliol]

Is there anything bad about deleting the encoding="latin-1" bit?

No there isn't. I originally wrote this script in Python 2.7, and then ported it to 3. I'm surprised as little has to be done to make it Python 2-compatible.

Python 2.7 actually supposes all read .txt files are in latin-1. This works for DF raws, but can cause corruption in other cases, so they changed it in Python 3. The "encoding='latin-1'" is telling it to use latin-1, despite being in Python 3.

[voliol]

Posted the Python 2.7 version as well, because why not? I should be able to make a binary version as well but I dunno about the timeframe on that.

[jecowa]

Thanks. Just used this to problem solve some Duplicate Object errors.

[voliol]

Np. Here's the first binary version. I'm not too used to compiling my python tbh so hopefully it works more often than not.

[Enemy post]

Thanks for doing this, I've been wanting something like this for a long time. This should make modding much more efficient.

[voliol]

I’m glad you like it ^^

I just hope this doesn’t lessen the use of prefixes/subfixes, because those are really a more longterm solution and ultimately better than this. That might be my hybris talking though :p

[voliol]

Now it states which .txt files has the duplicated raws in them, whenever any are found.
Also, it now has a proper name!

[voliol]

A new version is up. Added another feature inspired by overlapping mods, but that doesn't strictly have to do with duplication errors.

"name_mode" can be toggled on/off depending if you want to show "duplicated names". These are the in-game names (e.g "tiger:tigers:tiger" from "CREATURE:TIGER"), and so they don't cause duplication errors, but can still be confusing to the player. Note that some vanilla objects are recognized as "duplicated", i.e. skirt, short skirt, long skirt.

[voliol]

I'm reusing this thread for the spiritual successor of this tool, which I've been working on the last few weeks.
DF Diagnosipack is a collection of modding tools for Dwarf Fortress, with a focus on diagnosing problems and getting statistics on what is found within the raws. In other words, it's DF Duplisearch + DF Creatureclasser + two more tools, and a more proper base that not only should make developing more tools easier, but allows the "old" tools to handle creature variations. Oh, and a snazzy UI.

[Hetsin]

Hi, I just downloaded the .exe version. I'm getting a "double_print() missing 1 required positional argument: 'use_output_file'" error, when running some of the utilities.

Spoiler (click to show/hide)

Code: [Select]

Error using DF CVunpack 2021-07-19 04:48 :
Traceback (most recent call last):
  File "df_diagnosipack.py", line 152, in cvunpack_button_command
  File "df_cvunpack.py", line 39, in cvunpack_main
  File "df_diagnosipack_base.py", line 213, in unpack_creature_variations
  File "df_diagnosipack_base.py", line 1324, in apply_creature_variation
TypeError: double_print() missing 1 required positional argument: 'use_output_file'

Error using DF Duplisearch 2021-07-19 04:48 :
Traceback (most recent call last):
  File "df_diagnosipack.py", line 101, in duplisearch_button_command
  File "df_duplisearch.py", line 103, in duplisearch_main
  File "df_diagnosipack_base.py", line 213, in unpack_creature_variations
  File "df_diagnosipack_base.py", line 1324, in apply_creature_variation
TypeError: double_print() missing 1 required positional argument: 'use_output_file'

Is there something I overlooked when / before running the tools?

[voliol]

Nope, that was a mistake on my part. Thanks for telling me.

It’s related to an ”warning message” I put there to warn about an untested feature.
So: what mods are you using? What you encountered was the ”warning message” choking up, and that has already been fixed (and that version is now up), but the the untested feature is still untested, because vanilla didn’t use it, and I knew of no mods that did. Now is as good an opportunity as any to fix that, though, if you name the mods that could’ve caused it

[Hetsin]

Funny story about that. I've been working on a sprawling custom mod and while it runs on 32 bit DF, it does not run on 64 bit DF. I highly suspect it's the vast amount of CVs used (and specifically the dll function that does memory copies), so I was hoping that I could use the utility to 'flatten' the raws to verify that it's the case.

If you would like me to upload the current state of the mod, let me know. Thanks. 

[voliol]

The creature variant handling is what still needs work, so if you could send me the cvs and the creatures that use them, then I’d be very pleased