Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[thistleknot]

it sounds like your issue is with a bug in code, and not necessarily with the algorithm.

In fact this is exactly the problem which initially caused me to post.  Since Stakud has a preference for bucklers as well as spears she ends up being shown as equally suitable for all melee roles (since bucklers count for all of them).

It seems like your proposing two things:

fix the code [you think it's just wrong in the way it's logic is setup, not necessarily in an unintended bug]

fix the algorithm [ALGORITHM: spear pref bug]

but trying to relate the algorithm fix to fixing the code?

At least that is how I setup my last reply.

Because I think the algorithm is good/sound and positive for the game.  It only gives "additive" bonus' to dwarf's with multiple preference matches.  The issue with bismuth and bronze is not a problem with the conceptual algorithm, but with the way it is currently implemented aka bug.

I haven't heard from the all mighty one Splinterz on the issue beyond that you were discussing the spear preference; however, these new concerns about removing preferences raises my concern about all the work I just spent the last 3 months on.

"More deeply though it would be problematic (in my opinion at least) for a dwarf with a preference for bismuth bronze to have a higher suitability since in the vast majority of cases they will not be equipped with bismuth bronze."

It was in my prior post I addressed this concern with the ability to decrease group matches... however; to keep things simple, we opted for the matches as described.  Suggestion for match changes are more than welcome.  One can still set weights for their matches

Update

But in most cases I'm not going to have the ability and/or desire to equip them with bismuth bronze so in these cases

It sounds like all these issues your describing can be solved with setting individual preference weights (possibly removing group match preferences altogether, and just opt for single matches)

Your last statement, I do not disagree with in the slightest

...a dwarf with bed and wheelbarrow preferences is better off being a carpenter than a stone hauler (I'm assuming a custom stone hauler role with only the preference for wheelbarrows, which seems credible if not very useful).  This is since the stone hauler will potentially be more happy but the carpenter will also produce enhanced quality beds.  These beds are more valuable, but wait there is more, they also tend enhance room quality, and as a special offer if you read now, they also tend to be admired as fine pieces of furniture more.  So that's two extra ways you can improve the happiness of your fortress. 

I agree with your sentiment, everything you said is accomplished by comparing on a 1 dimensional level.  We accommodated that with the way the rankecdf calculates preferences.

Spoiler (click to show/hide)

I can dig up the algorithm.  I'm kinda in-between computers ATM...

When I say SETUP
I mean, you can set them up
I think the roles are "setup" correctly.   I believe they are properly accommodated by weights.

When I reworked this entire setup from the groundup, I had an understanding of how things worked on a fundamental level from the beginning to now.  Like a git commit history if you will of the algorithm that is used to derive the %.  Of course, Splinterz knows the algorithm as well.

Update 2
BTW, I think the goal of the role% should be like the stock market tries to achieve a price per share point [as in a number that is calculated based on more and more input variables]

I think a setup like that for DT will achieve more accessibility by the game.

[feelotraveller]

I am proposing one thing.  That the option to highlight preferences relevant to roles be added. 

Hopefully this could come with a filter to select only dwarfs which have one or more positive preferences (basically the ability to sort a role by this criteria).

I would prefer that this option also override the preference weightings for roles (i.e. set them all to zero) but that is merely for convenience.

Anything beyond that which I have said are descriptions of how I have seen Dwarf Therapist behaving at the moment or arguments/reasonings for why this option would be a good thing for players to have.

If all the weightings and mechanisms you are talking about are in some development version I can only shrug since I cannot check the validity of those assertions.  If you are talking about the way the program (that is v25.0) works at the moment you are simply wrong.

I am happy for you to disagree with me about the ease and value of producing a numerical value for preferences.  (I only hope that you do not insist that I use said schema.)

[Yaotzin]

How do I make it set the nickname to the profession name when I give them a profession? I seem to remember doing this last time I played DF but I've forgotten everything

[splinterz]

I am proposing one thing.  That the option to highlight preferences relevant to roles be added. 

Hopefully this could come with a filter to select only dwarfs which have one or more positive preferences (basically the ability to sort a role by this criteria).

I would prefer that this option also override the preference weightings for roles (i.e. set them all to zero) but that is merely for convenience.

Anything beyond that which I have said are descriptions of how I have seen Dwarf Therapist behaving at the moment or arguments/reasonings for why this option would be a good thing for players to have.

If all the weightings and mechanisms you are talking about are in some development version I can only shrug since I cannot check the validity of those assertions.  If you are talking about the way the program (that is v25.0) works at the moment you are simply wrong.

I am happy for you to disagree with me about the ease and value of producing a numerical value for preferences.  (I only hope that you do not insist that I use said schema.)

i've had some time to look at this again, and the issue isn't directly caused by the exclusion of zero matches. the cause is more to do with the default weights that are applied to the role preferences.

what happens in the back-end is that all role preference matches are analyzed, and they're assigned a rating. for example, having a single match in your game, was assigned a rating of 0.75. the role's use this rating for the preferences to determine the preference score. however, they also take into account the individual weights that have been assigned to the preference. so as an example of the problematic dwarf's roles:

Speardwarf Role:
  - Spear match, worth 0.75, with a weight of 0.75
  - Shield match, worth 0.75, with a weight of 1.00
Final weighted average: ((0.75*0.75)+(0.75 * 1.0)) / 1.75 = 0.75

Axedwarf Role:
  - Shield match, worth 0.75, with a weight of 1.00
Final weighted average: 0.75 * 1.0 = 0.75

now, adding the zero-matches back in, for this example, would drop the Axedwarf role's preference rating down: ((0.0*0.75)+(0.75 * 1.0)) / 1.75 = 0.43. but as you can see, that's very punishing and was the reason non-matches were excluded from the role preference ratings.

setting the preference weights equally with the current (25.0) method, bumps up shows the problem with the Speardwarf: ((0.75*1.0)+(0.75 * 1.0)) / 2.00 = 0.75. this is more reasonable than punishing the non-matches above.

a big part of the current issue is that the preference weights for the default roles need to be tweaked, especially after the recent changes to roles. for the default roles especially it would probably be a good idea to remove the preference weights entirely, essentially setting them all to 1.0

the highlighting is still an interesting idea though, because it would allow players to exclude all preference ratings from all roles (basically a global weight of 0) but still give an indication of which dwarves may have matching preferences. then, as you've explained, it's up to the player to decide if the preferences is actually applicable on the current fortress.

Edit: thinking about it a bit more, it might be worthwhile to modify the role cell tooltips to show the actual matching preferences...

Edit2: fix failed maths

[feelotraveller]

Edit: thinking about it a bit more, it might be worthwhile to modify the role cell tooltips to show the actual matching preferences...

That would be excellent.   

I'm not sure I understand the calculations for weighted averages (e.g. shouldn't the denominator in the final example be 2.0, leaving the speardwarf still at 0.75) but to be honest I am happy to leave that to the people who do have a head for it.

I started to make a pass through the roles but got quite unsure of myself.  I started with armoursmith/weaponsmith roles but cannot understand why Metals is specified as a preference rather than only the six (or seven for the weaponsmith) applicable weapons-grade metals.  Similarly for the armoursmith I did not understand why say Armour(Chest) was the preference criteria rather an enumeration of the chest armour pieces the smith can actually make (barring strange moods).

So I pulled up.  I can only assume things have been set up this way in order to facilitate the preferences calculations but then I don't get them.

One positive addition I did make before bogging down was to add Cloth as a preference for Wound Dresser.

[thistleknot]

I can only assume things have been set up this way in order to facilitate the preferences calculations but then I don't get them.

[ATM] quite the opposite.

preferences were calculated differently before (only Splinterz knows exactly); and the roles were based on those calculations.

A great reworking of all roles was done, but to allow for the same basic concept we were using before, weights and allowing the [existing] roles to still be used as is.  Hence, the roles need [may need] to be updated.

And no updating of the roles has been done since preferences have been updated.  The preferences were recalculated to work with the existing setup.

In other words, the user is free to recreate their own roles.  I have done so quite extensively.  The default weights are meaningless at the moment, and probably should all be set to 1 unless a user really wishes to change them.

I for one was actually all for removing "group" matches, but "group" matches allow a user to quickly create "roles" without having to enumerate each subtype.

In the end, the roles are not any where near 100% complete.

[feelotraveller]

Thanks for the speedy reply.

Okay, back to the coal-face with you Urist Mcfeelotraveller.

[splinterz]

I'm not sure I understand the calculations for weighted averages (e.g. shouldn't the denominator in the final example be 2.0, leaving the speardwarf still at 0.75) but to be honest I am happy to leave that to the people who do have a head for it.

/sigh absolutely, what a shameful mistake on my part. i've updated the post to use the examples to demonstrate the problem (again, due to the zero-matches being removed, as i'd first figured). so back to square one, preferences still need a fix.

Edit: thinking about it a bit more, it might be worthwhile to modify the role cell tooltips to show the actual matching preferences...

That would be excellent.   

I'm not sure I understand the calculations for weighted averages (e.g. shouldn't the denominator in the final example be 2.0, leaving the speardwarf still at 0.75) but to be honest I am happy to leave that to the people who do have a head for it.

I started to make a pass through the roles but got quite unsure of myself.  I started with armoursmith/weaponsmith roles but cannot understand why Metals is specified as a preference rather than only the six (or seven for the weaponsmith) applicable weapons-grade metals.  Similarly for the armoursmith I did not understand why say Armour(Chest) was the preference criteria rather an enumeration of the chest armour pieces the smith can actually make (barring strange moods).

So I pulled up.  I can only assume things have been set up this way in order to facilitate the preferences calculations but then I don't get them.

One positive addition I did make before bogging down was to add Cloth as a preference for Wound Dresser.

for the armor stuff especially, it was to aid in the previous problem where having lots of preferences pulled the rating way down. so only have one preference, even though it was a group, helped out significantly.

the other reason a lot of the roles are setup with general/group preferences was to support mods which may have many more types of armor, items, materials, etc. it's now possible for a mod to package their own set of roles though, so it's less of a concern if preferences are 'hard-coded' to only vanilla materials/items.

[feelotraveller]

A highlight box would fix it all up, at least for my purposes. 

Now for some more feedback since I've been going through the roles, albeit at the pace of a snail.  And please don't shoot me the goal is to make Dwarf Therapist even better...

The general categories (at least for an unmodded DF after reading the above) of Stone, Metal and Gems are at best useless since each dwarf has a preference for one item in each of these categories (=automatic match for the category).  (I leave aside the rare case where the 'stone' selection is not stone but porcelain or the like, but honestly this is infrequent enough to be ignorable.)  In fact these every-dwarf matches might not be helping with cross-role balance... and they certainly tend to mask relevant preferences.

More problematically, for me at least, after a small amount of investigation it seems that most of the food categories/items point to a dwarfs preference for consuming that thing.  That is all the entries under General Food, as well as the Seeds category further down the list.  The Plants and Plants (Alchohol) categories outside of the general food listing (i.e. the ones down the list) do seem to accurately point to liking those plants rather than preferring to consume them.

Although there is value in having consumption preferences for other purposes they actually get in the way of good assignation of labours, where it is only likes rather than prefers to consume that makes a difference.

If the information on dfwiki is comprehensive the only food type preference a dwarf can have - as opposed to the preference to consume - is 'a type of crop' which I'm pretty sure means any farmable/gatherable plant.  So that would be covered by the two categories down the list?

Perhaps, assuming I haven't erred somewhere, deleting the problematic categories would prevent future confusion?

[splinterz]

i had no idea that every dwarf has a preference for a metal, gem and stone. is that a recent change with 40.x? i did a quick test on a fort of ~100 dwarves and indeed it seems to be true. in that case, i'll certainly remove the general categories as they serve no purpose. i may be able to separate specific materials like porcelain, stoneware, glazes, etc. that show up as stone, into their own categories. Edit: this is already done under 'glazes & stoneware'

the general food->alcohols category can probably also be removed since every dwarf has a preference for some type of booze (like stone/gems/metal). it makes sense to remove the consumable foods as you've suggested. the sole purpose of the role preferences is to aid in finding a dwarf for a job, and consumable preferences are useless in that sense.

sounds like a good set of changes to slim down the list of role preferences. keep them coming!

[feelotraveller]

It is probably worth leaving the possibility to specify certain types of stone.  E.g. for a custom stonecrafter role chewing through some excess economic stone, or whatever

[splinterz]

It is probably worth leaving the possibility to specify certain types of stone.  E.g. for a custom stonecrafter role chewing through some excess economic stone, or whatever

you mean to have general categories for economic, non-economic stone?

[thistleknot]

feelotraveller:

I finally understand the issue.


Final weighted average:

ex...

Code: [Select]

foreach value: score = score + ((value* weight)/sumofWeights)
((0.75*0.75)+(0.75 * 1.0)) / 1.75 = 0.75

[part of] the problem is... We're doing weighted averages...

we should be doing a weighted sum.

ex.

Code: [Select]

foreach value: value = value + ((value * weight))I guess I should say,

Code: [Select]

foreach value; prerankecdfscore  = prerankecdfscore  + ((value * weight))
then the matrix of values prerankecdfscore is ran through a rankecdf to get a 0 to 100% (really 50% to 100% for preferences)

Matrix Function

Code: [Select]

score = rankecdf(prerankecdfscore)
The other part is the weights should be applied to the raw match values[maybe no], then a rankecdf is ran over the weighted data.

hence the algorithm [is the issue, as you have said ].

When we divide by the sum of weights, we're basically saying divide by n elements, but instead of elements, we use weights to represent our "n".

Doing so means if we have multiple successive matches, we end up dividing by n basically.

Not a good thing.

Since we only count successive matches, we get a basic ~binary additive number.

Ex.

1, 1.1, 1.2

However, the .75 Splinterz listed above is what rankecdf converts a value of 1 to.  1.1 would probably be converted to .80.

Continuing on...

So, say we have these "matches" that we evaluate as 1, 1.1, 1.2, etc... within preferences.

I'm thinking,

we can take each [raw] value and multiply it by it's weight.

vs taking it's rankecdf%(value) * weight. but...

we differ by not taking a weighted avg (i.e. dividing by sumofweights).

This additive number is ~equivalent to n number of matches.  We just don't wish to divide by n.

We differ again by implementing rankecdf after, converting these values to scores of 0 to 100% (50% to 100% in preferences case).

Update
So the above example would become (.75 represented 1 match after rankecdf was perormed):

Code: [Select]

((1*0.75)+(1 * 1.0))

1.75

Comparative to a single match of just (1 * weight of 1)...

compared to someone who

[khearn]

the general food->alcohols category can probably also be removed since every dwarf has a preference for some type of booze (like stone/gems/metal). it makes sense to remove the consumable foods as you've suggested. the sole purpose of the role preferences is to aid in finding a dwarf for a job, and consumable preferences are useless in that sense.

Hmmm, I just had an idea... It's not directly related to Therapist, though. What if you pick your miners based on what type of booze they prefer, and then set up booze stockpiles near the mines with only that kind of booze? And pick your metalworkers with a different sort of booze and stock that booze near the forges, etc.

The problem is that there is a very wide variety of booze preferences (they seem to pick randomly from every booze type that exists in the world), but most forts only have a limited selection. Heck most of my forts have lots and lots of dwarven wine, a decent supply of dwarven beer, ale, and rum, and a small stock of whatever the humans brought last summer (or whatever you can make with the fruit the elves or dwarves bought). Looking at a handy fort with 39 dwarves, only one of them has a preference for one of the "dwarven" drinks, and no drink of any sort has more than 2 dwarves that prefer it.

Ok, so that's not really a useful idea. Back to the on-topic discussion. Sorry for this pointless sidebar.

   Keith

[feelotraveller]

I've dabbled with forcing dwarfs to consume with nearby stockpiles and never really got it to work, by which I mean that it caused as much or more stuffing around with hauling things to and fro as the benefit produced in terms of happiness.  Often when dwarfs drink they also eat (or sleep) which takes them away from their work areas.  It could probably be done with burrows but it is still of questionable overall value.  I've found it a better approach to try to maintain a variety of drinks at any alcohol stockpile - dwarfs do a pretty good job of self-service. I think that they can also get fatigued with just drinking one kind of alcohol?

However alcohol consumption preferences (and the others) are handy to have sometimes for planning.  I find the preferences dock ideal for this.  If I have 5 dwarfs who like strawberry wine and none who like prickleberry wine then that information can be used to guide my crop plantings or my trading priorities.  It is just that consumption preferences are useless (as far as I can tell...) for defining roles.