Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[splinterz]

earlier today i altered the code to do as you proposed dreiche2, and it does seem to balance aspects out better. i just implemented the simplest version of what we talked about by specifying static absolute ranges to compare each aspect to for each dwarf, and then at the end adjusted all the final ratings via normal distribution. i haven't had enough time to play with it to see if it's resolving all the skill / weight issues, but it seems to so far.

it would be nice to incorporate something that gives a bit more value the farther from the average, but the only way this will work is if there's a way to transform or handle the massive skewing that skills have. we'll see...

[thistleknot]

thanks for the lognormal tip dreiche2

info of which can be found here.
  http://www.itl.nist.gov/div898/handbook/eda/section3/eda3669.htm

i wonder if I can use the info in the prior excel post on getting the shape... if so.  Then I think I can get a working formula up.

[dreiche2]

thistleknot:

For your future tests, just keep in mind that this issue wasn't about whether to ultimately compare the score to that of the other dwarves -- this was about whether computing the score initially from the relative values makes sense.

Let me give you an example. Let's say you want to decide who's a good dwarf for a job based on an equal weighting of their strength and agility. Let's say dwarf Urist here has agility of 10.1 (never mind the values DF actually uses) and a strength of 5. Let's say for simplicity that there are 100 other dwarves; in terms of agility, say all of them have agility 10. In terms of strength, let's say half of them have strength 5 and half have strength 15 [or something like that with larger stdev -- doesn't matter].

For agility across the population (including Urist), that gives me a stdev of about 0.01. For strength, a stdev of 5. If I normalise the values by dividing by their respective stdevs, this will essentially give a 5./0.01 = approx 500 times higher effective weight to the difference in agility compared to that in strength. I.e., if I'm not mistaken, your formula essentially assumes that the situation above is equivalent to one that looks like this (normalising by subtracting means and dividing by stdevs):

Urist has agility of about 10, while everyone else has one of -0.1.
Urist has strength of -1., while half of the population also has -1., and the other half +1.

Hence, as far as I can see in that case your formula would recommend us Urist over any other dwarf (in normalised terms, he's one stdev (=1.) below average in strength, but a whopping 10 stdevs above average in agility). In particular, it would prefer Urist at agility 10.1 and strength 5. over one of the dwarves with agility 10. and strength 15 (!) -- just because Urist happened to be 'so much better' in agility when measured in population stdevs.

Hope I didn't make a mistake somewhere, but if not, then that's maybe something to think about (or otherwise, to correct me on!).

Edit: typo etc.

[osmo]

I really like the current direction in relaying attribute information and matching dwarves to roles in Dwarf Therapist. It looks to make it an even more useful (even indispensable) tool. I would like to contribute a few points.

The first few are about presentation. The way I look at it, Dwarf Therapist presents data to inform player decisions, then facilitates the implementation of these. The latter aspect is achieved through a skills-oriented tab, by activating or deactivating individual labours. Now, in the branch splinterz and others have been working on, there is an added roles tab. Most, though not all, of these roles map to a labour just like the skills presented on the standard labour tab. To me it would thus make sense, and enhance the usefulness of the tool, to replace the 'roles' indicator by a duplicate of the first tab, with the same capabilities in terms of directly activating and deactivating labours, just with a different set of data being showcased, whatever you eventually decide it to be. This way you could easily implement your choices, just based on different information. As a bonus, you could optionally present the second data set on either tab through a colour code, like a temperature scale. Say, tab 1 indicates skill and tab 2 a weighted 'fitness' score. Then tab 1 could show the familiar squares and diamonds, but colour-coded on a temperature scale from cold purple to hot red. This would make all the information accessible on both tabs, with a different focus. In the long run you could even have a chooser menu allowing you to customize your tabs to present any two datasets relating to individual dwarves. So your custom tab could, for example, offer two differently weighted fitness functions indicated by square size and colour, respectively.
I've excluded roles that aren't directly tied to labours in this. Those are military functions and those appointed through the nobles screen. You could tack them on at the end of the labours grid, but without the option to activate the jobs. But a separate, special roles tab seems preferable. The selection of relevant information offered would be largely identical to the labour-tied roles.
This takes me to another consideration - what information should one focus on in the first place for any second tab? Here, I think the enhanced Therapist can help with two key questions. Which dwarves are good at a job, and which dwarves have great potential for a job. Which ones can I make good use of right now (and maybe should relieve from other duties like hauling) and which ones could be trained to be the legends of the future? I think it would be great if the two tabs could focus on these individual aspects. My impression is that the former is chiefly a function of skill, whereas the latter largely depends on attributes and maybe traits. So, as a first approximation, having two tabs showcase those properties looks like a good plan to me. That is, I would probably want to disentangle skills from the mix shown on the second one. If my suggestion about colour-coding in the respective other dataset were to be implemented, switching back and forth shouldn't really be necessary anyway to confirm your choices.
Finally, some thoughts on the details of the data. I would generally advocate to try to keep matters simple and do as little as possible to the data as offered up by the game. That is, avoid weighing data, taking means into consideration and so forth. If you do, however, wish to improve things I could think of two very good metrics for the two key features I mentioned above, capability and potential. The first would be the rate at which a dwarf can do a job. The second would be a rate at which a dwarf gains experience whilst doing a job. Both can be measured in-game. As exact attribute and skill values at any one point can also be determined, it should be feasible to work out their exact relationships by relating those properties and then to model them. All it takes is a great measuring exercise. A rather laborious task, mind, but a possible one.

I'll offer a few TLDR takeaways
-replace roles tab with a second labour tab with the same capabilities in terms of assignments, just different data shown
-integrate information shown in tab 2 into tab 1 through colour coding, and vice versa.
-have separate tab for special roles that cannot be assigned as labours.
-create customizable tabs allowing for choice of data source and form of presentation
-have labour tab 1 generally focus on actual capability, tab 2 on potential. I.e. tab 1 skills, tab 2 attributes as a first guess.
-don't convolute, average, apply statistics or w.e. unless it clearly improves the quality of data presented to the user in informing his choices.
-determine the exact relationship of job speed/ rate of experience gain and skills/attributes/traits through measurement. Then make these metrics available in Dwarf Therapist

[thistleknot]

I see where your going with this.  First, stdevs aren't that disparate, but ur saying stdev differences in sizes end up multiplying the results against each other.  I'm not entirely sure that is true cuz a new stdev is combined from the old ones... but I'll have to check ur claim out.  According to your example, str has a sdev of 5 and agil .01.  Any deviation from mean of 5 /.001 would and should rightfully give a 500 weight... cuz its 500 deviations from .01.  The examples provided were kind of screwy, but if you meant divided by .01 you were prolly referring to an agil score of around 15 compared to a mean of 10 w a sdev of .01

Update:
Basically, a sdev of .01 means that ~90% of the values are within .04 (or -2/+2 sdev's), so when you have 5 compared to .01, then you are talking about a x500 ratio!  That's all a z-score is, a ratio to a stdev, a 1 sdev represents ~34%, -1 to 1 sdev's represents about 66%, -2 to 2 represents ~90%.  So when you are 500 sdev's away, you should be weighted appropriately.

Update 2:
Yeah, okay, I see what your talking about.

Urist has agility of about 10, while everyone else has one of -0.1.
Urist has strength of -1., while half of the population also has -1., and the other half +1.

Hence, as far as I can see in that case your formula would recommend us Urist over any other dwarf (in normalised terms, he's one stdev (=1.) below average in strength, but a whopping 10 stdevs above average in agility)

And yes, this is exactly what I was going for.  Why?  Because he was far far out there on a stat that other dwarve's were not, and this is true for ALL STAT'S that the dwarf is far out on.  So it's equally applied to all stats.  I wanted to weight the distance from the mean at a (dare I say exponential) rate, which also applies to values below the average as well.

Which is why I think my formula was working, or should have worked regardless of skewed skill distribution.  As Splinterz had pointed out before, that the mean was still the mean of skills, i.e. the mean average value, so half the values were below the mean, and half above, so they would be rated exponentially the further they were away from this, but... I never thought a normal distribution should have been applied, but I didn't know how to do a skewed, I do now, but I'm not 100% sure on how to do a weighted deviation from mean with a skewed distribution, but... your log-normal distribution is showing promise as the log of values is supposedly a normal distribution, so I have some tests to run...

thistleknot:

For your future tests, just keep in mind that this issue wasn't about whether to ultimately compare the score to that of the other dwarves -- this was about whether computing the score initially from the relative values makes sense.

Let me give you an example. Let's say you want to decide who's a good dwarf for a job based on an equal weighting of their strength and agility. Let's say dwarf Urist here has agility of 10.1 (never mind the values DF actually uses) and a strength of 5. Let's say for simplicity that there are 100 other dwarves; in terms of agility, say all of them have agility 10. In terms of strength, let's say half of them have strength 5 and half have strength 15 [or something like that with larger stdev -- doesn't matter].

For agility across the population (including Urist), that gives me a stdev of about 0.01. For strength, a stdev of 5. If I normalise the values by dividing by their respective stdevs, this will essentially give a 5./0.01 = approx 500 times higher effective weight to the difference in agility compared to that in strength. I.e., if I'm not mistaken, your formula essentially assumes that the situation above is equivalent to one that looks like this (normalising by subtracting means and dividing by stdevs):

Urist has agility of about 10, while everyone else has one of -0.1.
Urist has strength of -1., while half of the population also has -1., and the other half +1.

Hence, as far as I can see in that case your formula would recommend us Urist over any other dwarf (in normalised terms, he's one stdev (=1.) below average in strength, but a whopping 10 stdevs above average in agility). In particular, it would prefer Urist at agility 10.1 and strength 5. over one of the dwarves with agility 10. and strength 15 (!) -- just because Urist happened to be 'so much better' in agility when measured in population stdevs.

Hope I didn't make a mistake somewhere, but if not, then that's maybe something to think about (or otherwise, to correct me on!).

Edit: typo etc.

[Chagen46]

I've unzipped the .rar with both Winrar and Zipeg and both of them cannot extract QtNetwork4.dll from the .rar, making the whole program useless.

Is there any particular reason why?

[Weazal]

Is there a listing of all the different user releases for Therapist?

Since mainly I seem to be using Splinterz releases, for simplicity's sake I'm looking for a spot to check for a latest release pushed by him if there is one, rather than browsing through the thread each time to check for the same. If there's not I'm fine with browsing as necessary. Just thought I might inquire and see if there is listing of sorts


Thanks for your hard work guys.

[thistleknot]

Finally, some thoughts on the details of the data. I would generally advocate to try to keep matters simple and do as little as possible to the data as offered up by the game. That is, avoid weighing data, taking means into consideration and so forth. If you do, however, wish to improve things I could think of two very good metrics for the two key features I mentioned above, capability and potential. The first would be the rate at which a dwarf can do a job. The second would be a rate at which a dwarf gains experience whilst doing a job. Both can be measured in-game. As exact attribute and skill values at any one point can also be determined, it should be feasible to work out their exact relationships by relating those properties and then to model them. All it takes is a great measuring exercise. A rather laborious task, mind, but a possible one.

Blasphemy!  I will defend my analysis of my new convoluted formula, why?  Because it's the only such analysis of data I've ever seen that attempts to weight deviations from mean (averaged weights does not do this).  And it took me a long time to come up with.

IF it is calculated wrong... well, that's another story, or if it's bad math... well, then that's another story.

But... I always was pushing for alternative formula's as being an option to choose from, as I believed weighted averages were an accurate description of stats, but... I thought that some descriptions of stats were more beneficial than others (especially for selection purposes).

So... your point of exp acceleration, and how quickly one does a job, those are nice... I think my formula tends to lead to the latter, but... your formula is hard to test and quantify (I think mine does quickness of job completion, but see my comment on weights), that's where the weights came in, the users could select them.

Maybe I'll push Splinterz to do multiple formula's...  Not a ton, but weighted averages would be one for sure...

[dreiche2]

Most, though not all, of these roles map to a labour just like the skills presented on the standard labour tab. To me it would thus make sense, and enhance the usefulness of the tool, to replace the 'roles' indicator by a duplicate of the first tab, with the same capabilities in terms of directly activating and deactivating labours, just with a different set of data being showcased, whatever you eventually decide it to be. This way you could easily implement your choices, just based on different information.

Agreed, that's what I meant earlier with "there's not much of a reason to treat that score differently from skills etc.".

Blasphemy!  I will defend my analysis of my new convoluted formula, why?  Because it's the only such analysis of data I've ever seen that attempts to weight deviations from mean (averaged weights does not do this).  And it took me a long time to come up with.

(emphasis mine) Yes, and I appreciate that! And I understand the intuition behind your choices, but I think at this point it might be useful to take a step back from different kinds of distribution and all those technical things and think about what outcome you actually want. In particular, do you really want a measure that does this:

In particular, it would prefer Urist at agility 10.1 and strength 5. over one of the dwarves with agility 10. and strength 15 (!) -- just because Urist happened to be 'so much better' in agility when measured in population stdevs.

Because no matter what distribution you use etc., as long as you do everything in terms of relative stdevs, this is what you're going to get. I mean if that's what you want, that's fine! I just think most other people will prefer other options 

Btw:

it would be nice to incorporate something that gives a bit more value the farther from the average, but the only way this will work is if there's a way to transform or handle the massive skewing that skills have. we'll see...

Could you elaborate a bit on what kind of behaviour you're looking for? What kind of situation gives you less weighting on exceptional values than desired?

[DwarfEngineer]

I'm actually in the process of moving right now - Toady's timing is impeccable, as usual.
Long story short, you guys are on your own regarding offsets until Monday.
Sorry! 

[narhiril]

I'm actually in the process of moving right now - Toady's timing is impeccable, as usual.
Long story short, you guys are on your own regarding offsets until Monday.
Sorry! 

No problemo.  Good luck with the move an thanks, as always, for the hard work.

[thistleknot]

dreiche2,

when splinterz mentioned

it would be nice to incorporate something that gives a bit more value the farther from the average, but the only way this will work is if there's a way to transform or handle the massive skewing that skills have. we'll see...

he was referring to the way my formula works (i.e. urist agility examples) (although, the same could be done using static means and stdev's vs populations).

Update:
Btw, I propopsed a quick fix to the skill issue to Splinterz.  Just assume their is only 1 0 skilled dwarf when calculating skills distributions.  It will put all 0 dorfs at 0% and scale a normal curve for those with ranking in skills.

Btw, I'm working on a log normal curve for skills, the above proposed fix may be a good temp fix in the meantime

[melkor]

i dont know if somebody spotted this but il just say it.

i have used the highlight cured dwarves ( i'm a dirty cheater) and spotted a vampire. his age was 337. when i turn of the option his age is still 337.
i dont know if you will see the vampire real age when you never turn on the option, but that should be fixed when you turn the option of

[slink]

This might be the correct addresses for 34_07 on Windows.

[info]
checksum=0x4f7592fe
version_name=v0.34.07 (graphics)
complete=true

[addresses]
translation_vector=0x0182f670
language_vector=0x00182f650
creature_vector=0x017eb758
dwarf_race_index=0x01617ebc
squad_vector=0x01829a58
current_year=0x0101e1f0

[Lockyy]

[snipped]

Seems to work fine for me.