Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[GoldenShadow]

I'd prefer if roles only took attributes and traits into account, since you can already sort by skill levels on the actual labors page.

[dreiche2]

on the other hand we could go back to using a weighted average of the aspects, after scaling them to a percentile of the absolute maximums. thoughts?

Again with the caveat that I haven't followed the details of the discussion so far (nor played DF much recently, nor ever used Therapist for that matter, so I don't know much about the inner workings of DF), but yes, I'm afraid the simplest solution might make the most sense here. I.e. scale everything to a range 0 to 1 (or 0% to 100%), linearly combine the aspects with the weights, and again renormalise to 0 to 1 (or have the weights be normalised to add to one to begin with, boils down to the same thing).

Then what you do with that score afterwards is the next question. I guess at that point there's not much of a reason to treat that score differently from skills etc. in terms of how it is communicated. E.g. if you want to implement a feature that tells the user how the dwarf fares relatively to the rest of the fortress population, you could also give that as optional info for the skills themselves, not just the role scores.

As for how that relative measure should be computed, I'm not sure. The percentages given with the z score for example only really make sense (as actual percentages I mean) if the population is normally distributed, which I would presume is unlikely to be the case. It also depends on what you think the user actually wants to know. You could also just report in what percentile the dwarf is by ranking the scores. On the other hand, that measure wouldn't give you the info how far she or he is above the mean, etc. ... so yeah, it depends.

I guess in case of doubt the only thing that gives you the full picture would be plotting the whole distribution/histogram of dwarves in comparison 

[thistleknot]

I'd prefer if roles only took attributes and traits into account, since you can already sort by skill levels on the actual labors page.

This would solve the skewed skills.

I honestly think the formula works fine for what it's supposed to do, just things like weights need to be taken into account and an understanding of skills skewness... but if we could apply the skewness to the skills distribution, I think it might be fixed.

Update: so we think we have a proposed solution/idea.

Calculate all cdfs as if they r skewed (weights might mess this up) or at the very least, just skills.  Then do the same at the global level.  Should allow for more accurate results taking skills skew into consideration

Update:
Any help on calculating a cumulative distribution function for skewed distributions http://en.wikipedia.org/wiki/Skew_normal_distribution#References is welcome.  However, there isn't a clearcut way to calculate shape.

so far...

for shape
http://www.weibull.com/LifeDataWeb/estimation_of_the_weibull_parameter.htm &
http://www.weibull.com/AccelTestWeb/calculating_the_parameters_of_the_weibull_distribution.htm

location
http://www.itl.nist.gov/div898/handbook/eda/section3/eda351.htm

scale
http://www.itl.nist.gov/div898/handbook/eda/section3/eda356.htm

Excel stuff
http://www.youtube.com/watch?v=M07Vp_TGBuM&feature=plcp&context=C43244afVDvjVQa1PpcFOBuqaE3P5Td9y03YG0nNcAu7sP5OzEwU8=

http://www.youtube.com/watch?v=PAdh3yv5pro

[DirtyChineseCigarettes]

Could someone help explain to me how I go about changing the roles formula for the splinterz version of Therapist.


I would like to customize them by changing the weighting of roles, a lot. for example i want my mechanics to be heavily intelligence weighted and my wardwarfs to put a 50% weighting on recouperability and disease resistance ONLY.

Looking at game_data.ini, I can find the formulas just right but the weightings dont make sense to me mathematically, they should add up to 1? For example:

26/name="Cook"
26/attributes/size=4
26/attributes/1/id=analytical ability
26/attributes/1/weight=0.5
26/attributes/2/id=agility
26/attributes/2/weight=0.5
26/attributes/3/id=creativity
26/attributes/3/weight=1.0
26/attributes/4/id=kinesthetic sense
26/attributes/4/weight=0.5
26/skills/size=1
26/skills/1/id=21

I want my cook to be more intelligent and creative than kinesthetic and agile, do I just make the weightings smaller? they total 2 instead of 1 so what if i change the total weights to equal 1.5???

Can I just delete weightings?

[thistleknot]

Could someone help explain to me how I go about changing the roles formula for the splinterz version of Therapist.


I would like to customize them by changing the weighting of roles, a lot. for example i want my mechanics to be heavily intelligence weighted and my wardwarfs to put a 50% weighting on recouperability and disease resistance ONLY.

Looking at game_data.ini, I can find the formulas just right but the weightings dont make sense to me mathematically, they should add up to 1? For example:

26/name="Cook"
26/attributes/size=4
26/attributes/1/id=analytical ability
26/attributes/1/weight=0.5
26/attributes/2/id=agility
26/attributes/2/weight=0.5
26/attributes/3/id=creativity
26/attributes/3/weight=1.0
26/attributes/4/id=kinesthetic sense
26/attributes/4/weight=0.5
26/skills/size=1
26/skills/1/id=21

I want my cook to be more intelligent and creative than kinesthetic and agile, do I just make the weightings smaller? they total 2 instead of 1 so what if i change the total weights to equal 1.5???

Can I just delete weightings?

they don't need to add to 1.
if you want mechanics heavility intelligent, just put your attribute weights for analytical at a ratio over what you want your non important skills to be.  Example analytical at weight 2, and the rest at weight 1.  So it will be weighted higher than the non intelligence attributes that are at 1 weight.

[thistleknot]

Okay, we may have a super close solution to fitting skewed curves to the formula, if anyone knows anything about regression lines.
http://www.weibull.com/LifeDataWeb/estimation_of_the_weibull_parameter.htm

Then we could use a Weibull distribution function (available in excel or scalc, scalc prefered as it's opensource), and maybe use the formula for skills.

[dreiche2]

I'm not sure how using a more skewed distribution is going to solve the problem...

Also, could I ask you not to retroactively add content to your posts via editing after further replies have been posted? I wouldn't even have seen what you wrote if I hadn't gone back for unrelated reasons.

[DirtyChineseCigarettes]

Damn thanks this role tab is pretty sweet, I rewrote a bunch of the formulas and loaded it up while my fort was already running. All my dedicated dwarves matched their role with a high 90%. ie: my farmer is 99.18% suited for the role. One issue I am seeing is with my meelee soldiers and ranged soldiers, shitty dwarves are turning up high percentages for these roles =/

[Kogut]

Is it possible to set different images for "no job" and "on break"?

[DirtyChineseCigarettes]

Also I thought I would just throw this out there... Old versions of dwarf therapist used to have a bar at the top (above all the rows in the labour tab) which enabled EVERY dwarf in whichever column you clicked's labour. ie: you could just click 1 box and EVERY dwarf you had would turn on/off stonehauling or whatever depending on the column you clicked. I remember using this feature SO MUCH and miss it a hell of a lot. I guess I will just have to set up custom professions...

[thistleknot]

Also I thought I would just throw this out there... Old versions of dwarf therapist used to have a bar at the top (above all the rows in the labour tab) which enabled EVERY dwarf in whichever column you clicked's labour. ie: you could just click 1 box and EVERY dwarf you had would turn on/off stonehauling or whatever depending on the column you clicked. I remember using this feature SO MUCH and miss it a hell of a lot. I guess I will just have to set up custom professions...

this feature is there, but you need to filter the dwarves by something, for example, gender/sex, and you'll have this bar at the location where the split occurs.

[thistleknot]

I'm not sure how using a more skewed distribution is going to solve the problem...

Also, could I ask you not to retroactively add content to your posts via editing after further replies have been posted? I wouldn't even have seen what you wrote if I hadn't gone back for unrelated reasons.

Well I updated prior posts so as not to "spam" the thread, but...it can make for hard following.

The problem seems to be not with how the attributes and traits are calculated, but in applying a normal curve to a skewed "skills" distribution.  I was never able to verify if the calculations were being performed incorrectly and is why said user had some %'s backwards; however,  I honestly think the original formula should work fine even with skewed data.  I was just hoping to improve upon the original by using skewed distributions for skewed data to ensure accurate %'s being reported amongst the population.

What Splinterz was proposing was something entirely different that doesn't take any consideration for where the dwarve's are in relation to each other, which I didn't agree with.

If you have a better solution, please, let us know.

I'm afraid the simplest solution might make the most sense here. I.e. scale everything to a range 0 to 1 (or 0% to 100%), linearly combine the aspects with the weights, and again renormalise to 0 to 1

Yeah, I think that's what I was going for with the skewed distribution (since you pointed out z-scores don't work for non normal data).   now most of the data does follow a normalish scheme, but we could convert to % more accurately if we did a skewed distribution.

Splinterz pointed out a way from a link I mined to possibly normalize skewed data (see bottom section, box-cot is one):
http://www.itl.nist.gov/div898/handbook/eda/section3/boxcoxno.htm

I don't know, I kind of give up at this point.  The formula for DT is probably gonna go in a different direction from deviations from mean if skewness is taken into account, and even if not, then it's probably not going to be counted at all, which won't count for individual ranges of attributes against each other.

I was thinking, maybe there's a way to remove all 0 valued skills from a skill % calculation, and incorporate the 0 value'ers as some sort of value in relation to those with actual exp.

[DirtyChineseCigarettes]

Also I thought I would just throw this out there... Old versions of dwarf therapist used to have a bar at the top (above all the rows in the labour tab) which enabled EVERY dwarf in whichever column you clicked's labour. ie: you could just click 1 box and EVERY dwarf you had would turn on/off stonehauling or whatever depending on the column you clicked. I remember using this feature SO MUCH and miss it a hell of a lot. I guess I will just have to set up custom professions...

this feature is there, but you need to filter the dwarves by something, for example, gender/sex, and you'll have this bar at the location where the split occurs.

ohhhh yeahhhhhh, you are one of the brighter peanuts in the turd.

[dreiche2]

What Splinterz was proposing was something entirely different that doesn't take any consideration for where the dwarve's are in relation to each other, which I didn't agree with.

If you have a better solution, please, let us know.

I'm afraid the simplest solution might make the most sense here. I.e. scale everything to a range 0 to 1 (or 0% to 100%), linearly combine the aspects with the weights, and again renormalise to 0 to 1

Yeah, I think that's what I was going for with the skewed distribution (since you pointed out z-scores don't work for non normal data).   now most of the data does follow a normalish scheme, but we could convert to % more accurately if we did a skewed distribution.

Well, my impression was that the problem isn't really that you don't fit the distributions properly (even though that is most likely true, too), but that it might not be ideal to combine the aspect values in the way you do. You say you don't like not taking into account how the dwarves compare to each other, but you'd still do that, only after you computed their respective role scores.

It might help clarifying what kind of behaviour you want the measure to have. Intuitively, to decide whether someone would make a good fighter by looking at their strength, agility, sword skill etc., would you really want to do go "oh look, he's in the top 0.00001% of sword fighters in my population, he most be optimally suited to be a fighter! I don't even care that he's below average in strength and agility by a good amount!"?... even if essentially everyone is shitty at sword fighting in your fort, he just happens to be the least shitty by a tiny little bit?

Because as far as I understand, that's what the current calculation will be doing (in a situation where sword skill has a very low standard deviation -- doesn't even have to have a mean at zero! -- and he's a tiny bit above mean, and where he's below average in strength and agility but those happen to have large standard deviations). And as far as I can see, that is not necessarily going to go away with better fitting distributions -- he might still be in the top 0.000001% in terms of sword fighting (because depending on how you measure it, he is.)

Sorry I didn't want to discourage you however! Maybe it's just a good idea to play around a bit with the different options. If you really need some more skewed distributions to fit your data, you might want to look into so-called heavy-tailed distributions, such as the log-normal distribution.

[thistleknot]

Spoiler (click to show/hide)

What Splinterz was proposing was something entirely different that doesn't take any consideration for where the dwarve's are in relation to each other, which I didn't agree with.

If you have a better solution, please, let us know.

I'm afraid the simplest solution might make the most sense here. I.e. scale everything to a range 0 to 1 (or 0% to 100%), linearly combine the aspects with the weights, and again renormalise to 0 to 1

Yeah, I think that's what I was going for with the skewed distribution (since you pointed out z-scores don't work for non normal data).   now most of the data does follow a normalish scheme, but we could convert to % more accurately if we did a skewed distribution.

Well, my impression was that the problem isn't really that you don't fit the distributions properly (even though that is most likely true, too), but that it might not be ideal to combine the aspect values in the way you do. You say you don't like not taking into account how the dwarves compare to each other, but you'd still do that, only after you computed their respective role scores.

It might help clarifying what kind of behaviour you want the measure to have. Intuitively, to decide whether someone would make a good fighter by looking at their strength, agility, sword skill etc., would you really want to do go "oh look, he's in the top 0.00001% of sword fighters in my population, he most be optimally suited to be a fighter! I don't even care that he's below average in strength and agility by a good amount!"?... even if essentially everyone is shitty at sword fighting in your fort, he just happens to be the least shitty by a tiny little bit?

Because as far as I understand, that's what the current calculation will be doing (in a situation where sword skill has a very low standard deviation -- doesn't even have to have a mean at zero! -- and he's a tiny bit above mean, and where he's below average in strength and agility but those happen to have large standard deviations). And as far as I can see, that is not necessarily going to go away with better fitting distributions -- he might still be in the top 0.000001% in terms of sword fighting (because depending on how you measure it, he is.)

Sorry I didn't want to discourage you however! Maybe it's just a good idea to play around a bit with the different options. If you really need some more skewed distributions to fit your data, you might want to look into so-called heavy-tailed distributions, such as the log-normal distribution.

I liked best fit approach, irregardless of whether the dwarf in the long run was a bad pick, as overtime, you really only wish to compare with fellow dwarves, and new dwarves will always adjust the calculation...  Take for example a lot full of old beat up cars.  Your job is to find the best car, not their ranking to all cars ever (which is far harder to calculate anyways), because you only have these set of cars to work with. 
What else is one trying to compare it to?  Is it worth knowing he's a bad fighter just because you know a static mean and sdev?  Maybe, but to me I want a dwarf for the job regardless of my forts progression state.

Well, I think I found a partial solution.  How to get shape parameter, which means a weibull function can be applied now.

http://www.qualitydigest.com/jan99/html/body_weibull.html

which means you can use the weibull function to get % for skewed distributions.

This doesn't figure out how to do deviations from means... but you can do weighted averages with these.

You might be able to do a deviations from mean's using a distance from 50%, but... I don't think that would work the same, as it would start throwing ranges off that I don't know how to reconcile like I did with z-scores.

btw, I LOVE MASTERS OF MAGIC!