Bay 12 Games Forum

Please login or register.

Login with username, password and session length
Advanced search  
Pages: 1 ... 90 91 [92] 93 94 ... 192

Author Topic: Dwarf Therapist (LATEST 0.6.12 5/23/12 see first post)  (Read 863674 times)

splinterz

  • Bay Watcher
    • View Profile
    • Dwarf Therapist Branch
Re: Dwarf Therapist (LATEST 0.6.12 3/6/12 see first post)
« Reply #1365 on: March 09, 2012, 04:34:56 pm »

Well I don't know how your going to get % when you comparing the value based off the middle.

I was curious as to how you were getting the results too (you were running Dwarf numbers through the formula, how were you doing that?  By hand?).  DT doesn't give the results for each dwarf, but now you should see that the formula should give you a below or above threshold.

How are you going to do %'s if 0 is the middle?  0 = 50%?  That might work.  I'm not a percent guy myself, I saw the histogram thing in the thread, but I like the way it is right now, big squares.  Mainly because % might be confusing.  % for racial max?  Or game max?  A dwarf prolly can never hit 5000 Strength, prolly reserved for golem's.  But what if someone runesmith's it?  Then display 100%+?

Here's the corrected formula, I just verified it works properly (thank god I'm home).

Spoiler (click to show/hide)

One last note.  This formula works for above or below average, but...
it doesn't account for the scale that is above average.

For example, a dwarf may have a higher ceiling range (higher upper limit, say average is 1000, max is 3000, but minimum is 0).  The only way to account for this, is to factor the value when it's above 0, in the prior example, prolly a factor of x2 when the value is above 0.  Confusing, but for the basics, it works.  However, I do not know dwarf max's.

If dwarf max's were known, then %'s could be done.

all i'm trying to do is determine if we can get the role formulas to fit into a standard so they can be easily displayed. so to take your formula again, i've got a range between -550 and -5. so how do we draw that? is -550 awfully bad, or is it still decent? what's the absolute awesome level that we'd want to draw the black diamond at?

thistleknot

  • Bay Watcher
  • Escaped Normalized Spreadsheet Berserker
    • View Profile
Re: Dwarf Therapist (LATEST 0.6.12 3/6/12 see first post)
« Reply #1366 on: March 09, 2012, 05:02:54 pm »

all i'm trying to do is determine if we can get the role formulas to fit into a standard so they can be easily displayed. so to take your formula again, i've got a range between -550 and -5. so how do we draw that? is -550 awfully bad, or is it still decent? what's the absolute awesome level that we'd want to draw the black diamond at?

I now understand what your asking, and I had to do some quick spreadsheet stuff.

if using my most recent formula in http://www.bay12forums.com/smf/index.php?topic=66525.msg3078785#msg3078785

I took the max value possible with my formula for every stat, and then averaged it.  So if someone runesmith'd there character to 5000 in everything, then the max value with my formula (with the weights I have) is 3847.614.
Lowest value is -1108.12
0 = middle.

Without weights (i.e. if your removed the *1.2 and *.8 you see in my formula's at the end of each attribute line).
Then the scale would be
-1130.7368421053 for the lowest possible value
0 = middle
3962.174 for the highest possible value

Does that help?


Update:
I might want to change the formula to not factor anything.  But to simply subtract the middle value (i.e Strength's middle value is 1250, Agility is 900, toughness is 1250, etc http://dwarffortresswiki.org/index.php/v0.31:Attribute) from each attribute.  This would yield a deviation from the center.

It would be interesting to see how they did that already with dwarf therapist in displaying attributes.  I should get a copy of runesmith and 31.25 to check it out.  They must have 0 strength listed as the biggest possible red square, and 5000 strength as the biggest possible black square right.  Or did they just use the attribute blocks as listed on the attribute site?  Each attribute has 8 categories, 4 above average, 4 below.  Seems like you got your 4 red/black squares right there.

Seems easier, would probably yield similar results, but for now, what I've displayed should work.
« Last Edit: March 09, 2012, 05:08:30 pm by thistleknot »
Logged

splinterz

  • Bay Watcher
    • View Profile
    • Dwarf Therapist Branch
Re: Dwarf Therapist (LATEST 0.6.12 3/6/12 see first post)
« Reply #1367 on: March 09, 2012, 05:15:11 pm »

all i'm trying to do is determine if we can get the role formulas to fit into a standard so they can be easily displayed. so to take your formula again, i've got a range between -550 and -5. so how do we draw that? is -550 awfully bad, or is it still decent? what's the absolute awesome level that we'd want to draw the black diamond at?

I now understand what your asking, and I had to do some quick spreadsheet stuff.

if using my most recent formula in http://www.bay12forums.com/smf/index.php?topic=66525.msg3078785#msg3078785

I took the max value possible with my formula for every stat, and then averaged it.  So if someone runesmith'd there character to 5000 in everything, then the max value with my formula (with the weights I have) is 3847.614.
Lowest value is -1108.12
0 = middle.

Without weights (i.e. if your removed the *1.2 and *.8 you see in my formula's at the end of each attribute line).
Then the scale would be
-1130.7368421053 for the lowest possible value
0 = middle
3962.174 for the highest possible value

Does that help?


Update:
I might want to change the formula to not factor anything.  But to simply subtract the middle value from each attribute, and average that.  Seems easier, would probably yield similar results, but for now, what I've displayed should work.

yeah that helps, but the big issue is that it's not calculating all the values in one go and storing them, as it displays the column it's putting in the values/rating so it doesn't really know at that point what the min/max is. i might have to try changing that if it's really impossible to get percentages with weighted formulas (i think it should be possible, dgc was doing it after all). however i haven't been on speaking terms with the maths for many a year.

as an example, i'm using that crappy unweighted formula i made before, and since i know the values will be between 0-100, i can draw them just like the attributes and skills, augmenting the result so it rests between -50 and +50, then scaling it down to between -20 and +20. this is the same range used for the attributes and skills. this way it doesn't matter what formula you throw it, as long as your min/max range ends up between 0 and 100, it will draw everything without issue.

edit:
i totally missed what you asked about attributes. they're basically hard-coded to use the ranges specified on the wiki and then assign a rating from there. as an example for strength, the lowest (<251) bracket is a -15 rating or the red diamond. the highest (>2250) is +15 and the black diamond.

« Last Edit: March 09, 2012, 05:36:31 pm by splinterz »
Logged

thistleknot

  • Bay Watcher
  • Escaped Normalized Spreadsheet Berserker
    • View Profile
Re: Dwarf Therapist (LATEST 0.6.12 3/6/12 see first post)
« Reply #1368 on: March 09, 2012, 05:47:57 pm »

UstWell the formula I gave you was what I had to work with in dwarf therapist.  I can give you a c++ formula that would normalize the distributions (that's if your actually implementing this into the source code).  It would give better results.  The problem w my formula is it normalizes the means but it changes the mins and maxs (especially when I use weights).  I could try a few things and do category blocks like those on the wiki, or I can force them all to a value between 1 and 100, 50 being the average.  Either way, the results aren't optimized nor correct for really high values (example highest willpower and focus w my formula is around 2900, str is like 4500).  So u can see their is a bias, and I don't think it should go into the source.

If dt supports if/then statements, then I can fix it with just dt
« Last Edit: March 09, 2012, 05:58:25 pm by thistleknot »
Logged

splinterz

  • Bay Watcher
    • View Profile
    • Dwarf Therapist Branch
Re: Dwarf Therapist (LATEST 0.6.12 3/6/12 see first post)
« Reply #1369 on: March 09, 2012, 06:01:10 pm »

Well the formula I gave you was what I had to work with in dwarf therapist.  I can give you a c++ formula that would normalize the distributions (that's if your actually implementing this into the source code).  It would give better results.  The problem w my formula is it normalizes the means but it changes the mins and maxs (especially when I use weights).  I could try a few things and do category blocks like those on the wiki, or I can force them all to a value between 1 and 100, 50 being the average.  Either way, the results aren't optimized nor correct for really high values (example highest willpower and focus w my formula is around 2900, str is like 4500).  So u can see their is a bias, and I don't think it should go into the source

maybe i'll take another look at what dgc did to get those results. i really thought it would be possible to create a weighted formula with the scripting that would spit out a percentage similar to what dgc comes up with.

splinterz

  • Bay Watcher
    • View Profile
    • Dwarf Therapist Branch
Re: Dwarf Therapist (LATEST 0.6.12 3/6/12 see first post)
« Reply #1370 on: March 09, 2012, 06:21:54 pm »

-snip-

If dt supports if/then statements, then I can fix it with just dt

the QScriptEngine does support if statements, i just tried the following and it's putting in the correct values: if(d.strength()>1000){0}else{1} 

i'm about done for tonight but i'll see if maybe sometime tomorrow i can get a build out that includes these changes so you and others can mess around with scripts to define roles.
« Last Edit: March 09, 2012, 06:23:47 pm by splinterz »
Logged

thistleknot

  • Bay Watcher
  • Escaped Normalized Spreadsheet Berserker
    • View Profile
Re: Dwarf Therapist (LATEST 0.6.12 3/6/12 see first post)
« Reply #1371 on: March 09, 2012, 10:47:16 pm »

I took a sample of 294 dwarves, and found their min/max values.

I updated the wiki (hopefully they don't pull it).

http://dwarffortresswiki.org/index.php/DF2012:Attribute#Sample_of_294_Dwarves_with_Min.2FAvg.2FMax_Values

New Link:
http://dffd.wimbli.com/file.php?id=5834

Min value for dwarves is 146 (Agility)
Highest value is 2537 (Musicality)

Which means my formula from earlier doesn't suffer too much bias for higher numbers, but I found some interesting results.

Dwarves have higher than center musicality and memory

They have a few stat's that are higher than the center's reported on attributes wiki.  I marked them after their name with either a < or a >.  Agility was pretty much right on.

I did discover that their is a bell curve to these values (possibly skewed, but I need a bigger sample), which is good to know.

In light of that information, I posted a new formula (that doesn't use factors anymore)
Spoiler (click to show/hide)
« Last Edit: March 11, 2012, 04:31:01 am by thistleknot »
Logged

ClubFoot

  • Bay Watcher
    • View Profile
Re: Dwarf Therapist (LATEST 0.6.12 2/29/12 see first post)
« Reply #1372 on: March 09, 2012, 11:48:48 pm »

I am, indeed, unsuccessful w/ the osx version.

Here's a new OSX beta version. Should fix the double icon issue, and the automatic download issue.

Dwarf Therapist OSX Beta (64-bit) v0.6.12b04

success!  Thanks.

I think the biggest outstanding issue for basic OSX support is the inability to change the professions.  Otherwise, this seems to be functioning fully.



Logged

thistleknot

  • Bay Watcher
  • Escaped Normalized Spreadsheet Berserker
    • View Profile
Re: Dwarf Therapist (LATEST 0.6.12 3/6/12 see first post)
« Reply #1373 on: March 10, 2012, 08:42:38 am »

Spoiler (click to show/hide)
0[/code][/spoiler]

Well, my 294 dwarf sample was pulled from the wiki, but on the wiki for attribute, it has 2500 listed as pretty much the defacto max for dwarves (though I had a dwarf with slightly above 2500).

http://dwarffortresswiki.org/index.php/DF2012:Attribute#How_Your_Dwarf_Gets_Attributes

I've found that the lower range tends to be closer to 200.

Dwarves have higher than center musicality and memory

They have a few stat's that are higher than the center's reported on attributes wiki.  I marked them after their name with either a < or a >.  Agility was pretty much right on.

Also, according to the wiki, dwarve's have a higher

Here's my 294 dwarf sample if anyone's interested.
http://dffd.wimbli.com/file.php?id=5834
« Last Edit: March 10, 2012, 09:04:43 am by thistleknot »
Logged

thistleknot

  • Bay Watcher
  • Escaped Normalized Spreadsheet Berserker
    • View Profile
Re: Dwarf Therapist (LATEST 0.6.12 3/6/12 see first post)
« Reply #1374 on: March 10, 2012, 10:10:15 am »

could we have traits added into dwarf therapist rather than having to apply filters?

splinterz

  • Bay Watcher
    • View Profile
    • Dwarf Therapist Branch
Re: Dwarf Therapist (LATEST 0.6.12 3/6/12 see first post)
« Reply #1375 on: March 10, 2012, 03:40:34 pm »

could we have traits added into dwarf therapist rather than having to apply filters?

i'm pretty sure you can add trait columns in a custom view if you want, however the drawing methods are missing (i think).

in regards to the roles i've been working on and after giving it more thought, i'm strongly leaning towards just implementing something similar to what dgc did in terms of allowing users to specify a role simply by adding lists of traits, attributes and skills. the advantage would be that it's much more user friendly to define a custom role. the disadvantage is it won't be as flexible as building some crazy weighted script.

the only question then is how to also factor in the levels of traits and skills? should they be weighted equally important as attributes, or something else? anyone have any thoughts on this?

thistleknot

  • Bay Watcher
  • Escaped Normalized Spreadsheet Berserker
    • View Profile
Re: Dwarf Therapist (LATEST 0.6.12 3/6/12 see first post)
« Reply #1376 on: March 10, 2012, 09:26:22 pm »

could we have traits added into dwarf therapist rather than having to apply filters?

i'm pretty sure you can add trait columns in a custom view if you want, however the drawing methods are missing (i think).

in regards to the roles i've been working on and after giving it more thought, i'm strongly leaning towards just implementing something similar to what dgc did in terms of allowing users to specify a role simply by adding lists of traits, attributes and skills. the advantage would be that it's much more user friendly to define a custom role. the disadvantage is it won't be as flexible as building some crazy weighted script.

the only question then is how to also factor in the levels of traits and skills? should they be weighted equally important as attributes, or something else? anyone have any thoughts on this?

I think that's best.  I just did a formula with a trait.

I had 3 attributes (for a diagnostician), and added helpfulness.  To get helpfulness to match the other stat, I weighted it by x17.

I chose x17 because attribute-50 = a range of 0 to 50 away from 0 (center).  The 3 attributes that (diagnostician used in DGC) were Analytical Ability, Memory, and Intuition.  Each had their own range of 800-900 from 0.  I got this from determining what the minimum vs maximum was (I had a sample of dwarves (WHICH HAS DIFFERENT RESULTS THAN THE WIKI FOR SOME AVERAGES), but one could just use the http://dwarffortresswiki.org/index.php/DF2012:Attribute site and guess what the max was.

Spoiler (click to show/hide)

The 3 attribute's had ranges from their average as 800, 800, 900 from average.  (averaged range from average had 833.33).  833.33/50 = 16.66.

I could have used 16.66, but I wanted to give helpfulness a higher weight, so I made it 17.

I weighted the other two, analytical ability, intuition, and whatever at 1 to 1.1, so I was giving helpfulness a weight of .33 higher (16.66 to 17 is .33 higher).

I would include the option of specifying weights, default could be 1 if none is entered.

In the end the formula was something like this
Spoiler (click to show/hide)

Hope that made sense.

Update:
The idea works, but I just realized those ranges are not from average.  Range from 0 is different  some have higher range to max than they do to min.  I guess I would have to average those two.

For example, for intuition I have a min of 452, max of 2250.  Range of 1790.  However, average is 1256.  So a min has a range of 804 from average, and max has a range of 994.  I would of course round all these numbers.  I would say the min has a range of 0 of 800, and from max 1000.  I would average the two to say intution has a range of 900 from 0.

I would then repeat the same for analytical ability and memory.
Analytical: 900 from average
Memory: 850 from averaage
(900+900+850)/3 = 883.333
883.333/50 = 17.66666666666667

New formula would be
Spoiler (click to show/hide)

If I were to use more traits, I would continue to base the traits multiplier on the value I derived (ex. 883.333/50 = 17.66666666666667), and increase the number I'm dividing by from /4 to how many attributes/traits are involved.

The 110 is a number I derived from the average center attribute value (i.e. an average of all attribute average values) * 10%. 1100 in this case is the average of ALL ATTRIBUTE averages.  However, it might be more accurate to use the averages of only the attribute's involved.  Example: Intution average is 1250, Memory: 1500, Analytical: 1000, average is 1250.  Again, I'm using the averages I derived from a sample of 294 dwarves, the averages for stats are listed in the wiki, and those can be used instead.

Update:
I just realized something, applying the .33 increased to 17.66 might not be right, I might want to do d.trait(20)*17.66*1.3.

New formula would be
Spoiler (click to show/hide)

Update 3:
Changed my < to > in my formula's

As a side note, the benefit of crazy weighted scripts isn't really important.  Dwarf Therapist has the ability already by just using the scripts I provided, one can change the >threshold quite easily.  I no longer think a column with a weighted script is a good idea anymore.  I think maybe some default scripts being provided with dwarf therapist is a better idea.  However, determining weights is a hard issue.  I'm thinking of starting a thread on dt filter scripts.

Here's a ranged combat script
Spoiler (click to show/hide)

Update:
A note on average discrepencies.  You can run a function that does the averages of all attributes of your dwarves at any given time, and calculates min/max's, and averages the range from the average.  Then use this for the formula's.  This creates a dynamic range of best available dwarves based on the makeup of your fortress.  This is only recommended if min/max ranges as well as averages are unknown.

Update:
Using the crazy convoluted weighted system (but everything described above), I came up with a close combat formula that takes into account the rage trait, as well as the perseverance trait.
Spoiler (click to show/hide)

Update:
Thanks for the info on the if/else stuff, I implemented it (shouldn't care if a close combat dwarf has below average rage, but it should be beneficial
Spoiler (click to show/hide)
« Last Edit: March 11, 2012, 12:10:03 am by thistleknot »
Logged

schismatise

  • Bay Watcher
    • View Profile
Re: Dwarf Therapist (LATEST 0.6.12 3/6/12 see first post)
« Reply #1377 on: March 11, 2012, 01:16:29 am »

You guys are nuts. I was following this entire thread to the letter and then suddenly thistleknot shows up and starts talking rocket science!

the only question then is how to also factor in the levels of traits and skills? should they be weighted equally important as attributes, or something else? anyone have any thoughts on this?

Anyway, i think i can help you with that at least - the obvious answer is they should be weighted based on how fast they are to level up. Many skills take years or even decades to go from 0 to legendary, while going from 'weak' to 'superdwarvenly tough' might only take a few seasons of constant sparring. I rarely, if ever, take attributes into consideration when picking military dwarves, for example. I'm more likely to pick an active, easily angered dwarf with high willpower who's weak as a stick than a friendly superdwarf, since traits, on the other hand, can't level up at all. So i suppose traits should be given the relevant merit, although it's not entirely clear how much more an angry dwarf will enter rage, let alone how much it effects combat.

TLDR: skills > traits > attributes, imo

Of course, if you're picking out soldiers to turn into vampires, it's a whole different story (since their attributes become locked permanently upon vampyrism).
Logged

splinterz

  • Bay Watcher
    • View Profile
    • Dwarf Therapist Branch
Re: Dwarf Therapist (LATEST 0.6.12 3/6/12 see first post)
« Reply #1378 on: March 11, 2012, 02:42:12 am »

You guys are nuts. I was following this entire thread to the letter and then suddenly thistleknot shows up and starts talking rocket science!

the only question then is how to also factor in the levels of traits and skills? should they be weighted equally important as attributes, or something else? anyone have any thoughts on this?

Anyway, i think i can help you with that at least - the obvious answer is they should be weighted based on how fast they are to level up. Many skills take years or even decades to go from 0 to legendary, while going from 'weak' to 'superdwarvenly tough' might only take a few seasons of constant sparring. I rarely, if ever, take attributes into consideration when picking military dwarves, for example. I'm more likely to pick an active, easily angered dwarf with high willpower who's weak as a stick than a friendly superdwarf, since traits, on the other hand, can't level up at all. So i suppose traits should be given the relevant merit, although it's not entirely clear how much more an angry dwarf will enter rage, let alone how much it effects combat.

TLDR: skills > traits > attributes, imo

Of course, if you're picking out soldiers to turn into vampires, it's a whole different story (since their attributes become locked permanently upon vampyrism).

yes that's what i was leaning towards as well. my intention was to do something much simpler than what thistleknot proposes by assigning a rating like so: take an equally weighted average of the attributes specified for the role, using the functions defined by dgc (example below). then take a weighted average of traits and skills (separately, but possibly using a similar function). finally i'd use user defined weights for the 3 groups to determine the final 'rating' as a weighted average of traits, skills and attributes.

Code: (dgc strength function. x = strength) [Select]
    var xpct;
    if (x <= 949) {
        xpct = ((0.0333988 * x) - 15.0295)
    } else if (x >= 950 && x <= 1149) {
        xpct = ((0.0837487 * x) - 62.8943)
    } else if (x >= 1150 && x <= 1249) {
        xpct = ((0.168333 * x) - 160.249)
    } else if (x >= 1250 && x <= 1349) {
        xpct = ((0.168343 * x) - 160.429)
    } else if (x >= 1350 && x <= 1549) {
        xpct = ((0.0837538 * x) - 46.4006)
    } else if (x >= 1550) {
        xpct = ((0.0238412 * x) + 46.3801)
    };
    return xpct;

i'll have to code it and then do some testing with thistleknot's formulas vs this method and see how much they differ. the disadvantage would be that it's not as flexible because you wouldn't be able to weight attributes/traits/skills individually, but i'm not convinced this is necessary seeing how useful dgc is while generating a rating purely on equally weighted attributes. the main advantages would be that it's simpler for the user to add roles (less weighted values to consider), and everything would be standardized to percentile results. additionally this means we don't have drawing issues because we know the resulting ranges we're dealing with. after all, what's the point of adding a column for a role when you can't see the results?

@thistleknot: i disagree that having role columns is a bad idea. the problem with filter scripts is that they, well... filter. you constantly have to tweak the formulas to make sure you're getting data. if you get no results, that's no good, because while nobody fits your precise filter, you still want to know who would best fit the role anyway. a column with the ability to sort and view the rating graphically does just that. i think something similar to this as an end result would be awesome.

thistleknot

  • Bay Watcher
  • Escaped Normalized Spreadsheet Berserker
    • View Profile
Re: Dwarf Therapist (LATEST 0.6.12 3/6/12 see first post)
« Reply #1379 on: March 11, 2012, 03:08:36 am »

Wow, so much in that post.  I liked the % thing at the end.  Yeah, I'm not for %.  Why?  Because this isn't exact.  However, the users are never gonna know the ranges either I guess.  So maybe % is good.  I believe with some research http://dffd.wimbli.com/file.php?id=5834 I've been doing we can figure the min/max/avg for all attributes (which differ from the wiki for some things).

YES THERE IS A BELL CURVE to attributes.  I was gonna try to figure out how to merge bell curves, but it makes my brain hurt.  That's why I just did simple Attribute-(Attribute's Average) to get a result that differed from the center of said attribute.  If doing %, then you need to find the min/max values from average of all attributes for that profession, and that will be your range from 0 that represents %.

Example:  A hypothetical profession that uses Analytical, and Agility

0 represents Attribute's average (i.e. attribute-average)

Agility: Averaged ranged from 0 is ~725
Analytical: Averaged ranged from 0 is ~885
Both Averaged = 805
var attributeAveragedRange = 805; (round to 800)

So, a -800 would be 0%, and a +800 would be 100%

Simple enough.

Traits already have their ranges setup (0-100, or -50 to 50 after you apply trait-50).

As explained earlier, traits need to be factored to equal the range of the attributeAveragedRange.

var avgAttrMean = the average of all Attributes involved (Agility is rounded to 900, analytical 1000), average is 950
var pctAwayFromMean = avgAttrMean*.15;
var traitTactor = attributeAveragedRange/50;
which is 800/15 (which results in 16).

Then the formula would be an average of all deviations from the center.

So

d.agility()-average is... (rounded to logical)
1 represents a factor, 1 means 100%
((
((d.agility()-900)*1)+
((d.analytical_ability()-1000)*1)+
((d.trait()-50)*16*1)
)/3)>pctAwayFromMean

Update:
I forgot to mention, pctAwayFromMean needs to be factored (although it's not really that important) to the average of all attribute weights.

Above example, everyone's weight is 1, so there's no need to mess with pctAwayFromMean, but if the weights were 1.3, 1.2, and .9, the average of that is 1.133333333333333, then pctAwayFrom would be factored by 1.1333, this is because the weights is going to change the averages (because it's weighting the results).
« Last Edit: March 11, 2012, 04:25:09 am by thistleknot »
Logged
Pages: 1 ... 90 91 [92] 93 94 ... 192