Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[thistleknot]

minor annoyance to report

when I sort by name (I did a startdwarf of 2000 dwarfs, similar to dfusion simple embark), it resizes the column of names by the largest name that is drawn on screen, but doesn't if that name is offscreen.

like I said a minor annoyance.

it would be nice if the name column was just drawn to a certain # regardless of longest name shown on screen (this is a remnant concern I've had with rows being resized...

[splinterz]

minor annoyance to report

when I sort by name (I did a startdwarf of 2000 dwarfs, similar to dfusion simple embark), it resizes the column of names by the largest name that is drawn on screen, but doesn't if that name is offscreen.

like I said a minor annoyance.

it would be nice if the name column was just drawn to a certain # regardless of longest name shown on screen (this is a remnant concern I've had with rows being resized...

i'm not sure it's a good idea to cut-off the name in the grid. the thing that annoys me is when the details pane resizes to long noble lists though. this isn't related in any way to the cells resizing on large fonts.

again, cells resizing to rectangles happens when you use a typeface that has a height larger than the cell width you've specified.

[Maklak]

Thistleknot, it takes most of my focus, concentration and linguistic ability for me to understand you, but I'll try to answer nonetheless.

the more and more i think about it. the more i dislike skill % being modified by skill rate. skill % is already modified by skill rate by attaining skill faster.  to me it would b better to have a group by option for it, ot own category (such as attributes, skills, traits, preferences, skill rate), or an option to modify existing skill by skill rate (but not enabled by default)

Current skill and the speed of gaining that skill are two different things. Both matter and should be separate categories.

what could be done to make it it's own category is to read all rate modifiers for castes, multiply this rate modifier by the castes frequency then do something like set the lowest value to 0, highest to 100 Something similar is being done for attributes

That could work, but this method ignores how skill rates really work. If you had two castes: one with 80% and another with 130% learning rates, the relative difference is +0.625 / -0.385. With your method one would be be set to 0 and the other to 100, which looses information about their relative importance. For a more extreme example, consider two castes having 95 and 105 learning rates in some other skill. That's not that big of a difference, but they still get set to 0% and 100%.
One way to fix this method would be to set minimum to 0 and bottom-cap maximum to 100. That way with the first example, you'd get weights 61.5 and 100, while for the second example 90 and 100, respectively. If for all castes the highest learning rate of something was 10%, you'd get 10 or less for all. Come to think of it, setting the minimum to always 0 is more important with this method than what you do with maximum.
Even with all that, I still say my proposition is better.

here's the code for attribute bins[...]

I don't see why you'd need bins like this for learning rates. I don't care about probability distributions. For traits, attributes and learning rates, just knowing min and max for all castes and everyone in the fort would be enough. Yes, you can get a nicer curve with bins and set the average properly, but I don't need this kind of behaviour. Besides, it might look nice in the grid view, but for weighting attributes and traits into recommendations, just multiplying their values by weights should be better.

"Cumulative distribution function" is What you call "density". Probability density would be (temp.probability / ((temp.max - temp.min !=0) ? (temp.max - temp.min) : 1)), since DF assumes uniform distributions within bins.

The code for traits looks OK, but the code for attributes looks strange. Here is what I see in there:
* Cumulative distribution function, which you for some reason call density, doesn't add up. For each bin it should be equal to that bin's probability plus it's value from the previous bin. Just needs two fixes.
* Why do you need a zeroth bin? By your code a Dwarf has a 0.01 chance of getting an attribute below the minimum for that caste. Is this the case? Does this emulate attribute rust? What is it?
* If you have the zeroth bin, why not also add a last one with min as the caste maximum and max as 5000? NVM, the code after the loop seems to be responsible for this, assuming raws[7] is always 5000. It seems to forget to add 0.01 to temp.density, though. There is nothing above it anyway.
* What is 0.16333333333? Did you mean (1.0 - 0.01 - 0.01) / 6.0 ? Just let the compiler calculate that for you.
* The fist bin generated by the loop has "density" 0. Did you mean 0.01? If so, delete the "if(i>0)".
* The max value for this bin should be one less than the min value for next bin.

the traits and attributes use known knowledge to find their mean of 50.  I did the research on the traits myself using 16,000 dwarf's, which provided me with about 46k data points to draw the statistics from.

Ah, so you did the research. The 0.01 probabilities for being below or above the racial minimum and maximum were there, yes?

The attributes were verified once it was found dfhack was exporting names of attributes in the wrong order, I started a big fight on the wiki saying stuff was inaccurate...  the attributes follow a simple 1/6 breakdown per bin category (remember what I was saying about frequency bins, at least that's what I told a frequency bin was).

I remember dfhack (the command to dump Dwarves to xml), Dwarf Therapist and Runesmith not quite agreeing on what attributes of my Dwarves were in 31.25. I didn't know which was correct, but I think I assumed dfhack to be wrong. 

we hope to incorporate better traits for castes by incorporating the castes min/max values that vanilla traits don't have.  The formula is pretty basic, it's[...]

Yeah, so long as Min and Max are correct, this will behave well. The Mean will be correct and everything above and below will scale linearly. That said, I'd be content with just:

Code: [Select]

return 100 * (Value-Min) / (Max-Min);

we hope to multiply these values by the castes frequency, so we can see where a dwarf fits within the whole fortress mode race.

Yes, that's the logical thing to do once you went that far to get the average. I imagine you get your averages by taking the middles of all bins, multiplying that by their probability, then dividing that by the number of bins (or rather 6/8 the number of bins).
But once again, I'd be pretty content with just linear scaling between absolute minimum and maximum for all castes and current citizens or displaying rounded values when I set the grid to display numbers.

I finally get what your formula is doing
double jobs_left = (29000 - XP) / rate;  // Ignore division by 60.
means treat each job * rate as the # of exp gained.
Interesting concept.

Yes, I omitted the 60 XP per job (or 120 per three mugs or whatever you get per weapon swing), but those are constants for any given skill (or reaction, in case of mugs) and therefore don't matter for comparison. All skill learning rates do is alter the number of XP gained. My formula is just an approximation, since I imagine XP is integer, while I use doubles. My formula is an answer to a question "How many jobs will it take this dwarf to max out his skill?" (except it ignores the 60 XP per job). It is also a good reason to ignore probability distribution for learning rates. You simply don't need this information to decide who is better for a job.

Here's another hopefully simple suggestion (should be much easier to fix than the current optimization/sorting formulas discussion). The optimizer has an "exclude active military" option. How hard would it be to change that to (or add an additional) "exclude squad members" option to avoid having soldiers that just happen to not have active orders at the moment getting included for optimization?

When I look for dwarves to assign them civilian labours, a workaround I use is to sort by military status, but I don't use the "train all the time" schedule, so the soldiers get off-duty and get mixed up with civilians. It would be useful to have an option to treat everyone having a squad as military, regardless of schedules, or if their alert is set to active/training.
I just use the grid view to do this, I didn't even know there was a job optimizer other than all the suggestions for roles and sorting by them.



I propose "soft capping" learning rates by 1/x function, not just cutting them off at 50%. That way for things like 0-10% learning rates you get values close to -2 (and you normally really don't want a dwarf with learning rate below 50 to perform a job) and for 50 and above you get a nice linear progression. While Masterwork never goes below 50%, I've seen people "disabling" skills in their mods by setting their rates to 0 or 10 or something like that, therefore for generality you want to take that possibility it into account. To patch this in replace with:

Code: [Select]

if(aspect_value < -1.0)
  aspect_value = -1.0;

Code: [Select]

if(aspect_value < -1.0)
  aspect_value = -2.0 - 1.0/aspect_value;
Looking at the code some more:

Code: [Select]

skill_rate_value = (skill_rate_value < 100) ? 100: skill_rate_value;Wait, what? You're capping skill rate at 100%?! It should be capped at 1% and only to avoid division by 0!

Code: [Select]

if(a->is_neg)
    aspect_value = 1-aspect_value;This looks suspicious, but I have no idea what it is.

Ah, so skill ratios are factored in last... Hm, looking at this, if skill ratios are very low, then the Recomendation can go below 0%. If this is a desired behaviour is up for debate.


Well, since I can't fight job recommendations, I might as well try to help you improving them. I believe the choice of the right Dwarf for the job (or the military) comes down to answering and prioritising these fundamental questions:

1) "Which one of you has the highest skill now?"
2) "Which one of you has the best attributes and traits and likes for the job?" OR "Which one of you will make the most quality items when you max out your skill?"
3) "Which one of you can get to legendary+5 with the fewest number of jobs, thus wasting the least materials?"
4) "Which one of you is the fastest?"

Are there any more fundamental questions?

I'll give you my perspective on them:

1) Old DT used to sort by this exclusively. You already have this included, but I'd like to add a twist. I was wrong when I wanted this to be sorted by XP, because that would work for sorting by skills alone, but not when this has to be aggregated with other criteria. I have another proposal. Instead of just taking integer skill levels into consideration, scale them linearly to next level according to XP. Just have an array with it, except set 0 as the minimum for Dabbling. http://dwarffortresswiki.org/index.php/Experience Instead of using just skill levels, scale them linearly according to XP. You might do something more sophisticated here, like fit a spline, but the old good linear interpolation is simple and good enough. An examples:
A Proficient (lvl5) Dwarf having 3600 XP, has effective lvl 5.1.
A Proficient (lvl5) Dwarf having 4000 XP, has effective lvl 5.5.

Code: [Select]

double eff_skill_level = dwarf.getSkill(skill_no);
if (eff_skill_level < 20.0)    // Don't care anymore for Legendary +5
{ double skill_xp = dwarf.getSkillXP(skill_no); // double is important. You want floating point arithmethic for this.
  double min_XP = XP_per_level[(int) eff_skill_level];  // static_cast<int>(eff_skill_level) if the compiler screams.
  double max_XP = XP_per_level[1 + (int) eff_skill_level];
  eff_skill_level += skill_xp / (max_XP - min_XP);
}
// Scale eff_skill_level from <0, 20> to whatever you need.
// Take it into account for recommendations, sorting or whatever.
If you want to make skill levels more apart, choose and alpha from  <0.5, 1> and use this instead:

Code: [Select]

eff_skill_level += alpha * skill_xp / (max_XP - min_XP);
2) This is actually a series of questions about attributes and traits and likes. I have very little clue to how this works, you are away ahead of me and so I don't have anything to contribute. In DT you can already have a grid view with attributes and see personality traits. There is even a normally hidden window where preferences can be seen and filtered by.

3) For Vanilla this is the same as 1, really. For mods with learning rates, this is a separate category, because a dwarf with 29000/2 XP and 50% learning rate is obviously better than one with 0 XP and 100% learning rate; even though they will both reach Legendary+5 after the same number of jobs, the first one will be better up to that point and work faster, while creating higher quality items. We have recently discussed this point.

4) Speed affects everything: Walking to the workshop, gathering materials and completing the job itself. There should be some way to get it from the game and I wouldn't mind a column in the grid view with speed. The speed that corresponds to "wait 9 frames" should be considered average. I'd scale the rest with 1/x function before using it for your aggregated goal function, but I would need some additional data to submit a proposal. A downside of using speed from the game might be that a dwarf's speed drops drastically when he is carrying a heavy cage or bin or something.


Right, so unless someone wants to discuss any of this, that's it from me for now.

[Meph]

Thistleknot, it takes most of my focus, concentration and linguistic ability for me to understand you, but I'll try to answer nonetheless.

It's good that I'm not the only one.

[splinterz]

@Maklak: i'll chime in here with some more information on the points you've brought up. first about the attribute bins, the code posted has some flaws/bugs in it, so i wouldn't worry about that. second, the bins described there are the available values when a dwarf is created, so the 1% above and below (if i remember) was used simply to accommodate the other values in the complete range of 0-5000. now, i'm not sure this is entirely correct, but it seems to work fairly well. i have a working alpha right now which takes castes into account and draws/sorts attributes on a 0-100 scale.

i also agree that applying this method to anything other than traits and attributes probably won't work. skill rates may need to be factored in separately rather than attempting to roll them into the skill level. i honestly hacked that in pretty quickly to get the release out with the bug fixes, but in my brief tests, i was happy with applying the skill rate as a bonus/penalty to the current skill's level.

i did some testing with the sorting on skills/labors with one of the formulas you supplied in particular: [50.0f * (1.0f - (((29000-current skill xp)/skill rate)/290.0f))]. now for me, this isn't very intuitive. i had a level 8.3 with 50% learn rate ranked below ~60 dwarves with ~0xp and 100% learn rate. personally it seems like it punishes skill rates below normal far too much. now if i compare with the current weighted method, sorry, current incredibly flawed incorrect method, the same dwarf gets ranked near the top (perhaps too near) but is still outranked by those with over 100% learn rate and a few levels less xp.

so thinking about this more, there's still two parts to the skill rates: the column sorting method, and how to add it into the roles. for the sorting, i'm trying to get something setup so you could right click on a column header, and choose the sorting method, either for all columns of that type, or that particular column. for example:

• total skill xp
• skill level (should be to a decimal as you suggested)
• skill rate
• role ranking

this way, there's no messing around with merging sort values, and it's very clear what you're sorting by.

for roles, well that's another issue, and i'm not completely decided on what to do there. your comment about skill ratings dropping below 0, i assume is also related to your function returning negative values. this is why it was mentioned that we prefer values are all normalized to an equivalent scale of 0-1.

1) "Which one of you has the highest skill now?"
2) "Which one of you has the best attributes and traits and likes for the job?" OR "Which one of you will make the most quality items when you max out your skill?"
3) "Which one of you can get to legendary+5 with the fewest number of jobs, thus wasting the least materials?"
4) "Which one of you is the fastest?"

1) the role calcs don't use the integer level, they already use the actual current xp.
2) as you said, the gridview can handle this somewhat
3) it may be best to consider the skill rate as a separate section of the roles, i'm not sure
4) getting the base speed is trivial, however the current speed is based on many factors: (environment, mood, race, caste, exhaustion, hunger, strength, agility, etc) if i add this i'll probably just show the base (race/caste) + attribute modifications and note that it's the uninfluenced speed or something

Thistleknot, it takes most of my focus, concentration and linguistic ability for me to understand you, but I'll try to answer nonetheless.

It's good that I'm not the only one.

yes this is normal

[Maklak]

i did some testing with the sorting on skills/labors with one of the formulas you supplied in particular: [50.0f * (1.0f - (((29000-current skill xp)/skill rate)/290.0f))]. now for me, this isn't very intuitive. i had a level 8.3 with 50% learn rate ranked below ~60 dwarves with ~0xp and 100% learn rate. personally it seems like it punishes skill rates below normal far too much. now if i compare with the current weighted method, sorry, current incredibly flawed incorrect method, the same dwarf gets ranked near the top (perhaps too near) but is still outranked by those with over 100% learn rate and a few levels less xp.

Level 8.3 with 50% learn rate will max out his skill slower, but will continue to be better most of the way there. If you use your weighted arithmetic means for some weights this formula should still be OK, although a more intuitive formula for skill rates would have to sort-of simulate getting to Legendary +5 and give points for how high the skill is along the way... or maybe just pretend performing 500 jobs at (60.0 / 100.0) * skill rate and add skill levels along the way, then divide them by 500. (with some optimisations to substitute multiplication for addition where applicable). The basic formula for current skill and skill rates would therefore be: 

Code: [Select]

double sr_factor = 0;
int xp = dwarf.get_xp_per_skill(skill);
for (int i=0; i<500; i++)
{ sr_factor += get_skill_level_from_XP(xp);
  xp += 60 * skill_rate;
}
sr_factor /= 500.0;You get a number in <0, 20> This is the simple and stupid version, without speed optimisations.

so thinking about this more, there's still two parts to the skill rates: the column sorting method, and how to add it into the roles. for the sorting, i'm trying to get something setup so you could right click on a column header, and choose the sorting method, either for all columns of that type, or that particular column. for example:

I'm guessing most people would want to apply the same method to all columns, so a radio button in the options would work better. Or maybe even your arithmetic mean with setting the weights, like it does now. I did some thinking on this and while there are other methods of aggregating goal functions, and some might be better, none is as intuitive as weighted arithmetic mean. It is for this reason that I changed my mind about it being "flawed and incorrect beyond redemption".

for roles, well that's another issue, and i'm not completely decided on what to do there. your comment about skill ratings dropping below 0, i assume is also related to your function returning negative values. this is why it was mentioned that we prefer values are all normalized to an equivalent scale of 0-1.

The "simulation" method above will always return values >=0. It may even replace both a goal function for skill rates and goal function for current skill levels as a suitable alternative for both.

Thistleknot couldn't quite make up his mind where he wanted me to scale those values, which caused some confusion. This is an example mapping into <0,1>. It even punishes low learning rates somewhat less.

Code: [Select]

int XP = (current_XP > 29000) ? 29000 : current_XP;
int rate = (skill_rate < 1)? 1: skill_rate;   // 0 or 1 doesn't really matter.
double jobs_left = (29000.0 - XP) / rate;  // Ignore division by 60.
double normalised_XP = 0.5 * (1.0 - jobs_left/290.0);    // Scaled to <-49.5, 0.5>
if (normalised_XP >= 0)
  normalised_XP += 0.5;    // Values <0, 0.5> reserved for those suitable for a task.
else
  normalised_XP = -0.5 / (normalised_XP - 1.0);

2) as you said, the gridview can handle this somewhat.

For better or worse, it also makes me prioritising attributes in my head rather than present me with an aggregating function. But I only ever care about attributes for my military and even that takes the form of "no penalties and preferably good recuperation and disease resistance".

4) getting the base speed is trivial, however the current speed is based on many factors: (environment, mood, race, caste, exhaustion, hunger, strength, agility, etc) if i add this i'll probably just show the base (race/caste) + attribute modifications and note that it's the uninfluenced speed or something.

You could get some error reports about the speed being incorrect, but if you can get it or dig up the formulas, it would be useful. If possible, I'd like this to factor in the carried equipment.

[splinterz]

i'll take another look at those skill rate functions later in more detail, thanks.

I'm guessing most people would want to apply the same method to all columns, so a radio button in the options would work better

i think it may be useful to change up the sorting on the fly, without having to go into the options. this would also be easier to add additional sort values in the future, potentially without having to modify the options. i'll wait and see if anyone else weighs in.

For better or worse, it also makes me prioritising attributes in my head rather than present me with an aggregating function. But I only ever care about attributes for my military and even that takes the form of "no penalties and preferably good recuperation and disease resistance".

why doesn't a custom role column work for this?

[thistleknot]

maklak:
as to assigning rate its own category (i.e. attribute, skill xp, trait, preferences, skill rate)
u touched on minor rate differences having large affects (i.e. 80% vs 90%). i wouldnt think this would b an issue since its merely a min/max approach (which you recommended for attributes). however. i was thinking of using a skill rate category that instead shows how many jobs is needed until legendary (similar to your original formula).

The reason I don't think a min/max approach would matter, is because the max represents the highest skill learner in the game... which uni formally applies throughout the game.  A different mod, would have a different feel for this value, but it would still mean the fastest learner.

alternatively... according to splinterz, max skill rate possible is 500%. maybe that can b the defacto 100%


also, we used to use min/max for attributes...  that was my original idea... but we changed it since the wiki had good information on bin's... and since the bins are not symmetrical, it makes more sense to use them.
the 0-1% and 99-100% represents dwarf's who have fallen below minimum, or gained above maximum attribute value (as defined in bin's).  This never happens on embark, it only happens through the life of the dwarf, and it is a good approximation of the # of current dwarf's at this level (there is no real way to calculate the possible gain a dwarf can get and throw it into this mix, so if you see a dwarf at 99.1%, it means he's above the bin maximum).  The wiki does state a dwarf can gain half his starting value (I believe it's half), but we just opted for this method instead, and it works.

as to the 16.333, I originally had it 98/6, but SOMEBODY changed it, supposedly makes for faster code.

I wasn't showing you the bins for learning rate purposes... I was just showing you the research we've put into coding up each different category.

Maklak:

Code: [Select]

[spoiler]int XP = (current_XP > 29000) ? 29000 : current_XP;
int rate = (skill_rate < 1)? 1: skill_rate;   // 0 or 1 doesn't really matter.
double jobs_left = (29000.0 - XP) / rate;  // Ignore division by 60.
double normalised_XP = 0.5 * (1.0 - jobs_left/290.0);    // Scaled to <-49.5, 0.5>
if (normalised_XP >= 0)
  normalised_XP += 0.5;    // Values <0, 0.5> reserved for those suitable for a task.
else
  normalised_XP = -0.5 / (normalised_XP - 1.0);[/spoiler]
I came up with this on my own, I just did a /2 at the end, and then added .5

[Maklak]

why doesn't a custom role column work for this?

I've never used custom roles before. I don't even know what those are. I'm mostly fine doing this on my own anyway.

u touched on minor rate differences having large affects (i.e. 80% vs 90%). i wouldnt think this would b an issue since its merely a min/max approach (which you recommended for attributes). however. i was thinking of using a skill rate category that instead shows how many jobs is needed until legendary (similar to your original formula).

If you just add skill rates on their own to a weighted arithmethic average, then yes, both methods are equivalent, but for <0, 100> you would want smaller weights. In any case I disagree with this approach. Time and time again I explained that skill learning rate only means something when you know the skill level. For a Legendary +5, skill rate 30% or 300% is irrelevant, but close to 0 XP those are very different. Therefore any method for taking skill rates into account must take skill levels into consideration as well and not just scale skill rates to some interval and add that with a weight to the goal function.

The reason I don't think a min/max approach would matter, is because the max represents the highest skill learner in the game... which uni formally applies throughout the game.  A different mod, would have a different feel for this value, but it would still mean the fastest learner.

Splinterz just bashed me for this with his example of a Dwarf with Level 8.3 with 50% learn rate and I must agree with him. The "simulation" method is the best I can think of, when it comes to skill level and it's learning rate. My formulas aggregated with current skill level are a distant second.

also, we used to use min/max for attributes...  that was my original idea... but we changed it since the wiki had good information on bin's... and since the bins are not symmetrical, it makes more sense to use them.

The way I see it, you did something stupid in a smart way. I'll try to explain the flaws of the current system from a user's standpoint.
1) The most recent DT only uses a few discrete values to represent attributes. There should be more of them.
2) Dwarves are scaled according to their castes, so you can't compare their attributes by just glancing at the grid. (Well, you can in vanilla, but not in caste-heavy mods.)

the 0-1% and 99-100% represents dwarf's who have fallen below minimum, or gained above maximum attribute value (as defined in bin's).

Except military dwarves will usually go above their maximum and there are more than 1% of them.

The wiki does state a dwarf can gain half his starting value (I believe it's half), but we just opted for this method instead, and it works.

I think it says that for adventurer mode. I've never seen a military dwarf cap agility in fortress mode, but I wasn't looking very closely. I would prefer to see the whole range of attributes, but capping them at max and min for all castes works too.

as to the 16.333, I originally had it 98/6, but SOMEBODY changed it, supposedly makes for faster code.

0.98/6.0 Compilers calculate constants and macros made of constants during compilation, so the speed is the same. Even if it wasn't, this is not a place where you need all the speed you can get, like the inside of a loop inside a loop. In most cases it is better to make code human-readable rather than micro-optimise it.

I wasn't showing you the bins for learning rate purposes... I was just showing you the research we've put into coding up each different category.

Well, there is no talking you out of this now, so have fun. I just want to add a few things:

Just remember to handle things like [1000:1000:1000:1000:1000:1000:1000] or [3000:3000:4000:5000:5000:5000:5000] and other edge cases gracefully.

My first method for attributes would be to just display real values, where 0 is min and 5000 is max. The problem here is that with the square / diamond method pretty much everyone would have a red square, so OK, this wouldn't work so good and you want the graphical and numerical representations to be consistent.

My second approach would be to find:
* Min of attribute for { all caste minimums, regardless of frequency; all current citizens }.
* Max of attribute for { all caste maximums, regardless of frequency; all current citizens }.
If Min and Max are very close, scale them further away from their mean. Say at least 300 points apart.
This would work better, except for two problems:
1) As military skills up agility, the others appear to be regressing in it.
2) 0 is not the average. This could be fixed by finding the average, like you do, and scaling everything above and below linearly. Personally I'd settle for this. 

Whatever you do with attributes, just keep these things in mind:
* The more values we get, the easier it is to compare dwarves at a glance. To keep attributes consistent with skills, you could use -20 to -1 (red), 0 (empty), 1 to 20 (blue). 41 different values would be enough for clarity.
* If dwarf A has a higher attribute than dwarf B, the value displayed in grid view for A should be at least as much as B, so I can see the difference at a glance. Don't just scale them inside their own castes.
* Average for entire population / all castes being displayed as 0 with red below and blue above has a lot of appeal... Though I think you want the median, not average for this.
As long as whatever method you use satisfies the first two guidelines, I should be happy with it.

[thistleknot]

My second approach would be to find:
* Min of attribute for { all caste minimums, regardless of frequency; all current citizens }.
* Max of attribute for { all caste maximums, regardless of frequency; all current citizens }.
If Min and Max are very close, scale them further away from their mean. Say at least 300 points apart.
This would work better, except for two problems:
1) As military skills up agility, the others appear to be regressing in it.
2) 0 is not the average. This could be fixed by finding the average, like you do, and scaling everything above and below linearly. Personally I'd settle for this. 

my initial idea was to use 10% below lowest bin, and 10% above highest bin for the final %

but we opted for the code the way it is now.

If dwarf A has a higher attribute than dwarf B, the value displayed in grid view for A should be at least as much as B, so I can see the difference at a glance. Don't just scale them inside their own castes.

this is now fixed, splinterz shared an alpha w me.

[splinterz]

Splinterz just bashed me for this with his example of a Dwarf with Level 8.3 with 50% learn rate and I must agree with him. The "simulation" method is the best I can think of, when it comes to skill level and it's learning rate. My formulas aggregated with current skill level are a distant second.

Code: [Select]

double sr_factor = 0;
int xp = dwarf.get_xp_per_skill(skill);
for (int i=0; i<500; i++)
{ sr_factor += get_skill_level_from_XP(xp);
  xp += 60 * skill_rate;
}
sr_factor /= 500.0;

i'd really like to try implementing this, but 60xp isn't a constant rate per job. it's different depending on the skill, and even varied for any military skill experience (training vs actual combat).

[Maklak]

i'd really like to try implementing this, but 60xp isn't a constant rate per job. it's different depending on the skill, and even varied for any military skill experience (training vs actual combat).

You're right, of course, it was too good to be true and I got too fixated on crafting. I made several beds and got 60 XP for each (so the wiki must be incorrect with 30 XP for some reason), so I guess 60 XP should be correct for most crafting skills and for things like smoothing, farming, etc, there are just different values, maybe even 10 XP per job, like the wiki says. For social skills this is probably different and military skills are notorious for erratic learning. There are different values for combat, sparring, different attacks with the same weapon, demonstration XP scales with several skills, etc.
Since it would be overly complex to have different methods of taking learning rates into account, just try adding the formula that spits out <0, 1> along with what you have for current skill level to your weighted average and see what happens. For some combination of weights level 8.3 with 50% learn rate should come ahead of 0 skill with 100%, which is what you want. My guess is this should happen around 75% skill formula, 25% rate formula.

While I would like the part based on learning rates to behave as nicely as possible, truth be told, a lot of time I'm just looking at lots of dwarves with 0 skill to choose whomever seems like a good replacement for my recently deceased worker. For this, even most Thistleknot's proposals (such as about scaling learning rates to <0, 1>, regardless of skill) would be enough. They just don't have as nice properties for skill levels in-between 0 and 20, so this is what should be a true test for learning rates formulas.

[thistleknot]

i havent tested but that formula should work w 1 vs 60 as well

nevermind, I just noticed the skill level variable in there.

I need to read this more

Well I tried a few things (after giving up on the above, here's the code I was using to test it, I'm not sure what numbers I'm supposed to be getting.

Spoiler (click to show/hide)

Code: [Select]

#include <iostream>

using namespace std;

int get_skill_level_from_XP(int xp)
{
    if ((xp >= 0) && (xp <= (500-1))) return 0;
    if ((xp >= 500)   && (xp <= (1100-1))) return 1;
    if ((xp >= 1100)  && (xp <= (1800-1))) return 2;
    if ((xp >= 1800)  && (xp <= (2600-1))) return 3;
    if ((xp >= 2600)  && (xp <= (3500-1))) return 4;
    if ((xp >= 3500)  && (xp <= (4500-1))) return 5;
    if ((xp >= 4500)  && (xp <= (5600-1))) return 6;
    if ((xp >= 5600)  && (xp <= (6800-1))) return 7;
    if ((xp >= 6800)  && (xp <= (8100-1))) return 8;
    if ((xp >= 8100)  && (xp <= (9500-1))) return 9;
    if ((xp >= 9500)  && (xp <= (11000-1))) return 10;
    if ((xp >= 11000) && (xp <= (12600-1))) return 11;
    if ((xp >= 12600) && (xp <= (14300-1))) return 12;
    if ((xp >= 14300) && (xp <= (16100-1))) return 13;
    if ((xp >= 16100) && (xp <= (18000-1))) return 14;
    if ((xp >= 18000) && (xp <= (20000-1))) return 15;
    if ((xp >= 20000) && (xp <= (22100-1))) return 16;
    if ((xp >= 22100) && (xp <= (24300-1))) return 17;
    if ((xp >= 24300) && (xp <= (26600-1))) return 18;
    if ((xp >= 26600) && (xp <= (29000-1))) return 19;
    if (xp >= 29000) return 20;


}

int main()
{
    bool test = 1;
    while (test)
    {
    double xp;
    int skill_rate;

    double sr_factor = 0;

    cout << "input experience" << endl;
    cin >> xp;
    cout << "input skill rate (1 to 500)" << endl;
    cin >> skill_rate;

    for (int i=0; i<500; i++)
    { sr_factor += get_skill_level_from_XP(xp);
    xp += 60 * skill_rate;
    }
    sr_factor /= 500.0;

    //cout << "Hello world!" << endl;
    cout << xp << endl;

    cout << "quit? 0, else 1";
    cin >> test;

    }



    return 0;
}


I tried the

Spoiler (click to show/hide)

int XP = (current_XP > 29000) ? 29000 : current_XP;
int rate = (skill_rate < 1)? 1: skill_rate;   // 0 or 1 doesn't really matter.
double jobs_left = (29000.0 - XP) / rate;  // Ignore division by 60.
double normalised_XP = 0.5 * (1.0 - jobs_left/290.0);    // Scaled to <-49.5, 0.5>
if (normalised_XP >= 0)
  normalised_XP += 0.5;    // Values <0, 0.5> reserved for those suitable for a task.
else
  normalised_XP = -0.5 / (normalised_XP - 1.0);

* straight xp / 29000

as well as
averaging it with straight xp /29000

for 18000 xp out of 29000:
0.6206896552

I got these %'s

Rate Series
50, 100, 150, 200, 500

Initial formula (interesting to note the 1st value matches my exp value)
0.6206896552   0.8103448276   0.8735632184   0.9051724138   0.9620689655

Factored:
0.385255648   0.5029726516   0.5422116528   0.5618311534   0.5971462545
Averaged:
0.6206896552   0.7155172414   0.7471264368   0.7629310345   0.7913793103

but this still doesn't solve the issue with learning rates below 50%.

[Maklak]

Thistleknot:
While Splinterz rejected the simulation approach on the grounds that it cannot be done for military skills (which is true), if you implemented it in a short program, you might as well get the proper numbers out of it. You want to print the "sr_factor", which is the average skill level over 500 jobs. This is obviously higher for better skill rates and skill levels. With this method "sr_factor" would be a good benchmark for both.
In a production code, you'd want to replace as many additions as you can with multiplications, but for testing, simplicity of a loop is OK.
I'd also use an array with XP levels and a function searching over them with bisection method, rather than hard-code lots of ifs into a function. That way it is easier to answer other questions, such as "How much XP till next level" by re-using the array.
Instead of using integer values for levels, it would also be OK to use interpolated ones based on XP, like I've shown in one of my earlier posts. The only difference is that the "sr_factor" would be slightly higher.

In your example using my formula the "Initial formula" looks good and this is what you want. I don't even know where "Factored" and "Averaged" come from.

but this still doesn't solve the issue with learning rates below 50%.

It does. You just neglected to test it for skill rates such as 35% or 20%. Even for the 50% rate you chose 18000 XP, which is closer to Legendary+5 than to Dabbling, so of course the formula gave you something above 0.5.
So what exactly makes you think that this is wrong?

Oh, and as for displaying skill rates, I recommend the following: If any skill rate is other than 100%, display it, even for dwarves with 0 experience in that skill. The 25% threshold in the big tooltip over a dwarf that recommends skills is good.

EDIT: After some thinking about it, I've come to the conclusion that the simulation method can still be used for military skills. Granted, it won't be using the exact number of jobs, but the slopes are the same and there will still be multiple (measuring pints? what word?) and the proportions between their number will be correct, which is what matters. With the simulation method for military skills the numbers will be distorted from what actually happens, but I believe they will still have enough value to be reliably used for comparison. I've come to this conclusions after staring at level(jobs, initial_lvl) graphs and thinking how to calculate integrals on them. Simulation method FTW! Try it.

[thistleknot]

Maklak

the factored average numbers were based on the second spoilered code in response to you stating this

Spoiler (click to show/hide)

For this, even most Thistleknot's proposals (such as about scaling learning rates to <0, 1>, regardless of skill) would be enough. They just don't have as nice properties for skill levels in-between 0 and 20, so this is what should be a true test for learning rates formulas.

, thanks the tip on the sr_factor (such an idiot).

I'm working up the array with the interpolated values