Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[thistleknot]

I've been working on your code maklak

I'm getting get_level to return a decimal version
http://pastebin.com/CdGs1gHn

[Maklak]

I don't see any problems with what you did, except for trivial things, such as using printf and cout in one program. You did something very similar to what I proposed near the bottom of this post. It looks good, except I wouldn't subtract 1 from upperDiff, so that when you are 1 XP from the next level you still don't get a number equal to that next level.

Once you have the get_double_level_from_xp function, it is a good idea to use it instead of XP for sorting by skill and suggested roles. Oh and of course comment out the couts when you know it works. In any case, it will help you to learn gdb rather than rely on gratuitous printing of variables.

I'm not sure what you want from me? To run the program and see how well it interpolates levels? It looks fine.

[thistleknot]

Maklak:
I removed the -1,
I also think I noticed a bug in your code (let me know if I'm totally wrong about this)

I propose this fix (but still am not getting the same #'s as sim1 with what I believe should be the fix)

Spoiler (click to show/hide)

Code: [Select]

    double avg_level = curr_level + (1-((low + high) / 2.0));   // Average scaled level for this iteration.

Spoiler (click to show/hide)

Code: [Select]

  { int xp_gap = xp_levels[curr_level+1] - curr_xp;         // How much XP till mext level?
    if (xp_gap > sim_xp)
      xp_gap = sim_xp;
    double low = (0.0 +curr_xp -xp_levels[curr_level]) / (xp_levels[curr_level+1] -xp_levels[curr_level]);
    double high = (0.0 +curr_xp + xp_gap -xp_levels[curr_level]) / (xp_levels[curr_level+1] -xp_levels[curr_level]);
    double avg_level = curr_level + (low + high) / 2.0;   // Average scaled level for this iteration.
    ret += xp_gap * avg_level;
    curr_level++;
    curr_xp = xp_levels[curr_level];
    sim_xp -= xp_gap;
  }


for 10000 50
your avg_level is 10.666 when it should be 10.333

I made the same mistake when I was posting averaged results and using the spreadsheet to record the interpolated levels.

Then I inversed the levels decimal formula.

btw, I got my debugging working with codeblocks, so no more need for cout.

I'm gonna propose the fix inside the code, and see if it remedies it.

I noticed the issue when I converted your function to do interpolation automatically, but was not getting the EXACT same results as sim2

Spoiler (click to show/hide)

Code: [Select]

double simulate_skill_gain(int xp, int rate)
{ if (xp >= max_xp)
    return 20.0 / 20.0;

  if (rate == 0)
    return get_double_level_from_xp(xp) / 20.0;       // Obviously stays the same.

  int sim_xp = max_xp;    // 29k seems like a good value to me.
  sim_xp = (sim_xp / 100.0) * rate; // This is how much XP will go towards skill learning.
  int total_xp = sim_xp;
  double ret = 0.0;
  //int curr_level = get_level_from_xp(xp);
  double curr_level = get_double_level_from_xp(xp);
  int curr_xp = xp;

  while ((sim_xp > 0) && (curr_level < 20.0))
  { int xp_gap = xp_levels[int(curr_level)+1] - curr_xp;         // How much XP till mext level?
    if (xp_gap > sim_xp)
      xp_gap = sim_xp;
    ret += xp_gap * curr_level;
    curr_level++;
    curr_xp = xp_levels[int(curr_level)];
    sim_xp -= xp_gap;
  }

  if (sim_xp > 0)
    ret += 20 * sim_xp;
  ret /= total_xp;
  ret /= 20.0;
  return ret;
}



and splinterz had this neat formula that worked, and mine was matching his

Spoiler (click to show/hide)

Code: [Select]

double getLevelFromXP (int xp)
{
    return (double(xp) /(225.0f + 5.0f*sqrt(2025.0f + (2.0f*double(xp)))));
}



Update:
wow, I love this part of the code

I was trying to figure this out the guessing way
  if (normalised_XP >= 0)
    normalised_XP += 0.5;    // Values <0, 0.5> reserved for those suitable for a task.
  else
    normalised_XP = -0.5 / (normalised_XP - 1.0);

it's beautiful.

here's a pic of it in action
the numbers on the bottom represent #x500xp

updated to include up to 29k

Spoiler (click to show/hide)

I'm not sure if that's right though, looking at my spreadsheet, I was assuming everyone would approach 1 near the end).  looking at the sheet (you guys can't see it), 1 skill exp 1 rate % even at 29k is only 36.71%

looking at my sheet, I realized I failed to put the 29k column in (58 *500 = 28500), but when I did, all the #'s jumped straight to 1.5, or 150%
I just realized if xp >= 29k, it returns 1.

btw, your code (untouched) won't run 27000 or 28000 exp at 100% rate...

Here's some final results on the interp1 output
averaged (.5 skill .5 rate)

Spoiler (click to show/hide)

.25 skill .75 rate

Spoiler (click to show/hide)

.75 skill .25 rate

Spoiler (click to show/hide)

[Maklak]

Ah, nice progress. I get the feeling that we're spending a bit too much effort here, but is doubles up as learning experience, so OK.

for 10000 50
your avg_level is 10.666 when it should be 10.333

Working as intended and there is nothing to fix. 10.3333 is the current interpolated level for 10000 XP, so low is equal to 0.3333. High is equal to 1, since we have enough sim_xp left to gain next level. I could just add 1 XP each time, then add the interpolated level for that, but I do the smart thing instead. Low, high and xp_gap form a trapezoid with two right angles and I calculate it's surface area. The "slope" of interpolated level stays the same until a level up. Just think of it as changing lots of additions into a multiplication by "average level" from 10000 to 11000 XP, which is 10.66666 for 10500 XP.

Actually sim1 makes more sense exactly because it always uses the current level without any interpolation. For any given level the quality and speed will be the same, no matter how close you are to the next level. I made sim2 first, then reconsidered, but left it in for comparison.

After you changed the formula for sim1, it uses the interpolated level for the current XP, then keeps on using non-interpolated level, as if it couldn't make up it's mind. Of course after your modification it does a different thing than both sim1 and sim2, so the numbers will be different. I'd not use your changes for production code, but by all means, play around with it, see what it does and ask me questions. It would be even better if you understood what these things are supposed to do, not treat them as black boxes with parameters. Try drawing sim1 and sim2 on a piece of paper. They divide XP from here to 29000 into trapezoids by level (or rectangles in case of sim1), then calculate the surface area of that (which is what happens when you compute and integral), then add a rectangle for lvl 20 if it it reached and finally normalise the result, dividing it by the simulated XP and 20.

LevelFromXP = (double(xp) /(225.0f + 5.0f*sqrt(2025.0f + (2.0f*double(xp)))));

I haven't figured out where it came from, but it looks very good on a graph. It gives the right numbers for all levels and there are only minor differences between this and my "linear between levels" approach. Plugging this directly into sim2 will introduce up to 2% additional error, which is OK. Good job on this one, Splinterz.

wow, I love this part of the code

Well, I think sim1 is better overall, but for this formula I had a piece of code that said how much effort it will take to get to Legendary+5 with current xp and skill rate. The problem was that for very poor rates it returned too negative values. One solution was to cap it at say 50% skill rate and call it a day, as you proposed, but I wanted something more elegant, so instead I used http://en.wikipedia.org/wiki/1/x When used properly, it can map a finite interval into an infinite one, or vice-versa and near the "fat part" it looks pretty nice. So I made a continuous function by glueing it with linear function for really good rates. That way I still have some kind of comparison going on for very low skill rates and nice linear properties if Legendary+5 is closer than 29000 xp after factoring in the skill rate. It could be further tweaked to make the linear part longer, but I'm unsure what would be better.
Your graph for this will look even better if you put more horizontal lines there, so you can see what skill levels and rates match each other according to this formula.
With 28500 XP and 1% rate you are still 500*100 = 50000 XP away from legendary+5 (and depending on how XP gains are rounded, will never get there, because for example 60/100 = 0). Legendary+4 is still very good, that's why for "best roles" you'd want to aggregate both this formula and current skill level to make a judgement. I'd be pretty happy with instant Legendary+4 myself, if it doesn't rust.
Both my simulation formulas would still return 19/20 or more in this situation, which means they aggregate skill levels and rates more how a player would want them to.

btw, your code (untouched) won't run 27000 or 28000 exp at 100% rate...

Come again?
Hm, infinite loop for 26601 XP to 28999 XP, which is level 19.
Damn it, no gdb on this computer
OK, the error was in get_level_from_xp where max should be initialised to 20, not 19. That removes the infinite loop. Sorry for the bug.

.25 skill .75 rate

This one looks the best to me, except 1% rate sucks no matter what, but then you normally don't get that low in mods. These graphs are why the pseudo-simulation approach is better. No matter how you tweak the weights for skill and skill_rate_eval, it still won't give you exactly what you want for all xp and skill rate values.

I suggest that you experiment with simulate_skill_gain, but change int sim_xp = max_xp; to less, say 20000, 15000 and 10000.

[thistleknot]

I'm still a bit confused then as to how your formula does that advanced math stuff on triangles and mine and splinterz come out the same...

but...

I guess that's not where the issue is at?

the issue is at where your formula follows up with a drawing of the difference to next level.

where mine doesn't do that, because mine is called before?

to be honest, I've had some fun with friends tonight with drinks and all

so to try and figure it out before I know you'll check it out around 3 AM my time will mean I haven't had the fortitude to figure it through yet.

so if you can understand my miscomprehension, and explain in detail why Splinterz sqrt formula is drawing the level at around the right range, yet is not enough to produce sim2 results, would be awesome!

[Maklak]

I'm still a bit confused then as to how your formula does that advanced math stuff on triangles and mine and splinterz come out the same...

Hey, last time you said, they weren't exactly the same. Anyway, my sim1 and sim2 formulas pretend they perform numerical integration, sim1 uses something similar to the rectangle method, while sim2 uses trapezoid rule, but in this particular case sim1 is better, I think. sim1 only cares about current level rounded down. Wikipedia has some nice pictures on numerical integration.

If you want to plug Splinterzes formula into sim2 (there is no reason to do it for sim1, except if you take the floor of it), here's how to do it: 

Code: [Select]

double getLevelFromXP (int xp)
{ return (double(xp) /(225.0f + 5.0f*sqrt(2025.0f + (2.0f*double(xp)))));
}

double simulate_skill_gain_slopes_2(int xp, int rate)
{ if (xp >= max_xp)
    return 20.0 / 20.0;
  if (rate == 0)
    return getLevelFromXP(xp) / 20.0;       // Obviously stays the same.

  int sim_xp = max_xp;    // 29k seems like a good value to me.
  sim_xp = (sim_xp * rate) / 100; // This is how much XP will go towards skill learning.
  int total_xp = sim_xp;
  double ret = 0.0;
  int curr_level = getLevelFromXP(xp); // Convert to int
  int curr_xp = xp;

  while ((sim_xp > 0) && (curr_level < 20))
  { int xp_gap = xp_levels[curr_level+1] - curr_xp;         // How much XP till mext level?
    if (xp_gap > sim_xp)
      xp_gap = sim_xp;
    double low = getLevelFromXP(curr_xp);
    double high = getLevelFromXP(curr_xp + gap_xp);
    double avg_level = (low + high) / 2.0;  // Average scaled level for this iteration.
    ret += xp_gap * avg_level;
    curr_level++;
    curr_xp = xp_levels[curr_level];
    sim_xp -= xp_gap;
  }
  if (sim_xp > 0)
    ret += 20 * sim_xp;

  ret /= total_xp;
  ret /= 20.0;
  return ret;
}It uses the exact same method as sim2, but the output will be slightly different.

EDIT: BTW, this is my formula for lvl(xp), from solving the quadratic equation:

Code: [Select]

min_xp_for_this_lvl = 500 * lvl + 100 * (lvl * (lvl -1))/2
interpolated_level(xp) = (-9.0 + sqrt(81.0 + (2.0*xp)/25.0)) /2
It gives exactly the same results as Splinterzes formula, but I don't know why the two are equivalent.

[thistleknot]

Okay, sobered up (man, too much tequila...)

So.  I realized why my question was stupid.  I was thinking that my formula and splinterz were doing something correct, but as you pointed out, sim1 is truncating the xp to the starting point for each level?  Which is why they are getting values NEAR each other (splinterz code, and mine), but are not the same as sim2.

I guess what I'm hoping to resolve is how to make it work like sim2, because I am having a hard time reading it.  But I think I might be able to get it up.  I think I just need to add to the current xp (of sim1) the interpolated amount of xp no?

I'm leaning more towards sim2, merely because an interpolated value more represents a dwarf's actual progress, combined with averaging it with just a dwarf's interpolated level, this will "reduce" the effect of sim2.  I'm not sure why you like sim1 more than sim2.

btw,
when I mentioned that i was thinking rate of 0 to 100 could be .01 to .5, and 100 to 500 could be .5 to 1, I was thinking along the lines of multiplicative inverse (and mentioned exponential methods), but I'm not so good with math as you are.  I was trying to create some #'s using logs, but it wasn't working.  that's why I was like "this is so beautiful"

update:
i see now that ur code isnt adding xps.... i think u mentioned that.

[gn0rt0n]

Hi everyone. I wanted to say thanks for all your hard work. I am amazed at the effort you all put in to this project. It is a must-have for me as I play DF.

I'm still quite new to DF, but have put a lot of time into research as I generally enjoy meta-gaming as much or more than the actual game itself. I like to work to automation as much as possible and don't like the overwhelming micro of certain tasks.

To this end, my games usually end after the first couple large migrant waves since I now tend to start to micro job management. I think that Optimization Plans are exactly what I am looking for. However, I would like to look over some sample plans from others on the forum to get a feel for how others are implementing the priority/ratio system.

If any of you have some plans you would be willing to share, I would love to look them over. As I mentioned, just having the auto assign feature is only part of what I am looking for. I want to understand the logic behind other players priority/ratio system as I feel I could be a bit more efficient.

Also, I love the request from a few days ago to add Exclude options for those in Squads in addition to active Military. I think that would help my calculations as well.

Thanks again,
G

[Maklak]

sim1 vs sim2:
I like sim1 more for 2 reasons:
1) The returned numbers are lower, so the results for all kinds of xp and rates should be slightly more spaced out. This is a good thing for comparison.
2) sim1 works a bit like the game. Yes, you have xp, but until a dwarf makes it to another level, his speed and quality and chance to hit and other things are based (presumably) on the current skill level, not some interpolated value. sim1 pretty much thinks "OK, so you get this much utility form this level, this much from the next one, this one from the level after that and so on." Then it averages that.

I made sim1 and sim2 as close as possible. Less than 10 lines of code are different between them. To make sim2 that looks more like sim1, I gave you some code recently, namely this:

Code: [Select]

    double low = getLevelFromXP(curr_xp);
    double high = getLevelFromXP(curr_xp + gap_xp);
    double avg_level = (low + high) / 2.0;There is no reason to make sim1 work like sim2 when you already have sim2 and the above replacement for it's interpolation.

i see now that ur code isnt adding xps.... i think u mentioned that.

Adding them to what? The "beautiful" formula actually subtracts current xp, while sim1 and sim2 add xp just fine.

gn0rt0n, I don't fully understand your question, but I think the answer might have something to do with a few last pages in this thread. We were discussing recommendations for jobs and sorting by skills and other factors quite furiously.

[thistleknot]

maklak:

your very original formula did a 500 run loop of adding factored xps (60 * rate).  Thats what i was referring to when i said sim1 and sim2 don't seem to be adding xps (everytime a new xp is calculated (at least for sim1), it's only the rounded down version of the next level (i.e. based on the array).  I was trying to figure out where I could include an option to add accrued xp (you mentioned that my modded sim1 formula starts with an interpolated level, but afterwards, all the levels are rounded down (xp wise)), I was trying to remedy that, but your sim formulas r doing geometric "shortcuts"

gn0rt0n: is referring to the labor optimizer (my latest and greatest contribution to dt, of course Splinterz recoded it optimally , I've been going on for months about it on this thread using a over complicated spreadsheet, but finally had the balls to sit down and hard code it in).

And the ability of the labor optimizer to exclude dwarfs assigned to squads vs just active militia (something I was bringing up to Splinterz attention when he was originally coding it, but since I knew the filter script to filter by active squads, it was a non issue for me, but it seems to be an issue for those who don't know how to do the scripting).

Here's my exported labor plan (note, a priority merely factors a dwarf's raw fit % down vs other roles)

Spoiler (click to show/hide)

Code: [Select]

New Plan,20,100,50,0,0,0,0
Engraver,12,0.6438,1
Stone Crafter,53,0.613,0.75
Lyemaker,62,0.6438,0.75
Trapper,25,0.5333,0.75
Fisherdwarf,41,0.3851,1.25
Miller,34,0.5171,0.75
Miner,0,0.9771,2.5
Armorsmith,47,0.7929,1
Carpenter,11,0.826,1.25
Potash Maker,61,0.1676,0.75
Soaper,31,0.2131,0.75
Siege Operator,58,0.8461,1.25
Leatherworker,27,0.9134,1.25
Bone Carver,54,0.613,0.75
Milker,37,0.0243,0.75
Mechanic,60,0.7433,1.25
Wound Dresser,21,0.7433,0.75
Cook,38,0.5495,1.25
Clothier,33,0.1379,1.25
Bowyer,59,0.7067,1
Siege Engineer,57,0.3078,1.25
Tanner,28,0.3078,1
Cheesemaker,36,0.1379,0.75
Animal Caretaker,16,0.0243,1
Farmer,39,0.9343,2
Shearer,66,0.0243,0.75
Animal Trainer,15,0.265,1
Weaver,32,0.2011,1.25
Wood Burner,64,0.0243,1.5
Woodcutter,10,0.4506,1.5
Plant Gatherer,40,0.5495,2.5
Bone Doctor,19,0.7433,0.75
Weaponsmith,46,0.7929,1
Hunter,44,0.6438,1
Suturer,20,0.7433,0.75
Gem Cutter,50,0.8461,1
Surgeon,18,0.7433,0.75
Mason,13,0.826,1.25
Wood Crafter,52,0.613,0.75
Fish Cleaner,42,0.3228,1.25
Brewer,29,0.9108,1
Chief Medical,17,0.826,1
Dyer,63,0.1081,1.25
Butcher,24,0.265,1.25
Thresher,35,0.7433,1
Spinner,67,0.3078,1.25
Furnace Operator,45,0.0243,1.25



Update:
Maklak, okay, I think I see what I need to change
ret += xp_gap * curr_level;

to something else, in sim2 it's
ret += xp_gap * avg_level;

so I'm thinking I need to add the xp to curr_level but interpolate curr_level?

arghhh... not sure, because then the level is still reset in the following statement
curr_xp = xp_levels[int(curr_level)];

okay, I think I got it... it's ret that I need to mess with

ah, nm.  I have no idea what to mess with to try to get sim1 to be more like sim2, but using actual increases in xp vs the geometric methods...  I'm not really sure what some of these different xp values are supposed to represent.

I guess sim_xp is supposed to be the distance from 29k?  But is updated using only the gp gap to next level?

Is this measuring the avg # of jobs to reach next level or something?

update:
okay. i see ur code on using splinterz formula.
its doing one iteration per level until it hits 20. that should help me figure things out.

[Maklak]

your very original formula did a 500 run loop of adding factored xps (60 * rate).  Thats what i was referring to when i said sim1 and sim2 don't seem to be adding xps (everytime a new xp is calculated (at least for sim1), it's only the rounded down version of the next level (i.e. based on the array).  I was trying to figure out where I could include an option to add accrued xp (you mentioned that my modded sim1 formula starts with an interpolated level, but afterwards, all the levels are rounded down (xp wise)), I was trying to remedy that, but your sim formulas r doing geometric "shortcuts"

Ah yes, my original formula was quick and dirty and I mentioned, it had obvious optimisations by substituting division with addition. For example when you have 10000 xp, at 60 xp per job and 130% rate, you get (11000 - 10000) / ((60 * 130)/100) = 12, but since (((11000 - 10000) % ((60 * 130)/100)) != 0) you need more than 12 jobs to get to next level, so it it actually 13 jobs at level 10 to get to level 11.
So instead of adding ((60 * 130)/100) xp 13 times, you can multiply it by 13, add it once, add 10 13 times to the sum of levels and subtract 13 from the number of iterations left. This is more complex than a simple addition loop, but works the same and for a large number of additions it should be faster. Oh yeah, and you want to check if ((60 * 130)/100) == 0 and treat it as a special case.

Here's my exported labor plan

You haven't written what those numbers are.

I thought this was quote self-explanatory, but I'll add even more comments.

Code: [Select]

double simulate_skill_gain(int xp, int rate)
{ if (xp >= max_xp)
    return 20.0 / 20.0;    // Already at max level. Getting more XP won't change the skill level.

  if (rate == 0)
    return get_level_from_xp(xp) / 20.0;       // Obviously stays the same.

  int sim_xp = max_xp;  // This is how much xp will be simulated. 29k seems like a good value to me. 10000 or 20000 might also work.
  sim_xp = (sim_xp / 100.0) * rate; // This is how much XP will go towards skill learning after factoring in the rate.
  int total_xp = sim_xp;                         // This is just used for division at the end.
  double ret = 0.0;                               // return value, based on the average level for this simulation.
  int curr_level = get_level_from_xp(xp);   // current (non-interpolated) level for our xp.
  int curr_xp = xp;                               // current xp in that skill.

  while ((sim_xp > 0) && (curr_level < 20))     // while there is xp to simulate gaining skill and not yet legendary +5
  { int xp_gap = xp_levels[curr_level+1] - curr_xp;         // How much XP till next level?
    if (xp_gap > sim_xp)               // If we run out of sim_xp before reaching the next level, we just use it all up and not get to next level.
      xp_gap = sim_xp;
    ret += xp_gap * curr_level;   // We stay on this level until enough xp is accumulated, so we use that xp as the weight for current level.
    curr_level++;                          // This is only correct if we make it to the next level, but if we don't the loop ends here anyway. 
    curr_xp = xp_levels[curr_level];  // This is the lowest amount of xp on next level, but really curr_xp += xp_gap;
    sim_xp -= xp_gap;                  // We subtract the amount of xp spent on this level from the total number of xp to simulate.
/* More correct formulas here, but since the loop ends when sim_xp runs out, what I have here now works just fine:
    ret += xp_gap * curr_level;
    curr_xp += xp_gap;
    curr_level = get_level_from_xp(curr_xp);  // Function call here, so you might preffer the ++.
    sim_xp -= xp_gap;
*/
  }

  if (sim_xp > 0)
    ret += 20 * sim_xp;    // Any leftover sim_xp means Legendary +5, so we take that into account.
  ret /= total_xp;            // Divide by total xp to get the average level for this simulation.
  ret /= 20.0;                // Normalise to <0, 1>
  return ret;
}


ah, nm.  I have no idea what to mess with to try to get sim1 to be more like sim2, but using actual increases in xp vs the geometric methods...  I'm not really sure what some of these different xp values are supposed to represent.

I have no idea why to mess with sim1 to be more like sim2 when you already have sim2 and I've even given you a sim3, which is sim2 using the sqrt formula for level from xp.
I use multiplication instead of addition, because:
1) It is faster for a large number of additions.
2) The "pseudo-integral" methods (sim1 and sim2) don't care if you get xp in chunks of 60, 10 or whatever else. Splinterzes criticism of the "add 60 500 times" method stemmed from things like military skill gains being erratic. sim1 and sim2 pretend that xp is earned in chunks of 1, but the total amount of it gained is governed by xp rate.
As I already told you, this introduces two errors:
1) Rounding of XP rates. I'm pretty sure the actual xp used after factoring in xp rates stays integer, so for exaple 104% * 60 is 62 and not 62.4. This can't be avoided by sim1 and sim2, because we don't know how much is 60, but should be relatively small and therefore the outcomes for sims with different methods should be in correct order, regardless of method used, except for very close values.
2) Rounding around levels. When you are very close to the next level, sim1 and sim2 will make that small amount count towards this level and jump to the next one. The "add 500 times" method will make 1 job count towards this level and add the xp, landing us well inside the next level.

Trying to fix something you don't understand is a recipe for a disaster.

[thistleknot]

thx. ur clarifications helped a lot.  i knew it had something to do with jobs till next level. thanks again

if u import the csv sheet into a labor optimization plan. the numbers become clear.
in some order its name of role (actually the name is not so important as it can be anything, the labors r actually assigned via labor id, this field is merely a descriptor), then the labors id, the ratio to assign (translates to a number based on dwarves selected) and a priority (ie factor to reduce a roles % by, which is used in relation to other roles/jobs)

update:
maklak:
now that I see your formula, I see trying to carry xp over into next iteration was silly, since each iteration starts at it's own level (although, starting at the midpoint of the 1st iteration kind of makes sense, but since that's already considered in the calculation... maybe not)

I had some ideas, the first one I think is a non starter, but what about a sim3 that measured average level each iteration from current to level 20 (i.e. distance from max skill), I figured the end result would be a higher rating, which probably wouldn't serve any purpose.

secondly, I don't know if this formula would help in optimizing your formula's.
XP=((50 * LEVEL) * (LEVEL + 9))

Which is very interesting, such a simple foil formula, basic algebra (not that I recall any of it).  Splinterz thought it wouldn't work for getting interpolated levels, but it does.  I don't know if it would help identifying the curve for your calculations, either way, it does away with an array, and splinterz other formula does away with the need for your binary search (which I ALSO thought was neat, never seen it, but makes perfect sense.

Update:

wow, the formula for xp is so friggin simple
500 xp for 1st level, +100/lvl (+ initial 500) for every level afterwards...

Update:
Here's maklak's original code with his updated posts (lacking mine)

Spoiler (click to show/hide)

Code: [Select]

#include<stdio.h>
#include <math.h>

const int max_xp = 29000;

const int xp_levels [] = {0, 500, 1100, 1800, 2600, 3500, 4500, 5600, 6800, 8100, 9500,
11000, 12600, 14300, 16100, 18000, 20000, 22100, 24300, 26600, 29000};
// xp(lvl) = 500*lvl + 100 * (lvl * (lvl -1)) / 2
// const int lev_size = sizeof(xp_levels) / sizeof(int);   // 21 , unused

double getLevelFromXP (int xp)
{ return (double(xp) /(225.0f + 5.0f*sqrt(2025.0f + (2.0f*double(xp)))));
}

int get_level_from_xp(int xp)   // Uses busection method. Assumes xp >= 0.
{ if (xp >= max_xp)
    return 20;

  int min = 0;
  int max = 20;
  int current = 10;

  while (1)
  { if (xp < xp_levels[current])
    { max = current;
      current = (min + max)/2;
    } else if (xp > xp_levels[current+1])
    { min = current;
      current = (min + max)/2;
    } else
      return current;
  }
}

/* Simulation method for judging a dwarf's skill level and rate.
   Simple version of the while loop, without the interpolation.
   Inputs: dwarf's current xp and learning rate for a skill.
   Output: rating based on simulating gaining experience in that skill. In <0, 1>
*/
double simulate_skill_gain(int xp, int rate)
{ if (xp >= max_xp)
    return 20.0 / 20.0;    // Already at max level. Getting more XP won't change the skill level.

  if (rate == 0)
    return get_level_from_xp(xp) / 20.0;       // Obviously stays the same.

  int sim_xp = max_xp;  // This is how much xp will be simulated. 29k seems like a good value to me. 10000 or 20000 might also work.
  sim_xp = (sim_xp / 100.0) * rate; // This is how much XP will go towards skill learning after factoring in the rate.
  int total_xp = sim_xp;                         // This is just used for division at the end.
  double ret = 0.0;                               // return value, based on the average level for this simulation.
  int curr_level = get_level_from_xp(xp);   // current (non-interpolated) level for our xp.
  int curr_xp = xp;                               // current xp in that skill.

  while ((sim_xp > 0) && (curr_level < 20))     // while there is xp to simulate gaining skill and not yet legendary +5
  { int xp_gap = xp_levels[curr_level+1] - curr_xp;         // How much XP till next level?
    if (xp_gap > sim_xp)               // If we run out of sim_xp before reaching the next level, we just use it all up and not get to next level.
      xp_gap = sim_xp;
    ret += xp_gap * curr_level;   // We stay on this level until enough xp is accumulated, so we use that xp as the weight for current level.
    curr_level++;                          // This is only correct if we make it to the next level, but if we don't the loop ends here anyway.
    curr_xp = xp_levels[curr_level];  // This is the lowest amount of xp on next level, but really curr_xp += xp_gap;
    sim_xp -= xp_gap;                  // We subtract the amount of xp spent on this level from the total number of xp to simulate.
/* More correct formulas here, but since the loop ends when sim_xp runs out, what I have here now works just fine:
    ret += xp_gap * curr_level;
    curr_xp += xp_gap;
    curr_level = get_level_from_xp(curr_xp);  // Function call here, so you might preffer the ++.
    sim_xp -= xp_gap;
*/
  }

  if (sim_xp > 0)
    ret += 20 * sim_xp;    // Any leftover sim_xp means Legendary +5, so we take that into account.
  ret /= total_xp;            // Divide by total xp to get the average level for this simulation.
  ret /= 20.0;                // Normalise to <0, 1>
  return ret;
}

/* Simulation method for judging a dwarf's skill level and rate.
   This version of the while loop is interpolating the level. It may be smoother.
   Inputs: dwarf's current xp and learning rate for a skill.
   Output: rating based on simulating gaining experience in that skill. in <0, 1>
*/

double simulate_skill_gain_slopes_2(int xp, int rate)
{ if (xp >= max_xp)
    return 20.0 / 20.0;
  if (rate == 0)
    return getLevelFromXP(xp) / 20.0;       // Obviously stays the same.

  int sim_xp = max_xp;    // 29k seems like a good value to me.
  sim_xp = (sim_xp * rate) / 100; // This is how much XP will go towards skill learning.
  int total_xp = sim_xp;
  double ret = 0.0;
  int curr_level = getLevelFromXP(xp); // Convert to int
  int curr_xp = xp;

  while ((sim_xp > 0) && (curr_level < 20))
  { int xp_gap = xp_levels[curr_level+1] - curr_xp;         // How much XP till mext level?
    if (xp_gap > sim_xp)
      xp_gap = sim_xp;
    double low = getLevelFromXP(curr_xp);
    double high = getLevelFromXP(curr_xp + xp_gap);
    double avg_level = (low + high) / 2.0;  // Average scaled level for this iteration.
    ret += xp_gap * avg_level;
    curr_level++;
    curr_xp = xp_levels[curr_level];
    sim_xp -= xp_gap;
  }
  if (sim_xp > 0)
    ret += 20 * sim_xp;

  ret /= total_xp;
  ret /= 20.0;
  return ret;
}

double simulate_skill_gain_slopes(int xp, int rate)
{ if (xp >= max_xp)
    return 20.0 / 20.0;

  if (rate == 0)
    return get_level_from_xp(xp) / 20.0;       // Obviously stays the same.

  int sim_xp = max_xp;    // 29k seems like a good value to me.
  sim_xp = (sim_xp / 100.0) * rate; // This is how much XP will go towards skill learning.
  int total_xp = sim_xp;
  double ret = 0.0;
  int curr_level = get_level_from_xp(xp);
  int curr_xp = xp;

  while ((sim_xp > 0) && (curr_level < 20))
  { int xp_gap = xp_levels[curr_level+1] - curr_xp;         // How much XP till mext level?
    if (xp_gap > sim_xp)
      xp_gap = sim_xp;
    double low = (0.0 +curr_xp -xp_levels[curr_level]) / (xp_levels[curr_level+1] -xp_levels[curr_level]);
    double high = (0.0 +curr_xp + xp_gap -xp_levels[curr_level]) / (xp_levels[curr_level+1] -xp_levels[curr_level]);
    double avg_level = curr_level + (low + high) / 2.0;   // Average scaled level for this iteration.
    ret += xp_gap * avg_level;
    curr_level++;
    curr_xp = xp_levels[curr_level];
    sim_xp -= xp_gap;
  }

  if (sim_xp > 0)
    ret += 20 * sim_xp;
  ret /= total_xp;
  ret /= 20.0;
  return ret;
}

/* An approximate formula to judge how close a dwarf is to legendary +5
   Inputs: dwarf's current xp and learning rate for a skill.
   Output: rating in <0, 1>
*/
double skill_rate_eval(int xp, int rate)
{ if (xp >= max_xp)
    return 1.0;
  rate = (rate < 1)? 1 : rate; // 0 or 1 doesn't really matter.
  double jobs_left = (29000.0 - xp) / rate;  // Ignore division by 60.
  double normalised_XP = 0.5 * (1.0 - jobs_left/290.0);    // Scaled to <-49.5, 0.5>
  if (normalised_XP >= 0)
    normalised_XP += 0.5;    // Values <0, 0.5> reserved for those suitable for a task.
  else
    normalised_XP = -0.5 / (normalised_XP - 1.0);
  return normalised_XP;
}

/* I could add fancy options to use this from command line or reading from a file,
   but that's not really necessary. Just use $ rates <file_in.txt >file_out.txt */
int main(void)
{ int xp, rate;
  int ret = 2;

  printf("xp, rate, slill_level, sim1, sim2, interp1\n");
  while ((ret = scanf("%d %d", &xp, &rate)) == 2)
    printf("%d, %d, %d, %f, %f, %f\n", xp, rate, get_level_from_xp(xp), simulate_skill_gain(xp, rate), simulate_skill_gain_slopes(xp, rate), skill_rate_eval(xp, rate));

  return ret;
}



[thistleknot]

if traits modify eligibility for skills. maybe they should affect the rate of learning?

[Maklak]

Splinterz said sim1 is good enough, so perhaps it it time to move on. We've spent what, 2 weeks and 3 pages tuning this? I'm sure there are other things on the TODO list for Dwarf Therapist.

if traits modify eligibility for skills. maybe they should affect the rate of learning?

They don't affect the rate of learning. AFAIK you get the same XP regardless of stats and traits. Well, there are exceptions, like honesty (straightforwardness?) affecting the ability to lie, but those are few. They do affect how good a Dwarf is at a job, so they should be factored in separately. I actually prefer the current system with arithmethic means for this than what you propose.

I had some ideas, the first one I think is a non starter, but what about a sim3 that measured average level each iteration from current to level 20 (i.e. distance from max skill)

I would have to see it to comment on it, but I think sim1 is good enough.

secondly, I don't know if this formula would help in optimizing your formula's.

// xp(lvl) = 500*lvl + 100 * (lvl * (lvl -1)) / 2;
// xp(lvl) = 50 * lvl * (10 + 2*(lvl -1)/2);
// xp (lvl) = 50 * lvl * (lvl + 9);   // Is legit.
// max_xp_gap(lvl) = 500 + 100*lvl;
Hm... to find the initial level from xp, you could use Splinterzes formula for level_from_xp, then round it to int.
To get the xp gap to next level you could use this.
Yes, you could skip the array and binary search with these two. There would be less code, but more comments that way.

EDIT:

so i was proposing modifying the rate (ie learning rate) of those skills to 0 if those skills couldnt be learned...

Ah, then I agree. If you can't learn liar because of honesty, set the learning rate for that to 0 (inside DT, not inside game) and proceed.

[thistleknot]

maklak
i wasnt proposing to change the arithmetic mean method. however, after reading the wiki. traits dont affect eligibility of labor skills but social skills that can be learned.  so i was proposing modifying the rate (ie learning rate) of those skills to 0 if those skills couldnt be learned...

but... since there social skills and not labor skills... they could only really affect some noble positions such as some mwmod nobles like the psychatrist that rely on social skills

Splinterz:
ctrl-a for selecting all currently listed dwarfs would be cool (I use script filters to exclude militia, nobles, and this simple hack would let me select the rest of dwarf's listed)