Bay 12 Games Forum

Please login or register.

Login with username, password and session length
Advanced search  
Pages: 1 ... 181 182 [183] 184 185 ... 192

Author Topic: Dwarf Therapist (LATEST 0.6.12 5/23/12 see first post)  (Read 870499 times)

Maklak

  • Bay Watcher
    • View Profile
Re: Dwarf Therapist (LATEST 0.6.12 5/23/12 see first post)
« Reply #2730 on: January 15, 2013, 11:57:26 am »

I generalised the simulation method of factoring in both current skill level and skill rate. I'm pretty sure this method is fine, but the code itself may need cleaning up. It may not even compile the first time and I might have missed some bugs.

Code: [Select]
const int max_xp = 29000;

const int xp_levels [] = {0, 500, 1100, 1800, 2600, 3500, 4500, 5600, 6800, 8100, 9500,
11000, 12600, 14300, 16100, 18000, 20000, 22100, 24300, 26600, 29000};


// xp(lvl) = 500*lvl + 100 * (lvl * (lvl -1)) / 2

// const int lev_size = sizeof(xp_levels) / sizeof(int);   // 21 , unused

int get_level_from_xp(int xp)   // Uses busection method. Assumes xp >= 0.
{ if (xp >= max_xp)
    return 20;

  int min = 0;
  int max = 19;
  int current = 10;

  while (1)
  { if (xp < xp_levels[current])
    { max = current;
      current = (min + max)/2;
    } else if (xp > xp_levels[current+1])
    { min = current;
      current = (min + max)/2;
    } else
      return current;
  }
}

/* Simulation method for judging a dwarf's skill level and rate.
   Inputs: dwarf's current xp and learning rate for a skill.
   Output: rating based on simulating gaining experience in that skill.
*/
double simulate_skill_gain(int xp, int rate)
{ if (xp >= max_xp)
    return 20;

  if (rate == 0)
    return get_level_from_xp(xp);       // Obviously stays the same.

  int sim_xp = max_xp;    // 29k seems like a good value to me.
  sim_xp = (sim_xp / 100.0) * rate; // This is how much XP will go towards skill learning.
  int total_xp = sim_xp;
  double ret = 0.0;
  int curr_level = get_level_from_xp(xp);
  int curr_xp = xp;


/*  // Version of the while loop interpolating the level. It may be smoother.
  while ((sim_xp > 0) && (curr_level < 20))
  { int xp_gap = xp_levels[curr_level+1] - curr_xp;         // How much XP till mext level?
    if (xp_gap > sim_xp)   
      xp_gap = sim_xp;
    double low = (0.0 +curr_xp -xp_levels[curr_level]) / (xp_levels[curr_level+1] -xp_levels[curr_level]);
    double high = (0.0 +curr_xp + xp_gap -xp_levels[curr_level]) / (xp_levels[curr_level+1] -xp_levels[curr_level]);
    double avg_level = curr_level + (low + high) / 2.0;   // Average scaled level for this iteration.
    ret += xp_gap * avg_lvl;
    curr_level++;
    curr_xp = xp_levels[curr_level];
    sim_xp -= xp_gap;
  }
*/

//  Alternate, simple version of the while loop, without the interpolation.
  while ((sim_xp > 0) && (curr_level < 20))
  { int xp_gap = xp_levels[curr_level+1] - curr_xp;         // How much XP till mext level?
    if (xp_gap > sim_xp)   
      xp_gap = sim_xp;
    ret += xp_gap * curr_level;
    curr_level++;
    curr_xp = xp_levels[curr_level];
    sim_xp -= xp_gap;
  }

  if (sim_xp > 0)
    ret += 20 * sim_xp;
  ret /= total_xp;
  return ret;
}
Logged
Quote from: Omnicega
Since you seem to criticize most things harsher than concentrated acid, I'll take that as a compliment.
On mining Organics
Military guide for FoE mod.
Research: Crossbow with axe and shield.
Dropbox referral

thistleknot

  • Bay Watcher
  • Escaped Normalized Spreadsheet Berserker
    • View Profile
Re: Dwarf Therapist (LATEST 0.6.12 5/23/12 see first post)
« Reply #2731 on: January 15, 2013, 06:44:19 pm »

I think the answer is in a exponential curve of some sort...

we shouldn't focus on raw 60 exp points, but some sort of curve that is adjusted from 1 to 100% at a low rate (i.e. exponential), and then jumps from 100% to 500% (at a higher rate), of course this has to be considered with the amount of experience a dwarf currently has in that skill... I'm not a math major, but I know an exponential fit based on rate would be the answer.

Update:
I think I'm wasting too much time on this and not playing...

anyways, here's my latest failed attempt on a program (maklak's simulated approach) I don't think best explains this problem, I still think an exponential approach is best, yet I have no idea how to implement it
Spoiler (click to show/hide)

heres an idea

rate:
0 to 100% = 0 to 50% (2:1 ratio)
100% to 500% = 50% to 100% (5:1 ratio)

exp = xp / 29000

factored = rate * exp

update:
nm... bad idea. .5 x 100% xp = 50%
« Last Edit: January 15, 2013, 11:39:54 pm by thistleknot »
Logged

Maklak

  • Bay Watcher
    • View Profile
Re: Dwarf Therapist (LATEST 0.6.12 5/23/12 see first post)
« Reply #2732 on: January 16, 2013, 02:39:42 am »

Sigh.

Code: [Select]
            lowerValue = exp [x-1];
            upperValue = exp [x]-1;
exp[ x ] is the threshold value and the lowest value for a level, not the highest value for a threshold level. Even so, I gave you a more elegant function to get_skill_level_from_XP(int xp) based on bisection. Didn't it work?
Oh, and that array with xp levels should be outside of any function. Otherwise at least make it static.

Quote
we shouldn't focus on raw 60 exp points, but some sort of curve that is adjusted from 1 to 100% at a low rate (i.e. exponential), and then jumps from 100% to 500% (at a higher rate), of course this has to be considered with the amount of experience a dwarf currently has in that skill... I'm not a math major, but I know an exponential fit based on rate would be the answer.
No, I don't think exponential function is the answer here.

The rest of the simulation program you posted is based on assuming that you get 60 XP per job. A newer version I posted yesterday doesn't care about it. It just multiplies total XP by xp_rate (which may introduce errors due to rounding) and then pretends you get XP in chunks of 1. Therefore it should be quite accurate, regardless of XP per job. I estimated less than 5% error based on levelling up and less than 10% error based on rounding, assuming you normally get at least 10 XP per job.

Quote
heres an idea

rate:
0 to 100% = 0 to 50% (2:1 ratio)
100% to 500% = 50% to 100% (5:1 ratio)

exp = xp / 29000

factored = rate * exp

update:
nm... bad idea. .5 x 100% xp = 50%
Dammit, I told you several times already, why this is a bad idea and you're still fixated on it. You didn't even learn that you should operate on xp left to Legendary +5 as skill rate has no influence on the xp that's already there.
Your method of scaling 1-500 to 0-100 introduces an undesired property: slopes around 100% are different, so 80 skill rate is 40% and 120 skill rate is 55%. Now, 120/80 = 1.5, while 55/40 = 1.375. This is not exactly what you want here.
I've already given you a method to do this using 1/x. It is not perfect either, but at least it uses skill rates in tandem with skill level, as they should be.
« Last Edit: January 16, 2013, 07:31:28 am by Maklak »
Logged
Quote from: Omnicega
Since you seem to criticize most things harsher than concentrated acid, I'll take that as a compliment.
On mining Organics
Military guide for FoE mod.
Research: Crossbow with axe and shield.
Dropbox referral

thistleknot

  • Bay Watcher
  • Escaped Normalized Spreadsheet Berserker
    • View Profile
Re: Dwarf Therapist (LATEST 0.6.12 5/23/12 see first post)
« Reply #2733 on: January 16, 2013, 06:48:22 am »

maklak: no need to get testy. im just trying sh** out (u have a bad habit of commenting negative things even when i posted/realized it didnt work). i have a learning process here.  i havent tried ur latest formula yet.

and yes i did learn to use distance from legendary.

if u had read my earlier post correctly i was messing w trying to use this as a weight against raw exp/29k which i still think is a good idea (which i took queue from you) but like i said i havent tried ur latest invention.

to be honest. it sounds like youve hit on something i was thinking/ said earlier. 60 xp shouldnt matter. i had no idea how to implement it so good on you.

i just realised your original formula
http://www.bay12forums.com/smf/index.php?topic=66525.msg3945110#msg3945110

if i add 99 and then divide by 100
results in a .01 to 1 scale for rates of 1 to 500

i dont know if this will resolve much but it could be used as a weight.

just fyi.
ill mess w ur latest formula laters
« Last Edit: January 16, 2013, 07:08:24 am by thistleknot »
Logged

Maklak

  • Bay Watcher
    • View Profile
Re: Dwarf Therapist (LATEST 0.6.12 5/23/12 see first post)
« Reply #2734 on: January 16, 2013, 07:50:07 am »

Quote
maklak: no need to get testy. im just trying sh** out (u have a bad habit of commenting negative things even when i posted/realized it didnt work). i have a learning process here.  i havent tried ur latest formula yet.
Well, I suppose maybe I owe you an apology.

For both:
a) Skill level
b) Skill rate, factoring in skill level
You want for the current system with weighted arithmetic means:
1) a value, mapped to <0, 1> and bigger means better.
2) a weight, chosen arbitrarily, but adjustable by the user.

For skill level, I think Splinterz already mentioned having a value that works. I submitted a sensible proposal, where you sort by interpolated skill levels, which may even be how it currently works.

For Skill Rates, I submitted the generalised simulation approach and a method for using 1/x for unfavourable rates.

Weights for aggregating those values into your goal function are something entirely different.

Maybe I should just write a program myself, that takes a file with lines "xp rate\n" as input and prints values for different forumlas in the output.  I'm pretty sure I gad a working gcc on this computer. Maybe even one of them fancy text editors that can highlight syntax.

EDIT: Here is my program with these formulas. sim1 (without slopes) seems to be the most sensible: http://pastebin.com/XAypc3Ka
It can accept a file with "xp rate\n" in each line and produce a CSV that you can use for Libre Office Calc or whatever. I didn't bother with all the fancy things, like options and reading / writing to / from a file, so you have to use it like this: "$ rates <input.txt >output.txt". It ends when a line contains anything but two integers. I didn't even add any safeguards against negative numbers. It is easily extensible with other formulas. Just add more functions.
« Last Edit: January 16, 2013, 08:40:50 am by Maklak »
Logged
Quote from: Omnicega
Since you seem to criticize most things harsher than concentrated acid, I'll take that as a compliment.
On mining Organics
Military guide for FoE mod.
Research: Crossbow with axe and shield.
Dropbox referral

splinterz

  • Bay Watcher
    • View Profile
    • Dwarf Therapist Branch
Re: Dwarf Therapist (LATEST 0.6.12 5/23/12 see first post)
« Reply #2735 on: January 16, 2013, 09:30:52 am »

thanks for the additional formulas Maklak.

so the plan is to fix the sorting by separating the skill rates from the skill level. the distinct column sorting methods would be as previously described: total experience, skill rate and whatever else. i don't know if i'll include an option to sort based on a merge of skill rate and experience as it's not entirely intuitive what's happening.

for the role calculations, i see less of an issue combining the skill rate and the skill experience as the roles are already something of a black box. if the results of the simulated method look good, then i'll use it. i haven't tested it yet, but my intent is to implement the simulated rating in place of the current skill rate/level weight system.

the only reason i see to use both a current xp/total xp rating and a merged xp/skill rate rating with weights applied would be to allow the user to adjust, or flat out remove the skill rate addition to the skill rating for the role. since the skill rate has a direct and significant impact on the experience gained and resulting level, i'm not sure this should even be optional.

those are my current thoughts and intended direction, if this doesn't seem sound or i'm misunderstanding anything please let me know.

Maklak

  • Bay Watcher
    • View Profile
Re: Dwarf Therapist (LATEST 0.6.12 5/23/12 see first post)
« Reply #2736 on: January 16, 2013, 10:09:55 am »

Splinterz, looks like our ideas of how this should work are converging.

If I get the options to sort a column by either skill level or skill rate, well, I'll sort by skill rate (which should be visible in tooltips even for 0 xp), then look at the top and see, if there's anyone with good skill level. There should be few enough for me to decide if I prefer lvl 5, 80% or lvl 0 120%, even without any formula that combines them (but it would be more handy for skill rates than just rates).

Quote
the only reason i see to use both a current xp/total xp rating and a merged xp/skill rate rating with weights applied would be to allow the user to adjust, or flat out remove the skill rate addition to the skill rating for the role. since the skill rate has a direct and significant impact on the experience gained and resulting level, i'm not sure this should even be optional.
That was the idea to combat your lvl 8.1 50% rave vs lvl 0 100% rate problem. You can have skill level formula and skill level plus skill rate formula and weight them. With the simulation approach, you could just lower the number of earned XP. Values within <5000, 30000> seem sensible. For lower ones higher skills would win. For higher ones it could go both ways.

Oh and of course anything related to skill rates should only be visible / usable if any rate for any skill is not 100. That would disable it for vanilla and enable it for caste-heavy mods, which may be what you want to avoid screen clutter. The downside is confusing users who use a mod for the first time.
Logged
Quote from: Omnicega
Since you seem to criticize most things harsher than concentrated acid, I'll take that as a compliment.
On mining Organics
Military guide for FoE mod.
Research: Crossbow with axe and shield.
Dropbox referral

thistleknot

  • Bay Watcher
  • Escaped Normalized Spreadsheet Berserker
    • View Profile
Re: Dwarf Therapist (LATEST 0.6.12 5/23/12 see first post)
« Reply #2737 on: January 16, 2013, 10:12:20 am »

Quote
the only reason i see to use both a current xp/total xp rating and a merged xp/skill rate rating with weights applied would be to allow the user to adjust, or flat out remove the skill rate addition to the skill rating for the role. since the skill rate has a direct and significant impact on the experience gained and resulting level, i'm not sure this should even be optional.

well that was my big beef, but if the #'s aren't too ridiculous, it's not a big deal.  I just think that there should be a way for dwarf's who have a significantly higher skill, but marginally lower rate, will be ranked higher, if this is done correctly, then great!

thistleknot

  • Bay Watcher
  • Escaped Normalized Spreadsheet Berserker
    • View Profile
Re: Dwarf Therapist (LATEST 0.6.12 5/23/12 see first post)
« Reply #2738 on: January 16, 2013, 04:27:43 pm »

Maklak
I couldn't figure out how to input values into your app.

I tried at the console (win32):
$ rates input.txt output.txt
rates input.txt output.txt
rate input.txt output.txt

I tried nameofexe.exe input.txt output.txt

input.txt file is

#, #
#, #
#, #

also tried
# #
# #
# #


anything I'm missing?


thanks splinterz

I now figured out rates was the name of your app, not a function call of your program
rates <input.txt >output.txt
worked with no comma's

thanks
« Last Edit: January 16, 2013, 05:27:19 pm by thistleknot »
Logged

thistleknot

  • Bay Watcher
  • Escaped Normalized Spreadsheet Berserker
    • View Profile
Re: Dwarf Therapist (LATEST 0.6.12 5/23/12 see first post)
« Reply #2739 on: January 16, 2013, 06:03:55 pm »

I did some numbers

I updated the numbers (because I noticed a pattern...) originally, had lower levels...

Spoiler (click to show/hide)

here's a chart of columns 5-6 (the default output)

Spoiler (click to show/hide)

It appears that the closer one gets to higher levels, the less the rate counts.  Which I guess is good.  However, rate pretty much dominates lower levels...

Idea #1:

I also did something that maklak would hate.  I wanted to see what would happen if I used these %'s as a basis for factoring a standard % against, respective columns are the last 3.

See chart

Spoiler (click to show/hide)

Not really what I was expecting...  Not dynamic enough (my modification).  I think the formula requires more than a simple a * b to do what I was hoping for...

Idea #2:

However,  I found what I was hoping for, by averaging this factor with Maklak's original numbers.

Check it out.

Spoiler (click to show/hide)

I'm working on getting this done with a way less convoluted formula, but still using Maklak's original

1-(((max xp - current xp)/rate)/290)

I'm hoping to combine it with my idea of scaling 1 to 100 from 0 to 50%, and from 100 to 500 using 50% to 100%...

we'll see if I can get it done before maklak can comment on it.
« Last Edit: January 16, 2013, 11:48:49 pm by thistleknot »
Logged

splinterz

  • Bay Watcher
    • View Profile
    • Dwarf Therapist Branch
Re: Dwarf Therapist (LATEST 0.6.12 5/23/12 see first post)
« Reply #2740 on: January 17, 2013, 02:14:50 am »

Quote from: Maklak
That was the idea to combat your lvl 8.1 50% rave vs lvl 0 100% rate problem. You can have skill level formula and skill level plus skill rate formula and weight them. With the simulation approach, you could just lower the number of earned XP. Values within <5000, 30000> seem sensible. For lower ones higher skills would win. For higher ones it could go both ways.
right, i see exactly what you mean after looking at a few of the outputs from the little simulator program. i also agree that sim1 is looking pretty good!
Quote
Oh and of course anything related to skill rates should only be visible / usable if any rate for any skill is not 100. That would disable it for vanilla and enable it for caste-heavy mods, which may be what you want to avoid screen clutter.
this is actually already done in the last version. it does a check for any significant (>=25%) experience bonuses in the castes and if none are found (vanilla) it hides all skill rate displays and doesn't bother using them in the role calculations. i'll probably adjust that 25% figure or improve the check.

Quote from: Thistleknot
It appears that the closer one gets to higher levels, the less the rate counts.  Which I guess is good.  However, rate pretty much dominates lower levels...
seems the whole purpose of this discussion was to have a model where learning rate was worth less the closer to the legendary +5 experience cap a dwarf is... anyway it seems to me the rest of this averaging with 'regular' (which appears to be current/total xp) is exactly what was meant with combining an experience rating with a simulated rating, so i guess this is proof of concept? again this could be expanded upon to allow a user defined weight or ratio to value the 'regular' vs simulated ratings. that might just be overkill though, i'm not sure it needs to be that flexible.

Maklak

  • Bay Watcher
    • View Profile
Re: Dwarf Therapist (LATEST 0.6.12 5/23/12 see first post)
« Reply #2741 on: January 17, 2013, 02:52:35 am »

I told you my program was quick and dirty, without command line options and what have you and I thought "$ ratings <in.txt >out.txt" was clear enough. Why write functions to use files when you can just redirect input and output streams :P

Quote
seems the whole purpose of this discussion was to have a model where learning rate was worth less the closer to the legendary +5 experience cap a dwarf is... anyway it seems to me the rest of this averaging with 'regular' (which appears to be current/total xp) is exactly what was meant with combining an experience rating with a simulated rating, so i guess this is proof of concept? again this could be expanded upon to allow a user defined weight or ratio to value the 'regular' vs simulated ratings. that might just be overkill though, i'm not sure it needs to be that flexible.

Yes, except for the current skill level I'd use interpolated levels, like 8.1, not however much XP that is. In theory if that was the only criteria the order would be the same, but when you're aggregating with other things, it is the level that matters, not XP. If you sort by XP, then adding 2k at low level may mean 2 or 3 level ups, while at Legendary it may be one. This is an undesired property. Skill level and not XP directly is what decides the quality of items, so skill level is what should be factored in.

Yes, I eventually settled for one formula based on interpolated skill level and one formula based on current xp and skill rate (where skill rate matters less and less the higher the xp), both aggregated into the goal function.

Quote
I also did something that maklak would hate.  I wanted to see what would happen if I used these %'s as a basis for factoring a standard % against, respective columns are the last 3.
Not really what I was expecting...  Not dynamic enough (my modification).  I think the formula requires more than a simple a * b to do what I was hoping for...
Good, now you understand why I was so against it. I foresaw it wouldn't work.

The sim1 formula takes some number of XP, which can be lower than 29000 to get more varied results (see what happens for 5000, 10000, 15000 and 20000, you just need to change "int sim_xp = max_xp;" to whatever you want), multiplies that with the skill rate, then simulates getting that many xp in very small chunks. It calculates the average level by multiplying current level by the amount of xp needed for the next one, adds those up and divides them by total gained XP. Then it normalises that to <0, 1>. This may be the most sensible approach, but the problem of how much XP to earn remains. 29000 is a good value, but it may not be the best value and something less may be better.

Quote
I'm hoping to combine it with my idea of scaling 1 to 100 from 0 to 50%, and from 100 to 500 using 50% to 100%...
we'll see if I can get it done before maklak can comment on it.

I've already commented on it. Anyway, now you can have fun and throw whatever formulas you can think of at the problem and see results, but I very much doubt you'll outperform sim1. I suggest that you make a version of sim1 that accepts sim_xp as the third parameter and run a few tests on that.

It is surprising how coding three functions and a simple loop for main() was something of a breakthrough for this discussion. :D
Logged
Quote from: Omnicega
Since you seem to criticize most things harsher than concentrated acid, I'll take that as a compliment.
On mining Organics
Military guide for FoE mod.
Research: Crossbow with axe and shield.
Dropbox referral

thistleknot

  • Bay Watcher
  • Escaped Normalized Spreadsheet Berserker
    • View Profile
Re: Dwarf Therapist (LATEST 0.6.12 5/23/12 see first post)
« Reply #2742 on: January 17, 2013, 03:27:57 am »

thanks, this has been a learning experience.

I believe your normalizing by dividing by 20 levels.  nice.

I was working on finishing up your coded version originally, but eh, you got it done way faster than I could have.

I hope we can apply a way to merge current level/simulated level to help with ranking dwarfs.

I did a spreadsheet merge using interpolated levels vs xp/29k

I did two types of averaging

one involves factoring (interpolated % * interpolated level %) then adding this to interpolated % again, and dividing by 2.

the second was merely a combination of (interpolated level % + interpolated level %) / 2

updated:
included maklak's interpolated values (I might update his program to include these averages, so I don't have to create them in scalc)
made an error with my original interpolated levels based on interpolated level % (didn't do a (next level exp - current)/(current - lower level xp) ).

Updated again:
had the decimal of the level %'s inverted... doh!  Fixed

Spoiler (click to show/hide)

Hopefully final update:

Here's sim2 averaged with the interpolated level % (which is probably what should be used) compared to the interp distance from legendary averaged with interpolated level %
Note: no factoring graph is shown, as is shown above.
Spoiler (click to show/hide)
« Last Edit: January 17, 2013, 03:39:56 pm by thistleknot »
Logged

Grumpy

  • Escaped Lunatic
    • View Profile
Re: Dwarf Therapist (LATEST 0.6.12 5/23/12 see first post)
« Reply #2743 on: January 17, 2013, 08:54:51 am »

New user trying to figure out a basic feature: How do I see attributes (not traits) in DT?  I have DT 0.6.12 installed on a Mac using Dwarf Builder.
I've seen screenshots of them displayed in the grid for example. I did a search but it seemed to mostly turn up discussion about user-modified versions or advanced questions. I tried filtering based on attributes but it always results in 0 dwarves in the list.
Logged

thistleknot

  • Bay Watcher
  • Escaped Normalized Spreadsheet Berserker
    • View Profile
Re: Dwarf Therapist (LATEST 0.6.12 5/23/12 see first post)
« Reply #2744 on: January 17, 2013, 10:23:30 am »

Quote
New user trying to figure out a basic feature: How do I see attributes (not traits) in DT?  I have DT 0.6.12 installed on a Mac using Dwarf Builder.
I've seen screenshots of them displayed in the grid for example. I did a search but it seemed to mostly turn up discussion about user-modified versions or advanced questions. I tried filtering based on attributes but it always results in 0 dwarves in the list.

attributes are only available in splinterz branch

http://code.google.com/r/splintermind-attributes/

be mindful, if you intend on using it with mods, don't throw the towell in when you see the castes being drawn funny, wait until next release, that'll be fixed (but if your playing vanilla, then disregard)
Pages: 1 ... 181 182 [183] 184 185 ... 192