Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[Scruiser]

I guess what is most surprising is that any random Urist could in theory discover the most complex of topics after only 50 times pondering. In theory.

The example calcs I gave were for only level 1 topics.  For a level 4 topic, it would take 400 time pondering.  And, according to the wiki, there are a few topics that have prerequisite topics, (although most topics have no prerequisites, at least as far as the diagram on the wiki page shows, but this might be a selection bias with research that has prerequisites being very rare to show up, since the wiki page only has 130 of the 300 topics total, i.e. just the ones players have identified before I think).  This isn't really that unreasonable if you ask me.  400 cycles is 400 days, so with various breaks for eating, drinking, and sleeping it is around 1.5 years of continuous pondering before a dwarf even has a chance of making a breakthrough on the level 4 topics.

I think you are misinterpreting some of the wall o' text, I think "However there is also the 2% (1/50 chance) of changing research topics.", is actually for when the right amount of research points is gotten(that is, there's research cycles, which pool, and then can produce research points(a different meter), and it seems each time research points are added to the second pool, the second set of rolls is done.

That actually was how I initially interpreted it (that you need a chance of making a breakthrough, i.e. at least one breakthrough "dice" before the 2% chance of switching topics kicks in), but then I had second thoughts...  I do have empirical predictions for the different possibilities that you may have enough observations in your science to give some initial feedback on:

If unskilled dwarfs regularly switch topics, about once in every 50 days, then that means that the switch topic chance is activating every time pondering, even when they are below threshold research credits (0 breakthrough dice).

If an unskilled dwarf has about even odds of making a breakthrough vs. switching topics, that means the 2% chance only starts activating.

If dwarf that has previously extensively pondered a topic (I guess we could check there research credit with the dfHack script?) has even odds of making a breakthrough vs. switching topics, that means that the 2% chance is there for each breakthrough dice.  (I.e. each additional breakthrough dice rolled also mean an additional roll for the odds of switching topics).  I don't think Toady meant to say this, but the wording wasn't 100% unambiguous.

[therahedwig]

-snip-

I do have empirical predictions for the different possibilities that you may have enough observations in your science to give some initial feedback on:

If unskilled dwarfs regularly switch topics, about once in every 50 days, then that means that the switch topic chance is activating every time pondering, even when they are below threshold research credits (0 breakthrough dice).

Yes, this is correct. I've seen my test scholar switch topics when they were at the start of research, but didn't finish their first ponder, going to sleep, and then getting back and having switched topics..

If an unskilled dwarf has about even odds of making a breakthrough vs. switching topics, that means the 2% chance only starts activating.

Unfinished sentence?

If dwarf that has previously extensively pondered a topic (I guess we could check there research credit with the dfHack script?) has even odds of making a breakthrough vs. switching topics, that means that the 2% chance is there for each breakthrough dice.  (I.e. each additional breakthrough dice rolled also mean an additional roll for the odds of switching topics).  I don't think Toady meant to say this, but the wording wasn't 100% unambiguous.

I think "Also: if they fail to get the breakthrough after the 50-sided rolls, they have a 2% chance of switching topics" does really suggest it happens as a separate check after failing all the 50-sided rolls.

(although most topics have no prerequisites, at least as far as the diagram on the wiki page shows, but this might be a selection bias with research that has prerequisites being very rare to show up, since the wiki page only has 130 of the 300 topics total, i.e. just the ones players have identified before I think)

No, the wiki has all 313~ topics, determined by dfhack, that diagram as well is just the names determined via dfhack, it doesn't actually show the techtree. The dev quotes suggest that topics in one branch determine whether topics in another can be chosen, and it's pretty clear there's a relationship between the astronomy topics in the astronomy branch, and those in the engineering branch. I've also sort of noticed some others, like economic maps always showing up after gaining the topic of maps, and so forth.

I suspect the best way to determine the techtree will be to write a dfhack script to spit out a csv correlation table thing, and if I got this right, if we do this with enough data, then topics should always have about a 100% correlation with themselves and the topics that are prerequisites. Except books kinda mucking it up, but still...

I've mostly been busy trying to figure out what skills are used for what topic right now. I also have enough experience now to understand that topics are chosen based on the highest rank of scholarly skill(once a scholar gains a proper rank in a single scholarly skill, they'll continue on pondering about those). Going through this I noticed that it sometimes requires multiple times of failing to advance a field before they'd get the topic, which is why I asked the fotf question. And then we got the full algorithm. I had not expected that part

[Scruiser]

I caught another oversight in my reading through of Toady's wall of text description... I misread how the odds of a research credit changes as they complete more cycles...

it rolls 0-50 vs. the number of completed cycles minus 50

So the first 50 research cycles have 0 odds of getting any research credit, and the 100th research cycle has 100% odds?  If so, I vastly misread it.  It still takes a while, but this means the first 50 cycles will have no research credit progress, the next 50 cycles will have an average of 50 credits progress (as I calculated) and the 100th and greater cycles will all have 100 credits progress per cycle (twice as fast as i calculated).  I suppose I could try using the dfhacl script you gave to check this? 

Yes, this is correct. I've seen my test scholar switch topics when they were at the start of research, but didn't finish their first ponder, going to sleep, and then getting back and having switched topics..

Toady didn't specify if Dwarfs switch topics after taking breaks, only that when do sequentially cycles of pondering there is a 2% chance to switch topics.  I suppose this could be tested with Dwarfs modded not to need sleep/food/drink and careful monitoring of the number of days they spend pondering (presumably each day corresponds to a cycle of pondering?)

Unfinished sentence?

Opps, I mean that if a dwarf, on average (with no food/drink/sleep breaks?) makes a breakthrough half the time, and switches topics of pondering the other half, then that means that the 2% chance of switching topics only starts getting checked after they have accumulated enough research credit to also have a chance of breakthrough.

No, the wiki has all 313~ topics, determined by dfhack, that diagram as well is just the names determined via dfhack, it doesn't actually show the techtree. The dev quotes suggest that topics in one branch determine whether topics in another can be chosen, and it's pretty clear there's a relationship between the astronomy topics in the astronomy branch, and those in the engineering branch. I've also sort of noticed some others, like economic maps always showing up after gaining the topic of maps, and so forth.

Ah... I looked at an old section of discussion on the wiki and misinterpreted the .svg on the Topics page.  In the case, that dependencies will be really hard to map out.  Your civilization can start with topics already known right?  So you either need a fresh world with no topics known yet, or you need to check in legends viewer beforehand and map out the knowledge that way?  And not researching a topic doesn't rule out that it is available.  So they best we can do through experimentation is determine which topics have no prerequisites?  I guess a dfHack script is the way to go.  Or just get Toady to outright give a map of the tech tree.

I've mostly been busy trying to figure out what skills are used for what topic right now. I also have enough experience now to understand that topics are chosen based on the highest rank of scholarly skill(once a scholar gains a proper rank in a single scholarly skill, they'll continue on pondering about those). Going through this I noticed that it sometimes requires multiple times of failing to advance a field before they'd get the topic, which is why I asked the fotf question. And then we got the full algorithm

Nice going with that!  So a way to test this quickly is to embark with Scholars that only have single scholarly skills and then run a fort for a few years to see what topics they breakthrough on?  If so I can help with that.  What skills have already been investigated, what skills still need to be tested?  While I am at it, I can try modding the dwarfs to have no need for sleep/food/drink to get them to ponder nonstop to test my assumptions about Toady's descriptions of the algorithms under more controlled conditions.

Also, with the way topics can switch after breaks or even just by chance, it seems that unskilled dwarfs with no scholarly skill will randomly change topics among too broad of a total pool of topics to ever actually make any breakthroughs (313 topics, randomly going from one to another, means an unskilled dwarf will almost never complete a breakthrough on anything).  Whereas at least a skilled dwarf will go back to the same category/skill of topics.  Also, do existing discussion put dwarf onto that topic?  If so, discussion could be used to keep small group of Scholars focused on particular topics even as individual dwarfs in that group take breaks.

Edit: Okay I redid the calculations... they got a lot worse:

Spoiler (click to show/hide)

With zero contribution from skill rolls it takes an average of 50+8.54 cycles to get a breakthrough credit. This means an additional breakthrough dice every 1464 cycles for level 1 topics, every 2528 cycles for level 2 topics, every 5056 cycles for level 3 topics, and every 10112 cycles for level 4 topics. From that point, it is an average of 2528 cycles to get a breakthrough, or to switch intentionally topics.

I got 8.54 cycles with the following Matlab code:

Code: [Select]

odds_of_finishing_at_ith_cycle(51)=0;
odds_of_finishing_at_ith_cycle(1)=0;
expected_value=0;
for i=1:50
   odds_of_finishing_at_ith_cycle(i+1)=(.02*i)*(1-sum(odds_of_finishing_at_ith_cycle(1:i)));
   expected_value=expected_value+odds_of_finishing_at_ith_cycle(i+1)*i;
end


[Bumber]

So the first 50 research cycles have 0 odds of getting any research credit, and the 100th research cycle has 100% odds?  If so, I vastly misread it.  It still takes a while, but this means the first 50 cycles will have no research credit progress, the next 50 cycles will have an average of 50 credits progress (as I calculated) and the 100th and greater cycles will all have 100 credits progress per cycle (twice as fast as i calculated).

No, the cycle resets after they get credit:

So at 51, they have a 2% chance, and at 100, they have a 100% chance.  Then, it resets the cycle number to zero and gives them breakthrough credit, [...]

That means they get 100 (+skill roll) credits after 51 to 100 cycles (with an equal distribution across that range, if my mental math is right.) So, for unskilled, that means 100 credits every 75.5 cycles. For an Easy topic, that's 1887.5 cycles (5.6 dwarf years at optimal non-stop pondering) for the first discovery dice roll with any chance (2%) of success.

I assume the dice are being rolled each time credits are awarded (even if you have 0 discovery dice,) risking a 2% chance of switching topics every 75.5 cycles. (Worst case would be every single ponder cycle, although that would make discovery quicker once you've gotten a die. This would likely be a bug, since you'd practically never reach 2 dice before succeeding.)

We have one cycle per day in the optimal case. Toady didn't mention the other factors besides pondering vs discussing. I assume mental attributes and focus (from met needs) might factor in. Could be skills involved.

[therahedwig]

So, for unskilled, that means 100 credits every 75.5 cycles.

Erm, in practice I got 1800~ research credit for a dabbling critical thinker, so there must be a little bit more to it. Also in practice: pondering gives enough xp to rank up at the least once after a year.

I have also never seen a scholar switch after they got their initial pondering research credit/pondering in, so it must be super rare to switch at that point...

Nice going with that!  So a way to test this quickly is to embark with Scholars that only have single scholarly skills and then run a fort for a few years to see what topics they breakthrough on?  If so I can help with that.  What skills have already been investigated, what skills still need to be tested?

Yeah, I've sort of staggered my research there, because I didn't want to spam the wiki too much. I've got 186 topics figured out in my local document (I messed a lot with dfhack to get my scholar to progress more quickly). I guess I'll fill in the ones on the topics page I've already got now, even though it's incomplete and messy .

Your civilization can start with topics already known right?

Not that I know of; it isn't necessary yet because there's very little stuff that unlocks right now. It might be that in the future dwarves will automatically have stuff like alloys, but that isn't necessarily the case right now.

Edit: updated the topics tables.

[therahedwig]

Ok, I went through all topics and put them on the wiki. For some reason my scholar doesn't want to start studying propositional logic. Has anyone seen a scholar research propositional logic or has seen it in their worlds? Mine just keeps being stuck in the research! job or writes.

Secondly, the mathematician descriptions are all muddled up, and I suspect there's multiple ones which have the same description(the two PI related topics, as well as the area of a circle and the method of exhaustion just all seem muddled. If someone who has a better grasp of math than me could go through them, that'd be great...)

Also, despite untrusted life's insistence, philosophers don't seem to actually to ponder values explicitely. I suspect this is a misinterpretation of the 'dialog' and 'essay' forms, which do allow them to talk about stuff like that.

EDIT:

Okay, I've written a lua script that somewhat reliably creates a csv table with numbers of how often one topic is know if the other is also known.

Spoiler (click to show/hide)

Code: [Select]

-- Create correlation table of academic topics in the world.
--[====[

world-knowledge-csv
==============
Create a table of known academic topics in your world and the topics they correlate with.

]====]

    knownKnowledge = {}
   
    saveDir = df.global.world.cur_savegame.save_dir
    worldName = dfhack.TranslateName(df.global.world.world_data.name)
   
    print("Getting topics for "..worldName)
   
    for progress, histfig in pairs(df.global.world.history.figures) do
       
        if histfig.info ~= nil then
   
            if histfig.info.secret ~= nil then
               
                knowledge = histfig.info.secret.knowledge
               
                if (knowledge~=nil) then
                   
                   
                    knowledges = {}
                    knowledges[0] = knowledge.philosophy
                    knowledges[1] = knowledge.philosophy2
                    knowledges[2] = knowledge.math
                    knowledges[3] = knowledge.math2
                    knowledges[4] = knowledge.history
                    knowledges[5] = knowledge.astronomy
                    knowledges[6] = knowledge.naturalist
                    knowledges[7] = knowledge.chemistry
                    knowledges[8] = knowledge.geography
                    knowledges[9] = knowledge.medicine
                    knowledges[10] = knowledge.medicine2
                    knowledges[11] = knowledge.medicine3
                    knowledges[12] = knowledge.engineering
                    knowledges[13] = knowledge.engineering2
                   
                   
                    for _, knowledge_type in pairs(knowledges) do
                        for name, known in pairs(knowledge_type) do
                            if (name ~= tonumber(name)) then

                                if knownKnowledge[name] == nil then
                                    knownKnowledge[name] = {}
                                end
                               
                                correlation = knownKnowledge[name]
                               
                                for _, knowledge_type_cor in pairs(knowledges) do
                                    for no, correlates in pairs(knowledge_type_cor) do
                                        if (no ~= tonumber(no)) then
                                            if correlation[no] == nil then
                                                correlation[no] = 0
                                            end
                                            if known==true and correlates == true then
                                                correlation[no] = correlation[no] + 1
                                            end
                                        end
                                    end
                                end
                            end
                        end
                    end
                end
            end
        end
       
        if (progress%1000 == 0) then
            print("handled "..progress.." histfigs")
        end
    end
   
    if worldName ~= nil then
   
        spaces = string.find(worldName, ' ', 1, true)
        while spaces ~= nil do
            worldName = string.sub(worldName,1, spaces-1)..'-'..string.sub(worldName,spaces+1)
            spaces = string.find(worldName, ' ', 1, true)
        end
   
    else
        worldName = "unnamed"
    end
   
    documentName = saveDir.."-"..worldName..'-topic-correlation'..'.csv'
   
    local log = io.open(documentName, 'w')
   
    stringHorizontalHeader = " "
    for name, correlations in pairs(knownKnowledge) do
        stringHorizontalHeader = stringHorizontalHeader..",  "..name
    end
    log:write(stringHorizontalHeader.."\n")
   
    for name, correlations in pairs(knownKnowledge) do
        row = name
        for no, value in pairs(correlations) do
            row = row..", "..value
        end
        log:write(row.."\n")
    end
    log:close()
    print("Succesfully saved data to "..documentName)


if I did everything right, this shouldn't error out. So, I think what we need to do next is have as many people run this script locally, share the csv files with us and then use something like python or something to add all the numbers together and then figure out the highest 10 correlating topics for each topic? I don't have a lot of experience with data crunching, so I am kinda wandering around blindly here...

Anyway, I put my two csv files in a zip here: http://dffd.bay12games.com/file.php?id=14659

[hun]

therahedwig, is you script "read-knowledge" updated to use now? I got an error when try to run it. Would be nice to track the progress of my scholars.

Code: [Select]

[DFHack]# read-knowledge

Printing data for `McMason' Mersethubbul
======================================

Books Read:
-----------
...8\Dwarf Fortress 0.44.12/hack/scripts/read-knowledge.lua:30: attempt to index a nil value (field 'secret')
stack traceback:
        ...8\Dwarf Fortress 0.44.12/hack/scripts/read-knowledge.lua:30: in local 'script_code'
        ...k 0.44.12-r08\Dwarf Fortress 0.44.12\hack\lua\dfhack.lua:680: in function 'dfhack.run_script_with_env'
        (...tail calls...)

I've also made one csv with correlations for the world I am currently playing, if it is still of any use for you: http://dffd.bay12games.com/file.php?id=14663

[Scruiser]

So I was busy with family stuff and then busy getting back to work after Christmas break... finally had some time now and I should have a lot of time over the weekend to run.  I've got modded dwarfs that are immune to stress and don't need to eat, sleep, or drink.  The read-knowledge script is working fine for me... You saved the script as a .lua file in /hack/scripts, hun?  It seems to be working fine for me, but I haven't had my dwarfs read any books yet, and your error seems to be happening with Books Read?

Some initial thoughts/ideas:

Toady mentioned the cycles take 1-2 days... I think based on the times pondered count and the time that has passed since I started my library the cycles are longer...  Does the time pondered count not increment for times that the dwarf was not the leader of a discussion?  That accounts for several days, but the time to ponder something still seems longer than 1-2 days... more like 2-3.  I imagine there is more stuff that Toady failed to mention or rounded off...

Will assigning the dwarfs to separate libraries make them all ponder instead of discuss?  It would be good to control for as many variables as possible this way.

[hun]

I gave it another shot, and it actually turns out that the script works for some dwarves (very few, but all of them read some books), but for the rest ~90% (including all my scholars) it does not. From looking into the code and the error text, and from some simple manipulations I figured out that the reason why it does not work is because the field histfig.info.secret returns nil for most of the dwarves (when I suppose it never should?).

This is as much as I can get without any knowledge of lua or dfhack internal structure.

[Scruiser]

Hmm... is there a difference between starting dwarfs vs. migrants vs. migrants that are historical figures?  I've only tried it on my starting 7 dwarfs so far, I'll try it on my migrants.  I don't know any lua, but I've done scripting in other languages... I think I can at least figure out how to comment out the books read part so that it just skips that portion of the code, best case, I think would be to add a check similar to the knowledge section's check and then skip the next section if it is nil...  Something like the following line right before the part that is returning an error:

Code: [Select]

if (histfig.info.secret.read_books~=nil) then
(with an additional end afterwards)

So the new script should be

Spoiler (click to show/hide)

Code: [Select]

-- View knowledge for currently selected unit
--[====[

read-knowledge
==============
View knowledge for currently selected unit

]====]

    local i,unit_name

    local unit=dfhack.gui.getSelectedUnit()
        if unit==nil then
            print ("No unit under cursor!  Aborting.")
        return
    end

    unit_name=dfhack.TranslateName(dfhack.units.getVisibleName(unit))

    print("")
    print("Printing data for "..unit_name)
    print("======================================")
    print("")
   
    histfig = df.global.world.history.figures[(unit.hist_figure_id+1)]

    firstname = histfig.name.first_name
    print("Books Read:")
    print("-----------")
    if (histfig.info.secret.read_books~=nil) then
        for _,book_id in pairs(histfig.info.secret.read_books) do
            for id, book in pairs(df.global.world.written_contents.all) do
                if (book.id==book_id) then
                    bookName = book.title
                    print("")
                    print("-\t"..bookName)
                    style = ""
                    for x, s in pairs(book.styles) do
                        sep = " "
                        if (x>0) then
                            sep = ", "
                        end
                        style = (style..sep..df.written_content_style[s])
                    end
                    print("\ta"..style.." ".. df.written_content_type[book.type])
                    if book.author > -1 then
                        maker = df.global.world.history.figures[(book.author+1)]
                        makerName = maker.name.first_name
                        print("\tby ".. makerName)
                    end
                end
            end
        end
    end   
    print("")
    print("Knowledge:")
    print("----------")
    print("")
    knowledge = histfig.info.secret.knowledge
    if (knowledge==nil) then return end
   
    knowledge_goal_category = knowledge.knowledge_goal_category
    knowledge_goal = knowledge.knowledge_goal
    research_points = knowledge.research_points
    research_percentage = (research_points/100000) *100
   
    print("Times pondered:", knowledge.times_pondered)
    print("")
   
    knowledges = {}
    knowledgeNames = {}
    knowledges[0] = knowledge.philosophy
    knowledges[1] = knowledge.philosophy2
    knowledges[2] = knowledge.math
    knowledges[3] = knowledge.math2
    knowledges[4] = knowledge.history
    knowledges[5] = knowledge.astronomy
    knowledges[6] = knowledge.naturalist
    knowledges[7] = knowledge.chemistry
    knowledges[8] = knowledge.geography
    knowledges[9] = knowledge.medicine
    knowledges[10] = knowledge.medicine2
    knowledges[11] = knowledge.medicine3
    knowledges[12] = knowledge.engineering
    knowledges[13] = knowledge.engineering2
   
   
    for no, knowledge_type in pairs(knowledges) do
        entry = 0
        for name, known in pairs(knowledge_type) do
            entry = entry+1
            if (known==true) then
                print("- ".. name)
            end
            if( knowledge_goal_category==no) then
                for entry_g, known_g in pairs(knowledge_goal) do
                    if (known_g == true and entry_g==entry-1) then
                        print("- "..name.." (Learning: "..research_points.." / 100000 or "..research_percentage.."%)")
                    end
                end
            end
        end
    end
   
    print("")



No idea if that works at all, this is my first attempt modifying a dfHack script.  I think it will cause the code to just skip the books section to avoid a nil result, as opposed to actually working around it...

[hun]

This also returns an error in my case. The problem is not in the empty histfig.info.secret.read_books. The problem is in returning nil by histfig.info.secret, which is also used later to get information about learning progress (field histfig.info.secret.knowledge). So if it is nil, then script cannot really do anything the way it is written (which is 90% of time).  Also, I am quite sure that many of the problematic dwarves read books (my scholars at the very least).

Either something is very broken with my DF/dfhack, of DF can store these information about books and knowledge in a field, different from histfig.info.secret, in some cases.

[therahedwig]

The read knowledge script is very flakey and doesn't actually represent the progress of your scholars because according to the January fotf answer, apparantly research is 90% dice-a-palooza. I'll need to rewrite the script to have it give actual useful info (and even then it only would track the chance-to-discovery level)

The problem is in returning nil by histfig.info.secret, which is also used later to get information about learning progress

Yes, this can be nil. It becomes an actual object when pondering is happening.

Thanks for the csv table.

Here's the script as I am using it now, but again, I need to collect my thoughts on how to get something useful out of the dice-a-palooza that can be understood by us mortals.

Code: [Select]

-- View knowledge for currently selected unit
--[====[

read-knowledge
==============
View knowledge for currently selected unit

]====]

    local i,unit_name

    local unit=dfhack.gui.getSelectedUnit()
        if unit==nil then
            print ("No unit under cursor!  Aborting.")
        return
    end

    unit_name=dfhack.TranslateName(dfhack.units.getVisibleName(unit))

    print("")
    print("Printing data for "..unit_name)
    print("======================================")
    print("")
   
    histfigOffset = 0
   
   
    print(unit.hist_figure_id)
    histfig = df.global.world.history.figures[(unit.hist_figure_id+1)]
   
    for _, hf in pairs(df.global.world.history.figures) do
        if hf.name.first_name == unit.name.first_name and hf.name.language == unit.name.language then
            same_lastname = false
            for w, word in pairs(hf.name.words) do
                if (word > 0) then
                    same_lastname = (word == unit.name.words[w])
                end
            end
           
            if same_lastname == true then
                print(dfhack.TranslateName(hf.name))
                if histfig.id ~= hf.id then
                    print("Mismatching Histfig ID")
                    print("Unit: "..histfig.id.." Actual ID: "..hf.id)
                    histfigOffset = hf.id-histfig.id
                    histfig = hf
                    break
                end
            end
        end
    end

    print(dfhack.TranslateName(histfig.name))
   
    for _, skill in pairs(unit.status.current_soul.skills) do
        print(df.job_skill[skill.id])
        print("\t"..skill.rating)
        print("\t"..skill.experience)
    end
   
    if (histfig.info.secret ~= nil) then
        print("Books Read:")
        print("-----------")
        for _,book_id in pairs(histfig.info.secret.read_books) do
            book = df.global.world.written_contents.all[book_id]
                if (book.id==book_id) then
                    bookName = book.title
                    print("")
                    print("-\t"..bookName)
                    style = ""
                    for x, s in pairs(book.styles) do
                        sep = " "
                        if (x>0) then
                            sep = ", "
                        end
                        style = (style..sep..df.written_content_style[s])
                    end
                    print("\ta"..style.." ".. df.written_content_type[book.type])
                    if book.author > -1 then
                        maker = df.global.world.history.figures[(book.author+histfigOffset+1)]
                        makerName = dfhack.TranslateName(maker.name)
                        print("\tby ".. makerName)
                    end
                   
                    for x, ref in pairs(book.refs) do
                       
                        if (ref._type==df.general_ref_value_levelst) then
                            print(book.ref_aux[x] , df.value_type[ref.value], ref.level)
                        else
                            printall(ref)
                            print(book.ref_aux[x] )
                        end
                    end
                end
        end
       
        print("")
        print("Knowledge:")
        print("----------")
        print("")
        knowledge = histfig.info.secret.knowledge
       
        poetic_forms = histfig.info.secret.unk_v4201_1
        musical_forms = histfig.info.secret.unk_v4201_2
        dance_forms = histfig.info.secret.unk_v4201_3
       
        if unit.status.current_soul.performance_skills ~= nil then
       
            if poetic_forms ~= nil then
               
                print("Poetic forms:")
                print("-------------")
                for _, form in pairs(unit.status.current_soul.performance_skills.poetic_forms) do
                    print(dfhack.TranslateName(df.global.world.poetic_forms.all[form.id].name, true, true).."\t"..form.rating.."\t"..form.experience)
                   
                end
            end
           
            if musical_forms ~= nil then
               
                print("Musical forms:")
                print("--------------")
                for _, form in pairs(unit.status.current_soul.performance_skills.musical_forms) do
                    print(dfhack.TranslateName(df.global.world.musical_forms.all[form.id].name, true, true).."\t"..form.rating.."\t"..form.experience)
                   
                end
            end
           
            if dance_forms ~= nil then
               
                print("Dance forms:")
                print("------------")
                for _, form in pairs(unit.status.current_soul.performance_skills.dance_forms) do
                    print(dfhack.TranslateName(df.global.world.dance_forms.all[form.id].name, true, true).."\t"..form.rating.."\t"..form.experience)
                   
                end
            end
       
        end
       
       
        if (knowledge==nil) then return end
       
        knowledge_goal_category = knowledge.knowledge_goal_category
        knowledge_goal = knowledge.knowledge_goal
        research_points = knowledge.research_points
        research_percentage = (research_points/100000) *100
       
        print("Times pondered:", knowledge.times_pondered)
        print("")
       
       
        knowledges = {}
        knowledgeNames = {}
        knowledges[0] = knowledge.philosophy
        knowledges[1] = knowledge.philosophy2
        knowledges[2] = knowledge.math
        knowledges[3] = knowledge.math2
        knowledges[4] = knowledge.history
        knowledges[5] = knowledge.astronomy
        knowledges[6] = knowledge.naturalist
        knowledges[7] = knowledge.chemistry
        knowledges[8] = knowledge.geography
        knowledges[9] = knowledge.medicine
        knowledges[10] = knowledge.medicine2
        knowledges[11] = knowledge.medicine3
        knowledges[12] = knowledge.engineering
        knowledges[13] = knowledge.engineering2
       
       
        for no, knowledge_type in pairs(knowledges) do
            entry = 0
            for name, known in pairs(knowledge_type) do
                entry = entry+1
                if (known==true) then
                    print("- ".. name)
                end
                if( knowledge_goal_category==no) then
                    for entry_g, known_g in pairs(knowledge_goal) do
                        if (known_g == true and entry_g==entry-1) then
                            print("- "..name.." (Learning: "..research_points.." / 100000 or "..research_percentage.."%)")
                        end
                    end
                end
            end
        end
   
    end
   
    print("")


[hun]

Here's the script as I am using it now

Thank you, it works now! At least I can check learned topics, that is already useful by itself.

[Scruiser]

I got my first migrant wave, and the old version of the script failed on all of the new migrants.   therahedwig's new script avoids returning an error and instead prints nothing for books/knowledge for the migrants.  When I assigned one of my migrants to be a scholar, it started printing out the times pondered for them.  I commented out the section printing all of their skills, I found it redundant with what I can get from therapist.

I need to collect my thoughts on how to get something useful out of the dice-a-palooza that can be understood by us mortals.

From my understanding, the first 50 times pondering have 0 chance of getting research credit, the next 50 times pondering each have an increasing chance of getting research credit (until it is guaranteed at the 100th time).  Getting a research credit resets the pondering count back to zero (I'll check that your script does this correctly)?  Research credit depends on 100+skill roll (which according to your past usages of the script is the major contributor to research credit).  When total research credit reaches a threshold (that varies) you starting rolling dice to get a chance at breakthroughs.  So your script covers two of the big useful things (times pondered and research credit), I think the main additional thing it needs then is the number of breakthrough dice you are rolling (research credit/threshold rounded down).  If you can't directly access the breakthrough count dice from the game, you could manually calculate it if you have access to the difficultly level of that research topic by dividing and then rounding down.

Edit: just got my first research credits.  I think my estimate of on average 58.54 times pondering is reasonable, it took 59 time pondering.  The pondering count in therahedwigg's script works correctly, it counter each time pondering and then reset to 0 with the first research credit obtained.  The skill roll and skill level makes a massive difference, 7000 research credits on my proficient mathematician.  If we can work out how much the skill roll/skill level contribute to research credits obtained, I think we can come up with an average time to get additional breakthrough dice and from that get average time to make research breakthrough.  This will also tell players how they should prioritize more dwarfs vs higher skill level when trying to maximize research.  Finally, I think Toady rounded down on the 2 days number slightly.  Does anyone know any good dfHack tricks for keeping exact count of ticks of time? 

Edit 2: Just got another success on research credits.  Something like 55-57th times pondering (I wasn't checking quite frequently enough).  I am more confident about my calculation of an average of 58.54 times pondering to get research credit.  Competent optics engineer got 4200 research credits.

Edit 3:  Proficient chemist on 60st-61th (again not sure exactly, only checking every 3-4 ingame days) got 7200 research credit.  Adequate astronomer on 61st-62nd time pondering got 2700 research credit.

[hun]

The skill roll and skill level makes a massive difference, 7000 research credits on my proficient mathematician. 

Edit 2: Just got another success on research credits.  Something like 55-57th times pondering (I wasn't checking quite frequently enough).  I am more confident about my calculation of an average of 58.54 times pondering to get research credit.  Competent optics engineer got 4200 research credits.

Edit 3:  Proficient chemist on 60st-61th (again not sure exactly, only checking every 3-4 ingame days) got 7200 research credit.  Adequate astronomer on 61st-62nd time pondering got 2700 research credit.

I had just one example of credits on my scholar so far. A skilled matematician, 6200 credits (didn't see how much time pondering did it take). From this and your numbers it seems that there is a start around 1000-1500 credits for a dabbling researcher, and then something around additional 700-1200 per every skill (clearely some roll is involved). So the skill seems to be super important, even a couple of levels matters. We can do more accurate calculations later after collecting more data.