Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[jseah]

I don't think computing power will constraint this suggestion.  Random numbers don't use alot of time. 

A few hundred numbers over a game day or so won't even be noticed next to the pathing calculations, however optimized you make those. 

It's the presentation of this information to the player that will be difficult.  I was thinking you could just give the player the grid(s) for the dwarf the player wants to see and colour in the squares accordingly. 

[counting]

I don't think computing power will constraint this suggestion.  Random numbers don't use alot of time. 

A few hundred numbers over a game day or so won't even be noticed next to the pathing calculations, however optimized you make those. 

It's the presentation of this information to the player that will be difficult.  I was thinking you could just give the player the grid(s) for the dwarf the player wants to see and colour in the squares accordingly.

Just listed as pron and con, I personally do like more realistic system. Why not And DF already is processes heavily. Maybe distributed processes can help.

[vintermann]

In programming perspective, It will require more process time.

Are you sure? There are assembly operations for counting the number of bits set in a bit array (they're called POPCOUNT or something), and these are very quick. So if you're using a 512 bit array to represent a skill, it just takes 16 popcount operations and 15 additions. Learning can be done with masking of random bit-arrays. In general, you can get ridicuoulsly efficient with bit-fiddling like this, I don't think this is going to be DF's performance bottleneck 

[counting]

In programming perspective, It will require more process time.

Are you sure? There are assembly operations for counting the number of bits set in a bit array (they're called POPCOUNT or something), and these are very quick. So if you're using a 512 bit array to represent a skill, it just takes 16 popcount operations and 15 additions. Learning can be done with masking of random bit-arrays. In general, you can get ridicuoulsly efficient with bit-fiddling like this, I don't think this is going to be DF's performance bottleneck 

Bitsting counting can be tricked to used masked and recursion into iterations of steps for the purpose of accelerating counting its elements, but it's not O(logN) additions it's O(logN) iterations. how fast the iteration is related to how big the chunk is,(how you divided the bitstring, 32-bits CPU max is 32, so when you shift it and mask will no longer be simple 1 steps, unless you have a 512 bitwise CPU). Any case it still requires "at least 15 additions" it's not a simple memory/register read. unless the size is only 1, it only reduce "the factor" of slowing down doesn't make it faster. At a factor of 15 (fastest possible in your case) it's still 1400% more time!!

As for random array, yes mask is quicker than multiply ops. Although not much if bit length is large, it has to be done separately in (N/bitwise) cycles. And it still need to "GENERATE" that random array. You are just trading one problem with another, unless you are talking about using some unused memory blocks, to be used as the source of random array. In that case it won't be pseudo random, and "less less random" when system running too long memory pools drop down. Or you may as well just used a fixed sequence of "rand_seq()%N" table look-up function, as long as the size of table large enough people won't notice, but it still require a modulation operation which is about the same as multiply, and after that require additional mask operations. At last several times of total operations cost. either way it's still "SLOWER".

I am not implying its not possible. In fact very possible in implementing, we already point out how, and I like realism, Just point out the extra costs. What makes them unnoticeable,is these functions can be distributed into like path-finding routine, and stored vale for lookup, and players won't know the differences. If these functions are not used often. as some other earlier replies indicate, it may not be a major slowing down factor (unless someone constantly checks skill page, and non stop training dwarf).

[Naryar]

I am fine with the current skill system. I would just like to see potash maker, soaper, lye maker be assembled into Alchemist and other more useless skills (how is Pump Operator even a skill when Hauler isn't?)

[harborpirate]

These operations may not have an ideal profile in Big O notation, but we're talking about something like 1024 bits. Performance on modern, dual core multi-gigahertz CPUs should not be an issue. Sure, that would be multiple registers in the machine; but when compared to the structure of, say, the map, its positively tiny. A simple structure for this storing this type of knowledge representation would probably be a byte array.

Using bitwise operations means that certain things that would be stupidly hard and slow in higher level structures get really really easy. For instance, a dwarf reading a book should be a simple bitwise OR. This type of operation is extremely quick in comparison to iterating and comparing over some higher level collection object.

Dwarves sharing knowledge would be more challenging. I would probably procedurally select a bit from the "teachers" knowledge array and OR it against the "learners" array in the same location, storing the result in the learners array location. Choosing the bit to share is undoubtedly the trickiest part.

Unlike pathing, which is running constantly, these operations are presumably going be relatively rare. They should probably only happen when dwarves that don't have a job gather together (the time when dwarves currently train social skills), or when they gather to train a specific thing (like the current military training).

[counting]

These operations may not have an ideal profile in Big O notation, but we're talking about something like 1024 bits. Performance on modern, dual core multi-gigahertz CPUs should not be an issue. Sure, that would be multiple registers in the machine; but when compared to the structure of, say, the map, its positively tiny. A simple structure for this storing this type of knowledge representation would probably be a byte array.

Using bitwise operations means that certain things that would be stupidly hard and slow in higher level structures get really really easy. For instance, a dwarf reading a book should be a simple bitwise OR. This type of operation is extremely quick in comparison to iterating and comparing over some higher level collection object.

Dwarves sharing knowledge would be more challenging. I would probably procedurally select a bit from the "teachers" knowledge array and OR it against the "learners" array in the same location, storing the result in the learners array location. Choosing the bit to share is undoubtedly the trickiest part.

Unlike pathing, which is running constantly, these operations are presumably going be relatively rare. They should probably only happen when dwarves that don't have a job gather together (the time when dwarves currently train social skills), or when they gather to train a specific thing (like the current military training).

1024 bits "Per skill Per Dwarf", almost 100 kind of skills, with 100 dwarf = 1.3MB (+pointers), comparing to 4 bytes per skill per dwarf = 40 KB (with far pointers 80 KB), that's quite a lot difference. You may not think its much, but consider cache aren't equal to memory, efficiency will be different. Operational time IS a problem, consider FPS issue, you don't just simulate 1 dwarf, its everything simultaneously, it already pushing some CPU to limit.

I think my older post already point out "gain_skill_books(skill_array*, book_index) will need O(1) to compare" its the only one which is better in efficiency than the other 2 (get_skill_level & gain_skill_training). But they are issue only when viewing the skill often and training a lot dwarf at the same time (which is constantly happening), and it require constantly calling rand() subroutine. And it needs some work to optimize (default random generator is bad and slow), or trick to make it like random. Otherwise it will be very predictable. (which will be interesting, considering using a static string with random distributions. And if you can crack its sequence, learn which numbers will NOT be touched by 'rand sequence', and you can read books to fill the blank, and learn much faster).

Sharing knowledge can also be done using "random". XOR 2 maps create a teaching table, Just pick a number and see if isn't a 1, than it's a skill can be exchanged. Set both teacher/students map bit to 1. if it's a 0, sharing fail, pick again. repeat. over. And who is teacher, who is student doesn't matter. They mutually teaching each other things they don't know.

[harborpirate]

1024 bits "Per skill Per Dwarf", almost 100 kind of skills, with 100 dwarf = 1.3MB (+pointers), comparing to 4 bytes per skill per dwarf = 40 KB (with far pointers 80 KB)...

No.

The original suggestion would become unworkable, you're correct. Probably my posts were unclear; thats not what I'm saying at all.

1024 bits for all skills per dwarf is what I've been advocating. And thats just an off-the-cuff number, we might need more or less than that. But with some bits overlapping multiple skills, using a single large byte array for all skills should be extremely efficient.

Even if it was twice that size (2048 bits), it still shouldn't be an issue, thats 256 bytes per dwarf. One hundred (100) dwarves would be 25.6 KB at that size. Seems pretty manageable.

The brilliance of the tweak to this idea lies in the details: putting all skills into a single byte array to minimize the total size and also better represent the way knowledge works. (Each bit representing some type of knowledge.)

Books are simple: we don't even have to care what type of knowledge they represent!* We can just use their values in bitwise operations. Same with learning/teaching, as you noted.

For the skill level displayed to the player, you'll need skill lookup maps/masks or hard coded algorithms, but you'll only need one of those per skill, and not per dwarf. Also I don't think there are 100 skills in the game.

*Books done this way can accurately simulate Medieval books that were generally conglomerations of a variety of topics instead of concentrating on a single thing.

[counting]

1024 bits "Per skill Per Dwarf", almost 100 kind of skills, with 100 dwarf = 1.3MB (+pointers), comparing to 4 bytes per skill per dwarf = 40 KB (with far pointers 80 KB)...

No.

The original suggestion would become unworkable, you're correct. Probably my posts were unclear; thats not what I'm saying at all.

1024 bits for all skills per dwarf is what I've been advocating. And thats just an off-the-cuff number, we might need more or less than that. But with some bits overlapping multiple skills, using a single large byte array for all skills should be extremely efficient.

Even if it was twice that size (2048 bits), it still shouldn't be an issue, thats 256 bytes per dwarf. One hundred (100) dwarves would be 25.6 KB at that size. Seems pretty manageable.

The brilliance of the tweak to this idea lies in the details: putting all skills into a single byte array to minimize the total size and also better represent the way knowledge works. (Each bit representing some type of knowledge.)

Books are simple: we don't even have to care what type of knowledge they represent!* We can just use their values in bitwise operations. Same with learning/teaching, as you noted.

For the skill level displayed to the player, you'll need skill lookup maps/masks or hard coded algorithms, but you'll only need one of those per skill, and not per dwarf. Also I don't think there are 100 skills in the game.

*Books done this way can accurately simulate Medieval books that were generally conglomerations of a variety of topics instead of concentrating on a single thing.

Yes, there are close to 100 skills (actually 98, you can count them, not all are implemented now though). But I guess a larger bitmap can't hurt much.

I do misunderstand what a bitmap skills should all be inside one. So how do we separately count the bitstring, to get a number about a specific skill level? divided them evenly on the bitstring? or by the number of books/skill bit they consisted and mash them up? If there is just 1024 bits divided into 100 groups, than I don't think counting how many 1 in 10 bits need any acceleration. But I guess you might originally think it's about 64 bits per skills on average? Some skill can share the same skill bit, How to determine the overlapping? DF are not predetermined, it needs some algorithm to make sure we won't have absurd overlapping of skills. (A brewer and archery book, teach you how to get drunk and not shooting your enemy, it will be FUN!)

[harborpirate]

1024 bits "Per skill Per Dwarf", almost 100 kind of skills, with 100 dwarf = 1.3MB (+pointers), comparing to 4 bytes per skill per dwarf = 40 KB (with far pointers 80 KB)...

No.

The original suggestion would become unworkable, you're correct. Probably my posts were unclear; thats not what I'm saying at all.

1024 bits for all skills per dwarf is what I've been advocating. And thats just an off-the-cuff number, we might need more or less than that. But with some bits overlapping multiple skills, using a single large byte array for all skills should be extremely efficient.

Even if it was twice that size (2048 bits), it still shouldn't be an issue, thats 256 bytes per dwarf. One hundred (100) dwarves would be 25.6 KB at that size. Seems pretty manageable.

The brilliance of the tweak to this idea lies in the details: putting all skills into a single byte array to minimize the total size and also better represent the way knowledge works. (Each bit representing some type of knowledge.)

Books are simple: we don't even have to care what type of knowledge they represent!* We can just use their values in bitwise operations. Same with learning/teaching, as you noted.

For the skill level displayed to the player, you'll need skill lookup maps/masks or hard coded algorithms, but you'll only need one of those per skill, and not per dwarf. Also I don't think there are 100 skills in the game.

*Books done this way can accurately simulate Medieval books that were generally conglomerations of a variety of topics instead of concentrating on a single thing.

Yes, there are close to 100 skills (actually 98, you can count them, not all are implemented now though). But I guess a larger bitmap can't hurt much.

I do misunderstand what a bitmap skills should all be inside one. So how do we separately count the bitstring, to get a number about a specific skill level? divided them evenly on the bitstring? or by the number of books/skill bit they consisted and mash them up? If there is just 1024 bits divided into 100 groups, than I don't think counting how many 1 in 10 bits need any acceleration. But I guess you might originally think it's about 64 bits per skills on average? Some skill can share the same skill bit, How to determine the overlapping? DF are not predetermined, it needs some algorithm to make sure we won't have absurd overlapping of skills. (A brewer and archery book, teach you how to get drunk and not shooting your enemy, it will be FUN!)

You joke about it, but Medieval books were often like this. Most were not written on a single subject, but actually contained information on a wide variety of topics that may or may not be interrelated.

The reason you mentioned (storage) would be the primary driver to mix all skills into a single byte array. It also makes bit sharing between skills much simpler, and it allows some skills to only have a few bits devoted to them (cleaning, hauling), whereas complex skill types could have much more fidelity (metalworking, alchemy). Some skills could have high fidelity but have almost all of their bits shared among other skill types, and others may have most of their skill bits devoted specifically to that practice alone. Its extremely flexible.

1024 is just an example number, it might have to be considerably larger to give enough fidelity to be useful. With 98 skills, even if sharing was pretty common that wouldn't give very many bits devoted to each skill type, so something like 8 times that size might be required. That would get it up to an entire kilobyte to store per dwarf, and assuming a fair amount of bit sharing you'd get over 100 bits to represent each skill.

Full calculation:

Spoiler (click to show/hide)

1 KB = 1024 bytes = 8192 bits. 8192 / 98 skills = 83 bits per skill. Skill sharing at 25% gives a boost of an extra 20 bits per skill, resulting in at least 103 bits per skill on average.

As I mentioned for calculating the "level" of each skill, you will have to have a map or mask to do that calculation. With proper care and some careful bitwise logic you can almost certainly do that quite rapidly, but that would be one area that Toady would have to be very wary of - because any calculation that uses the overall level of any particular skill would require that operation and that could definitely impact FPS.

Now you've got me itching to build a simulation of this system just to test the performance of it... and I've got enough work as it is!

[kaenneth]

As I mentioned for calculating the "level" of each skill, you will have to have a map or mask to do that calculation. With proper care and some careful bitwise logic you can almost certainly do that quite rapidly, but that would be one area that Toady would have to be very wary of - because any calculation that uses the overall level of any particular skill would require that operation and that could definitely impact FPS.

I was thinking that the total skill points would be cached in memory local to the core dwarf data, and only updated when something is learned or forgotten; that way the vast majority of the time no calculations would be needed.

As for gaining skill, there wouldn't be a situation where a dwarf 'must' gain a skill point, and the system would loop until it found an unset bit to set. attempting to gain skill would just be an 'or' operation; if the bit was already set, then nothing new is actually learned, and the fuction is done. This simulatates it getting hard and harder to discover/invent something new; just like the increasing points needed to level a skill at each level currently, and in most other games with 'levels'

I also erred in saying 'bitmap', maybe I should have said 'bitmask', I wasn't thinking of a 2d grid, or a line... just raw bits. with the only presentation to the user being the same description DF does now 'Dabbling' to 'Legendary', based up the total number of set bits (by native popcount instruction, or methods from http://graphics.stanford.edu/~seander/bithacks.html#CountBitsSetParallel

[counting]

I was thinking that the total skill points would be cached in memory local to the core dwarf data, and only updated when something is learned or forgotten; that way the vast majority of the time no calculations would be needed.

Cache only helps when the read out >> write cost. And while dwarfs are not idling, they are actually doing something at every time you refresh data of the world, and each refresh the system needs to readout the skill_level to determine the efficiency of the works dwarfs are doing, so it can advance the progress of each work(+ N%), hence I believe its best to read the skillmap and update the cache, ONLY after a fixed counter is reach, so the operation time of skill_read_n_writeto_cache(), can be spread. The less often you update, the more FPS, you get, otherwise, it will be lagging, but update more often, so you get more accurate reading when you view a dwarf. The rate should related to how often, a player check the skill on average. And you don't need to use a very efficient skill_level_read at first, and optimized it later, as long as the counter is large enough, OR a better idea, it only update after a dwarf went to sleep and at the point of waking up, so this function can be tied to the wake up function, think of Oblivion 4, and other RPG game element.

As for gaining skill, there wouldn't be a situation where a dwarf 'must' gain a skill point, and the system would loop until it found an unset bit to set. attempting to gain skill would just be an 'or' operation; if the bit was already set, then nothing new is actually learned, and the fuction is done. This simulatates it getting hard and harder to discover/invent something new; just like the increasing points needed to level a skill at each level currently, and in most other games with 'levels'

Its impractical, to call gain_skill_training(), every time the world is refresh, the same reason as before, but when the point where a certain job is done, a good is produce, a hauling is in place, a session of combat course is finished, so the cost of this function which contain time consuming rand(), can be spread. Or even better, as using the sleep-learning system, that while training, the training result is only store in a cache as an index as well, and only slowly "written in" the dwarf brain while sleep at a constant rate (Oh! the dream of a dwarf, was so dwarfy), and at a much slower rate while idling, whether or not the dwarf can used out the learning index, depend one how quick a learner he is (less counter), and how many time he is idling/sleeping. So we get a legitimize reason why a dwarf is idling and what's the need of sleeping. (Or is it already been done by Toady? just using current skill update, this technique can also increase FPS as well.) Hence an idling dwarf is actually a learning dwarf not a lazy dwarf, we all misunderstood them. (If you keep pushing every dwarf to do thing, and don't let him rest/sleep he will over-load his index, and never learned anything, we can even implement a new FUN! as a dwarf been working to death  )

I also erred in saying 'bitmap', maybe I should have said 'bitmask', I wasn't thinking of a 2d grid, or a line... just raw bits. with the only presentation to the user being the same description DF does now 'Dabbling' to 'Legendary', based up the total number of set bits (by native popcount instruction, or methods from http://graphics.stanford.edu/~seander/bithacks.html#CountBitsSetParallel

As a better image to the whole picture, I would say we can call this an actual "dwarf brainmap", and better yet, why 2d at all, it can be 3d, or higher! (your brain is not actually 3d, since area in one part of the brain, not just related to their surrounding area, but also connect to far parts of the brain, more like a 3.5d, and why neural system functioning). So certain skill is laying in one part of the brainmap (poor dwarf can only see black/white 0/1 ), it's neighbor skills are the only ones can be overlapping with it, the bigger the surface area(boundary in 2d), the more skill it connected to. The volume (or area in 2d), is not positive related to the surface (boundary), an area can be like a starfish shape, with a long boundary border but a minimum area; an area like a circle is the exact opposite. And a good idea for why book should work, the eyes connected to every part of the brainmap, and a book is a "bitmask represented with dots", and the function of the eye, is actually the OR function to combine these 2 arrays.

But the mechanism I was talking about previous post setting the overlapping part, still need to be worked on, it needs to be "draw" carefully to set some correct logical overlapping in a brainskillmap. The same dots belong to a skill should gather together tightly in 2d/3d spatial positions, for the purpose of aesthetics as well as for purpose of deigned, its easier to picture them in blocks, before putting into array. Skills group like in the real brain, things like mortar function(military, hauling, mining) as larger groups first. Skills set don't have to be clustering in one continuing shapes, it can containing many areas with different shapes like in real brain. (This didn't effect the programming at all, just to the purpose of actually writing the value preset in skill_mask array, or you won't even have a place to start. Implementing them sections by sections won't generate a 2d/3d related maps, but a num_section-dimension related blocks, and looked very chaos mapping into 2d/3d plane)

!WARNING suggestions below are extra thoughts, and will cause extra costs in memory/calculation

We can even have a smart(or dumb) potions system, that magically add dots intelligence/knowledge in a dwarf, how neat. And why set a fixed size/static brainmap, it can be grown/moved. So the younger dwarf can only access a part of the whole brainmap, and his brain capacity determine the whole potential, and only get wiser from learning, but before dwarfs are old enough. Its not possible to produce a GOD-like dwarf while he is young, It has a max possible size, so to be the complete knowledge of the wold. In implementing it will need extra process to check boundary, and while get_skill_level function doesn't need to be changed, but training will need extra brain size mask types (selected regarding age and intelligence level?), Then ((BrainMap OR LearnedSkillMap) AND BrainSizeMask), to get a new BrainMap.

Even cache has a real meaning, as it represent a dwarf's shorten memory, a temporary storage, and the brainmap actually store the long-term part. We can even tweak the cache so it has 3 variables to represent a skill, the base value from updating from brainmap, and a increment as calculated from learning index cache, and the real skill efficiency is the combination of both. So a dwarf can temporarily be quicker at a short time, but not really settle down to base value in the brainmap, and its possible to lost the index cache, when the cycle of sleep(idling can help to reduce the need) is interrupt before its count to 0, and the full skill potential is not as efficient as the previous working day when he just waking up, until he actually starts working again. And we can add a "church bell" mechanics, to wake up the dwarf in an area, to interrupt their sleep and make it more FUN! (Bells have real purpose not just for FUN, since it is possible to running out of index (index count to 0), and the dwarf still sleeping, and now the dwarf is actually lazy than helpfully learning.)

[harborpirate]

Nice post there, and interesting food for thought.

I think the expanding structure may not be necessary since an initialized baby dwarf would start with nothing and slowly learn over time. Interesting idea though, I haven't fully thought through the potential other uses. Super smart gods? Really dumb creatures incapable of learning certain skills? Hmmm, I'm coming back around on that one...

This opens up huge possibilities for temporary effects. You could swap out the dwarfbrain into a temp structure and modify the active dwarfbrain to have more or less knowledge than they really have. You could have a potion that caused a dwarf to suddenly know everything about metalworking, or a spell that caused a dwarf to forget everything they knew about making cheese. And these could be temporary things that could be fixed by some action that swaps the data back (or of course they could be permanent).

Interestingly a dwarf that temporarily learned say, alchemy for instance could share that with others and some of that knowledge could remain in your fortress even if/when the effect wears off for that one dwarf.

The more we discuss this system, the more I like it.

[counting]

Nice post there, and interesting food for thought.

I think the expanding structure may not be necessary since an initialized baby dwarf would start with nothing and slowly learn over time. Interesting idea though, I haven't fully thought through the potential other uses. Super smart gods? Really dumb creatures incapable of learning certain skills? Hmmm, I'm coming back around on that one...

This opens up huge possibilities for temporary effects. You could swap out the dwarfbrain into a temp structure and modify the active dwarfbrain to have more or less knowledge than they really have. You could have a potion that caused a dwarf to suddenly know everything about metalworking, or a spell that caused a dwarf to forget everything they knew about making cheese. And these could be temporary things that could be fixed by some action that swaps the data back (or of course they could be permanent).

Interestingly a dwarf that temporarily learned say, alchemy for instance could share that with others and some of that knowledge could remain in your fortress even if/when the effect wears off for that one dwarf.

The more we discuss this system, the more I like it.

Me like it too. And another thing I thought about is the personality that affects the learning. It's the queuing priority while absorbing the cache index into brainmap in sleep. When you put a dwarf to work on different things, he accumulate different potentials, but he choose to solidify which first. It he hates something, it may not absorbed them at all at low priority.

Plus, we can add something dreamy skills(or stuff) that doesn't actually has functions (or yet, it think with 8k-bits brainmap there is room left for this), and some lazy dwarfs can dream about it, get some random giving potential points with high priority, so while idling he doesn't trying to remember the skills player put them through (or he has no potential points left to learn already), because he is day dreaming. (Urist gains 1 idle-skill potentials while idling, he is now a proficient idler . I like to see at the end how many master idler we get) But even if he hates something, but you still put him to that work , he will have no choice to be better, eventually. Aren't we all saying : I hate my job, but in the end, we are actually professionals in our jobs. This can be good, when some dwarf may pre-training themselves with works not yet available, because he has a dream to give him potential points while idling.

(Urist while meditating has an epiphany on how to brew new ales quickly, he can actually write about it, and adding in a book for other dwarf to read. We are advancing in science through idling dreaming great ideas, they are extra points giving by talented traits dwarf while idling, beyond normal training/working/reading, which he has already learned then all what player put him through, so the state after idling-learning is the thinking-learning time, and spend extra time as a thinker maybe important as a dwarf with high idle dreaming potentials, however dreamy skills only has partial overlaps with actual working skills, so the theoretical part of some skills can be obtained this way, not all of it, no one can idle to a master, but true practices, and his book while useful in some way, but may content something just pure dreaming)

A partial extension idea of it, is the "bodymap", not just "brainmap", some physical attribute maybe less linked in brain skills that actual physical figures, like haulers and warriors/axeman, they require body mass, so part of the bitmap, is actually bodymap (a dot is a how good you can use that part of body, the motor nerves), so they maybe linked to body parts. I haven't think it through, But I think a dwarf without left hands, or a fingers, no matter how good he is before or training to be, never possible to be a true master marksman. (Some skills won't be affected I guess, like idler  )

[kaenneth]

I really appreciate the positive feedback.

Contempating the n-dimensional 'brain-map' idea, I could visualize it as a hyper-cube structure, with dimensions partly defined by dwarf soul attributes, and clusters assigned to particular skills... trying to think of an effective way to do that as there would be a lot of sparse space.

But at least overlapping sets of bit could be arranged, for example there would be a set of bits for 'stonecrafting', and a set of bits related to 'musicality', and there would be a small number of bits that are in the intersection set, so that a dwarf with high music sense working as a stonecrafter would be most likely to get bits in that particular area set.

This also feeds back into the 'brain injury' thread, as an injury/syndrome/curse that affects an attribute could then affect those particular skills appropriatly by masking out the affected bits.

For more efficent use of space a hybrid system might be better. Bits within a skill could be tiered by difficulty, so that (again, for example) while a skill has 256 bits only 32 are used in bitmask form, stored as an an offset and mask. so no skill would be "0x00,0x00000000", middle skill would be "0x7E, 0x0B4F5091", and max skill "0xDF, 0xFFFFFFFF"; that is, all bits below the offset are assumed set, and all bits beyond the mask are assumed unset, whenever the high bit would be set, the offset is incremented and the mask shifted by one. This would model the need to learn the basics before advanced techniques while also saving storage space.

At which point the overall skill is basically defined by the offset which can be used in skill calculations, and the mask is only used to model learning.  However, a dwarf might get stuck, unable to advance because he just dosn't get one particular bit... I don't know if that's a good thing or a bad thing.

I also like the idea of only updating when a dwarf sleeps; I could see it as bits in the mask might get set by training and working, but the offset only moves when they sleep. This jibes well with my concept of dreams in reality; I think of them as the time when the brain sorts out the things it learned the prior day, prepares the stuff it thinks it'll need for the next, and opens up short term space for new things.

That wouldn't apply to books, a book might contain only advanced knowledge, and be incomprehensible to a novice, or a scattering of basic and advanced. Just like a calculus textbook would only be useful for a 4 year old if he also had some crayons.