Phew! About six months ago I started on a little task that I thought would take me a couple of weeks to do, and this weekend I've finally finished it. I'm only 1,100% late - that's not too bad, is it? ;-)
But it's a biggie - if I couldn't get it working well enough we'd have been screwed, basically. The task was yang learning; in other words, how should a brain map learn a useful palette of actions out of the many that it could take, and learn which ones to take in any given circumstance to get to a given goal? It seemed like a pretty easy question compared to yin learning (where there's no feedback about how well the system is doing) but the problem is full of baroque twiddly bits that make it really hard to do in real-world scenarios. There aren't nearly enough neurons to represent every rule, for instance, so they have to find optimal uses for themselves so that the system can generalize new solutions from only vaguely similar circumstances without doing anything too dumb in the process. Similarly, in academic problems it's often fine if the network needs to see a set of examples hundreds of times before converging on a sufficiently good response, but that's not like real life - you only have to poke someone who's bigger than you once with a stick to find out it's not a good idea. And in the academic world problems are often broken up into nice neat time steps: the network gets presented with a problem, it generates a response and then along comes the next problem. But real life is messy and analogue, and decisions can vary from the discrete (I think I'll have the chicken korma) to the continuous (my left arm is a bit too low and moving too fast).
I can't say I'm proud of the way the learning happens right now - it's a bit top-down and cheaty - but the important thing is that it works. Quickly, effectively and hopefully without needing to be tweaked for different problem spaces. I can easily go back and alter it if a better idea comes to me later.
But I'm much more pleased with the decision-making process. Imagine a brain map is asked to get us from A to G, but it has never done this before so it doesn't have a ready answer. On the other hand, it HAS learned how to get from A to B, from E to F and from B to D. What my design can do now is sit there and think for a fraction of a second, and then realize that it can try taking the sequence of actions A->B, B->D, E->F, and hopefully that will get it from A to somewhere sufficiently close to G.
The exciting thing about this from my perspective is that the network is actually THINKING, not just reacting. Admittedly it's not the kind of thinking we normally mean by the term - the sort involving long chains of deliberate and conscious reasoning or modeling - it's more like the kind of "intuitive" thinking that goes on unconsciously between recognizing a problem and somehow knowing what to do about it. The sort of thinking that we don't even think we're thinking. I'd like some more coffee right now, but I don't have to sit here and think "well, if I push my chair back, that will mean I can get up, and then if I get up it will mean I'm able to get to the kitchen, but only if I move my left foot forward and shift my balance..." And yet my brain does have to do something like this. Somewhere it has to link up chains of smaller actions (each of which involves linking up chains of even smaller actions) until I have a plan to execute. All of this happens unconsciously - from a conscious perspective we just have a sense of an "urge", which we can sometimes feel building up to the point at which we trigger an action, which just happens to be the right action to kick off a sequence of events that lead us to the situation we want to be in. We're aware of the impetus and the sense of forming a decision, but we're not aware of the steps involved in getting there. In fact, if you monitor someone's brain for a few seconds around the time they decide to do somethig like go and have a coffee, you'll find there's a rapid build-up of brain activity for a little while prior to the initiation of the first action. It's called the Readiness Potential and it occurs before we are even aware that we've made a choice to do something, because it IS the process of us making a choice. (Lots of philosophical stuff about free will crops up here - look up Benjamin Libet if you're interested).
And that decision-making process is what these brain maps are now able to do. Signals start to flow from neuron to neuron and spread outwards from various points across the map. The signal builds up until one neuron reaches a threshold and then shazzam, its action is taken, the neurons fall silent and the build-up process begins again (the plan isn't stored; the entire predicted path from A to G is determined and then the first step is taken. This might not actually lead to where it was expected to lead, so the plan is reassembled at each step to account for errors or uncertainties along the way). As it stands at the moment, the system is a little too good, if anything, and sometimes takes the scenic route from the state it was in to the state it needs to get to, but at least it gets there and the dynamics of it feel pretty nice. The time taken to make an assessment is proportional to the amount of confidence the map has and the degree to which each step resembles earlier experiences, for instance. And if we don't have time to make a considered choice we'll leap at a less considered one instead. It's all pretty realistic.
So at last we have our first thoughts emerging, even if they're only subconscious ones. The network is making predictions and assembling simple plans, and it can then learn from its mistakes or inaccuracies as it carries them out, so that it makes better plans in future, even in very different circumstances. And it's doing this through simple, biologically plausible processes such as spreading waves of activation. This is much more interesting than many existing neural networks, which often tend to have no dynamics at all.
What we have working now is the yin system and the yang system. There are still missing parts, such as the (much more technically straightforward) mechanism for recording emotional associations, and some big unknowns, such as the exact rules for reflection - when and how signals coming up the yin channels get reflected back down as yang commands, and when speculative signals flowing down the yang should be reflected back up the yin as if the intention has actually occurred (we need this for dreaming and for more deliberate and speculative planning). But getting yin and yang to work together was THE key to the whole operation. It's now possible for a brain map to observe what happens to its inputs, learn to classify those inputs to produce a map of situations, accept commands from above that it can now meaningfully interpret in terms of this map of situations, and create short plans of actions that it thinks will lead from the sensory situation it finds itself in towards the sensory situation it, or another map, would like it to be in. Basically, it can recognize what's going on and figure out what best to do about it. How it feels about that is something that can wait for later, and whether it will be able to explore possibilities in its own mind without those things occuring in reality is still a big philosophical question, but not a pressing one. I have enough now to be able to go on and start building a functioning brain.
Phew! If you don't really understand what any of this means, all you need to know is that one brain map is now equivalent to, yet far more complex in its structure and behavior than the entire brain of a Norn. And these new creatures will have maybe a hundred such brain maps, all working together. What's more, I can now say with some confidence that the Grandroids will think ahead and have expectations in a way that Norns couldn't. This difference is qualitiative - you can't be disappointed unless you expected something to happen and it didn't; you can't be genuinely frightened unless you think something bad is about to happen; and you can't be worried unless you realize that you don't really know what to do about it. The things we humans use consciousness to do are still some way off in the future, but at last I've closed the loop between sensation and action (and the resulting sensation...).
I've been sweating quite a lot about this part, I have to admit, and it's a big relief to get it out of the way. I can't rest on my laurels, because now comes the challenge of creating the quite complex sequences involved in things like walking, where an odd dynamic exists between actions and rewards. The trouble with having invented a brain that's hundreds of times more powerful than a Norn's is that the problems these new creatures have to face are also hundreds of times more complex! But my confidence level has gone up a lot over the past few days, so after a short afternoon hike to celebrate (and to get out of this damn chair) I'll launch back into it again with gusto. I'm afraid there's still nothing much to see (although I could maybe do a short video just to show this new brain activity happening), but this past few months has been a real bottleneck to progress, so hopefully things will speed up quite a bit over the coming weeks and the progress will become a lot more tangible.
Onward and upward!
With any luck this could mean backers get something to toy around with within the next month or two.
Author
Topic: Grandroids: Secret Supporter Stuff! (Read 65902 times)