10 million dollar, out-of-pocket AI development that they just opened up
Are they serious with that "exponential growth" claim?
That a self-improving system improves exponentially? Yes.
Biological ecosystem is a self-improving system, but it sure as hell doesn't improve exponentially.
I don't see why it should even improve exponentially. I mean, what's the reasoning behind it?
Well, a biological system isn't truly self-improving, since it requires generational passage to improve. It's not a particularly apt comparison. It would be better to compare it to... as if you could re-write the structure of your brain to make it better. The math, though, is where the exponential part comes in.
So, if it takes an AI time n to make its ability to process 2x faster, that would imply that it would then take n/2 time to do another leap by 2x, then n/4, then n/8, and so on. As the time decreases by a factor of 2^n, the speed increases by a factor of 2^n. Hence the exponential.
NinjaEdit: And yes, physical constraints aren't being taken into consideration here.
Humans have a lot of physical constraints on the size of our brain that aren't specific to the brain itself. i.e. the amount of energy the body can provide, the size of the skull etc. Obviously artificial systems will scale much differently since they don't have to be constrained to one "box". Distributed computing is a thing.
We already have learning algorithms which have improved a ton of that hardware. For example they use genetic algorithms to fine tune the physical structure of CPUs. Since this optimization code could have run on that same CPU in the first place, that is already a case of a computer using AI techniques to improve itself. So the concept of sufficiently well-programmed computers being able to improve their design isn't pie in the sky, it's already a fact. And it really doesn't need us to make a "conscious" device. The factors that govern a computers facilities are pretty much driven by concrete rules, which regular computers can perfectly well process, and the goals for optimization can definitely be coded in logic too. One advantage of computer-driven design is that computers don't break things down into abstract levels like we do, they deal directly on the level of individual predicates, thus their designs are often things humans would never have thought up, since they cut across levels of abstraction. They can deal with extremely complex mathematical relationships without needing to "abstract" things to understand it.
Here is an example of an antenna for satellite/earth communications. It's machine evolved and a huge advance on existing antennas. By looking at it, it's pretty clear than no human engineer working from basic design principles would come up with this. This kind of thing is basic proof that human design skills have a lot of blind-spots that automated design cuts through.
We'd have to be really arrogant to think that we're anywhere near perfect in e.g. programming computers. We break things down into code that's easy for a human to read. It's clearly far from an efficient use of the computer's processing.
The real question is - can a computer design new software to achieve goals that we tell it to. (i.e. we just specify predicates for the outcome of the program, then another program designs the new software). If we can make this, it can clearly be fed it's own specs and be asked to come up with something better. There might be a counter-argument that the program won't be able to deal with things more complex that it is. But this is bullshit basically. Programs work on datasets, and the datasets can be much bigger and more complex than the actual program itself. i.e. a video player can be a very small application, but can deal with huge data streams.