I thought Mantle was supposed to replace OpenGL or something... last I heard about it there was a lot of hype and virtually no publicly available data on how to use it though (at least, as far as I could find via internet search engines). [ETA: And is AMD's Mantle going to be supported by non-AMD graphics hardware, or what? Despite the hype focusing on eliminating the need for bloaty compatibility layers, I was unable to learn how it's going to handle portability either. Just curious.]
The thing that you have to remember about using the GPU to run calculations is that it basically requires learning a whole new programming language... for the simple reason that compilers don't make your regular programming languages that normally do calculations run on the hardware that was designed to do calculate en masse efficiently for graphics' sake. It's extremely ironic: we built special hardware for graphics because graphics needed lots more calculation, then we developed programming languages just to control the graphics hardware, then we realized that could be a more efficient way to do regular program calculations when there's a lot of calculating to do, and we haven't yet got to the point of saying, "Shoot, why don't we just make the regular hardware that powerful so you don't need to learn separate programming languages for using the graphics hardware for calculating other stuff?" (Now, I'll grant you, if you want to use separate programming languages because the way they approach calculation is better for mass calculation than traditional programs -- you know, things like SIMD [Single-Instruction Multiple-Data] -- that's another matter, whether the hardware's specialized or not. But that's, well, another matter!)
End rant. That's not to say that programming on the GPU is a bad idea, just that you have to keep in mind it's extra work and might be unnecessary in a decade or two (so, not long from now in the Dwarf Fortress Development Cycle Scale of Time!) if we can work through the fit of irony that is the fact of GPU programming languages in the first place.
Going back to the OP's question: I imagine that taking any old program and dynamically splitting its load in parallel between multiple cores might be something operating systems will do for all programs by analyzing their code, at least for code that doesn't modify itself while running, in another decade or two (again)... Sort of like how certain programming languages' runtime engines optimize their scripts or bytecode as they run it and see what it actually ends up doing.