I think a nice balance is to have more than one pool of experience points. For instance, Fable separates skills in 3 groups: magic, "speed" and combat. Instead of leveling, you purchase more powers/trainings (I don't like levels, unless it is a "secondary" attribute, meaning it is calculated from the amount of skills you've developed). Personally I would use different groups tho, even separate very broad "schools" of magic, such as arcane or clerical (not narrow ones, like "enchant", "summon", "healing" etc) and put all "athletic" skills in a single pool, and "intellectual" skills in another, perhaps.
So yes, playing a lot of soccer may allow you to be better at fighting with axes, but only if you actually spend the points that way. It may sound silly, but at least you're working out... if that bothers you, maybe have an "upper body" vs a "lower body" exercise skill pool. But I think having better legwork is good for any kind of fighting.
Even if simplistic, Dungeon Siege had a system where you got better at either 1) fighting 2) shooting 3) nature magic 4) combat magic, by doing stuff (namely, fighting, shooting and using magic). The interesting part is, the higher your "total skill", the harder it was to advance in one of these. So you could have a jack-of-all trades, or a combat magic specialist, kinda organically, regardless of his starting attributes (unless they already started with a ton of points in one of these) and while you had to be careful not to make your mage fight too much, at least you didn't have to actively avoid things like jumping, running, dodging, talking, buying, sleeping, eating, thinking... like you had to do in Elder Scrolls Morrowind/Oblivion or you would accidentally level with crappy stats.