Magic is essentially "cheating"...it is a concept of something that breaks physics and the way the world works...I find it somewhat annoying when it is trivialized and weak. It also makes a game repetitive and boring when everything is within a short reach.
I am going to ignore "different settings, different rules" thing AND the "In what way is magic inherently strong?" I could give a speech about and instead focus on the gameplay aspect.
If you want magic to be 'special' that is fine but remember that there needs to be an appropriate risk versus reward (AND continued risk versus reward) but also you don't want to bar the player from being able to do anything interesting. The reason why magic is often given to the player from the start is because it allow you to develop magic as something consistent within the gameplay.
Which brings me to
Yeah, you're kinda missing the point - the idea would be that the fun is that becoming a mage/wizard is actually challenging rather than just 'press fireball + ice shield'.
Yes but your missing my point. That if you punish the player for wanting magic, just because you want magic to be this rare, expensive, and risky skill... It ceases to be fun and starts to be a chore. Remember that if you give the players a Bazooka, they will use it even if they have to write an essay every time... and they will rightfully call your game boring for doing so. If you want to make it risky so that every use has a chance to essentially wipe the character then people will simply not use it and it becomes useless.
Real costs and requirements (especially ones that make some sort of logical sense) are far more enjoyable than basic "mana" systems
You must have absolutely loved Ultima. Though I personally found reagent hunting to be a chore in the games that forced you to do it. (though they usually used both systems at once)
--
But ok... So you have magic that is rare and expensive. It might be "Powerful" but not only is a sword is as well... But because it is rare and expensive it also makes training it nearly impossible. So anyone who seriously considers magic is going to have to jump through quite a few hurdles.
Now what world changing abilities can magic have, to justify this? What can be done to mitigate it? What stops people from just stockpiling it? How can it not break the game?