imo, quas wex is a better build if you aren't sure you're going to win fast. If you've skilled exort primarily, your disables and stuns all suck mid-late game, which is when invokers exort damage starts to be less scary. It's not an issue if your team has plenty of cc already and you're playing a dps role, but the disable lengths are tremendously different between quas-wex and exort-quas invoker at around level 14 or 15.
It's pretty simple to balance him. nerf his bonus stats through qwe and the exort damage. I also don't think he's a hero where you can point to different skill levels and say,"He's bad here for the reasons he is good here", like you can with a hero like huskar. Most invokers will only use 2 or 3 spells for the entirety of a match and most invokers will only need to use 2 or 3 spells. Sunstrike, cold snap, forge spirit or tornado, emp are enough in most skill-levels. Even using more than that is a minor inconvenience, quickly overcome if you play him regularly, as astral clearly shows. Why should every hero cater to people who can't learn or practice? There are other AoS for them, that explicitly eschew heroes that require expertise or knowledge to play or play against.
I digress. The thing that makes him too strong is the fact that he can beat everyone mid. Sidelane invoker, skilling wex and quas, is still really strong, but his ability to out-lane mid with exort-quas, is what makes him imbalanced at all levels of play. The combination of really absurd auto-attack damage and excellent regen with quas makes him way too potent.
If my math is right, exort invoker is looking at 91 damage with one blade of attack and 1 ggbranch at level 5. Pugna, with the highest int gain, will hit only 75 damage by that time with the same items. If he skilled only stats to level 5, pugna could get 85 damage. Roofie can only give the individuals on his team +4 hp/s with maxed living armor (granted that comes out to 20 hp/s during the day for his team as whole). Basically, invoker can get the kind of damage and regen that most heroes can only get by crippling their build and still is capable of similar levels of cc and nukes. SF, with the same items, manages to get 100 damage by lvl 5 with max souls (3 raze, 2 necro).
----------------------------
I na'd and had some fun. Not much to do once they get mad and buy a gem and sentries, though. WAIT, I'll go blink and dagon 5!