There's still only so much matter around that can be repurposed. I assume, but cannot prove, that the sorts of magical powers often ascribed to superintelligent AIs would require so much sheer mass to be obtained, especially since the beginnings will be especially crude, that somebody will notice long before the thing vibrates some electrons in its power cord in a way that creates a wireless signal, which it uses to connect to the Internet and break into laboratory and manufacturing systems and start cranking out nanobots that kill us all in one coordinated attack with no chance of resistance. Surely, the chunk of computronium the size of NYC will be spotted.
More to the point, there are also scaling factors on how much hardware you can glob together to work on the same task. There's overhead on coordinating and reassembling the subproblem solutions and so on, which is why you don't get 2x performance by assigning 2 cores to do something you used to have 1 do, even if it is something that can be neatly done in parallel. Signal propagation delays also become relevant at large enough scales. So you can't just pour raw processing power into making more processing power and expect your returns to be anything but diminishing.
This is why we have bureaucracies.
We're still low on the sigmoid, but it's not really exponential in the long run, is the point I want to make.