I don't know why anybody would chose not to use an IDE, honestly.
Ever since the Visual Studio .net rewrite, I've found it to be terribly slow and bloated compared to a "simple" text editor like Emacs or vim. Still the best debugger on Windows, but unfortunately even the debugger has slowed down lately. There should not be a visible delay when I hit the button to step through a line of code... Besides that, I find that using an IDE encourages writing code that
requires an IDE to write, e.g., lots of unnecessary boilerplate everywhere because it's just so easy to crank out.
Emacs even has pretty competent autocomplete with Semantic now, which is all I would've used an IDE for anyway. I also really have an issue with the "project" style of organization most IDEs use, which makes writing tiny code snippets a real pain in the ass. Every time I've used VS it's shat out a half dozen folders god-knows-where to keep track of a bunch of completely meaningless data. With Emacs, I can just make a new file, then hit alt+m to compile. No hassle. If there's a makefile or build.bat then it'll call that, otherwise it just calls cl currentfile -Wx -Wall -Zi.
I don't actually think Emacs (or vim!) are particularly competent text editors, but they're as close to "good" as we've gotten. Still waiting for someone to write a great code editor that's not written in some awful interpreted language, runs quickly, and is extremely lightweight. Preferably with C bindings for plugins...