Oh come on, we were all "outsiders" once.
Most gamers of my age group (late 20s to early 30s) that I have met IRL are pretty casual (as in mobile gaming casual). Even the hardcore gamers (those that owned gaming stations and/or gaming laptops and also owned and played a prodigious amount of games) I know of have little to no idea what DF is. They are more preoccupied with the latest titles and hyped games from previously known quality game developers to bother with some indie game that has been around for some time that they've never even noticed. In my opinion, DF caters to a very VERY niche audience that knows what it wants from their gaming experience, isn't afraid of "insane" difficulty, non-existent "graphics" and horrible UI and has the patience to slog through a game as intimidating and time engaging as DF is or is rumored to be. Does that sound like the average gamer to you? It sounds pretty masochistic to me. Most people I know are looking for some quick entertainment provided in a neatly bundled package. Whereas all the entertainment value in DF is usually gleaned in between the lines. You'll have to have some imagination AND engage in that imagination while playing to derive entertainment from DF. Otherwise, its just a slog to play through. I reckon that's why I burn out on DF sometimes.
Most casual gamers I know would take a look at DF and decide then and there that they aren't interested in it at all. To them, DF is not even on their radar, so they wouldn't give it a second glance. If it wasn't for coming across, reading, enjoying and being intrigued by the quirky playthrough of Boatmurdered from the LP archive, I too would not have been interested enough to pick up DF. Even then it still took some convincing before I was sure that DF was the game for me, I read more playthroughs from the Hall of Legends here (and discovered that I really enjoyed them) and lurked around on the forums for quite awhile before even deciding to download the game and giving it a go myself. From then on, I was hooked. If not for a background in playing MUDs and ASCII games like The Two Towers, Nethack, Angband, as well as games that involved a modicum of reading like Kingdom of Loathing and so on, I doubt Dwarf Fortress would have sustained my interest. Indeed the ASCII had a certain charm of its on that harkens back to these older games.
Does that describe most us in a nutshell? I doubt it. Just like in real life, we are likely to be of a very diverse gaming backgrounds. Its just that if you aren't looking for something like DF, you aren't likely to stumble across it. Even then, it might take some time to harness the courage to take the dive and actually download and play. For the general gaming public, I reckon we probably don't exist. Does it even matter? Other than as some intellectual exercise, nope, not at all. I'll be happy building my own imaginary microcline castles in the sky, pursuing some megaproject or something and lining up my dwarves for the slaughter thank you very much.