Well my best friend does enjoy to play DF, I got him into it after we played a few games like clonk. Might have been from reading pcpowerplay. Then our other friend is pretty big on roguelikes never on ascii or anything lacking graphics, he was willing to do a share fort with us till I jokingly changed it so dwarf woman had beards and I told him. Meet someone else while playing Age of empires 2 they played dwarf fortress as well. Then there was this British guy in the clan I'm in, big on 40k and pretty much min maxed everything even space station 13 (He always liked being HoP making people fill out paper work and giving them the wrong ones), he ended up playing naming every dwarf after everyone.
Well thats mostly every friend I know that plays DF or has attempted, everyone else pretty much won't touch it because of the lack of grahpics mainly or that its overly complex. Hell I played DF in ascii first thing, when I got my friend playing it we both used phobus... took a while to see I wasn't liking the fact I knew who was each dwarf just by the picture and was hurting my imagantion and I hated the look of terrain for that set. Went through alot of tilesets CLA for a bit then through a few random ones before finding taffer's one with the diagonal hollow walls, that set is beyond perfect dwarfs might need a change but I love it sadly forgot what colour I have used with it.
I wouldn't really know of how the outside world sees us I think its just like everyone likes different games, though the fact that ascii seems like the biggest wall for people to get over makes it hard for people to accept that DF could be fun for them. Its much like everyone having a different play style while playing DF, I just don't use bridges or cage traps for defense its so much more fun to attempt to wall off when the danger arives and either fail or succeed. I pretty much avoid using exploits unless it fixes a annoying bug (quantum stockpiles because bins are broken) don't like min maxing either.