Alot of people here claim that the (lack of) graphics are the biggest hurdle for new DF players.
I just don't see that as a very strong dissuading force. The main points are, in my opinion, the unintuitive interface and the lack of content (read story). You say people don't look at gameplay, they do. There is very little good gameplay in DF.
People can get over graphics, unless they are the kind that play Halo or something like that all day. In that case who cares about them? We don't need those people.
However, the interface, when you first see DF, you don't understand it. You don't get anything. When I first played, I thought "Ok, how can I move these little men?" I tried selecting them and everything. Then I decided to stop playing, and took it up again a few months later. When at first you come to DF you just don't get it. It is so alien to any other game, you have to stop looking at it as a game and start looking at it as a simulator to understand it. Then you understand that the interface is merely a means to the an end, it does not have great gameplay, games with great gameplay have intuitive and fun interfaces that make sense, in DF the interface gets the job done. The interface is a vital part of gameplay.
There is also no content (in the context of story elements) or purpose in the game. At first its simple, you need to set up a base and learn how to play, then you can get to the good stuff! Yet the good stuff never comes, there is no story, the siege comes and it seems like some kind of a crappy automated mechanism. Suddenly you feel lonely, you see your dwarves now for what they are: data. Lifeless. Replaceable. There is no goal in playing the fortress. I really like RPG computer games (and with that I mean stuff like Avernum or Baldur's Gate II) so this hit me pretty hard, I like well-constructed and immersing worlds. There is a cold randomly generated world in DF, and it shows.
I'm not saying DF isn't a great game, well, maybe I should say DF is great, because I don't think its really a game. And as such we come to (in my opinion) the greatest problem of DF: false advertising. Everybody says what an awesome game DF is and how somebody should try it to feel the awesomeness. The fact is, most people expect something when it comes to awesome games. DF is straight edge awesome, but it is so different, difficult to describe in words, the only way to understand it is to try it, just like the only way to understand the scope of shittiness that is the Star Wars Holiday Special is to watch it, because it operates on terms outside of our normal thinking/boundaries.
(On a side note, it may seem I'm bashing DF, I'm not. I love it. I'm just approaching it from the view of somebody who doesn't know about it and is confronted by it. I was expecting something completely different when I first started playing DF and was operating it in ways I expected it to work, but I was wrong.)