It would probably lose the appeal for some players. As mentioned, the limited graphics allows imagination to fill in the blanks, aided by rather detailed descriptive text.
However, it the only change was nicer graphics, the gain of appeal would probably far outweigh the loss of appeal (apart from those suffering from the change, of course). Tile sets make a great job of widening the appeal of the "graphics" (under the constraints of the underlying logic), and probably relieves a fair bit of pressure on Toady to update the "graphics".
An important reason for why the graphics are as simple as they are is that they do not add a huge update burden when DF mechanics are changed, and so leaves Toady to spend most of the effort on the DF mechanics, rather than spending half of it on updating graphics (and he's told the story of the precursor to DF with it's 3D graphics [attempt] as a warning example).
Likewise, the UI is crude and confusing not because it's intended to be, but because that's where it's ended up when the focus has been elsewhere. Toady isn't happy about the UI, but it's still not high on the agenda (and personally I'd rather have Myth & Magic as the arc after the upcoming one than a UI change one, but on the other hand I would be happy to eventually see a change to 16-32 bit "graphics", allowing for full tile sets, and it wouldn't hurt if the color range was expanded from 15 to 255 or RGB).