Regardless of Dwarves being in regular contact with sand, or rivers, their mastering of glass still makes sense.
Glass can made without sand. Sand is just the easiest way to do it (which is why we do it that way).
Glass is made from silica. Sand is just the silica that remains when rocks erode.
The most common source that dwarves would likely find silica without using sand, is Quartz.
Quartz is one of the most common minerals in rocks (which is why its found as sand after the rocks are eroded). It's commonly found in many igneous, sedimentary, and metamorphic rocks. To name a few of the rocks which we commonly see in DF: granite, slate, pumice, rhyolite, sandstone (dwarves could easily turn this into sand too), gneiss (dependant on what the gneiss was originally), onyx and limestone, and of course, as raw quartz.
And, even without that source of silica, Obsidian is glass. It's natural glass. Dwarves find obsidian (and use it) all the time. It's not a difficult leap of faith to assume that they would learn how to make the stuff after being around magma (if you can make the leap of faith required to assume that dwarves work with magma, this should be an easy leap). In fact, being around magma, it was probably easier for them to learn how to make glass than it was for us humans.
The coloring of glass would be a natural step after they learned how to make glass. Red glass (tinted by gold) would be an easy thing to learn by accident for them. Gold veins are found in quartz-rich minerals (lode veins are often found in pure quartz, infused veins are often found in granite). Dwarf melts some quartz down that was originally holding some gold, makes glass, and lo and behold, it's red.
Silver, Tin and Copper is also found in quartz-rich granite, and would likewise result in easy, accidental colored glass discoveries for dwarves (and probably are how humans discovered them, as my own personal speculation)
(Hey, what do you know, some of that time I wasted in university taking Geology classes / listening to my grandfather (geologist/geology professor) paid off... discussing a game!)
(Wikipedia helped a bit too, to clarify the stuff I forgot)
Back to the original post - the (pre) medevial methods to make stained glassused the exact minerals to produce the exact colors in said post.
It's by no means a far-fetched idea for dwarves to know how to make these aditional colors.
(Personally, I'm not sure if we need all the colors, since it might be difficult to discern them all ("clear glass" is blue in-game)... but could be cool anyhow, and extra depth is certainly not a foreign concept to DF!)