Many gemstones make ideal mortar and pestle sets because of their hardness. A quern is a large stationary mortar so the same rule applies.
Example:
http://cgi.ebay.co.uk/rose-quartz-pestle-and-mortar-/330547860116I had an opaque white quartz mortar and pestle set at one point and it worked fine. It's not that unusual for a mortar to be made out of quartz; it has a mohs hardness of 7. You often see glass mortars in kitchen supply stores and quartz is significantly harder than most kinds of glass. I used to work at a lab that had a automatic zirconia mortar for grinding mineral samples; zirconia is more than 100 times harder than quartz but it's very expensive.
Sintered alumina mortars are another common lab staple; alumina can be easily manufactured but corundum, ruby, and sapphire are all forms of naturally-occurring alumina that could theoretically be put to the same use.
If you're cooking up food as opposed to chemicals, mortars made of smooth substances like quartz are better for mashing moist ingredients because they don't stain but they take more work to grind things as finely as coarse stone or unglazed porcelain will, and they can start to spall if they get chipped. Metal mortar and pestle sets are very strong but they can add undesired flavors to food and they're hard to keep clean.
Needless to say, diamonds are the hardest naturally occurring mineral. I've never heard of using one as a mortar though, as they're fairly brittle and sensitive to heat compared to the various forms of alumina.
In short, though, toady's use of gemstones as a material for mortar and pestle sets is entirely appropriate.