The above "rune smithing" is kind of what I had in mind. Engravers (or "rune smiths", etc, if you prefer a semantic distinction) would engrave Runes on the walls of your Fortress, which would be inlaid with gems/precious metals.
Here's what I have in mind for an (admittedly very elaborate) system, that I think would be broad enough to embrace a wide range of effects, but also interesting and rich/complex enough to be set into the incredibly detailed nature of the DF world:
The way I had in mind for it to function would be to make the various "runic alphabet" characters define what a given Rune would do, in a basic way.
Since it would be an alphabet, you could create spells by engraving "rune words" (perhaps called "Wyrds"? I'm fond of semantics, too...) onto the walls, in ordered groups.
The precious metals themselves would be used to define each separate runic word (or "Wyrd")---three characters set into silver or gold would define a given "Wyrd", that could then be laid beside four characters set in platinum or adamantium, or a single character "Wyrd" set in copper or iron, or 2 lead characters, whatever---and the metals themselves might indicate the overall nature of the spell (obviously, lead might be for malus effects, iron for combat spells, gold for resistance or "buffs", copper for healing, silver for transformation/change, what have you. "Demonic" magic might be governed by adamantium, with "holy" magic set in platinum.).
"Wyrds" might range from 1 to 7 characters in length, allowing for highly complex effects--and ofcourse tying into the mystical nature of the number 7.
Runes themselves could then further be defined in specific ways, by the gems inlaid into them. A powerful "demon-binding" rune might require specific runes set into adamantium, each inlaid with a single ruby. This fell "Wyrd" might be sufficient to dominate a single toad-demon, to, in effect, make it your friend and ally. To control a tentacle demon, 7 opals might be required instead. Control of a Spirit of Fire, on the other hand, might require 7 star-rubies.
If we assume 24-30 Rune characters (I'd suggest 27), and that each Rune could potentially be combined with all the others in "wyrds" of 1-7 characters, could then be set into various different metals, and inlaid with a host of different gems, then a near infinite number of effects might be possible.
The "spells" or effects the Runes caused could all be defined beforehand. Each effect could have it's own Raw, which would come into being in much the same way as a plate cuirass or a bone crossbow. You find the components (the runes), combine them in a formulaic fashion (on the walls, instead of in a forge), and then "create them" from whichever metal you chose. Gems would then act like spikes, as more or less of an afterthought.
A more important afterthought than spikes currently are ofcourse, since adding the right gem would define what a Rune exactly does.
To give an example: A Rune "Wyrd" that was being used to enchant a wand, might enable the wand to shoot fireballs, lightning bolts, OR a swarm of hornets, depending on the gem-type.
The same gems in a different Rune might enchant a ring so that it dealt fire damage, lightning damage, or poison, as a touch attack.
If those can be said to be "combat" attacks, set into iron, the same exact Runes, set into gold, might empower a ring to *resist* fire, shock, or poison damage,
If set into lead, the item worn might have a "curse" effect, by damaging the wearer with fire, shock, poison.
Set into silver, the same Rune-ring combination might allow the wearer to transform into a fire imp, a blizzard man, or an ant-man.
With each effect, each "Wyrd", or rune-effect, as a spell, completely defined from the beginning in the Raws, this system would have the advantage of being de-buggable before anything actually occurred in-game. I believe that this system should also lend itself well to Modding, and I think it won't end up being too out of place in the current overall "geography" of the game. Not a whole lot here requires the game to take a suddenly different direction, and everything here is already in the game in some form or another, so the pieces would just have to be strung together in a systematic way.
Another advantage is that, if the Runes are already being inscribed on the walls/floors/ceilings, and doors/windows/portals, etc. or whatever, then it would be easier for us, as players, to understand specifically *which* Rune "Wyrd" is doing *what*, *where*. If, for example, you have your dwarfs inscribe a "Wyrd" that keeps a door magically locked, you'd know exactly which door it is. It's the door with that specific "Wyrd" on it, and you'd just click on that door to get more information about it.
Weapons, armour, and other items could be handled in an only slightly more complicated way--every "Wyrd" in the game could be listed (like every animal, humanoid, etc.), on a given map, with a number beside it. Items linked to that "Wyrd" would then display a "Wyrd1" type message, telling you specifically which item had which "Wyrd" on it.
In keeping with the defensive nature of Dwarfs and Dwarf Fortress, having spells in the game exist as physical objects---the "Wyrds"---would allow them to then be destroyed, as well. Destroy the "Wyrd" and you instantly destroy the enchantment, rendering magical items inert, magical traps and locks disabled, magically bound creatures unleashed and hostile, etc. This could become as catastrophic as the erasure of a single "Wyrd" causing an entire floating fortress to fall out of the sky.
Dwarf rune-smiths might even have their very life-essence linked to the very first Rune they inscribe, forcing players to make strategic decisions about when and how they make the transition to magic, and who they risk allowing to use it.