I, too, struggled with maces and hammers in my own mod.
In reality, hammers specialised in their versatility, often having a spike or beak on one end along side a hammer bite on the other. Maces, on the other hand, were strictly blunt weapons with no (literal) armour penetration capability. They are virtually the same weapon, except the hammer has a bladed spike attack, which leads to problems with DF making bladed attacks get used more often than blunt, and which inevitably drove me to removing the spikes altogether, further stressing the weapons' similarity.
Historically, maces did have the niche of symbolism, much like the sword, and were often represented or used ceremoniously with connotations of authority. The hammer was strictly a brutal anti-armour weapon, through and through.
For what it's worth, as a weapon enthusiast: If you want to give maces their own identity while still respecting the reality of the weapon, it should be kept in mind that maces bite/strike rather than bash, unlike the war hammer. They do pin-point damage with very small contact areas to try and send their force directly through the armour, while the hammer attempts to turn the armour itself into an extension of the weapon with force distribution over a large area.