Actually, they do have to face their enemies on the plains-and in the deserts, the swamps, the forests, the taigas, etc. Dwarves do engage in offensive wars against elves, goblins and humans, so they WILL end up fighting in places other than the mountains. And even they don't really fight high up in the mountains (unembarkable for a reason), but in the foothills. Technically, cavalry aren't suited to forests either, but the elves have unicorns as mounts. And for that matter...swamps are HORRIBLE places to have cavalry (beak dog goblins, I'm looking at YOU!) for obvious reasons. Dwarven cavalry would make more sense from a practical standpoint, since they could start out by using that cavalry to transverse the mountains before they have their tunnels dog, and to scout out new sites.
BTW, chariots were widely used by the Egyptians, and were the first wheeled war vehicle to be widely used. They do provide an advantage over plain foot soldiers in that they are faster, which makes it far easier to flank or strike at the enemy's rear. They would be better to give to humans, though.
Edit: the American black bear is only 3ft high at the shoulder, just thought I should contribute that. And male bighorn sheep are about the same. And it's worth mentioning that the horns on a male's HEAD can weigh as much as the bones in the rest of their body.