An idea I've toyed with regarding castes is to make it so that earth ponies gain physical attributes faster (maybe just strength, toughness and endurance). In the past I've tweaked pony mods to give each caste some attribute variations like that, mostly by just giving earth ponies naturally better physical stats to compensate for the lack of a second grasper (for all that it matters). Pegasi just got a big agility boost, which was utterly broken in .31.25 and .34.11. I don't know how it would impact .40.x with its new speed system.
I've also toyed around with just setting some skill learn rate mods on the castes, but not by more than about 30% on any given skill. That way earth ponies learn farming, shearing, milling and so on faster than unicorns and pegasi, while unicorns learn administrative skills faster. It's also a bit fiddly to get right and ultimately doesn't have a huge impact on the game. In any case, I certainly wouldn't prevent other castes from learning skills altogether, or even severely penalize them.
In general I agree that the cutiemark matters more than the caste. It's probably mostly a matter of culture that causes earth ponies to gravitate toward farming and thus earning farming cutiemarks, for example. In the past when I've done this, I was always working with pony mods that didn't have a cutiemark system, so such a system probably does enough to completely remove the need for caste level skill modifiers.