Just to add my 2c: I feel that, physically speaking, "animal men" are useful critters to have running around in the game, for the sake of diversity, but I'm really in favor of expanding the "animalmen" designation into several separate, and more deeply defined, entities.
So that, instead of running into a random, generic "walrus man" group on a glacier, say, you might instead encounter a diverse northern tribe, or several, made up of several agrarian clans of what happen to be walrus people, penguin people, and seal people, for instance, who might be at war with the nomadic polar bear/arctic fox tribe, or tribes, in that area, but allowing both of these different and hostile tribes to each contain a population of wolverine, and porcupine people.
I would then give the larger tribal groups their own separate ethos and possibly even slightly different languages, created goods, etc., to differentiate them from each other, and to allow the possibility of adding additional entities, that happened to incorporate the same or similar types of "animal men", but which may have an entirely separate culture, morale system, and trading value.