The things affecting the terminal velocity in the natural environment are the power of the gravitational field (almost a constant near the surface of a planet) and the air resistance (varying heavily depending on the shape and other properties of an object). Thus heavier objects don't fall quicker.
They are, however, heavier, so the total energy/momentum of the impact are greater.
The helmets are designed in such a way that they try to distribute the force of the impact over larger area. In medieval times people wore quite thick padding underneath chain coifs/helmets that protected them against blunt trauma. That's why they worked at all, as getting hit in the head can easily cause concussion which makes you usueless in a fighting context long enough for the enemy to finish you off. Modern helmets are no different, they use various technologies to distribute the force applied to the outer hard surface of a helmet. Same thing applies to armour. It was padded, so that blunt force was dissipated more easily. Today, we don't se effective body armour on soldiers mainly because of that: often even if we stop the bullet so that it doesn't penetrate the shot victim, the force it carries is so big it will shatter the bones and internal organs hidden behind the protective gear, resulting in a quick death. There is simply no easy way to dissipate such a directionally focused energy in a small space comparable to the thickness of clothing.
Which gets us back to the energy/momentum. Assuming you wear a stiff piece of armour (like a plate breastplate) when you hit the earth (or the other way around, doesn't really matter) your kinetic energy at the moment of your "landing" gets used on plastic deformation of the ground (if any), plastic deformation of your armour (if any), plastic deformation of yourself and a little heat. So more energy means more potential deformation, what in turn means more damage. So heavier objects falling can potentially receive more damage from the fall. That's why bugs can fall 2m and do not harm themselves while humans may trip and die.
But at the same time, some of the energy went into the deformation of the armour itself (don't count on it when you're wearing chainmail though). So the (plate) armour at the same time both increases the damage you receive due to weight (energy) and reduces it due to stiffness. If there is padding underneath, it will help distribute the force so it dissipates more easily (probably converted into some form of molecule movement a.k.a. heat). Hopefully the energy that will inevitably reach joints and other vulnerable areas will not be enough to "deform" them permanently.
Same thing applies to internal organs. They are like boiled potatoes in a sac filled with dense soup. They can move a little inside your body, so they will make their own little "fall" on your skin, from the underneath. When you wear an armour the energy that gets used on deformation may slow you down a little, and in consequence maybe help your organs cope with the extreme deceleration. Hopefully your intestines won't then get squashed by the rest of your body, clothing and armour falling onto them (that's why you have a ribcage, but that's not infallible unfortunately).
TL DR; If I were to fall from the roof of my house I would wear a suit of plate armour with padding underneath if I had the choice.