If you want to attack with both in the same turn, yes. You can have Ambidextrous just to eliminate the off-hand penalty and be free to choose whether you attack with one weapon or the other like I do with my commissar in a different game. He has a pistol and a chainsword and he only attacks with one per turn, though due to the nature of Parry, a shield/sword/chainsword can be used for parrying even without Ambidextrous.
Let me explain how it goes:
You have neither Ambidextrous nor Two-Weapon Wielder. You have a pistol in your main hand and a sword in your off-hand.
You can't attack with both weapons in the same turn. The weapon in your off-hand suffers -20 to any attacks you attempt with it. Parrying with your off-hand suffers no penalties.
You have Ambidextrous but no Two-Weapon Wielder. You have a pistol in your main hand and a sword in your off-hand.
You can't attack with both weapons in the same turn. The weapon in your off-hand suffers no penalties. Parrying with your off-hand suffers no penalties.
You have Ambidextrous and one of either Two-Weapon Wielder (Melee) or Two-Weapon Wielder (Ranged). You have a pistol in your main hand and a sword in your off-hand.
You can't attack with both weapons in the same turn. The weapon in your off-hand suffers no penalties. Parrying with your off-hand suffers no penalties.
You have Ambidextrous and both Two-Weapon Wielder (Melee) and (Ranged). You have a pistol in your main hand and a sword in your off-hand.
You can attack with both weapons in the same turn (same Half Action), at a -10 Penalty to both attacks.
The weapon in your main hand suffers no penalties if you attack only with it. The weapon in your off-hand suffers no penalties if you attack only with it. Parrying with your off-hand suffers no penalties.
Fighting like an iconic IG Sergeant or Commissar (swinging both a sword and sidearm in melee) is quite costly, since you have to take
both Two-Weapon Wielder talents for it to work well.