Dove is a moisturizing body soap, but from what it sounds like, you are having a problem with hard water producing soap scum, which causes skin irritation.
Since this is an issue with alkalinity, try adding a few tablespoons of table salt to your bath water before getting in. (If you use a shower, consider a water softener-- but adding salt to the tub is cheaper.)
Dove is a moisturizing soap-- which may be part of your problem. Soaps form a complex, insoluble compound when dissolved into hard water. The hydrophillic sodium ion on the hydrophobic fatty acid chain gets replaced with calcium or magnesium ions dissolved in the water (from carbonate minerals, usually). This changes the number of bond points from 1 to 2, causing the soap to have 2 hydrophobic fatty acid chains attached to a single calcium or magnesium ion, which is then no longer soluble.
obligatory wikipediaTry using a mild detergent instead of actual soap. This may sound counter-intuitive, as often detergents are more harsh on skin that actual soaps, but where hard water is present, the reverse can be true, since the detergent requires less product to get the same clean, and leaves less residue.
Adding salt to the bathwater, stirring it up, then letting it sit a few minutes allows the calcium and magnesium ions in the water to re-associate with the chloride ions added, which are more electronegative than the carbonate complexes they are usually associated with, which then permits the formation of sodium carbonate, (baking soda) instead of calcium or magnesium carbonates in the water. Once this reassociation is complete, the formation of soapscum is far more difficult because the chloride ion holds onto those magneisum and calcium ions far more tightly, and wont allow them to replace the sodium of the soap complex. Soap lathers better, cleans better, and rinses cleaner.