Paladin's Kit seems fine. Feats are a bit off; Spirited Charge has Ride by Attack as a prerequisite, so you can't drop Ride By and keep Spirited Charge.
Spending wise, I see a couple things you could do:
1. Upgrade to Masterwork Full Plate. Despite what people had been saying before, Full Plate and Banded Armour have the exact same armour check penalty. Going to Masterwork lowers it by 1 and lets you get your armour enchanted later. Total cost: 1400 gold.
2. Speaking of enchantments, you could get your shield made magic for 1000 gold, or your sword made magic for 2000.
3. Wand of Cure Light Wounds. For 750 gold, you too can be the party healer.
4. A mount. Mounts you can buy include:
Can-fit-in-a-canoe-sized: Donkey (8gp), Riding Dog (25gp), Pony (30gp), War Pony (100gp), Dire Weasel (400gp), Worg (600gp and wages)
Too-big-for-canoes-sized: Mule (8gp), Camel (8gp), Light Horse (75gp), Heavy Horse (200gp), Light Warhorse (150gp), Heavy Warhorse (400gp)
Oh-gosh-its-flying-sized: Hippogriff (4000gp), Giant Eagle (5000gp), Spider Eater (6000gp) Giant Owl (6000gp and wages), Griffon (10000gp and wages)
1: The argument against full plate is that you need somebody to help you apply it. So I'd either need some sort of hireling or a fellow PC that my character trusts enough to help. Although, when I looked up full plate on the Wiki, I noticed nothing about time it takes to put on and how much help is required...
2: From what I have read on the Wiki, enchanting the shield only increases its hardness and HP. I think it would be cheaper to replace the shield than to enchant it and protect it from destruction. Am I missing something?
3: I don't have Use Magical Device as a Class Skill. In Pathfinder using a wand w/o investment in that skill isn't a great idea. Is that different in 3.5? I'll likely get some consumables, an as a Paladin I have a bit of built in healing power.
4: Heavy Warhorse looks to be right in my budget range. Kinda sad there are no amphibious options listed though.