Some good advice is to find heart statues, if possible, and add them to your arena (preferably set to a repeater timer). That should help with all arena battles.
What also helps is crafting as many status effect potions as possible of each kind, and carry all the useful for battle types on hand (3-5 of each). Just as the battle begins, hit 'B' to drink them all to gain the buffs at once.
Regarding battling The Destroyer, I advise you take the time to craft a Dao of Pow. It's a good weapon to have, and it doesn't require any materials from any of the late-game bosses to craft, and against The Destroyer, you can smash him a good 12 times per shot (accumulating a good 1500 damage per swing), provided you let him pass through you (regen potion, a cross necklace, and the heart statues will make you last longer than you think, with the occasional mega potion; the (obsidian) shield will also help in keeping you immobile for the best shot opportunities).
As for the other guys, kill Destroyer first and use it's remains to craft a Megashark, and use the Megashark with a full stock (1000x) of cursed bullets (Thanks to the 50% chance of no ammo spent, you should more likely shoot up to 1500 bullets) to take on the rest. That ammo will last. When facing off against the twins, take your time and focus on one eye at a time. I suggest the fireball one first, and then the laser one last. Fireball is a pest (chases you and sets you on fire; second phase being even worse), and laser eye keeps it's distance all the time. And for Skeleteron Prime, again, Megashark. Keep firing, and make sure your buffs are always active. It will take awhile, but you should be able to win more easily.
However, what always helps, armor, armor, armor. Take the time to craft a full set of adamantite armor, and just as well, earn enough cash to reforge all the equipment you're going to use to in battle to have the +4 defense buff each. A set of angel/demon wings won't hurt either (and have those +4 defense as well (barding, I think the prefix was called)). In my battles, I had the full set, and at least 4 of 5 equipment with +4 def, so I was well over 60, even 70, defense when I battled).
Oh yeah, and don't knock magic books, they help a good deal and they also make use of the mana regen buff while at it. Demon Scythe (especially Godly) works wonders in damaging these guys along with a followup volley of Megasharking (or Dao of Pow in Destroyer's case). With Demon Scythe, lead your target into the scythes as they begin their slow acceleration, and watch the life plummet (and like with Destroyer, shoot for the head as he goes through your guy, and then follow through with Dao when you run out of MP, and repeat once you're full again).
EDIT:
I was surprised at how easy Destroyer is compared to the other guys. I actually was able to kill it twice in one night with the same setup I mentioned.