I don't know how to do it myself [or else I'd currently be a very well off man]. I just know it's possible.
But if you were to; yes, you want to convert your US/whatever your domestic debt is to a currency you expect to fall in comparison to the currency you originally had debt in. Basically, if the currency you're exchanging to goes down compared to the value of US/whatever your currency is, it costs less in US/whatever your currency is to buy back the debt. It's a bit silly and easily exploitable, and only available if you have the funding and accounting suave to pull it off.
My previous example works pretty well as it's had a bit less up and down action. If you had 1,000$ in US debt circa February 2012 and had converted that to Egyptian pounds you'd have to pay approximately 860$ to buy back all of that debt in Egyptian pounds, before interest. Doing this can save you huge amounts of money if the percentage loss in [exchanged to-] currency value outweighs whatever interest rates you're paying.
Of course it's likely much harder to be 'in and out' on a trade like this. Not sure how well central banks around the world look upon such methods of exploiting currency divergences.
Hence why times like now would be a lovely lovey time to load up on Japanese debts [buying corporate debts], as the currency will likely continue to rebound thanks to Abenomics. Basically, go short long-term on currencies that are inflationary like the Indian rupee and the Yen and always watch how the USD trades in comparison to everything else. Currency markets are neat.
Sorry about that last paragraph, that's probably a bit in the weeds..