Or, you could just put the expensive garbage on display in your house. I never got into that much, but it seems like most of the people writing about the game online spent most of their time on interior decoration.
Well, you do get some fancy houses, and I collected every notable thing in the game (i.e. gems, scrolls, weird static items, and every weapon with a good material or too expensive to sell at price). I didn't really do any interior decorating, but I did have a floor in my Telvanni stronghold dedicated to holding all my special crap. I dunno, it just felt right.
Decorating Uvirith's grave was why I got my second mod: Book rotation, letting you place books every which way (my first mod was to get rid of the annoying magic glow).
Gathering books in Morrowind was amazing. Maybe it is just because I am a book-obsessed academic irl, but Morrowind got something right, that Oblivion and Skyrim didn't quite.
There were the skill books: collectible because precious. All that Oblivion and Skyrim kept as a legacy.
There were the Canticles of Vivec: nonsensical gibberish (+skill) for some, metatextual commentary on the structure of gaming piercing the fourth wall for others. I held the latter interpretation loving the tongue-in-cheek use of postmodernistic play as religious dogma (by Vivec, god of poets, liars and other tricksters).
Finally, there were the books on the Dwemer.
Some popular, some utterly unique.
The best of all, the Rosetta stone, central to the archmage quest, perhaps the best written quest (again to someone deeply traumatized by academia) in the game, where you solve an ageless mystery and displace the archmage, not by killing a monster, or finding a treasure, but by interviewing scholars, reading, comparing data and making a scholastic argument
Sure you could just kill the archmage, but beating him in academicism: priceless.