For value accumulation, I generally go with items that are listed as weighing <1kg (the gamma "weight" symbol is kilograms, apparently) and with a value of 200 or greater. This leaves valuable objects as:
Ordinary cut gems with a value modifier of 40 or greater (precious gems and up, which are admittedly fairly rare)
Large gems with a value modifier of 20 or greater (semiprecious gems - there's a LOT more of these)
Lightweight clothing or art objects decorated with gems/precious metals (decorations have a base value of 10, the same as large gems, and add zero weight to an item)
Note that the standard quality modifiers apply, so a sock decorated with -well-crafted gems- with a value modifier of 10 is worth over 200 (a lot of the lower-tier opals have a value modifier of 10).
For reference, some experiments I did a while back showed that gold coins have a value-to-weight ratio of about 1250/kg, so any item with significant weight that isn't worth at least that much probably isn't worth bothering with. This excludes, among other things, pretty much all metal objects - particularly armor - almost regardless of how decorated they are due to their high weight. Only keep armor that you plan on wearing.
Storage is accomplished with lightweight bags (rope reed bags seem heavier than leather or wool), typically one bag for gems and art objects, one for decorated clothing, adding extra bags as each become full. Stuff all the bags into a spare backpack or lightweight chest (willow is the lightest easy-to-find wood) for easy dropping in combat if necessary. Even sticking to the lightest objects, it's perfectly possible to acquire a large enough quantity of valuables to start slowing you down in combat. The fact that significant quantities of food and water are rather heavy doesn't help.
One nice benefit is that valuable objects work as trade goods in any civilization that you come across, unlike currency. I would recommend keeping at most a single stack of any given civ's currency. Major shops that you haven't visited before keep 200 gold coins in their money stores, so ask for 3000 currency and add valuables to the deal until the shopkeeper accepts. In other deals, don't be afraid to overspend a little or just drop change given in silver or copper- weight is important, and it's pretty easy to accumulate enormous amounts of value anyway.