The Upright Fun Stick is a special building that an adventurer can dismantle. All other buildings are workshops, and workshops cannot be dismantled by an adventurer.
If you're smart with military, you can actually get your stuff put away neatly!
1: Build a barracks and place a weapon rack and armor stand in it.
2: Make a squad of one dwarf, it doesn't matter who.
3: Assign that dwarf equipment individually, using the "Individual Weapon" option, find the artifact on the list - it'll be named so if you remember the name from the artifact list "l" (lower L) then you can find it quickly.
3: Assign this squad to "wear civilian clothes while off duty."
4: Give an order, wait, and cancel it.
This will cause the dwarf to go pick up his equipment - the artifact(s) - wear and/or wield, and then go put them away on the weapon rack and armor stand in the barracks. The only issue, is that these items are now free, and any dwarf will haul them to the armor and weapon stockpiles. Once they're placed on the rack, forbid the artifacts to make sure no one hauls them to the stockpiles.
The artifacts are now held on the building, much like items are held in the workshop, these are "inside" the weapon rack and will not be disturbed on abandon. To retrieve them, simply walk onto the weapon rack and pick them up.
This also works with all other equipment and also clothing. Anything that a dwarf can wear, can become a uniform. Any uniform can be placed onto an armor stand. Anything on an armor stand will remain stationary on abandon and can be picked up by an adventurer.