Hard drive space isn't at a premium. So clearing mapped areas to save space isn't worthwhile. However, storing saved maps for every retired adventurer does take some dedicated code to make it work, which Toady apparently didn't bother with it.
It's a good suggestion.
I'm not very familiar with Adventure mode but any map saved would probably be saved as a picture. A well traveled adventurer may make an image at about 2.5 - 3 kb. And if you have about 10 adventurers retired that can be 30 kb. That's not actually to bad from a taking up space point of view but when you think about a bunch of unused pointers in the code taking up memory in an already processor intense game it gets a bit more worrying. With how dwarf fortress saves it's worlds it might be able to get away with just saving what notable areas have been discovered instead of saving an image but that still doesn't get rid of the idea of unused pointers in the code.
I don't know what you mean by unused pointers being a problem. An adventurer map is probably not as simple as an image, but that's a good first order approximation of size. As long as the game does not load maps for inactive adventurers, there'll be no pointers to it, and it won't take up memory, just hard disk space. It's really just a matter of using a different file to store the map for each adventurer.