But accessing this data would be fastest if you used your noise function to fill out an array first, unless you need to dynamically get new values.
If I was to be repeatedly accessing the values of various points[1], then I think I'd have a scratch-storage for the most recent thousand-or-million points[2]. Discard the unused (or shunt values/deltas to long-term off-memory storage, if you want to keep some changed information).
If I was just wanting each value once, then if generating the huge array is no more complicated than "for p[v,w,x,y,z] I want fn(v,w,x,y,z)", then generating the entire set to storage and then randomly accessing it may give more overhead than merely doing the fn(v,w,x,y,z) as required.
But I'm absolutely not certain what requirements the OP to this issue is looking to fulfil, I'm just talking in general. And possibly also out of the top of my hat.
[1] Either the original ones or ones recalculated with reference to neighbours... Reminds me of something I once did, only that was just in three dimensions (excluding 'time', but that was fully deterministic to the initial three dimensions of randomness, so it wasn't even a fractal, technically).
[2] Depending on data size and overheads needed for such a hash-type storage, plus whatever 'age' data is needed to maintain the oldest-out mechanism once it hits the top limit. This way, a fairly quick and low cost re-accessibility of previously calculated data, on the understanding that stagnant data might well have shifted out of the short term storage if it takes a while to re-query it all. B-Tree, R-Tree and related methods might also be useful culling alternatives to the age one, and probably better with space-related arrays. You could even maintain a relatively small dynamic array that (through a jumping offset associated with each axis, plus a value to indicate where the nominally 'wrap-around' data's real 'edges' are, in the array (a mere two extra chunks (words?) of data per axis!), to save on excessive shuffling if you wander over a point-step or two) covers say 100^5 points surrounding your particular area of interest, and lets you handly a lot of data in bulk in short-term memory and fish out/throw back data from/to long term storage (if applicable) in much more efficient larger quantities at a time. Again, probably no relevance to OP, but a general idea. Probably badly/insufficiently/excessively[*delete as inapplicable] explained.