If something can take on near-infinite set of values with a random probability, in computing you would never do that as "infinite information per item", you'd have a central table of values, and do a random roll and pick one as needed. In that sense, a seemingly infinite set of possibilities per item actually implies that there isn't any data there, not that there's lots of data there.
Consider a set of chests in a game that you can open any time, and any one of a huge set of objects can appear in the chests, seemingly at random. Does this imply that lots of data is needed to specify what's in each chest? Or that there is really no data about each chest and it's drawn from a single data pool as needed? Entirely random processes imply there is no actual data to be had until you need it. So we can question the implications that quantum randomness imply to the true amount of information a system contains. Entirely random information that changes as needed doesn't really imply large data stores. They imply vastly reduced data storage requirements.
Also the idea that atoms can hold infinite amounts of data is sort of at odds with the implications of the heisenberg uncertainty principle, which heavily implies there's a limit to how much data they can actually represent. But ... if we accept that one atom can in fact hold infinite amounts of data, then we have also solved the RAM issue for a super-race running universe simulators right? They could easily set up each a single atom to represent as much data as needed, and simulate a large universe in only a finite amount of atoms, since each atom can hold infinite data.
Also with entanglement, if you have two piece of information digitally encoded, link two separate things to that one data field, you also get entanglement of the states of those separate pieces of information. Imagine a shared pointer in an existing computing system. You can save memory by sharing data fields between disparate objects, but tweaking one object will now change the state of the other object. This could explain the otherwise inexplicable "at a distance" effects of quantum entanglement that we can't explain in terms of normal forces or fields. In other words, many of these things can plausibly be explained as either the limits of simulation technology or "plain old quantum weirdness for no specific reason".