Well, no. It's not "This thing signifies one sound and one sound only." It's a "... What? Okay, that's another one for the ' pile," scenario.
What would be the point of that. It is assumed that the language in question has both pronounceable and unpronounceable sounds, and the former outnumber the latter (else what would be the point in a Latin orthography anyway). If there are several unpronounceable sounds, it makes sense to have several symbols for them, because while the human readers can't pronounce them, they can distinguish between them, and therefore read (which is the point of a Latin orthography for Chtulhuese).
There is literally no scenario in which the "another one for the pile" approach makes sense. None. If you're going that way, there's no point to having an orthography, since Rule Zero for any orthographic system is "Thy graphemes shall have phonetic meaning that can be derived from their shape or optionally, position, but it shall be a concrete meaning in the end." If an apostrophe in any position could be any of your, say, three unpronounceable sounds, then there's no point to having an apostrophe. Nothing will do just fine, since the apostrophe does nothing for increasing a reader's comprehension anyway if it has no precise meaning.
On the other hand, if we have several different symbols for Cthulhu noises (' for mandible rattle, " for side-spiracle hiss) then a human reader can distinguish between, say "kth"ha'" 'tree', "kth'ha'" 'forest', "kth"ha"" 'water' and "kth"ha"" 'ocean'. With an apostrophe, those would have been indistinguishable, and you'd get meaningless texts. The meaningless texts would be supposedly written by humans (since they're in Latin). Now you must assume your humans are slightly dumb and can't make up a functional transcription system for Cthulhuese.
I might see a reason. Say that the average human speaker, when confronted with such an ' is supposed to just leave a short gap in the word where normally one of the alien sounds goes (which might not be in the human hearing range). Why? Because to a native listener this does convey information, namely 'here goes a sound the puny human cannot make', which might help him interpret what said puny human is trying to say. Sure, the academics of that world probably have a whole system to represent the various unpronouncable sounds, but to the layman that has no real added value. Seeing a '&' or a '¥' or a '#' (using random symbols here), while each of them indicate the same thing for practical purposes, won't help him speak or understand the language better. Though I don't know the setting or the situation it was used, so this might not be valid.
Say dirtfarmer Bob sees an alien he knows approaching, he might say "hello Ma-(pause)-ry", giving "Ma'ry", even though its real name is 'Ma¥ry', where the '¥' sound is one dirtfarmer Bob can't pronounce.
Another reason might be that these humans simply haven't mapped the various alien sounds yet, and are representing them with ' until they can look into it better.
Yes, but if you're gonna pronounce them, then having at least an approximation for alien sounds is better, no? When you try to pronounce something foreign, you try to approximate since presumably, native speakers need to understand you. I mean, if you have no spiracles, you can just tap your sides with your arms and hiss, but just making a pause there isn't really the best idea. Why would he address her as ma-pause-ry if that adds nothing for both of them - for him it'd be easier without a pause, and she presumably knows that he has no spiracles and can't pronounce it, so why not just drop it altogether?
I must also reiterate that creatures that share most of their phonetic inventory with humans but have additional sounds that humans can't even begin to approximate are pretty improbable in the first place.
The last argument holds some weight, but then there are races that were supposedly familiar to mankind for a long time. For those, the "not mapped yet" hypothesis makes no sense. Besides, really, the more such unpronounceable sounds your aliens have, the less point there is for a human to speak their language instead of just writing on a markerboard or something, because with the number of unpronounceable and unadapted fragments increasing, his comprehensibility to them would decrease.
Sorry, I'm just feeling argumentative today.