Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[Shinotsa]

A clinical trial of 2000 showing no negative reactions means that the overall risk is less than 1/2000 anyway...

Plus, all adjuvants have multiple of such studies associated with them, allowing for far greater certainty

[Reelya]

Not necessarily. say the chance is 1/2000. the chance of not getting any "hits" in a sample of 2000 is (1999/2000)^2000 = ~0.3678, so a 1/2000 chance will turn up negative 37% of the time. Not very accurate.

But even a 1/1000 chance will turn up negative (999/1000)^2000 = 13.5% of the time. So you can only e.g. say there's a 63% chance that the odds are no greater than 1/2000 and a 86.5% chance that the odds are no greater than 1/1000

If you want to state a 95% certainty with 2000 trials that turn up blank, you can say that there's a 95% chance that the odds are no greater than 1/668, since (667/668)^2000 = ~5%. So we can put an upper limit on the odds, with a chosen degree of certainty that these odds are true. But it's not 1/2000 - the actual resolution is always quite a bit less than 1/(sample size).

Of course, the above math is only if you don't get any hits. It works out differently if you get 1 hit, 2 hits etc.

Say you got 1 hit, and the true odds were 1/1000. the chance of the first one being a hit = 1/1000 * (999/1000)^(2000-1). Then multiply that by 2000: since you add all the chance of getting 1 hit for each of 2000 trials together for the total odds of getting "1 hit out of 2000, when odds = 1/1000". The odds are then:

2000/1000 * (999/1000)^(2000-1) = ~27% chance of 1 hit, 1/1000 chance @ 2000 trials.

5% chance works out at 1/275 if you get 1 hit:

2000/275 * (274/275)^(2000-1) = ~5%. so if you get 1 hit out of 2000, the chance was no greater than 1/275 with 95% certainty.

I think i got this math right, this is all off the top of my head. let me know if I made any logical or math errors.

[Neonivek]

it is pretty much statistically impossible that 2000 trials to happen with no results.

Since you are asking that NONE of them to develop any sort of symptom after having a vaccine.

You would have results even if you were using Potato chips.

[10ebbor10]

That's the Nocebo effect though.

[Reudh]

you can simply compare lives vs. lives

Except that a lot of things people are being vaccinated against aren't fatal in the first place. I suspect that most people who get annual flu shots, for example, aren't really doing that because they expect to die if they don't. They get vaccinated to avoid the inconvenience of possibly being moderately sick for several days.

Or consider the Gardasil fiasco, when Texas got the bright idea of requiring 7th grade schoolgirls to be vaccinated against a non fatal STD that according to the CDC "in most cases goes away on its own and does not cause any health problems."

Bit late, but it's the primary precursor to some forms of cervical cancer.

[Neonivek]

That's the Nocebo effect though.

No it is just raw statistical inevitability.

If I were to do 2000 studies connecting eating potato chips to "Something bad happening"... I'd get at least one correlation. Even if it was a double blind test.

Bit late, but it's the primary precursor to some forms of cervical cancer.

True, a lot of diseases do not do any damage themselves.

[Skyrunner]

lrn2latex



(re: main post)

Sum ((chance of each risk) * (magnitude of risk)) / Sum ((chance of each benefit) * (magnitude of benefit)) = risk/reward ratio.

[penguinofhonor]

Reelya, where were you when I was taking statistics? That's a better explanation of statistical confidence (or whatever it's called) than my teacher ever gave me.

[GavJ]

Additionally, if you have ever done research you know that you have to disclude a handful of cases from the data for miscellaneous reasons

I have for years and years with human subjects, and it has never been considered acceptable anywhere I have published to not include itemized breakdowns of any non inclusions including specific reasons (and genders for that matter) The FDA is just more lax than journals, apparently. Take that as you may..

Like I said though, I ignored that issue for my estimate. I'm merely keeping track of conservative estimates made, not justifying any sort of math adjustment.

A) Repetition of studies

I have never seen a repeated clinical trial for any vaccine. Maybe it has happened here and there, but the vast majority have one clinical trial.

Also, as mentioned, I already generously multiplied the sensitivity by 10, already accounting for multiple repeated studies in case they exist (which again I haven't actually seen). Specifically in anticipation of stuff like this.

Large scale studies on vaccines are done not once, but multiple times

Okay, can you find me 5 different clinical trials on the MMR vaccine Priorix? (Keeping in mind that even if you do, I've already accounted for that hypothetical, but I'm still interested).

It seems you classified risk on a linear scale rather than by risk of spread - which is all but eliminated at 80% immunity.

No, I was using the terms risk and benefit from the perspective of vaccination. Lower risk of spread of disease is classified as a benefit. Please see the graph in section #5 of the OP. "Risk" here is risk of vaccine damage. it's simply the other way around from the words you're using. Notice that benefit slopes, and that it has a logarithmic shape which is indeed near its maximum benefit ("all but eliminated" depends on other nations not just immunization rate here) in my graph at around 80% immunity (please be sensitive to it being a hand drawn MS paint graph, I'm no wizard)

Immunological responses to a vaccine would be STRONGER in babies with healthy immune systems, plus the data itself would not be confounded by unrelated illnesses. Screening out genetic disorders and babies who aren't developing normally/are sick often is a great way to standardize results. Otherwise we'd be complaining about all of the cases they had to exclude to get any meaningful data, rather than just displaying a spread of problems all caused by external factors.

http://en.wikipedia.org/wiki/External_validity
Explaining how stuff would work really well in a made up world is not the point of a clinical trial.
In reality, drugs can have interactions that are greater than the sum of their parts and thus require clinical trials BOTH in isolation like you explain AND in any common cocktails (which for vaccines these are almost always given to kids in bigger bunches). As just one example. In the cocktail version, what you would do is then count up complications, and subtract off the expected complications from baserate + each of those other individual cocktails = the cocktail overall synergy effect, which you can both compare to other vaccines' and as an adjustment to overall risk compared to not vaccinating.

Again, keep in mind that I gave them a mulligan on this and didn't actually take it into account, am simply noting.

@Neonivek:

That is why you get a lot of babies and have a control group.

A lot of babies: Yes, they run thousands, but not a large enough "a lot" to get to the numbers they need to compare to the small disease benefits at current vaccination levels for some things (like measles)

Control group: Most clinical trials ALSO do the completely irresponsible thing mentioned earlier where they compare to another vaccine and not anything else, not to other ACTUAL control babies who for instance haven't been vaccinated with anything in the last month (never vaccinated at all would be scientifically preferable, but I'm being realistic). Since nearly every clinical trial does this, most vaccines are only testing their brand-specific dangers, not the actual danger of the whole vaccine that a child actually gets.

A reasonable actual experiment here would have 4 conditions, assuming ~1 month is a standard complications window, which it is:
Control 1) Babies with no vaccine in the last month
Control 2) Babies with whatever other vaccines in the last month an MMR receiving baby would typically have on average.
Experiment 1) Control 1 + MMR, i.e. Babies with only MMR in the last month
Experiment 2) Control 2 + MMR in the last month, i.e. full average cocktail as seen in normal usage.

vs reality most of the time:
"Control" 1) Another vaccine only in the last 30 days for the same disease
Experiment 1) This vaccine only in the last 30 days.
This is like using McDonald's as your control group for a study on the health effects of Burger King.

@Putnam

A clinical trial of 2000 showing no negative reactions means that the overall risk is less than 1/2000 anyway...

It is lower than that, due to uncertanties. As I mentioned previously in my example of a 5,000 subject trial, the most they can guarantee at alpha = 0.05 would be 1/1,100 rate or lower, for instance (so 5x less rate guarantee than the subject number).  It changes with population size. You have to use a binomial probability function. (A normal curve is sometimes used to approximate for large numbers)

Edit: to establish a rate of 1/10,000 or lower guaranteed at alpha 0.05, you would need a total population of 45,000 research subjects. A typical clinical trial has about 1,000-2,000, large ones go up to 5,000 or a little more.

To establish a rate of 1/60,000,000 at alpha 0.05, you'd need ~300,000,000 subjects

Edit #2: I use binomials because I'm still mainly focusing on death, and you're dead or not. You can also do e.g. differences in severity of something, but that typically requires even more subjects to establish a given rate.

[Shinotsa]

I mentioned that specific adjuvants (the parts of a vaccine that would cause a negative reaction, since otherwise it would simply be antigen which we are exposed to all the time) are tested repeatedly, generally through their inclusion in different vaccines. Very slight negative reactions were debated about some of the older ones, so now those are no longer used. Even for any single vaccine there are dozens of studies, it's just that many are post-marketing studies done on a vaccinated population.

Also - you didn't address the case of diseases such as tetanus that have a constant risk/reward ratio independent of others being vaccinated.

[Neonivek]

If you want to exclude data you VERY WELL better at least include the reason why as well as the result.

For example if you did a clinical trial of a drug... and someone got hit by a bus... obviously his fatality wasn't the result of the drug and if you excluded it and the data you collected on them is pretty unimportant... However, you better show your work.

Unexplained exemptions is just oozing with dishonesty.

and HECK a lot of scientists pretty much say that you include it anyway, the size of the trail alone is to exclude the noise.

As for the person talking about cherry picking subjects. Remember that your experiment was different then the baby one.

[andrea]

Also - you didn't address the case of diseases such as tetanus that have a constant risk/reward ratio independent of others being vaccinated.

He did. His logic returns an optimal vaccination rate of either 0% or 100% in such cases of constant risk/reward ratio. And in fact , without having to keep track of herd immunity, it should be easier to see which of those 2 extremes is the optimal point ( does the vaccine kill you more often than the disease?). I am going to make a wild guess and say that tetanus is worse than the vaccine.

[GavJ]

I mentioned that specific adjuvants ... are tested repeatedly, generally through their inclusion in different vaccines. Very slight negative reactions were debated about some of the older ones, so now those are no longer used. Even for any single vaccine there are dozens of studies, it's just that many are post-marketing studies done on a vaccinated population.

The whole point of these chemicals is that they interact with the system of the antigen and the immune system. It is their very nature and reason for existing in the vaccine to synergize with these things in non-purely-additive ways. They aren't just inert fillers or something. Therefore, only testing them individually, or relying only on results from their activity with regard to different antigens or mixtures of adjuvants is almost certainly not valid or meaningful.

...otherwise it would simply be antigen which we are exposed to all the time

For injections, no, it is NOT an everyday natural occurence for antigens to appear suddenly in the very middle of your arm muscle and nowhere else. You cannot assume that these will have the same effect as the same antigen dose in a natural exposure event (usually mucus membranes). Your immune system is not as developed or equipped for muscle teleportation as it is for mucus introduction. Introducing new threats from new areas may also potentially influence the immune system to become over-active to compensate, hypothetically leading to higher rates of myopathies or things like lupus or MS.

For both injections and nasal sprays: You also have to take into account antigen loads. Are they at natural levels nearly consistent with a normal natural contraction of a disease? Or are you dumping something like 20x more of the things on somebody (even if inactivated) than they would encounter in nature? This might have unexpected results and possibly also influence the nature of immune system function in general.

Speaking of inactivated antigens - depending on the method of inactivation, possibly in specific combination with characteristics of that antigen (virus? bacteria? toxins involved or not? more specific details?), they may represent things that your immune system is not accustomed to encountering or dealing with, and this could potentially have negative side effects in again re-calibrating your immune system away from an optimal state of expecting natural types of threats (either under- or over- calibration, or change in strategy of the immune system, who knows?). These require new testing as you change that kinds of stuff around, like everything else.

Also - you didn't address the case of diseases such as tetanus that have a constant risk/reward ratio independent of others being vaccinated.

As Andrea suggests, a disease that is primarily non-contagious would simply be a flat horizontal line for risk/benefits, and so you merely look at whether risk or benefits overall are higher or lower, and accordingly choose a blanket 0% or 100% target vaccination rate.

[Helgoland]

Or somewhere in between, if certain groups are more likely to catch the disease.

[Reelya]

I think the graph of benefit is wrong on the first page of posts. Many vaccines are not 100% effective, or even close to that. They're just effective enough to cause epidemics to peter out over time - due to herd immunity.

Say a vaccine is 30% effective. If you're the only person who had the vaccine, any negative effects of getting it might not be outweighed by your reduced chance of getting the virus. But if 90% of the population is immunized, each person has a much reduced chance of getting the disease - some people still get sick, even if immunized, but over time this diminishes to nothing.

So, the idea of looking at individual risk as each person decides one at a time if the vaccine is optimal for them - that concept fails to achieve the best result for the group, since no-one would make the decision to become the first person immunized. Effectively, it's fundamentally a group decision, not an individual one - "It makes sense for me to get immunized as long as everyone else is".

So, personal choice fails as the explanatory mechanism for the first person getting immunized. It only makes sense if everyone does it. But overall, it is in fact the optimal choice for each individual.

But remembering that if 100% of people are immunized then you might think "well it makes sense that I shouldn't be immunized if everyone else is because I can personally avoid anaphylaxis and other harmful effects of the vaccine, even if I believe in the good the vaccine does". But this logic works equally for all people,

In fact, if a vaccine isn't 100% effective, by not being immunized you also increase the risk to people who are immunized, thus reducing the benefit to them of being immunized themselves, if you are not. So "individual rational choice" actually drives the system towards zero immunization and brings back the disease epidemics. In this scenario, there is no equilibrium point from individual decisions, individuals looking out for themselves always drives the system to the worst possible state (zero immunization) even though it can be demonstrated that everyone was individually better off with full immunization. it's paradoxical but that's what you get when one person's decisions can harm other people and they don't factor that harm into their decisions.

I might put together a computer model exploring these ideas.