Re: Farmas USA
NVAX
Ya el último de hoy. Pongo los mejores post de una teoría vertida en IV que aunque no parece que se sustente demasiado (como argumentan entre otros Mark Reddish, que ya está fuera NVAX del todo), sí que ha generado un debate interesante. Es sobre el tema de falsos positivos en el brazo de placebo:
http://www.investorvillage.com/smbd.asp?mb=193&mn=7720&pt=msg&mid=16388447
flucurius. I teach advanced statistics to FDA regulated Fortune 500 companies. You can also forward my post to him
-
Today I decided to compute what the critical false positive rate had to be for the PIII trial to pass at p<0.05. Before you read on, what is your intuition about what that rate is? 10%, 5%, 2%. Guess again. You only need to remove a false positive rate of 0.25% in the vaccine group for the trial to pass. Let me go through this slowly...
Let's assume Johnny Sixpack gets his RSV shot and his Flu shot. Over the next 8 months he is infected by both the RSV virus and the Influenza B virus. His RSV shot works (produces high amounts of the appropriate antibodies) and his Flu shot fails to work. He goes to the doctor with a cough. RT PCR shows residual amounts of RSV virus in his blood and he gets counted as having one LRT symptom and being RSV positive. However I'm declaring this as a false positive because it is actually unabated Influenza B which caused this cough. Again we're assuming here that the Flu vaccine fails to produce the appropriate (or enough) antibodies in Johnny Sixpack. Remember the flu vaccine is only about 30% efficacious each year, and some years does worse.
Everyone with me? How many excess false positives do we need in the vaccine group vs the placebo group for the trial to pass? I'm not saying there isn't a false positive rate in the 5935 placebo group (though I believe this is less likely because they don't produce Anti-F antibodies; interesting asymmetry that NVAX failed to think about...); I'm asking what is the excess false positive rate in the 5921 vaccinated group? It turns out that critical value is 15 patients out of 5921 vaccinated - or 0.25%.
Let's look at this
If 15 of 5921 vaccinated are false positives then the efficacy rate for the primary endpoint is 50% and the p-value is 0.038. The efficacy rate for the secondary endpoint is 26% and the p-value is 0.035.
If 10 of 5921 vaccinated are false positives then efficacy rate for the primary endpoint is 31% but p fails at 0.23. The efficacy rate for the secondary endpoint is 21% but p fails at 0.084
You may wonder if I'm using the wrong statistical test but the one I'm using reproduced NVAXs p-values of 0.78 and 0.32 exactly
Comments, criticisms, clarifications....fire away. Could some of the symptomatic patients have actually had the flu? You decide
-
Unless you take a sample of the phlegm that the symptomatic patient coughs up and look for colonization by either Flu or RSV virus, how are you going to know for sure what's in the lungs?
-
There is no reason why the false positive rate would be any higher among the vax group vs the placebo group, so this should have a negligible impact. Obviously, the larger the trial, the lower risk here, and P3 was larger than our previous P2 trials. Unless I missed something, why would this be more likely to skew the results against us?
-
I do believe the false positive rate is biased towards the vaccine group in general. The placebo group won't produce and Anti-F antibodies so the presence of RSV and associated symptoms is more likely to be a true positive. The vaccine group will produce Ant-F antibodies, but just need a minute trace of RSV in the blood will force it to be counted as a positive if any symptoms show up, regardless of how poorly the flu vaccine works
-
He is saying the methodology for classifying symptoms as flu vs RSV is biased. A subject with flu like symptoms is more likely to be misdiagnosed as suffering from RSV in the vaccine cohort than in the placebo group. In experimental methodology we would say there is a confounding variable involved.
-
Exactly. There is a confounding variable. Look, in statistics one tries to compute a signal-to-noise ratio. And NVAX is trying to force flu symptoms in the noise part of the equation. But they can't do it without tissue sampling the lungs to see whether or not the colonies are influenza or rsv. PCR of blood is good, but it is not perfect, and I've written at length why not
All I'm doing is assuming a little bit of flu symptoms are miscounted as rsv symptoms and now your stats don't work. Just because you believe the false positive rate should be the same, doesn't mean that it actually is the same.
Everyone assumed the attack rate would be 5%, and it turned out to be less than 2%. There is a finite probability of all sorts of things happening, the combination of which can lead to failed p value.
-
Maybe I'm missing it, which is what I said in my first post, but why is "a subject with flu like symptoms more likely to be misdiagnosed as suffering from RSV in the vaccine cohort than the in placebo group?" It seems to me that a misdiagnosis of flu as RSV would be just as likely for either group, especially given the fact that its a blinded study...
-
Because all vaccine subjects would have RSV antibodies. Placebo subjects would have antibodies only if they contracted RSV. Any vaccinated subject who exhibits flu-like symptoms will be diagnosed as having contracted RSV, even when the infection was due to flu virus. perhaps this table will help:
Symptoms Diagnosis Validity
Placebo subjects
flu-like symptoms, no RSV antibodies Flu valid negative
flu-like symptoms + RSV antibodies RSV valid positive
Treatment subjects
flu-like symptoms + RSV antibodies RSV False positive
(due to flu)
flu-like symptoms + RSV antibodies RSV Valid positive
(due to RSV)
Hope this formatting holds up.
Would like to know if any treatment subjects were diagnosed as having flu.
-
I with ya except for the false positives using the RT-PCR test. My earlier reading on the test suggests it is very accurate for RSV, and the hypothetical you outline (flu versus RSV) makes not much sense as it is abundantly clear to me that trial investigators would have recognized this a potential problem at the onset.
--
(redplate/Mark Reddish)
All meaningless. You create the strange hypothesis that somehow residual RSV gives a false positive. If there is residual RSV it is not a false positive it is RSV. now you intended to say the symptoms were false positives. But the Pts are also tested for flu. Hypothesis is not viable.
It does not explain the data. But of course it is the fantasy of the day for a recount. The trial failed it will have to be repeated.
-
On what call or presentation did NVAX indicate that via PCR they test for the presence of all flu strains that might cause LRT symptoms? I would love to read or hear that first hand and put this issue to rest
-
It would seem one would need to come up with :
1) a rational basis for false positives in the first place, RT-PCR is always accompanied by controls for this - that said false positives do happen in PCR
2) a rational basis for false positive results to segregate to only the vax arm - this one is a very tough sell
And all that said my understanding is flu is checked for. On the matter of checking for each strain, were it me I would go pan-flu as I would not actually be studying flu, just want to know if it (any) is there.
-
Happy to respond to this. In your point 1 you say "false positive do happen in PCR". In this case we're talking about the mis-attribution of LRT symptoms to RSV when Inf A,B or C was also present in the blood. Not a false-positive reading of the instrument. I'm assuming that the PCR instrument is properly calibrated and maintained and works fine for this trial.
For your point 2. Let's take away the idea of systematic segregation. Let's just say in 100000 trials, on average, the mis-attribution rate of symptoms to rsv balances between the placebo and vaccine arm. I'm going to give you that. Does that imply that in any particular trial the mis-attribution rate is always the average. Clearly there will be some sort of distribution of error over one hundred thousand trials. Is that fair? Sometimes there will be more mis-attribution in the placebo arm and sometime more in the vaccine arm.
Maybe the average height of a man in the USA is 5'9". Does that mean every man I meet will be the average height. Clearly that's ridiculous. By the same reasoning I think it is unlikely to assume the mis-attribution rate of LRT symptoms to RSV where FLU IS PRESENT IN THE BLOOD will always be the same between the placebo and vaccine arms of 1000000 trials. Is that fair?
All I'm saying is that if the imbalance tilted towards the vaccine arm IN THIS trial by 15 out of 6000 people (notice I'm not saying in every trial, just in this trial) then you have a confounding problem with the stats that's serious.
-
If participants exhibit symptoms of respiratory disease they are screened for RSV infection using RT-PCR. RT-PCR confirms assignment of the symptoms to the infectious agent. Where, precisely, are you assigning the false positive event to if not the PCR?
On point 2 the purpose of the study is to determine if the vax alters disease outcomes. It did not as measured. It is not a dismissable point. It is the point. Again, one must present a rational basis for the results (that it didnt work) to be wrong and that in reality it did work. That requires systematic error or a confounding factor that independently determines the results of both arms (nullifying the validity of the results) Your "false positive " argument requires systematic error, I have seen no rational argument for systematic error to support that.
The alternative, brought up be the company, is that a superior independent variable (confounding variable) drove the results of both arms, thus voiding the ability to verifiably test the hypothesis (or null hypothesis if you will). The argument in this case being that low attack rate can serve as a confounding variable.
The whole height of average argument ignores that actual measurements are being made in the study. The height of the world does not matter. Within the study results were observed. That, and only that, must be explained. Given the poor attack rate one could make a confidence argument, but there were more events in the P3 than the P2 in any event.
-----
Otro hilo en el que se habla de la vacuna de gripe que se ha dado también en la Fase 3:
Remember those 'FREE' Flu shots given out for the Ph3 Resolve trial?
(redplate)
The immune system works sometimes in funny ways. Combining components together can create 'dominance' and suppress another component. It ain't a done deal until it is done. In particular any notion of superior responses etc, fact is we know nothing at this time and will not learn anything beyond the issues of symptoms that can confuse the flu vs RSV infection analysis.
-
Thanks for the expert feedback.
Any thoughts on why NVAX offered the free flu shots as part of the Ph3 Resolve trial?
Isn't it remotely possible that NVAX gave out the free flu shots for one or more of the following reasons:
1.) They hoped to duplicate the synergistic effect that was observed in the preclinicals where both the RSV and the flu vaccine seemed to work better together?
2.) They (NVAX, FDA, BARDA) are looking ahead to the RSV/Flu combo, a vaccine that could dominate both markets?
3.) A potential partner/buyer provided the free flu vaccines looking ahead to the combo?
4.) Preliminary data for the combo?
4.) Stan & Team NVAX are just really nice people ;)
I would say there has to be a reason.
-
I don't disagree with your basic comment but I think the synergy is not immunologic but a question of confoundning symptoms. RSV disease is flu like and flu is RSV like. So prevent both and see a lot less of the symptoms
I also agree that an individual with the flu is more likely to get RSV and vice versa. That is not a demonstration of enhanced immunity ofr the two together. It is demonstration that the immune suppression from having flu or RSV does not happen and leave one at increased risk.
The best part is that it cleans up symptoms.
And I strongly agree that the folks at NVAX are quite nice people
