What's evidence?

[Guest guest]

Evidence is information that demonstrates that a hypothesis is true - it's a

test that provides confidence the result was caused by the intervention and

that it predicts what others can experience.

The cornerstone of evidence is reproducibility - that the outcome was not

due to chance, or the natural history of the disease, or a confounding

variable, such as individual sensitivity to the prior treatment.

In drug testing the hypothesis might be:

* that drug A is superior to drug B when each or one is already proven

effective.

* That drug A is Active and provides Clinical Benefit ...

... that the benefit (disease control) is greater than the

risks (toxicities) of the disease untreated or treated differently.

That drug A is *active* is many times easy to demonstrate, particularly when

you get a dose-dependent response, consistently in a good percentage of

patients. But active is not the same as beneficial. The effect on the

disease could be offset by the toxicities on the body. Also, the response

might be short-lived, and the net effect could be harmful. This being the

most common reason that drugs fail to win approval for cancers.

Vaccines are a special class of experimental drugs ... a biologic type.

The hoped-for mechanism of action is to " train " the immune system to

recognize the cancer cells as not belonging by presenting a tumor antigen to

the immune system coupled with a highly immunogenic substance. The hope is

that this will trick the immune system into " seeing " an normal protein

(idiotype) on the tumor as being abnormal, leading to eradication of tumor

cells by immune cells.

So how do you set about proving that such an indirect strategy actually

works?

In early stage testing, you might give it to say 10 patients following

chemotherapy, and observe if they do better than historical controls. If

so, while recognizing this could be due to chance, you are encouraged to

test your hypothesis further. The controlled test you plan will prove or

disprove that adding vaccine after chemo is truly active - and, most

importantly, that it delays or prevents the recurrence of the lymphoma after

pretreatment with chemo.

Your statisticians work with FDA and agree on a sample size that's

sufficient to rule out the effects of chance. You apply random selection to

avoid patient-selection bias (cherry picking younger patients with low risk

disease). You give each group the same pretreatment. An independent data

monitoring team evaluates the results, blinded to who received the vaccine

and who received the control to avoid hanky panky, or biased intrepretations

of the results.

(Note: If you gave all participants identical placebo, there would be still

be a difference in outcomes, because the two arms will never match

identically in terms of sensitivity to the pretreatments, tumor burden,

immune status, etc. .. unless n = 10,000?)

So it's the comparisons of Net Outcome of such a trials which is the basis

for evidence (confidence), particularly if you can prove a survival

advantage for one group or another.

(Note: Survival is an unambiguous measure of clinical benefit, because it

accounts for all factors and effects, measured or not, short and long-term.

Since this study endpoint is not practical for indolent lymphomas,

investigators must use progression free survival (PFS) as the surrogate.

However, since vaccine is relatively low risk, PFS is a strong measure of

benefit I feel for vaccine studies.)

Do individual outcomes ever rise to the level of evidence? Unfortunately,

no.

.... Not even for advanced cancers with a highly predictable clinical course,

because spontaneous reversals can happen even if rarely is such instances,

and the reasons for the improvements can be unrelated to the credited

intervention in any single case.

For indolent lymphomas the need for a large control group in clinical

testing is much greater than for say pancreatic cancers, because spontaneous

remissions and long remissions to standard therapies occur commonly in FL.

It's a disease characterized by waxing and waning, by varying responses and

durations of response to the same treatments.

So size and study methods matter in the quest of evidence. Individual

results are interesting, provide reason to be cautiously hopeful, but not

evidence. I truly wish it was easier.

~ Karl