Hypothesis testing, part one

[Guest guest]

Hi Anu, Nrip, and friends,Thanks for giving the riddle a crack and setting up the discussions. Indeed, as Nip you explained it so well, the probability is 0.33:), This riddle came to mind thinking of hypothesis tests because this riddle relates to being skeptic in obvious answers and calls for re-examining where someone can go wrong in reasoning. These two principles (remaining skeptic and understanding errors) remain the cornerstone of all of science and indeed, play a very important role in hypothesis testing. Let's see if we can build our  discussion around this perspective.

In any clinical research, the first principle is that of skepticism and equipoise. Clinical equipoise, which is the most important aspect of conducting a clinical trial and thus helps in selecting the best control conditions, relates to the uncertainty and disagreement about which treatment (the experimental or the placebo) is better.  Skepticism in the context of a clinical trial relates to the position that outcome wise, an experimental treatment (or the novel treatment or novel drug) is no different from the placebo (or the usual treatment). This position is expressed by the null hypothesis which states that the effect is zero, or null, or that there is no difference in effect between the novel intervention and the alternative condition which can be a placebo or the standard treatment. The point here is this. The purpose of a clinical trial is to examine if a new intervention is superior to usual treatment or a placebo on measurable outcomes, but from the perspective of the trialist, one must start with the position that the new intervention is as effective as a placebo or the usual treatment (ie of no effect). More importantly, that position should be maintained till the completion of the trial. Only if the results of the study yield data that show that under the condition of the null, does the position gets changed. Therefore, clinical research is really about testing the hypothesis. However, Jerzy Nyman and Egan Pearson, who framed the theory of hypothesis testing also thought that one could not consider a null hypothesis unless one also conceived at least one plausible alternative hypothesis.

Now consider the second principle of clinical research, that a trial is about errors (!). Given our null hypothesis and data generation through a careful process of conducting the study, there are four positions here:

-----------------------------|------------------------------------------|-----------------------------------|-----------------------------| Null Hypothesis  True           | Null Hypothesis False | Study Conditions....| ------------------------------------------|----------------------------------|

Reject Null ------------| False Positive (Type I Error)  |  True Positive study    |-----------------------------|------------------------------------------|----------------------------------|Fail to reject null --- |  True Negative study ----------|  False negative (Type II error) |   

------------------------------|----------------------------------------|--------------------------------------------| As you can see in the above table, there are four positions really:1. The results of the study reject null whereas the null hypothesis is true (the study falsely finds a difference). This error is known as alpha error or Type I error

2. The results of the study fails to reject null whereas the null hypothesis is actually not true (the study fails to find a difference where there is a real difference). This error is known as beta error (or false negative error or Type II error).

Otherwise a study is either positive or negative. A positive study is one where the findings reject the null and the negative study is one where the results of the study fails to reject the null hypothesis (ie no effect, or the effects of the treatment and the control are same). Accordingly, power of a study is its probability of turning out to be truly positive and hence power = (1 - beta error rate).

When we are talk about hypothesis testing, alpha and beta errors, we talk in terms of probabilities. We talk in terms of conducting hundreds of studies and what proportion of these studies are open to errors. This also happens to the be point where we take into account alpha error rates, beta error rates and estimated effect size differences and their variability to select a specific sample size to detect the meaningful differences (we had discussions on this topic earlier, see the archives). Again, the emphasis is on a " population " of similar studies being conducted and what proportion of them are likely to commit the alpha and the beta errors.

As it turns out, according to convention, the alpha error rate is set at 5% (the probability of alpha error is 0.05) and the probability of beta error is set at either 0.20 or 0.10 (indicating that if these studies were to be conducted hundreds of times over, in about 10 out of 100 studies, one would have failed to find a difference where differences truly existed) The more conservative alpha error rates on the other hand indicate that if these studies were to be conducted over and over again, in about 5 out of 100 studies one would falsely find a difference where none existed.

The point to note here is that, this is not the same thing as p-values that get reported in the studies. P-values like alpha errors in hypothesis testing also take into account null hypotheses. However, in the context of clinical trials, the p-values indicate the probability that the observed findings are consistent with the conditions of null hypothesis. The lower the probability, the further away the statistic from the null condition, the less likely that the conditions of the null hypothesis are supported by the findings of the study. We shall visit the issues around p-values, and their relationship to hypothesis testing tomorrow.

Endpiece:Here's my question for the day. Djulbegovic (2009) and his colleagues recently analyzed published trials (RCTs) and found that (and I quote), " ... that we can discover no more than 25% to 50% of successful treatments when they are tested in RCTs. The analysis also indicates that this discovery rate is optimal in helping to preserve the clinical trial system; a high discovery rate (eg, a 90% to 100% probability of success)

is neither feasible nor desirable since under these circumstances [conditions of clinical equipoise. - Arin ], neither the patient nor the researcher has an interest in randomization " . Ben names this phenomenon as the " principle or law of clinical discovery " . Is there a paradox about discovery of effective treatments? What do you think?

Below is the reference for you. 1. Djulbegovic B. The paradox of equipoise: the principle that drives and limits therapeutic discoveries in clinical research. Cancer control : journal of the Moffitt Cancer Center. 2009;16(4):342-7. Available at: http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=2782889 & tool=pmcentrez & rendertype=abstract.

Enjoy,/Arin

On Sun, Jul 18, 2010 at 10:49 AM, Nrip Nihalani <n.nihalani@...> wrote:

 

Dear Arin, Thanks for the riddle..., Santa seems to be wrong, as usual That should be 0.33 ...Since the sentence is " atleast one of them is a boy " , this gives us 3 possible cases 3, {Boy, Girl} , {Girl, Boy}, {Boy, Boy} ... And considering the condition that the other has to be a Boy, only 1 of these cases will satisfy that...

Warm RegardsNrip--------------------------------------------------------------------------------------Nrip NihalaniCEO, Plus91 Technologies Pvt. Ltd.--------------------------------------------------------------------------------------

Phone:            +91 20 30496190E-mail:            n.nihalani@...Skype:            nrip.nihalaniTwitter:            http://www.twitter.com/nrip

Web:               http://www.plus91.inPersonal Web: http://nihalani.info--------------------------------------------------------------------------------------

What matters is not the idea a man holds,but the depth at which he holds it.--------------------------------------------------------------------------------------

On Sat, Jul 17, 2010 at 12:50 PM, Arin Basu <arin.basu@...> wrote:

 

Hi Vijay and friends,Thank you for the invitation to moderate hypothesis testing. I look forward to the discussions.My name is Arin Basu, I am an Otolaryngologist and an Epidemiologist by training. I teach Health Sciences and Epidemiology at the University of Canterbury in New Zealand (University home page http://www.hsci.canterbury.ac.nz/people/basu.shtml).

In this discussion, I am going to introduce hypothesis testing and I am planning to talk about p-values and confidence intervals. I hope that we shall round up the discussions with some limitations of hypothesis testing and p-value approach. If any of you'd like to see any other topic covered here, feel free to write.

Let's start the discussions in a light mood with a riddle/joke on probability as we shall be visiting this topic in course of our discussions. The riddle/joke goes like this:Sardar Santa Singh met Sardar Banta Singh after a long time. As they were chatting, Santa asked Banta,

" So, how many children do you have Banta? " Banta replied (he just took a course in math/stats), " Tell you what, I have two children and at least one of them is a boy. Now can you tell me what is the probability that the other is a boy too? "

Santa thought for a while and answered, " Why it's easy yaar, the probability is half (0.5) " Question: Do you think Santa is correct? Why or why not? Please post your response to the group.

, Arin

On Sat, Jul 17, 2010 at 2:37 PM, Vijay <drvijaythawani@...> wrote:

 

Hi,

We will be starting the second second segment of the workshop on research methodology tomorrow. It specifically concentrates on methodological issues. The opening topic is " Testing hypothesis " which will be moderated by Dr Arin Basu from 18-21 Jul 2002.

Arin, welcome to NetRUM as moderator. Kindly send your picture scan to me (on vijaythawani@...) for uplaoding in moderators' album of NetRUM. For those who would like to view our moderators, please be guided to visit photos menu in the left margin.

Arin, kindly take over NetRUM WEF today's eve and do introduce your self for all of us.

Vijay

Groupie