Teaching Clinical Research Methodology by Example

Introduction

I began medical school in Melbourne, Australia in 1953, 8 years after the end of the Second World War. At the time, effective therapies to treat diseases were few: sulfonamides, blood transfusions, and several surgical interventions. Penicillin and streptomycin had just become available. Peptic ulcers were common and caused chronic pain, and sometimes death from hemorrhage or perforation. We had no effective treatment for acute or chronic leukemia, which was invariably a death sentence, and operations for other cancers was limited to disfiguring and debilitating surgery and cobalt therapy, which was just starting to be used. Polio was a dreadful scourge; rubella in pregnant women caused blindness and malformations in their babies; oxygen treatment caused crippling lung disease and blindness in newborn infants; the number of soldiers dying of lung cancer after the World War II exceeded battle deaths during the war; young people with rheumatic heart disease were struck down by stroke and heart failure; high blood pressure was untreatable; heart attacks were on the rise, and we had no idea how to prevent or treat them. Conservative dentistry was nonexistent; our water and toothpaste were not fluorinated, and young people were offered sets of false teeth to replace mouthfuls of decayed teeth. The mentally ill were still being treated by having their frontal lobes skewered with ice picks and often ended up in a permanent vegetative state. Meningitis, diphtheria, whooping cough, scarlet fever, measles and other infectious diseases ravaged children, and patients with tuberculosis were isolated in sanatoria and had their lungs collapsed.

Organ transplantation, renal dialysis, heart surgery, pacemakers, deep brain stimulation surgery for Parkinson’s disease and many other life-transforming innovations were the stuff of science fiction. All of these changes have happened during my professional career, a span of 60 years, a blink of an eye! The instrument of these changes was the melding of basic, clinical and epidemiological research. Statisticians became essential members of clinical research teams and the discipline of clinical epidemiology was born. The term evidence-based medicine was introduced to describe the methodology of clinical research and the interpretation of its results by researchers and clinicians.

The journey of medical progress was not without its setbacks. Innocent children and adults were harmed, often by well-intentioned researchers whose work ultimately helped humankind; examples appear in the chapter on vaccines. Sometimes the setbacks were caused by well-intentioned doctors whose faulty research results were embraced by the public and media; a good example is found in the chapter on multiple sclerosis. Occasionally, the researchers were rogues motivated by greed, who foisted unproven and harmful treatments on innocent patients; one egregious example is ice-pick lobotomy. Unfortunately, with the profusion of unfiltered information transfer through social-media sites, coupled with the lack of oversight by our elected lawmakers, this form of quackery continues and even flourishes today.

Transformative clinical research requires creativity as well as an understanding of methodology. The latter is easy to teach, but creativity is not and requires an intangible quality, including what Louis Pasteur called a prepared mind. Major medical advances are usually made by well-trained researchers, but as highlighted in this book, groundbreaking discoveries have also been made by inquisitive, creative people untrained in research who had a prepared mind. These people include Edward Jenner (smallpox vaccine), Barry Marshall and Robin Warren (peptic ulcer), Sidney Farber (acute leukemia), and Norman Gregg (German measles in pregnant women).

Several excellent books have been written on the principles of clinical research methodology. This book is a departure from the standard texts because it teaches these principles through engaging stories that showcase some of the personalities behind the research. The book is written as a primer for health professionals, medical students, and graduate and undergraduate students in health sciences, but many of the chapters should be of interest to laypeople. Since much is learned by analyzing our mistakes, we include some monumental failures and their tragic consequences.

The book has two objectives: The first is to tell the story of the research process in action and to provide the reader with a glimpse into the minds of the researchers responsible for some of the major advances (and setbacks) in modern medicine—this is the entertaining part of the book. The second, and the main educational objective, is to explain the principles of evidence-based medicine by reviewing the research methods (the methodology) required to prove or disprove a theory. For our two sections, we have chosen two of the main reasons for performing clinical research: (1) to prove or disprove whether an association between a potential cause (e.g., human papillomavirus) and an outcome (e.g., cancer of the cervix) is causal or coincidental; and (2) to prove whether an intervention works, and if it does, whether its benefit outweighs its harm.

Part 1

CAUSATION

CHAPTER 1

Associations: How to Decide Whether They Are Causal or Coincidental

ARE ASSOCIATIONS CAUSAL OR COINCIDENTAL?

Associations between a possible cause and an outcome are often reported in the literature and taken up by the press and other public media. If the association is coincidental, it is of no real importance; but if it is causal, it can lead to the prevention and cure of important diseases. Associations are much more likely to be coincidental if the event is common in the population of interest. For example, it would be absurd to declare that being religious is a cause of criminal behavior if 100% of the population of interest is religious. Nevertheless, as described in chapter 6, a prestigious medical journal accepted a paper reporting that 8 out of 12 children had developed autism after being vaccinated. The paper was accepted despite the knowledge that 90% of all children in the geographic area had been immunized with the implicated vaccine. In contrast, it was only after decades of careful and painstaking research that the association of human papillomavirus (HPV) in the genital tracts of women with cancer of the cervix was declared to be causal, and eventually led to the development of an effective vaccine.

The most certain way to prove causality is to demonstrate in a randomized, controlled trial, that removing or neutralizing the cause also prevents or cures the disease. This was done to prove that peptic ulcers were caused by an unusual bacteria and to prove that the viral infection responsible for the precancerous lesion that precedes cancer of the cervix is prevented by the HPV vaccine. Unfortunately, many potential causes of disease cannot be evaluated in randomized, controlled trials, so other methods are necessary to demonstrate that their association with a disease is causal. For example, British researchers who identified cigarette smoking as a cause of lung cancer did not perform a randomized trial because they considered it unethical to ask nonsmokers to volunteer to start smoking.

Two giants in medical research have proposed rules for causation. The first was Robert Koch, the discoverer of the cause of tuberculosis and the second was Austin Bradford Hill, the man who performed and published the first two randomized clinical trials and who proposed criteria for establishing a causal relationship. We will not review the criteria here, but the following is the list of types of nonrandomized studies that are used to try to prove that an association is causal. They are listed from the least methodologically rigorous to the most rigorous:

1.Casual clinical observations

2.Case reports and case series

3.Case control studies

4.Retrospective cohort studies

5.Prospective cohort studies

CHAPTER 2

Smoking and Lung Cancer

This is the story of the research that demonstrated that smoking causes lung cancer and the two extraordinary researchers whose methods, creativity, and tenacity produced the evidence that saved millions of lives from lung cancer caused by smoking. Much of this information was taken from Conrad Keating’s informative biography of Sir Richard Doll.¹

Lung cancer was uncommon in the early 1900s, but after the First World War the rates of lung cancer began to increase in Western countries. By the 1940s, reports were published hinting at an association between smoking and lung cancer. But these case reports didn’t draw a lot of attention. People were enamored with smoking cigarettes, and it was becoming part of our culture. It was cool. Everyone smoked; film actors, sportsmen, politicians, and physicians. Tobacco companies were thriving, and billboards abounded with images of glamorous smokers.

In the 1950s, most medical students smoked, as did our tutors. In 1956 the report from England came out that claimed lung cancer was caused by smoking. I remember eight of us were crowded around a pathologic specimen of human lung cancer. Our tutor, a heavy smoker, expressed doubt about the publication because of its design. He said: Look boys, if you went outside, and it was raining and people were walking around with their umbrellas open, would you conclude that the open umbrellas brought on the rain? Just because an association is shown between A and B (smoking and cancer) doesn’t mean that A caused B. We laughed, thought that he was pretty smart, and were relieved that we could continue to smoke without doing ourselves any harm. After all, our mentor continued to enjoy his cigarettes!

Comment

Our tutor was pointing out to us that an association, even a strong one, is not necessarily causal. He was implying that it was coincidental.

As medical students, we had not read the seminal article by Richard Doll and Austin Bradford Hill and had not heard of its authors. They had convincingly shown that there is a strong and statistically significant association between smoking and lung cancer, but their conclusion was met with lots of resistance from experts who had financial and intellectual conflicts of interest. Some of the vocal dissenters were advocates for tobacco companies and many were hired guns.

Austin Bradford Hill was 15 years older than Richard Doll, and they had a mentor–student relationship. Bradford Hill was a distinguished statistician, and Doll was a young physician who had just returned from France where he had served in the army from 1939 to 1945. Bradford Hill was an economist by training, but his first love was medicine. In 1933 he was appointed to the staff of the London School of Hygiene and Tropical medicine. He wrote a book called Principles of Medical Statistics, which became a classic. In 1946, he persuaded the British Medical Research Council (MRC) to introduce randomization into its clinical trials, and he designed two of the first randomized controlled clinical trials (RCTs) ever published. The first was to evaluate the whooping cough vaccine and the second to evaluate the antibiotic streptomycin in patients with tuberculosis. The tuberculosis trial, which was published in 1948, was historic because Bradford Hill introduced double blinding. Both trials were positive. Since these pioneer studies were published about 70 years ago, randomized controlled trials have (appropriately) become the gold standard for evaluating treatments.

Death rates from lung cancer had risen to epidemic proportions, and by 1950 they had surpassed death rates from tuberculosis. The matter was of great concern to the Minister of Health, who in 1947 wrote to the MRC and asked them to look into the cause of the epidemic of lung cancer. Bradford Hill was the Director of the MRC Statistical Research Unit and was asked to lead the search. It was at this point that Richard Doll entered the scene. Doll had become acquainted with Bradford Hill while attending a course on statistics delivered by his future mentor. Doll had completed a successful clinical project on the causes of stomach ulcers at the Central Middlesex Hospital and had shown a passion for research and an aptitude for statistics. He also proved that he was tenacious, having achieved an impressive 98% follow-up of the patients included in the study. So when Hill started to form his team, he thought of Doll and put him in charge of the day-to-day operation of the research program. It was 1948 and Doll was 37 years old. The study was designed by Bradford Hill, using an ingenious new approach.

Comment

The importance of achieving high follow-up rates in longitudinal studies (randomized trials and cohort studies) is discussed later in the chapter.

Bradford Hill developed a questionnaire with a list of 50 questions. Doll, with the help of social workers (almoners), then administered the questionnaire to patients with lung cancer who had been admitted to one of 20 London hospitals. The questionnaire consisted of a variety of questions about lifestyle and environmental factors including several on smoking habits. Since Hill was investigating associations between environmental factors and lung cancer, the questionnaire was also administered to patients without lung cancer (controls), of the same sex and age who had been admitted to the same ward. The investigators had designed a case-control study.

Comment

Although controls had been used for years in nonmedical research, their use in medical research was uncommon in the 1950s. It was the norm for physicians to publish case reports and case series, and it took an outstanding biomedical statistician to introduce scientific rigor into medical research. The case-control study examining the association between smoking and lung cancer was among the first published in the medical literature.

When he developed the questionnaire, Hill had no preconceptions, and Doll thought that the likeliest cause of lung cancer was tar used to pave London roads. They were both smokers and included smoking in the questionnaire only because others thought that smoking might be a risk factor for lung cancer.

Comment

Hill and Doll had no preconceptions (they were both smokers) when they designed their study. In contrast, Ancel Keys (chapter 3) used his preconceptions to design the ecological study, which was pivotal to the dietary lipids–serum cholesterol–coronary heart disease hypothesis.

After more than a year of study, Doll wrote an interim report in which he stated that of all the candidate risk factors in the questionnaire, smoking emerged as having by far the strongest association with lung cancer. By the time the study was completed, Doll and his coworkers had enrolled 709 subjects with lung cancer and the same number of controls who did not have lung cancer. The rate of smoking was high in both groups, reflecting the popularity of smoking during these times, but the difference between the two groups was a highly statistically significant finding. The odds ratio (OR) was 16, indicating a very strong association between lung cancer and smoking. In 1950, when this study was published, it was the most rigorous case-control study ever performed. Unknown to Hill and Doll at the time, a smaller, less rigorously designed epidemiological study had been performed in Germany in 1939, and a second case-control study of similar design had been performed in the United States. All three studies produced similar results. The findings were not only statistically significant, they were consistent.

Comment

These two terms, statistically significant and consistent, are important in the field of biostatistics. If the results of one study are statistically significant, it implies that the findings are unlikely to occur by chance, and if the study is free from bias, it is likely to be correct. The likelihood that the finding is real (and not caused by the play of chance) is increased if it is confirmed by other researchers.

The results of Doll’s study were met with hostility by the medical profession, by the post-War government that was desperate for tax revenues from tobacco sales, and by indifference from the public. Many physicians did not believe the results; some wanted more solid evidence than could be provided by a retrospective study.

Doll and Hill were aware that their case-control study had limitations and knew that they needed more convincing evidence to prove that smoking caused lung cancer, but thought that a randomized trial would be unethical.

Comment

Doll and Hill had sufficient conviction that the relationship between cigarette smoking was causal to make it unethical for them to embark on a randomized trial. On the other hand, Ronald Fisher, a prominent figure in biostatistics, argued for a randomized trial, because he was not convinced that the association found by Doll and Hill was causal.

A case-control study is retrospective and subject to hidden biases. Although its results strongly suggest a true association, they don’t provide convincing evidence of a causal association. Bradford Hill and Doll recognized this weakness and decided to follow their retrospective or backward-looking case-control study by undertaking a prospective or forward-looking study to see whether smoking predicted lung cancer. Once again, the design of the study was Bradford Hill’s idea. Bradford Hill recognized that a prospective study would be more challenging because it required meticulous follow-up over several years. To address this issue of follow-up, he proposed selecting physicians as his subjects for the trial. He reasoned that physicians could be easily traced from the registry of physicians and thought that they were more likely to be compliant in filling out the questionnaire on their smoking habits than nonphysicians. Doll was inspired by the idea, and, in 1951, he wrote to British physicians to ask them to fill out a simple questionnaire, which included their smoking habits. Over 40,000 letters were sent out, and there were over 40,000 replies. The letters were sorted alphabetically, and the answers to the questions recorded. After a year of preparation, he selected about 20,000 male doctors over the age of 35 to follow, some of whom were smokers and some of whom were not. The physicians did not have lung cancer at the time of enrollment and were followed long term to compare the number in each group who developed cancer.

Comment.

Detailed descriptions of case-control and cohort studies are provided in the last chapter of the book. For now, it is sufficient to understand that a case-control study is retrospective and a cohort study is prospective. In the case-control study, participants with lung cancer and matched controls were identified and were asked to fill out a questionnaire, which included their exposure to environmental factors. Smoking was identified as a possible risk factor for lung cancer. In the prospective cohort study, Doll recruited physicians without known lung cancer, divided into them two groups, smokers and nonsmokers, and followed them up for over 20 years to determine the rates of lung cancer in each group.

By 1954, Doll’s prospective cohort study had confirmed the results of the case-control studies, and Doll also noted that smoking increased the risk of coronary thrombosis. Doll achieved a remarkable 99.6% follow-up in the prospective cohort study of British doctors. He knew that it was critically important to capture all deaths on follow-up, because the outcome of lung cancer is usually fatal, and missing deaths due to lung cancer could affect the study results. The prospective cohort study showed that smoking has a huge impact on the risk of developing lung cancer. The relative risk (RR) was over 20 times higher in smokers than nonsmokers, and increased in proportion to the number of years that a person had smoked. Furthermore, in people who smoked more than 25 cigarettes a day, there was a 50-fold increase in the risk of developing lung cancer.

Comment

The RR is the ratio of the probability of an event (lung cancer) occurring in an exposed group (smokers) to the probability of the event occurring in a comparison or control (nonsmokers) group. The increase in RR in heavy smokers is called a dose effect or a biological gradient, a criterion that increases the likelihood that an association is causal. The integrity of prospective studies, including cohort studies and randomized trials, is critically dependent on obtaining a very high level of follow-up. Because participants who die can’t attend follow-up visits, they could easily be excluded from the final analysis unless special efforts were made to identify them. If participants who died were excluded, the results might