Acta Psychiatr Scand 2009: 119: 443450 All rights reserved DOI: 10.1111/j.1600-0447.2009.01388.

2009 John Wiley & Sons A/S

ACTA PSYCHIATRICA SCANDINAVICA

Clinical overview

How to read and understand and use systematic reviews and meta-analyses
Leucht S, Kissling W, Davis JM. How to read and understand and use systematic reviews and meta-analyses. Objective: Systematic reviews and meta-analyses are increasingly frequently used in the evaluation of medical treatments. This review explains the principles of the methodology, signicance and limitations of systematic reviews. Method: Short review article. Results: In contrast to conventional reviews, systematic reviews use a structured approach in retrieving, analyzing and interpreting the evidence. A comprehensive search strategy is applied to identify all relevant trials. Study selection and data extraction is performed independently by at least two reviewers. The data are usually synthesized in a meta-analysis applying statistical methods to examine homogeneity. Funnel plots and other statistical methods are applied to detect publication bias. Conclusion: Due to the enormous amount of scientic information published every year, systematic reviews and meta-analyses have become indispensable methods for the evaluation of medical treatments.
Clinical recommendations

S. Leucht1, W. Kissling1, J. M. Davis2

1 Klinik and Poliklinik fr Psychiatrie and Psychotherapie, Technische Universitt Mnchen, Mnchen, Germany and 2Gilman Professor of Psychiatry, Department of Psychiatry, University of Illinois at Chicago, Chicago, IL, USA

Key words: systematic review; meta-analysis; psychopharmacology; evaluation Stefan Leucht, Klinik and Poliklinik fr Psychiatrie and Psychotherapie, Technische Universitt Mnchen, Klinikum rechts der Isar, Ismaningerstr. 22, 81675 Mnchen, Germany. E-mail: Stefan.Leucht@lrz.tum.de Accepted for publication March 5, 2009

The incredible ood of studies in the eld of medicine makes systematic reviews and meta-analyses important instruments for the evaluation of medical therapies, and an important source of information for clinicians and researchers. Clinicians should therefore know the principles of the methodology. Although they are based on the same numbers, relative and absolute risk reductions can lead to different interpretation. Therefore, clinicians need to consider in which form the data are presented and what the underlying percentages in each group are.

Additional comments

The term systematic reviews describes the systematic approach in retrieving, analyzing and interpreting the evidence, while meta-analysis refers to the statistical method of combining different studies on the same question. The main problem of meta-analysis is the apples and oranges question: how similar must the studies be that a meta-analytic combination is meaningful. Publication bias (the fact that studies with negative results are published less frequently than positive studies) is a major threat to evidence-based medicine. There are meta-analytic methods that can detect a possible publication bias.

Introduction

Systematic reviews and meta-analyses are considered important instruments for the evaluation of

medical therapies. Davis was the rst to use the method in psychiatry. In a systematic review and meta-analysis which is still being frequently quoted, he summarized all randomized double-blind studies 443

Leucht et al. on relapse prevention with antipsychotics and was able to show impressively that such a therapy reduced the relapse rates of schizophrenic patients (1). A systematic review and meta-analysis by the psychologist Glass (2) on the ecacy of psychotherapy published a year later introduced the method to the wider psychological psychotherapy audience but quickly came under re, being criticized to the eect that Glass was comparing apples and oranges. This heterogeneity problem is still being discussed today (see below). As there are thousands of clinical trials too many for an individual clinician to read the British physician and epidemiologist Archie Cochrane wrote at about the same time: It is surely a great criticism of our (medical) profession that we have not organized a critical summary, by specialty of subspecialty, adapted periodically, of all relevant randomized trials (3). Stimulated by Archie Cochrane, the Oxford epidemiologist Iain Chalmers founded the Cochrane Collaboration in 1993, an international non-prot organization, the goal of which is to produce, disseminate and maintain systematic reviews of medical treatments. These reviews are published quarterly in electronic form in the Cochrane Database of Systematic Reviews, assessable via the Internet (http://www.cochrane.org). In the eld of psychiatry ve review groups Cochrane Dementia and Cognitive Impairment Group, Cochrane Depression, Anxiety and Neurosis Group, Cochrane Developmental and Learning Disabilities Group, Cochrane Drugs and Alcohol Group and the Cochrane Schizophrenia Group cover the most important psychiatric illnesses.
Aims of the study Results Definition of terms: systematic review and meta-analysis

The terms systematic review and meta-analysis are often used synonymously, although they do not mean exactly the same thing. Systematic review describes the use of a structured procedure (such as a systematic literature search) in contrast to that of conventional reviews. Meta-analysis is a statistical method which combines the results of individual studies. Systematic reviews usually, but not necessarily, apply meta-analysis to examine a research question. Therefore, in the current article, we used the term systematic review when we described this broader concept, but we used the term meta-analysis when we explained the statistical method.
Methodology used in systematic reviews and meta-analyses

Despite their increasingly frequent use, systematic reviews and meta-analyses are still considered skeptically by many clinicians and researchers. The general fear is that these are mysterious, statistical methods of questionable use for clinical practice. Furthermore, without background knowledge the results are dicult to interpret. The present contribution therefore intends to explain the most important steps in producing a systematic review and discuss the signicance and limitations of the method. At the end, the reader should be able to read and interpret relevant publications.
Material and methods

Creating a protocol. Good systematic reviews rst dene in a protocol how to proceed in producing the reviews. This protocol establishes how the literature search is to be carried out, what study quality is to be considered (e.g. only randomized studies), which patients are to be included (e.g. schizophrenia according to DSM-IV), which outcome parameters are to be addressed, which outcome is the primary one and what statistical methods are to be used in the meta-analysis. In Cochrane reviews a protocol is always rst drawn up, which is reviewed by two editors of a review group and then published in the Cochrane Library. One of the editors assumes more of a consultant functions, the other one acts anonymously and takes a more critical stance. The protocol is then published in the Cochrane Library, which provides transparence, facilitating that the initially dened procedure should no longer be deviated from in order to avoid distortions. Search strategy. Good systematic reviews systematically search in a large number of sources. For example, Adams et al. (4) showed that a considerable number of randomized schizophrenia studies are not covered by MEDLINE. The Cochrane Schizophrenia Group regularly searches numerous electronic data bases (including Chinese), book articles, hand searches the abstract books of important conferences and searches the internet such as the web site of the Food and Drug Administration (FDA) in order to nd all relevant studies. In addition, Cochrane reviewers search the reference lists of already identied articles, and write to authors and manufacturers to provide additional articles or data. It has been shown that

We present a narrative overview on important denitions and methods that are applied in systematic reviews and meta-analyses. These are illustrated by examples from the literature. 444

Systematic reviews and meta-analyses negative studies are published less often (remain in the le drawer) than positive studies. This is termed publication bias or the le drawer eect (5). Some negative studies are not published as they contradict the ecacy of a commercial drug or an academics position. This can result in an overly positive view, sometimes called evidence-biased medicine (6). Turner et al. (7) recently showed that of the studies on antidepressants submitted to the FDA, companies preferably published those with positive results. Some negative studies are only presented at conferences (so-called gray literature, only a portion of the studies presented at conferences are actually published later). Finally, good systematic reviews include studies without language restrictions. For example, Egger et al. (8) showed that German authors preferably publish studies with signicant results in renowned English-language journals, not in German-language journals. Using just English-language articles would lead to a too positive result [so-called language bias, see also Ref. (9)].
Selection of the studies and data extraction. For quality assurance it is important that the selection of the studies and the subsequent data extraction is double checked. At least two reviewers should perform these steps independently of one another and then compare their results. Sometimes the study authors have to be contacted for clarication. It is important to document the search and the reasons for study exclusion, including a QUORUM diagram (10) (see Fig. 1). Sales such as the JADAD scale (11) can be used to quantify study quality. The JADAD scale rates the randomization and blinding procedures, as well as the reporting on drop-outs. Statistical methods used in meta-analysis. There exist a large number of statistical methods for various study types and questions. In the following, the principle for group comparisons in randomized studies, such as those customary in psychopharmacology, is illustrated [for more details, see textbooks, e.g. Refs (12, 13)]. Eect size measures. First of all, an eect size is calculated for each study and for each outcome parameter. An eect size is a statistical measure for the magnitude of the dierence between two interventions adjusted by some estimate of variability, whereas the P-value designates only the probability of whether a result has come about randomly. There are eect sizes for dichotomous (binary) and continuous data.
Potentially relevant RCTs identified and screened for retrieval (N =) RCTs excluded, with reasons (N = ...) RCTs retrieved for more detailed evaluation (N =) RCTs excluded, with reasons (N =) Potentiallly appropriate RCTs to be included in the meta-analysis (N =) RCTs excluded from meta-analysis, with reasons (N = ...) RCTs included in meta-analysis (N =)

RCTs withdrawn, by outcome, with reasons (N = ...) RCTs with usable information, by outcome (N =)

Fig. 1. The Quality of Reports of Meta-analysis (QUOROM) ow-diagram (10).

Eect size measures for dichotomous variables. Dichotomous (binary) outcomes are, for example, death yes no, relapse yes no or therapy response yes no, yielding such eect sizes as absolute risk difference, relative risk or odds ratio. Furthermore, there is the correlation coecient (R), which can be applied to both continuous and dichotomous variables. As this is frequently used in the psychological literature albeit less often in the medical literature, it will not be described here. The example in the following text uses the negative outcome relapse, but the same formula applies for positive outcomes such as response (see Table 1). Risk is the number of patients with an event (e.g. a relapse) divided by the total number of patients (e.g. three of 10 patients, thus 0.3 or 30%). Here risk is a generic term, and could also designate response (e.g. 30% of patients respond or recover). To obtain the absolute risk difference the risk in the one group is subtracted from the risk in other group (30% relapsed under drug, 70% relapsed under placebo, a risk dierence of 30%70% or 40%). To obtain the relative risk the risk in the one group is divided by the risk in other group (30% relapsed 70% relapsed (30 70 = 0.43 or 43%). To calculate the relative risk reduction the result is subtracted from 1 (10.43 = 0.57 or 57%). The odds ratio uses the same data as the absolute and relative risk dierence. In contrast to the risk, the odds ratio is, however, the number of patients with an event (e.g. relapse) divided by the number of patients without an event (and not by the total number of participants).

445

Leucht et al.
Risk measure Risk Relative risk Absolute difference Odds ratio Formula Explanation Table 1. Calculation of effect sizes for dichotomous outcomes

a (a + b) Number of patients with an event divided by the total number of patients [a (a + b)] [c (c + d)] Risk in the intervention group divided by the risk in the control group [a (a + b)] ) [c (c + d)] Risk in the intervention group minus risk in the control group (a b) (c d) Odds in the intervention group divided by the odds in the control group

a = participants with an event (e.g. relapse) in the intervention group; b = participants without an event (e.g. relapse) in the intervention group; c = participants with an event (e.g. relapse) in the control group; d = participants without an event (e.g. relapse) in the control group. The first three measurement figures can also be represented as percentages through multiplication by 100.

In meta-analyses, relative risk and odds ratio have the advantage in comparison with the absolute risk that they depend less on the baseline risk. The odds ratio has somewhat better mathematical properties than the relative risk. For rare events, it yields the same values as the relative risk. However, for frequent events (approximately over 20%), the odds ratio turns out signicantly greater than the relative risk, and falsely equating relative risk and odds ratio, then leads to an overestimation of the risk. For this reason the relative risk is generally used in Cochrane reviews. The advantage of the absolute risk dierence is that it is more intuitive in interpretation to the clinician. It is mathematically equivalent to the number needed to treat (NNT) being the number of patients who must be treated in order to avoid one negative outcome. NNT is simply calculated as 1 divided by the risk dierence (1 risk dierence), in the example above: 1 0.40, thus NNT = 2.5). For clinicians this measure is more intuitive. For a review on the concept of NNT see Citrome (14). Table 2 illustrates that absolute and relative risk reduction can lead to quite dierent interpretations depending on the baseline risk.
Eect sizes for continuous variables. Continuous variables are a continuum like blood pressure, a specic laboratory parameter or rating scales [e.g. Brief Psychiatric Rating Scale (BPRS) or Hamilton Depression Scale] with many points on a scale, which treated as continuous, although they are, strictly speaking, ordinal. There are basically two eect size measures: the difference of the mean values (DM) keeps the original

units and simply is the mean value of group X minus the mean value of group Y. For example, a BPRS Total Score Mean of 60 in group X minus a BPRS Total Score Mean of 50 in group Y, yielding a DM of 10. The DM is simple and grasped intuitively, but often dierent scales are used, for example either the Positive and Negative Syndrome Scale (PANSS) or the BPRS in schizophrenia. However, 10 point dierence on one scale does not mean the same thing as 10 point dierence on the other scale. For this reason the results of dierent scales must be standardized in order to be able to combine the dierent studies in a meta-analysis by a common unit; the standardized difference of mean values. This is carried out by dividing the DM by the pooled standard deviation of the two groups in the basic formula: SMD = (mean value of group X ) mean value of group Y) pooled standard deviation of the two groups. An SMD of 0.3 thus means a dierence of 0.3 standard deviations between two treatments. Various formulas for the calculation of SMDs exist such as, e.g. Cohens D or Hedgesg, the latter avoiding a bias in Cohens D, seen with small number of subjects. One disadvantage of the SMD is that it is more dicult to interpret than the simple dierence of the means. Cohen has given a widely publicized illustration of a simple, but coarse, rule of thumb according to which an SMD of 0.2 represents a small eect, 0.5 a medium one and 0.8 a large eect (15).
Calculation of a mean eect size. The eect sizes of the individual studies are combined to calculate a mean eect size using various formulae, weighting

Table 2. Examples how the absolute and relative risk difference can differ in various situations Rare side-effect (e.g. agranulocytosis) Frequencies (%) Absolute risk difference Relative risk Relative risk reduction Drug A: 3%, drug B: 7% 3%7% = |)4%| 3% 7% = 43% 100%43% = 57% Frequent side-effect (e.g. EPS) Drug A: 30%, drug B: 70% 30%70% = |)40%| 30% 70% = 43% 100%43% = 57% Percentage responders to treatment Drug A: 70%, drug B: 30% 70%30% = 40% 70% 30% = 2.3 (times more responders) Does not apply

The example illustrates how the absolute risk difference and the relative risk (reduction) differ largely depending on the frequency of the outcome.

446

Systematic reviews and meta-analyses according to variability of the results and sample size and occasionally other parameters, such as the study quality. The weight of study quality or other parameters runs the risk of being subjective, especially when only high-quality double-blind studies are included. Two general statistical models are used: xed effects models and random effects models in metaanalytic calculations. Fixed eects models assume that there is a constant true mean eect size around which the eect sizes of the individual studies are randomly distributed, with the variability of the individual eect sizes around the true value being a random sample error. This is tested by a homogeneity statistics. This statistics also helps in nding whether there are studies with deviating results. If yes, methodological or other dierences in such studies should be explored. Random eects models assume on the contrary that each study can have a somewhat dierent true mean, presumable caused by real dierences among studies (e.g. somewhat dierent inclusion criteria or dosages). Random-eects models incorporate this variability, which often leads to larger condence intervals and less statistically signicant results. As a limitation, random models give more weight to small studies. However, random eects models lead to results dierent from those of xed eects models only when the individual studies exhibit heterogeneous results.
Methods for examining a possible publication bias. As described above, publication bias represents a serious problem for evidence-based medicine. Meta-analyses nevertheless oer the chance to detect a publication bias. The funnel plot is a graphic method based on the assumption that

small negative studies are less likely to be published, in which the eect sizes of the individual studies are plotted against their number of participants or precision. If all studies on a given topic had been published, a symmetrical plot with the form of an inverted funnel must result (8). The large studies with high precision are namely located on top in the vicinity of the mean eect size. The smaller the studies, the more scatter they are around the mean value and lie randomly to the left or right. If the plot is not symmetric, this is an indication that some studies may not have been published. This is illustrated in Fig. 2. So-called fail-safe methods estimate how many unpublished studies with negative results must exist in order that a meta-analytical result be no longer signicant or the eect size take on a trivial value (16).
Presentation of the results. Good systematic reviews and meta-analyses attempt to be as transparent and thus reproducible presentation as possible. The search strategy, excluded and included studies are listed in detailed tables in Cochrane reviews. Particularly elegant and intuitively comprehensible is the presentation of the results in so-called Forest plots. Figure 3 is the forest plot of a systematic review which compared relapse rates after 1 year under treatment with second-generation and rstgeneration antipsychotic drugs (17). For each study, the eect size (here the absolute risk dierence was chosen) and the corresponding condence interval (mostly 95%) are presented. The number of relapses is also individually given for each study. The eect size at the bottom is the pooled eect size. The following rules make the funnel plots easy to interpret.
Hypothetical funnel-plot indicating potential publication bias
(n) 700 600 500 400 300 200 100

Hypothetical funnel-plot not indicating potential publication bias

(n) 700 600 500 400 300 200 100 0 0.3 0.2 0.1 0 0.1

0 0.3

0.2

0.1

0.1

Effect size
The effect Th ff sizes i of f the h single i l studies di were plotted l d on the h x-axis. the studies sample sizes on the y-axis. If all studies have been published a symmetrical figure resembling an inverted funnel results. Negative effect sizes reflect a superiority of the intervention group.

Effect size
Here the H h f funnel-plot l l is i asymmetrical. i l b because studies di i in the lower right hand corner are missing. This suggests a relevant publication bias in the sense that smaller studies with results in favour of the control group have not been published.

Fig. 2. Illustration of the funnel-plot method.

447

Leucht et al.
Secon d First-generation generation a nt ips y ch o t ic a n t i ps ych o ti c n/N % n /N % 2/33 10 % 6% 3 /30 6 5/ 1 88 35 % 1 2/10 9 1 1% 9 /31 0 / 14 15/48 4/ 14 2 /10 3 /14 29 % 0% 31 % 29 % 2 0% 21 %

Marder et al. 2003 (31)

Csernansky et al. 2002 (20) 41/177 23% Daniel et al. 1998 (21) Speller et al. 1997 (32) 2 /9 4 5/29 2% 17 % 4% 17% 29 % 22 % 13 % 13%

Effect size 95 confidence interval Here the confidence intervals do n o t o v er l a p w i t h t h e y - ax i s, t h u s t he r es u l t s a r e s t a t i s t i c a l l y si g n if i c an t

Tamminga et al. 1994 (18) 1/25 Essock et al. 1996 (33) 13/76

Rosenheck et al. 1999 (22) 10/35 Tran et al. 1998a (19) Tran et al. 1998b (19) Tran et al. 1998c (19) 10/45 6/48 71/53

T h e o n l y s tu d i e s w i t h a ( no ns i gn i f i c a n t ) t r e n d i n f a v o u r o f first-generation antipsychotic dr u g s

29/1 56 19%

Total

161/1096 15%

142/614 23%
0.5 0 0.5

P = 0.0001 in favour of second-generation antipsychotic drugs

N=number relapsed N=total number of participants in the group

Absolute Risk difference

In favour of secondgeneration drugs

In favour of first-generation drugs

Fig. 3. Example of a forest-plot meta-analysis of relapse rates of second-generation vs. rst-generation antipsychotic drugs (modied from Leucht et al. 2003 (17)).

i) All results on the left-hand side are in favor of the second-generation antipsychotic drugs, those on the right-hand side in favor of the rst-generation antipsychotic drugs: with the exception of the studies by Tamminga et al. (18) and Tran et al. (19), there is at least a trend in favor of the second-generation antipsychotic drugs. ii) If the 95% condence interval does not overlap the y-axis, the result is statistically signicant, as 95% of the results are expected to lie on one side. This is the case here only in the studies by Csernansky et al. (20), Daniel et al. (21) and the mean eect size. iii) Small studies or those with low precision have large condence intervals, e.g. (22); large studies have small condence intervals, e.g. (19). As a series of studies is taken together in the mean eect size, the case number is also large here and the condence interval small. In this sense, one possibility of meta-analyses is to increase the statistical power through the combination of several methodically good but small studies. Once these rules have been digested, forest plots considerably facilitate the interpretation of the 448

results. Whereas in narrative reviews, the results of the individual studies must be laboriously enumerated and at the end a more or less subjective conclusion results in the nal analysis, here a graphical impression of the statistical signicance, size of the eects and stray results are immediately obtained.
Methodological problems. It goes without saying that a systematic procedure in drawing up a review is preferable to an unstructured procedure as was previously customary. Therefore, there is little controversy about the concept of systematic review. In the statistical method of meta-analyses, however, as in all procedures, there are methodological diculties, several of the more important of which shall be discussed. The apples and oranges problem. The meta-analytical combination of individual studies has been criticized as comparing apples and oranges. A limited variability in the individual studies is permissible by all means, as in routine clinical practice there are patients with similar characteristics but individual dierences. This is presumably

Systematic reviews and meta-analyses what Gene Glass, one of the pioneers of the method, also meant with his quote that the apples and oranges combination is all right as long as one wants to make a statement about fruit (23). Clear inclusion criteria, the graphical presentation of the results in forest plots and heterogeneity statistics are an aid in keeping the apples separate from the oranges. Drawing the line, however, is admittedly often anything but trivial.
Diering quality of the individual studies. In metaanalyses, an additional weighting according to the study quality can be performed. Here, it should be remembered that one is usually already at a high level (randomized double-blind studies) and it is unclear how much weight is to be attributed to which quality parameters. There are, however, situations in which one should preferably rely, e.g. on two methodologically clearly higher level studies, than on, e.g. 10 poor ones (otherwise garbage in, garbage out). Meta-analyses are over-powered. There has been the criticism that through the combination of dierent studies, meta-analyses are statistically over-powered and thus overestimate the eects. Actually this is not the case, as in contrast to narrative reviewing the emphasis is placed on the eect size and not on the P-value. Meta-analytical averaging. It is claimed that metaanalyses form mean values and these mean values blur the results. Scientic results are, however, always mean values. For instance, a randomized study also shows nothing else but the mean value of the included patients. Thus if one cannot clearly show that certain studies are denitely better than others, there is no rational excuse for using only these studies. Missing data. If the necessary parameters are not given in a study, it cannot be included in the metaanalysis. Actually one would like to cover all relevant studies, but although the authors of the original studies are contacted for missing information, not all of them send this information unfortunately. Diering statistic models. There are debates as to which statistical model is most suitable in which situation, even though often no relevant dierences result from the choice in the case of robust data (a part from the dierent interpretation of absolute and relative risk reductions as explained above). Diering interpretation of the results. Although the method as such is objective, the results of

systematic reviews must be interpreted. Authors with dierent backgrounds and attitudes can arrive at dierent conclusions. Thus, for instance, the Cochrane reviews on amisulpride (24) and olanzapine arrive at dierent conclusions in spite of similar results (25). This is not a methodological problem in the strict sense, but there is an urgent need to work for a unication of the interpretation of systematic reviews and metaanalyses.
Discussion

As any method, systematic reviews have limitations. Nevertheless, they have become indispensable instruments for the evaluation of new treatments. At present, there is an incredible ood of information in the eld of medicine. It is estimated that approximately two million articles are published yearly in about 10 000 medical journals. Or that a general practitioner who wishes to read all contributions which are relevant to his area would have to read about 19 publications a day, 365 days a year [from Ref. (26)]. On the question of whether second-generation antipsychotic drugs are superior to rst-generation drugs, we recently found 38 double-blind comparisons with placebo (27), 150 double-blind comparisons with rst-generation antipsychotic drugs (28) and 78 blinded head-to-head comparisons of the second-generation antipsychotic drugs (29). The gures on studies dealing with the newer antidepressants are similarly vertiginous. In our opinion, it is impossible to objectively assess such enormous numbers of studies with conventional review methods. Objective evaluation is additionally hampered by studies sponsored by pharmaceutical often being presented in a distorted manner. Heres et al.(30) found that the abstracts of head-to-head comparisons of second-generation antipsychotic drugs turned out in 90% of the cases in favor of the medication of the sponsor. The advantages of the sponsors own substance are emphasized in the abstracts, and negative aspects are downplayed. A dierent approach would be to select specic studies, but this has its problems as well and opens the door to biased choices. One should remember that the studies selected for meta-analyses are usually all of high quality often it is an inclusion criterion that they are randomized and double blind. Making a further selection is often problematic. There is no doubt that systematic reviews and the Cochrane Collaboration will continue to play an important role in the evaluation of new treatments. 449

Leucht et al.
Declaration of interest
Stefan Leucht has received speaker and or consultancy honoraria from SanoAventis, BMS, EliLilly, Janssen Johnson and Johnson, Lundbeck and Pzer; and he has received funding for research projects from EliLilly and SanoAventis. Werner Kissling has received speaker and or advisory board consultancy honoraria from Janssen, Sano-Aventis, Johnson and Johnson, Pzer, Bayer, BMS, Astra Zeneca, Lundbeck, Novartis and EliLilly. John M Davis has none to declare. 18. exploratory meta-analysis of randomized, controlled trials. Am J Psychiatry 2003;160:12091222. Tamminga CA, Thaker GK, Moran M, Kakigi T, Gao X-M. Clozapine in tardive dyskinesia: observations from human and animal model studies. J Clin Psychiatry 1994;55(suppl. 9):102106. Tran PV, Dellva MA, Tollefson GD, Wentley AL, Beasley CM. Oral olanzapine versus oral haloperidol in the maintenance treatment of schizophrenia and related psychoses. Br J Psychiatry 1998;172:499505. Csernansky JG, Mahmoud R, Brenner R. A comparison of risperidone and haloperidol for the prevention of relapse in patients with schizophrenia. N Engl J Med 2002;346:1622. Daniel DG, Wozniak P, Mack RJ, McCarthy BG. Long-term ecacy and safety comparison of sertindole and haloperidol in the treatment of schizophrenia. Psychopharmacol Bull 1998;34:6169. Rosenheck R, Evans D, Herz L et al. How long to wait for a response to clozapine: a comparison of time course of response to clozapine and conventional antipsychotic medication in refractory schizophrenia. Schizophr Bull 1999;25:709719. Glass GH. In defense of generalization. Behav Brain Sci 1978;3:394395. Mota Neto JIS, Lima MS, Soares BGO. Amisulpride for schizophrenia. Cochrane Database Syst Rev 2002;2: CD001357. Duggan L, Fenton M, Rathbone J, Dardennes R, El-Dosoky A, Indran S. Olanzapine for schizophrenia. Cochrane Database Syst Rev 2005;2:CD001359. ther A, Stieglitz RD, Berger M. EvidenceBerner MM, Ru based medicine in psychiatry more rational psychiatry? Nervenarzt 2000;71:173180. Leucht S, Arbter D, Engel RR, Kissling W, Davis JM. How eective are second-generation antipsychotic drugs? A meta-analysis of placebo-controlled trials. Mol Psychiatry 2008; Jan 8. [Epub ahead of print]. Leucht S, Corves C, Arbter D, Engel RR, Li C, Davis JM. Second generation versus rst generation antipsychotics for schizophrenia: a meta-analysis. Lancet 2009;373:31 41. Leucht S, Komossa K, Rummel-Kluge C et al. A metaanalysis of head to head comparisons of second generation antipsychotics in the treatment of schizophrenia. Am J Psychiatry 2009;166:152163. Heres S, Davis J, Maino K, Jetzinger E, Kissling W, Leucht S. Why olanzapine beats risperidone, risperidone beats quetiapine, and quetiapine beats olanzapine: an exploratory analysis of head-to-head comparison studies of second-generation antipsychotics. Am J Psychiatry 2006; 163:185194. Marder SR, Glynn SM, Wirshing WC, et al. Maintenance treatment of schizophrenia with risperidone or haloperidol: two-year outcomes. Am J Psychiatry 2003;160:1405 12. Speller JC, Barnes TRE, Curson DA, Pantelis C, Alberts JL. One-year, low-dose neuroleptic study of in-patients with chronic schizophrenia characterised by persistent negative symptoms amisulpride v haloperidol. Br J Psychiatry 1997;171:564568. Essock SM, Hargreves WA, Covell NH, Goethe J. Clozapines eectiveness for patients in state hospitals: results from a randomized trial. Psychopharmacol Bull 1996;32: 683697.

19.

20.

References
1. Davis JM. Overview: maintenance therapy in psychiatry. I. Schizophrenia. Am J Psychiatry 1975;132:12371245. 2. Glass GV. Primary, secondary, and meta-analysis of research. Educ Res 1976;5:38. 3. Cochrane AL. 19311971: a critical review, with particular reference to the medical profession in Medicines for the year 2000. London: Oce of Health Economics, 1979. 4. Adams CE, Power A, Frederick K, Lefebvre C. An investigation of the adequacy of MEDLINE searches for randomized controlled trials (RCTs) of the eects of mental health care. Psychol Med 1994;24:741748. 5. Rosenthal R. Meta-analytic procedures for social research, Vol. 6, 2nd edn. New York: Sage Publications, 1991. 6. Melander H, Ahlqvist-Rastad J, Meijer G, Beermann B. Evidence b(i)ased medicine-selective reporting from studies sponsored by pharmaceutical industry: review of studies in new drug applications. BMJ 2003;326:11711173. 7. Turner EH, Matthews AM, Linardatos E, Tell RA, Rosenthal R. Selective publication of antidepressant trials and its inuence on apparent ecacy. N Engl J Med 2008;358:252260. 8. Egger M, Davey SG, Schneider M, Minder C. Bias in metaanalysis detected by a simple, graphical test. BMJ 1997;315:629634. 9. Heres S, Wagenpfeil S, Hamann J, Kissling W, Leucht S. Language bias in neuroscience is the Tower of Babel located in Germany? Eur Psychiatry 2004;19:230232. 10. Moher D, Cook DJ, Eastwood S, Olkin I, Rennie D, Stroup DF. Improving the quality of reports of meta-analyses of randomised controlled trials: the QUOROM statement. Quality of Reporting of Meta-analyses. Lancet 1999;354:18961900. 11. Jadad A, Moore A, Carroll D et al. Assessing the quality of reports of randomized clinical trials: is blinding necessary? Control Clin Trials 1996;17:112. 12. Higgins JPT, Green S. Cochrane Handbook for Systematic reviews of Interventions 4.2.5 in the Cochrane Library. Chichester: Wiley and Sons, 2005. 13. Cooper H, Hedges LV. The handbook of research synthesis. New York: Russell Sage Foundation, 1994. 14. Citrome L. Compelling or irrelevant? Using number needed to treat can help decide. Acta Psychiatr Scand 2008;117:412419. 15. Cohen J. Statistical power analysis for the behavioral sciences. New York: Academic Press, 1969. 16. Orwin RG. A fail-safe N for eect size in meta-analysis. J Ed Stat 1983;8:157159. 17. Leucht S, Barnes TRE, Kissling W, Engel RR, Correll C, Kane JM. Relapse prevention in schizophrenia with newgeneration antipsychotics: a systematic review and

21.

22.

23. 24.

25.

26.

27.

28.

29.

30.

31.

32.

33.

450