—eria. waewr- Corer, Research Methods in Applied LinguisticsPublished in this series sacrMan: Fundamental Considerations in Language Testing BACHMAN and paLaeR: Language Testing in Practice ‘rurrr: Individual Freedom and Language Teaching BRUMFIT and canrTER (eds.): Literature and Language Teaching, ‘CANAGARATAH: Resisting Linguistic imperialism in Language Teaching, ‘Cowen and Macano (eds): Language Learner Strategies ‘cook: Discourse and Literature cox: Language Play, Language Learning cox and srrmtstorer (eds: Principle and Practice in Applied Linguistics ‘aLLis: SLA Research and Language Teaching tis: Task-based Language Learning and Teaching ELLIS: The Study of Second Language Acquisition 11s: Understanding Second Language Acquisition [ELLis and BAREHUIZEN: Analysing Learner Language Fotos and wassajt (eds): Form-focused Instruction and “Teacher Education HOLLIDAY: The Struggle to Teach English as an International Language MOWATT: A History of English Language Teaching [JENKINS: English asa Lingua Franca {jenkiNs: The Phonology of English as an International Language xnaMscit: Context and Culture in Language Teaching xeRNs Literacy and Language Teaching, knaMscu: Context and Culeue in Language Teaching tanTot# (ed): Sociocultural Theory and Second Language Learning LaNTOL® and THORNE: Sociocultural Theory and the Genesis ‘of Second Language Development LARSEN-FREEMAN and CAMERON: Complex Systems and. Applied Linguistics ‘Mack (ed.): Conversational Interaction and Second Language ‘Acquisition eissto®: Language Learning in the Age of Satelite Television NATFINGER and DECARKICO: Lexical Phrases and Language Teaching PHILLIPSON: Linguistic Imperialism SeIDLHOFER (ed.): Controversies in Applied Linguistics seL1ceR and SHOHAMY: Second Language Research Methods SKEHAN: A Cognitive Approach to Language Learning ‘sTeRN: Fundamental Concepts of Language Teaching STERN (eds. P. Allen and B. Harley) Issues and Options in Language Teaching ‘TARONE and YULE: Focus on the Language Learner wippowson: Aspects of Language Teaching wippowson: Defining lsucs in English Language Teaching. wippowson: Practical Sylisties wippowson: Teaching Language as Communication ‘wray: Formulaic Language’ Research Methods in Applied Linguistics Quantitative, Qualitative, and Mixed Methodologies ZOLTAN DORNYEI OXFORDen ots Dornyei, Zoltan Research Methods in Applied Linguistics = ‘Quantitative, Qualitaive...Zoltan Darnyel. TNA ile She e we eae AE TOV le gas BE SE ION a HE we tsa Beene vean saver sins (6 se nga Lane pnts Shee ash ee WS tines sate tins pe as at «Zoltan Domyei ing! Research Methods in Applied Linguistics [Sp leds LE ak ATAP GEA A lg had Noe yg ale te ne erly gs LAE 2 tn hE la Sas le han bal te Mey Sys PEPPANAT cE PPEAMIPE «PPRMPIIN PPE ANY gH TY OLY {Bhd aay T+ MAREN AT GA AH ite alte ade Gh atlas ates GF SELLS AAEBWO Sg al sgn ep ME Ae cia OMe PERO igi gy He ld SES SePART ONE Contents Preface~9 Acknowledgements — rr Key issues in research methology — 13 Introduction: what is research and how does this book approach it? ~ r5 ror The good researcher ~ 16 1.2 My approach to research methology — 18 1.3 The content of his book - r8 1-4 Further reading —2r ‘Qualitative, quantitative, and mixed methods research ~ 24 2.1 The qualitative-quantitative distinction ~ 25 2.2 Quantitative research — 30 2.3, Qualitative research - 35 2.4 Mixed methods research — 42 2.5 My own paradigmatic stance - 47 ‘Quality criteria, research ethics, and other research issues ~ 48 31 Quality criteria for research ~ 48 3.2. Research ethics ~ 63 3.3. Research questions and hypotheses 72 3:4 Other essentials for launching a study: pilot study, research log, and data management 75 Longitudinal versus cross-sectional research - 78 4.1. Definition and purpose of longitudinal research—79 42, Longitudinal research in applied linguistics - 80 43. Main types of longitudinal designs - 8r 4-4 Longitudinal qualitative research ~ 86 4-5, Longitudinal mixed designs — 88 4.6 Choosing a design: longitudinal or cross-sectional? ~ 88 4-7 Quantitative longitudinal data analysis~ 90PART TWO, 5 PART THREE 9 Data collection - 93 Quantitative data collection- 95 5.1 Sampling in quantitative research— 95 5.2 Questionnaire surveys~ ror 3.3, Experimental and quasi-experimental studies ~ 325 5.4 Collecting quantitative data via the Internet — 121 ‘Qualitative data collection 124 6.x Qualitative data ~ 124 6.2 Sampling in qualitative research - 125 65 Ethnography 129 6.4 Incerviews~ 134 615, Focus group interviews — 44 66 Introspective methods ~ 147 67 Case studies— 152 68 Diary studies— 156 69 Research journals~ 159 ‘Mixed methods research: purpose and design ~ 163 7.1 The purpose of mixed methods research ~ 164 7.2. The compatibility of different research paradigms ~ 166 7.3. Main types of mixed methods designs ~ 168 3-4 Why don't people mix methods more? — 173 (Classroom research ~ 176 8.1 Origins and main types of classroom research ~177 8.2, Classroom observation = 178 8.3 Mixed methods classroom research ~ 186 8.4 Difficulties and challenges in classroom research ~ 187 8.5 Action research - 191 Data analysis - 195 (Quantitative data analysis ~ 197 9.x Computerized data analysis and SPSS — 197 $9.2 Preparing the data for analysis~ 98 ‘9.3, Data reduction and reliability analysis— 206 9.4 Key statistical concepts ~ 207 9.5 Descriptive statistics - 223 9.6 Comparing two groups: t-tests ~ 215 ‘9:7, Comparing more than two groups: analysis of variance (ANOVA) ~ 228 9.8 Correlation~ 223, 9.9 Non-parametric tests- 227 9.10 Advanced statistical procedures ~ 230PART FOUR % PART FIVE 4 ‘Qualitative data analysis 242 ro. Main principles of qualitative data analysis— 243 10.2 Qualitative content analysis— 245 x0.3 Grounded theory ~ 257 0.4 Computer-aided qualitative data analysis (CAQDAS) ~ 262 Data analysis in mixed methods research ~ 268 1.x Data transformation - 269 1.2. Extreme case analysis—272 11.3 Typology/category development —272 111-4 Maltiple level analysis ~ 273 11.5 Computer-aided mixed methods data analysis — 273 Reporting research results — 275 ‘Writing a quantitative report 277 12.1 Two functions of academic writing 278 12.2 Style manuals 279 12.3 The structure of the quantitative research report- 280 12.4 Reader friendly data presentation methods ~ 287 ‘Writing qualitative and mixed methods reports ~ 290 13.1 Writing a qualitative report ~ 290 13.2 Writing a mixed methods report 300 Summing up — 305 How to choose the appropriate research method ~ 307 4.1 General recommendatio adopt pragmatic approach 307 14.2. Rescarch content considerations ~ 308 114.3 Audience considerations - 309 4.4 Practical considerations - 309 114.5 Personal considerations - 310 114.6 General recommendation I: consider mixing methodologies ~ 313 14.7 Final words ~ 313, Afterword 315 Bibliography — 327 Index~ 329Preface ‘When I started my PhD research in the mid-1980s, I was unaware ofthe fact. thar there was a fierce ‘paradigm war’ raging around me between two camps of research methodologists: the constructivisv/interpretists, who used qualita- tive methods, and the positivistlempiricists, who used quantitative methods. Had I known about this division at the time, I would probably have been inclined, ideologically, to join the constructvistinteepretise camp. However, asithappened, he main research tool employedin the area of my interest—the social psychology of second language acquisition and atttudes/motivation in particular—was the ‘attitude questionnaire’. By using this instrument for my PAD research, I now realize that I was iresistibly propelled into the midst of the positivistcamp. was so ignorant of these issues that when I was first told—in the 19908— that was a‘positivis’,I needed to find out what ‘positivism’ actually meant. 1 learnt that referred to a scientific paradigm and worldview that assumes the existence of an objective and independent social realty ‘out there’ that can be researched empirically with standardized scientific instruments. However, Talso noticed that the term ‘positivist’ was almost exclusively used by ‘people who did not themselves align with this paradigm and that thei usage ‘of the term was definitely not positive: it seemed to indicate that someone was a bit thick-headed and had definitely lost touch (and probably also had ‘macho leanings). This was not good news and, to add to my sorrows, not only was Ia positivist but what many did not know, Iwas also a ‘positivist traitor’: in the mid-1990s I started to conduct tentative qualitative research to investigate the concepts of demotivation and group dynamics in SLA, and cover the past decade most of my PhD students have been engaged in either qualitative or—even worse—combined qualitative-quantitative research. I was clearly ina mess. However, my cheerless story seems to have taken a happier turn recently: Thave learnt that I now qualify to be a ‘pragmatist’ as a researcher, that i, « proponent of a respectable philosophical approach, and the research my students and I have been conducting is ‘mixed methods research’, deemed commendable and highly sought ater in some circles. What a fortunate time to be writing this book! In the light ofthe above, Ican now admit that the approach that guided me in prepating the manuscript of this volume has been pragmatic in every respect. My practical dilemma has been this: research is a complex, multi- faceted activity, such that itis not easy to provide novice researchers with10 Research Methods in Applied Linguistics relevant procedural knowledge without overwhelming and thus disempow- cring them. Furthermore, even though acquiring research expertise may well bea life-long process, we simply cannot hold back with our first investigation until we have learnt every relevant detail. Instead, what seems to me the most important thing about doing research is that we get down toi, ‘get our feet ‘wet’ and as a result get ‘hooked’ on what isa very exciting activity. So, how can research methodology be taught effectively and whatcan we do to prevent ‘young scholars from launching investigations that may violate key principles ‘of scientific inquiry? My answer has been to assume that there exists basic threshold of research. expertise which, once achieved, allows us to embark on the process of doing, decent research that we will not be ashamed of when looking back a decade later. We can find an interesting analogy to this idea in the psychoanalytic theory of the x960s: Winnicott (1965) introduced the concept of the ‘good enough mother’, which was then extended to parenting in general. The concept ofthe ‘good enough parent” suggests that in order to produce psycho- logical health in the child, we do not need to be perfect; instead, ‘good enough parenting’ only requires the parent to exceed a certain threshold of quality ‘parenting (including empathic understanding, soothing, protection and love) to promote healthy development (Bettelheim 1987), without necessarily having to be a ‘Supermum” or ‘Superdad’. Personally, I Have always found this notion encouraging and thus, following the ‘good enough’ analogy, ithas been my aim in this book to summarize those key components of research ‘methodology that are necessary to become a ‘good enough researcher’. strongly believe that che ‘good enough researcher’ needs to master some knowledge of both qualitative and quantitative research, as well as ways of combining them. In the Handbook of Mixed Methods in Social and Bebav- iowral Research, Tashakkori and Teddlie (2003¢) point out that perhaps the ‘worst residue of the ‘paradigm war’ in research methodology has been its impact on students. As they argue, many of our students are asa consequence suffering from a fractured ‘dual-lingualism’, which represents a spit person- ality in methods of study and ways of thinking, Atany given moment, they are asked tobe either qualitative or quantitative, and while in each modeto forget thatthe other exists. This is clearly undesirable and, luckily, applied linguists have by and large steered clear of such extreme positions. I hope that most readers ofthis book will gree with the need to educate a new generation of ‘good enough researchers’ who are sufficiently familiar with both qualitative and quantitative methods to be able to understand and appreciate the results coming from each school, and, perhaps, even to vary the methods applied in their own practice according to their particular research topic/question. ‘Thus, [am in full agreement with Duff's (2002: 22) conclusion that ‘a greater collective awareness and understanding (and, ideally, genuine appreciation) of different research methods and areas of study would be helpful to the field at large’. Accordingly, the writing of this book has been motivated byPreface 1x the wish to go beyond paradigmatic compartmentalization by highlighting the strengths of both approaches and by introducing ways by which these strengths can be combined in a complementary manner. Acknowledgements ‘Writing this book has been an enjoyable journey and have been encouraged and helped by a number of people along the way. First and foremost, I would. like to express my thanks to Cristina Whitecross from Oxford University Press, who has been much more than merely a good editor. Her ongoing support forthe projec right from the beginning has been invaluable. Special thanks realso dueto Simon Mutison-Bowie, who has not only done a careful job copy-editing the text buthas also provided useful suggestions with regard to the content and the clarity of presentation. Patsy Duff and Jean-Marc Dewaele gave their backing to the book when it was still ttle more than a proposal—thank you once again and I do hope that in the light ofthe final version you won't regret your decision. have been fortunate to have several friends and colleagues who were willing to read the whole or various parts of the manuscript and give me detailed suggestions on how to improve the book. My warm thanks are due toKathy Conklin, Judit Kormos, Maggie Kubanyiova, Patsy Lightbown, and Henry Widdowson for their detailed and constructive comments (which, of ‘course, caused me a lot of extra work...). have done my best to take as many of the comments on board as possible and I do believe that the manuscript has improved considerably as a result. In the end, however, this is my own personal take on research methodology, with—no doubt—several omissions and flaws. Looking back, this has been a much bigger project than originally “anticipated, but I do hope that I managed to convey some of the fascination that I fele while exploring what makes research work. 1 Iwas pleased to find out later that Hammersley and Atkinson (1995: 3) shared my perception; as they concluded: “Today, the term ‘positivism’ has become little more than a term of abuse among social scientists, and as a result its meaning has become obscured’. ‘The author and publisher are grateful to those who have given permission 10 reproduce the following extracts and adaptations of copyright material: p.r82 text adapted from Communicative Orichtation of Language Teaching Observation Scheme: Coding conventions and applications: Part A. (1995). With permission from the National Centre for English Language Teaching and Research. Copyright © Macquarie University. Pp 252,253, 255: NVivo 7 screen shots are reproduced with kind permis- sion from QSR International and are subject to copyright. NVivo and NVivo 7 ate trademarks or registered trademarks of QSR International. ‘www.qstinternational.com Pp 201, 204, 216, 236, 235: SPSS Software Inc.PART ONE Key issues in research methodologyI Introduction: what is research and how does this book approach it? ‘When students first encounter research methodology in the social sciences, they are likely to hear terms such as ‘statistics’, ‘sampling’, ‘reliability’ and ‘validity’, which make this area look very complex and technical, and also quite intimidating. Indeed, for a normal person the image of a social sci- entise submitting the results of Likert-sale items obtained from a stratified random sample to structural equation modelling is hardly more real than that of the sombre-looking natural scientist in a white coat doing all sorts of strange things in a laboratory. Such technical terms and images all suggest that research is something that only experts do and ordinary humans should steer clear of. In this book I would like to present a more user-friendly facet. ‘of research, trying to show that with a bi of care and lots of common sense all ofus can conduct investigations that will yield valuable results I do hope that by the end ofthis book most readers will agree that “You can do itifyou really want to’. So what is research? In the most profound sense ‘research’ simply means trying to find answers to questions, an activity every one of us docs all the time to learn more about the world around us. Let us take a very specific ‘example, say, wanting to find a music CD at the lowest possible price. To do s0 we can go to several shops to compare prices and nowadays we can also search the Internet for the best price. Throughout all this what we are doing is ‘research’, Not necessarily good research, 1 admit, because our investigation will probably not be comprehensive: even if we find what seems to be a really ‘g00d price, we will not know whether a shop or a website that we have not visited might have offered the same CD for even cheaper. To do good research ‘we need to be systematic so that by the end we can stand by our results with confidence. Thatis, research in the scientific sense is‘the organized, systematic search for answers to the questions we ask’ (Hatch and Lazaraton 1991: 1). (Or in short, research is disciplined inquiry. Following Brown (1988), we can identify two basic ways offinding answers to questions: