http://www.developingteachers.com/articles_tchtraining/researcharticlespf_mark.

htm

Mark's Critiquing Qualitative Research Articles print friendly 5 developingteachers.com

Critiquing Qualitative Research Articles

by Mark Firth

The purpose of this paper is to demonstrate how we can critique a qualitative research article according to
the criteria as recommended by McMillan & Wergin (1998:pp.7-12). The sample article being examined is
by Goh (1999), A cognitive perspective on language learners listening comprehension problems. Whilst it
is not possible to reproduce the article here, it is suggested that the critique procedure itself is being
exercised in order to demonstrate its usefulness to readers of qualitative research. The guiding questions
operate as effective criteria to understanding the quality and benefits of any given qualitative research
paper.

Introduction

With regards to qualitative designs, Punch (1998:p.150) explains that within any given piece of research a
combination of elements from different designs can overlap. As a result, a study may not necessarily be a
case study, an ethnographical study nor a work based on grounded theory but rather a mixture of two or
more of these approaches. Punch does however suggest that it is still useful to consider each separately.
This is important to keep in mind when critiquing an article such as this as in fact components of different
approaches and paradigms co-exist within the one research body as the author has attempted to uncover
what it is she is looking for. This does not necessarily mean however that a research article needs to be
labeled as being one type of research or another, but rather to take it as what it is; within the context of the
investigation and considering the limitations in which it was carried out, we can still grasp an idea of its
validity and usefulness.

Is the general purpose of the study clear? The introduction cites numerous references of previous
research describing general and specific factors affecting listener difficulties in a second language. The
author, Goh (1999), describes the different approach to be taken in this particular research concerning
categorizing real-time processing problems. Paradigm issues in regards to methods for gathering and
indeed the ways of treating data are revealed in the introduction. This provides the reader with a feeling for
the approach to which the researcher values and thinks about evaluating listeners difficulties.

Is the study significant? We are directed to go along with the assumption held by the researcher about the
potential value of understanding the mental processes relating to difficulties in listening and gaining
insights into students attitudes towards these difficulties. Again, the value of such research to the
discipline of language acquisition is relative to the approaches within the approach of using metacognitive
processing data and categorizing according to cognitive frameworks. For the purpose of thinking about
learner listening difficulties and how teachers can cater for students needs the research is justified.

Will it make a practical or theoretical contribution? The pragmatic potential benefit of this study is
mentioned in the introduction i.e. to be better able to identify the source of learner difficulties from a
cognitive point of view, but oddly enough a lengthy paper is presented after the summary of findings. Here
the researcher prescribes how teachers should deal with learners listening difficulties. Therefore, the
researcher could be seen to be trying to verify her own beliefs about teaching approaches and how to
assist learners in becoming better listeners and to support what certain others have suggested, namely
Field (1998) on p.69. Assuming good reliability the recommendations made by Goh could be viewed as
being potentially very useful.

Following
Page 1 of 10 this, the article serves another purpose regarding methodology; as introduced
Feb in
09,the abstract,
2017 the
02:04:06PM MST
http://www.developingteachers.com/articles_tchtraining/researcharticlespf_mark.htm
Following this, the article serves another purpose regarding methodology; as introduced in the abstract, the
author attempts to validate the methods used and suggests similar approaches for further research. This
suggests that if the reader is willing to accept the way in which the data has been treated and analyzed
then there is a potentially significant contribution to be made to the conceptualization of and in turn practice
of dealing with listening difficulties.

Is the introduction well organized and clear? Difficulties in reading the article may arise simply by the
layout of the document. Notably, the combining of the literature review with the introduction makes it hard
to distinguish between the empirical studies which form a historical context for thinking about the issues
involved and the current literature that has led the researcher to formulate the specific research questions.
Ideally it would have been better to separate these for their attributing purposes.

The organization of the introduction does however effectively establish a framework for the way in which
the researcher proposes to study difficulties in listening. Firstly, the idea of understanding problems from
learners own perspectives and secondly, the cognitive framework for understanding language
comprehension.

Review of literature

Is the review comprehensive and up to date? The literature that is cited is recent but tends to lack depth.
Much of the review consists of discussion of the tool that will be used for data analysis, which results in
lengthy explanation of its elements. While this can be seen as being necessary, little else is discussed.
Maybe little research exists within the specific area of evaluating listening through a cognitive framework,
however more relevant information concerning cognitive processes e.g. other possible models could have
deepened the discussion. By the nature of the study the researcher knows what she will be looking for;
through the use of an evaluative study, the narrative accounts (in this case students own reflections) are
intended to be explained and judged as described in Meriam (1998). More description of research that
suggests this approach could have been advantageous.

Is there an emphasis on primary resources? Put simply, we think so. Most of the citations appear to be
based on empirical studies but few details are given. It must also be mentioned however that this study (as
stated) was carried out in contribution to a larger body of research. Cohen et al. (2000:p.183) mentions that
case studies are often utilized to provide finer details to complement large-scale investigations. While this
body of research could not be termed as being a case study per se, by studying the listening difficulties a
small group of learners face, what is deemed common for this case may well also be considered common
for others as described by Punch (ibid:pp152-3). Little else resembles a case study-like investigation
except to say that perhaps the elements that make up a psychological case study may also be relevant.
Discussion on psychological research methodology is beyond the abilities of this author and so except for
the above-mentioned similarities, the article by Goh is generally considered to be ethnographical research.
Discussion of which is described later.

This may also explain the reason as to why the article doesnt contain many details on how the research
question was operationalized supported by the use of a large number of primary resources. As it reads by
itself the article tends to lack rich description in the literature review.

Is there a critical review or a review of findings? Whilst the review argues strongly for the strategic
(methodological) approaches it uses, no disadvantages or limitations are described. For example,
Andersons cognitive framework of language comprehension is lauded and backed up by other
researchers testimonies but no weaknesses or criticisms for conceptualizing an invisible process in this
way are presented.

Page 2 of 10 Feb 09, 2017 02:04:06PM MST

http://www.developingteachers.com/articles_tchtraining/researcharticlespf_mark.htm

Is the review well organized? As mentioned above the introduction should have been kept separate. The
layout does however clearly demonstrate the way in which the researcher intends to go about the study
the disadvantage of this being the somewhat predictability of results and certainly the approach to be taken
for data analysis (though this could be also seen as a strength of the study).

Does the review clearly relate previous studies to the current research problem? Looking at research that
has gone into listener difficulties in general, the review goes in to great detail about the stages of listening
comprehension according to Andersons model. It seems unfortunate that no other investigations are
referred to regarding listening processes within the field of psychology or language acquisition. In fact a
large body of research exists: Dirven & Oakeshott-Taylor (1984), (1985) have both researched and
reviewed the psychological processes involved in various levels of listening; and Nagle & Sanders (1986)
present an information processing model of listening comprehension of their own. If the author could have
referred to such models and discussions and pointed out where Andersons construct is more
advantageous, then a clearer link with previous research could have been established.

Does the review help establish the importance of the research? Whilst the conceptual framework for the
study is clear, the literature review does not strongly argue for the need for such a study. This is except to
say the author notes the extensive preceding research that has focussed on other types of factors affecting
listener difficulties. If a lack of research in an area does exist, this does not necessarily automatically
validate the need for research and so critical discussion about related material could be seen to be
necessary.

The author does refer to her own previous research, which we assume has led her to this investigation.
Other than this, the significance of the research is left up to the reader to decide. When a research article
refers to other documentation of the same author this is generally a sign of weakness as it suggests the
researcher has a good idea of what she intends to find in order to support some kind of argument,
McMillan & Wergin (1998:p146). Again to be fair however, we must bear in mind the study is part of Gohs
own larger research.

To sum up, the introduction and review of literature thoroughly introduce the approach to be taken and
elements comprising the design (use of learners self reports and analysis via Andersons model), yet we
still dont know how it is the author came to the decision for this particular design.

Research problem, question or hypothesis

Is the problem or hypothesis clear and concise? It appears that no specific research questions are
actually stated. Again this poses a problem because we know what the researcher is going to do but the
specific questions that need answering are not explicitly written. The purpose of this ethnographic research
is not necessarily to generate hypotheses but rather to assist practitioners to conceptualize learner
difficulties within a model framework. Similar to the characteristics of a case study the approach has a
specific idea of what it wants to find out and from whom. The article could have expressed its questions of
investigation something along the lines of:

From their own perspectives, what real-time listening comprehension difficulties do English as a
Foreign Language (EFL) learners face? And.
At what stages of the language comprehension model according to Anderson (1995) do these
difficulties occur?

What type of research is being carried out? Flecks of the research description resemble the processes
often associated with quantitative research. The deliberate use of constructs to acquire and analyze data

for3immediate
Page of 10 interpretation for example tells us the researcher not only knows what information she needsMST
Feb 09, 2017 02:04:06PM
http://www.developingteachers.com/articles_tchtraining/researcharticlespf_mark.htm
for immediate interpretation for example tells us the researcher not only knows what information she needs
but almost hypothesizes as to what the outcomes will look like. Qualitative research does not usually
impose structures on the data until the implications of the data begin to emerge Punch (ibid:p.25). And so,
such a vivid conceptual framework and design usually tends to be associated with quantitative research.

Without going into too much detail about methods, there is however a concern to reveal the subjective
beliefs of the learners being studied and an investigation into definitions of the situation. These purposes
are attributed to the approaches of participant observation and ethnography within the realm of qualitative
research as described by Wainwright (1997) in the defining of qualitative research.

A lot of flexibility is given to the qualitative researcher with regards to methodological issues as Punch
(ibid:p.161) tells us with regards to ethnographic research. Continuums of data collection techniques exist
from non-participant observation to words of the natives participant descriptions. In fact it seems as
though almost any kind of technique can be used in an attempt to gain a fuller picture of the live data.

Again to categorically label the type of research would limit the potential value of the research and we need
to take it for what it is worth. It could be said that the researcher in this instance has culminated the benefits
of a tight design, employing ethnographical data collection methods from a group of case study-like
participants. From the outset it is evident that the researcher has generalizability in mind as the study is
intended to inform teachers of how they can better assist learners by knowing where listening difficulties
can occur Goh (ibid:p.57).

Methodology: participants

Is the population described adequately? As a population, ESL students are identified as learners who at
different stages of listening proficiency experience various difficulties.

With regards to the confines and purposes of this study it could be argued that further description of the
population is not required. However, one point that could be worth mentioning is that we dont really know
of the differences (if any) that exist between learners of different first languages.

Is the sample clearly described? Whilst the article doesnt go to great length to describe the sample, for all
intensive purposes of this study the information could be said to be adequate. That is to say with regards to
the sample there doesnt appear to be a large number of extraneous independent threats, which could
affect the results.

Is the method of selection of the sample clear? It is not mentioned as to how or why 17 of the 40 students
were chosen or volunteered to participate in the small group interviews, nor how 23 students came to take
part in the retrospective verbalization procedure. While logically it would appear that in this kind of study
this should not affect the results, it would have been more comforting for the reader to have such details.

Could the method of selection affect the results? To answer this question it is first necessary to assertion
how indicative the difficulties faced by the sample are of foreign language learners in general. By using the
most practical and readily available sample to the researcher, an all-Chinese student sample poses
concerns about an important issue of how representative the sample is. It would have been beneficial for
the researcher to discuss the peculiar difficulties Chinese learners of English have in regards to phonetics.
For instance, the English language consists of certain phonemes not found in Chinese which are difficult to
learn, while others are similar to the phonemes found in Chinese but pronounced differently. In general,

Chinese
Page 4 of 10 speakers of English find English hard to pronounce, and have trouble learning to09,
Feb understand the MST
2017 02:04:06PM
http://www.developingteachers.com/articles_tchtraining/researcharticlespf_mark.htm
Chinese speakers of English find English hard to pronounce, and have trouble learning to understand the
spoken language Swan & Smith (1987:pp.224-5). Likewise other EFL learners face common difficulties
specific to interference from their first language or large phonetic differences Nunan (1991).

The type of data that is to be collected somewhat avoids these difficulties in the sense that it is to be
analyzed consistent with the cognitive model generally accepted to apply to the process of developing
listening comprehension. In doing so the specific difficulties concerning phonemes peculiar to any given
first language can be considered separately in the discipline of phonetics. It could be concluded therefore
that it is not likely that the participants backgrounds and characteristics will influence the results.

Are subjects likely to be motivated to give biased responses? This would be unlikely. Even if the learners
were well aware of what the researcher was looking for this could be seen to only enhance the reliability of
the data through the ethos between learners and interviewer. It would however have been enlightening to
have some reference to the relationship between the researcher and the participants i.e. we dont know
how frank the students were all the time in the interview sessions.

Methodology: instruments

Is evidence for validity and reliability clearly presented and adequate? Whilst it is generally accepted in
qualitative research that validity can not be achieved, to maximize validity should always remain a goal of
the researcher. Validity then, should always be seen as a matter of degree rather than as an absolute
state Gronlund (1981). The decision to use a combination of these data collection instruments was a
useful way to attempt to understand what kind of difficulties students were having. This eliminated the
possibility of mono-method bias or a threat to validity where a construct is measured by only one means
Trochim (1999). To this end, a reasonable level of depth and scope of learner difficulties could be attained.
If the interviews, diary entries and verbalizations were carried out honestly and authentically then these
instruments could be seen as some of the most valid and practical for ascertaining what invisible and
cognitive problems are occurring.

Reliability in quantitative research is often thought of as consisting of stability, equivalence and internal
consistency Cohen et al. (2000:p.117). Within the qualitative paradigms however, the canons of positive
reliability concerns are viewed at the very least debatable and to the greatest extent - irrelevant. This does
not in any way however take away from the pursuit of reliable and accurate research; but rather throws a
light back on to the concerns of validity and how effectively the researcher has encapsulated the truth
about the way a culture is behaving at a given moment and the ramifications for this knowledge. For this
reason reliability is often construed by qualitative researchers (Guba & Lincoln, 1985) to be referred to as
being dependability. The article by Goh refers to few measures taken to ensure dependability. For
example, we dont know how used to writing reflective journals students were nor do we have any details
about how the small group interviews were conducted. More reliability issues are discussed below.

Is there a clear description of the instrument and how it was used? The article does not explicitly give
details about what the students were asked write about in their listening diaries. We assume transcriptions
of the interviews were made by the references made in the data analysis but we were not informed clearly
enough of the procedures that the researcher planned to undertake in the methods section.

Similarly, what was to be done with the data is not mentioned until later in the article i.e. the way in which
labeling was done before categorization. Often qualitative research is seen as a process of going
backwards and forwards from the data to the design as the data reveals concepts to the researcher. In this
case however, by the nature of the pre-operationalized design, the researcher clearly envisaged what she
was going to do and could have detailed it here. Finally, no description of the verbalisation procedure

developed
Page 5 of 10 by Ericson & Simon (1987,1993) is given. If these principles for collecting verbal data02:04:06PM
Feb 09, 2017 could be MST
http://www.developingteachers.com/articles_tchtraining/researcharticlespf_mark.htm
developed by Ericson & Simon (1987,1993) is given. If these principles for collecting verbal data could be
verified as indeed being reliable, so too could the results and interpretations of this research be viewed as
more stable.

Is there a clear description of how the instrument was administered? The lack of details concerning the
administration of the diaries, interviews and verbalization process could be seen as the greatest weakness
of this study. It would have been better to have a full description of who conducted them and the duration of
each instrument. When reading this article, questions that come to mind which need answering include:

What instructions and guidelines were students given for completing diaries?
What was the nature of the listening texts? Were they monologues, dialogues, varieties of tapes,
videos and live conversations? What was the degree of difficulty?
How much were students required to write and were they given any assistance in spelling,
wording or writing the diaries?
Who conducted the small group interviews and were they trained for doing so?
Was any credit given to students for participating in the interviews or verbalisation procedure?
How many interview sessions and verbalisation sessions took place in turn producing how much
data?

Is it likely that subjects would fake their responses? One of the strengths of this study is the reliability
concerning trying to understand learner difficulties. In other words there is no immediate reason to see why
respondents would feel a need to perform or give politically desirable information, as there is a perceived
mutual benefit for researcher and participant alike.

Are interviewers and observers trained? This is a critical issue to ensure consistency and reliability of data.
Punch (ibid:p.175) contends that interviewers require training to maintain a common approach for the
questions and techniques to be in line with each other. Since we dont know how many interviewers there
were we dont know about their competencies.

Methodology: procedures

Are there any clear weaknesses in the design of the study? Paradoxically the strength of the study could
be viewed as being one of its weaknesses. For the intensive purpose of gathering reflective data for to
super impose a cognitive processing classification model discounts the data to speak for itself in any other
way. It would be interesting to use a different coding system and to try not to determine any pre-conceived
criteria for classification. This does not threat the validity of the study in any way but rather indicates the
extent to which a holistic approach wasnt undertaken.

Are the procedures for collecting information described fully? It would have been helpful to have more
detailed information about what kinds of questions made up the semi-structured interviews. We can only
suppose that the questions were not overly suggestive or leading student responses. Despite this, for
replicability in future studies, or just for reference, this would have been beneficial for professionals to know
about the kinds of questions they can ask learners having difficulties.

Is it likely that the researcher is biased? In this article Goh mentions some safeguards which she took
against bias, namely, the use of colleagues to triangulate classifications of learner problems. While there is
no possibility of observer bias effecting the results through the use of transcripts some misinterpretation of
the data in the coding is possible. This concerns the fact that the learners were required to carry out an

examination
Page 6 of 10 of their beliefs in such a metacognitive manner - which requires extensiveFeb
training
09, 2017as
02:04:06PM MST
http://www.developingteachers.com/articles_tchtraining/researcharticlespf_mark.htm
examination of their beliefs in such a metacognitive manner - which requires extensive training as
contended by Kohonen, Jaatinen & Lehtovaara (2001) and recommended in the discussion section of this
study by Goh herself.

Ironically the area of study (understanding meaning in a second language) in which this investigation takes
place is also the very concern for interpreting the data. Comprehension processes is the side of
psycholinguistics which looks at discourse analysis or put simply how we make sense of texts Finch
(2000:p199). According to Widdowson (1993) schema theory operates on two levels: systemic the
phonological, morphological and syntactic components of language; and schematic our background
knowledge. For comprehension of language, a match is required between the encoded systemic text and
our own schematic knowledge.

In sum, difficulties may have arisen at any of the following stages of communication:

When students were asked to explain the difficulties they had in listening they may not have fully
understood what they were supposed to respond.
When students voiced their various difficulties they may have easily had trouble verbalizing such
high order issues in a second language.
When the researcher analyzed the data we rely heavily upon her (maybe plus one or two
colleagues) schema for accurate interpretation.

It may have been useful to allow learners to report their difficulties in Chinese and English for a richer
conceptualization. We are also not told if the comments that were made in Chinese were translated and
recorded, after all, these may have been the most indicative verbalizations of the listening problems faced.

Results

Are the findings presented clearly? The article presents the findings systematically by problems the
learners faced making for a logical read. There could be an overuse of numbering however, especially the
need for numbering student comments doesnt seem to be necessary. Otherwise the document is most
readable with a convenient summary of findings at the end.

Is there appropriate use of tables, charts and figures? The tables and charts give an overview of the
results and point the reader to the text with an idea of the importance of each listener problem (as
measured by number of respondents with the attributing problem). It may have been useful to see some of
the tabulations that were made to illustrate the usefulness of each data collection method. After all Goh
does use this paper later to advocate the importance of developing metacognitive reflections in language
learners. It would be interesting to know if one method could be said to be more beneficial than another; a
table showing which mode of data collection revealed which problems would have been beneficial. This
would also clearly display evidence of the process of triangulation through the use of different data
collection methods.

Is the number of subjects taken in to consideration when presenting the results? A need could be argued
for a table summarizing the frequency each problem occurred. Whilst it would not be very statistically
reliable to do so we could still get an idea for how often listeners faced particular problems. Although the
sample size is not a great concern for the purpose of this study, cross-examinations could be interesting
with learners of different first languages.

Is there sufficient descriptive information to interpret the results? This is not seen as a problem with this
piece of qualitative research as the author uses only two charts and goes to great descriptive length of the

results.
Page 7 of 10 In fact many of the comments made in the results are more discussion or conclusion orientated
Feb 09, 2017 as MST
02:04:06PM
http://www.developingteachers.com/articles_tchtraining/researcharticlespf_mark.htm
results. In fact many of the comments made in the results are more discussion or conclusion orientated as
they describe the possible reasons for each problem. This allows for a more in-depth read of the results as
each problem is individually addressed.

Do illustrative quotes and specific instances accompany the results? The pragmatic benefit of the
research to practitioners could be said to lie here. By the use of direct quotes from the participants
themselves, teachers can readily identify with them with the comments made by their own learners. As Goh
discusses each problem, references are also made to the results of other empirical studies to clarify the
nature of the difficulties learners are facing, e.g. Craik & Lockharts (1972) theory ... without depth of
processing, information received will be quickly forgotten. This makes for a rather rich description of the
data but on the other hand leans on the side of discussion too.

Discussion and conclusions

Is the discussion based on the research problem and results, or is there a tendency to discuss unrelated
material or ideas? The results section is intertwined with the discussion making it difficult to know what
was exactly found in this case and what Goh simply contends as being good teaching practice. In section 5
Goh gives advice on helping learners become better listeners as a format of discussion. This is rather
political because the examples of student difficulties are being employed to rally for teaching methodology.
Goh does however discuss other professionals views on approaches to take before making her own
beliefs explicit i.e. supporting that of Field (1998).

Is there an adequate interpretation of the findings? One of the most significant conclusions of the article as
mentioned by Goh was what difficulties were not found, notably the non-presence of phoneme difficulties
as earlier suggested in this paper. This interpretation seems reasonable and certainly stimulates thought
about what knowledge we believe to have about second language acquisition. Goh discusses other
interpretations regarding the significance of the number of low-level comprehension problems being so
high and relates to recommended practice.

Is the interpretation separate from the results? As stated above, much of the results section is infiltrated
with discussion and it appears the discussion has been substituted by Gohs recommendations on helping
learners become better listeners. This leaves readers with a complex task of determining for themselves
what conclusions can be said to have derived directly from this research.

Are the results discussed in relation to previous studies? Relatively few studies are referred to in the
discussion. Rather, there seems to be a build up of Gohs previous work advocating how teachers can
foster listening strategy development. Certainly more citations from other empirical studies would have
been more convincing than mere references to certain others ho share the same beliefs.

Are limitations due to methodology included in the discussion? No they are not. The rationale for
considering learner listening difficulties within a cognitive framework was conceptualized in the articles
introduction and is seen as a major strength of the study. All research leaks (Nunan 2000) this
expression which has been adapted from all grammar leaks is particularly useful to keep in mind when
discussing research methodology. With regards to the procedures used, no limitations were recognized
which could be seen as a weakness. Specifically, internal validity is a concern because the
representiveness of the sample was not established.

Are the conclusions clearly stated and based on the results and the discussion? Goh describes what she
did in the research and summarizes her recommendations for adopting direct and indirect strategies. What
the results actually told us in this instance is not summarized however and certain recommendations for
further
Page 8 of 10study dont appear to be necessitated by this work. Furthermore, such recommendations are madeMST
Feb 09, 2017 02:04:06PM
http://www.developingteachers.com/articles_tchtraining/researcharticlespf_mark.htm
further study dont appear to be necessitated by this work. Furthermore, such recommendations are made
in jargon-like form, for example the term word-referent automisation (p.74) doesnt appear to be found in
most phonological and related texts.

Are the conclusions reasonable? Do they go beyond the interpretation of the findings? The findings that
relate to the types of difficulties faced and how we can build a construct for perceiving them are quite
sound. Although many of the recommendations made for teachers are still debatable, the logic used by
Goh to arrive at her position is plausible. This is to reiterate, depending on the readers personal view the
conclusions may be considered reasonable.

What is the external validity of the study? What factors would affect the external validity? Many threats to
external validity have been removed by the tight design of the study and by limiting possible internal
extraneous factors. Punch however (ibid:p.261) explains that for generalization (often referred to as
transferability) to be possible in qualitative studies three questions need to be asked: firstly, does the
sampling capture variation in a diverse enough manner? It would have been better to have more detailed
information of the learners profiles and of how the sampling was done (as mentioned earlier); secondly, Is
the context thickly described so that the reader can judge transferability to other settings. Again,
descriptive accounts of the learners abilities and learning experiences would be required for teachers to be
able to extrapolate to their own settings. Finally, how well have the concepts been extracted from the data?
As noted in the results section, a table showing which mode of data collection revealed which problems
would help readers compare the research setting with their own situations.

Conclusion

To restate, the purpose of this paper was not to be overly critical of a piece of qualitative research but
rather to demonstrate how we can critique a qualitative research article according to the criteria as
recommended by McMillan & Wergin. The guiding questions operate not only as effective criteria to
understanding the quality and benefits of a qualitative research paper, but they also highlight the level of
difficulty that is involved in undertaking empirically sound qualitative research.

References

Cohen, L. Manion, L. & Morrison, K. (2000). Research methods in education. 5 th edn. London: Routledge.

Dirven, R. & Oakeshott-Taylor, J. (1984). Listening comprehension (part 1). State of the art article.
Language Teaching, 17, 326-343.

Dirven, R. & Oakeshott-Taylor, J. (1985). Listening comprehension (Part 2). State of the art article.
Language Teaching, 18, 2-20.

Finch, G. (2000). Linguistic Terms and Concepts. Macmillan: London.

Goh, Christine C.M. (1999). A cognitive perspective on language learners listening comprehension
problems. On line. System 28 (2000) 55-75. www.elsevier.com/locate/system

Guba, E.J. & Lincoln, Y.S. (1985) In Cohen, L. Manion, L. & Morrison, K. (2000). Research methods in
education. 5 th edn. London: Routledge.

Helberg, C. (1995) Pitfalls of Data Analysis (or How to Avoid Lies and Damned Lies). (online) June 5-7
1995. http://www.com/-helberg/pitfalls/
[Accessed 17 August 2000].
Page 9 of 10 Feb 09, 2017 02:04:06PM MST
 [Accessed 17 August 2000].
http://www.developingteachers.com/articles_tchtraining/researcharticlespf_mark.htm

Kohonen, V., Jaatinen, R. & Lehtovaara, J. (2001). Experiential Learning in Foreign Language Education.
Pearson Education Ltd.:Essex.

McMillan, J.H. & Wergin, J.F. (1998). Understanding and evaluating educational research. Columbus,
Ohio: Prentice Hall.

Meriam, S.B. (1998). In Cohen, L. Manion, L. & Morrison, K. (2000:p.183). Research methods in education
. 5 th edn. London: Routledge.

Nagle, S. & Sanders, S. (1986). Comprehension theory and second language pedagogy. TESOL Quarterly
, 20, 9-26.

Nunan, D. (1998). Language teaching methodology. Phoenix ELT: Hertfordshire.

Punch, K. (1998). Introduction to social research: Quantitative and qualitative approaches. London: Sage

Swan, M. & Smith, B. (1987). Learner English: A Teachers guide to inference and other problems.
Cambridge: Cambridge University Press.

Trochim, William M.K. (1999) Research Methods Knowledge Base. 2 nd edn. (online). 06/29/00. {
http://trochim.cornell.edu/kb/consthre.htm}
[accessed 19 August 2000]

Wainwright, D. (1997). Can sociological research be qualitative, critical and valid? The Qualitative Report,
Vol. 3, No. 2. Online. http://www.nova.edu/ssss/QR/QR3-2/wain.html

Widdowson, H.G. (1983). Learning purpose and language use. In Nunan, D. (1998). Language teaching
methodology. Phoenix ELT: Hertfordshire.

Page 10 of 10 Feb 09, 2017 02:04:06PM MST