University of Brighton SASS

SS303 Research Methods

Interpreting and Validating Findings from Qualitative Data

Aims of session: y To introduce issues of reliability and validation pertinent to qualitative research; To relate these to small scale student research projects.

Grounding Qualitative Research A number of issues arise in qualitative research: y Too subjective? y y y Findings difficult to replicate?

Lack of transparency

Main focus on: y Trustworthiness (whether the findings can be considered true to the data gathered an honest interpretation) y Credibility (similarly whether the findings have authenticity to those in the settings described) Dependability (links to reliability whether similar findings would be generated if the study was to be done again) Transferability (whether the findings could be transferred to other similar settings) and Confirmability (similar to objectivity in terms of being able to be confirmed by other researchers who analysed the data for example) (Lincoln & Guba, 1985 cited in Flick, 2002).

roblems of generalisation

Reliability and Validity in Qualitative Research Reliability or dependability y y y y y y y Examining the quality of recording and documenting of data Separation of concepts/observations from interpretations Transcription rules Interview training for interviewers Piloting interview guides or questions Training observers so that they are looking for the same things Training and reflexive exchange about interpretation and methods of coding Systematic consideration interpretations of competing explanations/

Documenting the data gathering and analysis procedure

Validity (credibility/transferability) Issues of validity in qualitative research should not be about bias but about good practices throughout the whole research process. Ways to increase validity by evaluating 1. The production of data and 2. The presentation of the phenomena and the inferences drawn from them.

(1) The researcher should refrain from talking in the field but rather should listen as much as possible (2) He or she should produce notes that are as exact as possible, (3) begin to write early, (4) and in a way which allows readers of his or her notes and reports to see for themselves. This means providing enough data for readers to make their own inferences and follow those of the researcher. (5) The reports should be as complete and as (6) candid as possible. (7) The researcher should seek feedback on his or her findings and presentations in the field or from his or her colleagues. (8) Presentations should be characterised by a balance between the various aspects and by (9) accuracy in writing. From Wolcott (1990 cited in Flick, 2002)

Procedures should be used that illuminate the ways that the researchers values enter into the research by investigating how our findings are anchored in the data. Working with issues of bias is something that should be dealt with using reflexivity and self-corrective techniques rather than obscuring the origins of our conceptual frameworks. Reason & Rowan (1981) talk of objectively subjective inquiry and propose the use of self-corrective checks such as triangulation, reflexivity and member checking.

Triangulation is the use of different vantage points. Triangulation involves combining different perspectives, theoretical schemes, investigators, data and methods, and exploring the subject from multiple standpoints to create a fuller and potentially more valid explanation or interpretation. Crucially: The account of the research process should give the reader sufficient detail of the process of interpretation to allow her/him to make counter interpretations based on the data and to assess the validity of the claims made for herself/himself.

Four types of triangulation (Denzin, 1989 cited in Flick, 2002) y Data triangulation use of different data sources. Time, place, people study phenomena on different dates, places and from different people. Investigator triangulation different observers/interviewers employed to detect or minimise biases resulting from the researcher. Theory triangulation: approaching data with multiple perspectives and hypotheses in mind Various theoretical points of view could be placed side by side to assess their utility and power(p.239-40). Methodological triangulation: within method or between method. E.g between methods might mean using a questionnaire and interviews.

Reflexivity is one of the most well known ways of subjecting qualitative and critical inquiry to validity checking. Reflexive no claims to objectivity, making the researcher visible. It involves making explicit the process by which the research knowledge is produced and understandings formed. This involves acknowledging and incorporating the influences that we as individuals have on our research. 1) Personal dimension: ways in which researchers values, experiences, interests, beliefs, aims and social identity may have shaped the research. 2) Epistemological dimension How our design and method can construct our data and findings, how research questions define and limit findings, consideration of alternative interpretations and approaches Member checking Respondent/Interviewee Validation y y y Involving the research participants in the research process. Can lead to higher authenticity Excellent way of checking out your emerging findings and interpretations see whether the findings ring true Can be problematic - Participants may not consent to particular interpretations. People can be very self-conscious when getting their interviews back It can be a very time-consuming process

Evaluating your findings in terms of validity and reliability Some questions to ask about your own research: y Are the findings I have written up grounded in my research data? Remember it is very important to have a good fit between data and /findings and any emerging theories you may be suggesting from your data; Are the inferences I am making from the data logical? Are alternative explanations accounted for? Is the coding/categorisation structure appropriate? Are the quotes representative? What is the degree of researcher bias (premature closure, unexplored data in field notes, lack of search for negative cases, feelings of empathy)? What strategies were used for increasing credibility (second readers, feedback to informants, peer review, adequate time in the field)? Peer debriefing: regular meetings with other people who are not involved in the research in order to disclose ones own blind spots and to discuss working hypotheses and findings with them Provide sufficient contextualisation: historical, social and cultural location - the use of thick description to allow readers to draw their own conclusions.

y y y

You need to explicitly address the points above in your research report if only in brief.

References Flick, U. 2002. Introduction to Qualitative Research. London: Sage. Reason, P and Rowan, J. (1981). Human Inquiry: A Sourcebook of New Paradigm Research. Chichester: John Wiley and Sons.