ISSUES IN ADOPTING MIXED METHOD IN A RESEARCH ~ EXAMPLES FROM RESEARCH ARTICLES 1.

PURPOSE - is the purpose of adopting mixed methods made clear by the researcher?
* When its not mentioned, the researcher does not give recognition to the full contribution of each method
Article no 1 2 3 4 5 Title Asan, A. (2007), Concept Mapping in Science Class: A Case Study of fifth grade students, Educational Technology & Society, 10 (1), 186 195. Mitchell, Sidney N;Reilly,Rosemary;Bramwell,F.Gillian; Solnosky, Anthony; Lily, Frank ( 2004): Friendship and Choosing Groupmates: Preferences for Teacher-Selected vs Student-Selected, Journal of Instructional Psychology, 31(1), pg.20 Purpose in adopting mixed methods Not mentioned Whats mentioned was the purpose of the study only Not mentioned. Assumed for expansion purposes.?

2. METHOD Flexibility & Appropriateness

* When mixed without careful consideration of the assumptions / rules and expectation regarding their conduct, corruption of those methods can occur. Therefore, the findings are subject to question.
Articl e no 1 Hypothesis / Research Questions (simplified) 1. There will be a statistically significant difference between the experimental group (exposed to concept map program) with the control group (traditional teaching method) with respect to academic achievement. 2. Concept mapping as an instructional tool has a positive effect on students attitudes. Methods used to answer research questions H1: - Pre-test MCQs - Experimental grp: exposed to Inspiration concept mapping program - Control grp : traditional teaching - Post-test MCQs to both grps - CM scoring rubrics H2: - Interview for Exp grp students - One open-ended question, duration is not specified Method Limitation Weakness in instrumentation - metrics for attitude assessment - limited number of students - fixed on attitude only How did the researcher address this issue / What was suggested to overcome the limitation? Longitudinal studies - to explore students attitudes towards the use of concept map - increase sample size - diversify, example: - gender difference - motivation - cultural background

1. Choosing ones groupmates

would have a negative effect on subsequent attitudes toward choice . H1. Teacher as a researcher ( Quantitative ) H1. Pre post test non equivalent group design H1. Questionnaire ( CLS)
Assumed the attitudes is objective . The researcher used focus groups data to support evidence for questionnaire.

2. Students would be forced to choose their friends when given the chance to choose their groupmates.

H2. Focus groups involved students who had selected their groups

Students are facing similar quandaries as teachers in deciding who should choose groups Issues of student : In terms of responsibility, work effectiveness and task completion

-Training of students with good group skills and effective member roles, include the benefits of heterogeneity so that can construct groups in a more equitable manner.

3 4 5

3. SAMPLING
* Sampling issue must be resolved with respect to purpose of the research and in particular how are the results to be generalized to a population beyond the sample.
Article no 1 Hypothesis / Research Questions (simplified) 1. There will be a statistically significant difference between the experimental group (exposed to concept map program) with the control group (traditional teaching method) with respect to academic achievement. 2. Concept mapping as an instructional tool has a positive effect on students attitudes. Sampling Size & Method 23 students: - 10 control (Pre-test + traditional teaching + post-test) - 13 experimental (Pre-test + Inspiration CM + post- test + interview) Limitations Can the results be generalized? They are by no means conclusive for generalization purpose

Quantitative: - for comparison between two groups: minimum number of samples must be 10 Qualitative: - the minimum number is only 1 Quantitative : Missing values where N =121 How was the missing values treated?

1. Choosing ones groupmates would have a negative effect on subsequent attitudes toward choice . Questionnaire ( CLS)
2

139 students 5 science classes

LA T S S S 2 2 3 7

NA T S S S 2 2 2 5

HO TS/S S 24

2. Students would be forced to choose their friends when given the chance to choose their groupmates.

2. Focus groups involved 3 groups with students who had selected their own groups. Participants were purposively sampled

3 4 5

4. DATA ANALYSIS - Over-analyzed & Over-interpret: How to balance it off? Which method to be used must be judged on how well it inform the research purposes (answer the research questions)
* Difficulty arises in terms of generating a need to trade off between the intensive detail of hermeneustic analysis (interpretative / explanatory/ qualitative) and the extensiveness of statistical (quantitative) inference.

a. Qualitative part
no standard coding different researchers have different way to determine the coding system across data type multidirectional: possibility that analysis will reveal different dimensions for each code quantitizing code: meaning become fixed and single dimensional

b. Quantitative part
- different approaches to data analysis in treating variations
Articl e no 1 Hypothesis / Research Questions (simplified) 1. There will be a statistically significant difference between the experimental group (exposed to concept map program) with the control group (traditional teaching method) with Methods used to answer research questions H1: - Pre-test MCQs - Experimental grp: exposed to Inspiration concept mapping program - Control grp : traditional Analysis Method 1. T-test: Paired-Sample 2. CM Scoring Rubric 3. Concept Map and MCQs Correlation Assumptions Made in Selecting the Methods 1. T-test: To determine whether there has been an improvement in the mean score as a result of the intervention 2. CM Scoring Rubric:

respect to academic achievement. 2. Concept mapping as an instructional tool has a positive effect on students attitudes.

teaching - Post-test MCQs to both grps - CM scoring rubrics H2: - Interview for Exp grp students - One open-ended question, duration is not specified

4. Students Interview

To examine how many concepts have the students linked correctly 3. Concept Map and MCQs Correlation: To see content validity of both CM & MCQ whether CM scores could indicate students knowledge contents 4. Students Interview: To gather information on students reactions to CM

1. Choosing ones groupmates would have a negative effect on subsequent attitudes toward choice .

H1: - Pretest - Posttest non equivalent control group - Questionnaire ( CLS)

-A 3 ( group ) x 2 ( test ) repeated measures multivariate analysis of variance (MANOVA) -A 2 x3 gender by group Anova -A 5(class) x2 ( test) repeated measures multivariate analysis of variance

To determine whether there is a significant main effect for the pre/post test .

To explore the differences of interaction from pretest to posttest between group and gender. To explore if there were differences between classes on the question of preference regardless of treatment condition. To probe more deeply students responses on the questionnaire.

2. Students would be forced to choose their friends when given the chance to choose their groupmates.

H2: -Focus groups involved students who had selected their groups only - Use 3 structured questions

- Categorizing the responses of the participants on a flip chart according to themes .

3 4 5
the allocation of participants to groups.

THE POSTTEST ONLY DESIGN WITH NON-EQUIVALENT CONTROL GROUPS

Pretest-posttest designs are an expansion of the posttest only design with nonequivalent groups, one of the simplest methods of testing the effectiveness of an intervention. In this design, which uses two groups, one group is given the treatment and the results are gathered at the end. The control group receives no treatment, over the same period of time, but undergoes exactly the same tests. Statistical analysis can then determine if the intervention had a significant effect. One common example of this is in medicine; one group is given a medicine, whereas the control group is given none, and this allows the researchers to determine if the drug really works. This type of design, whilst commonly using two groups, can be slightly more complex. For example, if different dosages of a medicine are tested, the design can be based around multiple groups. Whilst this posttest only design does find many uses, it is limited in scope and contains many threats to validity. It is very poor at guarding against assignment bias, because the researcher knows nothing about the individual differences within the control group and how they may have affected the outcome. Even with randomization of the initial groups, this failure to address assignment bias means that the statistical power is weak. The results of such a study will always be limited in scope and, resources permitting; most researchers use a more robust design, of which pretestposttest designs are one. The posttest only design with non-equivalent groups is usually reserved for experiments performed after the fact, such as a medical researcher wishing to observe the effect of a medicine that has already been administered.

Read more: http://www.experiment-resources.com/pretest-posttest-designs.html#ixzz1qNEg07al