You are on page 1of 71

EDUCATION IN A COMPETITIVE AND GLOBALIZING WORLD

DOES ASKING PERTINENT


NON-ACADEMIC QUESTIONS
MAKE YOU A
BETTER RESEARCHER?

THE LINK BETWEEN
NON-ACADEMIC AND
RESEARCH QUESTIONS


No part of this digital document may bereproduced, stored in aretrieval systemor transmitted commercially
in any formor by any means. Thepublisher has taken reasonablecarein thepreparation of this digital
document, but makes no expressed or implied warranty of any kind and assumes no responsibility for any
errors or omissions. No liability is assumed for incidental or consequential damages in connection with or
arising out of information contained herein. This digital document is sold with theclear understanding that
thepublisher is not engaged in rendering legal, medical or any other professional services.
EDUCATION IN A COMPETITIVE
AND GLOBALIZING WORLD


Additional E-books in this series can be found on Novas website
under the E-book tab.


EDUCATION IN A COMPETITIVE AND GLOBALIZING WORLD


DOES ASKING PERTINENT
NON-ACADEMIC QUESTIONS
MAKE YOU A
BETTER RESEARCHER?

THE LINK BETWEEN
NON-ACADEMIC AND
RESEARCH QUESTIONS


MARA ANTONIA PADILLA VARGAS
UNIVERSIDAD DE GUADALAJ ARA
JAIRO TAMAYO
UNIVERSIDAD VERACRUZANA
AND
MARINA L. GONZLEZ-TORRES
UNIVERSIDAD AUTNOMA DE AGUASCALIENTES



New York

Copyright 2013 by Nova Science Publishers, Inc.

All rights reserved. No part of this book may be reproduced, stored in a retrieval
system or transmitted in any form or by any means: electronic, electrostatic,
magnetic, tape, mechanical photocopying, recording or otherwise without the written
permission of the Publisher.

For permission to use material from this book please contact us:
Telephone 631-231-7269; Fax 631-231-8175
Web Site: http://www.novapublishers.com

NOTICE TO THE READER
The Publisher has taken reasonable care in the preparation of this book, but makes no
expressed or implied warranty of any kind and assumes no responsibility for any
errors or omissions. No liability is assumed for incidental or consequential damages
in connection with or arising out of information contained in this book. The Publisher
shall not be liable for any special, consequential, or exemplary damages resulting, in
whole or in part, from the readers use of, or reliance upon, this material. Any parts of
this book based on government reports are so indicated and copyright is claimed for
those parts to the extent applicable to compilations of such works.

Independent verification should be sought for any data, advice or recommendations
contained in this book. In addition, no responsibility is assumed by the publisher for
any injury and/or damage to persons or property arising from any methods, products,
instructions, ideas or otherwise contained in this publication.

This publication is designed to provide accurate and authoritative information with
regard to the subject matter covered herein. It is sold with the clear understanding
that the Publisher is not engaged in rendering legal or any other professional services.
If legal or any other expert assistance is required, the services of a competent person
should be sought. FROM A DECLARATION OF PARTICIPANTS J OINTLY
ADOPTED BY A COMMITTEE OF THE AMERICAN BAR ASSOCIATION
AND A COMMITTEE OF PUBLISHERS.

Additional color graphics may be available in the e-book version of this book.

LIBRARY OF CONGRESS CATALOGING-IN-PUBLICATION DATA


ISBN: 978-1-62948-587-4


Published by Nova Science Publishers, Inc. New York









CONTENTS


Preface ix
Acknowledgment xi
About the Author xiii
Introduction xv
Chapter 1 Evaluating of the Relationship between
Academic and Non-Academic Questions 1
Chapter 2 Effects of Exposure to Different
Achievement Criteria on the Elaboration
of Non-Academic Questions 17
Chapter 3 Variables That Affect the Process of
Asking and J ustifying Research Questions 35
General Conclusion 47
Index 51











Approved for publication by the members of the Editorial Board of
the Centro Universitario de Ciencias Biolgicas y Agropecuarias, in the
capacity of a Scientific Committee, according to the Protocol
CINV.073/13.


Translated by:
Cristiano Valerio dos Santos














PREFACE


Answering the question of which criteria is best to identify a good
researcher is highly complex owing to the broad range of factors
involved in the scientific practice. Some criteria are institutional, while
others may be specific to a scientific community or even personal. On the
one hand, some may argue that being capable of publishing
autonomously (or at least independently of advisers) and of posing novel,
useful, viable, and pertinent research questions is the main criteria by
which a researcher should be judged. On the other hand, because
publications may result in financial benefits to the author, they are not a
good criterion to assess the academic value of a researcher. We do know
however that academic training has not received sufficient attention by
psychologists. Investigation on this topic has been conducted mainly by
sociologists and educators, using ethnographic and descriptive
methodologies. Psychological studies are scarce, fragmented and
unconnected.
Given the importance of producing and spreading knowledge, of
conducting novel research, and of training novice researchers to replace
retiring academics (in Mexico, at least, the average active scientist is
roughly 47 years old), it becomes important to analyze at least one
element that might make a good researcher: posing novel, pertinent
questions derived from a specific line of research. While in the process of
analyzing this topic, the relationship between the ability to pose research
questions and the skills involved in posing non-academic questions
became apparent, and received a more in-depth analysis. We tried to sum
up the results of these investigations and their conclusions in this book.











ACKNOWLEDGMENT

To the Academic Workers Union at the University of Guadalajara
for providing the financial aid needed to print this book.










ABOUT THE AUTHOR


Mara Antonia Padilla Vargas is a tenured professor at the University
of Guadalajara, working at the Center for Behavioral Studies and
Research (CEIC). She is a member of National Researchers Association
(SNI) and founded in 2006 the Mexican Association for Psychological
Research (SMIP), over which she has presided ever since. She has
published five books, 15 book chapters, and 26 research articles in
indexed international journals on topics related to scientific practice,
training new scientists, scientists interactions, and complex learning.














INTRODUCTION

Answering the question of which criteria are best to identify a good
researcher is highly complex owing to the broad range of factor involved
in the scientific practice. Some criteria are institutional, while others may
be specific to a scientific community or even personal. On the one hand,
some may argue that being capable of publishing autonomously (or at
least independently of advisers) and of posing novel, useful, viable, and
pertinent research questions are the main criteria by which a researcher
should be judged. On the other hand, because publications may result in
financial benefits to the author, they are not a good criterion to assess the
academic value of a researcher. We do know however that academic
training has not received sufficient attention by psychologists.
Investigation on this topic has been conducted mainly by sociologists and
educators, using ethnographic and descriptive methodologies.
Psychological studies are scarce, fragmented and unconnected.
Given the importance of producing and spreading knowledge, of
conducting novel research, and of training novice researchers to replace
retiring academics (in Mexico, at least, the average active scientist is
roughly 47 years old), it becomes important to analyze at least one
element that might make a good researcher: posing novel, pertinent
questions derived from a specific line of research. While in the process of
analyzing this topic, the relationship between the ability to pose research
questions and the skills involved in posing non-academic questions
became apparent, and received a more in-depth analysis. We tried to sum
up the results of these investigations and their conclusions in this book.
Three chapters comprise this book. In the first, we describe two
studies that were carried out with high school students. We evaluated
whether high school students were able to pose novel research questions
M. A. Padilla Vargas, J . Tamayo and M. L. Gonzlez-Torres xvi
based on technical material they had previously read (Experiment 1) or
whether they were able to pose non-academic questions without having
to master a particular line of research (Experiment 2). Results showed
that performance was poor with both types of questions and suggested
that we might need to take a step back and analyze more basic skills such
as reading comprehension and writing before exploring research
questions per se.
Given these results, in another study, described in Chapter 2, we
provided participants with differential information regarding the
achievement criteria (defined as the criteria a particular performance
must meet or to which it must adjust) when posing non-academic
questions, that is, related to daily life, for which no specific training is
required. Because high school students did not seem to be the most
appropriate population for the study of academic and no-academic
questions, we conducted this second experiment with high school,
undergraduate, and graduate students and found that more specific
achievement criteria facilitated the production of a greater number of
questions, which also were more pertinent to the daily situations we
presented. Higher-level students performed better and high-school
students again were not capable of posing non-academic questions, even
when they were given a specific achievement criterion.
Chapter 3 describes a series of studies, conducted with
undergraduate and graduate students and experienced researchers, whose
objective was to evaluate explicitly the variables that affect the
production of research questions. The standard experimental protocol
consisted of two groups (experimental and control), which were exposed
to a pre-assessment, two exercises with or without corrective training,
and a post-assessment (control groups were not given any training).
Participants were required to derive a novel and pertinent research
question from a series of empirical articles and justify it properly (pre-
and post-assessments), to identify some of their elements (Exercise 1), or
elaborate on them (Exercise 2). The empirical articles could either be
used as they were published or be modified in some of their elements to
make them unstructured, so we could evaluate whether this changes
affected performance. Results of the studies conducted so far have shown
that the performance of undergraduate and graduate students initially is
poor, but with adequate training, especially corrective, it improves
substantially from pre- to post-assessments. Experienced researchers
Introduction xvii
perform well from the start and their performance is not affected by how
unstructured the texts are, contrary to what happened with the students.
The most remarkable finding to us is that both undergraduate and
graduate students have poor reading and writing skills. Most are not used
to reading technical materials, and find it hard to interact with them (even
with corrective training, some are not able to obtain more than 60% of
correct answers). Because research questions are derived from the
comprehensive reading of empirical articles, it is understandable that this
task can be arduous and sometimes impossible for them. Moreover, when
required to justify the proposed research question, which implies writing
skills, an intimate relationship between reading and writing is observed:
1) skilled readers turn out to be skilled writers and vice versa, and 2)
improvements in one skill, brought about by training, relate to
improvements in the other skill.
With this book, we wish to help training more and better researchers.
Also, by presenting a detailed description of our methodology, we wish
to propose a way to study this topic experimentally, so we can gather
systematic data on the variables the affect the performance of
researchers, both novice and expert.








Chapter 1


EVALUATING OF THE RELATIONSHIP
BETWEEN ACADEMIC AND
NON-ACADEMIC QUESTIONS
1


The main objective of science education is to train novice
researchers to be able to work independently from their advisers and
teachers (Snchez Puentes, 2004), carrying out novel investigations that
are pertinent to the line of research under which they were trained (De La
Fuente, J usticia, Casanova & Trianes, 2005). This must include the
exercise of reading and writing skills, given that writing specialized
materials is essential to the scientific practice (Cassany, 2006). Not only
does this activity permit the diffusion of ideas and empirical evidence, it
also promotes the generation of new ideas (Keys, Hand, Prain & Collins,
1999).
Reading scientific texts is vital for conducting research. Due to its
collective nature, science requires constant literature reviews that allow
us to get in touch with what others have done in a particular line of
research, from which novel questions will be derived. This process also
contributes to the integration of the new piece of research into the
existing body of knowledge (Viniegra, 2002). However, it might be
worth mentioning the reading and understanding technical materials from
a line of research does not guarantee the derivation of pertinent
questions, as can be observed when participants perform well on a test

1
A preliminary version of the work was published in Padilla, M.A., Tamayo, J ., &
Gonzlez M.L. (2010). Anlisis de la posible relacin entre la elaboracin de
preguntas informales y de investigacin. IPyE: Psicologa y Educacin, 4, 7, 1-21.
Permission granted.
M. A. Padilla Vargas, J . Tamayo and M. L. Gonzlez-Torres 2
used to evaluate mastery of a particular domain and later fail to derive
novel and pertinent research questions from the materials they read
(Padilla, Suro, & Tamayo, 2010).
Despite its importance, the experience in research training facilities
within university contexts indicate that reading technical materials
(Canales, Morales, Arroyo, Pichardo & Pacheco, 2005; Pacheco,
Ramrez, Palestina & Salazar, 2007; Pacheco & Villa, 2005) and the
derivation of novel research questions from them are very problematic
for novice researchers (Padilla, Solrzano & Pacheco, 2009).
Because it is important to find strategies that permit novice
researchers to derive questions from a body of knowledge, we searched
for the variables related to improvements in reading and writing. We
found that specifying an achievement criterion to which the participants
had to adapt in order to solve a particular task promoted effective
performances (Tamayo, Padilla, & Gonzlez-Torres, 2009), so we
decided to explore whether manipulating this variable affected the
production of research questions derived from previously read scientific
materials.
Achievement criteria in educational contexts imply a set of
behavioral requisites specified for a concrete situation, to which an
apprentice must adapt in order to adequately solve a particular task.
Achievement criteria may be specified by the task itself, by the teacher,
or may be implicit.
Ibez (1999) investigated the correspondence between being able to
identify the criteria in a text and being competent in these criteria with a
sample of aspiring undergraduate students. Prior to a task, they were told
the type of activity they were about to do and the criterion they were
supposed to meet. After that, they were exposed to a criteria
identification task in which they were asked single out the texts that
provided the information that was necessary and sufficient to solve the
tasks. He observed that the participants that were able to correctly
identify a greater number of criteria also obtained higher scores in the
production tests.
Based on these results, Mateos and Flores (2008) claimed that,
although specifying and identifying achievement criteria were important,
so was studying the way these criteria were specified, and set out to
analyze the effects of exposing participants to different degrees of
information regarding the task to be solved. They evaluated the
Evaluating of the Relationship 3
relationship between three degrees of criterion specification (specific,
non-specific or absent) and the performance in identification and
production tasks, and found better levels of performance in both tasks
when participants were exposed to a specific criterion.
Assuming that a taxonomy of achievement criteria depending on the
type of information conveyed was needed, Ortiz, Gonzlez and Rosas
(2007) proposed a classification of instructions-descriptions that may be
given to a participant during an experimental task. This classification was
based on the amount and specificity of information that may be given
before, during and/or after solving a particular task, and may refer to
three elements: the stimulus conditions, the response conditions and the
consequence conditions. The instructions that include response
conditions also specify the achievement criterion.
Ortiz et al.s classification divides achievement criteria into clearly
defined and ordered categories, and includes, among others, the
following categories:
Pertinent and Specific Description (PS). A description that contains
all the elements of a contingency, and they correspond to the elements
that comprise the situation in hand.
Pertinent and Generic Description (PG). The description contains
only one or two of the elements of a contingency, and they correspond to
the elements that comprise the situation in hand.
Absent Description (A). The description does not mention any of the
elements (either relevant or irrelevant) of the situation in hand.
Because the specification of achievement criteria and the type of
information these criteria convey are considered relevant for a participant
to perform a task effectively, we wondered whether these aspects also
were relevant when participants were required to pose research
questions. In this sense, Ribes (1993) argued that the language games of
the scientific practice correspond to the game of posing pertinent
questions, which entails 1) posing research questions that contain
relationships between facts or events and their quantitative and/or
qualitative properties, and 2) establishing the categories that define the
pertinence of those relationships within a disciplinary framework.
We conducted an exploratory study to evaluate whether the distinct
information conveyed by the different achievement criteria affected the
production of pertinent research questions, assuming that an effective
M. A. Padilla Vargas, J . Tamayo and M. L. Gonzlez-Torres 4
behavioral adjustment might depend on the identification and
specification of these criteria.


STUDY 1

Method

Participants
Twenty-four high school students, 20 females and four males, were
randomly assigned to one of three groups (n=8). This population was
selected in order to reduce the probability that they were familiar with
the particular theoretical domain or with psychology in general.

Design

Table 1. Experimental design

Conditions
Groups
Exposure and
Evaluation 1
Exposure and Evaluation 2
Evaluation
3

Group 1
N =8
1) Read
area
x or
y

2) Eval.
1) Read
paper
x or
y

2) Ev.
of paper
Gen.
Specific
criteria
1)
Read
paper
y or
x

2) Ev.
of
paper
Gen.
Identificati
on
assessment

Group 2
N =8
General
criteria

Group 3
N =8
Minimal
criteria
Note: Read area x or y =read a document about the assumptions of an area
(animal or human behavior); Eval. =domain evaluation of the assumptions
read; Read paper. x or y =read a paper on animal or human behavior;
Ev. of paper. =domain evaluation of the paper read; Gen. =generate a
research question; Identification assessment =identification of the elements
of a research paper.

The criteria used varied in the type of information they specified: 1)
a specific criterion gave a detailed description of the task the participant
had to perform (in this case, posing a research question derived from the
line of research they had previously read, which had to include the
Evaluating of the Relationship 5
dependent and independent variables explicitly; 2) a general criterion
only specified that the question had to be derived from the line of
research they had previously read; and 3) the minimal criterion only
specified that they had to pose a question derived from the line of
research they had previously read, without further details.

Setting
The study was carried out in the high school classrooms where the
students had classes. Students had their own working area and performed
the task individually.

Materials
Conceptual framework of the research area (basic research on animal
behavior or applied behavior analysis with humans), which contained the
conceptual assumptions of each area (Skinnerian behaviorism. The order
of exposure to these areas was counterbalanced within groups); 2) a
glossary of technical terms of each area; 3) a reading comprehension test
about the conceptual frameworks with items that required not only
repetition but also elaboration of the material; 4) instructions with the
corresponding criterion depending on the group; 5) two research papers,
one on animal behavior and another on human behavior. The paper on
animal behavior compared the number of responses emitted by four rats
under continuous or intermittent reinforcement. The paper on human
behavior analyzed the effects of reinforcing child social behavior while
children interacted with their peers at school. Although both papers
contained technical terms, we looked for papers whose style was as
simple and clear as possible. They were three pages long and did not
contain any terms that were not in the glossary or in the conceptual
framework; 6) a reading comprehension test about the papers, with open-
ended questions; 7) instructions for the identification task. In this task,
participants were required to underline the following elements of one of
the research papers: main terms and concepts, reported evidence,
research justification and objective, research question, observed and
manipulated variables, type of subjects, procedure, data, and conclusions;
8) in addition, there were pencils, erasers, pencil sharpeners, and blank
sheets of paper.


M. A. Padilla Vargas, J . Tamayo and M. L. Gonzlez-Torres 6
Procedure
Participants were individually handed the conceptual framework and
the glossary for the first area (human or animal behavior). After they
finished reading the material, they were given the reading comprehension
test about the conceptual framework material, which was taken away.
When the participant finished the test, the experimenter immediately
scored it.
A passing grade corresponded to 90% correct answers. If this
criterion was not met, they were given feedback about their mistakes and
were asked to correct them using the material they read. Students could
not pass on to the following condition unless this criterion was met.
After that, they were given the instructions to the first test with a
minimal criterion, which said they had to pose a research question. In
addition, they were handed the first paper of the area corresponding to
the conceptual framework they had read. After they finished reading the
paper, they were given a reading comprehension test about the paper,
with open-ended questions.
Again, 90% of correct answers were needed to pass on to the next
condition, and they were given corrective feedback. Afterwards, they
were required to pose a research question related to the paper they read.
Having finished this task, participants were allowed to take a five-minute
break.
Before the second test, they were given the instructions that
corresponded to the group to which they were assigned. At the same
time, they were handed the second paper, which was directly derived
from the first. After they finished reading the paper, they were given a
reading comprehension test about the paper, with open-ended questions.
Again, 90% of correct answers were needed to pass on to the next
condition, and they were given corrective feedback.
Afterwards, they were required to pose a research question related to
the paper they read. Finally, they were exposed to the identification task,
in which they were asked to underline the elements of the last paper they
read. This study was conducted in a single session, in order to prevent
participants from talking amongst themselves about the activities and
possibly biasing the results.



Evaluating of the Relationship 7
Results
2


Each research question posed in tests 1 and 2 was evaluated
according to the classification system established by Padilla, Solrzano
and Pacheco (2009), which include three categories. The first includes
proposals to manipulate the same variables present in the papers,
changing only their values. The second category can be divided into
three subcategories: a) proposals to evaluate correlations between
variables not previously found in the literature they reviewed; b)
proposals to manipulate variables not previously found in the literature
they reviewed; and c) proposals to apply the theory they review to solve
other theoretical or technological problems. The third category includes
questions in which an experimental design is proposed that permits the
validation of the theory they reviewed. This classification system implies
an increasing level of complexity, where the first type of questions are
simpler and third type are more complex. In addition, the number of
elements correctly identified in the research papers was computed.
Figure 1 shows a comparison between the types of question posed in
the first and second tests. Only four participants (1, 7, 14, and 19)
showed any improvement from the first to the second test. However, the
questions posed in the second test were less complex. Only one
participant (19) posed a question at a level of complexity 2c in the
second test. The other participants either did not show an improvement
or did not pose any questions in neither test. There seemed to be no effect
of the achievement criterion, since performance was poor regardless of
the achievement criterion to which they were exposed.
Furthermore, there was no correlation between posing pertinent
questions and being able to correctly identify the elements that comprise
a research paper. Although most participants were able to correctly
identify the elements that comprised the research paper they read, they
still were not able to pose a pertinent question.



2
Participants answers were analyzed according to the criteria established in the systemof
conceptual and thematic content analysis (Raigada, 2002). In each answer, the
explicit and implicit terms were identified. The analysis was conducted by two
independent judges, with a mean reliability of 98.22% (%Reliability =1 [(n1-n2) /
(n1+n2)]*100).
M. A. Padilla Vargas, J . Tamayo and M. L. Gonzlez-Torres 8

Figure 1. Comparison between type of questions generated in Evaluations 1 and
2 by the participants in the three experimental groups.

Discussion

Several factors may account for these results. First, the population
might not be adequate, because participants anecdotally reported the low
reading levels when asked about it. Additionally, results of the reading
comprehension test were generally poor. All participants had to correct
their answers at least once for one or more questions, which is suggestive
of poor reading comprehension. Pacheco et al. (2007) suggested that
reading comprehension might be a basic competence needed to
effectively derive research questions, because it allows participants to
establish contact with the referents presented in the text, from which they
are supposed to derive their research questions. Second, materials may
have been too difficult for this type of population, since technical terms
from experimental psychology, with which they were not familiar, were
present in the texts.
In other words, high school students were not able to pose research
questions derived from a specific domain, and we wondered whether this
difficulty was limited to research questions or was more pervasive and
extended to a difficulty to pose pertinent questions in informal or non-
academic questions. Posing pertinent questions in non-academic settings
might be a precursor skill, and Pacheco et al. (2007) argued that using
referents within the ordinary language domain is vital for the study of
Evaluating of the Relationship 9
reading and writing skills, since the contact with such referents is
facilitated.
With that in mind, a second study was carried out, in which we
analyzed the type of questions posed by the participants of the previous
study in non-academic settings, in order to identify a possible
relationship between posing informal questions and posing research
questions. We found necessary to assess whether these high school
students were not able to pose research questions because they were
unable to pose pertinent questions in general.


STUDY 2

Method

Participants
Twenty-two of the twenty-four students from the previous
experiments (several months had passed between studies, so it was not
possible to locate two participants).

Setting
Same as the previous study. In this case, the tasks also were
performed individually.

Materials
Participants were given a written paper with four hypothetical
situations, from which they were requested to obtain a specific type of
information. These situations took place in an airport with a character
that was visiting a foreign destination for the first time. The information
they needed to obtain included the baggage claim site (situation 1), a
particular immigration proceeding (situation 2), the location of a hotel
and the more appropriate means of transportation to get there (situation
3), and a place to exchange currency (situation 4). The participants were
asked to imagine themselves in those situations and to pose a question
that was pertinent to each of them. They were told that the question had
to allow them to get all the information needed to solve the situation
adequately. In addition, there were pencils, erasers, pencil sharpeners,
and blank sheets of paper.
M. A. Padilla Vargas, J . Tamayo and M. L. Gonzlez-Torres 10
Procedure
The activity took place at the end of a class and was conducted by
the teacher in charge of that class.


Results
3


As can be seen in Table 2, most participants scored low when asked
to pose the questions pertinent to the four hypothetical situations. The
scores used to grade the questions were assigned according to the
following criteria: 0 when there was no question, 1 when the question
was neither grammatical nor pertinent, 2 when the question was
grammatical but not pertinent, and 3 when the question was both
grammatical and pertinent. Only some participants were able to pose
grammatical and pertinent non-academic questions in all four
hypothetical situations (P17 and 18), or in most of them (P4, 6, and 12).

Table 2. Score obtain by participants in each one of the four
hypothetical cases

Participants
Type of question generated
Case 1 Case 2 Case 3 Case 4
1 1 0 1 0
2 0 0 0 0
3 0 0 0 0
4 3 2 3 3
5 0 0 0 0
6 3 2 3 3
7 0 0 0 0
8 0 0 0 0
9 0 0 0 0
10 0 0 0 0
11 0 0 0 0
12 3 2 3 3
13 0 0 0 0

3
Participants answers were analyzed according to the criteria established in the systemof
conceptual and thematic content analysis (Raigada, 2002). In each answer, the
explicit and implicit terms were identified. The analysis was conducted by two
independent judges, with a mean reliability of 98.22% (%Reliability =1 [(n1-n2) /
(n1+n2)]*100).
Evaluating of the Relationship 11
Participants
Type of question generated
Case 1 Case 2 Case 3 Case 4
14 3 2 3 0
15 0 0 0 0
16 2 2 2 3
17 3 3 3 3
18 3 3 3 3
19 3 2 0 0
20 0 0 0 0
21 0 0 0 0
22 0 0 0 0

To assess the relationship between non-academic and research
questions, one of the four situations was randomly selected (situation 3)
and the results with this situation were correlated with the results of the
second evaluation of the first study, yielding a correlation index of r =
0.385. That is, there was not a strong correlation between non-academic
and research questions.


Discussion

The results from both studies, taken together, suggest that specifying
the achievement criterion may not be sufficient to make high school
students pose novel, pertinent research questions. In addition, these same
participants were unable to non-academic questions in an informal
setting.
What could account for these findings? We may put forward a
couple of hypotheses. Pacheco et al. (2007) have argued that the
referents
4
to which participants are exposed may affect their reading and
writing skills, and that referents belonging to the ordinary language may
facilitate performance.
With that in mind, the absence of correlation between the production
of non-academic and research questions might be related to the lack of
familiarity with the hypothetical situations we created, which reduced
our participants capacity to pose pertinent questions. So, it is necessary

4
Referent refers to objects, people, events, situations, etc., about which one reads, writes
or talks.
M. A. Padilla Vargas, J . Tamayo and M. L. Gonzlez-Torres 12
to evaluate and control, as much as possible, our reader-writers history
of contact with the referents to which they are exposed.
The history of contact with the referents is an element in a set of
factors that intervene or modulate reading and writing interactions
(others being the achievement criteria, dispositional characteristics, skills
and competences, etc). This history is divided into the history of
referentiality and effective situational history, the latter referring to the
number of direct and indirect contacts with the referent. Both factors are
dispositional elements that regulate the characteristics of the interactions
that may occur between the subject and the events (Pacheco et al. 2007).
The characteristics of the referent, in turn, are related to the nature and
the domain of the event about which one is talking, reading or writing.
Thus, the domain may be related to the ordinary language, technical
language, artistic language, etc.
However, using referents that belong to the ordinary language may
not ensure an effective interaction with them, since such interaction is
affected by the substitutive contacts the reader-writer had in their history
of referentiality with the concrete event to which they are exposed at a
given time. That might explain the low scores observed in the second
study. Given that participants did not have direct experience with the
situation and its referents, their performances could not meet the
achievement criterion, that is, they could not pose pertinent questions.
In an attempt to investigate whether their poor performance when
posing questions about the four hypothetical situations was due to their
lack of familiarity with the situations, we replicated the study with
students from the same population (not reported here due to lack of
space). We used a new hypothetical situation with elements we deemed
more familiar to the participants (Imagine you are fond of a boy or girl
in your classroom and want to get to know him or her better. Write down
what youd ask them to find out: a) what kind of music they like, b)
whether they want to be your friend, and c) whether they want to go out
with you some time). Results were generally similar to those reported in
the second study: most were unable to pose grammatical and pertinent
questions.
If we add to that the requirement to read technical materials and
write down research questions based on them, results from the first study
are understandable, because the materials used referred to animal and
human behavior. Although we looked for papers whose style was as
Evaluating of the Relationship 13
simple and clear as possible, the inclusion of technical terms may have
made them too abstract and complex for the participants. That is, because
the participants had not been exposed to similar referents throughout
their history, coupled with the novelty of the situation and the technical
elements of the materials, it is not surprising that they could not
adequately interact with the reading material and meet the criterion we
established for posing derived research questions.
It seems, therefore, that both the population was not suitable for the
study because of their low levels of reading and writing and the materials
we used may have been too complex for these participants. So, both the
history of contact with the referent and the type of referent might hinder
an effective adjustment to the task. These elements should be considered
and possibly controlled in future studies.
The need to consider the history of contact and the type of referent is
based on the assumption that the act of posing research questions is
related to problems of reading and writing adjustments. Thus, future
studies designed to analyze the production of research questions may
have to consider reading and writing adjustments as basic competences.
Only when a reader understands the material to which they were
exposed, will they be able to write down pertinent research questions.
So, the first step will have to be to ensure that participants make
effective contact with the critical referents of the texts to which they are
exposed. In the case of research reports, these elements may be the
research question and its justification, the dependent and independent
variables, as well as the evidence that gives support to it. Although a
reader has to adjust to the text as a whole, a systematic analysis has to
start from the consideration and evaluation of these separate elements.


REFERENCES

Canales, C., Morales, G., Arroyo, R., Pichardo, A., & Pacheco, V.
(2005). Anlisis Interconductual del ajuste lector en el mbito
educativo. En C. Carpio y J .J . Irigoyen (Eds.) Psicologa y
educacin. Aportaciones desde la teora de la conducta (pp. 33-50).
Mxico: UNAM.
Cassany, D. (2006). Taller de textos. Leer, escribir y comentar en el
aula. Espaa: Paids.
M. A. Padilla Vargas, J . Tamayo and M. L. Gonzlez-Torres 14
De la Fuente, J ., J usticia, F., Casanova, P. F., & Trianes, M.V. (2005).
Perceptions about the construction of academic and professional
competencies in psychologist. Electronic Journal of Research in
Educational Psychology, 3,1, 3-34.
Ibez, C. (1999). Conducta de estudio: el papel de identificar criterios
en el discurso didctico. Acta Comportamentalia, 7, 47-66.
Keys, C. W., Hand, B., Prain, V., & Collins, S. (1999). Using the science
writing heuristic as a tool for learning from laboratory investigations
in secondary science. Journal of Research in Science Teaching, 36,
10, 1065-1084.
Mateos, R., & Flores, C. (2008). Efectos de evaluar el grado de
explicitacin del criterio de ajuste sobre el desempeo de estudiantes
en tareas de identificacin y elaboracin. Acta Comportamentalia,
16, 1, 73-88.
Ortiz, G., Gonzlez, A., & Rosas, M. (2008). Una taxonoma para el
anlisis de descripciones pre y post contacto con arreglos
contingenciales. Acta Colombiana de Psicologa. 11, 1, 45-53.
Pacheco, V., & Villa, J . C. (2005). El comportamiento del escritor y la
produccin de textos cientficos. Revista Mexicana de Investigacin
Educativa, 10, 27, 1201-1224.
Pacheco, V., Ramrez, L., Palestina, L., & Salazar, M. (2007). Una
aproximacin al anlisis funcional de la relacin entre las conductas
de leer y escribir en estudiantes de psicologa. En J .J . Irigoyen, M.
J imnez y K. Acua (Eds.) Enseanza, aprendizaje y evaluacin en
psicologa. Una aproximacin a la pedagoga de las ciencias.
Hermosillo: UniSon.
Padilla, M. A., Solrzano, W. G., & Pacheco, V. (2009). The effects of
text analysis on drafting and justifying research questions. Electronic
Journal of Research in Educational Psychology, 17, 7, 1, 77-102.
Padilla, M.A., Suro, A.L. y Tamayo, J . (2010). Efectos de la exposicin
diferencial a los supuestos de una teora en la elaboracin de
preguntas de investigacin. Revista Mexicana de Psicologa, 27, 2,
247-256.
Raigada, J . L. (2002). Epistemologa, metodologa y tcnicas del anlisis
de contenido. Estudios de Sociolingstica, 3, 1, 1-42.
Ribes, E. (1993). La prctica de la investigacin cientfica y la nocin de
juegos de lenguaje. Acta Comportamentalia, 1, 1, 63-82.
Evaluating of the Relationship 15
Snchez Puentes, R. (2004). Ensear a investigar. Una didctica nueva
de la investigacin en ciencias sociales y humanas. Mxico: Plaza y
Valds.
Tamayo, J ., Padilla, M. A., & Gonzlez-Torres, M. L. (2009). Efectos de
criterios de logro diferenciales en la elaboracin de preguntas
informales, en estudiantes de preparatoria, licenciatura y posgrado.
Acta Colombiana de Psicologa, 12, 1, 27-39.
Viniegra, L. (2002). Un acercamiento a la crtica. Educacin y Crtica:
el proceso de elaboracin del conocimiento. Mxico: Paids.








Chapter 2



EFFECTS OF EXPOSURE TO
DIFFERENT ACHIEVEMENT CRITERIA
ON THE ELABORATION OF
NON-ACADEMIC QUESTIONS
5


Asking pertinent and grammatical questions is one of the daily tools
we can use to obtain information, to clarify a topic, to obtain knowledge
or to solve problems in general. When one faces a situation in which
there is an imbalance between what is known and what exists, a scenario
with anomalies, contradictions, and obstacles emerges, which leads to
questioning and the posing of questions in order to obtain answers that
might help clarify ideas, adjust behavior or carry out specific tasks
appropriately (Graesser & Olde, 2003). In a daily basis, humans are
faced with situations in which they must ask questions that are pertinent
to the situations.
In an educational setting, asking questions seems to be a key element
to complete a comprehensive learning process. Being able to ask good
questions may promote strategies to a better understanding of texts and
tasks, which may lead to better academic outcomes (King, 1991). Asking
questions is arguably related to the learning process, since it implies the
active participation of the student, who is not limited to receiving
information passed on by the teacher, and also allows the students to

5
A preliminary version of the work was published in Tamayo, J ., Padilla, M.A., &
Gonzlez-Torres, M.L. (2009). Efectos de criterios de logro diferenciales en la
elaboracin de preguntas informales, en estudiantes de preparatoria, licenciatura y
posgrado. Acta Colombiana de Psicologa, 12, 1, 27-39. Permission granted.
M. A. Padilla Vargas, J . Tamayo and M. L. Gonzlez-Torres 18
question and identify by themselves the flaws and conceptual voids that
they need to fill (Graesser & Olde, 2003).
In order to ask pertinent questions, one must attend to a large number
of elements present in the situations where those questions take place.
Identifying these different elements in a situation is vital for asking
adequate and pertinent questions. In an academic setting, this translates
into students that question the world and develop a critical attitude
towards the information they receive.
The studies that have been conducted to analyze individual processes
and contextual elements involved in asking questions in an academic
setting have been carried out within a cognitive framework. These
studies focus on the evaluation of strategies and techniques that allow
students to ask adequate questions or on the identification of the effects
that such strategies have on other academic activities.
For instance, Davey and McBride (1986) reported significant
improvement in the understanding and recollection of read materials after
training sixth-graders to ask technical questions. Also, they found that
the same kind of training promoted the production of more and more
pertinent questions
6
. In a similar vein, King (1991) found an
improvement in the typical classroom tasks, such as spatial reasoning,
when student were trained to ask questions classified in terms of
planning, monitoring, and evaluation.
Although asking questions, or being trained to ask questions, led to
an improvement in academic performance, it became apparent in these
studies that students ask very few questions spontaneously. Ninety-four
percent of the questions are asked by the teacher. When a student asks
questions, they usually lack depth, analysis, and inference (Graesser &
McMahen, 1993).
Tutorial programs have been shown to improve the quality and
increase the number of questions asked by 18% (Graesser, Person &
Huber, cited by Graesser & McMahen, 1993). Nevertheless, the
questions asked in these programs are usually based on literal
information that is explicitly present in the texts, and not on the
elaboration of inferences, relationships, and syntheses. It has been argued
that these deficiencies occur because students fail to understand technical
material. These failures consist of problems in identifying the critical

6
These authors do not make a clear theoretical or methodological distinction between
making a question and training to make a question.
Effects of Exposure to Different Achievement Criteria 19
elements that are relevant and not easily abstracted from the texts
(Graesser & McMahen, 1993).
However, teaching strategies also arguably affect the number and
quality of questions asked. In some cases, a teacher may allow students
to question the contents of what they are learning, whereas in other cases
a teacher may organize a class in such a way that questions are reduced
to a minimum, and the student becomes a mere passive agent (Graesser
& Olde, 2003). Thus, an educational context that either allows or not the
posing of questions may differentially affect this activity (King &
Rosenshine, 1993).
In a similar vein, Freire (1985) claimed that education has been
structured around what he called the pedagogy of the response. In this
model, the teacher is conceived as the center of knowledge, leading to a
mechanical process of passing on information, and students are passive
agents of the educational system, that is, they do not question that
knowledge; they only receive and retain the information passed on by the
teacher. In this sense, education is a mechanical process that deforms
students.
Freire (1985) proposed the abandonment of the pedagogy of the
response in favor of its counterpart, the pedagogy of the question. In
this model, the teacher is a facilitator in the development of active and
critical individuals. Under this assumption, education is not reduced to
the reproduction of mechanical methods centered on the teacher; it is
based on the interaction between teachers and students, on questioning,
and on critical thinking, which may promote social change in the short
and long term.
As a consequence, asking questions in different contexts is arguably
a topic that requires further analysis. However, research should not
center on the process of asking questions per se, but also on the type of
questions asked. Graesser and Olde (2003) found that student that scored
higher on comprehensions tests ask fewer questions, but these were more
concrete and pertinent, which permitted obtaining key pieces of
information. Conversely, students that scored lower asked more
questions, but they were less pertinent, deep, and specific, lacking the
critical elements of the situation.
M. A. Padilla Vargas, J . Tamayo and M. L. Gonzlez-Torres 20
It seems then that some dispositional
7
factors differentially affect the
production of questions and their functionality in different contexts. One
of such elements may be the achievement criterion. Effective
performance is indicative of an adjustment to a criterion required to solve
a particular task, that is, adjustment to a criterion is evidenced in an
apprentices effective performance.
Given that specification of the achievement criteria seems to affect
performance in a number of tasks, it may also affect the production of
questions, both in academic and non-academic contexts. One may
suppose the more clearly specified is the achievement criterion, the more
pertinent to the situations the questions asked will be. Thus, in the
present study, we attempted to evaluate the effect of the specificity of the
achievement criterion on the production of non-academic questions.
Non-academic questions are questions related to daily life, for which no
specific training in a particular area of knowledge is required.
Asking non-academic questions may be an important antecedent
8
to
the production of scientific and research questions. In the previous
chapter, we found that high school students were unable to ask pertinent
research questions even after being exposed to training in a particular
theoretical domain. This result led to the question of whether asking non-
academic questions was a pre-requisite skill necessary for asking
pertinent research questions, which meant moving back the starting point
for the research on this phenomenon, and conducting a parametric
evaluation of the production of questions in general. So, we exposed
those same participants to a task of asking pertinent questions that were
sufficient to obtain some piece of information required to solve a daily
task. Again, they were unable to succeed in the task (Padilla, Tamayo &
Gonzlez-Torres, 2007). These results suggested we needed to explore
the task of asking non-academic questions before studying the process of
asking research questions.

7
Dispositional factors are the set of events that affect an interaction so as to facilitate it or
hinder it, that is, that make it more or less likely (Ribes & Lpez, 1985).
8
Considering the absence of experimental evidence in this sense, we cannot argue that
being able to ask non academic questions is a pre-requisite to making research
questions. Neither can we say that the latter types of questions are caused or
automatically derived fromthe former type. We can say, however, that making non
academic questions precedes the production of research questions froma temporal
standpoint. More empirical evidence would be necessary to make causal claims.
Effects of Exposure to Different Achievement Criteria 21
As a consequence, we designed an experimental protocol to identify
the effect of providing differential information regarding the
achievement criteria on the process of asking non academic questions.
We decided to compare the performances of high-school, undergraduate,
and graduate students, because higher-level students usually ask more
questions (Graesser & McMahen, 1993; Graesser, Person & Huber, c.p.
Graesser & McMahen, 1993).
The main interest of this experiment was to analyze individual
performances. To do so, we designed an experimental protocol that
permitted to identify the effects of our main variable, the achievement
criteria, on the performance of individual participants. This type of
analysis is more compatible with the assumptions of interbehavioral
psychology, whose main subject is the interaction between an individual
and the objects and properties of their environment (Kantor, 1924-1926;
Ribes & Lpez, 1985).


METHOD
9


Participants

Twenty high-school students, 20 undergraduates majoring in
Psychology, and 20 graduate students from a Psychology program took
part in this study. High school and undergraduate students received extra
credit for taking part in the study. Graduate students did not receive any
form of compensation for taking part in the study.


Design

Participants were assigned to four different experimental groups
(n=5), exposed to two phases (see Table 3). During the first phase, they
were exposed to a hypothetical situation and a minimal achievement
criterion. This was used as a pre-assessment (or Baseline) in order to
identify the types of questions participants asked before introducing the

9
The studies described in chapter 1 and in this chapter received a grant fromCONACYT,
under protocol number 46262-H.
M. A. Padilla Vargas, J . Tamayo and M. L. Gonzlez-Torres 22
independent variable. During the second phase, each group was exposed
to a different achievement criterion and to a second hypothetical
situation. Participants were required to write down as many questions as
were necessary and pertinent to the second hypothetical situation,
according to the achievement criterion to which they had been exposed.
Questions asked during the first and second phase were compared.

Table 3. Experimental design

Groups
Pre-assessment Baseline

Phase 1
Hypotetical case 1
Evaluation with specificity of criteria

Phase 2
Hypotetical case 2
G1
N =5
Minimal criteria of
adjustment
Minimal criteria of adjustment (MC)
G2
N =5
General criteria of adjustment (GC)
G3
N =5
Specific criteria of adjustment (SC)
G4
N =5
Specific criteria of adjustment with
example (SCE)


Setting

High-school and undergraduate students were exposed to the task in
groups within classrooms, but they solved it individually. Graduate
students were exposed to the task individually in separate cubicles.


Materials

Participants were provided with pencils, erasers, blank sheets of
paper, and sheets of paper with the hypothetical situations used in this
protocol. The first hypothetical situation was presented along with a
minimal achievement criterion. In this situation, participants read the
following story: Imagine that a new teacher is going to give you classes.
You missed the first day of classes when the syllabus for the semester
Effects of Exposure to Different Achievement Criteria 23
was presented and explained, so you need to ask a few questions about
it.
In the second phase, all participants were presented with a different
hypothetical situation and the achievement criterion that corresponded to
the group to which they belonged. The group with minimal achievement
criterion was exposed to the following story: Imagine youre at an
appointment with a physician, because youve been feeling ill all week.
He orders a blood test and prescribes some medication. Youre interested
in knowing your case better and ask him some questions. Your task is to
ask the corresponding questions.
For the groups exposed to the other achievement criterion, the story
was the same except for the last sentence. The general achievement
group (GM) was told: Your task is to ask questions about the
diagnostics (situation 1), the tests you have to undergo (situation 2), and
the medication that the doctor prescribed (situation 3). Questions should
be written in a coherent manner.
The specific achievement criterion group was told: Your task is to
ask the corresponding questions. Questions should contain all the
necessary elements and be written in a coherent manner. Write down the
questions you would ask about the diagnostics (a blank space was
provided so that participants could write down the questions). Write
down the questions you would ask about the tests you have to undergo (a
blank space was provided so that participants could write down the
questions). Write down the questions you would ask about the
medication the doctor prescribed (a blank space was provided so that
participants could write down the questions).
The specific achievement criterion group with example was given
the same instructions as the specific achievement criterion group, but an
example of a question asked appropriately was given. The example was
not related to the domain of the second hypothetical situation.


Procedure

Each participant sat at an individual desk (high school and
undergraduate students) or in a separate cubicle (graduate students). We
thanked them for their participation and explained that the task was
supposed to be solved individually. Then each participant was handed a
M. A. Padilla Vargas, J . Tamayo and M. L. Gonzlez-Torres 24
sheet of paper with the first hypothetical situation and was asked to call
the experimenter when they had finished the task. At this moment, the
experimenter grabbed the sheet with the first situation and handed the
sheet with the second situation and the achievement criterion that
corresponded to the group.
When participants finished solving the second task, the experimenter
took the sheet of paper and thanked them again for their participation.


Results
10


In both phases of the study, the number, pertinence, sufficiency, and
structure of the questions asked by the participants were given a score.
Appendix A shows some examples of the questions asked by the
participants and their respective scores. The number of questions was
defined as the total number of questions asked in each situation. The
pertinence was defined as the correspondence between the question and
the situation under which the question was asked. Questions were given a
score of 1 when they corresponded to the situation and were necessary to
obtain the required information, and a score of 0 otherwise.
The sufficiency was defined in terms of whether the whole set of
questions asked by the participants permitted to obtain all the necessary
information in each case. The whole set of questions was given a score of
1 when it was deemed sufficient to obtain all the information needed, and
a score of 0 otherwise.
Finally, the structure refers to whether questions were grammatical.
We assigned them different scores based on the presence (or absence) of
syntactical elements needed to properly ask a question, such as the
question mark or spelling errors, among others. Questions were given a
score of 2 when they were grammatical, 1 when there was some mistake,
and 0 when questions were not structured.
Within-subject and between-group comparisons of the questions
asked in both phases were conducted. Figure 2 shows the number of

10
Participants answers were analyzed according to the criteria established in the systemof
conceptual and thematic content analysis (Raigada, 2002). In each answer, the
explicit and implicit terms were identified. The analysis was conducted by two
independent judges, with a mean reliability of 97.43% (%Reliability =1 [(n1-n2) /
(n1+n2)]*100).
Effects of Exposure to Different Achievement Criteria 25
questions asked by the participants belonging to the three populations.
There appears to be an effect of the specificity of the achievement criteria
in all three populations. Both groups with specific criteria showed an
increase in the number of questions asked when the two phases are
compared, whereas groups with minimal or general criteria did not show
a marked difference between phases. The differences among groups,
however, did not reach statistical significance. Differences between the
populations, on the other hand, were significant, both the in first (F =
4.074, p =0.022) and the second phase (F =6.616, p =0.003).


Figure 2. Comparison between the number of questions generated in phases 1
and 2 by the participants of the three populations evaluated. Left: high school;
middle: college; right: postgraduate. It also shows the type of criteria that were
exposed to the different groups. MC: Minimal Criteria; GC: General Criteria;
SC: Specific Criteria; SCE: Cpecific Criteria with Example.
Data for pertinence were more variable. For high-school students,
there seems to be an effect of the specificity of the achievement criteria,
because questions asked by groups exposed to general, specific and
specific with example achievement criteria were more pertinent in the
second phase, whereas the group exposed to minimal criterion shows no
difference between phases.
For undergraduate students, minimal and general achievement
criteria brought a slight increase in the pertinence of questions in the
second phase, whereas groups exposed to specific and specific with
examples showed either no difference between phases or a slight
decrease. For graduate students, there was no difference between phases
or among groups, possibly due to a ceiling effect (see Figure 3).
Differences that might have been related to the achievement criteria,
M. A. Padilla Vargas, J . Tamayo and M. L. Gonzlez-Torres 26
however, were not statistically significant. Again, differences between
the populations, on the other hand, were significant, both the in first (F =
8.186, p =0.01) and the second phase (F =3.53, p =0.036). Graduate
students asked more pertinent questions than undergraduates, who in turn
asked more pertinent questions than high-school students.


Figure 3. Comparison between the percentage of questions generated by group
and phase. Left: high school; middle: college; right: posgradute. MC: Minimal
Criteria; GC: General Criteria; SC: Specific Criteria; SCE: Specific Criteria with
Example.


Figure 4. Comparisons between the number of subjects per group generated
sufficient questions in relation to the hypothetical case. Phase 1: hypothetical
case related to class rules; phase 2 was divided in hypothetical case 1, related to
medical diagnostic; hypothetical case 2, related to exams and medical test; and
hypothetical case 3, related to medication use. Left: high school; middle: college;
rigth: posgraduate. MC: Minimal Criteria; GC: General Criteria; SC: Specific
Criteria; SCE: Specific Criteria with Example.
Effects of Exposure to Different Achievement Criteria 27
Figure 4 shows the number of participants that asked sufficient
questions. For high-school students, there seemed to be an effect of the
achievement criteria when phases 1 and 2 are compared. The effect
seems clearer for the group exposed to a specific criterion with example,
whose participants asked more sufficient questions for all three cases in
the second phase. Lower scores were observed for the second case.
For undergraduate students, there also seemed to be an effect of the
achievement criteria on the sufficiency of the questions asked when
phases 1 and 2 are compared: a more specific achievement criterion was
related to more sufficient questions. For graduate students, the number of
participants that asked sufficient questions was lower with minimal and
general criteria. For the group exposed to the general criterion, the
number of participants that asked sufficient questions was low in the first
phase and variable in the second, whereas this measure declined from
phase 1 to phase 2 for the group exposed to a minimal criterion. Again,
differences that might have been related to the achievement criteria,
however, were not statistically significant.
A comparison between the different populations showed that levels
of sufficiency for undergraduate and graduate students were higher in the
first phase, whereas most participants could ask sufficient questions
regardless of the population to which they belonged in the second phase,
especially related to the first case. With the other cases, undergraduates
and graduate students scored higher than high-school students, especially
those from the groups exposed to specific criteria. These differences did
not reach statistical significance.
Finally, Figure 5 shows the number of structural errors made. High-
school students made more structural mistakes (spelling errors, missing
question marks, etc.), followed by unstructured questions (sentences not
phrased as a question) and correct questions to a lesser degree.
Undergraduate and graduate students asked more questions with the
correct structure, followed by questions with an incorrect structure and
unstructured questions to a lesser degree. No differences were observed
regarding the specificity of the achievement criterion. A statistical
analysis showed that high-school students asked more incorrect or
unstructured questions in the first phase, but no in the second phase,
which suggests that the mere exposure to the task regardless of the
achievement criteria made the populations more homogeneous as far as
the structure of the questions is concerned.
M. A. Padilla Vargas, J . Tamayo and M. L. Gonzlez-Torres 28

Figure 5. Percentage and type of errors made by the participants in phases 1(P1)
and 2 (P2). Left: high school; middle: college; rigth: posgraduate. MC: Minimal
Criteria; GC: General Criteria; SC: Specific Criteria; SCE: Specific Criteria with
Example.

DISCUSSION

In this study, we investigated the effect of providing participants
with differential information regarding the achievement criterion in the
task of asking non-academic questions. Although there were some
differences among groups regarding the variables we evaluated (number
of questions, pertinence, sufficiency, and structure), the differences that
might have been attributed to the achievement criteria were not
significant.
In the first phase, participants were given minimal instructions as to
how they should ask the questions. In the second phase, different groups
received different instructions that varied according the specificity of the
information that was provided. However, participants given more
specific information did not perform any better than participants given
less specific information. These results differ from those obtained by
Ibez (1999) and Mateos and Flores (2008), who found that differential
achievement criteria and the type of information conveyed in them
affected the way their participant solved a task.
On the other hand, the scores for the number of questions asked,
pertinence, and structure of the questions asked by graduate students
were higher than the scores for the other populations. These differences
Effects of Exposure to Different Achievement Criteria 29
were significant in both the first and the second phase, except for the
score for structure, which did not differ among groups in the second
phase. This result suggests that the mere exposure to a criterion,
regardless of the amount of information conveyed in it, made the scores
more homogeneous among populations in the second phase.
Some tentative hypotheses may be put forward to explain this last
finding. First, academic experience seems to improve the ability to
structure and ask questions. Graesser and McMahen (1993) and Graesser,
Person and Huber, (as cited in Graesser & McMahen, 1993) reported that
students at higher levels of education, especially those under a tutorial
program, usually ask more questions than those in high school. In
addition, the quality of the questions asked also was superior, although
they revolved around the materials students read, and did not transcend
to other situations analyzed during tutoring sessions. So, graduate
programs, which usually entail a more direct relationship between
student and advisor, may promote inquiry, skepticism, and critical
reasoning toward the materials to which students are exposed.
Freire (1985) argued that education centers on the answer, while
ideally it should revolve around the question. Our results seem to
indicate that high-school educational programs and didactic strategies are
based on the pedagogy of the answer, which may prevent students from
making pertinent, sufficient, and structured questions. This suggests that
a mechanical model of knowledge transmission centered on the teacher is
predominant at lower levels of education (Graesser & Olde, 2003). It
seems that the quality of education has not received much attention when
the public policies extended educational coverage. The goal of extending
educational coverage caused classrooms to be filled with students that
need to be guided by a single teacher.
This situation makes it difficult for teachers to conduct an
individualized follow-up with their students and compels them to
organize their didactic strategies so as to cover as much of the syllabus as
possible in little time. Conversely, since graduate programs generally
consist of a reduced number of students, an individualized follow-up is
more workable. Moreover, because a higher level of mastery of some
specialized topic is required, teaching methods center on the more ideal
pedagogy of the question (Freire, 1985).
Other elements might have contributed to these results. First, higher
levels of experience and/or familiarity with the different hypothetical
M. A. Padilla Vargas, J . Tamayo and M. L. Gonzlez-Torres 30
situations may have promoted higher scores. Second, results from high
school students may have been affected by their level of understanding of
the instructions, since previous studies have shown that reading
comprehension in this particular population is poor (Morales, Pichardo,
Arroyo, Canales, Silva & Carpio, 2005). Finally, the type of task we used
might have affected the way participants asked questions. The materials
we used and the lack of familiarity with the technical terms in them, the
distinct hypothetical scenarios and the lack of familiarity with these
scenarios, among other factors, might have affected the type and number
of questions asked, as well as their pertinence and sufficiency. Further
studies with different experimental protocols might be necessary to
assess these possible biases.
So, the effects we observed in this study were more likely due to
characteristics of the population than the variable we manipulated
(achievement criterion). Thus, it may be worthwhile to carry out more
research controlling for participants experience/familiarity with the
hypothetical situations to which they are exposed, as well as to conduct
longitudinal investigations on the development of behavior of asking
questions from childhood through adulthood.
Likewise, it also may be worthwhile to assess the efficiency of the
various procedures designed to train the behavior of asking these types of
questions in more controlled situations (Davey & McBride, 1986;
Graesser & Olde, 2003; King, 1991). Finally, these results point to
deficiencies in the current teaching methods used in high school and
colleges, and it may be necessary to reevaluate the pedagogical strategies
employed in these settings.


REFERENCES

Davey, B., & McBride, S. (1986). Effects of questions-generation
training on reading comprehension. Journal of Educational
Psychology, 78, 4, 256-262.
Freire, P. (1985). Por uma pedagogia da pergunta. Rio de J aneiro: Paz e
Terra.
Graesser, A. C., & McMahen, C. L. (1993). Anomalous information
triggers questions when adults solve quantitative problems and
Effects of Exposure to Different Achievement Criteria 31
comprehend stories. Journal of Educational Psychology, 85, 136-
151.
Graesser A. C., & Olde, B. (2003). How does one know whether a person
understands a device? The quality of the questions the person ask
when the devise breaks down. Journal of Educational Psychology.
95, 3, 524-536.
Ibez, C. (1999). Conducta de estudio: el papel de identificar criterios
en el discurso didctico. Acta Comportamentalia, 7, 47-66.
Kantor, J . R. (1924-1926). Principles of psychology. New York: Alfred
Knopf.
King, A. (1991). Effects of training in strategic questioning on childrens
problem-solving performance. Journal of Educational Psychology,
83, 3, 307-317.
King, A., & Rosenshine, B. (1993). Effects of guided cooperative
questioning on childrens knowledge construction. Journal of
Educational Psychology, 61, 2, 127-148.
Mateos, R., & Flores, C. (2008). Efectos de evaluar el grado de
explicitacin del criterio de ajuste sobre el desempeo de estudiantes
en tareas de identificacin y elaboracin. Acta Comportamentalia,
16, 1, 73-88.
Morales, G., Pichardo, A., Arroyo, R., Canales, C., Silva, H., & Carpio,
C. (2005). Enseanza de la psicologa a travs de la lectura: un
ejemplo de abordaje experimental a la comprensin de textos. En C.
Carpio, & J .J . Irigoyen-Morales (Comp). Psicologa y Educacin:
aportaciones desde la Teora de la Conducta. Mxico: UNAM Fes
Iztacala.
Padilla, M. A., Tamayo, J ., & Gonzlez-Torres, M.L. (2007a). Efectos de
la especificacin del criterio de logro en la elaboracin de preguntas
de investigacin. En S. Carvajal (Editor). 2007 Avances en la
Investigacin Cientfica del Centro Universitario de Ciencias
Biolgicas y Agropecuarias. Mxico: Universidad de Guadalajara.
ISBN. 978-970-27-1280-0.
Padilla, M. A., Tamayo, J ., & Gonzlez-Torres, M. L. (2007b).
Elaboracin de preguntas informales y su posible relacin con la
formulacin de preguntas de investigacin. En S. Carvajal (Editor).
2007 Avances en la Investigacin Cientfica del Centro
Universitario de Ciencias Biolgicas y Agropecuarias. Mxico:
Universidad de Guadalajara. ISBN. 978-970-27-1280-0.
M. A. Padilla Vargas, J . Tamayo and M. L. Gonzlez-Torres 32
Raigada, J . L. (2002). Epistemologa, metodologa y tcnicas del anlisis
de contenido. Estudios de Sociolingstica, 3, 1, 1-42.
Ribes, E., & Lpez, F. (1985). Teora de la Conducta. Un Anlisis de
Campo y Paramtrico. Mxico: Trillas.


APPENDIX A

Examples of questions asked by our participants and the way each
was classified. Each criterion was independent, that is, a correct but
unstructured question could be either pertinent or non pertinent, and be
part of a set of questions that were either sufficient or insufficient as
whole.

a) Example of a question classified as pertinent to the first
hypothetical situation (class syllabus):
- What type of work is required and how is it graded?
b) Example of a question classified as non pertinent to the first
hypothetical situation (class syllabus):
- What are your impressions of the group?
c) Example of a question classified as pertinent to the second
hypothetical situation (medical exams):
- What are these tests for?
d) Example of a question classified as non pertinent to the second
hypothetical situation (medical exams):
- What if he only sends me to do the test without saying
anything?
e) Example of a set of questions classified as sufficient to the first
hypothetical situation (class syllabus):
- What rules apply to this term?
- Can I have a copy of the syllabus?
f) Example of a set of questions classified as non sufficient to the
first hypothetical situation (class syllabus):
- What is the name of the new teacher?
- Is it the same syllabus as the previous teachers?
- Did he leave homework?
g) Example of a set of questions classified as sufficient to the
second hypothetical situation (prescribed medicine):
Effects of Exposure to Different Achievement Criteria 33
- What effects will the medicine have in my body?
- How long do I have to take them for?
- How often do I have to take them?
- Do they have side effects?
h) Example of a set of questions classified as non sufficient to the
second hypothetical situation (medical exams):
- When do I have to pick them up?
- What time do I have to pick them up?
i) Example of a correct structured question
- How are we going to be graded?
j) Example of an incorrect structured question
- What is the rules in this classroom?
k) Example of an unstructured question
- Id ask him if he believes I have a serious illness.








Chapter 3



VARIABLES THAT AFFECT THE
PROCESS OF ASKING AND JUSTIFYING
RESEARCH QUESTIONS


One of the main goals of those who teach science is that novice
researchers may eventually conduct research in a novel and effective way
(De La Fuente, J usticia, Casanova & Trianes, 2005; Snchez Puentes,
2004). This entails a set of writing skills, because they are part of the
scientific practice (Cassany, 2006). According to Keys, Hand, Prain y
Collins (1999), besides spreading knowledge and divulging empirical
evidence, writing scientific texts promotes the generation of new
knowledge.
To do so, scientists must read and write at all functional levels,
especially at the more complex ones, which entails identifying the
distinct elements that comprise a text, relate them to previously read
materials, defend ideas and derive novel questions, justifying them based
on the technical literature reviewed (Fuentes, 2005).
In order to analyze the relationships between reading and writing
skills, Pacheco, Ramrez, Palestina and Salazar (2007) asked
undergraduate psychology students to read a scientific article and to
carry out a series of activities: to draft an abstract, to offer an opinion
about the text, to identify its elements, to ask a research question derived
from it, and to conceive a research project.



Table 4. Experimental design





Variables That Affect the Process 37
Most participants fared better in the tasks that involved
intrasituational interactions, as opposed to activities that required a more
complex level of interaction (extra or transituational).
To analyze further the variables upon which the process of asking
and justifying derived, novel, and pertinent research questions depends,
Padilla, Solrzano and Pacheco (2009) evaluated whether training to
identify and to draft different types of paragraphs of an experimental
article enabled graduate students to write down a justified research
question. Eleven graduate students were assigned to an experimental and
a control group. The experimental group was exposed to a pre-
assessment (or Baseline) over two sessions during which they read two
research papers, two training exercises during which they read two
distinct research papers, and a post-assessment. The control group was
exposed to the same conditions, except that they were trained neither to
identify nor to elaborate on the elements of the papers (see Table 4).
These participants only read the papers used for training the experimental
group, because they were supposed to derive a research question from
these articles (for control reasons, it was necessary that both groups were
exposed to the same articles). In addition, we wanted to make sure that
the act of reading these materials did not by itself cause participants to
ask and justify research questions.
Data were analyzed according to the following criteria: the elements
of the justification of the research question were first classified into title,
definitions, evidences, derived approaches and research question, and the
objective. A score of one was given if the answer was sufficient, half a
point if it was insufficient, and zero if it was inadequate or absent. One
other point was given if participants could identify the dependent
variable and another if they could independent variable in the
justification they wrote. These scores were added and expressed as a
percentage of the total correct answer. Based on a classification proposed
by Padilla et al. (2009), we identified the level at which the research
question was asked (see Table 5).
Before training, participants had a really hard time asking and
justifying their research questions, which suggests that reading and
writing occurred at a less complex level (intrasituational). After training,
however, their performance improved substantially, with questions asked
at extrasituational levels. In addition, the experimental group performed
M. A. Padilla Vargas, J . Tamayo and M. L. Gonzlez-Torres 38
better than the control group, at both asking the questions and justifying
them.

Table 5. Types and characteristics of the research questions

1. Variables are manipulated and evaluated and only chance the
value of one variable.
2. a) evaluate relationship (between variables) not considered in the
field of knowledge, b) is evaluated and manipulated variables that
are relevant to the problem of interest, and c) apply the principles of
scientific theory to explain a social concrete (related to technological
research) or a scientific problem (related to basic research).
3. Experimental preparations are proposed to validate the facts of a
theory.

Regarding the task of identifying the paragraphs that comprise a
scientific paper (Exercise 1), most participants had a hard time doing it,
although it involved behavior at a simpler level (intrasituational). In the
second exercise, when participants had to identify the different types of
questions, to recognize the experimental variables (dependent and
independent), and to write down a paraphrase of the materials they read
(extrasituational level), performance was generally poor, even worse than
in the first exercise, given the greater complexity involved in these tasks.


Figure 6. Levels that were generated the research questions, for each participant
in the pre-assessment (or Baseline) and in the sessions 1 and 2 of the Evaluation.
Variables That Affect the Process 39
Regarding the justification of the research question, the experimental
group performed better than the control group (Figure 6), which suggests
that the training to which they were exposed raised the level at which
their questions were justified. These results are consistent with the
suggestion by Pacheco et al. (2007) that the level at which one reads
corresponds to the level at which one writes. Apparently, exposing
novice researchers to technical materials and explicitly training them to
draft a scientific paper may improve the behaviors of asking and
justifying research questions derived from the papers they read
previously (Figure 7).


Figure 7. Percentage of corrects responses obtained by the participants in the
experimental and control groups on the justification of research questions asked
in the pre-assessment (or Baseline BL) and the two evaluation sessions.
In the same vein, Padilla, Fuentes and Pacheco (submitted) analyzed
the effects of a corrective training for the identification and elaboration
of some of the elements that comprise an empirical paper on the behavior
of asking and justifying research questions. Twenty undergraduate
students were assigned to an experimental or control group. The
experimental design was similar to the one in the previous study, except
that the experimental group was exposed to a corrective training in the
identification and the drafting of empirical papers. This was done to
promote performance at intrasituational and extrasituational levels,
respectively. Both quantitative (Figure 8) and qualitative analyses were
conducted. For the quantitative part, we recorded the percentage of
correct responses in each condition, using a scale especially designed for
this study (for simplicity, only the data from the pre- and post-
0
20
40
60
80
100
BL Eval uati on SS1 Eval uati on SS2
%

o
f

c
o
r
r
e
c
t

r
e
s
p
o
n
s
e
s
Experimental
Control
Experimentalconditions
M. A. Padilla Vargas, J . Tamayo and M. L. Gonzlez-Torres 40
assessments are presented). In addition, we also identified a) the number
of words used in the proposal, under the assumption that the number of
words used in the justification would be increased by training, since
participants would present more arguments, b) the propriety with which
the concepts were used in the pre- and the post-assessments, c) the
congruence between the proposed research question and its justification
based on the papers participants read, and d) the functional level at which
the research questions were asked and justified.
As shown in Table 6, the corrective training to which the
experimental group was exposed arguably promoted the development
and justification of research questions at extrasituational levels.

Table 6. Number of words used by the experimental and control
groups to develop and justify their research question and the
Propriety and Congruence of the research questions generated,
and the Functional Level at wich they were written
(E= Extrasituational; I= Intrasituational; X= No-response*)

Experimental Group

Participant Condition
No. of
Words
Propriety Congruence
Functional
Level
P1 Pre 380 No Yes E
Post 647 Yes Yes E
P2 Pre 552 No Yes I
Post 816 Yes Yes E
P3 Pre 27 No Yes X
Post 1002 Yes yes E
P4 Pre 310 No Yes X
Post 390 No Yes X
P5 Pre 380 No Yes I
Post 369 No Yes I
P6 Pre 140 No No I
Post 305 No No I
P07 Pre 269 No No I
Post 271 No Yes I
P8 Pre 388 Yes Yes E
Post 435 Yes Yes E
P9 Pre 410 No Yes I
Post 571 Yes Yes E
P10 Pre 352 No Yes I
Post 1974 Yes Yes E
Variables That Affect the Process 41
Control Group

Participant Condition
No. of
Words
Propriety Congruence
Functional
Level
P11 Pre 219 No Yes I
Post 140 No Yes I
P12 Pre 384 Yes Yes E
Post 846 Yes Yes E
P13 Pre 357 No No X
Post 585 No No I
P14 Pre 118 No Yes X
Post 263 No Yes I
P15 Pre 183 No No I
Post 467 No Yes I
P16 Pre 323 No Yes X
Post 233 No Yes I
P17 Pre 207 No Yes I
Post 209 No Yes I
P18 Pre 246 No No I
Post 209 No No I
P19 Pre 445 Yes Yes I
Post 406 Yes Yes I
P20 Pre 196 No No I
Post 493 No Yes I

Looking at the number of words used in the proposals, with one
exception, participants in the experimental group used substantially more
word in the post-assessment, compared to the pre-assessment, as
expected, which may suggest a greater number of evidences and further
reasoning. This effect was not as marked in the control group, although
these participants read the same papers.
Regarding the congruence between the proposed research question
and its justification based on the papers participants read, two
participants of the experimental group presented arguments that were not
congruent with the research questions they posed, as opposed to four
participants of the control group.



Figure 8. Percentage of correct responses obtained by the participants of experimental and control groups during the pre-assessment
(or Baseline) and the evaluation.
Variables That Affect the Process 43
As for the proper use of the technical terms when justifying the
research question, we found that five participants from the experimental
group started using them in a proper manner after being exposed to the
training (although one of them already did that in the pre-assessment),
which was related to a higher functional level (extrasituational). A
similar effect was not observed in the control group, which suggests that
only reading the papers without specific training in the identification and
elaboration of their elements does not promote the proper use of the
technical terms.
Considering these results, Padilla and Fernndez (submitted)
evaluated the effects of varying the textual characteristics on the
experimental paper (making them unstructured) on the percent correct
and the functional level (intrasituational or extrasituational) at which
undergraduate students proposed and justified research questions.
Twenty students from a public university were again assigned to an
experimental and a control group. The experimental design was similar
to that of previous studies. However, some of the elements of the
materials we used were modified to make the papers unstructured: 1) the
elements of the research question (e.g., the dependent variable, the
independent variable, and the relationship between them) could not be
clearly identified, 2) the pertinence of the research question was not
clearly stated and the arguments for it did not appear in the same
paragraph as the research question, and 3) the empirical evidence related
to the different elements of the research questions was not fully detailed.
These changes were brought about by the observation that a paper with
these three characteristics resulted in consistently poor performances in a
previous study (Padilla et al., 2009). Because these were the only
differences we could find with the materials we used in other sessions,
we decided to put to a test the hypothesis that these differences produced
the results we observed, by manipulating the characteristics of the
empirical papers to which the participants were exposed.
Figure 9 shows the results of this study. Compared to previous
studies, when participants were exposed to structured papers, the
percentage of correct answers, the functional level at which participants
and developed and justified their research questions, and the propriety
with which they used the technical terms were poorer.

M. A. Padilla Vargas, J . Tamayo and M. L. Gonzlez-Torres 44

Figure 9. Percentage of correct responses obtain by the participants in the pre-
assessment (or Baseline) and Evaluation conditions.
In addition, the percentage of correct answers in identification and
elaboration tasks also was poorer, despite corrective training. These
results are consistent with Padilla et al.s finding that performance was
generally poorer when participants were faced with papers later classified
as complex or unstructured.
Therefore, when untrained readers, such as a novice researcher with
no or little experience reading experimental papers, are exposed to a text
in which the research questions and its pertinence are not clearly stated,
its elements and the relationship between them are not easily identified,
or the evidence that supports it is not adequately presented or organized,
performance in tasks of identification and elaboration is hindered (Padilla
et al., 2009; Padilla & Gonzlez, submitted).
Further support for these claims is being currently obtained in our
laboratory. This time, we exposed six expert researchers to structured
and unstructured papers and analyzed the way those participants posed
and justified a research question (Padilla & Gonzlez, 2013, in
preparation). The experimental design was the same as the previous
study. Results showed in increment in the average percentage of correct
answers from the pre-assessment (61%) to the post-assessment
(93%), both for those exposed to structured articles and those exposed
to unstructured ones, which suggests an effect of training. It is
remarkable that those participants had a relatively good performance in
the pre-assessment compared to novice participants exposed to structured
texts, who usually obtain about 30% of correct answers in this condition,
which suggests that the experience of the expert researchers in reading
and writing technical papers minimizes the effects
of unstructured experimental articles. Moreover, the effect of training
0
20
40
60
80
100
P01 P02 P03 P04 P05 P06 P07 P08 P09 P10
%

o
f

c
o
r
r
e
c
t

r
e
s
p
o
n
s
e
s
Participants
Base Line
Evaluation
Variables That Affect the Process 45
seemed a little stronger when compared to undergraduate students
reading structured texts.
Of course, since we have only run six participants, we cannot make
bolder claims about the effect and its generality. So, we need to conduct
this experiment with a larger sample size, and continue to analyze the
phenomenon systematically, with a parametric analysis of the variables
that affect the production of questions, both academic and non academic,
as well as the relationship between them.


REFERENCES

Cassany, D. (2006). Taller de textos. Leer, escribir y comentar en el aula.
Espaa: Paids.
De la Fuente, J ., J usticia, F., Casanova, P. F., & Trianes, M. V. (2005).
Perceptions about the construction of academic and professional
competencies in psychologist. Electronic J ournal of Research in
Educational Psychology, 3, 1, 3-34.
Fuentes, T. (2005). Repertorios recurrentes de la comprensin lectora
reconstructiva. Tesis doctoral, Universidad de Guadalajara,
Guadalajara, J alisco, Mxico.
Keys, C. W., Hand, B., Prain, V., & Collins, S. (1999). Using the science
writing heuristic as a tool for learning from laboratory investigations
in secondary science. J ournal of Research in Science Teaching, 36,
10, 1065-1084.
Pacheco, V., Ramrez, L., Palestina, L., & Salazar, M. (2007). Una
aproximacin al anlisis funcional de la relacin entre las conductas
de leer y escribir en estudiantes de psicologa. En J .J . Irigoyen, M.
J imnez , & K. Acua (Eds.), Enseanza, aprendizaje y evaluacin.
Una aproximacin a la pedagoga de las ciencias (pp. 247-275).
Mxico: UniSon.
Padilla, M. A., Solrzano, W. G., & Pacheco, V. (2009). The effects of
text analysis on drafting and justifying research questions. Electronic
J ournal of Research in Educational Psychology, 17, 7, 1, 77-102.
Padilla, M. A., Fuentes, N., y Pacheco, V. (submitted). Anlisis de los
efectos de un entrenamiento correctivo en la elaboracin y
fundamentacin de preguntas de investigacin. Revista Mexicana de
Investigacin Educativa.
M. A. Padilla Vargas, J . Tamayo and M. L. Gonzlez-Torres 46
Padilla, M. A., y Fernndez, G. (submitted). Anlisis del efecto de
manipular algunas caractersticas textuales del referente en la
lectoescritura de textos cientficos. Revista Latinoamericana de
Psicologa.
Padilla, M. A., y Gonzlez, J . (2013). Anlisis del efecto de exponer a
artculos incompletos a investigadores experimentados. Manuscript
in preparation.
Snchez Puentes, R. (2004). Ensear a investigar. Una didctica nueva
de la investigacin en ciencias sociales y humanas. Mxico: Plaza y
Valds.











GENERAL CONCLUSION

Making proper non academic questions contribute to posing research
questions? If so, what is the relationship between these two types of
questions? The present book attempted to shed light on these inquiries.
The evidence we gathered so far with the studies we have conducted
suggest that the answer to the first question is affirmative. The behavior
of asking questions seems intimately related to reading and writing skills;
making a good research question implies that it should be not only novel,
pertinent, useful, and viable, but also derived from the literature of the
area of interest, because science is not a lonely enterprise, but highly
collaborative. This means that significant breakthroughs in science are
more likely with the participation of hundreds of researchers working on
the same problems, from different perspectives, in different laboratories,
with diverse experimental protocols and populations, with more or less
clear and unified criteria and boundaries. The advancement of science is
made possible by the contribution of ones work to the task of building a
solid and coherent body of knowledge.
Scientific knowledge is similar to a wall with irregular holes in it.
The job of scientists is to identify what kinds of research questions allow
for a pertinent experimental preparation that will provide the evidence
that might fill in a specific void in the wall of science. This is the only
way a significant contribution can be made. Otherwise, the work would
be of no use to anyone, since it would not contribute to the advancement
of a particular area and could remain an isolated piece of knowledge.
As consequence, it is imperative that reading comprehension is
fostered in novice researchers, so they will be able to pose a derived
research questions that is coherent with the existing body of knowledge
in a particular area. In order to propose a properly justified research
M. A. Padilla Vargas, J . Tamayo and M. L. Gonzlez-Torres 48
question that contributes to the advancement of a particular area, it is
necessary to conduct exhaustive searches into the existing literature, to
adequately understand the materials read, and to identify the holes in the
wall of science.
Of course, the research question has to be original and not a mere
repetition of what has already been done, varying simple parameters.
This has no academic merit, and it is a waste of time and resources. Our
results show that participants often repeated the questions that had
already been investigated in the papers they read, or only proposed
minimal non pertinent variations to the independent variables
manipulated in these materials. Additionally, participants often had a
really hard time justifying the proposed research questions, despite being
required to derive them from the evidence provided in the papers they
read. This difficulty may stem from the fact that justifying a derived
research question requires the exercise of writing skills at an
extrasituational level, and our participants writing skills were generally
poor, hardly exercised, and occurred at lower functional levels. This was
very evident when they only repeated the information presented in the
papers (sometimes even unrelated to the proposed question), whereas the
proper way of justifying would be to restructure and elaborate on these
bits of information so as to come up with arguments that were coherent
with the new questions they posed. Only experienced researchers were
able to do it, which suggests that academic and investigative practice
improves adjustment to the expected achievement criterion.
The results reported here seem to indicate a direct relationship
between educational experience and the ability to ask research questions.
Performance of undergraduate and graduate students was initially poor in
general, but improved substantially according to their academic status
after being exposed to a specific training in reading and writing technical
materials, especially if it was corrective. This training aimed at
developing skills at both intrasituational (identification) and
extrasituational (elaboration) levels. High school students, on the other
hand, did not seem to benefit from training and were able to pose neither
research questions nor non academic questions, even when the
achievement criterion was clearly specified.
Conversely, more experienced researchers performed relatively well
from the beginning, and achieved nearly perfect scores after training,
even when the papers to which they were exposed were modified to
General Conclusion 49
become disorganized and complex. When undergraduate students were
exposed to the same materials, their average scores did not exceed 50%
of correct answers.
We may conclude that reading and writing skills in general, and of
technical materials in particular, affect how we make both academic and
non academic questions, which may be related to our referential history,
that is, our lack of familiarity with these types of texts. To make things
worse, our data also suggests that education is based on what Freire
called the pedagogy of the answer, which conceives the teacher as the
center of knowledge, whose job is to impart bits of information
mechanically. The student, therefore, becomes a mere passive agent of
the educational system, someone that does not question those bits of
information, that does not make questions and that does not ponder on
them. That results in students that do not know how to make questions in
any form, neither non academic nor research questions.
Another noteworthy result is that students with greater educational
experience (undergraduate and graduate students) are usually better at
asking non academic questions, as shown in Chapter 2. The same was
observed for research questions. This relationship between being able to
ask pertinent research and non academic questions and ones educational
level might be due to the selection process prospective students undergo.
Those who enroll in higher education might be the ones that already had
better reading and writing skills, and that were more likely encouraged to
express a critical attitude towards knowledge throughout their formative
years, which means that questioning to themselves and posing those
questions to their teacher were common practices for them.
For that reason, it is important to restructure the education system so
as to promote critical reasoning at all educational levels, to create
students that question what they are told, that are skeptical, that inquire
their surroundings and that pose questions to their teachers. Also, it is
vital to explicitly promote training of reading and writing skills from the
more basic educational levels, so that students are able to read both
technical and non technical materials, which in turn will provide them
with the necessary precursors to make good research questions, should
they choose this career path. Several diagnostic analyses have shown that
undergraduate Psychology students at the Mexican National Autonomous
University (UNAM) have poor reading and writing skills. However,
there are no projects especially designed to improve these abilities in the
M. A. Padilla Vargas, J . Tamayo and M. L. Gonzlez-Torres 50
curriculum. Quite the opposite happens in Colombia, for example, where
there are programs especially designed to improve research skills in
which reading, textual analysis, and writing are explicitly trained
(Guerrero, 2007).
Finally, there seems to be a relationship between making good non
academic questions and making good research questions, which may be a
starting point in the training of novice researchers. In the same vein,
considering that the main objective of science is to produce knowledge,
teaching to conduct research must entail, among other things, training the
capacity to pose original problems based on the reconstruction of various
approaches to a particular subject matter. Part of the teachers job,
therefore, is to entice students curiosity, so that they are more likely to
pose creative and practical problems. Another part should be to
encourage the critical reading of what has been done regarding a
particular topic, so that students may come up with their own problems,
amenable to investigation. This might be the first step towards training
critical researchers that are engaged on making high-level science that
will ultimately impact on the development of a progressively less
unequal and more educated society.


REFERENCE

Guerrero, M. E. (2007). Formacin de habilidades para la investigacin
desde el pregrado. Acta Colombiana de Psicologa, 10, 2, 190-192.












INDEX


A
academic performance, 18
academic settings, 8, 9
adjustment, 4, 13, 20, 22, 48
adulthood, 30
adults, 30
advancement, 47, 48
animal behavior, 5, 6
assessment, xvi, 4, 21, 22, 37, 38, 39, 41,
42, 43, 44
B
baggage, 9
basic education, 49
basic research, 5, 38
behaviorism, 5
behaviors, 39
benefits, ix, xv
blood, 23
C
childhood, 30
children, 5, 31
classes, 5, 22
classification, 3, 7, 37
classroom, 12, 18, 33
colleges, 30
Colombia, 50
community, ix, xv
compensation, 21
complexity, 7, 38
comprehension, 5, 8
congruence, 40, 41
construction, 14, 31, 45
content analysis, 7, 10, 24
contingency, 3
control group, xvi, 37, 38, 39, 40, 41, 42,
43
correlation(s), 7, 11
critical thinking, 19
currency, 9
curriculum, 50
D
deficiencies, 18, 30
dependent variable, 37, 43
depth, ix, xv, 18
diffusion, 1
draft, 35, 37, 39
E
education, 1, 19, 29, 49
Index 52
educational experience, 48, 49
educational programs, 29
educational system, 19, 49
educators, ix, xv
elaboration, 5, 18, 39, 43, 44, 48
environment, 21
evidence, 1, 5, 13, 20, 35, 43, 44, 47, 48
exercise, 1, 38, 48
experimental design, 7, 39, 43
exposure, 5, 27, 29
F
financial, ix, xi, xv
flaws, 18
G
graduate program, 29
graduate students, xvi, xvii, 21, 23, 25,
27, 28, 37, 48, 49
H
high school, xv, xvi, 4, 5, 8, 9, 11, 20,
23, 25, 26, 28, 29, 30
higher education, 49
history, 12, 13, 49
homework, 32
hotel, 9
human, 4, 5, 6, 12
human behavior, 4, 5, 12
hypothesis, 43
I
ideal, 29
identification, 2, 3, 4, 5, 6, 18, 39, 43, 44,
48
immigration, 9
improvements, xvii, 2
independent variable, 5, 13, 22, 37, 43,
48
individuals, 19
inferences, 18
integration, 1
J
justification, 5, 13, 37, 39, 40, 41
L
lead, 17
learning, 14, 17, 19, 45
learning process, 17
light, 47
M
materials, xvii, 1, 2, 8, 12, 13, 18, 29, 30,
35, 37, 38, 39, 43, 48, 49
matter, 50
medical, 26, 32, 33
medication, 23, 26
medicine, 32, 33
methodology, xvii
Mexico, ix, xv
music, 12
O
obstacles, 17
organize, 19, 29
P
participants, xvi, 1, 2, 3, 5, 6, 7, 8, 9, 10,
11, 12, 13, 20, 21, 22, 23, 24, 25, 27,
28, 30, 32, 37, 38, 39, 40, 41, 42, 43,
44, 48
pedagogy, 19, 29, 49
permit, 1, 2
Index 53
poor performance, 12, 43
population, xvi, 4, 8, 12, 13, 27, 30
preparation, 46, 47
principles, 38
probability, 4
problem-solving, 31
project, 35
psychologist, 14, 45
psychology, 4, 8, 21, 31, 35
publishing, ix, xv
Q
question mark, 24, 27
questioning, 17, 19, 31, 49
R
reading, xvi, xvii, 1, 2, 5, 6, 8, 9, 11, 12,
13, 30, 35, 37, 43, 44, 47, 48, 49, 50
reading comprehension, xvi, 5, 6, 8, 30,
47
reading comprehension test, 5, 6, 8
reasoning, 18, 29, 41, 49
reconstruction, 50
reinforcement, 5
reliability, 7, 10, 24
reproduction, 19
researchers, ix, xv, xvi, xvii, 1, 2, 35, 39,
47, 48, 50
resources, 48
response, 3, 19, 40
rules, 26, 32, 33
S
school, xv, xvi, 5, 21, 22, 25, 26, 27, 29,
48
science, 1, 14, 35, 45, 47, 48, 50
scientific theory, 38
side effects, 33
social behavior, 5
social change, 19
society, 50
spelling, 24, 27
stimulus, 3
structure, 24, 27, 28, 29
style, 5, 12
T
taxonomy, 3
teachers, 1, 19, 29, 49
teaching strategies, 19
techniques, 18
training, ix, xv, xvi, xvii, 2, 18, 20, 30,
31, 37, 39, 40, 43, 44, 48, 49, 50
transmission, 29
transportation, 9
triggers, 30
tutoring, 29
V
validation, 7
variables, xvi, xvii, 2, 5, 7, 28, 37, 38
variations, 48
vein, 18, 19, 39, 50
W
waste, 48

You might also like