Journal of Mathematical Behavior 32 (2013) 377396

Contents lists available at ScienceDirect

The Journal of Mathematical Behavior
j our nal homepage: www. el sevi er . com/ l ocat e/ j mat hb
Multiple levels of metacognition and their elicitation through
complex problem-solving tasks
Young Rae Kim
a,
, Mi Sun Park
a
, Tamara J. Moore
a
, Sashank Varma
b
a
STEMEducation Center, University of Minnesota, 320 Learning & Environmental Science Building, 1954 Buford Avenue, St. Paul, MN
55108, United States
b
Department of Educational Psychology, 165 Education Sciences Building, University of Minnesota, Minneapolis, MN 55455, United
States
a r t i c l e i n f o
Article history:
Available online 25 June 2013
Keywords:
Metacognition
Denition building
Operationalizing denitions
Model-eliciting activities
a b s t r a c t
Building on prior efforts, we re-conceptualize metacognition on multiple levels, looking
at the sources that trigger metacognition at the individual level, the social level, and the
environmental level. This helps resolve the paradox of metacognition: metacognition is per-
sonal, but it cannot be explained exclusively by individualistic conceptions. We develop
a theoretical model of metacognition in collaborative problem solving based on models
and modeling perspectives. The theoretical model addresses several challenges previously
found in the research of metacognition. This paper illustrates how metacognition was
elicited, at the environmental level, through problems requiring different problem-solving
processes (denition building and operationalizing denitions), and how metacognition
operated at both the individual level and the social level during complex problem solving.
The re-conceptualization of metacognition has the potential to guide the development of
metacognitive activities and effective instructional methods to integrate them into existing
curricula that are necessary to engage students in active, higher-order learning.
2013 Elsevier Inc. All rights reserved.
1. Introduction
In the rst overview of the new science of learning, Bransford, Brown, and Cocking (2000) emphasized the critical role
of metacognition in successful learning. Metacognition is the process in which students monitor, assess, and modify their
own learning progress. It can help students develop their knowledge for teaching themselves and improve positive learn-
ing transfer to new settings and events. This has been demonstrated in numerous studies across multiple disciplines (e.g.,
Bielaczyc, Pirolli, & Brown, 1995; Borkowski, Carr, & Pressely, 1987; Muir, Beswick, & Williamson, 2008; Rasekh & Ranjbary,
2003; Schraw, 1998; White & Frederickson, 1998). These studies demonstrate the need for instructional approaches to help
students become more metacognitive about their learning. However, more needs to be understood about the mechanisms
of metacognition, howto effectively encourage students metacognition in problemsolving, and howto promote the devel-
opment of students metacognitive abilities a mechanism that enables one efciently to organize, monitor, and regulate
what one knows to reach a goal successfully.
Metacognition has traditionally been dened at the individual level, as thinking about ones own thinking (Flavell, 1976).
In the research presented here, we re-conceptualize the construct of metacognition on multiple levels, considering thinking
about thinking at the individual level, the social level, and the environmental level. At the individual level, a student has access

Corresponding author at: Department of Curriculumand Instruction, STEMEducation Center, University of Minnesota, 320 Learning & Environmental
Science Building, 1954 Buford Avenue, St. Paul, MN 55108, United States. Tel.: +1 612 807 0951; fax: +1 612 626 0993.
E-mail addresses: kimx1495@umn.edu (Y.R. Kim), parkx598@umn.edu (M.S. Park), tamara@umn.edu (T.J. Moore), sashank@umn.edu (S. Varma).
0732-3123/$ see front matter 2013 Elsevier Inc. All rights reserved.
http://dx.doi.org/10.1016/j.jmathb.2013.04.002
378 Y.R. Kimet al. / Journal of Mathematical Behavior 32 (2013) 377396
to internal sources to monitor or regulate her/his cognitive processes. However, this raises what we term the paradox of
metacognition: metacognition is personal, but it cannot be explained exclusively by individualistic conceptions (e.g., Iiskala,
Vauras, & Lehtinen, 2004; Iiskala, Vauras, Lehtinen, & Salonen, 2011; Vauras, Iiskala, Kajamies, Kinnunen, & Lehtinen, 2003).
For example, individuals may experience a time when they are stuck. If individuals only have access to their own internal
thinking to help them resolve their obstacle thinking which caused them to get stuck in the rst place how can they
make progress?
1
Another example might be individuals lacking good self-regulation abilities. How can they detect their
own (false) cognition when it goes awry, and adapt their thinking? One way to resolve this paradox is to observe that in
addition to their internal psychological resources, individuals also have access to external sources that trigger metacognitive
thinking. These external sources include bothsocial triggers that come fromother people interacting withthe individual, and
environmental triggers that come from interacting with the environment in which one is learning. This access to external
sources of metacognition informs how metacognitive failures, such as an absence of checking behavior (Stacey, 1992),
metacognitive blindness, metacognitive vandalism, and metacognitive mirage (Goos, 2002), can be prevented or resolved.
The research reported here shows howmetacognition, operating at multiple levels (individual, social, and environmental),
functions during complex collaborative problemsolving to overcome the paradox of metacognition, and ultimately informs
instructional practice.
We consider metacognition in the context of solving complex problems of the kind found in mathematics and science
classrooms. Inparticular, we focus ontwoprocesses requiredbycomplexproblemsolving, denitionbuilding andoperational-
izing denitions. These processes are strongly dependent upon (1) whether or not problems involve directed information,
includingclear denitions andunique solutionpaths, toaccomplishwell-denedgoals; and(2) the degree towhichproblems
are directed in the conceptualization and planning of the problem-solvers nal argument. Non-triviality and complexity in
denition building and operationalizing denitions are the representative sources that trigger metacognition at the environ-
mental level. They hold potential for revealing metacognition at the individual, and especially the social levels (e.g., Efklides,
2006; Iiskala et al., 2011; Prins, Veenman, & Elshout, 2006). The central research questions of this study are:
(1) Howis metacognitionelicited throughthe denitionbuilding and operationalizing denitions processes during complex
collaborative problemsolving?
(2) How does metacognition operate during complex collaborative problem solving at the individual, social, and environ-
mental levels?
The type of complex problems we consider here are Model-Eliciting Activities (MEAs). MEAs are team-oriented, interdis-
ciplinary, and realistic problem-solving tasks that reveal participants thinking (Chamberlin & Moon, 2005; Diefes, Moore,
Zawojewski, Imbrie, & Follman, 2004; Lesh & Doerr, 2003; Lesh, Hoover, Hole, Kelly, & Post, 2000; Moore & Diefes-Dux,
2004; Moore & Hjalmarson, 2010; Moore, Diefes-Dux, & Imbrie, 2006). They were initially created by mathematics educa-
tors as a research tool for exploring students conceptual understanding and problem-solving strategies (Lesh et al., 2000;
Lesh & Lamon, 1992). Thus, they work well as an authentic method for verbal protocol analysis, since students are required
to verbalize their thoughts while working on MEAs in teams in natural classroom settings. Here, the Paper Airplane Con-
test MEA (described in Section 3.3) was used to explore how metacognition on multiple levels can be fostered by complex
collaborative problemsolving.
2. Multiple levels of metacognition
In this section, we review a subset of the expansive literature on metacognition, focusing on the distinction among the
individual, social, and environmental levels. We also review the literature suggesting the utility of MEAs for studying the
development of metacognition on multiple levels as authentic methodological tools and sources of metacognition at the
environmental level.
This research study has adopted the Models and Modeling Perspectives (MMP) in order to study metacognition. MMP has
beenexpanded and applied to the teaching and learning of various subjects, inparticular inSTEM(Science, Technology, Engi-
neering, andMathematics) education. Thefocus of MMP is onhowstudents developconceptual systems of interpretationthat
include the complexity of their daily lives and their knowledge and experiences of various content domains in collaborative
modeling problem-solving settings (Lesh & Doerr, 2003). These perspectives provide newinsights into metacognition com-
pared with traditional viewpoints of metacognition (Lesh & Zawojewski, 2007; Lesh, Lester, & Hjalmarson, 2003). One focus
of MMP research is on individuals conceptual systems, including both cognitive (e.g., understanding, skills) and metacog-
nitive (e.g., beliefs, awareness) components. The MMP assumes that both cognitive and metacognitive components within
the holistic conceptual systems interactively and bi-directionally inuence each other. For example, as students increase
1
There are anecdotal examples of mathematicians and natural scientists being stuck on hard problems and then becoming unstuck on their own,
following anincubation periodlasting many days, weeks, or months (Hadamard, 1954). Indeed, the Gestalt traditionmade muchof the role of incubation
in problemsolving(Duncker, 1945). However, modernpsychological studies havefoundlittleevidencethat incubation plays afundamental roleinproblem
solving (Kaplan & Simon, 1990). Thus, the proposal that there are problems that people can solve on their own (i.e., given internal knowledge sources) and
additional problems they can solve in collaboration with others (i.e., given external knowledge sources) merits investigation.
Y.R. Kimet al. / Journal of Mathematical Behavior 32 (2013) 377396 379
Fig. 1. Diagramshowing howthe different internal and external triggers affect cognition and metacognition.
their understanding, their metacognition is triggered effectively because encouraging individual metacognition helps stu-
dents develop better understanding (Lesh & Zawojewski, 2007; Lesh et al., 2003). The MMP assumes that thinking becomes
metacognitive when the individuals shift from thinking WITH the cognitive components to thinking ABOUT them, by
monitoring, controlling, and regulating them (Lesh et al., 2003). The MMP assumes that metacognition could be devel-
oped along with dimensions similar to cognitive development. For example, when observing the development of students
metacognitive abilities in problem solving, we are expected to observe the gradual patterns of metacognitive behaviors,
such as external toward internal, concrete toward abstract, simple toward complex, and so on (e.g., Lesh et al., 2003).
Another focus of MMP research is on the situated, environmental, and social nature of metacognition. The MMP assumes
that metacognition is closely related to particular content and situations because it draws on individuals interpretations
of the content and situations based on their conceptual systems. By valuing the situated and environmental nature of
metacognition, the MMP assumes that the productivity of metacognitive functions often varies across problems, and even
the stages of problem solving. This is due to the focus of (sub)tasks being different across problems and changed during
the problem-solving processes. For example, brainstorming may be more productive in the early stages of problem-solving
processes rather than at later stages, such as assessment (Lesh et al., 2003). The social nature of metacognition is also
a research focus of the MMP. Research from the MMP often investigates problem solving while students are engaged in
collaborative teams, and compares teams with individuals. The MMP suggests that investigations focusing on a teamoften
provide a useful way to inform how one individuals mind works in context, in particular how one individuals thinking
becomes metacognitive (Lesh et al., 2003; Magiera & Zawojewski, 2011).
The three potential triggers of metacognition individual, social, and environmental affect the individuals cogni-
tion and metacognition. Fig. 1 shows the framework used in this study, based on the MMP view of metacognition and
a re-conceptualization of metacognitive triggers. The diagram in the top half of Fig. 1 indicates the extended sources to
which an individual has access for triggering metacognition. The sources of metacognition, as starting places for triggering
380 Y.R. Kimet al. / Journal of Mathematical Behavior 32 (2013) 377396
metacognition, are categorizedas internal andexternal. The internal sources are individuals conceptual systems that include
both cognitive and metacognitive components. The external sources are partitioned into (1) others conceptual systems in
collaborative activities and (2) environmental sources. The function machine in the bottomhalf of Fig. 1 shows howindivid-
uals thinking becomes metacognitive. For example, when an individual who works in a group uses a mathematical concept
of average to solve a problemwithout any evaluation, he/she thinks WITHthe cognitive component knowledge of the aver-
age fromherself/himself (IC) or fromgroup members (OC). On the other hand, when an individual evaluates whether the
concept of average is proper for the problemsituation, he/she thinks ABOUT the cognitive component, and her/his thinking
becomes metacognitive. In Fig. 1, the use of the Venn diagramemphasizes that the sources of metacognition are not active
agents but rather starting places for the metacognitive activities. The use of the function machine representation emphasizes
our focus on the functions of metacognition that an individual as the unique agent of metacognition triggers.
The following sections reviewliterature supporting the multiple levels of metacognition conceptualized for this study.
2.1. Metacognition at the individual level
Metacognition commonly refers to thinking about our own thinking, but the denition is not useful enough to study
metacognitive behavior of childreninmathematical problemsolving andinlearning mathematics (Schoenfeld, 1987; Wilson
& Clarke, 2002). Research on metacognition frequently presents knowledge of cognition and regulation of cognition as the two
main aspects of metacognition (e.g., Flavell, 1976; Garofalo & Lester, 1985; Schoenfeld, 1987). However, there have been
different views on the main aspects for the construct of metacognition, how to categorize these main aspects, and how to
establish the relationship between them. These discrepancies have caused confusion over the termmetacognition (Garofalo
& Lester, 1985; Wilson & Clarke, 2002). Schoenfeld (1987), for example, separated beliefs as a distinct category from the
knowledge of cognition while others consider them as a type of subjective knowledge (Efklides, 2006; Garofalo & Lester,
1985; Norman, 1981).
Focusing onthe functions of metacognitionitself, there is agreement uponthe operational denitionof metacognition, i.e.
metacognition as manifestations of the monitoring and regulatory function. For example, to dene the construct of metacog-
nition, Wilson and Clarke (2002) emphasized the two non-regulatory functions of metacognition: awareness individuals
have of their own thinking (Awareness) and their evaluation of that thinking (Evaluation). These monitoring functions
are distinguished fromthe regulatory function of metacognition, their regulation of that thinking (Regulation). Similarly,
Efklides (2006) presentedthree facets of metacognitiondistinguishedby their manifestations as a functionof monitoring and
control. The monitoring functions are metacognitive knowledge knowledge about ones own cognition and metacognitive
experiences metacognitive judgment and assessment based on monitoring the features or outcomes of the task at hand.
The control function of metacognition is metacognitive skills the knowledge of the procedures needed to control cognition
(Efklides, 2006).
However, the existing literature has paid less attention to the sources of metacognition that are not active agents, but
rather starting places for the metacognitive activities. For example, knowledge and attributes that individuals have are the
sources to which they have access for eliciting metacognition. That is why the current study takes the MMP as a theoretical
framework for the construct of metacognition. As mentioned above, a primary focus of MMP research is on individuals
conceptual systems that include both cognitive and metacognitive components. At the individual level, the sources of
metacognition are the individuals conceptual systems that are cumulated with prior knowledge and experiences. Individual
differences play a critical role because people possess differing conceptual systems.
For this study, we merged the viewpoint of the MMP with a portion of Goos (2002) metacognitive constructs framework,
which she developed based on the episode-based frameworks of Schoenfeld (1985) and Artzt and Armour-Thomas (1992).
We will use our framework to identify students specic metacognitive behaviors while collaboratively solving a complex
mathematical task. In Goos (2002) framework for analyzing verbal protocols in a collaborative problem-solving setting, she
considered the monitoring and regulatory functions of metacognition that would be appropriate and expected at each stage
of the problem-solving processes. In particular, the monitoring function involves assessment of ones own thinking: assess-
ment of knowledge, assessment of understanding, assessment of strategy appropriateness, assessment of progress toward goal,
assessment of strategy execution, and assessment of the accuracy or sense of a result. The regulatory function of metacognition
is triggered based on these assessing processes: identifying new (alternative) information (strategy), reinterpreting problem,
changing strategy, correcting errors, and so on. The current study adopted this classication of metacognitive activities as
manifestations of the monitoring and regulatory functions. However, Goos presented this framework as linear, because she
adopted a linear progression viewpoint of problem solving. MMP considers problem-solving processes to be iterative and
inherently tied to the prior knowledge of learners. Therefore, problemsolving is not linear, nor can we expect metacognitive
processes (monitoring and regulating) to be so. This has led to a redesign of the operational denition of metacognition
(Fig. 2). For our representation of metacognitive processes, we have rearranged Goos (2002) framework to represent the
MMP viewof howproblemsolving occurs.
2.2. Metacognition at the social level
All manifestations of metacognition, however, cannot bereducedtotheindividual level only(Iiskalaet al., 2004, 2011). The
paradox of metacognition posits that metacognition is personal, but it is not sufcient to completely explain metacognition
Y.R. Kimet al. / Journal of Mathematical Behavior 32 (2013) 377396 381
Fig. 2. An MMP viewof metacognitive activities during problemsolving.
by exclusively drawing on individualistic conceptions (Iiskala et al., 2004, 2011; Jost, Kruglanski, & Nelson, 1998; Vauras
et al., 2003; Volet, Vauras, & Salonen, 2009). For example, it is difcult for individuals who have little metacognitive ability
to monitor and evaluate their own learning and understanding. Therefore, how do these individuals overcome temporary
metacognitive failures? Questions such as this one have led researchers to call increased attention to the social nature of
metacognition (Efklides, 2006, 2008; Goos, Galbraith, & Renshaw, 2002; Iiskala et al., 2004, 2011; Jost et al., 1998; Magiera
& Zawojewski, 2011; Vauras et al., 2003).
One way to resolve this paradox is to consider a dual agent organization of metacognition, an individual and a group in
which the individual is engaged (Iiskala et al., 2004, 2011; Vauras et al., 2003; Volet, Summers, & Thurman, 2009; Volet,
Vauras, et al., 2009). In this perspective, a group is considered as a whole agent of metacognition at the social level. Several
researchers have conceptualized metacognition at the social level using language such as socially shared metacognition
(Iiskala et al., 2004, 2011), shared-regulation (Vauras et al., 2003), and co-regulation (Volet, Summers, et al., 2009). A group
as a social system (Salomon & Globerson, 1989) is recognized as an entity pursuing a common goal. The metacognition
triggered by the group members operates as a whole to jointly monitor and regulate a cognitive process toward a common
consensual goal (Iiskalaet al., 2011; Vauras et al., 2003; Volet, Vauras, et al., 2009). Thus, aunit for analyzingverbal protocols is
oftenanepisode that is a set of cognitive or metacognitive activities triggeredbythe members ina groupas a whole entity, not
individual activities (Iiskala et al., 2004, 2011; Vauras et al., 2003; Volet, Summers, et al., 2009). This perspective emphasizes
metacognition as social processes that are not context variables that facilitate metacognition as individual processes (Iiskala
et al., 2011).
Collaborative problem-solving settings have provided a useful windowinto research on metacognition through the lens
of individual cognitive theories, as well as through the lens of social cognitive theories (Lesh et al., 2003). However, the main
agent of metacognition is still an individual, regardless of whether the individual engages in collaborative teams or works
independently. Metacognition itself is a mental process within an individual drawing on the individuals conceptual systems
(Lesh et al., 2003). Iiskala et al. (2011) criticized this individualistic conception of metacognition. However, we consider an
individual as the unique agent of metacognition. The situated, environmental and social nature of metacognition needs to
be explored by keeping the focus on the unique agent of metacognition. Metacognition is not an autonomous entity over
which an individual has no control; thus, it can be practiced and developed (Lesh et al., 2003; Schoenfeld, 1987).
Re-conceptualizing metacognition on multiple levels, focusing on the sources that trigger metacognition at both the
individual and social levels, is a different way to resolve the paradox of metacognition because metacognition is supported
by external sources, in addition to internal ones. The agent of metacognition is an individual who has access to the sources
of metacognition at both the individual and social levels.
At the social level, one source is the description of the development of conceptual systems of other individuals, such as
peers or teachers. One person talking aloud about the development of her or his thinking around a particular conceptualiza-
tion of a problemcan provide feedback for another persons thinking about her or his own conceptualization. Considering
metacognitive feedback fromones mind to anothers helps resolve the paradox of metacognition, and is an important con-
sequence of having students solve problems collaboratively (Lesh et al., 2003; Lesh & Zawojewski, 2007). Interactions with
peers and teachers are the main sources that encourage individuals to retest their own learning process, to monitor their
current level of understanding, and to detect and repair their misconceptions (Carr & Biddlecomb, 1998; Goos & Galbraith,
1996; Goos et al., 2002; Goos, 1994, 2002; Kramarski &Mevarech, 2003; Lesh &Zawojewski, 2007; Lesh et al., 2003; Schraw,
1998).
Social sources inuencing metacognition are important variables in changing students attitudes and beliefs toward
themselves as mathematical problem-solvers, toward mathematical problems, and toward mathematics itself, all of which
are important components in mathematical metacognition (Schoenfeld, 1987). For example, Wilkins and Ma (2003) found
382 Y.R. Kimet al. / Journal of Mathematical Behavior 32 (2013) 377396
that social sources, such as high expectations from teachers and inuence from peers, have a signicant impact on the
development of positive attitudes toward mathematics. By contrast, individual sources, such as prior achievement and
educational aspirations, are not signicantly related to changes in attitude, and in fact, can have a negative impact.
2.3. Sources of metacognition at the environmental level
The external sources of metacognition are not limited to interactions among participants in collaborative activities (Goos,
1994, 2002; Goos et al., 2002). Interactions between a person (or persons) and the learning environment are also important
sources that trigger metacognition (Iiskala et al., 2011; Magiera & Zawojewski, 2011; Volet, Vauras, et al., 2009). Interac-
tions with the learning environment are potential sources encouraging students to develop metacognitive ability. These
interactions help students unpack misconceptions and repair themthrough metacognitive processes operating at both the
individual and social levels.
The learning environment, as an external source, supports metacognition through classroom activities and through
specic problem-solving tasks (Lesh et al., 2003; Lesh & Zawojewski, 2007). Each activity or task involves a different focus
of problem solving, such as analyzing and creating, which directly affects the focus of metacognition (Lesh et al., 2003;
Stacey, 1992). Problems requiring different levels in conceptual and cognitive demands of the problem-solving processes
can produce differing metacognitive functions involving different focuses of metacognition within problem-solvers (Lesh
et al., 2003). Task complexity is another important factor in the elicitation of metacognition. Metacognition is triggered
more during difcult problems (Helms-Lorenz & Jacobse, 2008; Iiskala et al., 2004, 2011; Prins et al., 2006; Stahl, Pieschl, &
Bromme, 2006; Vauras et al., 2003).
The following sections describe in more detail the sources of metacognition at the environmental level. In particular,
we use the primary data collection tools of MMP, Model-Eliciting Activities (MEAs). We use MEAs in this study for two
functions: as an authentic methodological tool for studying metacognition on multiple levels, and as an environmental
source for supporting metacognition.
2.3.1. The learning environment and metacognition
Traditional lectures and individual problem-solving tasks, such as textbook word problems, do not encourage or elicit
metacognition, particularly at the social and environmental levels, because they do not require higher-order thinking. There-
fore, these environments are insufcient for investigating our expansion of the metacognition construct. Teacher-centered
approaches to instruction allow limited metacognitive processes. However, this is highly dependent on the knowledge of
the teacher because these approaches are associated chiey with the transmission of knowledge. Brown (2003) argued that
teachers in direct instruction environments retain control over students learning and focus more on content rather than
on students processing. It is not expected that students will develop broad and exible metacognitive ability in this static
environment (Lesh et al., 2003). For example, traditional lectures include traditional word problems that encourage students
to apply learned processes in a rote way, but not to develop or practice their own ways of thinking (Cardelle-Elawar, 1995).
Traditional word problems provide students with the information necessary to solve problems. Students only have to
identify the proper procedure and the correct inputs to accomplish concrete, well-dened goals. These types of problems
encourage students to maintain a right path by removing obstacles in the path. To encourage metacognition through prob-
lemsolving, different types of problem-solving tasks are required. In student-centered approaches, students are encouraged
to build their own strategies and enhance their learning through monitoring and evaluating the processes and products
of social activities (Bransford et al., 2000). Social activities can be emphasized through many active learning environments
using pedagogical constructs such as MEAs, cooperative learning, and project-based learning.
2.3.2. Problem-solving activities and the focus of metacognition
The literature classies problemsolving into two distinct classes. Common labels for these classes are well-dened versus
ill-dened (Kitchener, 1983; Schraw, Dunkle, & Bendixen, 1995) and well-structured versus ill-structured (Jonassen, 1997;
Shin, Jonassen, & McGee, 2003). Well-dened problems are completely specied by the information given (initial state),
the end goal of the problem (goal state), the methods at hand (operators), and the space of possibilities (problem space)
(Newell & Simon, 1972). Examples of well-dened problems include proving logical theorems and completing puzzles such
as the Tower of Hanoi. Acharacteristic of well-dened problems is the existence of a correct or optimal path between the
initial and goal state. The focus of such problems is to make progress toward the goal state through sequentially applying the
operators tothe initial inputs. Students metacognitive processes (monitoring andregulating) focus onmoving along the path
without deviation, including removing obstacles. Thus, the focus of metacognition is to maximize positive characteristics
within initially (adequate) ways of thinking (Lesh et al., 2003, p. 387).
In contrast, in ill-dened problems, some of the information present in well-dened problems is missing. This can be the
initial state, the goal state, the operators for moving between states, and methods for achieving goals and subgoals (such
as removing obstacles). An important part of solving an ill-dened problemis nding or dening this missing information.
Thus, solving anill-denedproblemrequires extensive metacognitiontoformalize the informal, andtoevaluate the resulting
formalizations. Ill-dened problems typically involve complex real-world contexts, whereas well-dened problems tend to
look like traditional textbook word problems.
Y.R. Kimet al. / Journal of Mathematical Behavior 32 (2013) 377396 383
Table 1
Principles for guiding MEA development (Lesh et al., 2000).
Principle Description
Reality Requires the activity to be posed in a realistic mathematical context and to be designed so that the students can
interpret the activity meaningfully fromtheir different levels of mathematical ability and general knowledge
Model construction Ensures the activity requires the construction of an explicit description, explanation, or procedure for a
mathematically signicant situation
Model documentation Ensures that the students are required to create some formof documentation that will reveal explicitly howthey are
thinking about the problemsituation
Self-assessment Ensures that the activity contains criteria the students can identify and use to test and revise their current ways of
thinking
Generalizability Also known as the Model Share-Ability and Re-Usability Principle. Requires students to produce solutions that are
shareable with others and modiable for other closely related engineering situations
Effective prototype Ensures that the model produced will be as simple as possible, yet still mathematically signicant for learning
purposes (i.e., a learning prototype, or a big idea in mathematics)
Of course, many problems in everyday life may not be divide-able into the dichotomous categories indicated above, and
may involve different levels of complexity. The MEAs considered for this study do not completely t into the categories of
well-dened and ill-dened. They involve characteristics of both, a consequence of the six design principles to which every
MEA must adhere (Lesh et al., 2000; Moore &Diefes-Dux, 2004) (Table 1). The focus of problemsolving for MEAs is to develop
an effective method or model satisfying specic criteria for success and quality, rather than to identify the proper procedure
already existing (the focus of well-dened problems) or to nd or dene only one part of missing information (the focus
of ill-dened problems). Thus, the focus of metacognition when solving MEAs is to minimize negative characteristics of
current (inadequate) ways of thinking and develop beyond them (Lesh et al., 2003, p. 388).
2.3.3. Model-eliciting activities
The environmental considerations articulated in the prior section led us to consider MEAs for studying metacognition
on multiple levels. This section provides more details about MEAs and how they support the study of metacognition by
addressing several criticisms of self-report methods commonly used in research on metacognition, including: accessibility,
veridicality, retrieval issues, completeness, and reactivity due to articial setting (Ericsson & Simon, 1980; Garofalo & Lester,
1985; Goos & Galbraith, 1996; Schoenfeld, 1985; Wilson & Clarke, 2002).
MEAs are the most common type of problem-solving activity within the MMP (Lesh & Zawojewski, 2007). They are
complex, open-ended problems in which problem-solvers need to dene constructs for solving the task at hand (denition
building), then mathematize those denitions and develop arguments for why these mathematical constructs meet the
needs of the situation (operationalizing denitions). These types of problems encourage problem-solvers to think about
thinking, both their own and others, and to monitor or regulate possible alternative processes.
MEAs are problem-solving tasks related to real-world situations and involve the development or design of mathematical
models, where models are conceptual systems that describe, explain, or represent an experience, a complex series of experi-
ences, or another systemfor a purpose (English, 2008; Hamilton, Lesh, Lester, &Brilleslyper, 2008; Lesh&Doerr, 2003; Lesh&
Harel, 2003). An important attribute of MEAs is that they require students to develop a procedure, or model, to do something
for someone else, thus requiring students to document their thinking through speaking and writing (thus addressing the
accessibility of students mental processes). Students reveal their thinking about the given context based on their existing
knowledge or experiences and their development of newideas throughout participation in MEAs (thus addressing retrieval
issues: what students are saying is what they are thinking, rather than what they were thinking) (Diefes et al., 2004; English,
2008; Hamilton et al., 2008; Lesh & Doerr, 2003; Lesh et al., 2000). This enables tracing the conceptual process of students,
which provides opportunities to detect and repair students misconceptions (Hamilton et al., 2008).
MEAs require that student teams go through multiple cycles of revision, usually referred to as express-test-revise
cycles, where teams express their current ways of thinking about the problemsolution, test those ideas through information
provided by the problem or by other team members, and revise their thinking based on these tests. The nal models that
students develop are required to be transportable, reusable, and sharable (Hamilton et al., 2008; Lesh & Doerr, 2003), an
attribute that aids in the necessity of the revision cycles.
MEAs are fundamentally social, team-oriented tasks. In natural classroom settings (thus reducing the problem of reac-
tivity), three to four students in a group share their multiple perspectives to develop a model satisfying the needs of the
situation. This also addresses the issue of veridicality, because students in this environment spontaneously verbalize their
thoughts to achieve a common goal, rather than by request (English, 2008; Hamilton et al., 2008; Moore &Diefes-Dux, 2004).
Throughout the problem-solving processes in MEAs, students have to make several agreements. This collaborative aspect of
MEAs assures much more verbalization of students thoughts, which helps with completeness (Schoenfeld, 1985). Here, stu-
dents reveal their cognitive processes much more than in most learning environments due to the need to communicate their
thinking to one another in order to solve the problem. This provokes considerable metacognitive functions, with students
monitoring and regulating their own and each others thinking (Goos & Galbraith, 1996).
Workinginsmall groups, students alsodevelopcollaborationskills (Diefes et al., 2004; English, 2008; Hamiltonet al., 2008;
Moore & Diefes-Dux, 2004; Moore et al., 2006), which help teachers address the need for communication in mathematics
384 Y.R. Kimet al. / Journal of Mathematical Behavior 32 (2013) 377396
(NCTM, 2000). In this regard, MEAs are thought-revealing and collaborative activities (Lesh & Doerr, 2003; Lesh et al., 2000),
making themexcellent research sites in which to study our re-conceptualized denition of metacognition.
2.3.4. MEAs as the sources of metacognition at the environmental level
The individual, and especially the social levels of metacognition can potentially be revealed through solving complex
problems such as MEAs (Iiskala et al., 2004, 2011; Prins et al., 2006). MEAs are useful tools allowing students to engage in
and develop their metacognition because they are purposefully designed to provide enough information to allow students
to self-assess their understanding (individual) and require students to work in teams where multiple perspectives will be
presented (social) (Hamilton et al., 2008; Lesh et al., 2003; Magiera & Zawojewski, 2011).
MEAs are oftendesignedrequiring the needto dene nebulous constructs andoperationalize those denitions. Because of
the complexity and undirected nature of these processes, a teams rst solution attempt will likely be suboptimal. Therefore,
multiple cycles of revision, where students criticize each others thinking, are typically necessary. These cycles reveal how
students monitor and modify their own and others thinking in an active and dynamic way over the course of problem
solving (Lesh & Doerr, 2003; Lesh et al., 2003). The productivity of metacognitive functions provoked during these modeling
cycles varies because the focus of subtasks is different (Lesh et al., 2003). For example, during earlier cycles, brainstorming
may be more productive than assessment. The whole course of problemsolving provides students with a place to practice
and develop various metacognitive abilities (Lesh et al., 2003).
Students often have difculties evaluating their own solutions, but peers can help with this evaluation. When students
work in teams, they evaluate each others ideas, serving a metacognitive role for one another (e.g., Goos et al., 2002; Goos,
2002; Hurme, Merenluoto, & Jrvel, 2009). Multiple perspectives are typically useful when solving MEAs (Moore & Diefes-
Dux, 2004). MEAs lead to multiple solution attempts by students, and more sources for the development of metacognition
on the social level as teammembers criticize solution attempts that do not meet their own standards of goodness.
2.3.5. Task complexity, denition building and operationalizing denitions
Several research studies have identied task difculty in terms of task complexity as an important factor in the elicitation
of metacognition (Efklides, 2006; Helms-Lorenz & Jacobse, 2008; Iiskala et al., 2004, 2011; Prins et al., 2006; Vauras et al.,
2003). They suggest that metacognition tends to emerge more frequently in difcult versus easy tasks. Task difculty draws
onboththeconceptual andcognitivedemands of atask(Efklides, 2006; Stahl et al., 2006). Stahl et al. (2006) distinguishedtask
complexity fromtask difculty to refer to the cognitive demands of a task. For example, they argued that while memorizing
very difcult facts is simpler than applying an easy formula in terms of task complexity, the former could be much more
difcult than the latter because of the higher level of cognitive demand. During problemsolving, students can monitor task
complexity and regulate their goal setting and planning accordingly (Stahl et al., 2006).
We focus on task complexity as an environmental source for triggering metacognition. We assume that task complexity
involves both the conceptual and cognitive demands of a task. The cognitive demands of a task seemto be more contextual
thantheconceptual demands of ataskthat areafunctionof ones developmental level and/or of domain-specic knowledge
(Efklides, 2006, p. 6), and therefore draw on the individual conceptual systems. Research often classies the complexity of
tasks in terms of their processing complexity: how many steps are involved to get to a goal state, such as one-step versus
four-step addition, subtraction, multiplication and division problems (Iiskala et al., 2011). To classify task complexity, Stahl
et al. (2006) used the revised Blooms taxonomy (Anderson & Krathwohl, 2001), which provides a comprehensive set of
classications for learners cognitive processes of different complexity: remember, understand, apply, analyze, evaluate,
and create.
However, the levels of complexity in MEAs used for the current study cannot be classied in terms of their processing
complexity or the hierarchical taxonomy of cognitive processes. This is because, as shown in Section 2.3.3, multiple cycles of
revision (express-test-revise) are typically necessary for students to create transportable, reusable, and sharable models
(Hamilton et al., 2008; Lesh & Doerr, 2003; Lesh et al., 2003). Thus, we consider task complexity to refer to the different
levels in conceptual and cognitive demands of the problem-solving processes, denition building and operationalizing
denitions, required in MEAs. To differentiate the levels of complexity in solving problems, we look to the degree to which
problems require students to (1) build denitions and (2) operationalize those denitions. This analysis parallels the view
of metacognition as a management issue (Schoenfeld, 1987).
Denitionbuildingis theprocess bywhichproblem-solvers interpret givencontexts andconceptualizeproblems, building
meanings based on existing conceptual systems (Lesh & Doerr, 2003; Lesh & Harel, 2003). Denition building comes to the
forefront by putting students into situations requiring them to dene fuzzy ideas such as Which is the most rough? or
Who is the best volleyball player? These constructs are ill-dened. Roughness depends on the context in which you are
working; for example, blacktop used to pave streets may be rough to the touch, but may be smooth when driving on it. The
best volleyball player depends on multiple variables such as ability to serve, spike, jump, move quickly, and so on. These
examples illustrate situations where students must dene qualitative constructs before proceeding to solve problems.
Operationalizing denitions is the process by which a qualitative construct is made measurable in order to formalize
the goal of a problem. This process requires identifying evaluative criteria for what counts as a good or better solu-
tion. Denition building is inherently tied to this process, but these are separate ideas in problem solving. The process of
operationalizing denitions involves subtasks such as the following:
Y.R. Kimet al. / Journal of Mathematical Behavior 32 (2013) 377396 385
(1) quantifying qualitative information;
(2) converting all information to a homogeneous formof representational media;
(3) choosing variables that are consistent with the denition built;
(4) sampling fromdata to represent the given context; and
(5) choosing mathematical operations that are consistent with the denition built.
In business and industry, understanding and designing complex systems mainly involves the operationalizing of deni-
tions that are not straightforwardly mathematized.
In this study, we chose to use the terms denition building and operationalizing denitions in order to emphasize the
action involved compared to the more common terms of concept denitions and operational denitions (Harel & Koichu,
2010). For example, students in this study were required to dene the concepts for themselves within a particular context;
by contrast, concept denitions typically indicates that denitions are provided for students.
So far, we have described the theoretical model of metacognition in complex collaborative problem-solving activities
developedinthe current study, andthe rationale for using MEAs tostudymetacognition. The theoretical model of metacogni-
tionhelps make clear the distinctionof cognitive andmetacognitive behaviors, andthe distinctionof metacognitive functions
in the domain of problem solving. It is also expected to work as an effective basis to identify and interpret metacognitive
activities in collaborative modeling activities. Using the theoretical model as a framework and MEAs, a case study is designed
toexplore howmetacognitionfunctions during complex collaborative problemsolving, andhowit operates at the individual,
social, and environmental levels.
3. Methodology
3.1. Case study research design
A single-case naturalistic case study method was used to investigate metacognitive processes (Darke, Shanks, &
Broadbent, 1998; Yin, 2004). The purpose of the case study research was to explore in depth one teams individual and
social metacognition: how students thinking develops metacognitively while working within a group in a natural class-
roomsetting, in an MEA where both the denition building and operationalizing denitions processes were needed. Because
of these descriptive, naturalistic, and inductive characteristics of the study, qualitative research methods meet the needs
(Bogdan & Biklen, 2003; Creswell, 2006). Data were collected fromaudio recordings of the students in one teamworking on
the Paper Airplane Contest MEA. In an effort to establish rigor and credibility for the study, parameters such as triangulation
of data source and multiple researcher analysis was employed as described by Darke et al. (1998).
3.2. Setting
This study took place in a girls middle school located in a central part of South Korea during the 20082009 school
year. Two researchers implemented the Paper Airplane Contest MEA in a class of 32 female 8th grade students who had
mixed-level mathematics abilities. Eighteen of the students scored 20 or more points out of the total 30 on the 2009 National
Middle School Students Mathematics Exam, and four students did not pass the exam (the cut-off score is 13). Five of the
students failed in both the Computation and Word Problemsections. Two students did not reach the cut-off in the Function
section and seven failed the Geometry Problemsection.
The participants in this study were four team-members in the class. This team was chosen for this study because they
adoptedpositive attitudes duringtheir sessionandverbalizedtheir conceptualizationof problems over the course of problem
solving. The students worked on the MEA for about 135min including the time for a break and for lling out the consent
form. First, the students reada newspaper article that servedas anadvancedorganizer andthat engagedtheminthe problem
context. They also discussed a set of warm-up questions about the newspaper article for 20min as a whole class. Then, they
worked on the problemstatement involving the best oater problemand the most accuracy problemfor 70min. Finally, they
made short group presentations to share their solutions to the whole class for 20min. The implementation was videotaped
and each group of four students was audio recorded to explore student conversations throughout the MEA group activity
period. The group conversations of the four students in this study were translated fromKorean into English for analysis.
3.3. Paper airplane contest MEA
This studyusedanMEA entitledPaper Airplane Contest that was translatedfromEnglishintoKorean. This MEA provided
students with a complex problem-solving experience in which multiple levels of metacognitive functions were invoked. We
explore how the different sub-tasks within the MEA required different levels of conceptual and cognitive demands of the
two problem-solving processes, denition building and operationalizing denitions, and therefore how the sub-tasks
functioned differently as the sources at the environmental level for eliciting students metacognition at the individual and
social levels. The following sections contain an overviewof the problem.
386 Y.R. Kimet al. / Journal of Mathematical Behavior 32 (2013) 377396
Fig. 3. Landing points for four paper airplanes thrown by three pilots.
3.3.1. Math-rich newspaper article as the individual warm-up
The students were rst asked individually to read a math-rich newspaper article that (a) described how to make and
toss a variety of different types of paper airplanes and (b) provided information about the paper airplane contest. The article
described several problems that occurred at last years paper airplane contest. Some ight characteristics that were tested
were: (a) how far the planes ew, and (b) how long the planes stayed in the air. However, it proved difcult to judge some
of these characteristics because the planes performance depended on which pilots tossed them. For this reason, one year
later, the organizers of the paper airplane contest decided that three pilots should y each plane, and that the same three
pilots should y all of the planes. Then students individually answered several warm-up questions about the newspaper
article and discussed their answers as a whole class.
3.3.2. Teams creating a judging scheme for a paper airplane contest
Teams of four students then worked on the MEA problemstatement. They were asked to write a letter to the judges of
the paper airplane contest. The letter needed to provide a procedure which would allowthe judges to decide which airplane
was (a) the most accurate ier and (b) the best oater. We refer to these as the most accurate problem and the best oater
problem, respectively. The teams were givenTable 2, a sample of data froma trial contest to developandtest their procedure.
Fig. 3 is a graph of the data in Table 2, separating out the landing points for each plane regardless of pilot and the landing
points for each pilot regardless of plane. Both the table and the gure showthe results fromthe trial contest in which three
pilots ewfour different paper airplanes. The pilot stood at a point S(0, 0) on the oor, and their goal was to toss the planes
so that they came as close as possible to the point X(25, 25), which was the target for the ights.
3.4. Analysis methods
The resulting verbal protocols were parsedintothe problem-solving processes denitionbuilding andoperationalizing
denitions for each sub-task within the MEA. A ner grained analysis of conversational statements was then carried out
to identify each members behaviors, using a coding scheme consistent with the framework developed in Figs. 1 and 2. For
example, thecodingof NI (newidea) andIC(individual cognitivecomponents) was givenwhenpotentiallyuseful information
or alternative strategy was mentioned based on the individuals prior knowledge or experience. While coding the data, some
conversational statements t into two or more categories, so cross-indexing was used. For example, if a student assessed
her understanding of the problem and revealed a way of potentially useful thinking, and if the assessment and idea were
made based on her prior knowledge and experience but prompted by others way of thinking, then her response would
be coded by AU (assessment of understanding)/NI (new idea), and IL (individual level) & SL (social level). Table 3 shows
the coding scheme used in this study for deciding the distinction between thinking WITH and thinking ABOUT and the
distinction among the multiple levels. It also includes more detailed examples related to the distinctions and several notes
for the coding decisions made in this study.
To reduce coding errors, two researchers carefully read the transcript several times. Each researcher coded the data based
on the coding schemes as mentioned above that were agreed upon before data analysis. The Cohens K coefcient of inter-
rater agreement for coding the data was 0.94. This value is within a range that indicates an acceptable level of reliability
Y
.
R
.

K
i
m

e
t

a
l
.

/

J
o
u
r
n
a
l

o
f

M
a
t
h
e
m
a
t
i
c
a
l

B
e
h
a
v
i
o
r

3
2

(
2
0
1
3
)

3
7
7

3
9
6
3
8
7
Table 2
Information about four paper airplanes own by three different pilots.
Flight Pilot F Pilot G Pilot H
Distance from
start
Time in
ight
Distance to
target
Angle
from
target
Distance from
start
Time in
ight
Distance to
target
Angle
from
target
Distance from
start
Time in
ight
Distance to
target
Angle
from
target
Plane A 1 22.4 1.7 15.2 16 30.6 1.6 14.5 23 39 1.8 7.5 10
2 26.3 1.7 16.7 26 31.1 1.6 11.9 19 36.3 1.7 4.3 6
3 31.6 1.7 7.1 10 26.7 2.2 8.9 4 35.9 2.2 9 14
Plane B 1 32.1 1.9 7.6 11 35.9 1.9 14.3 23 43.7 2.0 9.5 6
2 42.2 2.0 9.2 9 39 2.1 11.1 16 29 2.0 7.6 7
3 27.2 2.1 10.2 11 25.6 2.0 11.7 12 36.9 1.9 12.4 19
Plane C 1 19.2 1.8 16.6 8 42.9 2.0 9.8 9 35.1 1.6 2.8 4
2 28.7 1.9 9.3 11 44.6 2.0 9.3 1 37.2 2.2 2 1
3 23.6 2.1 17.3 25 35.7 2.2 3.2 5 42 2.1 9.8 10
Plane D 1 28.1 1.5 8.9 9 37.2 2.1 20.2 32 41.7 2.2 10.1 11
2 31.6 1.6 14.8 24 46.6 2.0 11.4 2 48 1.9 14.1 8
3 39.3 2.3 9.1 12 34.7 1.8 22.2 36 44.7 1.7 11.5 9
3
8
8
Y
.
R
.

K
i
m

e
t

a
l
.

/

J
o
u
r
n
a
l

o
f

M
a
t
h
e
m
a
t
i
c
a
l

B
e
h
a
v
i
o
r

3
2

(
2
0
1
3
)

3
7
7

3
9
6
Table 3
Coding scheme for deciding the distinction between thinking WITH and thinking ABOUT and the distinction among the multiple levels.
Problem-solving behaviors Sources to which an individual has access
Distinction Description Individual Level Social Level Environmental Level
Cognitive activities:
Think WITH
Without any evaluation, an individual
thinks WITH a cognitive component:
Thinking WITH the cognitive
component due to oneself
Thinking WITH the cognitive
component due to others
Thinking WITH the cognitive
component due to a learning
environment
e.g., Given a set of numeric data,
student S1 thinks with knowledge of
the average to solve a problem.
e.g., Student S1s thinking with her
own knowledge of the average
e.g., Student S1s thinking with
knowledge of the average is
triggered by another students idea to
use average.
e.g., Student S1s thinking with her
own knowledge of the average due to
something in the problemthat
indicates that concept, such as What
is the average of the numbers?
Metacognitive Activities:
Think ABOUT
With evaluation, an individual thinks
ABOUT a cognitive or metacognitive
component:
Thinking ABOUT the cognitive or
metacognitive component due to
oneself
Thinking ABOUT the cognitive or
metacognitive component due to
others
Thinking ABOUT the cognitive or
metacognitive component due to a
learning environment
e.g., Given a set of numeric data,
student S1 thinks about knowledge of
the average as to whether it is proper
to solve a problem.
e.g., Student S1s thinking about her
own knowledge of the average is
triggered by her own realization of a
mistake.
e.g., Student S1s thinking about her
own knowledge of the average is
triggered by another student pointing
out a mistake.
e.g., Student S1s thinking about her
own knowledge of the average is
triggered by something in the problem
that makes her original solution not
viable, such as competing variables in
the problem.
Notes on howthe coding scheme is implemented:
1. The unit for analyzing verbal protocols is each comment made by an individual.
2. The coding decision for each comment is made on the basis of the overall scenario of students dialogs within the group rather than on the basis of each individual statement made.
Ex. (1) When student S4 led a problem-solving process by suggesting a newidea with the comment, We need to compare the speed to distance. . .dont we? (Statement 60, p. 39) This comment could be coded
at the individual level not because of the subject we, but due to the overall scenario.
Ex. (2) When student S1 made the comment, . . .we need to calculate this again. . . (Statement 61, p. 39) This comment could be coded as thinking WITH because her thinking still stuck in calculation without
any evaluation based on the overall scenario.
Ex. (3) When student S1 made the comment, Why is this furthest? (Statement 48, p. 34) This comment could be coded at both individual and social levels because her thinking ABOUT a way of thinking (the
individual level) occurred due to thinking ABOUT S2s way of thinking (the social level).
3. Non-verbal and verbal cues are considered as important factors in making coding decisions.
Ex. When students made comments with some cues of evaluation, such as I think. . . (Statement 1, p. 31), um. . . (Statement 33, p. 33), . . .dont we? (Statement 60, p. 39), Ah. . .compare. . . (Statement 61,
p. 39), and so on. The comments could be coded as thinking ABOUT because they indicate an evaluation occurring.
Y.R. Kimet al. / Journal of Mathematical Behavior 32 (2013) 377396 389
(Altman, 1991; Fleiss, 1981; Landis & Koch, 1977). The coding discrepancies were resolved via discussion and consensus so
that 100% agreement was reached.
The conversational statements presented here are assigned with a consecutive numbers. According to the coding results,
they are annotated to indicate students problem-solving behaviors, their monitoring and regulatory activities, and the levels
of sources that triggered them.
4. Results and discussion
Recall that the two research questions are:
(1) Howis metacognitionelicited throughthe denitionbuilding and operationalizing denitions processes during complex
collaborative problemsolving?
(2) How does metacognition operate during complex collaborative problem solving at the individual, social, and environ-
mental levels?
We rst address research question (1) by exploring how the complexity of a task (in this case, a subtask) can encour-
age students metacognition, especially focusing on denition building and operationalizing denitions. We then address
research question (2) by focusing on students metacognition on multiple levels as they participated in the MEA, particularly
howstudents monitored or evaluated their own thinking processes within the group, and howstudents thinking processes
were transferred to others through the teaminteractions.
4.1. Effect of task complexity on the construct of metacognition at the environmental level
This section focuses on the effect of task complexity on metacognition. It provides data from student interactions to
address research question (1). Recall that the MEA involves solving two problems, with regard to denition building and
operationalizing those denitions. The most accurate problem involves less complexity than the best oater problem, and
thus less social metacognition. This is perhaps because the phrase most accurate is somewhat clearer than the phrase
best oater for students to conceptualize based on their existing knowledge and experiences. For example, the word
accurate might remind students of words like target or aim, which bring to mind games of accuracy, and this elicits
prior experience. By contrast, the word oater could be associated in students mind with words such as levitate or
hover, which are much more nebulous for students to dene. In addition to the different levels in conceptual demands
of the problem-solving processes, another possible reason why the most accurate problem involves less complexity than
the best oater problem is the graphical representations with the target point X (Fig. 3). This might reduce the cognitive
demand of the problem-solving processes for the most accurate problem. These speculations about the complexity of the
different sub-tasks within the MEA were supported by the following description of the students problem-solving session
with transcript excerpts.
4.1.1. The most accurate problemas a source of metacognition at the environmental level
For the most accurate problem, students exhibited relatively clear goals, suchas dening the most accurate ier as closest
to target, which in turn led to a common solution path without many iterations needed to meet the goal. For the process
of denition building, they made a tacit agreement without much argument. All the students merely looked at the data to
nd which ier was most accurate without a specic discussion on what most accurate means (Statements 12, 14, 15, and
16 below). The students partially revealed their denition of most accurate as closest to the target in the conversations
for testing the accurate ier (Statements 18, 20, and 22). (In contrast, they clearly discussed the denition of the best oater
amongst themselves at the beginning of solving that problem, as we will see below.) The tacit agreement of what most
accurate means was illustrated as follows:
1. S1: I think rst we need to knowwhat our tasks are. (Assessmentunderstanding/Newidea: Thinking ABOUT a way of thinking,
Individual Level)
2. S2: Tasks?
3. S1: I mean. . .we need to knowwhat they (the judges) want us to do?
4. S2: Right, we should nd them. We rst need to nd which ight is the accuracy. (Assessmentunderstanding: Thinking ABOUT S1s way of
thinking, Social Level)
5. S1: We need to nd this. . .
Denition building for the most accurate problem:
6. S2: What is the accuracy?
[Students then worked for a fewminutes to try to gure out their overall goals for the task. The students also discussed what a denition of best
oater should be.]
Operationalizing dention for the most accurate problem:
390 Y.R. Kimet al. / Journal of Mathematical Behavior 32 (2013) 377396
12. S1: The criteria of the accuracy. . .we should knowthe criteria rst. . .so, I think we can use the graphs (Fig. 3) for that. (New idea: Thinking
ABOUT a way of thinking, Individual Level)
13. S2: Really!
14. S1: First, lets nd the accurate ier. . .then. . .what should be the criteria of (the accurate) ier?
15. S3: The criteria of (the accurate) ier. . .well. . . (Assessmentstrategy appropriateness: Thinking ABOUT S1s way of thinking, Social Level)
16. S2: The criteria are. . .? (Assessmentstrategy appropriateness: Thinking ABOUT S1s way of thinking, Social Level)
17. S3: What are they (the graphs in Fig. 3) saying?
18. S1: . . .um. . .Lets see. . .this has accurate. . .arrival point and ight path. . .(New idea: Thinking ABOUT a way of thinking, Individual Level)
19. Other Ss: Oh. . .yes. (Assessmentstrategy appropriateness: Thinking ABOUT S1s way of thinking, Social Level)
20. S1: (After) deciding the range. . .and we can see the ight in the range. . .(New idea: Thinking ABOUT a way of thinking, Individual Level)
[Students dened an in bounds and out-of-bounds range and were only considering the in bounds ights for accuracy.]
21. S3: What to do next? . . .(Assessmentstrategy appropriateness: Thinking ABOUT S1s way of thinking, Social Level &
Assessmentprogress toward goal, Individual Level)
22. S1: Then, in the graphs, we should nd the ight nearby the target. . .(Assessmentprogress toward goal, Individual Level & Social Level)
23. Other Ss: Ok. . .lets see. . .
24. S1: By the pilot. . .
25. S3: By the Pilots. . .
26. S1: When we see this graph (Landing points for the four planes, A, B, C and D in Fig. 3), closest to X (target) is C. . .C is three times in the
range. . . (Assessmentstrategy execution, Individual Level)
27. S2: No. . . (Assessmentstrategy execution: Thinking ABOUT S1s way of thinking, Social Level)
28. S1: Here. . .C, C, C. . .Three times. . .
29. S2: Aha! (Assessmentstrategy execution: Thinking ABOUT S1s way of thinking, Social Level)
[Students spent a fewminutes to explore the graphs.]
30. S4: Lets just calculate C (average of distance to target). (New idea: Thinking ABOUT a way of thinking, Individual Level)
31. S1: But. . .when we see the next graph (Landing points for the three pilots, F, G and H, in the Fig. 3). . .the most accurate (that is) the
closest (to target) is H and F, and the smallest angle is F. . .F and G. (Assessmentstrategy execution, Individual Level)
[One student (S1) reminded the group what the best oater means, and then the students went back to working on calculations for the most accurate.]
34. S1: The accuracy. . .
35. S2: The accuracy is . . ..we decide as H and C. . .
36. S1: Right. H is the oater (who threwthe accurate ier) . . .and C is most accurate. . .because the accuracy is the closest to target.
(Assessmentaccuracy of result: Thinking ABOUT S2s way of thinking, Social Level)
37. S3: We are done for one question.
The students did not question what most accurate meant at all. They quickly tried to nd the most accurate ier without
dening what accuracy in this context means (Statements 12, 15, and 16 above). The students had a clear common goal
in approaching this part of the problem, and they felt condent and certain of the solution path (e.g., Statements 20, 22, 26,
30, etc.). In other words, this problem, in which outcome goals seemto be apparent, gives students condence to monitor
their own problemsolving.
4.1.2. The best oater problemas a source of metacognition at the environmental level
This section illustrates how the best oater problem differentially worked as an environmental source for effectively
encouraging students metacognition at both the individual and social levels, compared to the most accurate problem. For
the notion of the best oater, each student had different denitions, such as the longest time (Statement 8 below), the
furthest distance (Statement 9), and the slowest ight (Statements 32 and 33). This is evidence of differences in the
individual conceptual systems based on prior knowledge and experience. The different level of complexity in the denition
building process for the best oater versus the most accurate problem elicited multiple perspectives from students. One
student (S1) made an initial denition of the best oater as the slowest ight (Statement 32) with group members
passive agreement in the beginning of the problem solving (Statement 33). The group had a concrete denition of the
problem at that time because no members argued about it anymore. The excerpts below show how the students reached
agreement on their rst denition of the best oater.
Denition building for the best oater problem:
7. S3: We also need to knowthe best oater.
8. S4: We. . .um. . .howabout nd the longest time? (Newidea: Thinking ABOUT a way of thinking, Individual Level)
9. S1: The furthest distance fromthe start and the shortest time in ight. . .(New idea: Thinking ABOUT a way of thinking, Individual Level)
10. S4: Dont we nd the longest time? (Assessmentstrategy appropriateness: Thinking ABOUT S1s way of thinking & Thinking ABOUT her
own way of thinking, Social Level & Individual Level)
11. S1: Ah. . .the longest time in ight. . . (Assessmentstrategy appropriateness, Social Level & Individual Level)
[Team spent about 10min talking about the most accurate problem]
Denition building for the best oater problem:
32. S1: We already decided the best oater as the slowest ight. . .right? (Assessmentprogress toward goal, Individual Level)
33. S2: um. . .the slowest ight. . .(Assessmentunderstanding: Thinking ABOUT S1s way of thinking, Social Level/Assessmentstrategy
appropriateness, Social Level, & Assessmentprogress toward goal, Social Level)
[Students returned to calculating average of distances to target.]
The initial denition of the best oater subsequently caused some controversies within the students in the group. They
repeatedly monitored and evaluated their conceptualization of the problemas individuals and against others feedback or
criticism. The following excerpt illustrates this.
Y.R. Kimet al. / Journal of Mathematical Behavior 32 (2013) 377396 391
Operationalizing dention for the best oater problem:
38. S2: Best oater?
39. S1: First, lets write down our decision. C oater and H pilot (for the accuracy). . .best oater. . . the furthest distance and the longest time
in ight. . .distance divided by time. . .(best oater) that speed is smallest. . . (Assessmentprogress toward goals, Individual Level, & Thinking
WITH knowledge of speed)
40. S2: To get the speed. . .they gave us the calculator. . . (Assessmentstrategy appropriateness: Thinking ABOUT S1s way of thinking, Social
Level, & Environmental Level)
41. Other Ss: Right. . .right. . .(Assessmentstrategy appropriateness: Thinking ABOUT S2s way of thinking, Social Level)
42. S1: Distance divided by time. . . (Thinking WITH knowledge of speed)
43. S2: Distance divided by time. . .? (Assessmentknowledge: Thinking ABOUT S1s knowledge of speed, Social Level)
44. S3: Lowest speed (for the best oater)? (Assessmentunderstanding: Thinking ABOUT understanding of the denition, Individual Level)
45. S1: Yes, the lowest speed. . .(Assessmentunderstanding: Thinking ABOUT S3s understanding of the denition, Social Level)
46. S1: So, we need to rst calculate the average of these three (distances fromstart and times in ight for each plane). (Assessmentprogress
toward goals: Thinking ABOUT Ss way of thinking, Social Level)
[Calculating average and then speed]
47. S2: Something strange. . .strange. (Assessmentsense of result, Individual Level)
48. S1: Why is this furthest? (Assessmentsense of result: Thinking ABOUT S2s way of thinking led thinking ABOUT a way of thinking, Social
Level & Individual Level)
49. S2: We are correct. But, (somethings) strange. . .(Assessmentaccuracy or sense of result, Individual Level)
50. S3: Whats the problem? (Assessmentunderstanding: Thinking ABOUT S2s way of thinking, Social Level)
[Students tried to gure out what caused the problem. They spent about 5min for re-calculation of speed.]
Operationalizing dention for the best oater problem:
60. S4: We need to compare the speed to distance. . .dont we? (New idea: Thinking ABOUT a way of thinking, Individual Level)
61. S1: Ah. . .compare. . .we need to calculate this again. . .(Assessmentstrategy appropriateness: Thinking ABOUT S4s way of thinking,
Social Level, & Thinking WITH a strategy, calculation)
62. S3: C has the longest time but. . .(Assessmentsense of result, Individual Level)
63. S2: Speed is . . .
64. S4: We need to compare the speed to distance. . .howabout that? (Assessmentprogress toward goals, Individual Level)
65. S1: Right. . .(Assessmentprogress toward goals: Thinking ABOUT S4s way of thinking, Social Level)
66. S2: We need to calculate distances again. (Thinking WITH a strategy, calculation of average)
67. S4: No, we already have distances on the data. (Assessmentstrategy appropriateness: Thinking ABOUT S2s strategy of average, Social
Level)
[For a fewminutes the group discusses their initial denition of best oater.]
Operationalizing dention for the best oater problem:
72. S1: But, in this case, the shorter distance is, the smaller speed is. . .uh. . .whats the problem? (Assessmentsense of result, Social Level &
Individual Level)
73. S3: The larger a denominator is. . .(Assessmentknowledge: Thinking ABOUT knowledge of fraction, Individual Level)
74. S2: The longest time is. . .
75. S4: The less time, the faster speed is. (Assessmentsense of result: Thinking ABOUT S1s way of thinking, Social Level, &
Assessmentknowledge: Thinking ABOUT knowledge of speed, Individual Level)
76. S1: But, if the time is longest and the distance is shortest, then speed is lowest. . .but distance is shortest. . .then we need to decide an
arbitrary distance for this. . .howabout this? [paused for a time] Lets nd speed with the arbitrary distance (a constant) after dividing all
of them, but howto decide the arbitrary distance? (Assessmentknowledge: Thinking ABOUT knowledge of speed, Social Level & Individual
Level, & New idea: Thinking ABOUT a way of thinking, Individual Level)
77. S4: Is it possible? (Assessmentstrategy appropriateness: Thinking ABOUT S1s way of thinking, Social Level)
78. S2: We have to have a condence for our strategy. (Assessmentstrategy appropriateness: Thinking ABOUT S1s way of thinking, Social
Level)
The students kept their own conceptualizations of the problem, such as the longest time in air, the furthest distance
fromthe start, and the slowest, during problemsolving. Based on these conceptualizations, they tried to operationalize
denitions to develop mathematical models for evaluating criteria for the best oater (e.g., Statements 39, 46, 60, 76, etc.
above). As a result, the students continuously conicted with each other on the processes of denition building and oper-
ationalizing denitions (e.g., Statements 47, 50, 62, 77, etc.). However, feedback and criticismfromeach other encouraged
the students to monitor and evaluate their thoughts during problemsolving (e.g., Statements 47, 48, 49, 50, etc.). This case
is a good exemplar of how MEAs can encourage students metacognition on the social level by promoting the interactions
with peers, encouraging themto go beyond their current ways of thinking (e.g., Statements 60, 72, 75, 76, etc.).
4.1.3. The sources at the environmental level for eliciting students metacognition
In sum, for the most accurate problem, students quickly approached the goal because they had a relatively clear common
goal without much doubt. When monitoring their own problem solving, students might expect an easy route to the goal
with tacit agreement. Thus, each student might depend on the individual level of metacognition, rather than welcoming
or needing social metacognition. This well supports the Staceys (1992) warning about group behaviors in collaborative
problem-solving settings. She cautioned that in the group situation, the attraction of simplicity is strong: Students who
see simple ideas produce simple answers quickly without any careful considerations of them. The most accurate problem
is relatively simplistic for denition building in comparison to the best oater problem, and therefore gives students con-
dence to monitor their own problem solving. However, sometimes this condence may prevent students from taking the
opportunity to engage in metacognitive processes on the social level. This is in line with previous metacognition research
(Artzt & Armour-Thomas, 1997; Stacey, 1992) which has shown that self-condent attitudes have contributed to the lesser
degree of metacognitive behaviors in collaborative problem-solving settings. There is less tendency for students to unpack
392 Y.R. Kimet al. / Journal of Mathematical Behavior 32 (2013) 377396
their multiple perspectives and make an argument-based negotiation, which may negate the need for social metacognition
even though the problemsolving is team-oriented.
However, for the best oater problem, each student had a perspective derived from their own conceptualization of the
problem, andtheconict aroundtheir conceptualizations encouragedstudents tocometoanagreement. Therst attempt the
group made was far fromoptimal, and therefore students criticized each others thinking. The best oater probleminvolves
more complexity in the problem-solving processes of denition building and operationalizing denitions, and consequently
it provided a sufcient source for metacognition at the social level, in addition to metacognition at the individual level.
Finally, one unexpected but interesting nding related to the sources of metacognition is the possibility of a negative
role of calculators as an environmental source for eliciting metacognition (see Statement 40). We provided calculators to
each student group as a complementary tool, even though they are not allowed in a regular math class in South Korea. This
existence of calculators in the classroomsetting might be able to reinforce the students belief that mathematics problems
call for a formal computation. Schoenfeld (1987) argued that this students wrong belief about what mathematics is all
about has a very strong negative effect on their mathematical behavior.
4.2. An analysis of how one students metacognition operated at the multiple levels
This section focuses on how one students interactions with the problem and with other group members became a
catalyst for thinking metacognitively at the individual level and the social level. Student S1 actively verbalized her thoughts
throughout much of the MEA, so she was chosen as the unit of analysis for this section. Student S1 took the lead in the activity
to reach solutions in both parts of the problem. However, she revealed a misconception about speed that was expected to
be used as the groups model to judge the best oater. Her misconception was that the furthest distance and the longest
time in ight result in the slowest speed (Statement 39 below). She was unable to monitor this misconception in her own
problemsolving until other students repeatedly criticized her thinking (e.g., Statements 47, 49, 54, etc.). This is illustrated
in the following excerpt:
32. S1: We already decided the best oater as the slowest ight. . .right? (Assessmentprogress toward goal, Individual Level)
33. S2: um. . .the slowest ight. . .(Assessmentunderstanding: Thinking ABOUT S1s way of thinking, Social Level/Assessmentstrategy
appropriateness, Social Level, & Assessmentprogress toward goal, Social Level)
[In the middle of working on the most accurate problem, S1 tried to remind the other group members of her denition of the best oater.
Students then returned their attention to calculations for the most accurate.]
[After a fewminutes]
38. S2: Best oater?
39. S1: First, lets write down our decision. C oater and H pilot (for the accuracy). . .best oater. . . the furthest distance and the longest time
in ight. . .distance divided by time. . .(best oater) that speed is smallest. . . (Assessmentprogress toward goals, Individual Level, & Thinking
WITH knowledge of speed)
[Calculating average and then speed]
47. S2: Something strange. . .strange. (Assessmentsense of result, Individual Level)
48. S1: Why is this furthest? (Assessmentsense of result: Thinking ABOUT S2s way of thinking led thinking ABOUT a way of thinking, Social
Level & Individual Level)
49. S2: We are correct. But, (somethings) strange. . .(Assessmentaccuracy or sense of result, Individual Level)
50. S3: Whats the problem? (Assessmentunderstanding: Thinking ABOUT S2s way of thinking, Social Level)
51. S2: We need to calculate it again? (Assessmentprogress toward goal: Thinking ABOUT Ss way of thinking, Social Level)
52. S3: No, there are differences fromeach result. . .(Assessmentaccuracy of result: Thinking ABOUT S2s way of thinking, Social Level)
53. S2: Is it a problem? (Assessmentaccuracy of result: Thinking ABOUT S3s way of thinking, Social Level)
54. S4: No. . .5.7. . .it is right. . .something strange. (Assessmentaccuracy or sense of result, Social Level & Individual Level)
55. S1: Ah, this is the problem! It is 17.21. . . So, D is the best oater. . .and. . .H. . .(Assessmentaccuracy of result, Social Level)
56. Other Ss: Why? Why? Why?
57. S1: Because it is the slowest speed. The best oater is the . . .slowest (Student S2 talked simultaneously). . .speed. . .
um. . .(Assessmentunderstanding, Social Level)
58. S3: Isnt (the best oater) the fastest oater . . .(Assessmentunderstanding, Social Level)
59. S2: (The best oater is) Slowest and farther oater. . .(Assessmentunderstanding, Social Level)
Student S1 slowly developed her thinking in reaction to other students conceptualizations (Statements 48 and 57 above;
Statements 61, 72, 76, and 84 below), even though she did not thoroughly evaluate her thinking, and could not correct the
misconception by the end of the activity (Statements 85 and 87 below). This is illustrated in the following excerpt:
9. S1: The furthest distance fromthe start and the shortest time in ight. . .(New idea: Thinking ABOUT a way of thinking, Individual Level)
10. S4: Dont we nd the longest time? (Assessmentstrategy appropriateness: Thinking ABOUT S1s way of thinking & Thinking ABOUT her
own way of thinking, Social Level & Individual Level)
11. S1: Ah. . .the longest time in ight. . . (Assessmentstrategy appropriateness, Social Level & Individual Level)
[The teamspent about 10min talking about the most accurate problem, and then spent about 13min working on the best oater problem.]
60. S4: We need to compare the speed to distance. . .dont we? (Newidea: Thinking ABOUT a way of thinking, Individual Level)
61. S1: Ah. . .compare. . .we need to calculate this again. . .(Assessmentstrategy appropriateness: Thinking ABOUT S4s way of thinking,
Social Level, & Thinking WITH a strategy, calculation)
[After calculating themagain]
62. S3: C has the longest time but. . .(Assessmentsense of result, Individual Level)
63. S2: Speed is . . .
64. S4: We need to compare the speed to distance. . .howabout that? (Assessmentprogress toward goals, Individual Level)
65. S1: Right. . .(Assessmentprogress toward goals: Thinking ABOUT S4s way of thinking, Social Level)
66. S2: We need to calculate distances again. (Thinking WITH a strategy, calculation of average)
Y.R. Kimet al. / Journal of Mathematical Behavior 32 (2013) 377396 393
67. S4: No, we already have distances on the data. (Assessmentstrategy appropriateness: Thinking ABOUT S2s strategy of average, Social
Level)
68. S1: Our denition of the best oater is the farthest distance and the longest time. . . (Assessmentprogress toward goals, Individual Level)
69. S2: Right! (Assessmentprogress toward goals: Thinking ABOUT S1s way of thinking, Social Level)
70. S1: So, thats the slowest speed. . .because distance divided by time. (Assessmentstrategy appropriateness, Social Level & Individual Level)
71. S2: Yes. . .then . . . ight C. . .(Assessmentstrategy appropriateness, Social Level & Individual Level)
72. S1: But, in this case, the shorter distance is, the smaller speed is. . .uh. . .whats the problem? (Assessmentsense of result, Social Level &
Individual Level)
73. S3: The larger a denominator is. . .(Assessmentknowledge: Thinking ABOUT knowledge of fraction, Individual Level)
74. S2: The longest time is. . .
75. S4: The less time, the faster speed is. (Assessmentsense of result: Thinking ABOUT S1s way of thinking, Social Level, &
Assessmentknowledge: Thinking ABOUT knowledge of speed, Individual Level)
76. S1: But, if the time is longest and the distance is shortest, then speed is lowest. . .but distance is shortest. . .then we need to decide an
arbitrary distance for this. . .howabout this? [paused for a time] Lets nd speed with the arbitrary distance (a constant) after dividing all
of them, but howto decide the arbitrary distance? (Assessmentknowledge: Thinking ABOUT knowledge of speed, Social Level & Individual
Level, & New idea: Thinking ABOUT a way of thinking, Individual Level)
77. S4: Is it possible? (Assessmentstrategy appropriateness: Thinking ABOUT S1s way of thinking, Social Level)
78. S2: We have to have a condence for our strategy. (Assessmentstrategy appropriateness: Thinking ABOUT S1s way of thinking, Social
Level)
79. S1: C (ier) by F pilot.
80. S4: Lets see that. . .
81. S1: Look back at the graph. . ..
82. S4: C is the accurate. . .the most accurate ier equals to the best oater. Right? (Assessmentsense of result, Individual Level)
83. S2: We can do like that. . .(Assessmentsense of result: Thinking ABOUT S4s way of thinking, Social Level)
84. S1: Lets think more. . .
[Students were writing the letter to the judges]
85. S1: The best oater is the problem. . .the highest speed is ne. . . (Assessmentsense of result, Individual Level)
86. S3: Longest time in ight. . . (Assessmentsense of result, Social Level & Individual Level)
87. S1: Ok. . .we can say that the best oater is measured by longest time and smallest speed. (Assessmentsense of result, Social Level &
Individual Level)
Slight but meaningful changes in student S1s thinking during problemsolving revealed howmetacognition on the social
level developed her thinking about speed in this context (Statement 76 above). This case illustrates howone person talking
aloudabout their conceptualizationof a problemcanprovide metacognitive feedbackfor another persons conceptualization.
In sum, student S1s thinking about her misconception exemplies how metacognition on both the individual and the
social level can be used to change current ways of thinking. First, her talking aloud about her conceptualization of the
problemdemonstrated howshe repeatedly monitored and evaluated her thinking by herself, at the individual level, during
the best oater problemsolving. Second, her talking aloud about her conceptualization of a problemalso provided feedback
for another students conceptualization. In this way, she served as a source for the development of others metacognition
at the social level. Finally, she was also encouraged to develop her own metacognition at the social level through feedback
fromothers, especially fromstudent S4 (e.g., Statements 60 and 75). To summarize, she was able to monitor and regulate
her thinking processes in the best oater problemat both the individual and social level, in ways that would likely not occur
during a less complex problem.
5. Implications and future directions
This micro-lensed study of metacognition addresses the sources for developing students metacognition. We re-
conceptualized metacognition at multiple levels, looking at the sources that trigger metacognition at the individual, social,
and environmental levels. By drawing on this re-conceptualization of metacognition and adopting a portion of Goos (2002)
framework, a theoretical model of metacognition in collaborative problemsolving was developed based on the models and
modeling perspectives (Figs. 1 and 2).
This study made three potentially important contributions to research in metacognition. First, it provided a coherent
model of metacognition with the potential to make more or less clear the distinction of cognitive (thinking WITH cognitive
components) and metacognitive (thinking ABOUT cognitive or metacognitive components) behaviors, and the distinction
of metacognitive functions in the domain of problemsolving, which are challenges for research on metacognition (Garofalo
& Lester, 1985; Wilson & Clarke, 2002). While Goos (2002) framework adopted a linear progression viewpoint of metacog-
nition, this model of metacognition developed in this study adopted a multiple dimensional progression viewpoint of
metacognition, representing the MMP viewof howproblemsolving occurs.
Second, this study addressed the need for theories that can bridge the cognitive and social realms to inform how an
individuals mind works in context (Schoenfeld, 1999). In particular, the re-conceptualization of metacognition on multiple
levels helps resolve the paradox of metacognition: It canexplainhowindividuals who are absent of self-regulationtranscend
their conceptual limitations when cognition goes awry by drawing on the conceptual systems of others.
Finally, this study addressed the lack of an authentic method to observe and analyze peoples mental processes, which is
another obstacle in the study of metacognition (e.g., Garofalo &Lester, 1985; Wilson &Clarke, 2002). In this study, we used a
Model-ElicitingActivityfor anauthentic methodfor verbal protocol analysis, as well as anenvironmental sourcefor triggering
metacognition. This study showed how MEAs could be substituted for self-report methods in research on metacognition,
in particular in collaborative problem solving, addressing several criticisms of self-report methods, including accessibility,
394 Y.R. Kimet al. / Journal of Mathematical Behavior 32 (2013) 377396
veridicality, retrieval issues, completeness, andreactivity due to articial setting (e.g., Ericsson&Simon, 1980; Goos &Galbraith,
1996).
This case study investigated howtwo subtasks of an MEA with different demands on denition building and operational-
izing denitions fostered the development of students metacognition on multiple levels. The results suggest the importance
of different levels of complexity in problem-solving tasks in designing metacognitive learning environments. On one hand,
problems that do not require complex negotiation to dene constructs, but that provide roomfor multiple conceptions give
students the condence to monitor their own problemsolving. Put differently, even though the problem-solving situation
is team-oriented, it may require metacognition only on the individual level. On the other hand, problems requiring students
to grapple with nebulous constructs and negotiate their individual and social meaning encourage metacognitive processes
on both the individual and social levels. Such problems require students to share their multiple perspectives, promoting
multiple solution paths, and in turn, requiring multiple solution attempts to meet goals with teamagreement. Thus, problem
solving may involve several iterative cycles of revision, demonstrating that more complex metacognitive functions are being
evoked.
In addition, this case study illustrated howone students (student S1s) participation in an MEA could work as a catalyst
for the development of individual and social metacognition within a team. This demonstrates that the problemof providing
metacognitive feedback can be off-loaded fromone individuals mind to anothers, and therefore implicates the importance
of social sources such as interactions with peers for improving metacognitive learning environments. Social sources enable
one to go beyond the individuals knowledge or regulation of cognition, which may support only limited metacognitive
processing, thus increasing opportunities to develop metacognition. These ndings enrich our understanding of how to
designinstructionthat fosters the development of metacognition. For example, it is important to consider problemsituations
requiring students to dene qualitative constructs and negotiate their individual and social meanings in order to create
metacognitive learning environments.
A limitation of the current study is that it is a case study; thus, the ndings are not generalizable. Another limitation is
that a small number of metacognitive functions (i.e., assessment and newideas) were present within the case study. Further
research, such as multiple case studies spanning a range of problems, is required for investigating additional metacognitive
functions. Further research is needed to understand metacognition on multiple levels, particularly what types of environ-
ments encourage metacognition on the social level. One smaller scale qualitative research question is, Howdo different types
of teaching strategies and styles (based on traditional view versus constructivists view) encourage students metacognition on
multiple levels? Another is, Howdo different external sources (teachers, peers, class artifacts, and technology) encourage students
metacognition on multiple levels? Answering these questions would set the stage for larger-scale research on how different
problems with different levels of complexity support the development of metacognitive abilities in problem solving. In
addition, we expect that the theoretical model of metacognition on multiple levels would allowfurther research to explore
developmental patterns of students metacognitive activities within and across several problem-solving sessions, dimen-
sions along which students metacognitive abilities develop, and critical events that facilitate or interfere with students
metacognition.
6. Conclusion
At the individual level, students are limited in their ability to self-monitor and self-evaluate their problemsolving. This
is due, in part, to the fact that their individual conceptual systems are based on prior knowledge and experiences. By having
students work together in a social setting, students in this study were able to access richer sources to potentially overcome
their individual limitations through feedback and criticismfromothers. The nature of the problem(i.e., an MEA rather than a
well-dened problem) also served as a metacognitive catalyst by requiring students to dene constructs and operationalize
those denitions. The results suggest that re-conceptualizing metacognition on multiple levelsthe individual level, the
social level, and the environmental levelis important for developing metacognitive activities and incorporating theminto
school curriculum. Finally, we hope that our theoretical model of metacognition provides a windowinto howan individuals
mind works in contexts, in particular the nature of students metacognitive behavior in collaborative problemsolving.
Acknowledgements
This material is based in part upon work supported by the National Science Foundation under Grant No. 0717529. Any
opinions, ndings, and conclusions or recommendations expressed in this material are those of the author(s) and do not
necessarily reect the views of the National Science Foundation.
References
Altman, D. G. (1991). Practical statistics for medical research. London: Chapman and Hall.
Anderson, L. W., & Krathwohl, D. R. (Eds.). (2001). A taxonomy for learning, teaching, and assessing: A revision of Blooms taxonomy of educational objectives.
NewYork: Addison Wesley Longman.
Artzt, A. F., & Armour-Thomas, E. (1992). Development of a cognitive-metacognitive framework for protocol analysis of mathematical problemsolving in
small groups. Cognition and Instruction, 9(2), 137175.
Y.R. Kimet al. / Journal of Mathematical Behavior 32 (2013) 377396 395
Artzt, A. F., & Armour-Thomas, E. (1997). Mathematical problem solving in small groups: Exploring the interplay of students metacognitive behaviors,
perceptions and ability levels. Journal of Mathematical Behavior, 16(1), 6374.
Bielaczyc, K., Pirolli, P., & Brown, A. L. (1995). Training in self-explanation and self-regulation strategies: Investigating the effects of knowledge acquisition
activities on problemsolving. Cognition and Instruction, 13, 221252.
Bogdan, R., & Biklen, S. (2003). Qualitative research for education: An introduction to theory and methods (4th ed.). NeedhamHeights, MA: Allyn & Bacon.
Borkowski, J., Carr, M., & Pressely, M. (1987). Spontaneous strategy use: Perspectives frommetacognitive theory. Intelligence, 11, 6175.
Bransford, J. D., Brown, A. L., & Cocking, R. R. (2000). How people learn: Brain, mind, experience, and school. Washington, D.C.: National Academy Press.
Brown, K. L. (2003). Fromteacher-centered to learner-centered curriculum: Improving learning in diverse classrooms. Education, 124(1), 4954.
Cardelle-Elawar, M. (1995). Effects of metacognitive instruction on lowachievers in mathematics problems. Teaching & Teacher Education, 11(1), 8195.
Carr, M., & Biddlecomb, B. (1998). Metacognition in mathematics: From a constructivist perspective. In D. J. Hacker, J. Dunlosky, & A. C. Graesser (Eds.),
Metacognition in educational theory and practice. Mahweh, NJ: Lawrence ErlbaumAssociates.
Chamberlin, S. A., & Moon, S. M. (2005). Model-eliciting activities as a tool to develop and identify creatively gifted mathematicians. Journal of Secondary
Gifted Education, 17(1), 3747.
Creswell, J. (2006). Qualitative inquiry and research design: Choosing among ve traditions (2nd ed.). Thousand Oaks, CA: Sage.
Darke, P., Shanks, G., & Broadbent, M. (1998). Successfully completing case study research: Combining rigour, relevance, and pragmatism. Information
Systems Journal, 8, 273289.
Diefes, H. A., Moore, T. J., Zawojewski, J., Imbrie, P. K., & Follman, D. (2004). A framework for posing open-ended engineering problems: Model-eliciting
activities. In Proceedings of the 34th annual ASEE/IEEE frontiers in education conference Savannah, GA, October 2023.
Duncker, K. (1945). On problemsolving. Psychological Monographs, 58, 1113.
Efklides, A. (2006). Metacognition and affect: What can metacognitive experiences tell us about the learning process? Educational Research Review, 1(1),
314.
Efklides, A. (2008). Metacognition: Dening its facets and levels of functioning in relation to self-regulation and co-regulation. European Psychologist, 13(4),
277287.
English, L. D. (2008). Introducing complex systems into the mathematics curriculum. Teaching Children Mathematics, 15(1), 3847.
Ericsson, K. A., & Simon, H. (1980). Verbal reports as data. Psychological Review, 87(3), 215251.
Flavell, J. H. (1976). Metacognitive aspects of problemsolving. In L. Resnick (Ed.), The nature of learning (pp. 231236). Hillsdale, NJ: Erlbaum.
Fleiss, J. L. (1981). Statistical methods for rates and proportions (2nd ed.). NewYork: John Wiley & Sons.
Garofalo, J., &Lester, F. K. (1985). Metacognition, cognitive monitoring, and mathematical performance. Journal for Research in Mathematics Education, 16(3),
163176.
Goos, M. (1994). Metacognitive decision making and social interactions during paired problem solving. Mathematics Education Research Journal, 6(2),
144165. Retrieved 30.05.11, fromhttp://www.merga.net.au/documents/MERJ 6 2 Goos.pdf
Goos, M. (2002). Understanding metacognitive failure. Journal of Mathematical Behavior, 21(3), 283302.
Goos, M., &Galbraith, P. (1996). Do it this way! Metacognitive strategies in collaborative mathematical problemsolving. Educational Studies in Mathematics,
30(3), 229260.
Goos, M., Galbraith, P., & Renshaw, P. (2002). Socially mediated metacognition: Creating collaborative zones of proximal development in small group
problemsolving. Educational Studies in Mathematics, 49(2), 193223.
Hadamard, J. (1954). An essay on the psychology of invention in the mathematical eld. NewYork: Dover.
Hamilton, E., Lesh, R., Lester, F., & Brilleslyper, M. (2008). Model-Eliciting Activities (MEAs) as a bridge between engineering education research and
mathematics education research. Advances in Engineering Education, 1(2), 125.
Harel, G., & Koichu, B. (2010). An operational detion of learning. Journal of Mathematical Behavior, 29(3), 115124.
Helms-Lorenz, M., & Jacobse, A. E. (2008). Metacognitive skills of the gifted froma cross-cultural perspective. In M. F. Shaughnessy, M. V. Veenman, & C. K.
Kennedy (Eds.), Metacognition: A recent review of research, theory, and perspectives (pp. 343). Happauge, NY: Nova Publications.
Hurme, T-R., Merenluoto, K., & Jrvel, S. (2009). Socially shared metacognition of pre-service primary teachers in a computer-supported mathematics
course and their feelings of task difculty: A case study. Educational Research and Evaluation, 15(5), 503524.
Iiskala, T., Vauras, M., &Lehtinen, E. (2004). Socially-sharedmetacognitioninpeer learning? Hellenic Journal of Psychology, 1(2), 147178. Retrieved 30.05.11,
from http://www.pseve.org/journal/UPLOAD/Iiskala1b.pdf
Iiskala, T., Vauras, M., Lehtinen, E., & Salonen, P. (2011). Socially shared metacognition of dyads of pupils in collaborative mathematical problem-solving
processes. Learning and Instruction, 21(3), 379393.
Jonassen, D. H. (1997). Instructional design models for well-structured and ill-structured problem-solving learning outcomes. Educational Technology
Research and Development, 45(1), 6594.
Jost, J. T., Kruglanski, A. W., & Nelson, T. O. (1998). Social metacognition: An expansionist review. Personality and Social Psychology Review, 2(2), 137154.
Kaplan, C. A., & Simon, H. A. (1990). In search of insight. Cognitive Psychology, 22, 374419.
Kitchener, K. S. (1983). Cognition, metaognition, and epistemic cognition: A three-level model of cognitive processing. Human Development, 4, 222232.
Kramarski, B., &Mevarech, Z. R. (2003). Enhancing mathematical reasoning in the classroom: The effects of cooperative learning and metacognitive training.
American Educational Research Journal, 40(1), 281310.
Landis, J. R., & Koch, G. G. (1977). The measurement of observer agreement for categorical data. Biometrics, 33(1), 159174.
Lesh, R., & Doerr, H. M. (2003). Foundations of a models and modeling perspective on mathematics teaching, learning, and problem solving. In R. Lesh, &
H. M. Doerr (Eds.), Beyond constructivism: Models and modeling perspectives on mathematics problemsolving, learning, and teaching (pp. 333). Mahwah,
NJ: Lawrence ErlbaumAssociates.
Lesh, R., & Harel, G. (2003). Problemsolving, modeling, and local conceptual development. Mathematical Thinking and Learning, 5(2 & 3), 157190.
Lesh, R., Hoover, M., Hole, B., Kelly, A., & Post, T. (2000). Principles for developing thought-revealing activities for students and teachers. In A. Kelly, & R.
Lesh (Eds.), Research design in mathematics and science education (pp. 591646). Mahwah, NJ: Lawrence Erlbaumand Associates.
Lesh, R., & Lamon, S. (1992). Assessing authentic mathematical performance. In R. Lesh, & S. J. Lamon (Eds.), Assessment of authentic performance in school
mathematics (pp. 1762). Washington, D.C.: American Association for the Advancement of Science.
Lesh, R., Lester, F. K., & Hjalmarson, M. (2003). A models and modeling perspective on metacognitive functioning in everyday situations where problem
solvers develop mathematical constructs. In R. Lesh, & H. M. Doerr (Eds.), Beyond constructivism: Models and modeling perspectives on mathematics
problemsolving, learning, and teaching (pp. 383403). Mahwah, NJ: Lawrence ErlbaumAssociates.
Lesh, R., & Zawojewski, J. (2007). Problemsolving and modeling. In F. K. Lester Jr. (Ed.), Second handbook of research on mathematics teaching and learning
(pp. 763804). Reston, VA: National Council of Teachers of Mathematics.
Magiera, M. T., & Zawojewski, J. (2011). Characterizations of social-based and self-based contexts associated with students awareness, evaluation, and
regulation of their thinking during small-group mathematical modeling. Journal for Research in Mathematics Education, 42(5), 486520.
Moore, T. J., & Diefes-Dux, H. A. (2004). Developing model-eliciting activities for undergraduate students based on advanced engineering context. In
Proceedings of the 34th annual ASEE/IEEE frontiers in education conference Savannah, GA, October 2023.
Moore, T. J., Diefes-Dux, H. A., & Imbrie, P. K. (2006). Assessment of teameffectiveness during a complex mathematical modeling task. In Proceedings of the
36th annual ASEE/IEEE frontiers in education conference San Diego, CA, October 2831.
Moore, T. J., & Hjalmarson, M. A. (2010). Developing measures of roughness: Problem solving as a method to document student thinking in engineering.
International Journal of Engineering Education, 26(4), 820830.
Muir, T., Beswick, K., & Williamson, J. (2008). Im not very good at solving problems: An exploration of students problem solving behaviors. Journal of
Mathematical Behavior, 27(3), 228241.
396 Y.R. Kimet al. / Journal of Mathematical Behavior 32 (2013) 377396
National Council of Teachers of Mathematics. (2000). Principles and standards for school mathematics. Reston, VA: The National Council of Teachers of
Mathematics, Inc.
Newell, A., & Simon, H. A. (1972). Human problemsolving. Englewood Cliffs, NJ: Prentice Hall.
Norman, D. A. (1981). Twelve issues for cognitive science. In D. A. Norman (Ed.), Perspectives in cognitive science (pp. 265295). Norwood, NJ: Ablex.
Prins, F. J., Veenman, M. V. J., & Elshout, J. J. (2006). The impact of intellectual ability and metacognition on learning: New support for the threshold of
problematicity theory. Learning and Instruction, 16(4), 374387.
Rasekh, Z. E., & Ranjbary, R. (2003). Metacognitive strategy training for vocabulary learning. Electronic Journal for English as a Second Language, 7(2), 115.
Salomon, G., & Globerson, T. (1989). When teams do not function the way they ought to. Journal of Educational Research, 13(1), 8999.
Schoenfeld, A. H. (1985). Mathematical problemsolving. Orlando: Academic Press.
Schoenfeld, A. H. (1987). Whats all the fuss about metacognition? InA. Schoenfeld(Ed.), Cognitive science andmathematics education(pp. 189215). Hillsdale,
NJ: Erlbaum.
Schoenfeld, A. H. (1999). Looking towards the 21st century: Challenges of educational theory and practice. Educational Researcher, 28(7), 414.
Schraw, G. (1998). Promoting general metacognitive awareness. Instructional Science, 26(12), 113125.
Schraw, G., Dunkle, M. E., & Bendixen, L. D. (1995). Cognitive processes in well-dened and ill-dened problem solving. Applied Cognitive Psychology, 9,
523538.
Shin, N., Jonassen, D. H., &McGee, S. (2003). Predictors of well-structured and ill-structured problemsolving in an astronomy simulation. Journal of Research
in Science Teaching, 40(1), 633.
Stacey, K. (1992). Mathematical problemsolving in groups: Are two heads better than one? Journal of Mathematical Behavior, 11(3), 261275.
Stahl, E., Pieschl, S., &Bromme, R. (2006). Task complexity, epistemological beliefs and metacognitive calibration: Anexploratory study. Journal of Computing
Research, 35(4), 319338.
Vauras, M., Iiskala, T., Kajamies, A., Kinnunen, R., &Lehtinen, E. (2003). Shared-regulation and motivation of collaborating peers: Acase analysis. Psychologia:
An International Journal of Psychology in the Orient, 46, 1937.
Volet, S., Summers, M., & Thurman, J. (2009). High-level co-regulation in collaborative learning: Howdoes it emerge and howis it sustained? Learning and
Instruction, 19(2), 128143.
Volet, S., Vauras, M., &Salonen, P. (2009). Self- andsocial regulationinlearning contexts: Anintegrative perspective. Educational Psychologist, 44(4), 215226.
White, B. Y., & Frederickson, J. R. (1998). Inquiry, modeling and metacognition: Making science accessible to all students. Cognition and Instruction, 16(1),
3117.
Wilkins, J. L. M., & Ma, X. (2003). Modeling change in student attitude toward and beliefs about mathematics. Journal of Educational Research, 97(1), 5263.
Wilson, J., & Clarke, D. (2002). Monitoring mathematical metacognition. In Paper presented at the annual meeting of the American educational research
association NewOrleans, LA, April 15.
Yin, R. K. (2004). Case study research: Design and methods (3rd ed.). Thousand Oaks, CA: Sage Publications.