A process analysis of heuristics use in games under time constraints

Leonidas Spiliopoulos
Max Planck Institute for Human Development

Andreas Ortmann
University of New South Wales

Le Zhang
Curtin University

Abstract
We test empirically the strategic counterpart of the ADM (Adaptive Decision Maker) hypothesis (Payne
et al., 1993), which states that decision makers adapt to increasing time pressure in predictable ways both
in information search and choice behavior. We find our conjecture that tightening time constraints in a
strategic context similarly aects players confirmed. Specifically, our subjects are more likely to ignore
others payos, thus transforming games eectively into non-strategic decision situations. At least for
our selection of strategic interactions this behavior is adaptive in the sense that there were no significant
payo losses resulting from the change in behavior. We use Bayesian latent class modeling to estimate
the frequency of heuristic use or types in the subject pool conditional on time constraints. Under time
pressure, we observe an increase in the proportion of non-strategic types (particularly Level-1) at the
cost of other strategic heuristics including Nash equilibrium choices.
Keywords: Response time; time pressure; experimental economics; procedural rationality; games;
strategic decision making; Level-k reasoning; Mouselab; process tracing

Email addresses: spiliopoulos@mpib-berlin.mpg.de (Leonidas Spiliopoulos), a.ortmann@unsw.edu.au (Andreas

Ortmann), Lyla.Zhang@curtin.edu.au (Le Zhang)

1. Introduction
In this paper, we will empirically test the adaptive decision-maker (ADM) hypothesis (Payne et al.,
1988, 1993) for strategic interactions. This hypothesis contends that decision strategies (or heuristics)
are chosen so that they match (and take advantage of) the structure/characteristics of the environment
and the task, one of which is the time available to make a decision. While the ADM hypothesis generated
significant interest in its application to individual decision making, significantly less work has been done
in the context of strategic decision-making (DM). Arguably, the ADM hypothesis may be even more
relevant for strategic DM as a strategic environment is typically more complex than a non-strategic one.
For example, in the former a DM must also grapple with uncertainty about strategic characteristics such
as the type or beliefs of their opponent and how these will aect the game outcome. Closely related to
the ADM hypothesis is the work conducted by Hogarth and Karelaia (2005, 2006, 2007) and Gigerenzer
(1988); Gigerenzer et al. (1999), inspired largely by the work of Herbert Simon, e.g., Simon (1956).
We test the ADM hypothesis in the context of strategic games by leveraging two mutually compatible research paradigms, which have not been simultaneously used before in the existing game theory
literature. The first paradigm is that of process-tracing and the second is that of response-time analysis.
Process-tracing refers to the collection and analysis of data regarding the processes used by DMs, such
as what information they acquire, how long they look at it and the order of information acquisition
see Schulte-Mecklenbeck et al. (2011) for an overview of process tracing and its application to decision
making. Response-time (RT) analysis refers to the analysis either of the (unconstrained) time taken for
a decision to be made or the impact of a time constraintsee Luce (2004); Svenson and Maule (1993)
for an overview. We will show that RT and information search analyses are invaluable tools for testing
the ADM hypothesis.
Economists and psychologists have traditionally diered in how they attempt to estimate individual
models of behavior. While process-tracing has a long history of use by cognitive psychologists, economists
have relied on the revealed preference framework to infer individual preferences by observing only their
choices. For example, Expected Utility Theory is an embodiment of this preoccupation with choices and
outcome-based rather than process-based modeling. In cognitive psychology, a greater emphasis has been
placed on process models and the collection of information about the actual decision-making process.
Several researchers have used Mouselab to uncover the underlying decision processes in strategic
interactionssee Crawford (2008) for a general discussion. Camerer et al. (1993); Johnson et al. (2002)
exploit the implied information search pattern of the backward induction algorithm to estimate the
relative contributions of social preferences and bounded rationality to subjects deviations from the
normative subgame-perfect solution. Mouselab has also been used to explore behavior in guessing games
(Costa-Gomes and Crawford, 2006) and normal-form matrix games (Costa-Gomes et al., 2001). Process
tracing using eyetracking has also been used more recently to explore consumer choice behavior (Krajbich
et al., 2010, 2012; Reutskaja et al., 2011) and normal-form matrix games (Devetag et al., 2015; Knoepfle
et al., 2009).
The use of response time has a long tradition in psychology dating back to Donders (1868)for contemporary advances see Luce (2004); Svenson and Maule (1993). Until recently, response time analyses
in economics were scarce. Awareness of the advantages of response-time analysis, however, is increasing
in experimental economicssee Spiliopoulos and Ortmann (2014) for a comprehensive overview of the
literature and a critical exposition of the value of RT for experimental economics. There are two main
research threads that dier in their treatment of RT. First, Rubinstein (2004, 2006, 2007) examine the
relationship between unconstrained (or endogenous) RT and behavior, i.e., without imposing any exogenous restrictions on RT. Second, Kocher and Sutter (2006); Sutter et al. (2003) examine the eects
of time constraints on behavior. Our paper addresses both of these research threads but leans toward
2

the second. We use the terms time pressure and time delay to refer to situations where players have
an externally imposed maximum time and a minimum time to make a decision respectively. The term
time constraints encompasses both time pressure and time delay, i.e., any case where response time is
not endogenously chosen by the player.
The paper is arranged as follows. Section 2 describes the experiment and Section 3 lists the hypotheses
and decision strategies (or heuristics) that we will study. Our results are divided into two sections. Section
4 includes descriptive statistics and basic analyses of our experimental data. In Section 5, we specify
a Bayesian mixture model of the types of players in our subject pool based on their heuristic use and
report the results. Finally, Section 6 concludes with an overview and critical discussion of the paper
including interesting extensions.
2. The experiment
2.1. Design
We implemented a between-subjects design for the three treatments: Baseline or NC (no constraint), Min45s, and Max20s. The only dierence between the three treatments is the type of time
constraint imposed on subjects. In the Baseline treatment (NC), participants were not constrained in
any way, i.e., they could spend as much time as desired to make their decisions.1 In the Max20s treatment, participants had to submit their choices within 20 seconds since the program would automatically
proceed to the next game after this. If they failed to make a choice within the time limit then they would
not get paid if that game was randomly selected for payment.2 In the Min45s, we forced participants to
stay in each game (i.e., delay the submission of their choice) for at least 45 seconds as the submission
button was inactive until then. After this they could spend as much additional time as they desired.
The Min45s treatment allowed us to assess whether subjects took seriously our exhortation not to rush
through decisions quickly in the NC baseline, i.e., this treatment also served as a robustness check. Our
key treatment is Max20s in which we imposed a time constraint that was determined from pilot sessions
to be binding.
Within each treatment, participants made choices in the same 29 games derived from four published
studies (Devetag et al., 2015; Haruvy and Stahl, 2004; Rey-Biel, 2009; Rydval et al., 2009) using the following criteria. The games must dierentiate between competing decision heuristics, thereby improving
identifiability. For example, the games from Rey-Biel (2009) were designed to separate Level-k heuristics
from the Nash equilibrium. The set of games must cover a wide range of dierent game classes (i.e.,
strategic characteristics) of importance to the existing literature. The game set includes coordination
games that permit us to explore equilibrium selection, games with unique and multiple equilibria, dominance and non-dominance solvable games, games that manipulate eciency and fairness in outcomes,
and both constant-sum and variable-sum games; the latter are required to dierentiate between MaxMax, MaxMin and Nash equilibrium behavior. From these 16 games we created a total of 29 games by
including both player roles from the asymmetric gamessee Table 1 for the complete list of games and
Table 2 for a summary of their characteristics.3 In pre-testing we determined that approximately 30
games could be completed within the targeted time frame of one hour net on average. The order of game
presentation was chosen to temporally separate the two instances of games created from the two player
1 We

acknowledge that there is an implicit time constraint for each session. Participants would not stay longer than 1.5
hours as required by the recruitment protocol. However, we knew that this implicit time constraint was not binding from
pilot sessions that we conducted.
2 If a player did not submit his/her choice within 20 seconds, we randomly drew a decision to calculate the payo of the
participant that is matched with him/her. Obviously we encoded this as a missing observation in the dataset.
3 We scaled the payos for some of the games compared to the original studies to ensure that the monetary incentives
for all the games were relatively similar in magnitude. Payos for games from Rey-Biel (2009) were scaled up by a factor
of 7 and we set M = 80 for the games from Rydval et al. (2009).

Table 1: The normal form games

Original study

Games

Rydval et al. (2009)

Game 1
40,40
0,80

Devetag et al. (2015)

Game 8
80,0
40,40

40,40
0,80
0,80

Game 12
60,60
45,35
35,5

35,45
45,45
35,35

60,60
35,45
35,35

5,35
35,35
35,35

60,60
60,35
35,5

Rey-Biel (2009)

35,25
50,25
10,15

60,35
60,60
35,5

35,20
5,55
10,20

35,49
35,49
35,49

35,30
5,85
40,25

80,20
5,55
10,20

Haruvy and Stahl (2004)

7,77
70,14
77,7

35,25
50,25
10,15

35,35
5,35
35,35

45,45
35,45
35,35

45,35
60,60
35,5

Games 6, 21
35,30
5,85
40,25

10,25
50,25
10,15

80,20
5,55
10,20

10,25
50,25
10,15

15,30
5,85
40,25

15,30
5,85
40,25

Games 11, 25
7,77
35,49
49,35

56,42
28,42
14,49

Games 7, 22
63,21
70,14
56,28

5,35
5,35
35,35

Games 13, 26

Games 4, 19
56,28
35,49
14,70

35,45
60,60
35,35

Games 5, 20

Games 3, 18

Games 10, 24
35,20
5,55
10,20

80,0
80,0
40,40

Games 2, 17

Games 9, 23
5,35
5,35
35,35

80,0
40,40
0,80

14,42
49,42
14,35

Games 16, 29
7,77
21,42
28,28

56,28
35,49
14,70

35,49
35,49
35,49

7,77
35,49
42,42

Games 14, 27
56,28
63,21
49,35

42,7
49,7
35,7

14,56
70,28
77,21

28,14
63,49
35,35

Games 15, 28
70,70
60,60
45,45

30,30
60,60
45,45

20,20
30,30
40,40

For asymmetric games, the first number refers to the game for the row player, the second number for
the column player.
Shaded cells denote the Nash equilibria.

Table 2: Game characteristics

Rydval et al. (2009)

Rey-Biel (2009)

Type
22
33
33

Devetag et al. (2015)

33
33
33
Haruvy and Stahl (2004) 3 3
cs=constant-sum,

# of NE # of games
1
1
1
1
1
1 (cs)
1
1 (vs)
1
1
1
2 (cs,vs)
3
1
3
3
1
4
3
1
vs=variable-sum

Total
1
1
2
2
2
4
1
6
8
2

roles of a single game, thereby precluding the possibility that players may realize that the same game
has been presented earlier. We will report all results on the basis of the 28 3 3 games, i.e., exclude the
2 2 Game #1. This game was presented first as a warm-up for subjects before proceeding to the 3 3
games. Note that Game #1 is the 2 2 strategic equivalent of Game #8, and the results were extremely
similar for these two games. Consequently, excluding the 2 2 game from the analysis does not aect
any results, and is done to avoid having to constantly report a second set of results for only one game.
2.2. Implementation
Students from the University of New South Wales were recruited using the ORSEE system (Greiner,
2015) in October and November 2014. We conducted two sessions for each treatment; the number of
subjects in the NC treatment was 50 (24+26), 48 (22+26) in Max20s, and 50 (24+26) in Min45s. Each
session took approximately one hour (one hour for the NC treatment, one hour and 10 minutes for the
Min45s treatment, and 50 minutes for the Max20s treatment). At the beginning of each session, the experimenter read aloud the instructions to ensure that every participant understood they were going to make
decisions under identical conditions. We stressed that rushing through the decisions did not make sense
since we could do the matching only after everyone was done. We urged participants to work through
the tasks carefully and in a focused manner. Subjects were asked to turn o their cell/smartphones and
we enforced this ban. After the instructions, participants completed an understanding test where they
were required to correctly complete a payo bi-matrix according to the description of the game and the
presented payo information. The experiment started when all participants figured out the payo matrix
correctly; we note that no subject failed this understanding test. At the end of the experiment, we asked
three volunteers to randomly draw three numbered cards that determined the games whose outcomes
would be paid out. The mean (and median) payment to subjects in Australian dollars was $20 ($21).
We conducted the experiment using Mouselab (Johnson et al., 1989), see Figure 1. Information
in each box of a payo matrix was hidden until participants moved the mouse pointer over that box
revealing the payo; the payo would be hidden again when the mouse was moved away from the box.
Mouselab records the lookup time and number of lookups per cell, and search order, i.e., the order with
which information was revealed.
Our experimental implementation was similar to that of Costa-Gomes et al. (2001). Each participant
was treated as a row player and called You, and their decisions were given abstract labels (#, *, %),
while the other participants decisions were also given abstract labels (&, @, !). As in Costa-Gomes
et al. (2001), to distinguish between decision strategies, the screen of the payo matrix was separated
into two parts: the participants payos (Your Points) were in the two/three left-most columns, and
his partners (Her/His Points) were in the two/three right-most columns.
5

Figure 1: Mouselab screenshots

(a) Baseline or no time constraint treatment (NC)

(c) Time pressure treatment (Max20s): 10-20s left

(b) Time delay treatment (Min45s)

(d) Time pressure treatment (Max20s): 5-9s left

(e) Time pressure treatment (Max20s): 0-4s left

The submission button looked dierent conditional on the treatment. In the Baseline treatment
(no time constraint), a submission button marked OK was shown at the bottom of the screen. In
the Min45s treatment, the submission button was not activated until 45 seconds after the start of that
game (Figure 1b). Finally, in the Max20s treatment, there was a progress bar reminding participants to
submit their choices within 20 seconds as play would automatically proceed to the next game after that.
The time left for a decision was clearly conveyed in two ways. First, numerically by a countdown timer
as in Figures 1c-1e. Second, graphically with a colored bar that cycled through three dierent colors
indicating the degree of time pressureblue indicated 10-20s left (Fig. 1c), orange indicated 5-9s left
(Fig. 1d), and red indicated 0-4s left (Fig. 1e). There were only six cases were subjects did not respond
on time during the Max20s time limit, i.e., only 0.5% of the choices in this treatment. We discovered
that the software had not recorded one observation (out of 1400 total observations) for a single game by
one subject in the NC baseline treatmentwe have encoded this as a missing observation in subsequent
analyses.
3. Hypotheses and behavioral modeling
Spiliopoulos and Ortmann (2014) presented a framework for adaptation to time constraints by extending the Miller (1960) framework to include game theoretic concepts. The hypotheses from this
framework that we will be testing are listed and described in Table 3. Before proceeding further we
define the following observable variables in our experiment. Let rt be the total response time from the
presentation of a game until the subject confirms an action. We sub-divide rt = lt + lt0 + dt into: time
spent looking up own payos (lt), time spent looking up an opponents payos (lt0 ), and time spent in
the choice area of the mouselab window or decision time (dt), i.e., non-information lookup time. We
consider information to have been acquired if it has been looked up at least once. We measure the degree
of attention to information by the mean lookup time per cell. These measures of information acquisition
and attention are not necessarily perfectly correlated to the weighting of information in the decision process (during information integration).4 However, due to memory constraints, a piece of information that
is deemed important is more likely to be repeatedly looked up (higher number of fixations) and viewed
for a longer lookup time. Since the true attention or weighting of information during the decision process
is generally not observable it can be concurrently deduced both from information acquisition/attention
and from the observed choices.
Hypothesis 1 The acceleration hypothesis states that decision processes are performed more quickly
in Max20s relative to NC. In our setup, this can be captured in the following ways: a) reducing
the lookup time per payo cell (Hypothesis 1a), and/or reducing the decision time, dt (Hypothesis
1b).
Hypothesis 2 The filtration hypothesis states that in Max20s less information is acquired (Hypothesis
2a) and the relative degree of attention shifts from opponents payos to own payos in terms of
information acquisition and lookup times (Hypothesis 2b) relative to NC.
Hypothesis 3 The strategy shift hypothesis states that in Max20s the distribution of subject types
(i.e., using a specific heuristic) is dierent from the distribution of types in NC.5

Table 3: A framework and associated hypothesis for adaptation to time constraints

Hypothesis
1 Acceleration
1a
1b
2 Filtration
2a
2b
3 Strategy shift

Explanation
A speeding up of existing decision processes
A reduction in the mean lookup time per payo cell
A reduction in the decision time (non-lookup time)
A shift in the information used and its role in arriving at a decision
The acquisition of less information, i.e., outcomes of fewer cells
A shift in the relative degree of attention to acquired information
A shift to a dierent strategy/heuristic

Table 4: The decision heuristics

Heuristics

Abbrev.

Description

Non-strategic
Maxmax
Maxmin

MaxMax
MaxMin

Choose the action(s) that includes the highest payo

Choose the action(s) that includes the highest
minimum payo
Choose the action(s) that is the best response to the
assumption that an opponent is choosing randomly

Level-1
Strategic
Social preferences
Social maximum
Equality
Bounded sophistication
Dominance-1

L1

SocMax
Eq
D1

Level-2

L2

Level-3

L3

Normative
Nash eq.

NE

Payo dominant Nash eq.

PDNE

Choose the action(s) that maximize the sum of own

and opponents payos
Choose the action(s) that minimizes the dierence
between own and opponents payos
Choose the action(s) that is the best response to the
assumption that an opponent is choosing randomly
over his/her undominated actions
Choose the action(s) that is the best response to the
assumption that an opponent is L1
Choose the action(s) that is the best response to the
assumption that an opponent is L2
Choose the action(s) consistent with the Nash
equilibrium
Choose the action(s) consistent with the payo
dominant Nash equilibrium

3.1. The decision heuristics

Our analysis includes ten strategic heuristics that we identified from the existing literature as reasonable candidates, ranging from normative solutions (e.g., Nash equilibrium) to bounded rational solutions
(e.g., Level-k heuristics or cognitive hierarchy theory) to relatively naive solutions such as maximizing the
maximum own payo (MaxMax). Level-k heuristics and cognitive hierarchy theory have been successful
in earlier attempts to model bounded rationality in the literature, e.g., (Camerer et al., 2004; CostaGomes et al., 2001; Nagel, 1995; Stahl and Wilson, 1994, 1995). Table 4 lists the decision heuristics and
describes how they operate. We distinguish between two sets of heuristics based on their information
requirements. Non-strategic heuristics require knowledge only of the set of players own payos , not
their opponents payos 0 . Strategic heuristics require information about both own and opponents payos to prescribe an action. Strategic heuristics can be further subdivided into heuristics that incorporate
social preferences, heuristics with levels of bounded sophistication and normative solutions. Heuristics
can also dier in the computational cost of integrating and processing information. For example, SocMax requires the relatively simple calculation of summing own and opponents payos for every possible
outcome of the game. However, the calculations required to compute the Nash equilibrium are arguably
more complex and demanding. Consequently, the heuristics dier in the amount of time they require
both in terms of information acquisition and processing. Therefore, our main hypothesis is that under
time pressure subjects will shift to using simpler, and consequently faster, heuristics.
4. Descriptive statistics and basic results
As expected, the results in the NC and Min45s treatments were mostly consistent in terms of economic
and statistical significance. We will focus in the main text on comparisons between the NC and Max20s
treatments, thereby highlighting how subjects adjust to time pressure. However, all tables and figures
will include the results from all three treatments, and we will include more detailed results about Min45s
in footnotes.
4.1. Choice proportions
Before proceeding with the behavioral modeling, we first investigate how choice proportions in each
game are aected by the Max20s treatment compared to the NC baseline. Fisher exact tests on the
choice proportions calculated separately for each game reveal the following. In eight of the 28 games
there was a statistically significant dierence at the 5% level and the probability of finding at least eight
significant results from 28 tests is p = 5 10

. We are also interested in the eect size of these changes

in proportions, i.e., how large they are and whether they are economically significant. The eect size
P3
0
between two dierent treatments T and T 0 is defined as 1/3 i=1 | pTi,g pTi,g | where pTi,g denotes the
probability of choosing action i in game g. The median absolute change in the probability of playing an

action is 0.09 (10 percentage points) and the 95th percentile ranges from 0.01 to 0.23; we consider these
to be economically significant changes in behavior.6
4 For example, information that has been acquired may be completely ignored or severely diminished in importance
during the decision process.
5 This may be driven either by filtration performed in the information acquisition phase (which reduces the set of
heuristics that can be applied) or by integrating the same information in a dierent manner.
6 Comparing NC to Min45s, we find only one game with choice proportions that are significantly dierent at the 5%
level; the probability of finding at least one significant result out of 28 games is p = 0.76. The median absolute change in
choice proportions is 0.04 and the 95th percentile ranges from 0 to 0.11. We conclude that imposing the Min45s constraint
on subjects only weakly aected behavior relative to the NC treatment.

4.2. Response time and information search

Figure 2 displays the distributions of rt, lt, lt0 , dt for every treatment and Table 5 documents the
mean times, number of fixations of own and opponent payo cells (f and f 0 respectively) and lookup
time per fixation. We have restricted the maximum of the domain for rt in the figures to 100s for
practical purposes; 34 observations ranged from 100s to 221s in the NC treatment and 18 observations
ranged from 100s to 227s in the Min45s treatment.
Table 5 compares the components of rt across the treatments. Average response time was 44.3s in the
NC treatment and fell substantially to 13s in Max20s. We perform non-parametric Wilcoxon rank-sum
tests for every possible pairing of treatments for each time measure and find that they are all statistically
significant at p < 0.0001taking into account multiple-comparisons all results would still be significant
at a level of p < 0.01.
Figure 2: Distribution of response time and components

Density

Response time (rt)

Decision time (dt)

0.3

0.3

0.2

0.2

0.1

0.1

20

40

60

80

100

Density

Own payo lookup time

0.3

0.2

0.2

0.1

0.1

20

40
60
Time (s)

40

60

80

80

100

NC

Max20s

20

40
60
Time (s)

80

Min45s

Table 5: Mean response/lookup times and fixations across treatments

rt

dt

lt

lt0

100

Opponent payo lookup time

0.3

20

f0

lt/f

lt0/f 0

p (f = 0)

p (f 0 = 0)

NC
44.3 10.5 17.9 15.9 32.9 29.1 0.54 0.54
0
0.05
Max20s 13.0 4.8
5.7
2.5
14.5 6.9
0.39 0.38
0
0.27
Min45s 54.1 16.3 20.1 17.7 36.1 32.2 0.57 0.56
0
0.014
rt =response time, dt =decision time, lt (lt0 ) =lookup time for own (opponent) payos
f (f 0 ) =number of fixations for own (opponent) payos
10

100

Result 1a (Hypothesis 1a: Confirmed) Accelerationa decrease in the mean lookup times per cell
is observed in subjects behavior under time pressure (Max20s).
Compared to the NC baseline, subjects in Max20s significantly decreased both the total time spent
looking at their own (17.9s versus 5.7s) and opponents payos (15.9s versus 2.5s). The mean lookup
times per cell by treatment are displayed in Figure 3. For all cells, the mean lookup time is less in
Max20s than NC, and greatest for Min45s. The probability that the mean lookup times for all cells
move in the same direction across two dierent treatments is 0.518 = 4 10

Figure 3: Mean lookup times per cell (s)

Own choices

# 2.67

% 1.66

Own choices

&

Max20s

1.88

1.89

2.32

1.79

1.76

2.11

1.89

1.76

1.81

1.83

1.82

1.93

1.64

1.43

1.55

!
&
@
Opponent choices
Min45s

# 2.86

2.04

2.08

2.36

1.96

1.84

2.23

2.34

2.22

2.03

2.08

1.94

% 2.01

2.04

2.31

1.93

1.74

1.77

&

!
&
@
Opponent choices

Own choices

NC
# 0.75

0.6

0.6

0.34

0.27

0.26

0.68

0.7

0.6

0.32

0.31

0.27

% 0.56

0.59

0.58

0.28

0.25

0.25

&

!
&
@
Opponent choices

The first 3 columns correspond to own payos, the latter 3 columns to the opponents payos
Result 1b (Hypothesis 1b: Confirmed) Accelerationa decrease in decision timeis observed in subjects behavior under time pressure (Max20s).
Decision time fell drastically (by approximately 54%) from 10.5s in NC to 4.8s in Max20s. Consequently,
subjects spent less time integrating the information they had acquired to arrive at a decision under time
pressure.
Result 2a (Hypothesis 2a: Confirmed) Significant filtrationa reduction in the level of information
acquisitionis observed in subjects behavior under time pressure (Max20s)
Table 6 documents the percentage of payo cells that were opened at least once per game. Subjects
accessed the most information in Min45s, followed by NC and finally Max20s. The dierence between
the latter two is strikinga reduction of 21.1% points of acquired information in Max20s.
11

Table 6: Information acquired (% of payos) across treatments

NC
Max20s
Min45s

96.6
91.7
98.2

0
91.9
54.7
96.9

and 0
94.3
73.2
97.6

Result 2b (Hypothesis 2b: Confirmed) Significant filtrationa shift towards the acquisition of relatively more information (and relatively higher lookup times) for own versus opponent payosis
observed in subjects behavior under time pressure (Max20s).
Subjects filtered information about their opponents payos significantly more than their own. While
own payo acquisition decreases only slightly in Max20s compared to NC (from 96.6% to 91.7%) the
change is significantly larger for the opponents payos (91.9% to 54.7%)see Table 6. This means that
many subjects eectively ignored the strategic aspect of these games and treated them as individual
decision-making tasks.7 Also, lt0 fell proportionately more than lt (84% versus 68%) leading to relatively
more attention being directed at own payos under Max20s. The sharp decrease in lt0 was associated
with 27% of decisions being made without consulting any of an opponents payos. Note, the decline in
lt and lt0 is driven both by a decline in the mean number of fixations f and f 0 , and a decline in the mean
lookup time per fixation. Furthermore, the probability that there exists at least one own payo cell that
was not looked up is 0.13, 0.30, 0.08 in NC, Max20s, Min45s respectively. The probability that at least
one opponent payo cell was not looked up is 0.15, 0.70, 0.07 in NC, Max20s, Min45s respectively.
Readers knowledgable with the process tracing literature will notice that we have only analyzed search
in terms of information acquisition, and have not used search order, i.e., the order in which information
is acquired. Search order has been extensively used in prior studies to further identify decision rules
some assumptions are required to infer what a typical search order should be for each decision rule,
e.g., forgetting implies that specific payo comparisons that are necessary for a decision rule should
be temporally adjacent. The results above suggest that information search will not be as informative
in our case, and in fact will be quite dicult to identify. The reason is that, especially under time
pressure, many payo cells were not looked up at all. Recall that in Max20s for 70% of the decisions
there exists at least one opponents payo cell that was not searched, and 27% of decisions were made
without consulting any of an opponents payos. Consequently, matching decision rules to information
search over partially searched games (with significant heterogeneity in which cells are excluded across
games/players) is not only conceptually dicult but will be much less informative and conditional on ad
hoc rules imposed by us to deal with this.
4.3. Expected payos
We investigate whether there is a relationship between rt and the expected (population-matched)
payos in the NC treatment. We define expected payos as the average payos a subject would have
received had s/he played the whole population of subjects. Each observation is the expected payo
for a subjects choice in a specific game. In general, very little can be said a priori about the speedperformance relationship in strategic games as it depends not only upon how players adapt to time
pressure but also on the characteristics of the game. Consequently, this relationship may be positive,
negative, or non-monotonic.8 We use a fixed-eects specification to capture across-game heterogeneity in
7 While subjects acquired the largest proportion of and 0 information in Min45s, the level of information acquisition
is approximately the same as in NC.
8 Consider the following game and assume that under no time constraints players both use the L2 heuristicpayos are
10 and 40. Now assume that under time pressure, both players revert to a lower level heuristic, namely L1both players
are now better o, achieving payos of 60 each. Note that both players using the L3 heuristic leads to a worse outcome

12

Table 7: The relationship between rt and expected payos

Coef.

Std. err.

95% conf. interval

rt
2
(rt)
c

0.031
-0.00017
43.33

0.0127
0.00067
0.43

2.40
-2.51
101.04

0.016
0.012
0.000

0.006
-0.0003
42.49

F (2, 1369) = 3.22

p = 0.04

0.055
-0.00004
44.18

38

Expected payoffs (population matched)

40
42
44

46

Figure 4: The relationship between rt and expected payos

20

40

60

80

100
120
140
Response time (s)

160

180

200

220

expected payos. The independent variables are the response time taken for the decision and its square to
allow for a non-linear relationship.9 Table 7 presents the results of the regression. There is a statistically
significant non-linear (inverse-U shaped) relationship between expected payos and rtsee also Figure
4. However, the eect size is not particularly large, especially for the range of response times with the
majority of observations (and consequently, narrower confidence intervals). The statistical significance
appears to be driven by the very high end of response times, i.e., the outliers, which likely includes
observations of subjects that were not particularly focused or attentive to the task. We conclude that
there is an economically insignificant relationship between performance and response times, particularly
over the range of the most frequently observed RTs.
The mean expected payos are 44.2, 43.0 and 43.2 for the NC, Max20s and Min45s treatments
for both players compared to L1, i.e., the relationship is non-monotonic.
Player 2
L
C
R
Player 1
U
60,60L1
45,35L3
80,7L4 =NE
M
90,35
30,85
10,40L2
D
40,95
20,40
50,25
Superscripts denote the outcomes of both players using a Level-k heuristic (Lk) or the Nash equilibrium (NE).
9 A quadratic relationship in rt
(F (1, 1370) = 0.12, p = 0.73).

was employed as a linear relationship failed to account for the data

13

Table 8: Expected payos from simulated rematching

NC

Max20s

Min45s

NC

Max20s

Min45s

NC

Max20s

Min45s

2
3
4
5
6
7
8
9
10
11

55.6
33.6
35.2
45.4
28.6
60.2
40.0
54.6
32.7
27.1

45.1
37.0
32.0
43.8
26.3
61.1
40.0
48.3
31.5
30.7

49.6
36.5
34.7
43.5
28.3
61.0
40.0
50.1
34.5
25.5

12
13
14
15
16
17
18
19
20
21

46.0
28.8
59.6
64.8
34.9
55.3
51.2
48.8
46.0
46.6

44.2
25.9
62.3
62.7
33.9
46.3
52.9
52.0
43.8
51.9

43.4
30.0
59.3
63.8
34.1
51.9
49.8
49.3
42.4
46.7

22
23
24
25
26
27
28
29

23.8
54.9
32.4
46.8
29.7
40.9
64.8
49.1

22.9
47.7
34.9
45.1
30.9
36.5
62.7
50.1

23.0
51.7
30.9
45.1
27.9
42.3
63.8
49.9

Mean

44.2

43.0

43.2

respectivelysee Table 8 for a breakdown of the expected payos per game. A Wilcoxon signed-rank
test does not reject the null hypothesis of no dierence in the expected payos per game across the NC
and Max20s treatments (z = 1.36, p = 0.175).10 Of course, the sensitivity of expected payos to response
time is mediated by the curvature of the payo function of our games. If this is relatively flat, then we
would not expect payos to change significantly when behavior changes due to time pressure. We present
two statistics that provide evidence that the flat-maximum critique is not applicable. First, we calculate
the standard deviation of the row player payos per game and then average across gamesthe value is
20.1. A second more conservative estimate performs the same calculations except instead of using all
nine row player payos per game, it uses only the maximum payo per action per gamethe value is
17.8. Both of these estimates of the curvature of the payo space are at least an order of magnitude
greater than the observed changes in payos across treatments. Consequently, subjects have responded
to a strong time constraint (that eectively halved the time they had to make a decision compared to
NC) in an eective manner that had only a marginal impact on their earnings. This evidence supports
the notion of an adaptive DM, and that the observed acceleration/filtration that we documented in the
previous section was an eective response to time constraints.
4.4. Dominance
In this section, we investigate how the treatments aect the rate of violation of dominance principles
in ten games that had either a (weakly) dominant strategy or a dominated strategy. Table 9 presents
this information.
Table 9: The percentage of dominated choices across games and treatments

Game #
NC
Max20s
Min45s

8wdnt 18dnt 21dnt 24dnt 26dnt 7ded 22ded 14ded 27ded

10.2
40
48
14
16
2
4
2
0
29.2? 45.8
45.8
18.8
22.9
12.5 6.4
2.1
0
22
54
46
6
16
10
2
6
0
?
Statistically dierent at the 5% level from the NC treatment
(w )dnt
=(weakly) dominant strategy, ded =dominated strategy

All
15.3
20.4?
18.0

Result: We find moderate evidence that time-pressure increases the likelihood of choices violating the
dominance principle. The pooled violation rate for all games (with non-zero violation rates, i.e., excluding game #27) is significantly lower in NC compared to Max20s according to a two-sided exact Fisher
test, p = 0.05. For seven out of the eight games with non-zero violation rates, the percentage of choices
10 The null hypothesis of no dierence in the mean expected payos in the NC and Min45s is rejected (z = 2.47, p = 0.014).
However, the economic significance of this dierence is smallthe expected mean payos in Min45s are only 2% (or 0.9
points) lower than NC.

14

violating dominance were greater in treatment Max20s compared to NCthe probability of observing at
least seven outcomes is 0.035 (Binomial test). However, only the individual dierence in game #8 from
Rydval et al. (2009) was statistically significant at the 5% level using a two-sided exact Fisher test. If
we exclude this game, then we do not reject the assumption of dierent violation rates (p = 0.21). We
conclude that there is moderate evidence that in Max20s subjects violated the dominance principle more
often than in NC, but qualify that this result is driven primarily by one game. Larger sample sizes (and
therefore more statistical power) are required to reach stronger conclusions.11
4.5. Heuristics and actions
The proportion of actions in each treatment that are predicted by the heuristics is presented in Table
10. We have performed two adjustments to these adherence rates. The first was to deal with the fact that
heuristics can sometimes make a non-unique prediction in a game. We penalize for non-unique heuristics
by encoding success in such cases not as a value of 1, but as a value of n

, where n is the number of

predictions made by the heuristic. Furthermore, we make use of the information search information and
encode a strategic heuristic as not consistent if f 0 = lt 0 = 0.
The L1 heuristic had the highest adherence rate on average (59%) and specifically in the NC and
Max20s treatments, and the second-highest adherence rate in Min45s. D1 had the second highest adherence rate on average (51%) and was virtually identical to the adherence rates of L1, bar the Max20s
treatment where it fell significantly. The L3 heuristic and PDNE exhibited the next highest adherence
rates on average (48% and 47% respectively); however, both fall to near random adherence (33%) in the
Max20s treatment. In Max20s, non-strategic heuristics perform relatively better than in the NC and
Min45s treatments. For example, the second highest adherence rate in Max20s was that of MaxMax
(44%), followed by MaxMin (40%).
Despite the fact that we have adjusted the adherence rates for the number of predictions and information search, we prefer the structural econometric model presented in the next section for numerous
reasons. Model comparisons without a stochastic specification for choices using only deterministic adherence rates at the global level can be problematic in identifying the true use of heuristics, e.g., Broomell
et al. (2011); Davis-Stober and Brown (2011); Hilbig (2010). Firstly, adherence rates are compromised
by the fact that heuristics often make the same predictions, despite the choice of specific games with the
identifiability of heuristics in mind.12 Secondly, adherence rates implicitly assume deterministic use of
a heuristic without any errors or noise, i.e., models are misspecified. Finally, adherence rates implicitly
disregard probabilistic information, i.e., the strength of preference, by measuring the performance of a
heuristic as a 0-1 variable. While some of the issues can be addressed by further refining adherence rates
in various ways, we believe the Bayesian mixture model with stochastic errors that we implement below
is significantly more appropriate and informative for our application.
Table 10: Heuristic adherence rates (%), adjusted for number of predictions and information search

MaxMax

MaxMin

SocMax

Eq

D1

L1

L2

L3

PD

PDNE

NC
Max20s
Min45s

44
44
42

38
40
38

43
35
44

38
28
39

58
36
60

59
59
59

51
32
51

53
35
55

48
30
50

54
34
54

Average

43

38

41

35

51

59

45

48

43

47

11 The pooled violation rate for all games (excluding game #27) is higher in Min45s compared to NC; however, a twosided Fisher exact test does not reject the hypothesis that the violation rates are equal (p = 0.32). The hypothesis that
violation rates are dierent in Min45s than NC is rejected for each game at the 5% level (two-sided exact Fisher test).
12 Consider that an n n game can, at best, identify n heuristics in the best case scenario that they make dierent
predictions. Given experimental and practical constraints on the size of n, we are necessarily in a world were we must deal
with partial identifiability whenever the number of heuristics exceeds the size of the action space.

15

5. Strategy classification using a Bayesian latent class model

The misspecification of econometric models by incorrectly assuming a homogeneousrather than
heterogeneouspopulation can lead to significant estimation biases (Cabrales and Garcia-Fontes, 2000;
Cohen et al., 2008; Erev and Haruvy, 2001; Estes, 1956; Estes and Maddox, 2005; Wilcox, 2006). Latent
class (or mixture) approaches are an important tool in the modeling of heterogeneous decision makers. In economics, finite mixture models have predominantly been estimated using maximum-likelihood
techniques (Bruhin et al., 2010; Conte et al., 2011; Costa-Gomes et al., 2001; El-Gamal and Grether,
1995; Harrison and Rutstrm, 2009; Spiliopoulos, 2012). Notable exceptions that use Bayesian estimation include Houser et al. (2004); Shachat et al. (2012, 2015). Estimation of finite mixture models using
maximum likelihood suers from the possibility of settling in a local rather than global maximum. This
problem is particularly acute for higher-dimensional multi-modal functions; our model is susceptible as
it includes a large number (ten) of finite mixtures. Consequently, we settled on a Bayesian estimation
technique using the Markov Chain Monte Carlo (MCMC) method implemented in the software package JAGS (Plummer, 2003). Convergence of the MCMC simulations to the posterior distributions was
verified by running five dierent chains, each comprised of 10,000 samples after a burn-in of 20,000 samples. We chose not to thin the samples based on the arguments put forth in Link and Eaton (2012).

Convergence of the chains was checked visually and using the R-statistic
(Gelman and Rubin, 1992).
Let i, g and h index subjects, games and heuristics respectively. The action chosen by a player in a
specific game is denoted as ai,g . Each player is a single type that plays all games according to a unique
heuristic, possibly with an error in applying the heuristic. Since the heuristics dier in the complexity of
the required calculations, we model the stochastic error as heuristic-dependent. Each heuristic prescribes
a 3-tuple probability vector over the available actions in game g, denoted by dh,g , where each element is
equal to n 1 if it corresponds to one of n actions prescribed by the heuristic or zero otherwise. A player
is assumed to choose actions according to dh,g with probability 1 h and to choose an action randomly
with probability h , where 0 h 1this follows the error structure chosen by Costa-Gomes et al.

(2001). The resulting 3-tuple qi,g specifies the decision probabilities with which a player i chooses each
of his/her three actions in game g conditional on the heuristic hthe function f maps dh,g and its error
rate h to qi,g .
Information acquisition and information integration are related in the following way. While the
acquisition of a piece of information is a necessary condition for its integration or use in the decision
process, the converse is not true. DMs may acquire information but choose to ignore it, i.e., not integrate
it, for various reasons. We incorporate information search data in a way that respects this important
distinction. If a decision heuristic requires information about 0 but the player has not looked up
any of his opponents payos at least once, i.e., f 0 = lt 0 = 0, then random choice is assumed, i.e.,
qi,g = (1/3, 1/3, 1/3). Therefore, information search can refute, but not confirm, the use of a specific
heuristic.
The complete model is specified by equations 1-5. We use non-informative priors for the stochastic
error h and the mixture probability vector p. The former is uniformly distributed between the bounds
of zero and one. The latter is distributed as a symmetric Dirichlet distribution with a concentration
parameter equal to one. Consequently, we assume that each type is equally likely and that the density
of every possible combination of probabilities is uniformly distributed in the 9-simplex. A categorical
distribution assigns a subject to one of the ten heuristics, indexed by the variable ci . The decision
probabilities qi,g are determined by the assigned type of a subject, ci . Finally, the actions ai,g are
distributed according to a categorical distribution with probabilities qi,g .

16

U (0, 1)

(1)

Dir (110 )

(2)

ci

Cat (p)

(3)

qi,g

f (dci ,g , ci )

(4)

ai,g

Cat (qi,g )

(5)

Throughout the paper we will use the Bayesian notion of a credible intervalinstead of frequentist
confidence intervalsto test parameter hypotheses. The 95% Highest Posterior Density Interval (HPDI)
is the smallest possible interval that contains 95% of a distributions densityin the text HPDIs will be
presented in the format [, ], and can be literally interpreted as the probability that the parameter lies in
this interval.
The results from this Bayesian mixture model are the following. Table 11 presents the estimated
prevalence of types in the subject pool including the 95% HPDI for the estimates. Figure 6 is a graphical
representation of the same data. Before proceeding with a discussion of these results, we first establish
that subjects were classified into types with a high probability or certainty, i.e., that the classification
procedure was successful and that the model was appropriate. In particular, it is important to show that
our assumption of subjects using a single heuristic for all games, is appropriate. Figure 5 displays the
inverse cumulative distribution function of the probability that subjects belong to their most likely class.
The results are encouraging as the proportion of subjects classified with probability greater than 0.95
ranged from 0.34 to 0.71 across the treatments, from 0.54 to 0.83 for classification probabilities greater
than 0.8, and from 0.86 to 0.94 for classification probabilities greater than 0.5. Recall that the prior
assigned an average probability of 0.1 of belonging to any of the ten types.
Figure 5: Inverse cumulative distribution function of the maximal classification probability

Inverse cumulative distribution function

0.8

0.6

0.4

0.2

0.1

0.2

0.3
0.4
0.5
0.6
0.7
Maximum classification probability
NC

Max20s

17

Min45s

0.8

0.9

Result 3 (Hypothesis 3: Confirmed) Significant strategy shift is observed across treatments, particularly between the NC and Max20s treatments.
Result: Compared to no time constraints, time pressure induces a significant increase in the percentage
of non-strategic types, a significant increase in social types and significant decreases in bounded rational
and normative types. The percentage of subjects classified as non-strategic types decreases significantly
in Max20s compared to NC by 13% points [8.6, 17.1]. Social types increased by 18.0% points [14.5, 21.5],
but bounded and normative types fell significantly by 19.5% [14.6, 24.3] and 11.5% [6.9, 16.2] respectively.
Result: Compared to no time constraints, time delay induces a significant increase in the percentage of
strategic types, social types and bounded types. Normative types did not significant change in prevalence.
The percentage of subjects classified as strategic types increases significantly in Min45s compared to
NC by 10.8% points [6.4, 16.0]. Social types increased by 9.5% points [6.4, 12.8], and bounded types by
5.6% [0.004, 11.2]. Normative types fell by -4.3%; however this was not significantly dierent from zero
[ 9.6, 0.8].
Result: Nash equilibrium types were not found in any treatment; however, non-zero proportions of PDNE
types were detected and were the third or fourth most prevalent type in the three treatments. In all the
treatments the 95% HPDI for NE types includes zero, and the mean posterior probability is never greater
than 0.02. However, the payo-dominant NE solution is used by a higher proportion of subjects. It is
the third and fourth most prevalent type in the NC and Min45s treatments respectivelythe probability
of subjects classified as PDNE was 0.16 and 0.12 respectively. However, as expected in the Max20s
treatment, this fell significantly by -10.7% points [ 6.2, 15.1] to only 0.048 of the subject pool; however
it is still the fourth most common type.
Result: The most common type in all three treatments was L1; importantly, the proportion of L1 types
increased significantly in Max20s. The posterior probability of subjects belonging to the L1 type is
the highest across all three treatments, ranging from 0.32 to 0.53. Under time pressure (Max20s), the
proportion of L1 types increases significantly above NC by 17.5% points [13.7, 21.8].13 This finding is
corroborated by the information lookup data since L1 requires only information about ; recall that a
large proportion of subjects did not acquire 0 information in Max20s. The prevalence of higher-level
heuristics L2 and L3 is significantly lower, never surpassing 0.11 for any treatment.
Result: The second most prevalent type in the NC and Min45s treatments is D1; however, D1 types are
essentially non-existent under time pressure (Max20s). The proportion of subjects classified as D1 are
0.25 and 0.21 in the NC and Min45s treatments respectively. This heuristic is computationally more
complex than its simpler relative, L1, as it requires first establishing whether an opponent has any
dominated strategies and eliminating them before applying the L1 heuristic on the remaining action
space. The prevalence of D1 falls to 0.01 in Max20s and is not significantly dierent from zero [0, 2.1]
the sharp drop can be attributed to the fact that under time pressure subjects often did not acquire
information about 0 .
Other results. The prevalence of the remaining heuristics is quite low (below 0.1) with only one exception.
The prevalence of SocMax (24%) was the second highest in Max20s. This is significantly higher than
the percentages in NC by 21% points [17.7, 24.3] and 8% in Min45s [4.9, 11.1]. The MaxMin heuristic is
never statistically dierent from zero. The MaxMax heuristic never exceed a proportion of 0.08 in any
treatment.
13 However,

the dierence between NC and Min45s is not significantly dierent [ 0.01, 0.8].

18

Figure 6: Classification of types across treatments

NC
Max20s
Min45s

Proportion of types in the population

0.7
0.6
0.5
0.4
0.3
0.2
0.1

nd
ed
N
or
m
at
iv
e

ou

ci
al

So

gi
c

St
ra
te

PD
N
E

N
E

L3

L2

L1

D
1

Eq

ax
cM

in
So

ax
M

ax
M

ax

Types

Table 11: Classification of types across treatments

NC

Max20s

Min45s

Heuristic

Mean

95% HPDI

#s

Mean

95% HPDI

#s

Mean

95% HPDI

#s

MaxMax
MaxMin
SocMax
Eq
D1
L1
L2
L3
NE
PDNE

7.7
0.1
3.0
3.7
25.2
35.6
3.7
3.7
1.7
15.5

6.8
0.0
0.8
2.0
21.2
32.0
1.2
1.2
0.0
11.6

9.2
0.4
5.6
5.6
29.2
39.2
6.8
6.0
3.2
19.2

4
0
1
2
13
18
1
3
1
7

2.1
1.3
24.0
0.7
1.0
53.1
10.3
1.8
0.9
4.9

2.1
0.4
22.1
0.0
0.0
50.8
7.9
0.4
0.0
2.9

2.5
2.5
26.3
1.7
2.1
54.6
12.9
3.3
2.1
7.1

1
1
11
0
0
27
5
1
0
2

0.1
0.8
16.0
0.2
20.8
31.7
7.8
9.6
0.7
12.3

0.0
0.0
14.0
0.0
16.8
28.8
5.6
6.0
0.0
8.8

0.8
2.0
18.0
0.8
24.4
34.8
10.0
13.2
1.6
16.0

0
0
8
0
10
16
3
5
0
8

Non-strategic
Strategic
Social
Bounded
Normative

43.4
56.6
6.7
32.6
17.3

39.6
53.2
4.4
28.4
13.2

46.8
60.4
9.2
36.8
21.2

22
28
3
17
8

56.5
43.5
24.7
13.1
5.7

54.2
41.3
22.5
10.4
3.3

58.3
45.4
27.1
15.4
7.9

29
19
11
6
2

32.6
67.4
16.2
38.3
13.0

30.0
64.4
14.0
34.8
9.6

35.6
70.0
18.0
41.6
16.8

16
34
8
18
8

19

Summarizing the results, we conclude that boundedly rational heuristics L1 and D1 are the most
prevalent when not under time pressure, followed by the normative solution as exemplified by the PDNE.
Under time pressure, subjects acquire significantly less information about the opponents payos, thereby
leading to an increase in the use of heuristics that require only . Consequently, time pressure increases
the prevalence of L1 and MaxMax primarily at the expense of the D1 heuristic and the PDNE.
Comparison with previous studies. Our results for the NC treatment are largely in accord with the
existing literaturethe only exception is the high prevalence of NE behavior in Rey-Biel (2009), particularly in constant-sum games. Recall that the most common type in NC was L1, followed by D1, and
then by PDNEwe present the following lists also in decreasing order of prevalence. Stahl and Wilson
(1994) classify subjects as L2, Naive Nash, and L1, whereas Stahl and Wilson (1995) classified subjects
as Worldly, L1, L0 and Naive Nash. Costa-Gomes et al. (2001) classified subjects as L2, L1/Maximax
(not identifiable from the data) using actions only; incorporating search patterns changed this order to
D1, followed by L1 and L2. Costa-Gomes and Weizscker (2008) conclude that actions in their 3 3

games were most consistent with L1 behavior, followed by D1, L2 and NE. Rey-Biel (2009) finds that in
variable-sum 3 3 games, D1 best describes behavior, followed by NE, whereas in constant-sum games

this is reversed, i.e., NE best describes actions followed by D1. Overall, the prior literature has amassed
significant evidence that under no time constraints, L1, D1, and Nash equilibrium types are the most
common. Recall that our prior over the types in the Bayesian model was non-informative, i.e., we did not
incorporate these prior results. However, since our results in NC are strongly in line with this existing
literature, incorporating a stronger prior would have reinforced our findings.
In the structural mixture model, we assumed that each subject used a single heuristic for all games but
allowed for errors in its application. An indication of whether this assumption was appropriate is given
by the estimated compliance rates (derived from the estimated error rates)the compliance rate is the
percentage of choices that a subject made that are consistent with any of the heuristics predictions. Let
qh1 , qh2 , qh3 be the proportion of games where a heuristic makes one, two and three predictions respectively.
Then the compliance rate is given by h = qh1 1 23h + qh2 1 3h + qh3 .
If the assumption of subject types was not appropriate, then the model would capture this with low
compliance rates, i.e., high error rates. Table 12 presents the compliance rates by heuristic and treatment.
The mean compliance rates pooled over all heuristics and treatments is approximately 76%. The mean
compliance rates per treatment ranged from 77% in NC, 77% in Min45s to 75% in Max20s. These high
compliance rates are evidence that subjects are well described by the heuristics and the specific error
specification we chose.
Table 12: Compliance rates (%) h per heuristic and treatment

MaxMax

MaxMin

SocMax

Eq

D1

L1

L2

L3

NE

PDNE

All

NC
Max20s
Min45s

80%
83%
70%

87%
86%
87%

80%
87%
84%

88%
81%
82%

71%
64%
75%

69%
75%
68%

74%
69%
82%

72%
64%
72%

81%
73%
76%

74%
72%
72%

77%
75%
77%

All

77%

86%

83%

84%

70%

70%

75%

69%

77%

73%

It is not clear a priori whether we should expect compliance rates to be worse for heuristics requiring
more information and more complex calculations or integration of said information. The ambiguity
arises from the opposite influence that can arise from selection eects. For example, subjects with higher
cognitive abilities may be more likely to use complex heuristics, or subjects choosing a complex heuristic
may exert greater eort. The highest mean compliance rate across all treatments was 86% for MaxMin,
followed by 84% for Eq and 83% for SocMax. The lowest mean compliance rates were found for D1, L1
20

Figure 7: Compliance ratesmean and 95% HPDI

Heuristic compliance rates

0.8

0.6

0.4

0.2

NC
Max20s
Min45s
A
ll

PD
N
E

N
E

L3

L2

L1

D
1

Eq

ax
So

cM

in
ax
M
M

ax
M

ax

Types
and PDNE (70%, 70% and 73% respectively). Figure 7 plots both the mean and 95% HPDI intervals
for the compliance rates per heuristic/treatment. It is clear that for types that were not common in
the population the HPDI is quite wide, therefore the majority of comparisons of compliance rates per
heuristic/treatment will not be significantly dierent. The only exception is a comparison between
SocMax and L1 for the Max20s and Min45s treatments. Compliance rates were higher for subjects
classified as the former type, which is in line with the hypothesis that adherence rates are decreasing in
the computational complexity of a heuristic.
6. Discussion and conclusion
Our informal replication of existing studies in the baseline treatment without any time constraints
supports their findings of bounded-rational behavior in one-shot normal form games. These results are
in line with both studies using process tracing (Mouselab or eye-tracking) and studies not using process
tracing; therefore, our conclusions are not unduly influenced by the process-tracing procedure. The most
common type of player that we found used the L1 heuristic, which ignores the strategic aspect of the
games (or assumes complete uncertainty about an opponents type). Our investigation of decision making
under time pressure using process tracing (Mouselab) revealed that decision makers are adaptive. Our
hypotheses that time pressure leads to acceleration (faster execution of the same decision processes) and
filtration (acquisition of less information from the payo matrix and a change in the relative attention)
were verified. Subjects responded to time pressure by becoming less strategicthis was evident both
in their information search patterns that placed relatively less importance on an opponents payos
compared to own payos and in final choices. In fact, 27% of decisions under time pressure were
made without searching for an opponents payos at all this result highlights the informativeness of
information search techniques. The proportion of L1 types, which requires information about own
payos only, was significantly greater under time pressure and came mostly at the expense of the more
computationally complex D1 heuristic and the payo-dominant Nash equilibrium solution. Also, under
21

time pressure there was an increase in the proportion of subjects choosing the social maximumthis
type was now the third most common in this treatment.
Another interesting result is that we did not find a significant drop in expected payos to players
under time-pressure. This implies that players adaptation was quite eective, although this is admittedly
also a function of the types of games and their strategic characteristics.
We sampled a diverse set of games from the existing literature in the spirit of Erev et al. (2015, 2010);
Ert et al. (2011) and therefore we anticipate our results to be robust. However, we encourage further
work investigating whether decision makers behavioral adaptation is conditional on the strategic characteristic or class of games, stake size or other implementation details. Future directions for this work
include refining the econometric model to permit within-subject adaptation that is game-dependent. For
example, consider the addition of a type of player who uses a combination of the two existing heuristics,
possibly in a prespecified order. For example, when a subjects primary heuristic does not yield a single
prescription, then perhaps the set of prescriptions is refined according to second heuristic or criterion
rather than randomly choosing between the set of actions prescribed by the first heuristic. Another
example is a player who first searches for a focal point in a game and chooses this if it exists; otherwise,
in the absence of a focal point proceeds to use another heuristic. With respect to the experimental implementation an interesting extension involves another treatment that explicitly imposes an opportunity
cost of time. For example, game payos could be a decreasing function of response time.
In conclusion, we have found that strategic behavior is strongly dependent on the time available
to sample information about the game and arrive at a decision. Many decisions in the wild are made
under some time constraints, whether explicit or implicit (i.e., a non-zero opportunity cost of time).
Consequently, assessing rationality within the context of the properties of the environment (e.g, realistic
information search and time constraints), is an important endeavor that ultimately improves the external
validity of laboratory experiments.

22

References
Broomell, S. B., D. V. Budescu, and H. Por (2011). Pair-wise comparisons of multiple models. Judgment and Decision
Making 6 (8), 821831.
Bruhin, A., H. F. Duda, and T. Epper (2010). Risk and rationality: Uncovering heterogeneity in probability distortion.
Econometrica 78 (4), 13751412.
Cabrales, A. and W. Garcia-Fontes (2000). Estimating learning models from experimental data. Universitat Pompeu Fabra
Economics and Business Working Paper No. 501 .
Camerer, C. F., T.-H. Ho, and J.-K. Chong (2004). A cognitive hierarchy model of games. The Quarterly Journal of
Economics 119 (3), 861898.
Camerer, C. F., E. J. Johnson, T. Rymon, and S. Sen (1993). Cognition and Framing in Sequential Bargaining for Gains
and Losses. In K. Binmore, A. Kirman, and P. Tani (Eds.), Frontiers of Game Theory, pp. 2747. Cambridge,MA: MIT
Press.
Cohen, A. L., A. N. Sanborn, and R. M. Shirin (2008, August). Model evaluation using grouped or individual data.
Psychonomic Bulletin & Review 15 (4), 692712.
Conte, A., J. D. Hey, and P. G. Moatt (2011, May). Mixture models of choice under risk. Journal of Econometrics 162 (1),
7988.
Costa-Gomes, M. A. and V. P. Crawford (2006). Cognition and behavior in two-person guessing games: An experimental
study. American Economic Review 96 (5), 17371768.
Costa-Gomes, M. A., V. P. Crawford, and B. Broseta (2001). Cognition and behavior in normal-form games: An experimental study. Econometrica 69 (5), 11931235.
Costa-Gomes, M. A. and G. Weizscker (2008). Stated beliefs and play in normal-form games. The Review of Economic
Studies 75 (3), 729762.
Crawford, V. (2008, August). Look-ups as the windows of the strategic soul. In A. Caplin and A. Schotter (Eds.), The
Foundations of Positive and Normative Economics. New York: Oxford University Press.
Davis-Stober, C. P. and N. Brown (2011). A shift in strategy or error? Strategy classification over multiple stochastic
specifications. Judgment and Decision Making 6 (8), 800813.
Devetag, G. M., S. Di Guida, and L. Polonio (2015). An eye-tracking study of feature-based choice in one-shot games.
Experimental Economics, 125.
Donders, F. C. (1868). Over de snelheid van psychische processen. Onderzoekingen gedaan in het Physiologisch Laboratorium der Utrechtsche Hoogeschool. Tweede reeks II, 92120.
El-Gamal, M. A. and D. M. Grether (1995, December). Are people Bayesian? Uncovering behavioral strategies. Journal
of the American Statistical Association 90 (432), 11371145.
Erev, I., E. Ert, and O. Plonsky (2015, February). From Anomalies to Forecasts: A Choice Prediction Competition for
Decisions under Risk and Ambiguity. Mimeo, 156.
Erev, I., E. Ert, and A. E. Roth (2010). A choice prediction competition for market entry games: An introduction. Games 1,
117136.
Erev, I. and E. Haruvy (2001). On the potential uses and current limitations of data driven learning models. Working
paper .
Ert, E., I. Erev, and A. E. Roth (2011). A choice prediction competition for social preferences in simple extensive form
games: An introduction. Games 2 (December), 257276.
Estes, W. K. (1956, March). The problem of inference from curves based on group data. Psychological Bulletin 53 (2),
134140.
Estes, W. K. and W. T. Maddox (2005, June). Risks of drawing inferences about cognitive processes from model fits to
individual versus average performance. Psychonomic Bulletin & Review 12 (3), 403408.
Gelman, A. and D. B. Rubin (1992). Inference from Iterative Simulation Using Multiple Sequences. Statistical Science 7 (4),
457511.
Gigerenzer, G. (1988). Bounded rationality: The study of smart heuristics. In D. Koehler and N. Harvey (Eds.), Handbook
of Judgment and Decision Making, pp. 163. Oxford: Blackwell.
Gigerenzer, G., P. M. Todd, and the ABC Research Group (1999). Simple heuristics that make us smart. Oxford University
Press, USA.
Greiner, B. (2015, May). Subject pool recruitment procedures: organizing experiments with ORSEE. Journal of the
Economic Science Association, 116.
Harrison, G. W. and E. E. Rutstrm (2009). Expected utility theory and prospect theory: one wedding and a decent
funeral. Experimental Economics 12, 133158.
Haruvy, E. and D. O. Stahl (2004). Deductive versus inductive equilibrium selection: Experimental results. Journal of
Economic Behavior & Organization 53 (3), 319331.
Hilbig, B. E. (2010). Precise models deserve precise measures: A methodological dissection. Judgment and Decision
Making 5 (4), 272284.
Hogarth, R. M. and N. Karelaia (2005). Simple models for multiattribute choice with many alternatives: When it does
and does not pay to face trade-os with binary attributes. Management Science 51 (12), 18601872.
Hogarth, R. M. and N. Karelaia (2006). Regions of rationality: Maps for bounded agents. Decision Analysis 3 (3), 124144.
Hogarth, R. M. and N. Karelaia (2007). Heuristic and linear models of judgment: Matching rules and environments.
Psychological Review 114 (3), 733758.
Houser, D., M. Keane, and K. McCabe (2004, May). Behavior in a dynamic decision problem: An analysis of experimental
evidence using a bayesian type classification algorithm. Econometrica 72 (3), 781822.
Johnson, E. J., C. F. Camerer, S. Sen, and T. Rymon (2002, May). Detecting failures of backward induction: Monitoring
information search in sequential bargaining. Journal of Economic Theory 104 (1), 1647.
Johnson, E. J., J. W. Payne, D. A. Schkade, and J. R. Bettman (1989). Monitoring Information Processing and Decisions:
The Mouselab System.
Knoepfle, D. T., J. T.-y. Wang, and C. F. Camerer (2009, April). Studying Learning in Games Using Eye-Tracking. Journal
of the European Economic Association 7 (2-3), 388398.
Kocher, M. and M. Sutter (2006, November). Time is moneyTime pressure, incentives, and the quality of decision-making.
Journal of Economic Behavior & Organization 61 (3), 375392.

23

Krajbich, I., C. Armel, and A. Rangel (2010, September). Visual fixations and the computation and comparison of value
in simple choice. Nature Neuroscience 13 (10), 12921298.
Krajbich, I., D. Lu, C. F. Camerer, and A. Rangel (2012). The attentional drift-diusion model extends to simple purchasing
decisions. Frontiers in Psychology 3 (June), 118.
Link, W. A. and M. J. Eaton (2012, February). On thinning of chains in MCMC. Methods in Ecology and Evolution 3 (1),
112115.
Luce, R. D. (2004, August). Response Times: Their Role in Inferring Elementary Mental Organization. Oxford University
Press, USA.
Miller, J. G. (1960, February). Information input overload and psychopathology. The American Journal of Psychiatry 116 (8), 695704.
Nagel, R. (1995, December). Unraveling in guessing games: An experimental study. American Economic Review 85 (5),
13131326.
Payne, J. W., J. R. Bettman, and E. J. Johnson (1988). Adaptive strategy selection in decision making. Journal of
Experimental Psychology: Learning, Memory, and Cognition 14 (3), 534552.
Payne, J. W., J. R. Bettman, and E. J. Johnson (1993). The Adaptive Decision Maker. Cambridge University Press.
Plummer, M. (2003). JAGS: A program for analysis of Bayesian graphical models using Gibbs sampling. In Proceedings
of the 3rd International Workshop on Distributed Statistical Computing.
Reutskaja, E., R. Nagel, C. F. Camerer, and A. Rangel (2011). Search dynamics in consumer choice under time pressure:
An eye-tracking study. American Economic Review 101 (2), 900926.
Rey-Biel, P. (2009). Equilibrium play and best response to (stated) beliefs in normal form games. Games and Economic
Behavior 65 (2), 572585.
Rubinstein, A. (2004). Dilemmas of An Economic Theorist. In Presidential Address, North American Summer Meeting of
the Econometric Society.
Rubinstein, A. (2006). Dilemmas of an economic theorist. Econometrica 74 (4), 865883.
Rubinstein, A. (2007). Instinctive and Cognitive Reasoning: A Study of Response Times. The Economic Journal 117,
12431259.
Rydval, O., A. Ortmann, and M. Ostatnicky (2009, October). Three very simple games and what it takes to solve them.
Journal of Economic Behavior & Organization 72 (1), 589601.
Schulte-Mecklenbeck, M., A. Kuehberger, and R. Ranyard (Eds.) (2011, May). A Handbook of Process Tracing Methods
for Decision Research. A Critical Review and Users Guide. New York: Taylor & Francis.
Shachat, J., J. Shachat, L. Wei, and L. Wei (2012). Procuring Commodities: First-Price Sealed-Bid or English Auctions?
Marketing Science 31 (2), 317333.
Shachat, J., J. T. Swarthout, and L. Wei (2015). A hidden Markov model for the detection of pure and mixed strategy
play in games. Econometric Theory, 124.
Simon, H. A. (1956). Rational choice and the structure of the environment. Psychological Review 63, 129138.
Spiliopoulos, L. (2012). Pattern recognition and subjective belief learning in a repeated constant-sum game. Games and
Economic Behavior 75 (2), 921935.
Spiliopoulos, L. and A. Ortmann (2014).
The BCD of Response Time Analysis in Experimental Economics.
ssrn.com/abstract=2401325 .
Stahl, D. O. and P. W. Wilson (1994, December). Experimental evidence on players models of other players. Journal of
Economic Behavior & Organization 25 (3), 309327.
Stahl, D. O. and P. W. Wilson (1995). On players models of other players: Theory and experimental evidence. Games
and Economic Behavior 10 (1), 218254.
Sutter, M., M. Kocher, and S. Strau (2003, December). Bargaining under time pressure in an experimental ultimatum
game. Economics Letters 81 (3), 341347.
Svenson, O. and A. J. Maule (1993, November). Time Pressure and Stress in Human Judgment and Decision Making.
New York: Springer.
Wilcox, N. T. (2006). Theories of learning in games and heterogeneity bias. Econometrica 74 (5), 12711292.

24

Appendix A. Experimental instructions

Appendix A.1. No time constraint treatment (baseline)

25

INSTRUCTIONS
WELCOME!
[Introduction experimenters]
Please turn off your cell/smart phone now. Switch it on only after you are told to do so.
You are about to participate in an experiment in interdependent decision making. If you follow
the instructions and pass the Understanding Test, you will be allowed to continue in the
experiment. If you make good decisions, you may then earn a considerable additional amount of
money. This additional amount will be determined both by your decisions and by those of other
participants in the experiment.
Note: Since we can do the matching only after everyone is done, there is no sense in rushing
through your decisions. That said, we urge you to work through the tasks carefully and in a
focused manner.
You will be paid in private, in cash, after today's session. Payoffs may differ considerably based
on your decisions and those of other participants.
It is important to remain silent and not to look at other people's work. If you have any questions
or need assistance of any kind, please raise your hand, and an experimenter will come to you.
Otherwise, if you talk, laugh, exclaim out loud, etc., YOU WILL BE ASKED TO LEAVE.
The experiment consists of 29 games (decisions). You will be anonymously and randomly
matched with one of the other participants. We will refer to the other participant as "S/He". In
each round, You and S/He will be presented with a decision problem. Each of you, separately
and independently, will make a DECISION.
Together, the two decisions will determine the numbers of POINTS each of you earn per game,
which may be different. Earning more points increases your payment at the end of the
experiment, as explained below. For todays experiment we shall select randomly three
games from those that everyone has done and will convert 7.5 points into one dollar.
Once a game is over, you will not be able to change your decision. Neither you nor the other
participants will learn anyone else's decisions for any game until the entire session is over.
To repeat: Since we can do the matching only after everyone is done, there is no sense in
rushing through your decisions.
The next screen displays an illustrative decision problem and its table of points. IT IS ONLY AN
ILLUSTRATION; the decision problems you will face in the 29 games will be different from
this one, and will change each round.

S/HE: &

S/HE: @

S/HE: &

S/HE: @

You: #

53

21

84

22

You: *

87

49

38

65

YOUR POINTS

HER/HIS POINTS
!

Please make your choice!

You want to choose #
You want to choose *
OK

In the actual decision problems, You will be shown a table like this (but with different numbers
of points) on your screen, and asked to choose one of your decisions, here labeled # and *.
Note that your task is hence to choose one of the two rows.
The other participant with whom you are matched will be asked, independently, to choose one of
her/his decisions, here labeled & and @.
Note that her/his task is hence to choose one of the two columns.
The combination of your decision and her/his decision is called an OUTCOME. The number of
points you and s/he receive for an outcome will be whole numbers from 0 to 99.
Your points appear in the boxes on the left side of the table, labeled "YOUR POINTS"
underneath.
Her/His points appear in the boxes on the right side of the table, labeled "HER/HIS POINTS"
underneath.

To interpret the table, consider the results of the possible outcomes (that is, combinations of
decisions):
- If you choose # and s/he chooses @, s/he earns 22 points.
- If you choose * and s/he chooses @, you earn 49 points.
- If you choose # and s/he chooses &, you earn 53 points.
- If you choose * and s/he chooses &, s/he earns 38 points.
- If you choose # and s/he chooses &, s/he earns 84 points.
- If you choose # and s/he chooses @, you earn 21 points.
- If you choose * and s/he chooses &, you earn 87 points.
- If you choose * and s/he chooses @, s/he earns 65 points.
In each round of the actual decision problems, you will see a new table.
As in this problem, the points that you and s/he earn will depend on both your decisions.

In the actual experiment, the points in a table, like the one on the previous screen, will not be
openly displayed. Instead they will be "hidden" in the boxes, as if the boxes were covered.
However you will be able to open any box, just by MOVING the mouse (that is, moving the
cursor over the box by sliding the mouse). You may open as many or as few boxes as you wish, as
often and as long as you wish, and in any order.
However, you will be able to have only one box open at a time.
YOU ARE NOT ALLOWED TO WRITE DOWN THE NUMBERS IN THE BOXES.
If you would like to know the number of points in a box that you do not remember, just open that
box again.
The points will also be hidden on her/his screen, and s/he will be able to open the boxes in the
same way, subject to the same restrictions. Importantly, s/he face the same time constraint.
Please note: What decisions you make has payoff consequences. Giving your decision some
thought is likely to pay off. In other words, nothing is gained if you try to rush though the
decision problems.
Any questions? Please raise your hand if so.
***
Once the screen comes on, please
1.! Please enter your seat number as your user name
2.! Please enter 0 as condition number

Understanding Test:
Please make now sure you understand the table by entering the outcomes in bi-matrix form. In the
bi-matrix form below, each cell (the area enclosed by a rectangular border) corresponds to
hypothetical DECISIONS of You and S/He (or an OUTCOME). In the blue areas, please fill in the
payoff of You and in the green areas fill in the payoff of S/He. For example, the upper-left cell
corresponds to the hypothetical situation where You played # and S/He played &.

S/He: &

You: #

S/He: @
.

You: *

....
Raise your hand if you!need!further!explanation

Appendix A.2. Max20s treatment

31

INSTRUCTIONS
WELCOME!
[Introduction experimenters]
Please turn off your cell/smart phone now. Switch it on only after you are told to do so.
You are about to participate in an experiment in interdependent decision making. If you follow
the instructions and pass the Understanding Test, you will be allowed to continue in the
experiment. If you make good decisions, you may then earn a considerable additional amount of
money. This additional amount will be determined both by your decisions and by those of other
participants in the experiment.
You will be paid in private, in cash, after today's session. Payoffs may differ considerably based
on your decisions and those of other participants.
It is important to remain silent and not to look at other people's work. If you have any questions
or need assistance of any kind, please raise your hand, and an experimenter will come to you.
Otherwise, if you talk, laugh, exclaim out loud, etc., YOU WILL BE ASKED TO LEAVE.
The experiment consists of 29 games (or decision problems). You will be anonymously and
randomly matched with one of the other participants. We will refer to the other participant as
"S/He". In each round, You and S/He will be presented with a decision problem. Each of you,
separately and independently, will make a DECISION.
Together, the two decisions will determine the numbers of POINTS each of you earn per game,
which may be different. Earning more points increases your payment at the end of the
experiment, as explained below. For todays experiment we shall select randomly three
games from those that everyone has done and will convert 7.5points into one dollar.
Once a game is over, you will not be able to change your decision. Neither you nor the other
participants will learn anyone else's decisions for any game until the entire session is over.
Note: You will have only 20 seconds for each of the decisions. Note that failure to make a
choice will lead to no earnings for the game (and hence zero earnings if the game is one of
the games that are randomly selected as payoff-relevant.)
You will be told for each decision problem how many seconds you have to make your
choice. Please make sure to click the OK button at the bottom of the page to submit your
choice! A progress bar at the bottom of the page, just before the OK button, indicates how
much time you have left to make your choice.

S/HE: &

S/HE: @

S/HE: &

S/HE: @

You: #

53

21

84

22

You: *

87

49

38

65

YOUR POINTS

HER/HIS POINTS
!

Please make your choice!

You want to choose #
You want to choose *
You have seconds to answer the question (make your choice).
[Progress bar here]

OK

(Screenshot)
The above screen displays an illustrative decision problem and its table of points. IT IS
ONLYAN ILLUSTRATION; the decision problems (games) you will face will be different
from this one, and will change from game to game.
In the actual decision problems, You will be shown a table like this (but with different numbers
of points) on your screen, and asked to choose one of your decisions, here labeled # and *. Note
that your task is hence to choose one of the two rows.
The other participant with whom you are matched will be asked, independently, to choose one of
her/his decisions, here labeled & and @. Note that her/his task is hence to choose one of the
two columns.

The combination of your decision and her/his decision is called an OUTCOME. The number of
points you and s/he receive for an outcome will be whole numbers from 0 to 99.
Your points appear in the boxes on the left side of the table, labeled "YOUR POINTS"
underneath.
Her/His points appear in the boxes on the right side of the table, labeled "HER/HIS POINTS"
underneath.

To interpret the table, consider the results of the possible outcomes (that is, combinations of
decisions):
- If you choose # and s/he chooses @, s/he earns 22 points.
- If you choose * and s/he chooses @, you earn 49 points.
- If you choose # and s/he chooses &, you earn 53 points.
- If you choose * and s/he chooses &, s/he earns 38 points.
- If you choose # and s/he chooses &, s/he earns 84 points.
- If you choose # and s/he chooses @, you earn 21 points.
- If you choose * and s/he chooses &, you earn 87 points.
- If you choose * and s/he chooses @, s/he earns 65 points.
In each round of the actual decision problems, you will see a new table.
As in this problem, the points that you and s/he earn will depend on both your decisions.

In the actual experiment, the points in a table, like the one on the previous screen, will not be
openly displayed. Instead they will be "hidden" in the boxes, as if the boxes were covered.
However you will be able to open any box, just by MOVING the mouse (that is, moving the
cursor over the box by sliding the mouse). You may open as many or as few boxes as you wish, as
often and as long as you wish, and in any order. !
However, you will be able to have only one box open at a time.
YOU ARE NOT ALLOWED TO WRITE DOWN THE NUMBERS IN THE BOXES.
If you would like to know the number of points in a box that you do not remember, just open that
box again.
The points will also be hidden on her/his screen, and s/he will be able to open the boxes in the
same way, subject to the same restrictions. Importantly, s/he face the same time constraint.
Please note: What decisions you make has payoff consequences. Giving your decision some
thought is likely to pay off. That said, recall that you will have only 20 seconds for each of
the problems. Recall that failure to make a choice will lead to no earnings for the game
(and hence zero earnings if the game is one of the games that are randomly selected as
payoff-relevant.)
You will be told for each decision problem how many seconds you have to make your
choice. Please make sure to click the OK button at the bottom of the page to submit your
choice! A progress bar at the bottom of the page, just before the OK button, indicates how
much time you have left to make your choice.
See the screenshot for Game 3 on the back of the Understanding Test.
During the experiment, please do not reload, or go back or forward to a webpage. If you do
not follow this instruction, you will not be paid.
***
Any questions? Please raise your hand if so.
***
Once the screen comes on, please
1.! Please enter your seat number as your user name
2.! Please enter 0 as condition number

Understanding Test:
Please make now sure you understand the table by entering the outcomes in bi-matrix form. In the
bi-matrix form below, each cell (the area enclosed by a rectangular border) corresponds to
hypothetical DECISIONS of You and S/He (or an OUTCOME). In the blue areas, please fill in the
payoff of You and in the green areas fill in the payoff of S/He. For example, the upper-left cell
corresponds to the hypothetical situation where You played # and S/He played &.

S/He: &

You: #

S/He: @
.

You: *

....
Raise your hand if you!need!further!explanation.

Appendix A.3. Min45s treatment

39

INSTRUCTIONS
WELCOME!
[Introduction experimenters]
Please turn off your cell/smart phone now. Switch it on only after you are told to do so.
You are about to participate in an experiment in interdependent decision making. If you follow
the instructions and pass the Understanding Test, you will be allowed to continue in the
experiment. If you make good decisions, you may then earn a considerable additional amount of
money. This additional amount will be determined both by your decisions and by those of other
participants in the experiment.
Note: Since we can do the matching only after everyone is done, there is no sense in rushing
through your decisions. As a matter of fact, the program is written in such a manner that
you can advance to the next decision at the earliest after 45 seconds. That said, we urge you
to work through the tasks carefully and in a focused manner.
You will be paid in private, in cash, after today's session. Payoffs may differ considerably based
on your decisions and those of other participants.
It is important to remain silent and not to look at other people's work. If you have any questions
or need assistance of any kind, please raise your hand, and an experimenter will come to you.
Otherwise, if you talk, laugh, exclaim out loud, etc., YOU WILL BE ASKED TO LEAVE.
The experiment consists of 29 games (decisions). You will be anonymously and randomly
matched with one of the other participants. We will refer to the other participant as "S/He". In
each round, You and S/He will be presented with a decision problem. Each of you, separately
and independently, will make a DECISION.
Together, the two decisions will determine the numbers of POINTS each of you earn per round,
which may be different. Earning more points increases your payment at the end of the
experiment, as explained below. For todays experiment we shall select randomly three
games from those that everyone has done and will convert 7.5 points into one dollar.
Once a game is over, you will not be able to change your decision. Neither you nor the other
participants will learn anyone else's decisions for any game until the entire session is over.
To repeat: Since we can do the matching only after everyone is done, there is no sense in
rushing through your decisions. As a matter of fact, the program is written in such a
manner that you can advance to the next decision at the earliest after 45 seconds.
The next screen displays an illustrative decision problem and its table of points. IT IS ONLY AN
ILLUSTRATION; the decision problems you will face in the 29 games will be different from
this one, and will change each round.

S/HE: &

S/HE: @

S/HE: &

S/HE: @

You: #

53

21

84

22

You: *

87

49

38

65

YOUR POINTS

HER/HIS POINTS
!

Please make your choice!

You want to choose #
You want to choose *
OK

In the actual decision problems, You will be shown a table like this (but with different numbers
of points) on your screen, and asked to choose one of your decisions, here labeled # and *.
Note that your task is hence to choose one of the two rows.
The other participant with whom you are matched will be asked, independently, to choose one of
her/his decisions, here labeled & and @.
Note that her/his task is hence to choose one of the two columns.
The combination of your decision and her/his decision is called an OUTCOME. The number of
points you and s/he receive for an outcome will be whole numbers from 0 to 99.
Your points appear in the boxes on the left side of the table, labeled "YOUR POINTS"
underneath.
Her/His points appear in the boxes on the right side of the table, labeled "HER/HIS POINTS"
underneath.

To interpret the table, consider the results of the possible outcomes (that is, combinations of
decisions):
- If you choose # and s/he chooses @, s/he earns 22 points.
- If you choose * and s/he chooses @, you earn 49 points.
- If you choose # and s/he chooses &, you earn 53 points.
- If you choose * and s/he chooses &, s/he earns 38 points.
- If you choose # and s/he chooses &, s/he earns 84 points.
- If you choose # and s/he chooses @, you earn 21 points.
- If you choose * and s/he chooses &, you earn 87 points.
- If you choose * and s/he chooses @, s/he earns 65 points.
In each round of the actual decision problems, you will see a new table.
As in this problem, the points that you and s/he earn will depend on both your decisions.

In the actual experiment, the points in a table, like the one on the previous screen, will not be
openly displayed. Instead they will be "hidden" in the boxes, as if the boxes were covered.
However you will be able to open any box, just by MOVING the mouse (that is, moving the
cursor over the box by sliding the mouse). You may open as many or as few boxes as you wish, as
often and as long as you wish, and in any order.
However, you will be able to have only one box open at a time.
YOU ARE NOT ALLOWED TO WRITE DOWN THE NUMBERS IN THE BOXES.
If you would like to know the number of points in a box that you do not remember, just open that
box again.
The points will also be hidden on her/his screen, and s/he will be able to open the boxes in the
same way, subject to the same restrictions. Importantly, s/he face the same time constraint.
Please note: What decisions you make has payoff consequences. Giving your decision some
thought is likely to pay off. In other words, nothing is gained if you try to rush though the
decision problems.
Any questions? Please raise your hand if so.
***
Once the screen comes on, please
1.! Please enter your seat number as your user name
2.! Please enter 0 as condition number

Understanding Test:
Please make now sure you understand the table by entering the outcomes in bi-matrix form. In the
bi-matrix form below, each cell (the area enclosed by a rectangular border) corresponds to
hypothetical DECISIONS of You and S/He (or an OUTCOME). In the blue areas, please fill in the
payoff of You and in the green areas fill in the payoff of S/He. For example, the upper-left cell
corresponds to the hypothetical situation where You played # and S/He played &.

S/He: &

You: #

S/He: @
.

You: *

.
....

Raise your hand if you!need!further!explanation.

!