Parunak, SPRING

Swarming Predictive Inference for Geospace (SPRING)

H. Van Dyke Parunak and Michael Quist, Soar Technology

Abstract
Conventional tracking technology depends on regular acquisition of the object of interest, and is of
little use when location data is sparse. The context of an object constrains its movement, allowing
prediction of its possible future locations even when its current location is not known.
Conventional tracking methods either use an impoverished distributional model of the objects dy-
namics, or require advance knowledge of its location and objectives. SPRING (Swarming Predictive
Inference for Geospace) uses a biologically-inspired constructive approach. An innovative multi-agent
search algorithm generates possible trajectories for an object moving on land. It represents the ob-
ject by hundreds or thousands of agents, each sampling different starting points, objectives, and de-
tails of the terrain. These agents generate the probability of finding the object at a given location in
space and time.

1 Introduction
A critical function in many domains is tracking an object of interest. The object might be a terrorist
suspect in the mountains of Afghanistan, or a panther roaming through suburban areas in southeast
Michigan,1 or a lost hiker in a wilderness region. Tracking an object of interest that is periodically re-
ported by sensors is a commodity capability. But surveillance assets are expensive, and allocating
them efficiently to detect an object of unknown location is an unsolved problem. In some cases,
sensors may simply not be available.
SPRING addresses such challenges with two core insights: contextual constraints and parallel
constructive modeling of alternative futures.
The first insight is that even when we do not know an objects exact location, the objects context
may constrain its movement, identifying regions in which it is likely to be found. For example: weath-
er, currents, fuel limitations, and commercial airline and shipping routes greatly limit the movement
of an object through the air and at sea; terrain features constrain movement on the ground. Some-
times the entity may adapt to changed context, as when a person travels by foot over off-road ter-
rain, then catches a bus or jitney upon reaching a road.
The second insight is that in the absence of sensor hits, bottom-up parallel constructive modeling of
alternative futures can estimate the objects possible locations more accurately than top-down statis-
tical methods, and faster (by leveraging biologically based algorithms). SPRING shifts the focus in
location estimation away from abstract distributions, managed with an eye to computational conven-
ience, and toward the concrete behavior of the object itself.
Section 2 reviews the state of the art in tracking. Section 3 outlines the theory behind SPRING.
Section 4 describes SPRINGs behavior representation language, and Section 5 summarizes the
functionality of the current SPRING prototype.

2 State-Of-The-Art
We begin our survey with a general mathematical framework applicable to any location prediction
method. We discuss the limitations of the most successful current tracking approach, Bayesian filter-
ing. Then we discuss two constructive models of object dynamics (field-based and graph-based), and
their limitations, and outline how SPRING applies them.

1
http://www.michigancougar.com/black.htm

1
 Parunak, SPRING

2.1 A Mathematical Framework for Location Prediction

Any effective location prediction method (including SPRING) iterates two steps: predict where the
object is likely to be, based on its last location, and correct that prediction based on sensor observa-
tions. Abstractly, let xt be the objects location at time t, and zt be the sensor observation at time t.
We seek the belief Bel(xt) = p(xt|z0, , zt) that the object is at a given location at t, conditioned on
the series of observations. Methods differ in how they perform these two steps. Prediction relies on a
model of the dynamics of the object, which we can represent abstractly as p(xt|x0, , xt-1), and
yields a predicted estimate, which we can describe abstractly as
Equation 1 , , ! ! !
Update corrects this predicted estimate, using a model of the behavior of the sensor as a function of
actual state, p(zt|xt). Abstractly,
Equation 2
where the proportionality is set so that the beliefs sum to 1.
The state of the art in location prediction is various forms of Bayesian filtering [10], including
Kalman filters, multihypothesis tracking, grid-based approaches, topological approaches, and particle
filters. These approaches seek to implement Equation 1 and Equation 2 as directly as possible. To
allow mathematical analysis, they assume Markovian dynamics, p(xt|x0, , xt-1) ~ p(xt| xt-1), and fo-
cus on the distributions Bel(xt), p(xt|xt-1), and p(zt|xt) as the fundamental entities of analysis, often
constraining them to mathematically tractable forms (usually Gaussians). (The particle filter, an im-
portant exception, is related to our approach, as discussed in Section 3.2.)
Both the Markovian assumption and the need for tractable distributions discard information
about contextual constraints that might allow a more accurate prediction. Consider first the Markov-
ian restriction. A real agents speed or range depends on its energy, which in turn depends on the
length and difficulty of its past trajectory. Dynamics of the form p(xt|xt-1) depend only on current
object location, and ignore the past trajectory. Tractable distributions also compromise accuracy. In
complex terrain, the direction and length of an objects next step depend on the objects current lo-
cation: if it is facing a steep cliff, the probability of movement up and over the cliff is extremely low;
if it is in hilly terrain, the probabilities depend more subtly on the gradient in each direction, etc. A
Bayesian filter is easier to implement, and runs faster, if p(xt|xt-1) is a Gaussian around xt-1, perhaps
displaced to account for the general direction of assumed movement, but ignoring contextual details.
In conventional tracking applications, the inaccuracies introduced by such simplifications are reme-
died by the update step, which continually corrects the predictive estimate based on new observa-
tions.
This model faces severe challenges if sensory input is limited or absent, limiting the update step.
In this case, we need to extract the most information possible from the context. To address this re-
quirement, we need a more sophisticated model of the dynamics of the object, one that can take into
account a wide variety of contextual details, and that may not be Markovian. SPRING provides such
a method.

2.2 Constructive Models of Object Dynamics

Without frequent observations, Bayesian filtering is handicapped by simplistic assumptions in its
model of the objects dynamics, p(xt|xt-1). SPRING replaces this distributional model with a constructive
one. It uses software agents to act out possible movements of the object, based on each agents cur-
rent context. Two broad categories of path-planning algorithms are field-based and graph-based.
Field-based methods predominate in situations where the object can move anywhere as long as it
avoids collision with a limited number of obstacles. The idea is to define a computational potential
field. In physics, the gradient of a potential field is a force, which imparts an acceleration to objects

2
 Parunak, SPRING

that are in the field. Analogously, computational potential fields define the movement of objects
(though in this case the force must be defined computationally rather than resulting from primitive
physical interactions). These methods originated with the work of Khatib in 1980 [18] and have been
extensively refined through the years, primarily in planning paths for robots operating in open envi-
ronments. Numerous innovations have been made both in defining the potential field on the basis
of the obstacles, and in defining the movement of the object in response to the local field. In partic-
ular, non-conservative fields can be defined that permit kinds of movement (e.g., rotational) that are
not possible with real physical potentials.
Graph-based methods are appropriate when the areas in which the object can move are highly re-
stricted, and everything else is an obstacle. The classic example is route planning for cars and trucks,
which are usually constrained to road networks. Mathematically, these networks form graphs, and
the problem then becomes finding paths on the graph that satisfy certain properties (such as short-
est distance, or least congestion). Much of the work in this area is based on Dijkstras algorithm [8]
for the shortest path through a weighted graph, or its enhancement, A* search [12]. The navigation
programs available in modern cars and in Google Maps are based on graph-based route planning.
Both field-based methods (for limited obstacles) and graph-based methods (for limited roads)
operate from the point of view of the object, seeking to move in a rational way (not collide with ob-
stacles; satisfy conditions on time and distance of travel). There are two challenges in applying these
methods directly to geospatial predictive inferencing.
First, the object is presumed to know its current location, its movement resources, and the desti-
nation to which it is moving. In our problem, we know none of these factors. Without knowing the
objects resources, we cant tell whether a field-based or a graph-based approach would be more ap-
propriate. Without knowing its current location and proposed destination, neither of these methods
can tell us how the object will move.
Second, from the perspective of the object, it needs only a single path, with some assurance that
it is better than the alternatives. It knows its own objectives, and can use search methods such as
dynamic programming or A* that explore only a fraction of the search space by heuristically pruning
regions likely to be of low value. In our problem, we do not know the objects objectives. We would
like a distribution over the entire space, such as the Bel(xt) produced by Bayesian filtering (but with-
out the loss of information imposed by the assumptions of Markovian dynamics and tractable distri-
butions).

2.3 SPRINGs Use of Constructive Dynamics

As discussed in more detail in the next section, SPRING estimates Bel(xt) by focusing, not on the
distributions of the Bayesian formulation, but (using path planning techniques) on the objects be-
havior. Thus it avoids the simplifying assumptions that make it difficult to incorporate diverse con-
textual information in Bayesian reasoning. Because we know neither the objects location nor its
preferences, SPRING represents the object by many (hundreds or thousands) of very simple agents,
and estimates Bel(xt) from their locations as they individually interact with their context. Running
many agents in parallel requires that they be simple, so we restrict their reasoning to data from their
immediate spatial neighborhood. Thus their movement is inspired by potential field methods, but
our fields can be much more complex than those used in robotics. If sensory information is availa-
ble, SPRING can do updates, in two ways: by evolving the behavior model of the agents to match the
objects recently observed behavior [28], and by replacing them to correct their spatial location [26].
However, the current prototype presumes no sensory updates.

3 Theory of SPRING
We review the SPRING concept, then give a detailed technical description and review the overall

3
 Parunak, SPRING

architecture of the system. Table 1: SPRING addresses key challenges in geospatial

predictive inference
3.1 Concept Overview Challenge SPRINGs Approach
SPRINGs central idea is shift- Detailed agent behavior driven by local context
Lack of regular surveillance
ing the focus in location estima- Opportunistic use of observations in agent re-
data on object
generation
tion away from abstract distribu-
Represent as fields over area of interest
tions, managed with an eye to Many context features must
Agents respond to a numerical combination of
computational convenience, and be considered these fields
toward the concrete behavior of Computational cost of run- Agents are tropistic (guided by local field
the object itself. It replaces a dis- ning many agents in parallel strengths), so can execute very rapidly
tributional model p(xt|xt-1) of the Different domains (road, SPRINGs cognitive engine plans each agents
objects dynamics with a construc- open areas, sea, air) re- movement across multiple domains, and sets up
quire different predictive trigger points at which the agent switches from
tive model, allowing more de- technologies one detailed predictor to another
tailed use of the context in pre-
dicting the objects movement. To accommodate uncertainty in the objects location and motives,
SPRING runs many agents in parallel. Each agent modulates its moves based on its local environ-
ment. Together, the agents develop a distribution over the objects possible location. (SPRING does
not use a distributional model of the objects dynamics, but it does generate a distribution over the
objects likely location.)
Different domains (road networks, open terrain, sea, air) offer different movement possibilities,
and an object might very well combine several such processes. For example, it might move by foot
from a hiding place to a village, from which it could take a car to an airport and then depart by air.
SPRINGs cognitive generator for behavioral templates emulates the human planning process by
giving each agent a set of execution phases, each drawing on a different prediction engine. The pro-
totype will demonstrate the functioning of this mechanism with roads and open terrain. The use of
the template generator permits future extension to other domains as well. Air and sea movement is
broadly constrained by vehicle range, weather patterns, and desire to favor or avoid regular routes (if
the object controls its own plane or boat), or more narrowly constrained by published endpoints and
transit times (if the object uses commercial carriers).
SPRING addresses key challenges in predicting the context-constrained movement of an object
whose location is uncertain, shown in Table 1.
SPRING enables an analyst to explore the context-constrained movement of an object under
various assumptions about the objects current location, likely objectives, and movement capabilities.
SPRINGs agents run very fast, as documented in the next section. On commodity desktop com-
puters in 2006, we explored 240 alternative trajectories of a single object 10,000 times faster than re-
al time, so each agent was running 2.4E6 times faster than real time. The processing is highly paral-
lelizable, allowing implementation on a general-purpose graphical processing unit (GPGPU) [21] for
even tighter time constraints. Thus predictions are almost instantaneous, allowing the user to exper-
iment with alternative sets of assumptions.

3.2 Detailed Technical Description

SPRINGs swarming technology is inspired by biological examples. Mathematically, it combines ide-
as from two forms of Monte Carlo search.
Biological Inspiration: Swarm intelligence is inspired by numerous biological systems, notably so-
cial insects [22]. The behavior of such populations can often be explained without requiring agents
to communicate directly with one another. Instead, they interact through a shared environment.
Each agent has a location in this environment, and can read and modify the state of the environ-
ment only in its immediate vicinity (thus bounding its processing requirements). Biologists call this

4
 Parunak, SPRING

process of environmentally-mediated agent coordination, stigmergy [11]. It is most commonly

studied in insect systems, but has also been documented extensively in more complex animals, in-
cluding humans [23]. Algorithms inspired by these animal systems have been applied to a wide range
of practical problems [1], including telecommunications, scheduling and logistics.
In SPRINGs application of swarming, the agents do not coordinate with each other. They do
generate a presence field by depositing a mark of their location at each step, and they sense their
context through a set of fields (e.g., physical gradient, gradient leading to possible movement objec-
tives, ground cover, presence or absence of a road, presence or absence of an obstacle). Some of
these fields correspond to commonly available map layers; others are generated from the regions
that the analyst marks on the interface. Each of these fields is either attractive or repulsive. Each
agent has an energy supply that is depleted as it moves (something that is impossible in a Markovian
movement model). The agent independently executes the following steps:
1. Come into being at a location sampled from the best current estimate of the location of the
object (which might be a broad region).
2. Increment the presence field at the current location.
3. Sample the values of the context fields at its current location and locations to which it can
move next.
4. Combine these values into a composite field value, weighting them according to the agents
mobility characteristics (for example, a wheeled vehicle is attracted to roads and avoids for-
ested areas).
5. Probabilistically select its next location based on the composite field value.
6. Move to that location.
7. Decrement the energy supply.
8. If energy is gone, die (and generate a new agent at step 1); otherwise go to step 2.
This process proceeds asynchronously with surveillance. Each time a new agent is generated, its
location (step 1) is selected based on the latest current intelligence. If no new intelligence is available,
new agents are distributed over the starting region specified by the user. This process, which we
have demonstrated in modeling urban riots [26], keeps the population of agents (and thus execution
speed) at a constant level, and also allows updating agent locations based on sporadic intelligence.
Monte Carlo Foundations: Mathematically, our application of swarming to prediction combines
ideas from Monte Carlo Tree Search [17] and particle filtering [9].
Particle filtering, a popular form of sequential Monte Carlo search [9], is a form of Bayesian filtering
(Section 2.1). Unlike approaches such as Kalman filtering [16], it makes no assumptions about the
distributional form of the belief function Bel(xt), but represents it discretely as a collection of particles,
each situated at a specific location. Particle filtering iteratively weights each particle by how closely its
characteristics match the observed characteristics of the object being tracked, then samples this distri-
bution to get a new set of particles, and shifts the location of each particle by sampling the dynamics
distribution p(xt|xt-1). Typically, the dynamics underlying this distribution are very simple (for exam-
ple, a Gaussian distribution centered around the current location estimate).
Swarming resembles particle filtering because it represents the Bel(xt), not as a unified distribu-
tion, but as a population of many individual entities. It differs from particle filtering in three im-
portant ways. First, it does not require regular sensor readings to calculate weights associated with
each agent, but repeatedly iterates the particle shift operation. Second, it uses a constructive model
of object dynamics that takes into account the objects local context in estimating its next location,
rather than using a distributional model of dynamics p(xt|xt-1). Third, when surveillance information
is available, it is applied by regenerating new agents according to the information, rather than by
weighting existing agents.
Monte Carlo tree search (MCTS) is a powerful mechanism for planning in complex decision trees,

5
 Parunak, SPRING

and is currently the engine behind the most successful computer programs for playing Go [2,6]. At
each move in a game, a player faces several choices, or branches in the tree. Each of these leads to
several choices by the opponent, to each of which the first player can respond, and so forth. Such
trees grow very rapidly. In making a choice, a player would like to explore every possible sequence
of moves that could result from that choice, but because of the size of the tree, this exhaustive
search is not feasible. The tree is large, but it is also constrained, because the moves available at each
point are limited by the current state of the board and the rules of the game. MCTS recommends a
move from a given point by sending multiple agents to explore the further branches. It uses more
agents than there are immediate options for a move, but still far fewer than the possible paths
through the tree. At each point, each agent chooses randomly from the legal moves open to it, with
weights determined by simple heuristics. When it reaches the end of the game, it reports back the
score it achieved. The algorithm then selects the immediate move that had the most agents reporting
victory for the player.
Swarming resembles MCTS because the individual agents make multiple decisions without senso-
ry input, and without complex reasoning, based only on local constraints (board state and game rules
in Go, features of the objects context in SPRING). In some implementations of swarming predic-
tion, agents paths are weighted based on the outcome they achieve, just as MCTS agents report the
final scores they realized. The SPRING prototype will not include score reporting, but it is one di-
rection for further refinement.
Swarming predicts by representing the context as fields over the terrain, and repeatedly updating
the location of each agent based on the field values at its current location. Each agent chooses its di-
rection of movement stochastically, weighted by a combination of the attractive and repulsive fields
in each adjacent cell. It chooses the length of its step by the features of the terrain (e.g., it moves
more slowly over steep gradients than over shallow ones, and more rapidly over roads than over
fields covered with scrub).
SPRING produces reasonable predictions in the absence of any surveillance information. How-
ever, its architecture (though not the current prototype) can also update based on surveillance input.
If input is available sporadically, the architecture uses it asynchronously to locate the rebirth of
agents [26]. The system need not wait for sensory input, but uses it when available.
In a setting in which input is available more frequently, SPRINGs underlying polyagent engine
can evolve the behavior models of agents based on their recently observed actions [28].
A realistic object might have a complex movement plan, perhaps including walking through re-
mote areas to reach a road, then riding on a wheeled vehicle to an airport in order to leave the coun-
try. Different segments of such a plan may require different predictive modalities (e.g., field-based
swarming for open terrain; graph search for roads). SPRINGs cognitive generator can furnish each
agent with a behavioral template that al-
lows it to change its behavior as it moves
from one domain to another. We use the
Soar cognitive architecture [19], which can
encode and execute decision processes
that realistically mimic human thought.
For example, consider an object be-
lieved to be located within the dark blue
polygon in Figure 1, with an hypothesized
objective of leaving the country. Yellow
lines are major roads, regularly traveled by
cars and busses. Light blue lines are dirt
tracks accessible to four-wheel drive vehi- Figure 1: Illustration of Cognitive Planning

6
 Parunak, SPRING

cles and ox-carts. The green star marks an international airport, while the red star marks a military
base controlled by forces hostile to the object. Given the starting polygon and the objective leave
the country, the generator might yield templates that bias agents to move by foot to a dirt track,
continue to move by foot at higher speed to get to a main road, then hitch a ride to the airport, but
stay away from the hostile base. An agent with this plan will change its behavior, and possibly the
prediction engine that drives it, as it moves from one stage to another of this plan.
To demonstrate the feasibility of cognitive coordination of different movement segments, the
SPRING prototype will incorporate distinct agent behaviors for open terrain and paved roads, and a
template generator based on Soar to demonstrate switching between them.
SPRING offers a fundamentally new capability in tracking: faster-than-real-time estimation of an
objects context-constrained movements in a surveillance-impoverished environment. Todays dom-
inant tracking technology (Bayesian inference) requires regular updates to correct errors from its
simplistic prediction methods, while more sophisticated path predictors (whether field-based or
graph-based) require specified start and end points, and predict only one path per start-end pair.
SPRING does not require updates (though it can use them if they are available). It works with start
and end regions rather than points, regions that can be very broad (e.g., destination is move
south), and even with the same start and end points, its stochastic mechanisms will sample alterna-
tive routes that the object may follow. The agents simple decision logic lets them run very rapidly,
executing thousands of times faster than real time.
Two important aspects of performance are execution speed and predictive accuracy.
Speed Performance: Our previous work offers encouraging speed results. In our implementations
to this point, we divide the total prediction period into a series of epochs, and each agent makes one
decision per epoch. In the DARPA RAID program, epochs were one minute long, and we looked
60 minutes into the future. In DeepGreen, epochs were on the order of 30 minutes, and we looked
ahead 24 hours. (More decisions per epoch may be needed for longer epochs.) On stock WinTel
platforms current in 2005, we were able to execute the decision cycle for 24,000 agents in less than
one second, so our agents moved through time about 100x faster than the wall clock. Those 24,000
agents represented 100 distinct entities, looking at 240 different futures for each one. For a single
object, with 240 agents, we would be running 10,000x faster than the wall clock for 60 second
epochs, and even faster for longer epochs. These figures reflect the performance of single-processor
machines in 2006. In addition to hardware speed improvements since then, our swarming algorithm
lends itself to a parallel implementation on an architecture such as a GPGPU. Each agent interacts
only with a limited neighborhood, so the map could be distributed across multiple processors, allow-
ing many agents to execute at once.
Accuracy Performance: We have concrete indications of accuracy from two applications: predict-
ing locations of enemy forces at the company level in the DARPA RAID program, and predicting
locations of IEDs based on past events under JIEDDO and ONR funding. This experience suggests
two useful accuracy metrics for SPRING: CEP and probability landscape.
In RAID, our predictions on human-staffed wargames were compared with a game-theoretic de-
tector, a statistical (Bayesian) predictor, and the estimates of military officers with combat experi-
ence. Our locational accuracy dominated all of them [24], as measured by CEP (circular error prob-
able), the radius of a circle around each prediction large enough so that together the circles enclose
50% of the actual objects being predicted. The prediction in this case is the high point of the
probability field generated by the agents representing an object. As envisioned, SPRING will be
predicting the location of a single object instead of the 100 being predicted in RAID, so the 50%
would have to be computed over successive trials of the system instead of over multiple objects in a
single trial. Alternatively, we could simply report the distance from the high point of the predictive
field to the object. In both cases, small numbers indicate more accurate prediction.

7
 Parunak, SPRING

Although IEDs are not moving objects, our

evaluation mechanism from our JIEDDO and
ONR projects suggests another accuracy metric.
Appropriately normalized, the presence field de-
posited by the agents as they move yields a prob-
ability field over the area of interest. This field has a.# b.#
a value at every point in the area of interest, and
can be visualized as a probability landscape with Figure 2: Probability landscape a) with cut-
multiple peaks (Figure 2a). This landscape can be off level; b) as search regions
cut off at any level to yield search regions, whose
area relative to the total area increases, the lower the level (Figure 2b).
Figure 3 shows a set of regions generated in this way from an IED forecast for Baghdad. As one
lowers the level and increases the area of the search regions, the number of actual IEDs contained in
the regions increases. A curve that shows the relation between the search area (as a percentage of to-
tal area) and the percentage of IEDs contained in the search ar-
ea can be interpreted as one would a ROC curve (Figure 4).
The lens-shaped area around the diagonal is 1, the region in
which the curve would be expected to fall if the probability
field were randomly generated. We successfully forecast 90% of
the IEDs with search areas that, restricted to roads, covered
only 3% of the area of interest. In addition, a test with human
observers with combat experience with IEDs shows that they
were no better than a random predictor. Again, swarming dom-
inates human prediction.
The probability landscape approach could be applied to
SPRING by measuring the relative search area needed to cap- Figure 3: Actual search areas
ture the object of interest. It differs from the CEP approach from an IED forecast for
because it selects the search area based on the topography of Baghdad
the probability field, rather than arbitrarily looking at circles
around the high points of the field. This difference makes it a more stringent metric. With the CEP
method, an objects actual location might be assigned very low probability, yet it could still fall with-
in a modest radius of a circle drawn from the high point of a
distribution. With the landscape approach, the objects actual
location must be assigned a relatively high probability to score
well.
In both RAID and the IED work, we trained our swarming
agents on recent observations, a process that we have not ap-
plied in the SPRING prototype. However, since swarming
dominates other predictors (including humans) when observa-
tions are available, we expect it to continue to dominate with-
out observations, since updates cannot correct the much sim-
pler dynamics models used by alternative methods. Our main
benchmark for the SPRING prototype is that it outperform al-
ternative solutions (including human experts) on test problems. Figure 4: Pseudo-ROC curve.
Bounds around diagonal show
3.3 System Architecture one standard deviation limits for
Figure 5 shows the overall SPRING architecture. SPRING in- the curve that would be generat-
gests map data (currently, in either SRTM or DTED format). ed by a random forecast.

8
 Parunak, SPRING

The Cognitive Generator reasons about dis-

continuities in agent behavior (such as a shift
from offroad to onroad travel), generating
movement plans and weight vectors as de-
scribed in the next section.

4 SPRING Behavior Plans

The behavior of a stigmergic agent includes
stochastic hill-climbing on a field that is a
weighted combination of its local state varia-
bles (e.g., the primitive fields that are defined
in over the environment).2 A necessary step in Figure 5: SPRING Architecture
designing stigmergic agents is mapping from a
description of an agents personality (framed in terms of goals and personal traits) to the weights
that define its behavior. We have two ways to achieve this objective: a linear transformation from
personality characteristics to behavior weights, and a goal-based analysis that seems more cognitively
realistic. The first section gives examples of the types of parameters involved. The second outlines
the linear approach, and the third summarizes the goal-based approach.

4.1 Personality and Behavioral Parameters

Our task is to translate from personality parameters to behavioral ones.
Personality Parameters. We characterize an agents personality in terms of its priorities, its objec-
tives, and its traits.
Its priorities are ordinal values, nominally high, medium, low. These might include
avoid-detection.High means the agent wants to avoid being detected.3
achieve-objective-quickly.High means the agent wants to move as quickly as possible.
conserve-energy.High means the agent wants to minimize the drain on its stamina.
Its objectives are nominal values with geospatial reference. The first three refer to named regions:
reach-region.A named region that the agent wishes to reach
leave-region.A named region within which the agent is currently located, and which it wants
to leave.
repel-region.A named region that the agent wants to avoid.
move-in-direction.One of {north, south, east, west}.4
Its traits are internal state variables:

2
Its behavior may also include updating its local state and augmenting certain fields at its current lo-
cation, for example, a Presence field to indicate its presence, or a WeaponsFire field if it decides to fire a
weapon. Each such action decision is a separate function of its environmental fields and its own
state.
3
In the prototype, we do not distinguish between detection by friendly vs. adversarial agents. One
could imagine another priority, partisanship, which would indicate how sensitive the agent is to the
alliance of other agents. To make use of this, we would include fields that distinguish between
friendly and adversarial features (e.g., in terms of buildings).
4
This could be generalized to a compass heading, but with some care because of the step disconti-
nuity in going from 359 to 0

9
 Parunak, SPRING

stamina.A ratio value, the agents current energy level. This is decremented as a function of
the agents movement, and (F: max meters traveled w/o rest)
travel-modality.A subset of the nominal values {foot, car, boat}, indicating the forms of travel
available to the agent. The agent is presumed to use the modality most appropriate for its
current terrain.
max-speed.X.A ratio value, the maximum speed in m/s that the agent can achieve with a
given X travel-modality.
The current prototype does not implement internal state variables.
Behavioral Parameters. An agents behavior depends on the local environmental fields that it
senses, and the weights it applies to each in its combining function.
Scalar fields on the map include the following:
Steepness (absolute terrain gradient, in meters of elevation change/kilometer).
Elevation (in meters)
Latitude (in degrees)
Longitude (in degrees)
GroundCover.trees (from 0 to 1)
GroundCover.water (from 0 to 1)
GroundCover.rocks (from 0 to 1)
DistanceToBuilding (in meters)
DistanceToRoad (in meters)
DistanceToReach.X (in meters -- X is a region name)
DistanceToLeave.X (in meters -- Y is a region name)
The agents external state is the field vector at its current location, and is to be contrasted with its
internal state, which is given by its personality traits.
Let X be the set of field types in Section Error! Reference source not found., and vxi the value
of field type x X at cell i. For each field type x X, an agent has a weight wx (-, ). The agent
computes a score for each cell i in its Moore neighborhood,
Equation 3
! = ! !"
!!
where we abuse the gradient notation to indicate the component of the gradient in the direction
from the agents current location to the cell under consideration. A common refinement is to restrict
weights to a range, WLOG (-1, 1), by passing them through a logistic function w = f(w), for exam-
ple,
Equation 4
2
= 1
1 + !!"
where controls the steepness of the logistic function. We may also modify vxi with a nonlinear
function (current options include Tanh, Max, Min, Abs, Inverse, Power, and Abs),
either before or after taking the gradient. For instance, an agent might safely ignore distance to a re-
mote threat, but pay increasing attention to it as it gets closer. Equation 3 simply computes the gra-
dient of the distance field, which is a constant, no matter how far the agent is from the target.
By applying the (say) Tanh transformation to the distance field before taking the gradient (and
adjusting the weight to account for the constant generated by the gradient operator), we can allow

10
 Parunak, SPRING

the gradient itself to be strong close to the target, but close to 0 far away.
Then the agents next step is chosen from among the nine cells in its Moore neighborhood,
where the probability of choosing cell i is drawn from a Boltzmann distribution,
Equation 5
!!!!
!!!!

!
where can be varied to tune from uniform selection over the options (when = 0) to determinis-
tic selection of the cell with the largest score (as ).
To accommodate agents moving at different speeds, consider the basic execution loop for the
system as a whole:
Select5 agent
WITH selected agent
Choose next cell
Move to next cell
Update traits
HTIW
There are various ways to modify this scheme to support agents with different speeds. All of
them depend on quantizing the range of agent speeds so that each agents speed is an integer k [1,
n].
1. In RAID, we modified Choose next cell to include cells up to k steps away from the
agents current location. The disadvantage of this approach is that the agent skips over interme-
diate cells and the information they contain.
2. The agent loop k times before selecting the next agent:
Select agent
WITH selected agent
FOR i = 0; i < k; i++
Choose next cell
Move to next cell
Update traits
ROF
HTIW
Now the agent covers intervening cells, but we reduce the asynchroniety across agents.
3. In Select agent, select agents randomly, but nonuniformly, with each agents probability of
selection proportional to its speed. Now agents continue to interleave their execution, but on
average faster agents are selected more often and thus move farther.
Our task now is to derive the weights wx from the personality parameters.

4.2 Linear Mapping

The simplest approach is to assign a numerical value to each of the agents priorities (e.g., low 0,

5
The simplest approach is round-robin scheduling (picking agents in a fixed order), which can gen-
erate artifacts [14]. Shift scheduling, in which agents are sampled randomly without replacement in
each shift, avoids these artifacts without distorting time across agents, since each agent is never more
than one shift off from the state of the other agents. Simple random sampling with replacement is
simpler, and in many cases gives adequate results.

11
 Parunak, SPRING

medium 1, high 2), then let each weight be a linear combination of the priorities. That is, set
R = (avoid-detection, achieve-objective-quickly, conserve-energy)T
A = a scalar matrix with three columns and as many rows as there are relevant fields (which
depends on how many objectives the agent has).6
The values in each row of A indicate how relevant that priority is to that row. Then the weight
vector W is given by the matrix multiplication
Equation 6
=
For example, consider the rows in A for DistanceToRoad and TimeToReach.X, where X is the
agents reach-region objective. Reasonable values for these rows
might be as in Table 2. A positive weight leads the agent to de- Table 2: Example rows in lin-
scend the gradient, preferring to move toward lower field values, ear mapping matrix
while a negative weight leads the agent to ascend the gradient, DistanceToRoad -0.4 0.4 0.2
toward higher field values. TimeToReach.goal 0 1 0
Now consider the weights that result for two agents.
The first has R = (0, 1, 2)T (doesnt care about detection, and is more concerned about conserv-
ing energy than about achieving the objective). Its weight for the gradient of DistanceToRoad is 0 +
0.4 + 0.4 = 0.8, so it will prefer to move down the gradient (toward the road), and its weight for the
gradient of TimeToReach.goal is 0 + 1 + 0 = 1, so it will move down the gradient toward the goal.
The two weights are of comparable strength.
The second agent has R = (2, 2, 0)T. Its in a rush but also very much concerned about detection.
Now it puts 0 weight on the gradient to the road (its fear and its speed concern balance each other
out), and it assigns 2 to the time gradient to the destination.
This approach is a baseline with which we might compare a more cognitive approach. The disad-
vantage is the assumption that the mapping defined by A is independent of the agents current state
(both external and internal), and the impact of that state may itself depend on the agents priorities.
For example, if an agent has high achieve-objective-quickly, its attraction to the road will be greater if car
travel-modality than if it is not. The next section discusses how to generate W as a function of agent
priorities and state.

4.3 Goal-Based Mapping

A cognitive agent plans its actions by recursively decomposing its goals until it reaches actions that it
can perform directly. Then it executes the plan by conducting these actions. To allow rapid Monte
Carlo sampling of agent trajectories, the agents in SPRING do not reason in this way, but our cogni-
tive generator formulates such a plan in the form of a sequence of weight vectors conditioned on its
state. Each weight vector is designed to move the agent to a state that will activate the next weight
vector in the plan. To realize this scheme, we show how weight vectors can function as subplans.
Then we need to address the problem of conflicting goals. The basic idea is to define weight vectors
for each subplan appropriate to an agent priority, then to combine these to reflect different levels of
each priority. 7 The current prototype handles only one priority at a time.

6
If all fields are of equal potential relevance to the agents movement, we normalize the sum of the
absolute values of each row.
7
In previous implementations, weve handcrafted weight vectors (and the changes among them). We
include a simple version of the Cognitive Behavior Generator in our prototype to show how it can
be accommodated in the architecture.

12
 Parunak, SPRING

4.3.1 Weight Vectors as Table 3: Plan for achieve-objective-quickly

Subplans Subgoal Transition Condition Next Sub-
goal
Here we develop example plan
DistanceToRoad = 0 2
structures for each agent pri- 1 Move to road
TimeToObjective.foot < TimeToRoad.foot 3
ority. The basic schema is the 2 Move along road TimeToObjective > 0 along road
agents current subgoal, a tran- toward objective 3
sition condition on agent state 3 Move to objective DistanceToObjective = 0 Success
indicating when it should tran-
sition to a different subgoal, and the subsequent subgoal. When there are two transition conditions
from a given subgoal, they should be evaluated in the order listed. One effect of these plans is an
additional internal agent state variable, its subgoal in a given plan (necessary to determine which
transition condition to watch for next).

4.3.1.1 Plan for achieve-objective-quickly

Table 3 gives an informal plan for an agent with the high-level goal of achieve-objective-quickly. Some
details require attention.
TimeToObjective and TimeToRoad must be computed based on the environment field value Dis-
tanceToObjective or DistanceToRoad and the max-speed trait for the best travel-modality the agent has
for the current terrain. In subgoals 1 and 3, we assume that only foot is useful; in subgoal 2 Time-
ToObjective will depend on whether the agent has car or not.
The transition from subgoal 2 to 3 requires evaluating the gradient of TimeToObjective.foot along the
road in each direction.
Table 4 suggests weight vectors for each subgoal.

4.3.1.2 Plan for avoid-detection.

The priority avoid-detection doesnt require a change in weight vector. Table 5 is a suggested weight
vector.

4.3.1.3 Plan for conserve-energy

Table 6 shows a plan for conserve-energy. If the agent has a car, this plan resembles achieve-objective-
quickly, though it places more of a premium on gentle gradients during walking segments.

Table 4: Weight Vectors for achieve-objective-quickly

Subgoal Subgoal
2 2
Subgoal
Environmental Field Subgoal 1 car car Intuition
3
travel- travel-
modality modality
Steepness 5 1 5 5 Steepness matters less in a vehicle than on foot.
Elevation 0 0 0 0
Latitude 0 0 0 0
Longitude 0 0 0 0
GroundCover.trees 2 5 2 2
Obstacles such as trees and rocks matter more
GroundCover.water 10 10 10 10
to vehicles than to a walker.
GroundCover.rocks 2 5 2 2
DistanceTo-
0 0 0 0
Reach.building
DistanceToReach.road 10 0 0 0
DistanceTo- We never want to forget where were headed,
2 10 10 10
Reach.objective but may need to backtrack to get to a road.

13
 Parunak, SPRING

Table 7 suggests weight vectors Table 5: Weight Vector for avoid-detection

for each subgoal in the conserve- Environmental Field Weights Intuition
energy plan. Steepness 2
The Cognitive Behavior Gen- Elevation 0
Latitude 0
erator generates agent plans as
Longitude 0
JSON files. Figure 7 illustrates the GroundCover.trees -5
format of one of these files. Trees afford cover; rocks less
GroundCover.water 10
so, and water blocks progress
GroundCover.rocks -2
4.3.2 Combining Weight Vec- DistanceToReach.building -5
Both of these risk disclosure
tors DistanceToReach.road -5
At this point, for each combina- DistanceToReach.objective 10 Still need to get there
tion of <agentPriority, agentState> we have a plan and a weight vector. In principle (though not in the
current prototype), we allow an agent to have multiple concurrent priorities, which implies the pos-
sibility of multiple possible plans and weight vectors for a given agentState. How would we reconcile
these? We are evaluating Table 6: Plan for conserve-energy
three options. Currently, Subgoal Transition Condition Next Subgoal
we favor #2. car travel-modality 1
1. Favor the weight vec- Start
car travel-modality 3
tor for the highest- DistanceToRoad = 0 2
ranked priority; if 1 Move to road
TimeToObjective.foot < TimeToRoad.foot 3
theres a tie, pick ran- 2 Move along road TimeToObjective.foot > 0 along road 3
domly. This can be toward objective
done once for each 3 Move to objective DistanceToObjective = 0 Success
agent, when it is initialized. It seems rigid, but with multiple agents with the same set of priori-
ties, we will get the right distribution across the alternatives.
2. Choose stochastically among the weight vectors, weighted by the numeric value of the agents
priorities. This can be done at agent initialization. Alternatively, and more satisfyingly, it could be
done each time the transition condition associated with the currently active subplan is satisfied.
3. Sum the weight vectors appropriate at a given state, weighted by the numeric value assigned to
each of the agents three priorities. This approach is likely to yield flat, uninteresting weight ta-
bles.

Table 7: Weight Vectors for conserve-energy

Subgoal 2 Subgoal 2
Subgoal Subgoal
Environmental Field car travel- car travel- Intuition
1 3
modality modality
Now theres a premium on avoiding
Steepness -10 -1 -10 -10
steepness
Lower altitudes are better for con-
Elevation -3 0 -3 -3
serving energy
Latitude 0 0 0 0
Longitude 0 0 0 0
GroundCover.trees -2 -5 -2 -2 Obstacles such as trees and rocks
GroundCover.water -10 -10 -10 -10 matter more to vehicles than to a
GroundCover.rocks -5 -5 -5 -5 walker.
DistanceToReach.building 0 0 0 0
DistanceToReach.road -10 0 0 0
We never want to forget where
DistanceToReach.objective -2 -10 -10 -10 were headed, but may need to
backtrack some to get to a road.

14
 Parunak, SPRING

5 The SPRING
Prototype
This section contains a
paper demonstration of
the SPRING prototype.

5.1 Interface
The SPRING inter-
face has two windows.
One (Figure 6) displays
the terrain, and allows
the user to select what is
displayed and control
execution. The other
(Figure 8) shows the
currently active plan, and
allows the user to
change the plan.
On the terrain win-
dow (Figure 6), the user Figure 7: Example plan in JSON format
can
Select the fields and
User6dened'polygons'
artifacts displayed Origin'
(using the check A;ractor/goal'
Fields'and'
boxes at the upper Repeller'
ar+facts'
left); displayed'
Switch between edit-
ing polygonal areas
of interest (which
may be attractors, Edit'polygons'vs.'
repellers, or origin run'simula+on'
areas for the entity)
and running the Execu+on'control'
simulation;
Control the execu-
Start'+me'of'paths'
tion of the simula-
tion, adding paths Length'of'paths'
and extending their
length; Determinism'
Adjust the start time Figure 6: Terrain Window
and length of the
displayed paths and the degree of nonlinearity in the composition of the roulette wheel (the in
Equation 5).
The plan window (Figure 8) displays the plan being executed. The figure shows the plan window
for the plan given in Figure 7. The Load plan button brings up a file dialog through which the us-
er can load a different plan.

15
 Parunak, SPRING

5.2 Impact of Terrain on Movement

The first example (Figure 9) illustrates the basic impact of
terrain on movement. The initial location of the entity is be-
lieved to lie in a broad region just north of a valley system, and
we suspect that the entity wants to move south by foot, which
would imply avoiding steep terrain. Here is a plan that captures
this intuition: Figure 8: Plan Window
{"comment": "Agents move south, preferring routes with minimal steepness to
save energy",
"first" : "start",
"subgoals" : [
{
"name" : "start",
"weights" : [
{ "fieldname" : "Ycoord", "weight" : 0.2 },
{ "fieldname" : "Steepness", "weight" : 1}
] },
]
}
Initially, the agents are distributed uniformly
across the north of the valley system, reflecting
our uncertainty as to the entitys location. How-
ever, the agent paths are not at all uniform. They
cluster into areas favored by the terrain, and only
three paths make it past the valley system to the
plateau beyond. Clearly, these trajectories should
receive high priority for analysis.

5.3 Behavior Shift

To illustrate SPRINGs ability to execute plans
with a shift in behavior, consider this plan, with
two subgoals: move to the nearest road, and then
move along the road to the nearest building.
{
"comment" : "Agents move first to Figure 9: Impact of Terrain on Movement
the nearest road, then along the
road to the nearest buildings. Offroad, they prefer gentle, low-altitude
terrain.",
"first" : "offRoad",
"subgoals" : [
{
"name" : "offRoad",
"weights" : [
{ "fieldname" : "Distance.Road", "weight" : 0.6 },
{ "fieldname" : "Distance.Repellers", "weight" : -0.4 },
{ "fieldname" : "Distance.Building", "weight" : 0.3 },
{ "fieldname" : "Elevation", "weight" : 0.6 },
{ "fieldname" : "Steepness", "weight" : 1}
],
"test" : [
{ "condition" : "isNearRoad", "next" : "onRoad" }
]
},

16
 Parunak, SPRING

{
"name" : "onRoad",
"weights" : [
{ "fieldname" : "Distance.Road", "weight" : 0.5 },
{ "fieldname" : "Distance.Attractors", "weight" : 0.5 },
{ "fieldname" : "Distance.Building", "weight" : 0.5 }
]
}
]
}
Figure 10 shows the agent behaviors. We have activated the Distance.Building field to
highlight the locations of buildings.
The left-hand display selects the actual agent trajectories. Each agent seeks out the nearest road,
which differs depending on where in the large initial area the agent is initiated. It then heads toward
that road, and when it reaches it, proceeds in the direction of the nearest buildings.
The right-hand display shows the agent displacements, which are straight lines for each subgoal
from the agents location at the start of the subgoal to its location when the subgoal changes. In this
display, the displacement color indicates the subgoal: black indicates that the agent is moving off-
road, while red indicates movement over the road.

5.4 Attractors and Repellers

Figure 11 illustrates the effect of attractor and repeller regions, resulting from this plan:
{ "comment": "A nonlinear repeller with a linear attractor.",
"first" : "start",
"subgoals" : [

Figure 10: Two-stage movement, overland and via road. Left: agent paths. Right:
agent displacements, showing shift between offroad (black) and on-road (red) phases.

17
 Parunak, SPRING

{
"name" : "start",
"weights" : [
{ "fieldname" : "Dis-
tance.Repellers", "weight" : -4000,
"transform": ["tanh", 4000] },
{ "fieldname" : "Dis-
tance.Attractors", "weight" : 0.5}
] },
]
}
We apply the Tanh transform to the Dis-
tance.Repellers field, so that agents ignore
it when they are far away. The result is a smooth
movement of agents around the obstacle to the
destination.

5.5 Crossing an Obstacle Figure 11: Impact of Attractors and Repel-

A particularly interesting behavior appears lers
when the agents must cross an obstacle. Consider
this plan, which requires them to move toward
the nearest building, but avoiding roads to evade
detection, as in this plan:
{ "comment" : "Agent moves toward
the nearest building, avoiding roads
and repeller regions. ",
"first" : "start",
"subgoals" : [
{
"name" : "start",
"weights" : [
{ "fieldname" : "Dis-
tance.Road", "weight" : -4000,
"transform": ["tanh", 4000] },
{ "fieldname" : "Dis-
tance.Repellers", "weight" : -1},
{ "fieldname" : "Dis-
tance.Building", "weight" : 1 }
]
}
]
}
As in the previous example, we transform the
distance field with Tanh before taking the gradi-
ent so that the agents respond to the road only as
they approach. Figure 12 shows the resulting be- Figure 12: Crossing an Obstacle
haviors. The agents sampling the leftmost part of
the area where we suspect the entity of interest may be divert their paths to skirt the road, but those
on the right choose to cross the road.
Figure 13 magnifies the western end of the east-west road segment to show more clearly the way
in which they cross the road. (We have changed the background color to make the behavior clearer.)
Note the leftmost trajectory, showing a single agent. The agent moves in the direction of the desti-
nation, until it senses the presence of the road. At this point, the attraction of the destination is

18
 Parunak, SPRING

roughly balanced against the repulsiveness of the road. Because of

the stochastic element in selecting its next cell, the agent executes a
random walk, until two random steps in the direction of the destina-
tion take it across the road. At this point, the repulsion of the road
reinforces the attraction of the destination, and the agent moves
rapidly away from the road. The agents in the cluster of trajectories
on the right show similar hesitation when approaching the road,
manifested by the dense black region where their trajectories repeat-
edly cross each other.
Hesitation in the presence of conflicting stimuli is widely docu-
mented in physical agents [13,15,20,29]. It is intriguing that it
emerges naturally from the simple mechanisms in our agents. At
Figure 13: Hesitation when
first glance, one might view this alignment as a happy coincidence.
Crossing an Obstacle
However, it actually reflects a close mathematical alignment be-
tween the behavioral rules of our agents, and a respected psychological model, decision field theory
[3-5]. This model explains hesitation in difficult decision settings by reference to an opposition be-
tween numerical fields whose mathematics directly maps to the execution of SPRINGs agents.
While the original design of SPRINGs agents was independent of decision field theory, they are
consistent with that theory, and thus can yield realistic behavioral forecasts in spite of their computa-
tional simplicity.

6 Conclusion
SPRING is an innovative approach to modeling the movement of entities in constrained envi-
ronments. Conventional tracking systems rely on frequent observations of the entity of interest to
make up for information discarded by mathematical simplifications (notably, Markovian dynamics
and idealized probability distributions) made to allow closed-form analysis of the underlying formal-
ism. SPRING uses a constructive rather than a distributional modeling approach that can exploit
contextual information, information that methods based on idealized distributions must ignore.
Thus it can function in applications in which sensory acquisition of the target is uncommon. At the
same time, when sensor information is available, it can be incorporated into the processing by repre-
senting it as another field.
SPRINGs underlying field-based representation can easily incorporate the users input concern-
ing the origin, attractors, repellers, preferred routes, and similar features of the target. Thus it pro-
vides a powerful tool to let analysts explore the consequences of alternative hypotheses about entity
behavior. Its computation can be extremely rapid, because it uses numerical computations, and be-
cause those computations draw their inputs from a local neighborhood around each agents current
location. This locality means that the computation can be readily distributed over a highly parallel
computing architecture such as a general-purpose graphic processing unit (GPGPU). The speed of
computation in turn allows the constructive exploration of many alternative futures for the target,
producing a distribution over possible outcomes that feed naturally to many decision processes.
The prototype described in this paper forecasts the movement of an entity in space, but
SPRINGs fundamental forecasting mechanism can be applied to any entity movement with the fol-
lowing characteristics:
The environment over which the entity moves can be structured as a graph (the current
prototype represents space as a square lattice).
Different features characterize the nodes in the graph, so that they can be represented as
fields over the nodes.

19
 Parunak, SPRING

The probability of the entitys choice of a next node, giving its current node, depends on
these features.
Examples of other possible domains that satisfy these requirements include:
The evolution of alternative trajectories through a causal graph [27];
The execution of alternative behaviors specified by a hierarchical task network [25];
The flow of resources through a social network [7];
The movement of cyber-intruders through a computer network.

7 References
Reprints of the authors publications are available at http://www.abcresearch.com/papers .
[1]E. Bonabeau, M. Dorigo, et al. Swarm Intelligence: From Natural to Artificial Systems. New York,
Oxford University Press, 1999.
[2]B. Brgmann. Monte Carlo Go. In Proc. the AAAI Symposium on Games: Playing, Planning, and
Learning, AAAI, 1993.
[3]J.R. Busemeyer, A. Diederich. Survey of decision field theory. Mathematical Social Sciences,
43(3):345-370, 2002.
[4]J.R. Busemeyer, R.K. Jessup, et al. Building bridges between neural models and complex decision
making behavior. Neural Networks, 19(8):1047-1058, 2006.
[5]J.R. Busemeyer, J.T. Townsend. Decision Field Theory: A Dynamic-Cognitive Approach to
Decision Making in an Uncertain Environment. Psychological Review, 100(3):432-459, 1993.
[6]R. Coulom. The Monte-Carlo Revolution in Go. In Proc. the Japanese-French Frontiers of Science
Symposium (JFFoS'2008), 2009.
[7]J. Crossman, R. Bechtel, et al. Integrating dynamic social networks and spatio-temporal models
for risk assessment, wargaming and planning. In Proc. The Network Science Workshop, 2009.
[8]E.W. Dijkstra. A note on two problems in connexion with graphs. Numerische Mathematik, 1:269
271, 1959.
[9]A. Doucet, N. de Freitas, et al., Editors. Sequential Monte Carlo Methods in Practice. Statistics for
Engineering and Information Science, Berlin, Germany, Springer, 2001.
[10]D. Fox, J. Hightower, et al. Bayesian Filters for Location Estimation. IEEE Pervasive Computing,
September:24-33, 2003.
[11]P.-P. Grass. La Reconstruction du nid et les Coordinations Inter-Individuelles chez
Bellicositermes Natalensis et Cubitermes sp. La thorie de la Stigmergie: Essai d'interprtation du
Comportement des Termites Constructeurs. Insectes Sociaux, 6:41-84, 1959.
[12]P.E. Hart, N.J. Nilsson, et al. A Formal Basis for the Heuristic Determination of Minimum Cost
Paths. IEEE Transactions on Systems Science and Cybernetics, 4(2):100107, 1968.
[13]W.E. Hick. On the rate of gain of information. Quarterly Journal of Experimental Psychology, 4(1):11-
26, 1952.
[14]B.A. Huberman, N.S. Glance. Evolutionary Games and Computer Simulations. Proceedings of the
National Academy of Science USA, 90(16):7716-7718, 1993.
[15]R. Hyman. Stimulus information as a determinant of reaction time. Journal of Experimental
Psychology: General, 45(3):188-196, 1953.
[16]R.E. Kalman. A new approach to linear filtering and prediction problems. Journal of Basic
Engineering, 82(1):3545, 1960.
[17]M. Kearns, Y. Mansour, et al. A Sparse Sampling Algorithm for Near-Optimal Planning in Large
Markov Decision Processes. In Proc. the Sixteenth International Joint Conference on Artificial Intelligence,
pages 1324-1331, Morgan Kaufmann, 1999.
[18]O. Khatib. Commande dynamique dans l'espace oprationnel des robots manipulateurs en prsence d'obstacles.

20
 Parunak, SPRING

Ph.D. Thesis at Ecole Nationale Suprieure de l'aronautique et de l'espace, 1980.

[19]J.E. Laird. The Soar Cognitive Architecture. Cambridge, MA, MIT Press, 2012.
[20]K. Lewin. A Dynamic Theory of Personality. New York, McGraw-Hill, 1935.
[21]M. Lysenko, R.M. D'Souza. A Framework for Megascale Agent Based Model Simulations on
Graphics Processing Units. Journal of Artificial Societies and Social Simulation, 11(4):10, 2008.
[22]H.V.D. Parunak. Go to the Ant': Engineering Principles from Natural Agent Systems. Annals of
Operations Research, 75:69-101, 1997.
[23]H.V.D. Parunak. A Survey of Environments and Mechanisms for Human-Human Stigmergy. In
D. Weyns, F. Michel, and H. V. D. Parunak, Editors, Proceedings of E4MAS 2005, vol. LNAI 3830,
Lecture Notes on AI, pages 163-186. Springer, 2006.
[24]H.V.D. Parunak. Real-Time Agent Characterization and Prediction. In Proc. International Joint
Conference on Autonomous Agents and Multi-Agent Systems (AAMAS'07), Industrial Track, pages 1421-
1428, ACM, 2007.
[25]H.V.D. Parunak, T. Belding, et al. Stigmergic Modeling of Hierarchical Task Networks. In Proc.
the Tenth International Workshop on Multi-Agent-Based Simulation (MABS 2009, at AAMAS 2009),
pages 98-109, Springer, 2009.
[26]H.V.D. Parunak, H.S. Brooks, et al. Dynamically Tracking the Real World in an Agent-Based
Model. In Proc. Fourteenth International Workshop on Multi-Agent-Based Simulation (MABS 2013) at
AAMAS 2013, IFAAMAS, 2013.
[27]H.V.D. Parunak, S. Brueckner, et al. Swarming Estimation of Realistic Mental Models. In Proc.
Thirteenth Workshop on Multi-Agent Based Simulation (MABS 2012, at AAMAS 2012), pages 43-55,
Springer, 2012.
[28]H.V.D. Parunak, S.A. Brueckner. Extrapolation of the Opponent's Past Behaviors. In A. Kott
and W. McEneany, Editors, Adversarial Reasoning: Computational Approaches to Reading the Opponent's
Mind, pages 49-76. Chapman and Hall/CRC Press, Boca Raton, FL, 2006.
[29]J.R. Stroop. Studies of interference in serial verbal reactions. Journal of Experimental Psychology:
General, 18(6):643-662, 1935.

21