Ga 4 GT

Genetic Algorithms and
Application to Adaptive
Automata for Game
Theory
Rawan Ghneamat
Master's degree in Computer Science
Al-Balqa' Applied University
May 2005
Rawan Ghneamat - Master's degree in Computer Science - Al-Balqa' Applied University - May 2005
Contents
Chapter 1
Introduction...........................................................................................................................4
Chapter 2
Genetic Algorithms presentation.........................................................................................5
2.1. Evolutive systems background.......................................................................................5
2.2. History of evolutive systems...........................................................................................5
2.3. Genetic algorithms as tools for optimization problems..................................................6
2.4. Genetic Algorithms: the variation-selection principle......................................................8
2.5. Genetic Operations ......................................................................................................10
2.5.1. Reproduction/ duplication.......................................................................................................................10
2.5.2. Crossing-over..........................................................................................................................................10
2.5.3. Mutation..................................................................................................................................................10
2.5.4. Schematic vue of the operators..............................................................................................................10
2.6. A first hand made sample ............................................................................................11
2.7. General algorithm ........................................................................................................12
2.8. Variants.........................................................................................................................13
2.8.1. New genetic operators derived from the basic ones..............................................................................13
2.8.2. Ecological kennels..................................................................................................................................13
2.8.3. Preselection ...........................................................................................................................................14
2.8.4. Evolutive strategies.................................................................................................................................14
Chapter 3
Genetic Algorithm Implementation....................................................................................15
3.1. Basic overview of genetic libraries ..............................................................................15
3.2. Description of a GAlib...................................................................................................16
3.2.1. GAlib programming concepts features...................................................................................................16
3.2.2. Genetic concepts supported by GAlib....................................................................................................16
3.2.2.1. Individual based representation..........................................................................................................16
3.2.2.2. Genetic operators................................................................................................................................16
3.2.2.3. Population evolution management......................................................................................................16
3.2.3. A first basic exemple of C++ program....................................................................................................17
3.2.4. Class hierachy........................................................................................................................................17
3.3. Example of use for classical optimization problems.....................................................18
Chapter 4
Genetic operators on automata..........................................................................................19
4.1. Automata: from abstract computing devices to operational process............................19
2/36
4.2. Automata with multiplicities...........................................................................................22

4.3. A linear representation for the automata with multiplicities..........................................23
4.4. Genetic operators on automata with multiplicities........................................................24
Chapter 5
Applications of genetic automata to game theory............................................................26
5.1. Game theory and economical modelling......................................................................26
5.2. Adaptive probablistic automata for iterated prisonner dilemma...................................27
5.2.1. From adaptive strategies to probabilistic automata...............................................................................27
5.2.2. From probabilistic automata to genetic automata..................................................................................29
5.2.3. General GA process for genetic automata.............................................................................................29
5.3. Implementation and Experiments.................................................................................30
5.3.1. TNT - Template Numerical Toolkit, a C++ linear algebra library...........................................................30
5.3.2. A class for automata with multiplicities with linear representation.........................................................30
5.3.3. Adaptive strategy implementation for iterated prisonner dilemma using GAlib and TNT......................30
Chapter 6
Extensions of Genetic Algorithms.....................................................................................31
6.1. Genetic Classifiers........................................................................................................31
6.1.1. Rules based system ..............................................................................................................................31
6.1.2. Credit assignation system......................................................................................................................31
6.1.3. Genetic algorithms for genetic classifiers...............................................................................................32
6.1.4. Functionning schema .............................................................................................................................32
6.2. Genetic Programming...................................................................................................32
Chapter 7
Conclusion..........................................................................................................................34
Chapter 8
Bibliography........................................................................................................................35
8.1. Books............................................................................................................................35
8.2. Articles..........................................................................................................................35
8.3. Web links.......................................................................................................................36
8.4. Computer libraries used................................................................................................36
3/36
Chapter 1
Introduction
This work is based on the usage of genetic algorithms. This kind of algorithms
was proposed in the last decades (their origins are usually according to John
Holland in the end of 1960s) as metaheuristics which goals are to obtain
solutions to optimization problems. They can be adapted to hard non linear
problems where local optima exist. Based on biological inspired methods,
genetic algorithms consist of buiding a suited metaphor of the life evolution. In
such algorithms, we use in one hand individual primitive data called
chromosomes which are usually a kind of sequence and in another hand a
population representation for these chromosomes. Genetic operators processes
on individual data and selection operators processes on chromosomes
population.
The aim of this work is to show the basis of genetic algorithms and their use on
evolvable automata which can be models for game theory and economical
modeling.
In this report, we begin in chapter 2 by explaining the principles of genetic

algorithm and we give some classical samples of its use in optimization
problems. In the chapter 3, we show how such algorithms can be implemented
using a library written in C++. We show the use of this library on some classical
optimization problems. In chapter 4, we focus our attention on a specific
application which is a less usual one. It consists on the use of genetic
algorithms over some specific automata with output, called in their general form,
automata with multiplicities. In chapter 5, we show how the genetic automata
can be used in game theory. Game theory is used for economical modeling. We
show some experiments on such problem, using the library previously
presented. Chapter 6 shows some extensions of genetic algorithms. Genetic
programming is a well-known extension and allows to develop some self-
adaptable programs which can let evolve their behaviour during the execution.
We conclude in chapter 7 and give some bibliographical references in the 8 and
last chapter.
4/36
Chapter 2
Genetic Algorithms presentation
We present here the aim of evolutive systems and genetic algorithms. We
explain the basis of the genetic operators and the classical applications to
optimization problems.
2.1. Evolutive systems background
Evolutive systems take their background in evolutionnist theory, synthesis of

natural selection theory. We will not assume here these theories whose
fundations belong to Darwin, and heredity theory which is due to Mendel. Of
course, we do not here enter the debate about the validity of genetic theories of
evolution. We just take locally the mechanisms induced by: population genetic
modification, reproduction and selection. We can sum up the basis of the
principles with the three following concepts:
● Evolution is the result of progressive alteration of living entities along

generations.
● Reproduction is based on the genetic character which undergo during

generations, some recompositions and some mutation.
● Natural selection consist of keeping the well adapted individuals to the

context of their living environment.
One of the charateristic of living is that individuals have not been programmed
to answer to a specific problem but they evolve inside their environment on the
impulse - among another - of adaptation.
The aim of evolutive systems is to buid algorithms based on these concepts and
models.
2.2. History of evolutive systems
Evolutionary computation starts from the end of 1950s. As many research

domains, there exists different concurrent approaches that have been
developped by several scientists independently. To sum up these different
approaches, we can simplify and consider two major classes:
● The "German" school which introduced with Rechenberg [8] the concept of
evolution strategies, a method used to optimize real-value parameters, for
example and initialy concerning airfoils. Fogel, Owens and Walsh [9]
5/36
developed then the evolutionary programming which consists in the definition

of evolvable machines to code part of programs.
● The "American" school led by John Holland [3] and his students from the
University of Michigan. J. Holland is usually considered as the inventor of
genetic algorithms as we define them in the following. The original goal of J.
Holland is to study the phenomenon of adaptation as it occurs in nature.
From this study concerning natural phenomenon, he wants to design artificial
systems that retain the important mechanisms of natural systems [1]. He
proposes a original life simulation called "echo" [24].
2.3. Genetic algorithms as tools for optimization problems
Beside the original goal of J. Holland, Genetics Algorithms (GA) are usually
considered as tools for a search and optimizing methodology [1]. In such
problems, we have to find the best parameters set characteristic of an optimal
situation.
Traditional optimization methods based on analytical approaches are usually

build by solving a non linear set of equations resulting from setting the gradient
of the function to optimize equal to zero. To solve such non linear systems, we
use an iterative method which consist in
● defining an initial approximation of the solution as start point of the iterative

processus,
● building a recurrent sequence which follows the slope of the function. For
exemple, the gradient method follows such process.
Illustration 1: one optimum function (from [1])
These analytic methods generally lack in robustness and need to have a

smooth function with good convex properties. For exemple, if the function to
6/36
optimize has only one optimum as in the illustration 1, analytical methods are
able to find the good solution. If the function has more than one optimum as in
illustration 2, analytical methods may generate sequences that converge
towards a local optimum or a saddle point (for which the gradient is also zero)
and not toward the global optimum. In this illustration, if the initial approximation
is nearer the local optimum then a gradient-type method will follow the slope
toward the local optimum and are not able to found the global optimum.
Illustration 2: Two-optima function (from [1])
More generally, many functions to optimize can be the results of previous non
uniform processes and can look like the illustration 3. In this case, such
optimization problems are not in the scope of analytical methods because they
can fall in various local optima, without finding any global one.
Illustration 3: multi-optima function (from [1])
To solve these difficulties which are inherent to classical analytical methods,

other kinds of methods have been defined. Random schemes can be used and
7/36
consist in choosing in each iteration a random position to evaluate the function.

This random choice are also able to go out a local optimum attraction. This kind
of methods is penalized because of the huge number of computations needed.
To improve this, we have to define an heuristic. The Genetic algorithm can be
considered as such process. We also point out two other methods that propose
a heuristic and generally a small part of random calculus, we however do not
develop their heuristic with precision and rather refer to adapted documentation:
● the simulated annealing uses random processes to help to guide its form of
search for minimal energy states [10];
● the ant system which is a new method of optimization based on bio-

inspirated model using the social insects paradigm [11].
Following D.E. Goldberg [1], GAs have good robustness properties and they
differ from classical optimization methods on the 4 following aspects:
● GAs work with a coding of the parameter set, not on the parameters
themselves
● GAs search from a population of points, not a single point,
● GAs use objective function information, not partial derivatives or other
auxiliary knowledge,
● GAs uses probabilistic transition rules, not deterministic rules.
From the capabilities to solve optimization problems, GAs can find practical
applications in a great amount kind of domains and for exemple in the following
ones [6]:
● Filtering
● Computational intelligence
● Speech recognition systems
● Production planning and scheduling problems
● Communication systems
2.4. Genetic Algorithms: the variation-selection principle
The major aspect of genetic algorithms is the adaptation property. This property
can be considered as the search of the optimum of a specific function.
Genetic algorithms work on a population of individuals. The individuals are

represented with chromosomes which are composed with primitive informations
called alleles. In term of computable formalization, chromosomes are generally
strings or sequences of informations over a finite alphabet.
To begin the algorithm, we generate a population of chromosomes.
8/36
Illustration 4: GA Individual and population levels
The algorithm is based on a variation-selection processus (cf. illustration 4):
● The variation step concerns the basic genetic operators on the individual
level and so this step acts on the chromosomes. These basic operators are
described in the following section. The result gives another population with a
greater number of chromosomes than the initial one.
● The selection step concerns the population level inside which a selection
function modifies the whole population constitution. This step leads to keep
only some of the chromosomes that have been generated during the
variation step and which satisfy to specific constraints.
The principle of selection is based on an evaluation function, called objective

function. This function measures the individual performance. So we can deduce
one probability for each individual as its ability to reproduce or to generate some
clones. This probability is called fitness.
The variation-selection processus is included in an iterative method which is

stopped after a defined number of iterations or when some specific conditions
are verified.
To conclude about this principle, this processus consists in the production of

one population which has better adaptive property and so allows to converge
toward an optimum.
9/36
2.5. Genetic Operations
During the variation step, we usually compute three genetic operators on the
chromosomes:
● Reproduction/duplication,
● Crossing-over,
● Mutation.
We describe in the following each of these operations.
2.5.1. Reproduction/ duplication
The first operator is the reproductive one. This level consists in the copy of the
string or sequence describing chromosomes and generating an identical
chromosomes. This process is made by each chromosom itself, in function of
the value of the selective function which is a kind of measure of efficience.
2.5.2. Crossing-over
Crossing-over is a process which, from two chromosomes, is able to generate

two others ones. The crossing-over consist in the cuting of the initial
chromosomes in the same place choosen at random. We permute the two parts
of the chains with a crossing between them. Then we pass the permutted parts
to the other part of chromosomes.
2.5.3. Mutation
In this step, we chose a few chromosomes at random to be candidates for the

mutation process. Usually, the probability used for this selection is low. Then for
each candidate chromosomes, we choose at random, one allele from each
chromosome and we modify its value at random. The mutation step allows to go
out from a local attractor during the optimization process.
2.5.4. Schematic vue of the operators
In the following illustration, we show a schematic vue of these operators based

on an initial chromosom which is a sequence of bits drawn with white squares
(for 0) and black squares (for 1). The sequence is copied in the
reproduction/duplication step. A specific position is pointed on the two instances
of the sequence during the crossing-over step and the ends of the sequences
from this position are exchanged between the two sequences. In the mutation
step, another specific position is pointed and in function of a probabilistic value,
10/36
the associated bit is changed from white to black or black to white.
Illustration 5: Successive getnetic operations on bit sequence
2.6. A first hand made sample
We consider the basic optimization problem as following (from D.E. Goldberg

[1]):
We want to search the maximun of the function f(x)=x² on the interval [0; 31].
The population considered is composed of the binary representation of each

natural number in the interval [0; 31], from 00000 to 11111.
The initial population is constituated by the 4 following individuals:

● 13 which binary code is 01101
The performance function for each individual correspond to the function f(x)=x².
We deduct the fitness function as the percentage repartition from these
performances:
Individual Performance Fitness

13 169 0,14
11/36
Individual Performance Fitness

24 576 0,49
8 64 0,06
19 361 0,31
Reproduction step
We choose at random, 4 new chains from the existing, the probability is
proportional to the fitness. For exemple, we can obtain: 13, 24, 24 and 19.
Crossing-over step
We take individuals two by two. We cut the chromosoms at one position
choosen at random and then we cross the cuted parts:
13: 0110||1 \ / 01100: 12

24: 1100||0 / \ 11001: 21
24: 11||100 \ / 11011: 27

19: 10|| 011 / \ 10000: 16
Mutation step
We choose at random to modify one binary position in the chain, using a
predefined mutation probability (usually low). In this sample, we take it as 0,05.
So over the 20 binary positions of the 4 individuals, we suppose that only one
binary position is modified:
12 : 01100 is transformed using the mutation on the third binary position and
we obtain 01000 : 8
New generation
After the whole set of transformations, the new generation is the following: 8,
21, 27 and 16. If we sum the performances, we obtain 1490 for this new set and
previously, we obtained 1170.
2.7. General algorithm
The following abstract algorithm gives the accurate scheduling of the whole set
of operators which allows to implement a genetic algorithm for a general
problem. The first step consists in defining the elementary chromosoms which
support the evolutionary aspect of the problem.
12/36
Genetic algorithm()
{
t:=0; // initial time
initPopulation(P) // initialisation of the
// chromosoms population
evaluate(P) // evaluate the fitness of
// all chromosoms
while not terminated do // evolution cycle
{
t:=t+1;
P':=generate(P) // augmented population obtained by
// reproduction step
cross_over(P') // crossing over step
mutate (P') // mutation step
evaluate (P') // P' chromosoms fitness evaluation
P:=select(P') // selection step
}
}
2.8. Variants
There are many variants on the basic algorithm which has been defined
previously. We give in the following some of them.
2.8.1. New genetic operators derived from the basic ones
Other operators can be defined in replacement or in addition to the basic ones.

Some of them can be
● sub-chain sequences inversion
● crossing over using a binary mask. The mask values indicate the crossing-
over positions
● boolean function
● ...
2.8.2. Ecological kennels
This variant is inspired from ecological kennels and species theories. A part of
the precedent population is saved in the new one respecting these theories
13/36
2.8.3. Preselection
This variant concerns the selection step. In this one, a child (eg a chromosome
that has been created in the present iteration) replaces its parent if its
performance is better, elsewhere the parent is kept and not the child.
2.8.4. Evolutive strategies
This variant is a major one and represents the german school of evolutive
algorithms. These approaches have been developed in concurrence whith the
american J. Holland school. In fact these two classes are quite similar and
mainly differ in the following concepts:
● The individuals are not coded with chromosomes but with real values;
● The random aspect is less developed and the reproduction is adeterministic

processus;
● The mutation operator is a major one and mainly acts. The crossing-over
operator is less major and simply acts to assure a good population diversity
when needed.
14/36
Chapter 3
Genetic Algorithm Implementation
We focus our attention in this chapter on the implementation of genetic
algorithms. The selected library used is called GAlib, initially developped by
Matthew Wall in the Massachusetts Institute of Technology. We describe here
the functionalities of this library and show its use for some classical optimization
problems.
3.1. Basic overview of genetic libraries

There is a lot of computer libraries that propose an implementation of Genetic
algorithms. We give here only a few of them, referenced in the web site "GA
Archives" (http://www.aic.nrl.navy.mil/galist/src/). This web site is an index of
source code for implementations of genetic algorithms.
● DGENESIS is a distributed implementation of a Parallel Genetic Algorithm.

Each subpopulation is handled by a Unix process and communication
between them is handled with sockets.
● LITHOS is a stack based evolutionay computation system. It is written in

langage C.
● GAEiffel is a library based on bit-string GAs written in the langage Eiffel.
● ECJ is a java-based evolutionary computation and genetic programming. It

was designed to be highly flexible.
● GAOT (GA Optimization Toolbox) implements evolution in the Matlab

environment. This implementation is very flexible for genetic operators,
selection functions, ...
● GECO is a toolbox for constructing genetic algorithms. It provides a set of

extensible classes and methods designed for generality.
● Koza.gp is a Common Lisp implementation of the Genetic Programming

Paradigm. It has been written by John Koza who is a major reference in
Genetic Programming.
● GAlib is a general purpose toolkit which is described in the next paragraph.
15/36
3.2. Description of a GAlib
GAlib is a C++ library for genetic algorithms programming. This library allows
the user to define the genome/chromosome class corresponding to the
individual data representation of the evolution problem. The library allows also
the user to define the genetic operators and to manage the population
evolution. There is many built-in representation or operators that make a easy
use and implementation of genetic algorithms. Using the object paradigm and
the inheritage concept, the user can also redefine or overload many of these
buil-in buidings and so adapt the basic concept to more specific ones.
3.2.1. GAlib programming concepts features
● Object implementation using classes hierarchy;
● Template usage in optional way;
● Coupling with PVM for parallel implementation on multiple CPUs.
3.2.2. Genetic concepts supported by GAlib
3.2.2.1. Individual based representation
● Chromosome can be built from any C++ data type.
● Buit-in chromosome type can be used like real number array, list, tree, 1D,
2D or 3D array, binary strings of variable length, ...
3.2.2.2. Genetic operators
● Various built-in mutation operators and crossover operators are defined and
can be used.
● An objective function can be implemented. They can be population- based or

individual-based.
● All chromosome initialization, mutation, crossover, and comparison methods

can be customized.
3.2.2.3. Population evolution management
GAlib proposes 4 methods to manage the population evolution at each time
16/36
step:
● Simple genetic algorithm: this algorithm uses non-overlapping populations. A

new population is entirely created at each generation (eg at each iteration).
● Steady-state algorithm: this algorithm uses overlapping populations. In this

variation, you can specify how much of the population should be replaced in
the each generation.
● Incremental genetic algorithm: this algorithm allows custom replacement

methods to define how the new generation should be integrated into the
population. So, for example, a newly generated child could replace its parent,
replace a random individual in the population, or replace an individual that is
most like it.
● "deme" genetic algorithm: this algorithm make evolve multiple populations in

parallel using a steady-state algorithm. Each generation migrates some of
the individuals from each population to one of the other populations.
3.2.3. A first basic exemple of C++ program
float Objective(GAGenome&);
main()
{
// create a genome:
GA1DBinaryStringGenome genome(length, Objective);
GASimpleGA ga(genome); // create the genetic algorithm
ga.evolve(); // do the evolution
cout << ga.statistics() << endl; // print out the results
}
float Objective(GAGenome&)
{
// your objective function goes here
}
3.2.4. Class hierachy
This library is a build on object-oriented programming. The different classes are

defined using inheritance and are organized in a hierarchical way. We show in
the following only the class hierarchy based on the general genetic algorithm
and on genome:
GAGeneticAlgorithm
GASteadyStateGA (overlapping populations)
GASimpleGA (non-overlapping populations)
17/36
GAIncrementalGA (overlapping with custom replacement)

GADemeGA (parallel populations with migration)
GAGenome
GA1DBinaryStringGenome
GABin2DecGenome
GA1DArrayGenome<>
GA1DArrayAlleleGenome<>
GAStringGenome (same as GA1DArrayAlleleGenome<char>)
GARealGenome (same as GA1DArrayAlleleGenome<float>)
GA2DArrayGenome<>
GA3DArrayGenome<>
GATreeGenome<>
GAListGenome<>
3.3. Example of use for classical optimization problems
Genetic algorithms are well-suited to solve many optimization problems, as was

explained previously. Crossing-over and mutation are operators that allow the
algorithm to go out some local optima.
One of the well-known problems is the travelling salesperperson (TSP). This

example uses an order-based list as the genome to figure out the shortest path
that connects a bunch of towns such that each town is visited exactly once. It
uses an edge recombination crossover operator.
18/36
Chapter 4
Genetic operators on automata
In this chapter, we deal with automata which are usually considered as the
basis of abstract computer devices. Inside the huge classes of automata, we
focuse our attention on the automata with outputs, and specificaly the automata
with multiplicities which allow the definition of powerful operators. This class of
finite state machines [12, 15, 20] is sufficiently versatile (by means of the
scalars and transition matrices) to cover the NFA (Nondeterministic Finite
Automata - with B as set of output), the finite state Markov chains (matrices are
stochastic), transducers, discrete events systems, etc. Using their
representation with matrix formulation, we obtain an effective descriptions on
which we define some genetic operators based on the original approaches
defined in [18].
4.1. Automata: from abstract computing devices to operational

process
Automata are initially considered as theorical tools. They are created in the
1950's following the works of A. Turing who previously deals with the definition
of an abstract "machine". The aim of the Turing machines is to define the
boundaries of what a computing machine could do and what it could not do.
The first class of automata, called finite automata corresponds to simpler kinds
of machines. They are studied by a great number of researchers as abstract
concepts for computable building. In that aspect, we can recall the works of
some linguist researchers, for example N. Chomsky who defined the study of
formal grammars.
Following [12], we can define a finite automaton, also called Finite State
Machine (FSM) as a model composed of states and transitions. A transition
links to states and represents the way to go from one state to another state. The
transition is characterized by a condition that would need to be fulfilled to enable
the transition. In illustration 6 from [17], we show a two-state automaton which
represents the 2 states of a door (open or close) and the 2 transitions between
these states.
In many works, finite automata are associated to an operator which allow to

describe a language. In such works, the condition of a transition is simply a
symbol from an alphabet. From a specific state S, the reading of a symbol a
allows to make the transition which is labeled by a and come from S. A whole
automaton is in that way, associated to a language, the recognized language,
which is the set of words. These recognized words are composed of the
sequence of letters of the alphabet which allows to go from a specific state
called initial state, to another specific state, called final state.
19/36
Illustration 6: Open/close door : a basic FSM

with 2 states and 2 transitions
A formal mathematical definition of a finite automaton is following.

Definition (Finite Automaton):
A finite automaton is the 5-uple  Σ,Q,I,T,δ  where
● Σ is a set of symbols, called alphabet of the language the automaton

accepts, the alphabet is usually, but not necessarily, assumed to be finite;
● Q is the set of states to be finite;
● I ⊂Q is the set of initial states, that is, the states in which the automaton is
when no input has been yet processed;
● T ⊂Q is the set of the final states called accepting states;
● δ is the transition function, that is δ:Q ×Σ  Q which from a state S1 and

the letter a go to a state S2 if it exist a transition labelled with a from the state
S1 to the state S2.
20/36
In the illustration 7 from [17], we represent an automaton with one initial state,
labelled with 1 and two final states, labelled 6 and 7. This automaton allows to
recognize the word "nice".
Illustration 7: Finite automaton which recognizes the word "nice"
A first classification is based on the geometric aspect : DFA (Deterministic Finite

Automata) and NFA (Nondeterministic Finite Automata)
● In deterministic automata, for each state there is atmost one transition for
each possible input and only one initial state.
● In non-deterministic automata, there can be none or more than one transition

from a given state for a given possible input
Besides the classical aspect of automata as machines allowing to recognize

languages, another approach consists in associate to the automata a functional
goal. In addition of accepted letter from an alphabet as condition of transition,
we add for each transition an information which can be considered as an output
data of the transition, the read letter is now called input data. We define in such
way an automata with outputs or weighted automaton.
Such automata with outpout give a new classification of machines:
● Transducers: they generate output based on a given input and/or a state

using actions. They are used for control applications. Here two types are
distinguished:
● Moore machines: Output depends only on a state, i.e. the automaton
21/36
uses only entry actions. The advantage of the Moore model is a

simplification of the behaviour
● Meally machines: Output depends on input and state, i.e. the FSM uses
only input actions. The use of a Mealy FSM leads often to a reduction of
the number of states.
Finally, we focuse our attention on a special kind of automata with outputs

which are efficient in operational way. This automata with output are called
automata with multiplicities and they are defined in the following.
4.2. Automata with multiplicities
An automaton with multiplicities is based on the fact that the output data of the
automata with output belong to a specific algebraic structure, a semiring. In that
way, we will be able to buid effective operations on such automata, using the
power of the algebraic structures of the output data. And we are also able to
describe this automata in matrix representation with all the power of the linear
algebra.
Definition (Automaton with multiplicities):
An automaton with a multiplicities over an alphabet Σ and a semiring K is the

5-uple  Σ,Q,I,T,δ  where
● { }
Q= S 1 ,S 2 ,. . . ,S n is the finite set of state;
● I:Q  K is a function over the set of initial states, which associates to each
initial state a value of K, called entry cost, and to non-initial state a nul value ;
● T:Q  K is a function over the set of the final states, which is associated to
each final state a value of K, called final cost, and to non-final state a nul
value;
● δ is the transition function, that is δ:Q ×Σ×Q  K which from a state S 1 , a

letter a and a state S 2 go to a value z of K if it exist a transition labelled with
a from the state S 1 to the state S 2 . and with the output z.
Remark 1: Automata with multiplicities is a generalisation of finite automata. In

fact, finite automata can be considered as automata with multiplicities with for
the semiring K, the boolan set B={0,1}. To each transition we affect 1 if it exists
and 0 if not.
22/36
Remark 2: We have not yet, on purpose, defined what a semiring is. Roughtly it
is the least structure  K ,, x  that allows matrix computation with units (one
can think of a ring without the "minus" operation'
4.3. A linear representation for the automata with multiplicities

The previous automata with multiplicities can be expressed by a linear
representation which is a triplet
ρ=  λ,μ,γ 
● with λ∈K 1 ×n is a row-vector which coefficients are λi =I S i ,  

● γ∈K n×1 is a row-vector which coefficients are γi =T S i ,  
● μ:Σ K n×n is a morphism of monoids such that
∀ a∈Σ , the coefficient on the ith row and jth column of all transitions

labelled with a is μ  a ij =δ S i ,A,S j 
In the following, we describe the linear representation of the automata with
multiplicities which corresponds to the illustration 8.
Illustration 8: automata with multiplicities sample
In this illustration, the input states are characterized with input arrows from
nothing and the output states are characterized with output arrow to nothing. On
each transition, we indicate before the semicolumn, the input data and after the
semicolumn, the output associated for each input data.
23/36
The linear representation is the following:
λ=  0 . 5 ; 0 ; 0 ; 0 ; 0 . 5 ; 0 
γt = 0 ; 1 ; 0 ;1 ; 0 ; 0 
[ ]
0 . 2 0 . 5 0 . 3 0 0 0
0 0 0 1 0 0
0 0 1 0 0 0
μ  a =
0 0 0 1 0 0
0 0 0 0 1 0
0 0 0 0 0 1
[ ]
1 0 0 0 0 0
0 0 0 . 2 0 .8 0 0
0 0 1 0 0 0
μ  b =
0 0 0 0 .8 0 . 2 0
0 0 0 0 1 0
0 0 0 0 0 1
4.4. Genetic operators on automata with multiplicities
We define the chromosome for each automata with multiplicities as the

sequence of all the matrices μ  a  associated to each letter a from the
alphabet Σ . The chromosomes are composed with alleles which are here the
lines of the matrix.
In the following, genetic algorithms are going to generate new automata

containing possiblly new transitions from the ones included in the initial
automata.
The genetic algorithm over the population of automata with multiplicities follows
a reproduction iteration broken up in three steps:
● Duplication where each automata generates a clone of itself;
● Crossing-over concerns a couple of automata. Over this couple, we consider

a sequence of lines of each matrix μ  a  for all a∈Σ . For each of these
matrices, a permutation on the lines of the chosen sequence is made
between the analogue matrices of this couple of automata.
24/36
● Mutation where a line of each matrix μ  a  is randomly chosen a sequence of

new values is given for this line.
Finally the whole genetic algorithm scheduling for a full process of reproduction
over all the population of automata is the evolutionary algorithm:
1. For all couple of automata, two children are created by duplication, crossover
and mutation mechanisms;
2. The fitness for each automata is computed;
3. For all 4-uple composed of parents and children, the performless automata,
in term of fitness computed in previuous step, are suppressed. The two
automata, still living, result from the evolution of the two initial parents.
Remark: The fitness is not defined at this level of abstract formulation, but it's
defined corresponding to the context for which the automata is a model.
25/36
Chapter 5
Applications of genetic automata to game theory
In this chapter, we deal with some specific domains of applications to game
theory which is one of the major class of models in the new approaches of
modelling in the economical domain. First, we will explain how adaptive
strategies are useful concerning these domains and so how genetic automata
are well-adapted for such modeling. We focus our attention on the "iterated
prisonner dilemma" and we buid an original evolutive probabilistic automaton for
strategy modeling.
5.1. Game theory and economical modelling
Game theory has become since the five last decades a major aspect in
economical sciences modelling and in a great number of domains where
strategical aspects has to be involved.
Game theory is usually defined as a mathematical tool allowing to analyse

strategical interactions between individuals.
Initially funded by mathematical researchers, J. von Neumann, E. Borel or E.

Zermalo in 1920s, game theory increase in the 1940s with a major work from J.
von Neumann and O. Morgenstern and then with the works of John Nash in the
1950s . John Nash has proposed an original equilibrium obtained with adaptive
process, mainly presented in its phd of 27 pages and a scientific paper of 1
page [ref]. His oversized character both on scientific and personal aspects has
inpired the well-known movie "a beautiful mind".
In game theory, the Nash equilibrium is a kind of optimal strategy for games
involving two or more players, whereby the players reach an outcome to mutual
advantage. If there is a set of strategies for a game with the property that no
player can benefit by changing his strategy while the other players keep their
strategies unchanged, then that set of strategies and the corresponding payoffs
constitute a Nash equilibrium.
We can understand easily that the modelization of player behavior needs some
adaptive properties. The computable model corresponding to genetic automata
are in that way a good tool to modelize such adaptive strategy. We use them
for the game presented in the next paragraph.
26/36
5.2. Adaptive probablistic automata for iterated prisonner

dilemma
5.2.1. From adaptive strategies to probabilistic automata
The prisoner dilemma is a two-players game where each player has two
possible actions: cooperate ( C ) with its adversary or defect it ( C ). So, four
outputs are possible for the global actions of the two players. A relative payoff is
defined relatively to these possible outputs, as described in the following table
where the rows correspond to one player behaviour and the columns to the
other player one.
C C
C (3,3) (0,5)
C (5,0) (1,1)
In iterative version of the prisoner dilemma, successive steps can be defined.

Each player do not know the action of its adversary during the current step but
he knows it for the preceding step. So different strategies can be defined for a
player behaviour, the goal of each one is to obtain maximal payoff for himself. In
the illustration 9, we describe two strategies with transducers. Each transition is
labeled by the input corresponding to the player perception which is the
precedent adversary action and the output corresponding to the present player
action. The only inital state is the state 1, recognizable by the incoming arrow
labeled only by the output. The final states are the states 1 and 2, recognizable
with the double circles. In strategy A, the player has systematically the same
behaviour as its adversary at the previous step. In strategy B, the player
chooses definitively to defect as soon as his adversary does once (see
illustration 10).
Illustration 9: Tit-for-tat strategy

strategy
27/36
Illustration 10: Vindictive strategy automata
The previous automata represent static strategies and so they are not well
adapted for the modelization of evolutive strategies. For this purpose, we
propose a model based on a probabilistic automaton described by the
illustration 11.
This automaton represents all the two-states strategies for cooperation C and
competitive C behaviour of one agent against another in prisoner dilemma.
The transitions are labeled in output by the probabilities p_i of their realization.
The state 1 is the state reached after cooperation action and the state 2 is
reached after defection.
For this automaton, the associated linear representation, as described

previously, is:

λ= p1 ; 1 − p1 
γt = 1 ;1 
μ  C =
[ p2
p3
1 − p 2
1 − p3 ]
μ  C =
[ 1 − p 4
1 − p 5
p 4
p5 ]
28/36
Illustration 11: Probabilistic multi-strategies two-

states automata
5.2.2. From probabilistic automata to genetic automata
With the linear representation of the automata, we can compute genetic

automata as described in previous chapter. Here the chromosomes are the
sequences of all the matrices associated to each letter.
We have to define the fitness in the context of the use of these automata. The
fitness here is the value of the payoff.
5.2.3. General GA process for genetic automata
A population of automata is initially generated. These automata are playing

against a predefined strategy, named SO.
Each automaton makes a set of plays. At each play, we execute the

probabilistic automaton which gives one of the two outputs: C or C . With
this output and the SO's output, we compute the payoff of the automaton,
according with the payoff table.
At the end of the set of plays, the automaton payoff is the sum of all the payoffs
of each play. This sum is the fitness of the automaton. At the end of these set of
plays, each automaton has its own fitness and so the selection process can
select the best automata. On the end of these selection process, we obtain a
new generation of automata.
This new generation of automata is the basis of a new computation of the 3

genetics operators.
29/36
5.3. Implementation and Experiments
5.3.1. TNT - Template Numerical Toolkit, a C++ linear algebra

library
The implementation of a linear algebra library has began with the BLAS libraries
which is considered as the basic functionalities of matrix operations. These
libraries have been developed since many decades and originally for Fortran.
With the use of object oriented programming, the major evolution in this
direction has been made with Lapack++ which proposes a C ++ extension for
BLAS subroutines.
A major evolution of C++ has been made with the concept of template which
allow abstract and reusable programmation. The conceptors of Lapack++ have
begin to make evolve this library toward template concept but finally stop this
evolution. The lapack++ development team finally propose to the people which
want to use the template notion to use another library which is named TNT for
Template Numerical Toolkit.
TNT propose a collection of interfaces and some implementations for numerical

objects concerning scientific computation in C++.
The goal is to propose reusable componants. The basic data structures are
mainly mutidirectional arrays.
5.3.2. A class for automata with multiplicities with linear

representation
The TNT library is well adapted to generate generic classes to represent

automata with multiplicities, under their linear representation. The template
concept allow the building of the genericity concerning the output which can be
simply described as elements of a semiring.
5.3.3. Adaptive strategy implementation for iterated prisonner

dilemma using GAlib and TNT
The two libraries GAlib and TNT can be combinated to implement genetic
probablilistic automata which are able to modelize adaptative strategies. This
implementation is under construction. The validation phase and solution
analysis still need a important work which is presently under progress.
30/36
Chapter 6
Extensions of Genetic Algorithms
6.1. Genetic Classifiers
Genetic classifiers are systems which are able to learn specific rules. The
genetic processus allow to optimize these rules performance. These systems
perceive informations from outside as message and the evolution is based on
the systema daptation and reaction to these messages.
Genetic classifiers are contituated with 3 main parts:
● rules based system;
● credit assignation system;
● genetic algorithm.
6.1.1. Rules based system
A rule can be written with the following description:
<cond> : <result>
Example:
01#0 : 01111
● The condition means that we select all binary messages composed of 4 bits,
starting with 01 and finishing with 0;
● If the message has been accepted, we send another message which is 0111.
6.1.2. Credit assignation system
This corresponds to a competitive system between the rules. Each rules is

affected with a specific credit which value can evolve.
When a message appears:
● The rules which are able to select it, make a proposal which is proportional
with their present credit;
31/36
● The attribution system select the rules which make the best proposals and
then make them active;
● An actived rules send its own result message associated with a reward
corresponding to its own credit. The reward is given to the rules which will
select the message.
6.1.3. Genetic algorithms for genetic classifiers
A genetic algorithm is implemented on the rules population. The performance

function is simply the credit associated to each rule. The genetic operators,
reproduction, crossing-over, mutation and selection are applied to the rules
which are the best performances.
6.1.4. Functionning schema
Illustration 12: genetic classifier schema
6.2. Genetic Programming
Genetic programming is an automated methodology to find computer programs

that best perform a user-defined task. It's based on genetic algorithms principles. It
is therefore a particular machine learning technique that uses an evolutionary
algorithm to optimize a population of computer programs according to a fitness
landscape determined by a program's ability to perform a given computational
task.
Computer programs in GP can be written in a variety of programming languages.
32/36
In the early (and traditional) implementations of GP, program instructions and data
values were organized in tree-structures, thus favoring the use of declarative
languages that naturally embody such a structure (an important example
pioneered by Koza is Lisp).
33/36
Chapter 7
Conclusion
In this study, we present genetic algorithm as a new methodology to explain
evolution processus in one hand and to give a new heuristic to find optima and to
solve search problems.
Genetic agorithm is based on adaptive properties that can be usefull to manage

for modeling adaptive strategies in game theory. We show how automata with
multiplicities with evolvable probabilities in output can be appropriate computable
models for such strategies.
We presently start the implementation work which is based on the use of TNT and
GAlib in C++ environment. Because of the template ehancement proposed in each
of these libraries, we can buid generic classes to represent automata with
multiplicities in abstract way concerning the set of its output which has only to
respect their belonging to semi-ring.
Such works of modeling and experimentation has novelty aspects. This study has
been made with the coordination of a french research team lead by Prof. G.
Duchamp and C. Bertelle, and including K. Khataneh, PhD candidate who come
from BAU. In that team, beside some implementations using C++, other
implementations are also currently made using a CAS (computer Algebraic
System) called MuPAD. These different works will conduct to some publications
inside international conferences (ECCS 2005 at Paris in November and IEEE-
ICECS 2005 in Tunisia in december).
34/36
Chapter 8
Bibliography
8.1. Books
1. David E. Goldberg "Genetic Algorithms", Addison-Wesley, 1989

2. Melanie Mitchell "An introduction to Genetic Algorithms", the MIT Press, 1996
3. John H. Holland "Hidden Order - How adaptation builds coimplexity", 1995
4. Fritjof Capra "The web of life", Anchor Books, 1996
5. K.F. Man, K.S. Tang and S. Kwong "Genetic Algorithms", Springer, 1999
6. J.E. Hopcroft, R. Motwani, J.D. Ullman "Introduction to automata theory,
Languages and Computation", Addison-Wesley, 2001
7. R. Axelrod "The complexity of cooperation", Princeton University Press, 1997
8. I. Rechenberg "Evolution strategies", Fromman-Holzboog, 1973
9. L.J. Fogel, A.J. Owens, M.J. Welsh "Artificial intelligence through simulated
evolution", John Wiley, 1966
10.L. Davis (Ed.) "Genetic algorithm and simulated annealing", Pitman, 1987
11.E. Bonabeau, M. Dorigo and G. Theraulaz "Swarm intelligence", Oxford
University Press, 1999
12.S. Eilenberg "Automata, languages and machines", Vol.A et B, Academic
press, 1976
13.R.P. Stanley "Enumerative combinatorics", Cambridge University Press,
1999
14.J.S. Golan "Power algebras over semirings", Kluwer Academic Publishers,
1999
15.J. Berstel and G. Reutenauer "Rational series and their language", EA TCS,
1988
8.2. Articles
16.John Koza "Genetic programming", Encyclopedia of Computer Sciences and

Technology, 1997
17.C. Bertelle, M. Flouret, V. Jay, D. Olivier, and J.-L. Ponty "Adaptive
behaviour for prisoner dilemma strategies based on automata with
multiplicities." In ESS 2002 Conf., Dresden (Germany), October 2002.
18.C. Bertelle, M. Flouret, V. Jay, D. Olivier, and J.-L. Ponty "Genetic algorithms
on automata with multiplicities for adaptive agent behaviour in emergent
organizations" In SCI'2001, Orlando, Florida, USA, 22-25th July 2001
19.A. Bechara, H. Damasio and A.R. Damasio "Emotion, Decision Making and
the orbitofrontal cortex", Cerebral Cortex, 10:295-307, march 2000
20.M.P. Schutzenberger "On the definitiion of a family of automata", Information
and Controm 4, 245-270 (1961)
35/36
8.3. Web links
Wikipedia Encyclopedia:
16.http://en.wikipedia.org/wiki/Genetic_Algorithm
17.http://en.wikipedia.org/wiki/Automata_theory
Rennard web site:

18.http://www.renard.org/alife/english/gavintrgb.html
GA archives:
19.http://aic.nrl.navy.mil/galist
20.http://gaul.sourceforge.net/intro.html
21.http://www.estec.esa.nl/outreach/gatutor/Default.htm
John Koza web site:

22.www.genetic-programming.com
Game theory:
23.http://www.math.wisc.edu/~stefanss/520report.html
John Holland Echo web site from the Santafe Institute:

24.http://www.santafe.edu/projects/echo/
8.4. Computer libraries used
25.GAlib: http://lancet.mit.edu/ga/
26.TNT: http://math.nist.gov/tnt/
36/36

Ga 4 GT

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Ga 4 GT

Uploaded by

Copyright:

Available Formats

Genetic Algorithms and

4.2. Automata with multiplicities...........................................................................................22

In this report, we begin in chapter 2 by explaining the principles of genetic

2.1. Evolutive systems background

Evolutive systems take their background in evolutionnist theory, synthesis of

● Evolution is the result of progressive alteration of living entities along

● Reproduction is based on the genetic character which undergo during

● Natural selection consist of keeping the well adapted individuals to the

2.2. History of evolutive systems

Evolutionary computation starts from the end of 1950s. As many research

developed then the evolutionary programming which consists in the definition

2.3. Genetic algorithms as tools for optimization problems

Traditional optimization methods based on analytical approaches are usually

● defining an initial approximation of the solution as start point of the iterative

Illustration 1: one optimum function (from [1])

These analytic methods generally lack in robustness and need to have a

Illustration 2: Two-optima function (from [1])

Illustration 3: multi-optima function (from [1])

To solve these difficulties which are inherent to classical analytical methods,

consist in choosing in each iteration a random position to evaluate the function.

● the ant system which is a new method of optimization based on bio-

2.4. Genetic Algorithms: the variation-selection principle

Genetic algorithms work on a population of individuals. The individuals are

To begin the algorithm, we generate a population of chromosomes.

Illustration 4: GA Individual and population levels

The algorithm is based on a variation-selection processus (cf. illustration 4):

The principle of selection is based on an evaluation function, called objective

The variation-selection processus is included in an iterative method which is

To conclude about this principle, this processus consists in the production of

2.5. Genetic Operations

We describe in the following each of these operations.

2.5.1. Reproduction/ duplication

Crossing-over is a process which, from two chromosomes, is able to generate

In this step, we chose a few chromosomes at random to be candidates for the

2.5.4. Schematic vue of the operators

In the following illustration, we show a schematic vue of these operators based

the associated bit is changed from white to black or black to white.

Illustration 5: Successive getnetic operations on bit sequence

2.6. A first hand made sample

We consider the basic optimization problem as following (from D.E. Goldberg

The population considered is composed of the binary representation of each

The initial population is constituated by the 4 following individuals:

Individual Performance Fitness

Individual Performance Fitness

13: 0110||1 \ / 01100: 12

24: 11||100 \ / 11011: 27

2.7. General algorithm

2.8.1. New genetic operators derived from the basic ones

Other operators can be defined in replacement or in addition to the basic ones.

● sub-chain sequences inversion

2.8.2. Ecological kennels

2.8.4. Evolutive strategies

● The random aspect is less developed and the reproduction is adeterministic

3.1. Basic overview of genetic libraries

● DGENESIS is a distributed implementation of a Parallel Genetic Algorithm.

● LITHOS is a stack based evolutionay computation system. It is written in

● GAEiffel is a library based on bit-string GAs written in the langage Eiffel.

● ECJ is a java-based evolutionary computation and genetic programming. It

● GAOT (GA Optimization Toolbox) implements evolution in the Matlab

● GECO is a toolbox for constructing genetic algorithms. It provides a set of

● Koza.gp is a Common Lisp implementation of the Genetic Programming

● GAlib is a general purpose toolkit which is described in the next paragraph.

3.2. Description of a GAlib