Artif Life Robotics (2005) 9:117122 DOI 10.

1007/s10015-004-0332-6

ISAROB 2005

ORIGINAL ARTICLE

Ivan Tanev Thomas Ray

Evolution of side-winding locomotion of simulated limbless, wheelless robots

Received and accepted: October 18, 2004

Abstract Inspired by the efcient method of locomotion of the rattlesnake Crotalus cerastes, the objective of this work was the automatic design through genetic programming of the fastest possible, side-winding locomotion of simulated limbless, wheelless artifacts. The realism of the simulation is ensured by employing open dynamics engine (ODE), which allows accounting for all the physical forces resulting from the actuators (muscles), friction, gravity, collisions, and joint constraints. The empirically obtained results demonstrate that the complex side-winding locomotion emerges from relatively simple motion patterns of morphological segments (vertebrae). The robustness of automatically evolved locomotion is veried by (i) the reasonable performance degradation when partial damage to the artifact is inicted, and (ii) the ability to tackle obstacles. Contributing to the better understanding of the unique, side-winding locomotion, this work could be considered as a step toward building real limbless, wheelless robots, featuring unique engineering characteristics, which are able to perform robustly in difcult environments. Key words Genetic programming Locomotion Open dynamics engine Side-winding Snake robot

1 Introduction
1.1 Advantages and drawbacks of snake robots The worthwhile wheelless, limbless snake robots feature potential robustness characteristics beyond the capabilities of most wheeled and legged vehicles, i.e., the ability to operate in hostile (dusty, damp) or hazardous (radiation, chemical toxicity, pressure, structural weakness) environments, the ability to traverse terrain that would pose problems for traditional wheeled or legged robots, and insignicant performance degradation when partial damage is inicted. Moreover, owing to their modular design, snake robots are usually cheaper to build, maintain, and repair. The useful features of snake robots include smaller crosssectional areas, stability, terrainability, good traction, high redundancy, and complete sealing of the internal mechanisms.1,2 Robots with these properties open up several critical applications in exploration, reconnaissance, medicine, and inspection. However, compared to the wheeled and legged vehicles, snake robots feature (i) a smaller payload, (ii) more difcult thermal control, (iii) more difcult control of locomotion gaits, and (iv) relatively inferior speed characteristics. Considering the rst two drawbacks as beyond the scope of our work, and focusing on the drawbacks of control and speed, we intend to address the following challenge: how to control such a snake robot in order to achieve the fastest possible speed of locomotion. 1.2 Automated design of the locomotion of a snake robot While for many tasks handcrafting the code of robot locomotion can be seen as the natural approach, it might not be feasible for developing the control code of the snake robot owing to its morphological complexity. Moreover, any complex system, and a snake robot in particular, is more than the sum of its individual entities, more than the sum of the parts that compose it. The overall locomotion gait of the snake robot might emerge from relatively simply dened motion patterns of morphological segments of the robot.

I. Tanev (*) Department of Information Systems Design, Faculty of Engineering, Doshisha University, 1-3 Miyakodani, Tatara, Kyotanabe, Kyoto 610-0321, Japan, and ATR Network Informatics Laboratories, 2-2-2 Hikaridai, Keihanna Science City, Kyoto 619-0288, Japan Tel. 81-774-65-6699; Fax 81-774-65-6801 e-mail: {i_tanev, ray}@atr.jp T. Ray Department of Zoology, University of Oklahoma Norman, OK, USA This work was presented, in part, at the 9th International Symposium on Articial Life and Robotics, Oita, Japan, January 2830, 2004

118

The phenomenon of emergence additionally complicates the task of handcrafting the code of robot locomotion since neither the degree of optimality of the developed code nor the way to incrementally improve the code is evident to the human designer.3 Thus, an automated mechanism for solution evaluation and corresponding rules for the incremental optimization of the intermediate solution(s) would be needed. An additional argument in favor of an automated design for robot locomotion is that often the challenge in problem-solving is to develop a solution which is competitive or even better than a human-designed one. Such a solution might be beyond the abilities of a human to handcraft it. The proposed approach of employing genetic programming (GP) implies that the code which governs the locomotion of a snake robot is automatically designed by a computer system via simulated evolution through selection and the survival of the ttest in a way similar to the evolution of species in nature.

1.4 Objective The objective of our work was to investigate the feasibility of applying GP for the automatic design of the fastest possible locomotion of a realistically simulated snake robot side-winding. Inspired by the fast locomotion of the rattlesnake Crotalus cerastes, this work is motivated by our desires to (i) better understand the underlying mechanisms of the fascinating side-winding locomotion of natural snakes, (ii) explore the phenomenon of the emergence of the locomotion of complex bodies from simply dened motion patterns of the morphological segments comprising these bodies, (iii) verify the feasibility of employing ODE for a realistic software simulation of a snake robot, and (iv) investigate the practicality of building real snake robots. The remainder of this document is organized as follows. Section 2 emphasizes the key features of GP proposed for the evolution of the locomotion of a simulated snake robot. Section 3 presents empirical results of evolving locomotion patterns of a snake robot and discusses the emergence of side-winding locomotion. Finally, Sect. 4 draws a conclusion and discusses directions for future research.

1.3 Evolving a realistically simulated snake robot Evolving a snake robots locomotion (and in general, the behavior of any robot) is often performed as a rst step in the sequence of simulated off-line evolution (phylogenetic learning) of the robots software model, followed by on-line adaptation (ontogenetic learning) of the evolved code on a physical robot situated in a real environment.4 The justication to incorporate off-line software simulation into the process of robot controller design comes from the facts that the verication of robot behavior on physical robots is extremely time-consuming, costly, and often dangerous for the robot and the surrounding environment. Moreover, in some cases it is appropriate to initially model not only the locomotion, but also (to co-evolve) the most appropriate morphology (i.e., the number of phenotypic segments, the type and parameters of the joints which link them together, the actuators power, the type, number, and location of sensors, etc.) of the artifact,5,6 and only then (if appropriate) to physically implement it as hardware. The software model used to simulate the robot should fulll the basic requirements of being quickly developed, adequate, and fastrunning.7 The typically slow development time of GP stems from the highly specic semantics of the main attributes of GP, such as representation, genetic operations, and tness evaluation. The development time of GP can be signicantly reduced through incorporating off-the-shelf software components and open standards in the software engineering of GP. To address this issue, we developed an XML-based GP framework which, employing the open XML standard, contributes to a signicant reduction of the development time of GP by utilizing (i) the off-the-shelf document object model (DOM) parsers to represent simulated individuals in GP, and (ii) built-in API of DOM parsers to implement genetic operations (crossover and mutation). To address the issues of the adequacy and runtime efciency of snake robot simulation, we applied the open dynamic engine (ODE) a freeware software library for simulations of rigid body dynamics.

2 Approach
2.1 Representation of a snake robot The snake robot is simulated as a set of identical spherical morphological segments (vertebrae), linked together via universal joints. All joints feature identical (nite) angle limits and have two groups of actuators (muscles) that perform the rotation of the joint in the horizontal and vertical plane, respectively (Fig. 1). Considering the representation of the snake robot, the task of designing the fastest locomotion can be rephrased as discovering such patterns (in the time domain) of the desired turning angles of the horizontal and vertical actuators of each segment of the snake that result in the fastest overall locomotion. In order to realistically simulate the overall locomotion of a snake robot for a given pattern of desired turning angles of actuators, accounting for all the forces which result not only from actuators, but also from gravity, the mass of morphological segments, friction with the surface (the major driving force of wheelless, limbless locomotion), collisions, and joint constrains, we implemented the model of a snake robot incorporated into the ODE environment.8

Universal joint Horizontal plane

Vertical plane Segment #i Segment #i+1

Fig. 1. Morphological segments linked via a universal joint. Actuators of the joint perform the rotation of segment No. i 1 in the horizontal and vertical plane, respectively

119 Table 1. ODE-related parameters of the simulated Snakebot Parameter Number of phenotypic segments in snake Model of segment Type of joint between segments Initial alignment of segments in Snakebot Number of actuators per joint Orientation of axes of actuators Operational mode of actuators Maximum force of actuators Actuator stops (angular limits) Friction between segments and the surface () Sampling frequency of simulation Value 15 Sphere, R 0.2 Universal Along the Y-axis of the world 2 Horizontal along the X-axis; vertical along the Z-axis of the world dAMotorEuler 12 50 5 20 Hz

Table 2. Main parameters of GP Category Function set Terminal set Population size Selection Elitism Mutation Fitness Trial interval Termination criteria Designation {sin, cos, , , *, /} {time, segment_ID, pi, random constant} 200 individuals Binary tournament, ratio 0.1 Best four individuals Random subtree mutation, ratio 0.01 Velocity of the snake during the trial 600 time-steps (Fitness 100) or (Generations 30)

ODE is a free, industrial-quality library for simulating articulated rigid body dynamics. It is fast, exible, and robust, and it has built-in collision detection. The ODErelated parameters of the simulated Snakebot are summarized in Table 1. 2.2 Genetic programming Genetic programming (GP) is a domain-independent problem-solving approach in which a population of computer programs (individuals genotypes) is evolved to solve problems. The simulated evolution in GP is based on the Darwinian principle of reproduction and the survival of the ttest. The tness of each individual is based on the quality with which the phenotype of the simulated individual is performing in given environment. Applying GP for the evolution of a snake robot implies that the genotype of the snake is associated with two arithmetical expressions, which represent the patterns of the desired turning angles of both the horizontal and vertical actuators of each morphological segment. Since locomotion gaits are periodical, in the function set of GP we include the sin and cos functions in addition to the arithmetical functions. Terminal symbols include the variables time, and an index of the current morphological segment of the snake, and two constants: pi, and a random constant within the range [0,2]. The main parameters of GP are shown in Table 2. As depicted in the table, the tness function accounts for the velocity of snake and it is estimated from the distance, which the center of the mass of the snake travels during the trial. The tness evaluation routine is shown in Fig. 2.
9

function Evaluate(GenH, GenV: TGenotype): real; // GenH and GenV is the pair of arithmetical // expressions, which represent the genotype of snake var AngleH, AngleV :real; //desired angles CurrAngleH, CurrAngleV:real; //current angles InitialPos,FinalPos: 3Dvector; //(X,Y,Z) begin InitialPos:=GetPosOfCenterOfMassOfSnake; for each time step of the trial t do begin for each segment s do begin AngleH := EvalHorisontalAngle(GenH,s,t); AngleV := EvalVerticalAngle(GenV,s,t); CurrAngleH := GetCurrentAngleH(s); CurrAngleV := GetCurrentAngleV(s); SetDesiredVelocityH(CurrAngleH-AngleH,s); SetDesiredVelocityV(CurrAngleV-AngleV,s); end; Obtain the new properties (position, orientation, velocity vectors, etc.) of morphological segments of snake as a result of applying all forces. end; FinalPos := GetPosOfCenterOfMassOfSnake; return GetDistance(InitialPos, FinalPos)/(time steps); end;

Fig. 2. Fitness evaluation routine. Fitness is evaluated from the distance which the center of mass of the snake travels during the trial

Inspired by the exibility and recently emerged widespread adoption of the document object model (DOM) and the extensible markup language (XML), we represent the evolved genotype of the simulated snake robot (expressions GenH and GenV in Fig. 2) as DOM-parse trees featuring corresponding at XML text. Our approach implies that both the evaluation the desired turning angles during tness evaluation (functions EvalHorisontalAngle and EvalVerticalAngle, shown in Fig. 2) and the genetic operations are performed on a DOM-parse tree using off-theshelf, platform- and language-neutral DOM parsers. A corresponding XML-text representation (rather than an Sexpression) is used as a at format, feasible for the migration of genetic programs among the computational nodes in the eventual distributed implementation of GP. The benets of using DOM/XML-based representations of genetic programs, as documented by Tanev,10 can be briey summarized as follows: (i) W3C-standard XML schema offers a generic way of representing the grammar of STGPE; (ii) using standard built-in API of DOM-parsers for maintain-

120 Fig. 3. Snapshots of the evolved best-of-generation locomotion gaits: initial (a), intermediate (b), and nal stages of evolution (c). The best-of-run locomotion gait is shown in (d). The trailing circles depict the trajectory of the center of mass of the snake. Since the time-stamp interval between each of these circles is xed, and is the same (10 timesteps) for all four snapshots, the distance between circles comparatively illustrates the achieved velocity of locomotion

Fig. 4. Snapshots of the sample evolved best-of-run side-winding locomotion of the simulated snake (left, right; top, down)

ing and manipulating genetic programs; (iii) the OS neutrality of parsers; (iv) algorithmic language neutrality of DOMparsers, and (vi) inherent Web-compliance of the eventual parallel distributed implementation of GP.

100

Fitness

80 60 40 20 0 0 5
FAVR=96 FAVR=14

3 Results
Figure 3 shows sample snapshots of the evolved best-ofgeneration locomotion gaits. As the gure illustrates, in the intermediate stages of evolution a forward direction of locomotion emerges (Fig. 3b). However, despite the fact that tness is measured as velocity in any direction, the sidewinding locomotion emerged in all independent runs of GP, suggesting superior speed characteristics for the terrain considered and the morphology of the robot. Snapshots of side-winding locomotion are shown in Fig. 4. In order to verify the superiority of side-winding locomotion, we compared the tness convergence characteristics for the following two cases: (i) tness is measured in any direction, and

10 15 20 25 Generation #

30

10 15 20 Generation #

25

30

Fig. 5. Fitness convergence characteristics of ten independent runs of GP for cases when tness is measured as the velocity in any direction (a) and in the forward direction only (b). The average of the tness FAVR of all independent runs is estimated at generation No. 30

(ii) tness is measured in a forward direction only. The corresponding diagrams are shown in Fig. 5. As the gure illustrates, compared to the side-winding (Fig. 5a), on average the forward locomotion (Fig. 5b) features many (by nearly seven times) inferior speed characteristics. The desired horizontal and vertical velocities of the actuators of each morphological segment of the snake

121

are shown in Fig. 6. These velocities result in the characteristic circular motion pattern of segments around the center of mass (Fig. 6), suggesting that during side-winding the shape of the snake can be illustrated as a three-dimensional helix wrapped around an imaginable, rolling cylinder. Figure 7 demonstrates that the articial evolution of locomotion via GP is able to invent the improvised wheel of a side-winding snake as a necessary attribute of efcient locomotion. The robustness characteristics are illustrated by the ability of the snake robot to retain its functionality when partial damage is inicted. Table 3 shows the degradation in veloc-

ity for various numbers of damaged actuators in evolved sample best-of-run locomotion. The damage is evenly distributed along the snake. As Table 3 illustrates, even when more than 50% (8 out of 15) of the actuators are damaged, the snake retains 39% of its original performance. Additional damage yields a sharp decrease in the performance of the snake due to the unfavorable effect of the chain of three consecutively connected dead actuators on the locomotion gait. Another aspect of robustness the ability of the snake to overcome obstacles is illustrated in Figs. 8 and 9.

0.2

1.2 0.8 0.4 0 -0.4 -0.8 -1.2 0 50 # Time step 100

Velocity

Vv Vh

Elevation, Zcs-Zcm

0.1 0 -0.1 -0.6 -0.4 -0.2 0 0.2 Lateral displacement, Xcs-Xcm 0.4 0.6

Fig. 6. Desired horizontal (Vh) and vertical (Vv) velocities of the actuators of the sample evolved best-of-run side-winding locomotion

Fig. 7. Trajectory of the central segment (cs) around the center of mass (cm) of the snake for the sample evolved best-of-run side-winding locomotion

Fig. 8. Ability of a snake with evolved sample best-of-run locomotion to emerge from burial under a stack of boxes: initial (left), intermediate (middle), and nal (right) stages of the trial

Fig. 9. Ability of the snake with evolved sample best-of-run locomotion to clear a pile of boxes: initial (left), intermediate (middle), and nal (right) stages of the trial

Table 3. Performance degradation of the snake with partial damage Number of damaged actuators (out of 15) 0 Fitness 102 1 70 2 46 3 51 4 36 6 51 8 39 10 19 12 4 14 4

122

4 Conclusion
We have presented an approach for the automatic design through genetic programming of side-winding locomotion of simulated limbless, wheelless artifacts. The software model used to simulate the robot should fulll the basic requirements of being quickly developed, adequate, and fast-running. To address the rst of these issues, we employed an XML-based GP framework. To address the issues of the adequacy and run-time efciency of the snake robot simulation, we applied the open dynamic engine (ODE) freeware software library for the simulation of rigid body dynamics. The empirically obtained results demonstrate that the complex locomotion of side-winding emerges from relatively simple motion patterns of phenotypic segments (vertebrae). The evolved locomotion pattern of each segment is such that the segment is rotating in a circle-like trajectory around the center of mass of the simulated robot. This suggests that evolved side-winding locomotion can be viewed as a process of rolling of the body of a simulated robot in a three-dimensional helix shape. The efciency of side-winding locomotion is far superior to locomotion in the forward direction, suggesting that side-winding is the fastest possible locomotion far simulated limbless wheelless robots with the characteristics used in this study (morphology, limits of actuator forces, joint type, joint movement limits, etc.). The robustness of automatically evolved locomotion is veried by (i) the reasonable performance degradation when partial damage to the artifact is inicted, and by (ii) the ability to tackle obstacles. Contributing to the better understanding of the unique, side-winding locomotion, this work could be considered as a step toward building real limbless, wheelless robots, in which unique engineering characteristics are able to perform robustly in difcult environments.

Better adaptation (through epigenetic development employing histone code modications) to unpredictable, changeable environments and/or partial damage, and coevolution of locomotion patterns and relevant morphological traits (power and angular limits of actuators, number of segments in the snake, etc.) are planned for the near future.
Acknowledgment This research was supported in part by the National Institute of Information and Communications Technology of Japan.

References
1. Dowling K (1997) Limbless locomotion: learning to crawl with a snake robot. Doctoral dissertation, technical report CMU-RITR-97-48, Robotics Institute, Carnegie Mellon University, December 2. Mahdavi S, Bentley PJ (2003) Evolving motion of robots with muscles. Proceedings of EvoROB2003, the 2nd European Workshop on Evolutionary Robotics (EuroGP), Essex, UK, April 1416, 2003, Springer, p 655664 3. Morowitz HJ (2002) The emergence of everything: how the world became complex. Oxford University Press, New York 4. Meeden L, Kumar D (1998) Trends in evolutionary robotics: soft computing for intelligent robotic systems. Jain LC, Fukuda T (eds) Physica, New York, p 215233 5. Sims K (1994) Evolving 3D morphology and behavior by competition. Articial Life IV Proceedings, MIT Press, Cambridge, p 28 39 6. Ray T (2001) Aesthetically evolved virtual pets. Leaonardo 34: 313316 7. Jacobi N (1998) Minimal simulations for evolutionary robotics. PhD thesis, School of Cognitive and Computing Sciences, Sussex University 8. Smith R Open dynamics engine, 20012003. http://q12.org/ode/ 9. Koza JR (1992) Genetic programming: on the programming of computers by means of natural selection. MIT Press, Cambridge 10. Tanev I (2004) DOM/XML-based portable genetic representation of morphology, behavior and communication abilities of evolvable agents. Artif Life Robotics 8:5256