Solving the Traveling Salesman Problem through Genetic Algorithms with Changing Crossover Operators

Ryouei Takahashi Hachinohe Institute of Technology ryoei@hi-tech.ac.jp

Abstract
In order to solve the Traveling Salesman Problem (TSP) through Genetic Algorithms (GAs), a method of changing crossover operators (CXO), which can flexibly substitute the current crossover operator for another suitable crossover operator at any time, is proposed. This paper reports experimental validation of CXO through C software by using data of 200 cities. Key Words GAs, TSP, Changing crossover operators

1. Introduction
For solving the Traveling Salesman Problem [2], [11], [9] by Genetic Algorithms [1] many crossover operators which are countermeasures against yielding lethal genes have been invented. They are classified into two groups. The first group views an array of n genes which compose a chromosome and represent n cities arranged in the visited order as a permutation of n characters. Such as Grefenstette's method [3], Goldberg's PMX [2], Oliver's CX [4] and Davis's OX [5] belong to the first group, and they devise those methods for permutation representations or their exchanging forms to avoid lethal genes. The second group tries to perform crossover operation well from the view point of epistasis, of which a low degree is realized in the TSP by retaining some useful information about links of parents edges between adjacency cities, quickly to converge to the optimum solution. In epistatic domains, links of selected-parents edges partially have to coincide with the optimum solution. Hence we take into consideration of the required distances between adjacency cities for selecting the next visiting city. Such as Whitleys EX [6], Kobayashi-groups SXX [7], and Nishikawagroups EXX [8] belong to the second group. We investigated an improved EX which is an application of Whitleys EX. The idea of the improved EX comes from greedy optimization [2], [12]. Our experiments show that with the improved EX, which selects the next visiting city through only information about distances between

adjacency cities on a pair of parental chromosomes, we can obtain an optimum solution on earlier stage of generations. But it is a local optimum solution and it can not always agree with the global optimum solution. On the other hand, with SXX a parent changes his sub-path A with another parents sup-path B, where A has the same set of edges as B has. If we could select a pair of parent chromosomes who have higher fitness, SXX supplies us with better opportunities to yield children who have higher fitness. But our experiments show that SXX can not succeed on creating a pair of parents with higher fitness on earlier stage of generations and that it takes about a bit of time to converge to the best solution compared with other crossover operators. Hence, in this study, we investigate how to realize GAs which enable us to dynamically change crossover operators on any arbitrary time in order to efficiently select the shortest cyclic path with the least amount of time. We name this changing crossover operator CXO.

2. Changing Crossover Operators

A method of changing-crossover-operators (CXO) substitutes a crossover operator O1 for another suitable operator O2 on any arbitrary time. It observes the maximum-fitness MF of individuals generated by O1, and it exchanges O1 for O2 on a suitable time when MF becomes stable, where fitness is defined as (the summation of all the distances between two cities) / (the total length of the cyclic path ). The purpose of CXO is to find an optimum path with the least required time. Our experiments show that in early stage of generations improved EX that selects the nearest city from the current city successively is available for selecting the last path with the minimum required distance, and that after generations SXX that generates individuals in the next generation by performing a crossover operation on a pair of the selected parents that have local optimum sub-paths is available. Hence, in our experiments, we change improved EX for SXX.

Proceedings of the Fourth International Conference on Machine Learning and Applications (ICMLA05) 0-7695-2495-8/05 $20.00 2005

IEEE

Figure 1. The concept of Changing Crossover Operators (CXO). The concept of CXO is illustrated on Figure 1. Figure 2 shows the flowchart of the GA with CXO.

these selected sub-tours, four candidate children are generated as follows. (a) Sub-tour SX in the parent tour CX is exchanged with SY or its reverse order sub-tour SY, (b) Sub-tour SY in the parent tour CY is exchanged with SX or its reverse order sub-tour SX. We thin out children, leaving two that have higher fitness in the above four candidates. In our investigation, in order to improve efficiency for selecting the most proper sub-path, SXX randomly determines a length and a starting site of a basic sub-path which is used for selecting the corresponding sub-path in another parent. With this method, we can shorten the search time for selecting the optimum sub-path from order of n3 to that of n1, where n is the number of cities. Improved-EX (Edge Recombination Crossover) Improved EX is an application of greedy algorithm [2], [12] and NNB (Nearest Neighbor Algorithm) [10]. With improved EX we successively select the city that has the shortest distance from the current city out of adjacency cities. Each city A has a list of adjacency cities N (A) that are next to A on a pair of parental cyclic paths denoted by CX and CY. The first city A1 that the first child C1 visits is the first city on the parent CX. The second city A2 that he visits is selected out of N (A1) such that the distance between A1 and A2 is the shortest, where A2 is a city not visited by him and it belongs to N (A1). In this way, the child C1 repeatedly chooses the next-visiting city. If N (A) is empty and cities not visited remain, C1 selects the city that has the shortest distance from the current city. This selection continues until a set of cities not visited become vacant and we could finally determine C1s cyclic path. Similarly we determine the childs C2s cyclic path, beginning with selecting the first city on the parent CY as the first city that the second child C2 visits.

3. Case Study
3.1 Case data
The experiment was performed by using 200 cities positioned in Figure. 3. Their coordinates are (i, j), where i=0, 1, 2, ,19 and j=0, 1, 2, , 9. City_NO in this Figure is i+1+20*j. This Figure indicates a two dimensional Euclidian space. The distance between two cities is calculated by the Pythagorean theorem.

3.2 Setting GA parameters

In order to compare effects of crossover operations which take measures against lethal genes on the performance of GAs under the same condition, GA parameters except for crossover operation are set as follows. pc: the crossover-rate of mating 0.8 population size 1000 the maximum number of observed generations 1000

3.4 The best solution CXO finds

CXO which exchanges improved EX for SXX on the 32-th generation can find an optimum path P with the minimum length (=200) on the 97-th generation, and computer execution time ET for finding P is 211 seconds. The optimum cyclic path that CXO (improved EX SXX) finds is shown in Fig. 3.

3.3 An evaluation of CXO by using improved EX and improved SXX

In our C programming experiments, the validity of CXO is confirmed by substituting improved-EX for SXX. They produce offspring as follows. SXX (Sub-tour Exchange Crossover Sub-tours SX and SY are sub-paths in the parent tour CX and in another parent tour CY respectively, where SX consists of the same cities in his tour as SY does. From

3.5 Performance and functional evaluation of CXO

For not generating lethal genes, many crossover operators are invented. CXO changes crossover operators by using those operators. They are Grefenstettes operator, PMX (Partially Matched Crossover), OX (Order Crossover), CX (Cycle Crossover), improved EX (Edge

Proceedings of the Fourth International Conference on Machine Learning and Applications (ICMLA05) 0-7695-2495-8/05 $20.00 2005

IEEE

Recombination Crossover), SXX (Sub-tour Exchange Crossover) and EXX (Edge Exchange Crossover). Results of performance evaluation of these crossover operators are shown in Table 1. In Table 1, the minimum length, the generation on which the best solution is found, the required time to find the best solution seconds , and

seed_id used for generating the initial population are illustrated for each crossover operator. GA performance depends on seed_ids. Seed_ids examined in this study are 1, 10, 20, 30, 40, 50, 60, 70, 80, 90, and 99.

Initialize Set GA parameters such as pc (probability of crossover), population size population size, and the maximum number of generations observed Select the first crossover operator (OP1) and the second crossover operator (OP2) from seven candidates for the crossover operation. Default operators of OP1 and OP2 are improved EX and SXX respectively. Determine the optimal generation to change crossover operators from OP1 to OP2 under a proper seed_id Set the first crossover operator to perform crossover. Set the generation counter for zero. Create initial population and evaluate fitness of each individual with the above seed_id.

Execute genetic operations Is the current generation an optimum one for changing crossover operators? NO YES Set the second crossover operator to carry out crossover

Select a pair of mates at random from members of the current generation with the Roulette wheel operator

Perform crossover stochastically on selected parents to generate two offspring without lethal genes Thin out children, leaving one in two, and apply the 2-opt method to the survivor to generate a new one Store the offspring to the temporary pool YES Is number of members of the temporary pool less than population size? NO Evaluate fitness of each individual in the temporary pool and select the last chromosome with the shortest path by the end of the current generation

YES Is the current generation the end of generations observed? NO Take the place of members of the current pool by those of the temporary pool Increment the generation counter

Stop

Figure 2. Flowchart of the GA with a CXO (Changing Crossover Operators).

Proceedings of the Fourth International Conference on Machine Learning and Applications (ICMLA05) 0-7695-2495-8/05 $20.00 2005

IEEE

The result of maximum fitness evaluation of crossover operations is illustrated in Fig. 4. In this Figure, process how each crossover operator shown in Table 1 converges to the best solution is illustrated. Y-axiss value is the maximum fitness found by the generation and is a kind of the off-line performance [2]. Main results of this experiment are as follows. Improved EX itself can find the same optimum path P with length=200 on the 83-th generation. The execution time ET for finding the P is 229 seconds, and improved EX is inferior to CXO in point of ET. SXX itself can find the next optimum path P with length=209.1 on the 785-th generation. The execution time ET for find the P is 1131 seconds, and SXX is also inferior to CXO in point of ET. EXX has a similar result as SXX. Such as PMX or OX, which perform crossover operation from a viewpoint of permutation exchanging or representation, lack information about links among edges and their sub-total lengths. Hence, with PMX or OX the best solution does not converge to the optimum solution. The minimum length we can find by using OX is over 1,000 as shown in Table 1. With Grefenstette or CX, minimum lengths converge to values of 220 to 240, which are longer than the best length (=200) found with EXX or SXX but are shorter than those found with PMX or OX.

not fixed, and it varies corresponding to sites where cities are positioned. In our experiments, we try to find the best generation between the first generation and the 83-th generation which is the best generation with the shortest path we can find through improved EX. Especially, the execution time for searching optimum paths with the shortest length (=200) corresponding to the generation are illustrated in Figure 5. Experimental results are as follows. We can find out the optimum cyclic path with the minimum length (=200) in the shortest time (=211sec) at the 32-th generation, which is the earliest time to change operators to succeed in finding out the best path. Among 83 candidates for altering crossover operators, we can find out eleven optimum generations with shortest paths. As generations undergo for changing crossover operators, the best search time to find the optimum path becomes inferior to that of changing operators on the 32th generation.

3.7 Validation of CXO

Figure 6 validates an efficiency of the CXO that exchanges crossover operators from improved EX to SXX on the 32-th generation. It can find the best model with the minimum length by the least of time. Figure 6 indicates computer execution time ET and maximum fitness values of the optimum model that CXO, improved EX and SXX can find by the time ET. In Fig. 6, a symbol signifies the suitable time CXO changes crossover operators. In this experiment, it is the end of the 32-th generation and it

3.6 Selecting the best generation to change crossover operators

The best generation to change crossover operators is

: An optimum solution

length=200

Figure 3. An optimum cyclic path CXO (improved EX

SXX) finds ( in a case of 200 cities).

Proceedings of the Fourth International Conference on Machine Learning and Applications (ICMLA05) 0-7695-2495-8/05 $20.00 2005

IEEE

Performance evaluation of crossover operators (in a case of 200 cities). The generation Seed_id The best fitness The required on which the used for (=total The time to find the generating best solution Crossover operators distances/ minimum best solution with the the initial the minimum length seconds population minimum length length) is found * 1605.9 200 97 211 10 CXO improved EXSXX Improved EX 1605.9 200 83 229 10 SXX 1535.9 209.1 785 1131 70 EXX 1512.0 212.4 595 199 40 Grefenstette 1449.1 221.6 998 2549 20 CX 1375.7 233.5 999 37 70 PMX 282.3 1137.7 963 87 1 OX 313.1 1025.7 534 49 1 <Remarks> improved EXSXX *: On the 32-th generation, crossover operators are changed from improved EX to SXX. The probability of crossover operation is 0.8, population size is 1000, and the number of generations observed is 1000.

Table

Figure 4. Maximum fitness evaluation of crossover operators. takes 95 seconds since crossover operation starts. Figure 6 shows that regarding maximum fitness how CXO gradually realizes better improvement compared with crossover operation only with improved EX as execution time increases. Furthermore, as crossover operation only with SXX produces offspring with maximum fitness less than 1000 under this observation, we can not describe its graph in Fig. 6. Table 2 describes maximum fitness values of the optimum models that CXO (improved EX->SXX) and improved EX and SXX can find on the execution time during 206 and 217. This period includes the 211 second at which CXO can find the optimum path with the best fitness (=1605.9). In fact, by the time improved EX and SXX take its maximum fitness values of 1596 and 504.7 respectively and cannot find the optimum path. Besides, the minimum lengths of maximum models which the CXO, improved EX and SXX find at 211 second are 200, 201.2, and 636.4 respectively. Our experiments show that CXO is superior in functionality and efficiency to both improved EX and SXX.

4. Conclusion
Our experiments by using data of 200 cities in an Euclidean space show that CXO which selects an improved EX in early generation and selects SXX after several generations can provide with an approximate solution of the TSP. This experimental result suggests that changing crossover operators at arbitrary time according

Proceedings of the Fourth International Conference on Machine Learning and Applications (ICMLA05) 0-7695-2495-8/05 $20.00 2005

IEEE

Table 2.

Validation of CXO (improved EX - > SXX ).

Figure 5. Best_search_time for selecting the optimum cyclic path in CXO(improved EX->SXX).

Figure 6. Validation of CXO(improved EX->SXX). to city data structure is available to improve the performance of GAs.

References
[1] J. H. Holland, Adaptation in Natural and Artificial Systems, Univ. of Michigan Press (1975), MIT Press (1992).

[2] D. E. Goldberg, Genetic Algorithms in Search, Optimization, and Machine Learning, Addison-Wesley Publishing Company, Inc., 1989. [3] J. Grefenstette, R. Gopal, B. Rosmaita, D. Van Gucht, Genetic Algorithms for the Traveling Salesman Problem, Proc. of 1st Int. Conf. on Genetic Algorithms and Their Applications, pp.160 - 168, 1985. [4] I. M. Oliver, D. J. Smith, and J. R. C. Holland, A Study of Permutation Crossover Operations on the Traveling Salesman Problem, Proc. Of 2nd Int. Conf. on Genetic Algorithms, pp. 224- 230, 1987. [5] L. Davis, Applying Adaptive Algorithms to Epistatic Domains, Proc. Of 9th Int. Joint Conf. on Artificial Intelligence, pp. 162-164, 1985. [6] D. Whitley, T. Starkweather and DAnn Fuquary, Scheduling Problems and Traveling Salesman: The Genetic Edge Recombination Operation, Proc. of 3rd Int. Conf. on Genetic Algorithms, pp. 133-140 (1989). [7] M. Yamamura, I. Ono and S. Kobayashi, Emergent Search on Double Circle TSPs using Subtour Exchange Crossover, Proc. of 1996 IEEE Int. Conf. on Evolutionary Computation, pp.535-540 (1996). [8] K. Maekawa, N. Mori, H. Tamaki, H. Kita and Y. Nishikawa, A Genetic Solution for the Traveling Salesman Problem by means of a Thermodynamical Selection Rule, Proceedings of the 1996 IEEE International Conference on Evolutionary Computation (ICEC'96), pp. 529-534, 1996. [9] M. Garey, and D. Johnson, Computers and Intractability. A Guide to the Theory of NP-Completeness, W. H Freeman and Company, 1979. [10] S. J. Russell and P. Norvig, Artificial Intelligence A Modern Approach, Prentice Hall, Upper Saddle River, New Jersey, 1995. [11] R. L. Haupt and S. E. Haupt, Practical Genetic Algorithms, John Wiley & Sons, Inc., 2004. [12] E. Aarts and J. K. Lenstra, Local Search in Combinatorial Optimization, Princeton University Press, 2003.

Proceedings of the Fourth International Conference on Machine Learning and Applications (ICMLA05) 0-7695-2495-8/05 $20.00 2005

IEEE