Computational Aesthetics as a Tool for Creativity

Gary R. Greenfield Department of Mathematics & Computer Science University of Richmond Richmond, VA 23173, USA ggreenfi@richmond.edu
ABSTRACT

We consider the problem of designing software tools that automatically evaluate the aesthetic content of images. Our purpose is to enhance the creative potential of generative art systems. We propose a number of metarules for enhancing creativity.
Author Keywords

BACKGROUND

Generative art system, computational aesthetics.

ACM Classification Keywords

J5. Arts and humanities: Fine arts.

INTRODUCTION

Following descriptions of interactive generative art systems by Sims [10] and Latham [13], a cadre of artists began exploring this new paradigm [2]. Since such systems require users to make hundreds of aesthetic decisions over an extended period of time, it is reasonable to ask whether software tools can assist this process. Digital artists such as Cohen, Knowlton, and Mohr have attempted to integrate software tools into their software systems to help structure the aesthetics of their personal artistic styles [4], however their tools are special purpose and do not address the general problem of developing principles and frameworks for guiding creativity, especially in the generative art setting. Previous research devoted to the computational evaluation of aesthetic images has focused upon assessing the complexity of simple compositions using statistical methods [12], or assessing the complexity of chaotic compositions using global measures [11]. We consider the problem of implementing software hooks for generative art systems in such a way that they help structure and guide the creative process. It is our claim, at least at a meta-level, that such mechanisms help foster creativity by helping to quantify what it really means.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. C&C'05, April 12-15, 2005, London, United Kingdom. Copyright 2005 ACM 1-59593-025-6/05/0004$5.00.

Software systems using simulated evolution for creating fine art are called generative, or evolutionary, systems. Using the simple genetic algorithm, such systems maintain a population of images that can be culled and repopulated for generation after generation. These systems use image fitness to decide which images will survive to re-populate the next generation. Even though generative systems may differ in how they represent image genotypes and how they transform image genotypes into image phenotypes, they are all able to recombine and mutate image genotypes and to associate numeric aesthetic fitness values to image phenotypes. In an interactive generative system, image fitness is determined by requiring a user to rank order images in the population. The challenge of automating the fitness ranking procedure is to do so in such a way that the aesthetic merits of image phenotypes are propagated from generation to generation, and the aesthetic potential of image genotypes is improved from generation to generation, yet the overall image making capability (i.e. creativity) of the system is not diminished. In simpler terms, the computational aesthetics challenge is to exploit the image making capability of the system by exploring the space of genotypes. Assigning image fitness non-interactively was initially considered for generative systems that used variations of the evolving expressions method introduced by Sims [10]. With Sims method, image genotypes are binary trees that can be viewed as image processing networks, and image phenotypes are obtained by evaluating the primitives found at the nodes during tree traversal. To oversimplify, we assume that systems of this type differ only in the sets of primitives they use. Using a Sims inspired system Baluja, Pomerleau, and Jochem [1] trained neural nets to rank order images within the population. Though much was learned from the experiment, their neural nets did not fare well either at producing aesthetic imagery or exploring genotype space. By implementing a coevolutionary algorithm that maintained populations of images and critics, Rooke [personal communication] was more successful in producing aesthetic imagery using his Sims inspired system. Rookes critics were trees packed with primitives for making statistical assessments

232

of selected portions of an image. However, because his evolved images were so similar from run to run, Rooke was not wholly successful at exploring genotype space.
CO-EVOLUTION USING A SIMS SYSTEM

In Greenfield [9] we also considered a co-evolutionary approach using a Sims style generative system. In order to concentrate on image aesthetics, the genotype-tophenotype mapping was constrained to produce grayscale images. To each host image phenotype 3x3 digital convolution filter parasites were attached. Parasites were used to create imperfections in the image by convolving their filters over small patches of the image. Image fitness was calculated on the basis of differences between the convolved patch and the underlying patch, while filter fitness was calculated on the basis of similarities between the convolved patch and the underlying patch. A sample image obtained from this generative system is shown in Figure 1. Similarities between images evolved using our system are most easily explained by the fact that the mathematical basis for the fitness calculation is entropy, thus image populations co-evolve to a state of equilibrium where white-noise phenotypes are predominant. The observation that co-evolution allows one to reach this equilibrium state following different evolutionary trajectories supports the claim that exploration of genotype space is occurring and creativity is being enhanced. Because the fitness computation is so unusual, it is natural to ask why our implementation is successful at producing different image styles. Co-evolutionary dynamics help ensure that during most runs no image species will dominate for too long, whence time devoted to exploitation, as opposed to exploration, will be bounded. Moreover, our design implements a local-global fitness principle that introduces a tension during simulated evolution as image genotypes must change the

characteristics of their phenotypes within patches and, as a consequence, also alter the overall appearance of their phenotypes. In other words, our implementation obeys the creativity meta-rule: Local modifications made to enhance creativity must have global consequences.
IMAGE COMPOSITION USING A SIMS SYSTEM

We now modify the genotype-to-phenotype mapping of the previous system so that pixels are assigned colors from a fixed palette by re-mapping their gray-scale numeric values. Aesthetic fitness calculations will be determined by the color information available in the phenotype. Our goal is to evolve images whose compositions have aesthetic merit. Although the details do not concern us here (see Greenfield [5]), in order to assign image fitness we first color segment each phenotype until it is reduced to a small, fixed number of regions. While segmentation is determined by image color, image fitness is determined by the compositional information these regions provide. We introduce three geometric assessments using the color segmentation: a weighted sum of areas A; a sum of boundary lengths B; and a sum of region adjacencies J. These sums help measure proportion, complexity, and connectivity respectively. We formulate fitness functions using linear combinations of these terms summed over fixed subsets of regions. Figure 2 shows an evolved image that resulted from a fitness function that rewarded images for having only a few large areas. Enhanced creativity is embedded within this system by a software tool that places decisions about the weights to be assigned to each of the terms A, B, and J in the fitness function as well as the region subsets to be summed over directly in the hands of the artist. The end result is that artists are encouraged to more carefully articulate design criteria and are better able to experiment and explore the consequences of those criteria. Once a fitness function has been selected, repeated evolutionary runs using this function tend to produce images that have a similar style. Thus choosing a fitness function greatly restricts the exploration of genotype space and further suggests that for generative systems there may be a relationship between the parameterization of fitness functions and the parameterization of image styles. A meta-rule arising from hand-crafting aesthetic fitness functions is: Software tools for enhancing creativity must be flexible.
A MULTI-OBJECTIVE OPTIMIZATION SYSTEM

Figure 1. An image obtained using co-evolution where fitness is based on an entropy calculation.

To help overcome limitations in exploring genotype space when using only one aesthetic fitness function, in [6] we considered multi-objective optimization. The NSGA-II implementation [3] was chosen. It uses multiple fitness functions in order to maintain species within the population by fostering image allegiance to different

233

Figure 2. An image obtained using an aesthetic fitness function rewarding large areas.

Figure 4. The same multi-objective run also supports an image with desirable area and boundary properties.

fitness functions in the hopes that hybrid images exhibiting several of the traits that fitness functions reward will arise. The price pay is that at the conclusion of each simulation run a pool of candidate solutions must be examined. Figure 3 and Figure 4 show two images that were bred during an evolutionary run that used three fitness functions. Each fitness function used only two of the three assessment terms that color segmentations provide. Sometimes aesthetic imagery did not evolve at all during runs using this system. At times this could be explained by the technical limitations of the underlying genetics, but more often it was caused by the fact that species were unable to produce hybrids due to the fact that the design objectives they were trying to fulfill were incompatible. Since it was so difficult to construct systems of objective functions to fulfill multiple design criteria we propose as a meta-rule: Design criteria for enhancing creativity must be compatible.

A MULTI-AGENT SYSTEM

We now consider generative systems that use radically different image genotypes and genotype-to-phenotype mappings. Our objective is to consider the difficulty of formulating and developing creativity tools from one generative system to the next. In [7] we considered the problem of constructing fitness functions that aesthetically rank ants in a small colony whose task was to aesthetically paint a grid. Ants are placed on a uniformly colored grid and their genotypes control their behavior. Different colors represent different scents and genotypes encode rules to be used to determine which colors ants search for, which colors ants deposit, and their movement tendencies as they traverse the grid. Since the style of an ant painting is dictated by individual ant behavior and not by the overall composition of the painting itself, and since ant behavior is the result of local interactions with colors and with other ants, the measures we used to assess ant fitness were V, the number of times an ant was able to follow scent, and F the number of

Figure 3. Multi-objective optimization supports an image with certain boundary and connectivity properties.

Figure 5. An evolved ant-painting from ants rewarded for balancing their scent following and exploration behavior.

234

assessment parameters. An aesthetic principle to ensure different color schemes could be explored was enforced by bounding the minimal variance of the three color cell products away from zero. The image in Figure 6 used this principle but also imposed additional composition design criteria by using Na to introduce a reward term and Nd to introduce a penalty term to image fitness. Surprisingly, very few of our ``ideas for articulating aesthetic criteria via fitness functions actually produced aesthetic imagery with this system. Presumably this is because this system is not particularly rich in aesthetic imagery. This observation leads us our final meta-rule: Creativity enhancement tools require a sufficiently rich generative environment.
REFERENCES Figure 6. An evolved gene-regulatory network rewarded for its color differences and high level of cellular activity.

1. Baluja, S., Pomerleau, D. and Jochem, J., Towards automated artificial evolution for computer generated images, Connection Science (1994), (6)325-354. 2. Bentley, P., Evolutionary design by computers, Morgan Kaufmann (1999). 3. Deb, K. Multiobjective optimization using evolutionary algorithms, John Wiley & Sons (2001). 4.Edmonds, E. and Candy, L. Creativity, art practice and knowledge, Comm. Of the ACM, ACM Press (2002), 10(10), 91-95... 5. Greenfield, G., Color dependent aesthetics for evolving expressions, 2002 Bridges Conference Proceedings, Central Plains Publishing, 9-16. 6. Greenfield, G., Evolving aesthetic images using multiobjective optimization, 2003 Congress on Evolutionary Computation Proceedings, IEEE Press, (3) 1903-1909. 7. Greenfield, G., Evolutionary methods for ant colony paintings, submitted for publication. 8. Greenfield, G., Genetic learning for a biologically inspired aesthetic process, submitted for publication. 9. Greenfield, G., On the co-evolution of evolving expressions, International Journal of Computational Intelligence and Applications (2002), 2(1) 17-31. 10. Sims, K., Artificial evolution for computer graphics, Computer Graphics, ACM Press (1991), (25) 319-328. 11. Sprott, J., The computer artist and art critic, Fractal Horizons, St. Martins Press (1996), 77-115. 12. Staudek, T., Computer-aided aesthetic evaluation of visual patterns, 2003 ISAMA-Bridges Conference Proceedings, University of Granada (2003), 143-150. 13. Todd, S. and Latham, W., Evolutionary art and computers, Academic Press (1992).

distinct cells an ant was able to visit during its traverse. Ant paintings were evolved using fitness functions proportional to F, F+V, and FV. Figure 5 shows an example using FV. Since it is remarkable how closely V and F are analogous to exploitation and exploration, we propose as a creativity meta-rule: Exploration and exploitation are universal creativity enhancement concepts.
A REGULATORY GENE NETWORK SYSTEM

In [8] we considered a model for simulating cellular morphogenesis in order to implement yet another generative system. In this system images were visualizations of cellular processes. Two types of simulated cells were randomly placed on a grid. The genotypes of these cells implement regulatory gene networks that manage cellular processes over time. Changes in cell product concentrations are governed by ``rules extracted from cell genotypes on the basis of current cell product concentrations. The key point is that because diffusion occurs between cells, rules extracted for cells with identical genotypes vary over the course of time. Concentrations of three of the cell products are used to assign colors to cells. In order to evaluate the aesthetics of the visualizations, the distribution of the concentration of each cell product x is used to compute its variance Sx. Two other global parameters calculated are the number of active cells in the grid Na, and the number of dormant cells in the grid Nd. Considerable computational effort is required to simulate the morphogenesis of the cells, whence small grids, small population sizes, and a limited number of generations were used in the generative system. Figure 6 shows an evolved image that was obtained from fitness functions formulated using our

235