the articulation of visual experiences through algorithm

Carlos Lunetta

The articulation of visual experiences through algorithm by Carlos Lunetta

Thesis submitted in partial fulllment of the requirements for the degree of Master of Fine Arts and is approved by the MFA Design Review Board of the Massachusetts College of Art in Boston.

May 2005

Jan Kubasiewicz Thesis Advisor

Joe Quackenbush Thesis Document Advisor

Brian Lucid DMI Faculty

Toby Bottorf External Reviewer

Abstract

The Articulation of Visual Experiences Through Algorithm explores the concepts and possibilities computation presents as a creative medium for design. This thesis investigates the relationship between code and visual language. From this relationship, a new vocabulary of new media will emerge, driven by ideas that stretch their existence simultaneously through all means visual, aural, verbal, haptical, temporal and logical. My methodology was based on discrete experiments native to the computational medium which investigated a range of topics including: musical intervals; generative and interactive form systems; interactive op art; database visualization; and the plastic components of time.

Aknowledgements

I traveled 4817 miles from So Paulo to Boston to improve myself as a designer. This would have been impossible without the support of my family, but I own absolutely everything to my wife Margarida, and her love, support and patience. Id like to express my gratitude to Jan Kubasiewicz, not only my mentor, but also a good friend. To Joe, Gunta and Brian, for all the worthy hard work. To my fellow classmates, Stephen, Keiko, Kate, Karolina and specially Lynn (and Ronaldo) for all the support and friendship. To my friends Colin Booth and BJ Jones, for trusting and receiving a Brazilian weird student at the US. To Dave and everyone at Beam interactive, for the opportunity of practicing the thesis preach. To Ben Fry, Mario Klingemann and all Processing community. And nally to Google, for returning 264,000 thesis acknowledgments to my amusement. There are surely a lot of grateful people out there!

Table of Contents

Denitions Roots Concepts Praxis Experience Case Studies

Sound in Space Form Systems Riley Patterns Logic Molecules Patterns of Time

17 27 39 53 67 71
73 85 99 109 115

Findings Bibliography

121 125

11

Preface

Every work of art is the child of its age [...] Each period of culture produces an art of its own which can never be repeated. Wassily Kandinsky

13

Disclaimer

To dene is to limit. Oscar Wilde, in The Picture of Dorian Gray The limitation serves a purpose - better understanding. This thesis document denes a series of concepts and ideas, many of them vast, open and entangled. By any means those denitions intend to limit concepts they mean to raise understanding by disclosing relevant issues. Ideas are not and shouldnt be - limited to their verbal counterparts.

15

Denitions

Visual experience, dened.

Design is an act of assembly of form through composition.1 A composition that has a purpose: to communicate. The idea of communication implies that a message will eventually be received by someone; there is an audience, a receptor. Also, communication implies the existence of a medium, a dimension where composition becomes reality. Composition, audience and medium are linked together: each one inuences and is inuenced by the other, in a mutual, constant change. The term visual experience is dened by the connection of these three concepts; a composition acknowledged of its audience and aware to the nature of its medium.

a model for visual experience

Design is the method of putting form and content together. Design, just as art, has multiple denitions, there is no single denition. Design can be art. Design can be aesthetics. Design is so simple, thats why it is so complicated. Paul Rand

17

Computer: a new medium

The personal computer has worked as a tool for design production and artistic experimentation since its tender age. During the mid-nineties, though, a shift in the role of computing occurred: the advent of the internet, that allied with the proliferation of household computers gave birth to a new mass communication medium. Computational devices2 became a medium, with an important difference from other known media they allow direct feedback and participation from the audience. The concepts of broadcaster and viewer got blurred, since one can become the other after few keystrokes. As a medium, computation affects the audience. Consequently, graphic design is inuenced by the change; the new medium opens a myriad of possibilities for creation in time, space, shape and sound. The participation of the audience raises the concept of interactivity. Computers have became no longer just a production tool, but also the living space for the design creation.

New media creativity

Once a new medium has been established and the gates for new dimensions in design have been opened, a question arises: how is the conception of visual experiences changed? How can new media affect creativity? The initial transformation in design was an adaptation; new media design was a collection of items from different media bonded together. A print design piece would receive navigation buttons, a small animation intro and be published as a new media piece. The adaptation evolved according to the features that mainstream graphic software packages would offer; software companies like Adobe and Macromedia ended up dictating the aesthetics of interactive communications by offering canned solutions, like templates and plug-ins,

Although this document aims the exploration of personal computers, like desktop or portable systems, any micro processed object is considered a computational medium: cell phones, palm tops, text messengers, DVRs and others

18

and generating design elements according to their own proprietary algorithms, like vector antialiasing and motion tween functions. Up to this point in time, the creative process per se changed very little more dimensions were added, but the visual experience conception remained the same. Under the navigation buttons and animated items, conceptually print media pieces live online.

Birth
The computer is both a medium and a tool for design but the creative process seems to still be far from the machine. Why is it the case that the principles of computation are not present in the creative framework, in the conception of a visual experience? They are present, but for a very small number of designers, still far from the mainstream. Theres a gigantic gap between the number of designers that use computers as a tool and the number of designers that use the computer as a part of the concept in the design work. There are a small number of design pieces that can only exist and be understood inside the digital medium. The language of design is in constant evolution, and now it faces the opportunity to embrace the concepts and processes of its new hosting medium, becoming native, being reborn.

What are the concepts of computation?

Computation is about change. The personal computer is a highly evolved calculator, performing billions of arithmetical and logical operations per second. The processor works by getting certain inputs, performing calculations and outputting them; data is constantly being chopped to small instructions, calculated and rebuilt. Lets use a keyboard input, for example; the physical keystroke is converted to a binary signal; the signal is processed and converted to a visual output, the graphical representation of the key that is displayed

19



a model for computation

on screen. The core of the computational process is about translating, converting a certain input to a certain output. The creative use of computation takes advantage of the nature of data processing in favor of design. There are situations where traditional design meets the limits of human labor; a computer can address a different scale than human hands in artistic creation, dealing with the very large, the very small, the very precise, the very exhausting, so on and so forth. Some of the computation characteristics are: Transformation: as stated above, the main concept of computation is to transform data. Repetition: a processing instruction can repeat itself eternally, as long as its physical part works. An element can repeat itself over time or over space the replication is lossless. Randomness: true randomness comes from nature, in events like radioactive decay. Computer systems feature pseudo-random numbers, a formula that produces no repetition, smooth numeric distribution, and more important: the lack of predictability. Precision: computer processing elements can be extremely exact and precise when necessary. Predictability: once a set of rules is created, the code behaves accordingly. Just like the physical world is attached to the laws of physics, a program can be attached to a set of unbreakable laws that will be always obeyed.

20

 Discretion: program elements can be discrete; events can be triggered by any kind of input rather than linear time. This is the key for interaction. Complexity: computation can achieve and display unbearable amounts of visual or logical information. Simplicity: the opposite of complexity; the ability to complete a task with a clear, austere and minimal output. Remembrance: the ability to store data, and recall it later, with no logical capping of time or quantity the only limit is physical. The concepts of computation are implemented in a computer code as formulas, or sets of instructions known as algorithms.

The origins of the gap

There is a resistance in incorporating computer processing characteristics in the concepts of design. The resistance has mainly two reasons; The rst reason is the difculty in communication between humans and computers; the computer language is different than common verbal language. Some specic training is necessary so one can be fully able to read and write computer code. At the same time, the amount of training today is a small fraction from the amount of training twenty years ago. Computer language is evolving at a similar speed of the computer hardware evolution. Coding is more accessible now than ever, and theres a prognostic for future improvements. The challenge is to learn what to tell the computer and not simply how to tell it. Language syntax depends on the available technology, but the idea to be coded is timeless. The second reason is cultural and social; western education branches arts and humanities from math and science. From high school to postgraduate education, theres a strong sense of separation between different areas of knowledge. Computational design merges art, math, physics and even psychology, when dealing with the study of interactive design reaction. The union of different branches of knowledge is required accompanied by a mindset prepared to create blissfully in any necessary eld of study.

21

Verbal, visual and computational

How different is the computational language from visual language? The human skill of communication is granted by sensorial channels; ideas, facts and feelings are understood and expressed whether in verbal, visual or any other sense. The compounding elements of a channel have similar counterparts on another channel; correlations can be addressed between elements with similar conceptual functions The correlation of elements has a clear role in the understanding of language; think about the word red. This word has strong visual information attached to it, representing the visual hue known as red. The experience of red can be expressed verbally, through the word, or visually, through the hue. It also can be learned both ways. We can understand a concept by contextualizing it in a communication channel; words can describe visual objects, phrases can describe visual actions - and vice versa.

correlations between verbal and visual domains

22

domain correlation examples

The computational language acts as an extra channel - it has similar structural properties to other communication means, thus, correlations of similar elements in the structure can be established. The computer language, in essence a binary series of logical and arithmetical operations, can be understood by contextualizing other channels especially, but not limited to, verbal and visual realms. We can communicate to a computer by typing verbal commands; the verbal will become a digital process, returned and converted to any output verbal, visual, haptical, sonorous or any other available. Language is a system4, where elements are composed to form meaning; The structure of the computational language is similar to the structure of visual, verbal or any other language; a structure of elements interacting in a composition. Elements can be arranged to work together in a logical environment just as in the visual or verbal form.
4

A language is a system of pure values Ferdinand Saussure

23

correlations between verbal, visual and logical domains

domain correlation examples

24

Module and System

Module is an independent part, a single element that exists by itself, having built-in functionalities. System is the working interrelation of modules, comprised as a whole. Scale is a determining factor to dene something as a module or a system; an atom can be taxonomized as a module a unique, independent part or a system, formed by the particles and sub particles that interact within its apparent unity. The building blocks that dene a language are considered modules; each module is unique, and represents a concept, an idea. Modules interrelate, creating a system - the composition. Words interact in a sentence, shapes interact in a canvas, and programming objects interact in a code.

the structure of model/system, with an example of application.

Articulation
The articulation of ideas in the visual, verbal or computational dimensions is based on the composition of elements. As initially stated, design is composition - acknowledged by its audience and medium. The articulation of visual experiences can grow along the creative potentials of the new computational medium; Its time to articulate visual experiences through algorithm.

25

Roots

Art, Science and Religion

The underlining concepts of computation and design have met many times through history; accessible computation may be a new format, but mathematical and scientic concepts aligned with art are not. The branches of knowledge have been treated as one through time; the knowledge of understanding nature, the knowledge of understanding the spirit and the knowledge of expressing both. Science, religion and art are linked together in the history of civilization. Pyramids, temples and statues that remain still prove how engineering and art were combined in service of belief.

a solid articulation of geometry.

Notable thinkers in history also show that the path of science and art can be one; Greek philosophers like Plato and Aristotle, for example, investigated math and physics along with aesthetics and poetry all inquiries about the real world. Jumping in time and space, one of the best examples is Leonardo da Vinci. Da Vinci had a strong grasp of mathematics and engineering, showed in his fabulous inventions, and strong artistic ability, proofed with his invaluable paintings and sculptures. His creativity had no borders.

27

The renaissance era of da Vinci gives us an important demonstration on the incorporation of mathematical and scientic concepts in the artistic creation. Before this movement, medieval paintings were visually at. Space and volume were represented, but did not reect reality. The artistic composition, as a concept, was focused in the symbolical components; the concept of composition was centered in the meaning.

Sassettas The Meeting of Saint Anthony and Saint Paul:the vanished vanishing point

The renaissance marks the advent of perspective techniques. Perspective drawing was initially a work of architects only, used in construction projects. But the new aesthetic of space spread and many talented painters embraced it when creating their work. Art as opposite of science? Masaccio, who is often hailed as the rst painter of modern times, was a member of the guild of apothecaries and grocers Karl Gerstner In a non-digital way, Renaissance paintings are visual experiences articulated through algorithm; the formulas of perspective were carefully implemented when picturing space, scientic data was used to create visual compositions and geometry became part of the creative process.

28

Masaccios Trinity. Masaccio employed complex mathematical detailing when producing his work.

Advancing more in time, we arrive at the modern model of knowledge categorization; each eld of study is highly dened and specialized. As civilization evolves, science grows in complexity its intangible to grasp all elds of knowledge. Art became a distant branch, and even the lower stages of education impose a learning path choice. After a certain high level of technical skill is achieved, science and art tend to coalesce in esthetics, plasticity, and form. The greatest scientists are always artists as well. All religions, arts and sciences are branches of the same tree Albert Einstein
29

Physicists like Albert Einstein and Werner Heisenberg believed that knowledge has similarities in different specializations and theres a mutual benet of connecting those similarities. The Massachusetts Institute of Technology (MIT) plays an interesting role in this relationship. Renowned for the development of groundbreaking technologies in different elds of science, MIT maintains the CAVS Center for Advanced Visual Studies, founded by Gyorgy Kepes. Kepes was a designer that did not believe in the separation of art and science; his explorations on visual language and his books reect this idea, and for almost thirty years (1946-74) his presence at the MIT maintained this ideal alive. An experimental effort to encourage such interthinking between different disciplines in the visual arts and scientic and technical elds is more than overdue. As the twentieth century has grown older, such intercommunication has become seemingly more improbable The dynamic interdependence of the wisdom of the senses and wisdom of reason is fundamental to a balanced growth Yet, history shows us that art and science, two basic human activities springing from a common love of nature are interdependent. Each achieves stronger growth when cross-pollinated by the other. Gyorgy Kepes

Computation
The domains of visual and computational languages met right after the advent of processing machines, during the fties. The early explorations involved tweaking the graphical output of huge computers to record lm frames and create animations, or by simply photographing an oscilloscope display that could be programmed. John Whitney, Sr. was one of the most important founding fathers of computer graphics.

30

The union of color and tone is a very special gift of computer technologies Time has become visual My computer program is like a piano. I could continue to use it creatively all my life. John Whitney, Sr. These quotes regard Whitneys explorations on computational abstract animation and music1. In 1962, A. Michael Noll at New Jerseys Bell Labs created a computer program that featured an algorithm whose function was to build visual compositions; the reference for the algorithm rules was the composition style of Mondrian.

Oscilloscope recording by Whitman

The work of Noll has extreme importance; besides Mondrians mimic, his programs investigated composition from a probability stand point visual forms were derived from a computational concept. Lines would be drawn (or not) according to the visual density of the composition. Other Noll
1 Two of his short movies can be accessed at http://www.siggraph.org/artdesign/prole/whitney/nobio_intro.html 31

works dealt with geometric randomness or repetition; he clearly took advantage of the newborn computation process in a creative way. The machinery evolved a lot since then, but Nolls ideas are still fresh there is no conceptual obsolescence. At the same time, Douglas Engelbart was creating the foundations of electronic communications and computational ofce work. Engelbart developed the mouse, the hyperlink, video conference, le structure navigation, word processing and more. Although he wasnt pursuing artistic objectives, his work supports the employment of creativity in the computer medium, proposing to use the machine to augment the human intellect.

Nolls Mondrian experiment

The MIT made strong contributions in the graphic computing eld. In 1963 the Sketchpad was created the rst CAD (Computer Aided Design) interface. Sketchpad was extremely advanced for its time; the user would interact with the application directly on screen, with a pen, through a graphic interface. All the interaction concepts of this system are the basically the same we use today.

In the early seventies, Xerox Corporation founded the Palo Alto Research Center, also known as PARC. It was inside the PARC that most of the modern concepts used in computation were invented, like the icon, object oriented programming, laser printer, WYSIWYG (what you see

32

The SketchPad

is what you get), computer notebooks, paint and animation applications (before that, everything had to be coded or use dedicated hardware), midi composition and many other ideas that shaped the terms multimedia and graphical interface. The PARC computers operating system was called SmallTalk and it featured something that is still lacking in todays operating systems: seamless integration between the graphic interface and the code behind it. The user was able to customize, change and create new functions whenever it was necessary, since there was no barrier to explore the operating source code. PARC successfully conducted experiments where teenagers were taught to code in SmallTalk language, and soon they would use their creativity to generate graphic applications. SmallTalk can be considered the golden age of computational design, when the visual and the logical sides were always connected and evident. The contemporary contribution from the MIT comes from a new generation of scientists/artists/designers developing projects regarding the

PARC SmallTalk - 1976

33

interaction of humans and technology. This happened at the Visual Language Workshop, in 1973, and later, in 1982 when the Media Lab was founded by Nicholas Negroponte. The specic topic of visual language mixed with computation was more intensely researched years later by Muriel Cooper, a pioneer in digital typography who worked to develop the language and structure of electronic design, and to expand the graphic design culture to four dimensions. John Maeda, considered Coopers successor, founded the Aesthetics + Computation2 group, where many researchers were involved in visual issues through programming, and how the visual language could be expressed and interacted with an audience through computation. The Aesthetics and Computation group started its operations at the same period that the internet blossomed and desktop computers became present in common households. Designers started to create for the computational medium; the graphic software packages would keep the programming language underneath a friendly interface, protecting designers from the necessary internet coding. The edge of programming for artists was consolidated with Macromedia Flash and Macromedia Director, commercial software packages where advanced users could script their own visual behaviors code. The term Aesthetics + Computation wasnt just university lab concept anymore it was present in the society. One of the projects from Maeda dealt with accessible computer coding focused in visual elements, called Design by Numbers a coding interface whose strength was in its minimalism, where the communication between designer and machine could be clear. Maedas project appears to have strong inuence from the previous Turtle project from MIT, a software application where an element on screen called turtle could be moved through code in order to build images. The turtle project later was know as Logo, and was largely used in schools during late seventies/early eighties. Logo helped to teach the basic concepts of computing the same idea that Design by Numbers explore. John Maedas work also helped to spread the idea that design could be achieved through computational means, that creativity could be expressed through code with his interesting explorations of algorithmic visuality.
2 http://acg.media.mit.edu/

34

Maedas Design by Numbers interface

Logo interface

The second project from MIT towards a visual creation code was created by Ben Fry and Casey Reas, and is called Processing. It adds a great number of features and extend Design by Numbers, providing more creative freedom and keeping the accessibility for non-computer scientists. Processing has today around 1000 users a small number compared to the whole interactive design community, but a big number of participants in a non-commercial, non-advertised university project, that is still entering its beta stage of development. The most important fact: the massive majority of the users are interactive designers, who were initiated to scripting languages when developing for the internet, and sought ways for doing more. Processing has no timeline, no visual thrills, and its interface is a simple text editor. But the concept of the application is clearly targeted to the artistic creation, hence its success among the design community.

35

Processing Beta interface

36

Concepts

Creative Framework
The idea of a creative framework can be explained through examples; lets use painting and photography. Two forms of expression that share similar characteristics theyre visual, compositive, bi-dimensional, have color hues or tones that form shapes and they communicate through visual information. Although similar, theyre also very different. Its easy to discern one from another; they have different aesthetic values, and generate completely different reactions. Photography strongly reects reality, augmenting or diminishing an aspect of it by light, texture and positioning; photography is subtractive the artist takes the physical world and frames a theme out of it, a part out of the whole. The kinds of camera, lenses, time of the day or season of year are manipulable elements that the photographer can use to achieve his intended visual result. Painting has its strength on expressing the authors internal world; even a photo-realist painting translates the artists view of light, color, form. A painting is built from a clear canvas; it is additive, colors and shapes are added to build a visual piece. Pigments, solvents, brushes and stroke techniques are elements that the painter can manipulate on creating his work. Both examples - painting and photography - are visual experiences; compositions that communicate in the visual realm. But theyre built through different creative concepts, using different creative tools. Creative concepts are the mechanics of expression in a medium the ways that an expression can become reality. Light is a concept in photography; space is also a concept. Both determine how the visual experience of a shot will be. In the same way, shapes, colors and dimensions are concepts

39

in painting; each medium has its own set of procedures. The instruments that allow those concepts to materialize - lenses, brushes, pigments, lm are creative tools.

Computational Concepts, revisited

As stated initially, computation is about change - and its mechanics feature concepts like transformation, repetition, randomness and others. Pure computation, just like pure mathematics, is an abstraction; it does not necessarily have a component in the visual domain in order to exist. If a piece of code is written on screen, the written letters are just a visual representation. The computational language can be reduced to a series of electrical signals traveling trough logic circuits. The elemental state of computation is a traveling electron. It might appear difcult to correlate such abstract medium to the creation of visual experiences, but we can compare it to cinema; the nucleus of the moving picture is time the articulation of visual experiences through time. What is the visual component of pure time? Can time be touched, seen? Time is an abstraction that applied to the photographic medium produces montage composition in time, a movie. We are able to understand pure abstraction by contextualizing it in a tangible medium. Film, for example, contextualizes time through image montage - ashbacks, slow motions, time lapses - the lm strip is able to demonstrate properties of time visually. The pure abstraction of computation can be contextualized - visually, through pixels, or verbally, through code. The intangible traveling electron can, nally, generate a visual experience.

40

Modules
Scale determines if a concept should be understood as a module or as a system the category depends on the point of view. In a standard user level point of view, the general computational module is called data. Data is a computational fact, the declared digital information. primitive and composite. Primitive data is single information, attached to a value. Numbers are primitive data. There several categories of numbers, according to the amount of bytes they take into the computer memory. Logical states (true or false) and single characters (a, b, c) are also primitive data. The value of a declared data can be xed, or vary according to a function; data with mutating values is simply called variable. The category of data is called data type. There are two kinds of data types:

1 1.0 1e20 1.77e-3

true false

hello

{1, 2, 3, 4}

class example { var example1; var example2; }

41

Composite data is the name given to primitive data combined; a word, for example, is a combination of characters, and is called string. Arrays are another kind of composite data: theyre groups. They can be groups of other groups (nested arrays) number, characters, objects or any other datatype. Classes and Objects are composite data. Theyre an important concept to be grasped, since almost all modern computation is based on Object Oriented Programming. Object is a philosophical term: it denes something, a particular item that has properties and relations. A software object can be compared to a physical world object, or a visual object.

> this.teaPot

a beautiful physical object

a reference for a logical object

Every object belongs to a class: the class is the template, the category that denes the possible properties that an object can have.

class Teapot { oat size; oat color; String lid String spout }

things the make a real teapot be a teapot: 1.ability to hold liquids 2. handle 3. lid 4. spout

class Teapot: things the make a teapot object be a teapot. The initial dened properties are size, color, the kind of lid and the kind of spout.

42

The main feature of programming through objects is the modularity: objects can be multiplied, reused, changed, and are independent beings inside a system. Another important computational concept is the function. A function describes an action to be taken, the relation between elements to be realized; it drives the change on any kind of computational module.

Modules in action: system

Computation is about change; through functions or other methods, computational elements can be put to work, perform a concept act as a system. Lets review some of the systematical concepts of computation: Repetition A system is never truly static except when data is stored in a physical media, its existence lies within electrons. A static module is an illusion: it is a module that is repeating itself over time, reafrming its existence on each circuit cycle. The invisible repetition can be visual, when a graphic element stands in time - or repeats itself over space. The simultaneous repeated copy is called instance. In the following example called repetition.pde, coded using Processing, a rectangle object is repeated over space - and innitely, over time. The code also performs two iteration loops to proper position the instances along a grid.
void setup() { size(100,100); ll(0); } void draw() { background(255); for(int i=0; i<3; i++){ for(int j=0; j<2; j++){ rect(10+i*30, 25+j*30, 20, 20); } } }
43

 Transformation When a property of a module is being changed over space, or time, or any kind of input, a transformation is occurring; almost everything in computation is an example of transformation. The following simple code called transformation.pde increases the value of the x coordinate of a square over time. When the edge of the screen is reached, the value is reset, and the transformation starts again.
int x = 0; // coordinate in x axis void setup() { size(100,100); ll(0); } void draw() { background(255); x = x+1; if (x>=width) { x = 0; } rect(x, 40, 20, 20); }

Randomness As initially written, true randomness comes from uncertain events in nature. A known source of uncertainty is the radioactive decay, or the movement of gas atoms in vacuum. Randomness is a controversial topic in physics and mathematics, but no technical background is required to use the idea of unpredictability in while designing. Random algorithms produce a number inside a certain range; the range can be a space coordinate, size, or a moment in time. Any property can be tweaked in random ways. The following example (randomness.pde) generates a random coordinate for the square each frame:
oat x; // coordinate in x axis oat y; // coordinate in x axis void setup() { size(100,100); ll(0); } void draw() { background(255); x = random(width); y = random(height); rect(x, y, 20, 20); }

44

 Precision The generation of images through algorithm sometimes requires a great deal of mathematical processing; certain values go through hundreds, thousands of calculations and the visual output depends on how exact those calculations are perfomed. The case study of the harmonograph (page 78), for example, shows musical intervals converted to images; the purity of a interval determines if the image will be symmetrical and stable or not. The visuality of the whole piece changes with any small deviation of the interval value, even a 0.0000001 unit. The concept of precision thus is valuable when obtaining certain visual pieces.

harmonograph: any change in the interval value creates a different visual representation

Predictability Predictability comes from nature; for example, its absolutely predictable that all physical objects on Earth will obey the laws of gravity. Physical, chemical and biological processes occur with condence that they were supposed to happen. In a computation system this concept can be recreated; a function can set behaviors to any module, and this behavior will happen, always, while the system is active. Creatively, it allows the designer to create his own rules: like that all objects should vibrate all the time. Or everything should become red when touched and so forth. Computer bugs are the outlaws of predictability.

45

 Discretion Being modular and being discrete are almost synonyms; events in computation dont need to be linear; events can happen in modules they can be handled through any kind of input. A computer code can have a set of functions, and these functions be called to act any time. A user interacting with a mouse is an example; when the mouse clicks an item, an event is called. The mouse was the event that determined the moment in time where the action took place; that action could happen again and again or not. Theres no need for linear time in logical modules. Complexity The concept of complexity is to use computation to achieve a degree of element profusion and element interaction that is intangible to human hands; based on the principles of repetition and transformation, complexity can become more an aesthetical statement, searching beauty in the intricate.

Peter de Jong algorithm shown with 2,000,000 iterations.

Simplicity Simplicity is the holy grail of computation, since the complexity is a natural tendency; bulk processing powers opens extensive creative options. A visual experience can be successful either from complexity or simplicity.

46

Malevichs Black Square and Red Square : simple and powerful.

Memory There are two kinds of memory in a computer system; the volatile, logical memory where the necessary data for realizing a code is stored, and the physical storage. The logical memory has a temporary nature; to perform operations, the computer stores data of any datatype in its functional memory banks. A computer can retrieve with precision any item from its electronic memory. The physical memory is permanent1; the role of time is very clear on the concepts of computation, and the physical memory is the place where time stands still. Like a printed book, binary data is printed onto an optical or magnetic surface, where it can be physically stored. The creative value of storage is in its power: a database can hold an unimaginable amount of information, in any kind of format. The internet itself is a gigantic database, that can be retrievable through search engines. The Human Genome Project and the CERN nuclear collision database are examples of huge unique data storage projects, with billions of bytes that can be retrieved, sorted and shaped in any way.

Subject to physical decay through time; under controlled storage conditions, the decay is minimal.

47

Ben Frys Haplotype Lines: a creative visualization of the massive genomic database.

The concepts of computation can be visualized, or verbalized, in countless forms; the key to achieve visual experiences through them is to employ artistic creativity in all imaginable or unimaginable - ways.

The issue of generative design

Generative design is a term vastly employed to describe visual pieces generated through computational processes; but its true concept is not clear. Between its diverse denitions, it can be used as design generated by a computer or design generated through a computer very different ideas. Can a computer design? Algorithms can generate visual compositions from ether; visual elements can be built and tweaked in any amount, following all the concepts of computation. The illusion that a computer is designing comes from the capacity of variation and randomness on the construction of new compositions. Sadly, random and abstract are not synonyms. Abstraction is a thought process. Computers dont think (at least yet).

48

The computer intelligence is a widely discussed topic; one of the most famous writings on it comes from Alan Turing, a mathematician that published in 1950 a test where machines would be considered intelligent if they could engage a non discernable conversation with a human being. The validity of the Turing test is questioned today, but no machine could achieve the victory so far. Despite the advancements on articial intelligence, it remains articial - a mimic, or a source of algorithmic variations. The conception of visual experiences is lead by original ideas; algorithms can be programmed to follow the principles of visual language proportion, distribution, color harmony; they also can be programmed to follow style guidelines use certain color harmonies, certain line styles, and achieve an overall aesthetic sense. But the leading idea remains in the hands of the designer.

The term generative design shouldnt be used to dene design created by the computer it should be used to describe through the computer instead. In a broad sense, any design that is visualized through a computer can be considered generative, but the usage of the term becomes more appropriate when describing automated processes - the computer is programmed to build automatically certain visuals according to a set of variables and pre dened laws with no need of human interference.

49

the language of the computer is the language of technology, not the language of design. Without a knowledge of design, the computer (like the pencil) is more than useless, for it is capable of producing enough superuous material to create the illusion, that one is inventing when in fact, one is merely producing variations of a theme, often of nothingness. Paul Rand, Design Form and Chaos, 1993

50

Praxis

The concepts of computational design need to be applied somehow; In order to transform abstract concepts into real projects, tools need to be used. The computer is obviously the most necessary tool for the task. It needs to feature a screen a device capable of displaying an array of pixels, light elements that are lit or not to build shapes and colors. It also needs to feature input devices, like keyboard, mouse or joystick. Finally, it needs the software, a logical layer between the computer circuit and the designer. The choice of software is a controversial issue: different packages offer completely different experiences for the designer and for the audience. Also, every software title imprints different aesthetical details to the pieces generated by them. A software title that helps the creation of visual experiences will act as an interpreter, a translator between the designer and the machine. Different interpreters will build the translated content in different ways; the overall meaning might be the same, but the lexical construction will differ. Also, software packages use libraries of functions pre written formulas that help the development of a visual piece. When the functionality of pre-written libraries grow, an effect called pasteurization might happen: little individual differences in the code between one author to another are suppressed; the code, thus the visual (since one generates the other) gain a very similar aesthetic aspect. They look the same. Commercial packages often break the barrier of canned aesthetic subtleties and offer completely industrialized, homogenous visual solutions - templates. Less, easier work but evident less visual value in design.

53

Forms of Code
Canned or artisan, the software tools provide a clean slate for creativity. There are many options to design through algorithm and these options have mainly three categories: compiled, scripted (aka interpreted) or markup languages. Their denition is a little fuzzy; the main idea is that a code written in a compiled language is converted to machine language compiled before it is distributed and executed. The nal product does not contain the original written code, but a translated machine code. For example, C and Pascal are purely compiled languages. C++ and Java are semicompiled, they stand in an intermediate structure between compiling and scripting modes. Scripting languages are interpreted at run time; the code is converted to machine instructions at the same time the program is running. The distributed/executed code contains the original language written by the programmer, and it needs the support of a compiled structure to receive. The performance of scripting languages is worse than purely compiled ones, but its exibility is greater; Postscript, JavaScript and Flashs ActionScript are popular examples of scripting languages. Markup language is the name for content that is tagged to be correctly displayed and interpreted. HTML and XML are markup languages texts, pictures or any data with the correct tags for the future interpretation.

Forms of Image
Another important distinction to be noted in graphic software is the denition of bitmap and vector images. Vector graphics are shapes dened in the computer memory by the formulas of its form, the formulas of how lines and curves are positioned and connected. They became very popular on the web thanks to Macromedia Flash, and very popular in print design thanks to Adobe Postscript. Vectors can be written as a map of coordinates in two or three dimensions, creating 3d models. The vector formulas need to be

54

converted to pixels before being shown on the screen; depending on the software that is interpreting them, different rendering results are noted, different visual styles are achieved. Pixels are square elements, and theyre required to represent points in curved shapes; The rendering engine needs to decide which pixels should t in the curved lines, and also decide how the jagged edges can be smoothed (also know as antialiasing) to complement the illusion of curviness

a vector image, and a blown-upof its pixel representation.

Raster (or bitmapped) graphics are visual les where an array of pixel color values is stored; all the pixel values together form an image. Photographs are examples of bitmapped images; bitmaps can be compressed through algorithms that perform pixel statistic operations, discarding information that it judges not relevant to the image and making them smaller. Bitmaps are already made out of pixel information, so unlike vectors they dont need to be converted to be displayed on screen.

Software
The most common software tools available today for the different code form categories are: Adobe Photoshop/Illustrator/After Effects the Adobe software titles are used as graphic production tools, not coding environments; but the reality is that algorithms can be written

55

and implemented on then. After Effects, the broadcast motion graphics standard has support to the common JavaScript. Illustrator can import postscript les that can be generated through code; Photoshop also can have plug-ins developed, automate functions through scripts or deal with postscript les. All these coding doors can be taken as advantage in the creative framework. Macromedia Director One of the oldest multimedia authoring platforms; Director is rst of all, an aggregator: it can unite almost any kind of media asset inside a project. Director has a powerful, but very extravagant code language called Lingo - extravagant because its syntax is very different from the coding standards used by other popular multimedia applications. Director has full support for real time, hardware accelerated 3D graphics and is largely used in complex networked projects, where ash is not powerful enough. Macromedia Flash Flash is one of the most successful pieces of software in history; the initial plug-in was designed to visualize vector graphics in a web browser, and animate them through tiny le sizes. The original l996 software called CelAnimator evolved, having more and more features added every year. Flash is a program that can be used in innumerous ways as a linear animator, as an illustrator or as a creator of applications and web sites. Flash has a scripting language called ActionScript, in part responsible by the actual movement of designers working through code. It is necessary to learn ActionScript in order to build interactive movies; the basic scripting of a site navigation is an easy task for non programmers which mean a smooth transition from traditional design to the possibilities of design through code. ActionScript can be very simple, but it is also capable of doing complex tasks as the user learns more coding techniques. There are limits in ash though; the package still does not offer control of the actual pixels being drawn on screen; the user can only code through vectors, and the closed ash engine process the conversion to screen pixels.

56

 Java Java was created in 1995; its initial concept was to be an extremely portable language that could be executed in any computational device from desktop computers to kitchen fridges. The Java-enabled consumer electronics were never fully employed, with the exception of cell phones, but the technology turned out to be very useful for network applications including the web. Java is a programming language modeled after the Object Oriented concept, and is considered a semi compiled language, designed to be easily ported to different computer systems. Processing Processing a programming language created aiming artistic and visual applications developed through code. Programs generated through Processing are usually called sketches, and can be run in any computer platform, or over the internet. Processing is built over Java, having added its own graphic library and functions. Java language has a steep learning curve, and requires lots of technical background. Processing does not have a graphical input interface; the sketch is designed in a textual way, but the logic within it is very visual, making sense for visual oriented users, as artists and designers. Processing grants pixel-level control; the screen array can be manipulated directly. Also, it has great extensibility through additional libraries like sound, midi, HTML or physical interfaces. Almost anything can be integrated into a artistic sketch project.

Every software package demands time from its user to fully learn all the concepts, functions and possibilities. The technology of the future is almost unpredictable, making hard to decide which platform will have more acceptance and more capabilities in few years. Macromedia Director, for example, was really popular for the CD-ROM production in the nineties; its usage has been decreasing, being now restricted to some specic 3D projects and its likely to become less popular in few years. Other factor to consider when learning a language is the commercial versus free software factor; commercial packages usually have, at least in theory,

57

more technical support, because theyre products being sold to customers; the designer is paying the developers. In fact, free software communities end up having a good network of users who support each other, even in a better way that paid application service. The real advantage on commercial titles is the marketing that convinces users to adopt means that will allow projects to be seen. Recently, Microsoft decided to cut the free Java as a pre-installed plug-in for the Explorer browser; meanwhile, the Macromedia Flash player still comes pre-installed, and is present in 95% of the user browsers. Using Java/Processing as a creative tool also features the advantage of controlling the visual rendering process to a low level depth. As said before, pre-programmed functions end up having an aesthetic inuence to the visual product; programs like Processing allow the designer to customize the aesthetic imprint by writing their own set of visual rendering functions.

still from Bzier Ballet, by Mario Klingemann (http://www.quasimondo.com). Unhappy with the Processing default antialiasing algorithm, Klingerman wrote his own, capable of displaying the smoothness and delicacy of thin bzier lines.

58

Mathematical Tools
Each software package has its own features and its own coding syntax to be learned. But one thing is common among them all: Math. It is necessary to know some math concepts in order to create visual experiences through algorithm; and this is one of the reasons that people are driven away from this kind of creative process. Art education has been far from Math, and mathematical ideas are seen as impossible problems by most of designers. Happily, the truth is that the necessary math for visual creations is very tangible, and all it requires is the high school math remembrance. 95% of the creative visual work produced through algorithm today employs high school level math only, including algebra, trigonometry, physics and geometry. The key is not to know complex math; the key is to use creatively the math you already know. Math is a tool, and the complexity of the tool never meant the quality of the creation neither in arts or sciences.

The Leica camera used by Cartier Bresson had only basic photographic functions. It is a quality simple instrument, the tool of Bressons beautiful photographic expression.

59

Remembering high school

What is the basic math a designer is supposed to know? A good general review at the high school books is recommended; sometimes, easy formulas or concepts that may be forgotten can help to transform an idea into a visual. There are some topics that deserve special attention; a good foundation on trigonometry, for example, will solve a great number of issues related to circular motion and visual oscillation. The Pythagorean Theorem, that solves the right triangle lengths (c2 = a2+b2) , will also solve questions regarding positioning, distance and linear motion along the space of a visual piece. Basic operators (addition, subtraction, division, multiplication) can also have creative potentials - since a block of code will loop indenitely, and a number can go through an operation and change its value on each cycle. The modulo (the operator that returns the remainder of the division between two numbers) is one of the forgotten high school arithmetic topics that can have a great use on computation modulo values are cyclic, they repeat themselves when the operand is constantly incremented in a loop. Colors, forms and position can be cycled along the modulo result. The author of this document knows very little math beyond the topics related above and even so, produced a series of intriguing mathematical visuals; the best advice to overcome the Math-phobia is to roll up the sleeves and work on visual projects learn the math that will solve each specic visual inquiry, one at a time. Following, a code example built in Processing that shows the application of modulo and trigonometry; a box rotates in circular motion achieved by calculating the x and y coordinates according to trigonometric rules; its color switches to black and white according to the modulo value of the loop cycle. The sketch can be found on the attached DVD under the name Trigonometry.pde.

Art and Mathematics are complementary different; that is to say: they differ each other just as much as they complement each other. Karl Gerstner

60

create a loop counter variable: create a circle radius variable: create the x and y coordinates variables: congure the piece: set the velocity of the animation: set the dimensions of the sketch: set the initial value of the counter: set the value of the circle radius: end of the conguration block: loop starting point: set background color: modulo operation: if the value is even, ll the square with black: if odd, ll it white: x axis value according to trigonometry circle: y axis value according to trigonometry circle: set the rectangle coordinates and size; increase the counter value: end of the loop block:

oat counter; int radius; oat x, y;

void setup() {

framerate(20); size(200,200); counter = 0; radius = 50;

void draw() { background(122); if(counter%2==0) { ll(0); }

else { ll(255); } x = sin(counter/10)*radius; y = cos(counter/10)*radius;

rect (90+ x, 90+ y, 20, 20); counter = counter+1;

The Code Language

Theres not a universal language for computer coding; each coding platform has its own syntax. Different languages can share similarities, but they have different ways to handle the code structure, making them unique - and requiring unique training. Someone whos aspiring to write code should choose which language ts best to his needs; for commercial internet creation, Flash is surely the best choice today. For artistic experimentation, Java and C++ are better options, since they offer less limitations and more performance than Flash. The Processing language has a lot of advantages for the artistic eld, including strong community support and a balanced learning curve. There are many other options Python, C#, Objective C all computer languages can be a source for creative work. Theres also no single method of learning the code syntax; each author proposes a different path through the functions and commands of each platform. A recommendation for the learning process of a visual-oriented user can be something like:

61

 learn to compose a basic code structure setup and loop blocks. create and congure the visual space for a piece (the stage) learn the methods of drawing basic shapes; learn the methods of tweaking the attributes color, size, rotation of those shapes; learn how to move things around. learn how to make the user interact with elements. This sequence would start out of graphic functions, instead, for example, the common foundation on Object Oriented Programming that involves more abstract and less visual thinking. The Processing platform has a lot of support for learning from the artistic point of view; the language reference includes simple graphic examples on the syntax of each topic at http://www.processing.org/learning and http://www.processing.org/reference.

62

The Praxis of Coding

The process of writing code is similar to the process of writing a narrative text. A narration tells a story; there are characters, each character has a role and things happen as the story develops, coming to an end. The same narrative story can be written in different styles, different languages; it can be written in a very erudite form, or it can have a teen literature lexicon. A computer code with a determined objective also can have different styles, and different structures. The style and structure of coding can be very personal; the way variables and objects are named, and how an algorithm is written or a problem is solved. But there are principles that should be followed. The basic principles of coding are simplicity, clarity, generality and automation. By denition, computation is a problem solving task. And there is usually more than one way to solve a problem; the principle of simplicity is that the simpler way to solve a problem is the best less computational resources are allocated, leaving more room for other operations. Clarity is an issue for both man and machine; code should be clear for other people rather than the author to read it, and for the machine to understand it. The clarity is achieved through good naming practices, using parentheses to clear ambiguities (sometimes its use is not mandatory, but helps the human reading), commenting the chunks of code, and describing any relevant information in the program body. Another item for clarity is the avoidance of contradictions; for example, the usage of lower and uppercase. A variable named Test and a variable named test are completely different in a case sensitive environment. If a piece of code is using a variable in lowercase, all references to it should also be in lowercase. Generality is a principle that follows the spirit of object oriented programming. A program can have very specic functions for each task;

63

sometimes, these functions are similar, and could have been written as one and reused. General functions allocate less memory in a computer system and allows better performance of the code. Automation is a principle where the computer should handle repetitive tasks, not the designer. It is an idea related to the principle of generality; Instead of writing a commands or functions that perform a similar task hundreds of times, it is better to write one function that operate the same way with different parameters, saving time on the development and making the debugging process easier. A good practice for someone who is not comfortable with the task of coding (or for someone with a very complex task in hands) is to rst write the program statements in plain English; list all the actions, variables and describe all possible functions. With a cohesive and well structured English text in hands, just translate it to the programming language syntax and the code is done.

64

Experience

Graphic design which evokes the symmetria of Vitruvius, the dynamic symmetry of Hambidge, the asymmetry of Mondrian; which is a good gestalt, generated by intuition or by computer, by invention or by a system of coordinates is not good design if it does not communicate Paul Rand A Visual Experience involves the audience; designing a visual experience does not contain only the what to do issue and the how to do issue; it also contains the whom to do, the concern on how a visual piece will be experienced by the audience. The experience is an individual process; each person responds to a communication stimulus in a particular form. The creation of visual experiences deal with probabilities; dening the most probable response to each presented stimulus, and planning how the visual piece will develop according to the most probable situations. Theres no real passive audience; if a visual piece is not interactive, theres always a choice for someone to see it or not; to ignore it or enjoy it. The act of ip a page or not, close a browser window or not is a basic level of interactivity. Interaction is any reaction to the design stimulus. A positive reaction is the successful communication. The designer should compose a piece in visual terms but also in interactive terms; how a piece will be originally seen? Is there a waiting time to see it? How much information is given at each step, each section, each second? Dosage: the concentration of information on every design node, part, section, screen considering information every visual, aural or any media output. The design of interaction deals with the dosage of how and when things will be experienced along a piece.

67

Regarding the visual terms, the shape of information should raise identity. Identity means that the each person in the audience will recognize aesthetical ideas in common between him and the presented experience. The recognition can be simple as a I like it response and it does not mean that the user identies the output as something already seen before, but as something in his personal taste, a match to his preferences. The aesthetic values should be shared between both ends of communication, the designer and the audience. The development of visual experiences through algorithm opens doors of limitless customization, interactivity and the generation of customized information channels: each piece can be designed and react according to the viewer. Despite of the unpredictability of a human response, the audience should be acknowledged during the creation process - the designer should not lock himself in an ivory tower, design is about public experience design is about communication.

68

69

70

Case Studies

The following projects were developed during the MFA program at MassArt; they relate, identify and explore issues on the task of articulating visual experiences, and they are the product of a designer whose education was centered in humanities dealing with computation as a tool for the visual domain.

71

72

Sound in Space

Music is geometry in time Arthur Honegger (1892-1955, Swiss composer)

Whats the relation between the aural and visual domains? Is there an image for sound? Sound is powerful enough to survive without our eyes. Music is a legitimate expressway from and to our souls; music strikes the heart with no need of visual alliances. At the same time, once we add a visual component to sound, a relationship is formed - visual and aural domains stick together, building an undeniable bond. The bond can be formed in two ways; the rst is by converting the character of the musical signal into a visual representation; Apple Itunes does this very well. Its visualizations uses the pitch and volume to feed graphical algorithms. The second way evokes the meaning of music along with the meanings of the visual domain; this idea is seen (and heard) in cinema. A dramatic scene has a dramatic soundtrack; dramatic is an adjective for the musical meaning; the pitches and volumes alone are not dramatic but the musical movement altogether can be. The relationship of visual and aural has been explored throughout history in many different ways; in eastern civilization, one of the oldest scientic approaches to it comes from Isaac Newton. In 1672, he studied the color spectrum and related 7 colors to 7 musical notes. Later, Louis Bertrand Castel built the rst visual keyboard the Clavecin Oculaire that displayed a different color to each played pitch. The idea of a keyboard that would display colors for each note reappeared in different places and moments in time; there was, though, never a rm proof that a certain

73

color is a certain pitch. The relationship always had an arbitrary element in its formation. Different color/pitch tables were developed, and many authors agree on the color distributions but there was never a scientic standard. Intuition and feeling, personal factors, remain present. Besides color, sound always had a strong relation to math and geometry. The notable study of this relationship is harmony. Harmony is the study of the simultaneity of sounds; Pythagoras, more than 2,500 years ago started to study why some sounds had a pleasant character when played together. Pythagoras studies had a strong mathematical sense, and he established the concept of intervals. Each musical note is a vibration; each vibration has a frequency. Our perception of pitches, or musical notes, has a cyclic nature every time a frequency doubles, we perceive it as the same pitch, on a higher tone. Its called an octave the space between two frequencies with the same note. Numerically, the octave interval is notated as 2:1 proportion the frequency of the interval note is 2 times bigger than the whole fundamental note. A fundamental note can be divided in any other proportion, and some of them are used as standards in music, like 3:2 (1.5 times bigger frequency than the fundamental - fth interval), 4:3 (fourth interval), 5:4 (major third), 6:5 (minor third), so on and so forth.

the proportional lenght between some musical intervals.

74

A mathematician called Jules Lissajous developed an experiment in 1857 that projected a vibration movement through a light dot in a screen. He managed to reect the light beam through 2 mirrors, each one vibrating in an axis, vertical and horizontal. The resulting projected shapes became known as Lissajous Figures

original Lissajous experiment

Different vibrations could be applied to the mirrors, producing different shapes; musical notes are vibrations, so different pitches could be reproduced and visualized - and when different pitches were projected together, musical intervals could be seen. The Lissajous gure might be the most scientic approach to a visual representation of sound; it is based on the physical properties of musical notes that cannot be easily contested like the previous color/sound experiments.

75

The Chordinator

Chordinator interface screen

The Chordinator re-creates and expands Lissajous experiment as an interactive computer application. Lissajous projection was limited to one single interval; two notes, because his physical projection system could only be bi-dimensional. The starting point of the Chordinator is to take advantage that a computer system can create the illusion of depth easily. There are 3 spatial dimensions that the human perception can discern: height, width and depth. Musical chords are 3 (or more) notes played together; two intervals in the same period of time, in a chord structure ( root, II degree, III degree, the button labels) The idea is to assign a spatial dimension to a sound frequency, and explore the plasticity of sound as a sculpture.

76

The sketch allows the independent control of the intensity of each chord component, as well as its phase. Phase can be understood as the coordination point between sound waves

phase example: three sound waves with same amplitude and wavelenght with different alignments in time.

To help the sense of 3 dimensional space, a cube was added marking the capping of the volume; rotation was added so the point of view can change over time, allowing a full visualization of sound in 3D. Despite of the strong sound concept, the Chordinator experience aims to sculpt a visual structure over time: the keyboard is controlling a form in space, not music. Production notes The Chordinator employs a very simple programming concept: parametric equations, where independent equations drive different coordinate axis. The musical note visual component is a graph plotted through time; each coordinate is being fed by an algorithm that determines the amplitude, pitch and phase of sound, independently. Theres an independent module that creates the cube and another that controls the interface. The algorithm follows trigonometric principles, where the sound vibration can be expressed as a sine wave, with dened amplitude, length and period. The 3D visualization was built in Processing, and the keyboard interface in Flash. they both work together in a browser window, allowing this visual experience to be shared online.

77

The Harmonographer

Harmonographer interface screen

The harmonograph was an invention built and sold in the nineteen century and it consisted in pendulums attached to a pen; the pendulum movement set the pen coordinates, drawing the resultant of two (or three) harmonic motions. The nal drawing is a frequency interval, like the Lissajous experiment; the pendulums also had the nice feature of simulate the damping of a musical note, the slow decay to silence, thanks to the mechanic energy that is dissipated within the motion. Another feature of the harmonograph was the use of rotary motion; it added a new dimension and perspective to the visualization. The frequencies were more dimensional, more representative of their physical nature in space han the Lissajous projection.

78

an original Harmonograph

There was a strong relation between musical harmony and visual harmony in the interval drawings of the Harmonograph; perfect intervals were symmetrical and visually stable; numerical approximations in the frequencies produced more unbalanced visuals, with a more intricate nature. The Harmonographer project is the translation of the physical device to the computer; it has the feature of dialing in the musical notes, obtaining the correct intervals. It also simulates the musical damping, and the phase oscillation. The images produced by the Harmonographer should be understood as drawn from the outer edges (maximum volume) to the center (silence) - the path of decay. The constant rotation of the drawing has a purpose: show the interval at all possible sound coordination points, and its circular motion comes from the phase value. The phase is a cyclic event (it repeats according to the length of a wave), so theres a constant rotation impression to the coordinates. The sketch also features acceleration from one interval to another and variable interval change speed; the function of both is to induce a smooth transition between different visuals, letting the user know what is happening from one note to another or just y through all possibilities.

79

A Portamento button is available to turn on the innite sliding between notes; in the portamento mode, the interval grows eternally, and the user can keep on watching and observing how the intervals grow and their visual elements change with the growth. The main objective of the Harmonographer is let the user explore the interval relations through their visual representation; or simply explore the visual patterns and their mandala-like formations, tweaking the possible graphical formations that the algorithm can build. A special version of the Harmonographer was also built, that allows the user to export the drawing as a vector le, that can be used in any graphic application or printed. The user can also save the drawings as bitmaps, and create video animations. Production Notes The coding behind the harmonographer is based on parametric equations, independent plot instructions for x and y values. The code is slightly more complicated than the Chordinator, since there are formulas for the damping - that use the harmonic damping formula from physics books. The damping was implemented as an algorithm, and the phase rotary motion was added as well - using again the trigonometric circle concept. The code has separate sets of acceleration and portamento functions, and all the interface commands function, along with the Flash virtualkeyboard element.

80

The Near Dorian Experience

Near Dorian Experience frame still

How can a musical piece be visualized through the harmonograph? To answer this question, a video called The Near Dorian Experience was developed, visualizing an original music composition. The musical soundtrack intentions for the Near Dorian were to work with few and strong musical intervals, and create a hypnotic trance of repetition - but at the same time calm and meditative. Theres one guitar with a mellow, woody tone and another with a twang slide sound. The percussion is nothing more than a triangle being hit twice in the music and the base sound is one single chord being played in a Hammond organ, oscillating through rotating speakers. The intervals explored in the song were pre selected through their visuals the interesting visuals had a granted place. The video has layers for the different instruments and their visuals; color was added, in the form of blue tones denoting the cold electric resonance, and yellow/orange denoting the warmness of the guitar strings. The white denotes the purity of the bell sound, the

81

moment of illumination. Of course, this description of the soundtrack is from a author standpoint, and the actual experience of musical meanings may vary. The piece shows the coordination between aural and visual intervals. The title, Dorian, is a musical mode of the natural musical scale, known by its simplicity. Since the song escapes the scale in a part, and the visual escapes the simplicity due to the overlaying, its a near Dorian experience. This project has the conceptual importance of being a completely authorial project in new media the author developed the expression in all domains: visual, temporal, musical, and logical. All the pieces were original and neither contained parts from other sources or projects. Beyond that, it has the proof of possibility to develop a concept in the all domains at the same time. A software application was created to achieve the visual of the video, and it couldnt be done in another way; even if a physical harmonograph was constructed, for example, it wouldnt animate the phase variations of rotating speakers - there are features that can only be achieved in a logical medium. The sound in space project was essentially pseudo-scientic; it had scientic fundaments, but embraced creative freedom to show a personal view of the spatial information of music. The personal view was expressed through a computer program that helped the development of a song and a video - the freedom of articulation came through algorithms.

82

83

Form Systems

A visual composition is formed by individual elements; these elements interact. The study of how and why the elements in a composition interact is the subject of visual language. Many artists and writers published their research on the nature and properties of visual language, and one of these authors is Karl Gerstner, Swiss painter and graphic designer. Gerstner has a remarkable point of view regarding form and color according to him, theyre body and spirit. Gerstner considered math an aid to ultimate master of ones medium. His most notable characteristic is the usage of structure as a mean of expression; many of Gerstner art pieces were not xed compositions, they were xed formulas where the components could be interchanged. Although Gerstner didnt employed computers, he created artistic algorithms; his pictures are variation machines. This computational spirit is the inspiration for the series of form systems experiments in which the nature of variation is explored. Gerstner work is the inspiration for a series of experiments, where concepts of the computational medium that connect with the idea of form as a system are explored.

The Monkey+Gerstner and No_More_Mondrian

A good example of Gerstners art is the Carro 64 piece. It consists in 64 aluminum blocks, in 16 colors (a 16 step color gradient) that form a do-it- yourself composition: the blocks can be arranged in any way. The arrangements can follow symmetrical operations or be random; there are trillions of possible combinations in the random mode, but the number of options is smaller when compositing laws are inserted, forming deter-

85

Carro 64 advertisement

mined color patterns. As an experiment, I rebuilt Carro 64 as a java sketch where the compositing possibilities would be alternated but never repeated. When translated to the electronic medium, the blocks location exchange is effortless. The canvas becomes an almost innite art generator, rebuilding the color blocks in a different conguration on each determined time interval. Are the color compositions interesting? Eventually. Being random, theres a random possibility that an interesting or visually attractive composition

86

the original Carro 64, and one of its random computer recreations

will emerge. An interesting experiment was built on the web by the software company AardAsNails, called Monkey Shakespeare Simulator1. Launched in 2003, it proposes to prove the mathematical innite monkey theorem, by the use of bulk grid computing and random letter string generation a monkey simulator, that would run on each user computer. If you have enough monkeys banging randomly on typewriters, they will eventually type the works of William Shakespeare. The record so far is 24 letters from King Henry II and it took only 2,737,850 million billion billion billion monkey-years to be completed. Eventually is a tricky word that tends to the impossible. Maybe thats why Gerstner, even though creating an innite art form, exposed congurations with carefully planned symmetric harmonies that required plenty of artistic sensibility to be achieved. The point of this experiment is to explore the role of authorship in the so called generative design; the absence of an authors guidance creates a monkey situation for the machine. Theres value on the concept of an eternal change art canvas, but the isolated result of the variation is not guaranteed. In a form system, the author can create rules, directions and behaviors that will channel the creation interesting levels, instead of abandoning the visual dimension to its own random generative luck.
1 http://user.tninet.se/~ecf599g/aardasnails/java/Monkey/webpages/index.html#results An experiment with real monkeys was also conducted with hilarious results at http://www.vivaria.net/experiments/notes/publication/NOTES_EN.pdf

87

generative design processor

The other side of a generative form system occurs when a complete, vast set of rules is laid to control the unpredictable randomness. The uncertain composition is tamed, but the absence of the monkey factor can also create banal pieces that always look the same. The predictability usually occurs on systems designed to mimic an abstract process. During the early days of graphic computing, Michael Noll developed experiments where a computer was programmed to mimic Mondrians Composition with lines. Nolls conclusion? Viewers of his experiment could not tell whether the painting was created by the human Mondrian or the electronic Mondrian. Recreating Nolls experiment in Java has brought me some interesting conclusions; rst, its complex to mimic a human artistic behavior. Second, its really easy to create a sketch that draws geometric forms in random patterns, similar to many abstract paintings but no abstract painting is like that. Even simple geometric paintings deal with a complex mental process of decision making: should a line be on the left or right? Should a square be big, small or medium? What will be the horizontal to vertical lines proportion? To answer these questions, rules need to be written in the code. Rules that determine average size, average proportion, placement limits, so on, so forth. Even a minimalist painting mimic needs a great amount of rules; the more rules, the closer the generated image will be to the original but it will also be more predictable. The image wont bring new, interesting formations that could exist outside the boundaries of a certain rule. Any painting can have an innite set of rules they can grow in complexity, more the rules, less the randomness, more faithful to the original

88

artwork. For the No_More_Mondrian sketch only a limited set of rules were written, a number enough to resemble the original experiment, and the sketch runs forever always trying to add a new layer of ruled shapes. Dull, as expected.

clash of generations: the original Noll experiment and the actual Java recreation.

The Squircle
The key to creating generating form systems is striking a balance between rigid rules and anarchic randomness. The Squircle is a small experiment toward nding this balance. Based on Gerstners Color-Form Continuum, in which he explored transitory sequential states between different shapes and colors, the Squircle explores all the shapes that lie between a circle and a square and the visual compositions that derive from them. The Squircle was built using the formula of Bzier curves - a common tool for Postscript, CAD and vector illustration applications. Bzier curve is a parametric cubic equation that plots a path with initial and nal points inuenced by control points along the way. The control points can be tweaked to change the overall shape of the curve. By coding a closed shape with symmetrical control point changes, the result is a regular polygon whose sides bend and morph in all possible positions - the

89

screen captures from The Squircle and Gerstners Color Form structure model

90

square and the circle are the two extremes of the possible forms. Instead of copying a Gerstner piece or simply creating all possible shapes in a random fashion, the Squircle uses an oscillatory pulse that changes the control points position, and the amplitude of the change can be set by the mouse movement. The sole purpose of the piece is for the user to enjoy creating forms with symmetrical aspect and experiencing the possibilities between two basic geometric shapes. Interactivity is a useful tool when theres a quest to create an endless form system that is not random; user interactivity determines a composition in the Squircle, that is a unique and endless, but not random.

The Anaconda Paint

In another experiment, Anaconda Paint, the user builds a volumed form according to mouse coordinates. The visual composition is obtained by varying the color and position of a circle, with either a dented or plain stroke texture. The illusion of a dimensional painting is created in a at surface by overlapping circles through time. To enhance the organic aspect of the composition, the shape moves with acceleration toward the cursor, avoiding abrupt movements.

It looks better in motion.

91

The trick that makes the Anaconda Paint interesting is the color shifting; all the colors in the screen are constantly changing, in a cyclic way. Each circle is assigned with a different initial color and when all the screen colors cycle, the illusion of volume is reinforced. The Anaconda visual experience is supposed to be fun to play with, letting the user create endless snake gures. This piece is completely different from the Monkey+Gerstner, but both have the same concept: endless compositions, based on a xed shape, with multiple instances and varied color and location. There are innumerous ways to translate a single concept.

The Outline Anaconda

Roto-Champ
The idea of keeping a shape xed, but to vary its color, size, and position over time is full of possibilities; the Roto-Champ experiment explored another point of view on this theme. The project is based on Marcel Duchamp Roto-reliefs, a series of gramophone discs painted with patterns that would rotate and create unexpected kinetic results. The Roto-Champ is an exploration of Duchamps concept translated into the computational medium. The shapes have xed form and color, and rotate at a constant speed. Illusions of volume and form distortion are created by the visual interaction of the concentric rings, just like the original piece.

92

Marcel Duchamp invites you for the Roto-relief experience

The experiment has two different approaches; the rst regards the geometric nature of the circle. The circle is a pure mathematical form, and its presence in a plain color background has a strong abstract and ethereal sense. I questioned how to add more physical, concrete value to this form - and the answer was to tweak the relative size and rotation of the rings. The values inserted were transformed according to a value very common to Botanic: golden section . The sacred mathematical proportion is found on the growth of plants, on the patterns of seeds and on nearly every natural organic form. Both size and rotation would respond to organic values, linking the ethereal form to a more physical dynamic.

Roto-Champ: motion blur version

the proportion resulting from the division of a straight line into two parts so that the ratio of the whole to the larger part is the same as the ratio of the larger part to the smaller Oxford English Dictionary

93

Roto-Champ: golden proportion and standard versions in perspective.

The second approach deals with interactivity; with one mouse coordinate, the user can change the perspective of the piece something that would happen in the original Duchamp piece, since the perspective would change according to the position of the viewer. By visually cropping parts of the concentric gure, the illusion is changed and different visual impressions occur. The other mouse coordinate was used to displace the center of each ring in relation of its inner sibling, generating then different frequency patterns, and different aesthetic experiences.

94

My Friend Jong
Form systems can be taken to extremes; as in the case of strange attractor visuals. This kind of visual has been explored by a number of designers and scientists, and it is known by algorithms that plot points with an apparent random, chaotic, order but that gravitate around certain xed values (hence the name attractor), generating interesting visuals. My own recreation of a strange attractor used the Peter de Jong algorithm, written in 1991 by Paul Bourke. It was chosen because of its simplicity - a small mathematical abstraction that can lead to complex, visually rich results. The Jong algorithm simply subtracts a cosine from a sine value, in both screen axis (x and y). xn+1 = sin(a*yn) - cos(b*xn) yn+1 = sin(c*xn) - cos(d*yn)
Peter de Jong - algorithm by Paul Bourke.

The trick is: given a starting point, the sine and cosine values are calculated based on the previous operation result, in a loop. This loop is called iteration, and the number of iterations can be predened ranging from 1 pair of coordinates to billions. Organic patterns emerge from the plotted dots the cold mathematical abstraction is transformed in a shape that looks and behaves like organic dynamic forces, as the smoke of an incense stick. Given a smaller (few thousand) iteration cycles, the attractor can be animated in real time, or respond to user interaction, like mouse movement. Attractors are systems with little visual control (they have an unpredictable nature) but are a good playground where small changes can be multiplied and re applied millions of times. The concept of iteration can be used as a creative tool theres not much control over the form itself, but on how the form will emerge and unfold through iteration. This is an important abstraction on the realm of generating images through code: sometimes, a visual cant be grasped; it has to be tamed through the numbers that feed its creation.

95

My friend Jong: 150,000,000,000 iterations.

(I counted them myself to double check if there was any missing.)

96

Riley Patterns

Op art is an artistic movement that blossomed in the middle sixties, through the work of artists like Victor Vasarely and Bridget Riley. The main characteristic of op art is a sensorial overload; the perception is challenged with tricky impressions of volume, color and motion. Op art is extremely kinetic - but in a perceptual, and not physical level. Another trademark of op art is the creative use of patterns; visual rhythms with distortions and modulations that conduct the composition experience. The use of patterns and modulations usually follow a dened set of rules a strong sense of algorithm. Riley and Vasarely works were not computational their creations were set in a time when computers where scientic secrets. My intention is to investigate how optical visual patterns can be created and modulated through code, and explore the computers advantage as a variation machine: visual ideas can be developed in endless ways, especially when the idea has an algorithmic nature. What are the changes in op art when it is reclaimed by the computational medium?

Baba O Riley
The art of tomorrow will be a common collective treasure or it will not be art at all. -Victor Vasarely, 1960 Vasarely professed the mass media possibilities at an early date. In which ways can art be collective? Baba O Riley is an attempt to answer this question, following Gerstners spirit of having structure as art. Baba O Riley contains no composition: it contains the structure for endless compositions. The authorship is shared:

99

the designer authors the aesthetic framework; the audience experiences the creation of a composition from that framework. Its a true form of collective art everyone can do it.

Baba oRiley interface screen

The user has the ability of drawing lines and circles, with different color frequencies - always alternating black and white. Even being just a computer application experience, the decision process on how and where the drawing elements should be placed is similar to the physical painting creative process. The medium is different, but the abstract thought process is alike The exibility on instantaneous composition is interesting for the artist/ designer; it allows painless experimentation. The successful tryouts can be saved, the unsuccessful discarded and the saved ones can even become sketches for physical paintings. One of the major difculties of creating physical real world op art creations is the technical production of visual patterns; they require a lot of effort and planning. A computer application can help a lot the process of planning, easing the technical barriers and amplifying the creative freedom.

100

Development Baba O Riley was developed from scratch, programmed into the Processing environment. It involved an interesting aspect of computer graphics: drawing a circle. Processing was chosen because it permits great control of the screen; pixels can be drawn and manipulated directly, feature absent in programs like Macromedia Flash. Designers take the computational circle drawing for granted; in Photoshop, Illustrator, Flash or even on the simple Windows Paint, theres a tool that allows the cursor to draw the basic gure. But how does the tool achieve that work? That was the starting point of the investigation. Pixels are square; the screen is a big group of squares. How can such a round gure exist in this harsh environment? The rst answer: it cant. A completely perfect abstract curve cannot be drawn to the square based screen. Its necessary to build a function that returns the coordinates of the pixels that better match the path of a curve. Curve lines on a screen are approximations; thanks to the resolution, they look acceptably good. The principles of trigonometry grant the knowledge of discovering such coordinates; once the radius of the circle is known, the Pythagorean Theorem can be applied to every pixel in the x coordinate it will return a matching y, creating coordinate pairs the rst step for a circle algorithm.

square pixels failing to follow a curved path

Unfortunately, the resulting curve line is not continuous; the frequency of y coordinate steps wears thin toward the edges the pixels imprecision appears. The second step of the algorithm is to ll those gaps with

101

straight lines. The result is a perfect circle built through inconstant coordinates and straight lines, in a computational illusion. This is the trick that Photoshop does all the time for its users. The third aspect of the algorithm is to build the curve with the right color. It rst evaluates the present color in the x/y coordinate, and redraws it with the opposite value the law of high contrast is followed all the time. The remainer of the code deals with mouse/keyboard response and interface parameters. The program is very simple, and although there are much better techniques of implementing the idea, it uses the simpler ones in an extensively commented code. Only one basic mathematical concept was employed proong that just a little math is already very useful to express ideas through code. The Baba ORiley project was a success in terms of audience the public enjoyed the online experience (2,000 unique visitors so far) and it shows how an interactive art project can be technically simple, but visually interesting at the same time.

Current; Flow
One of the rst computer graphic images in history, along with Mondrian experiments, was a recreation of Rileys Current piece, built by A. Michael Noll, called Ninety Parallel Sinusoids with Linearly Increasing Period2. I decided to explore the Nolls experiment, due to its historical value and also due to its visual value; Current one of the most interesting op art paintings. Current consists in a repetition (for Riley, repetition is a mean for amplication) of waves with increasing periods (the length of each vibration), and constant amplitude (the height of the wave). Its a very well mathematically dened piece, and maybe thats the reason with Noll chose it for his early computer experiments. The objective of the rst experiment was to explore the multiplicity of means to an end; Nolls experiment was rebuilt through different com2

Nolls version was actually a recreation of Plate 3, another painting from the Current series.

102

original Noll experiment

line polygons

Catmull-Rom curves

Bzier curves Current: many means to one form.

pixels

103

putational techniques different processes for a single end. The visual sketches were built using line polygons, Catmull-Rom splines, Bzier Curves, and nally, direct pixel manipulation. Also, the mouse coordinates were mapped to tweak amplitude and wavelength settings, helping to mark visually appealing values. Each technique has strong and weak features. The rst sketch used line polygons, which are fast and easy to manipulate, but are not real curves theyre small line segments that resemble a curve. Working with straight lines requires less processing power, but the fake nature of the curves might become visible. The second implementation used a curve called Catmull-Rom; it is a curve formula, where the path goes through the control points. Also, a control point in the curve line affects only the segment where it belongs, and not the whole curve. Catmull-Rom curves also have easy implementation, and are fast. Closed sinusoidal shapes designed with Catmull-Rom are difcult to be properly lled, at least using Java ll algorithms; the ll functions have a hard time determining which portions of the sine wave should be convex or concave to apply the correct lling. Catmull-Rom is an appropriate solution if the target code is not supposed to become a print piece. This kind of curve is not native to PostScript, as the Bzier curves are. For a truly cross-media piece, the solution would be to use the famous Bzier lines. Its harder to achieve the perfect variable period sine wave using Bzier; while the Catmull implementation was effortless, this required a lot of planning and calculation on the control points positioning. It also required many more control points to achieve the right smooth curve visual; on the other hand, the sketch could be the size of the screen or the size of an ocean liner, since the vector is size and resolution independent Finally, the pixel implementation: the best way to avoid the difcult lling of sinusoidal shapes is to draw the pixels directly. Instead of drawing a sine shape and lling it, I can set the screen pixels with different colors according to the form of a sine wave. The pixel version is not suitable for printing it is limited to the 72 dpi screen dimension but it works ne on screen and despite not being optimized for speed, it runs the lled sine transformations much faster than the vector versions. Another difference

104

is that while vector drawings can have anti-aliasing functions to work on top of them, pixel manipulation cant the antialiasing function has to be written specically to the sketch. The Current process exploration helps to demonstrate not only that a visual idea can be unfolded in different ways, but also that the chosen technique inuences the nal aesthetic of the piece: the speed of motion, the antialiasing algorithm, the curve rendering are unique in each version. The concept is drawn on screen in a similar way, but the style of the drawing is inherited from the coding techniques. In a computational visual experience, the process matters. The second part of the experiment dealt with the translation of a concept from a gallery wall to the screen. Riley had very interesting ideas about her painting she says that her paintings arent about optical illusion, but about optical reality. Riley explored the relationships between nature and vision, and how the dynamic forms of nature could be represented in their essence. Current has a very organic motion feeling into it, and I wondered how to translate this feeling taking advantage of the computational medium. The Flow sketch was the result of this exploration. It has two versions; an interactive, where a user input whether the mouse or any physical device triggers different wave settings and an automatic version, where the waves oscillate by themselves. The sketch intends to capture the transition from a geometric state the straight lines to an organic behavior resembling water currents. Dots were used instead of lines for more delicacy, and also to imprint the idea that a current too strong becomes noise; the dots mix themselves and the line pattern disappears. Flow is a very simple sketch; it has few lines of code and uses line vertex no curves and is even suitable for printing in any size. A simple concept, simple implementation, interesting results. Flow was also converted to an openGL version; openGL is a graphics library that allows geometry to be calculated by the computer graphics card, with increased performance in relation to the computer main processor screen rendering. The Vanishing Flow mixes the renaissance perspective with the sixties op art, creating a different aesthetic experience from the at version.

105

Flow: dot and perspective versions.

The op art explorations are product of my personal pursuit on exploring and expanding this traditional art form to new possibilities; visual patterns are great form systems that t naturally into the creative concepts of computation, and my role was to make the natural inclination become true.

106

Logic Molecules

Database: a collection of information, stored in a retrievable system. The database concept is the foundation of the internet today; the web is formed by uncountable servers that store all possible categories of data, constantly retrieved by browsers worldwide. What does a database look like? A database has no face; physically, it is stored in a binary mode, like any digital information - theres no difference between a music CD and a complex relational database source in its physical register. Logically, it has no face either. The interface that manages the relation between user and database has a face but it is a middle man, an agent, and it can be simple as a blinking command prompt. The database is intangible. The nearest a human can get to it is manipulating a query program, that will dive into the structured information and return with a completed request. Amazon, Internet Movie Database, eBay, Craigslist, Google: the major websites serving content and services are a shell for database querying engines all their power lie on the amount of information they contain, and how the information can be retrieved. How big is Amazon? How big is eBay? Where does it begin, and where does it end? How can a database system be understood in a graspable form? The Particle Database project investigated this issue. The intent was to connect the idea of database to a visual materiality, and raise the identity between databases and the physical world. The database is a mass with different categories of information in different amounts. It has a similar structure to physical matter: a mass built by different molecules, from different categories in also different amounts. When a database is queried, the relevant bits of information are retrieved by a query agent; information is added and removed of the system as well.

109

screenshots from the Particle Database querying system.

110

Building a parallel to physical matter, a database would be a exible, uid material where the molecules could react and be attracted by different agents. Another feature of physical matter is density: the amount of molecules per spatial area. A database representation would have the screen as its total spatial area, and the density would be a key idea to represent the amount of information populated in a system. A potential viewer of the physical database would be able to directly manipulate it, molding the visual data according to particular queries. Two concepts were developed for this task; rst, is the compliance to the mechanical laws of physics. Data molecules would need to be inuenced by gravity, and their mass generate gravitational elds. Acceleration and inertia would need to be present by reecting the real world, a sense of comfort is generated in the user experience, the comfort of being acquainted to physical rules like the known real world. Second, the idea of chemical agents; database queries would be formulated and assume a physical form; like a magnet or a chemical compound that would react attracting certain types of data and repelling others. Information could be attracted and repelled manually, allowing a sense of scale between the amount of attracted molecules and the whole molecule universe. The molecule/universe relationship has a clear metaphor in the computational medium: object/system. A molecule, with its components and properties, is similar to the logical object - an element with a series of functions and properties bound together. The concept of universe, the multi dimensional space where molecules exist, is the same concept of a computer system, the space where logical objects exist. Objects are abstract data concepts; they can be modeled to contain any visual presence on screen, and respond to any physics algorithm. Besides an arbitrary visual representation, an object can contain actual entries from a database. There are scientic applications where a large number of objects with properties need to be displayed and manipulated on screen; wind and aerodynamics simulations, atomic reaction research, astronomical models and many others. The name particle system is given to the style

111

of representing a large number of individual objects that have dened properties, and that interact according to common rules. The Particle Database project followed the visual style of a particle system, having each node of information represented as a glowing pixel. The pixels would stand grouped in a screen sized pixel sphere the universe, with variable density. The attraction/reaction agents would make the particles ow around; other tools would allow certain particles to be hidden, labeled or reveal their data content on screen. The project was built as animated demo, non functional, where an example query is done to envision the advantages and exibility of direct manipulation of data.

Coded Particles
Particle systems are the most explicit example of object oriented programming; a logical object does not need to be visual to exist but in a particle eld, each pixel is an object instance. The task of coding a particle system consists in coding the behavior of a particle kind. This kind is called class, the group of rules, characteristics and behaviors of an object. The class describes all topics in the life of a particle its appearance, motion and dynamic responses for different stimulus caused by the user or other particles. The Database Field sketch is a study of how the Particle Database would be coded; for experimental purposes, attracting and repelling behaviors were written, and the particles behave according to a faux Brownian motion random in space. The sketch adds the idea of the real interactivity, absent on the application demo.

interactive particle attraction/repulsion tests

112

Patterns of Time

Edwin Abbott, English writer and clergyman, published in 1884 a short book called Flatland. The book is an interesting and very early exercise into the perception of space; the human being lives in 3+1 dimensions 3 spatial planes and a punctum of time. The characters of Flatland live in a 2+1 universe; thus the atness of the title, the completely eliminated perception of volume. It is an interesting exercise to alter the relation between spatial and temporal dimensions; the Patterns of Time project explored an inversion on the perception of time and space; instead of 3+1 dimensions, 1+3. How can we expand time in more dimensions, while compressing space to a punctum? What is the spatial component of time? The computational medium is a good support for this kind of experiment; it allows great exibility for experimentation, and helps the imagination in such a entangled topic. The project proposal is an interface that allows the compression/expansion of dimensions to occur; it does not intent to be scientic, but poetic. It is easy to imagine all the special dimensions compressed to a point; timelines do that. A timeline transforms all the physical events to a printed point; time becomes a line, dot after dot representing day after day, year after year. But is time a line? The characters of Abbots Flatland had the rm belief that a third dimension didnt exist at all things were at by nature. The visualization of time as a line of events might be stuck in a Flatland mentality. Pattern is a concept of many denitions, but they all agree in the idea that a form may exist and be recurrent according to a rule a conguration. Patterns are everywhere in nature the conguration of clouds, waves, the growth of plants, structure of minerals. Common laws sew different realms together, binding them by mathematical concepts like golden ration and prime numbers.
opposite page: Patterns of Time Studio openGL version
115

The idea of pattern is usually employed to describe spatial or logical events; the project idea was to extend this application to time as well escaping temporal Flatland. What are the components of time? According to a personal point of view, corroborated by the quantum physics and approved by poetic license, time is light. Light is a periodic wave vibration, thus time probably shares the same properties of periodic waves - known as amplitude, wavelength and phase. What is the amplitude, wavelength and phase of time?

Phase is the point of the amplitude at which a wave begin. Applied to time, phase should be neutral time starts at the beginning of time (a recursive tautology). Amplitude and wavelength are related to the notion of facts. What marks the passage of time? Facts1. The happening of facts in the physical world is a beacon for proong the time ow. Each fact in time is unique2; but facts share similar congurations. Facts form patterns; theres a summer in every year its a predictable event. Each summer is unique but events that characterize the fact summer happen every year. In different scales, all facts have patterns of occurrence some of them are obvious, other subtle. The facts in time are points in the amplitude wave. Each fact follows a pattern of occurrence, so each fact has its own frequency, wavelength.

1 2

The world is the totality of facts Wittgenstein, in Tractatus Upon those who step into the same rivers different and ever different waters ow down. - Heraclites

116

How to represent the concept of patterns in time visually? The developed model of time used the line as a starting metaphor, added with the pattern idea. The facts have a frequency of occurrence, so they can have a cyclic nature. The end of each cycle does not mark the return to the beginning; the line of time continues to the future. So the cycle is not a closed circle it is a line with a coil appearance. Different patterns have different frequencies; bigger patterns encapsulate smaller ones like an year encapsulate 365 days. So the coil is not absolute it is formed by another coil; which is formed by another and other in an innite progression. The computer is a helper to achieve such concept in a visual way; the initial project was done after research on coil algorithms, and how could they be applied. A recursive coil model was developed in 3D, and a Flash demonstration of the concepts, was built around it. The Location of Time is a demonstration of the concepts of time and space, and it was projected to be the initial point of a functional interface.

Patterns of Time Studio: interactive online version

117

Patterns of Time: concept demo screens

118

Patterns of Time Studio: openGL version

The interactive version of the Location of Time was partially coded at the cost of much sweat since the concept is very challenging mathematically. Innite levels of recursiveness in a three dimensional structure is not exactly the everyday problem solving task of a designer. The rst levels of time recursion can be explored through different points of view, and different time line compression. Technically, the code describes a big three dimensional polygon with variable sides, and variable vertex frequency. The user can tweak and experiment on every possible parameter of the timeline and the code is suitable for screen or print, in any given size.

119

120

Findings

As a result of creative process, the product of design has its structure formed by concepts, form and medium. Concepts are ideas, consolidation of the human imagination. The concepts become real through form (the aesthetic faculties). And form needs the support of a medium, wherein it can be expressed.
concepts
form

medium

Noteworthy design work contains a sense of equilibrium in those components. Such work can usually be characterized as based on strong conceptual thinking, executed by a pleasing aesthetical form within full employment of the medium.

concepts
form
medium
time

equilibrium of the creative components

We in live in a moment in time marked by the proliferation of the personal computer and the fast growth of the internet - a computer network as a medium. The computer, as a vast new territory for design, is offering an interactive and dynamic space for creation.

121

At this moment, the focus on computer as media seems to have outrun the concepts in design creativity.

concepts
form

medium

time

media seems to have outrun the concepts in design creativity

The focus on media innovation may have eclipsed the role of concepts and form in creation. Instead of being based on ideas, design solutions became based on the medium; if something can move, move it; if something can blink, let it blink. There are innumerous examples of senseless employment of computational strengths that conrm Henry D. Thoreau Arcadian quote Our inventions are wont to be pretty toys, [] they are but improved means to an unimproved end. A hundred years ago, a similar event occurred: the advent of cinema. A new medium was introduced; a new technology was available for expression. But there were neither an aesthetic nor language of cinema; a new vocabulary needed to be built, where time was a plastic element that could be manipulated through montage. The cinematic medium component leaped ahead of available creative forms and concepts, and only after years of worldwide experimentation a proper language of the motion pictures emerged. The concepts of computation should be taken and used to build a new vocabulary of graphic design, where the computing logical plane shall be seamlessly welded to the visual, aural, temporal and haptical dimensions. The incorporation of computation is the key for conceptual thinking to reclaim its leadership in design creativity, bringing reasoning and meaning to interactive experiences.

122

The scientic, mathematical approach of computation and the whimsical nature of art are not truly opposites, and have already met numerous times through history. We are facing a new instance of this integration, where algorithms are agents of translation from concept to form, from form to medium. Designers confront the challenge of learning the praxis of computation; creative freedom in new media can only be achieved by knowing how to deal with the transformation machine. This knowledge does not imply that designers should become computer scientists, but that designers should become wiser. The visual expression through code is not and should not be hype, a trendy toy of ephemeral design. Neither should it be an obligation, a mandatory solution to all communication questions. The visual expression through code should be considered as a current stage of the evolution in design which is a process where different media can coexist. The articulation of visual experiences through algorithm is the comprehension of new media design as an integrated whole: when conceptual and technical barriers are tore down, creativity can be expressed simultaneously in all dimensions. Creativity is borderless.

123

Bibliography

Arnheim, Rudolph. Art And Visual Perception. University of California Press, 1974. Ashton, Anthony. Harmonograph. Walker, 2003. Barthes, Roland. Camera Lucida. Hill and Wang, 1982. Bense, Max. Estetica. Perspectiva, 1975. CAVS-MIT. Gyorgy Kepes The MIT years. MIT Press, 1978. Chediak, Almir. Harmonia & Improvisao. Lumiar, 1986. Dondis, Donis. A primer of visual literacy. MIT Press, 1973. Eisenstein, Sergei. The Film Form. Harcourt, 1969. Eisenstein, Sergei. The Film Sense. Harcourt, 1969. Gerstner, Karl. The Spirit of Colors. MIT Press, 1981. Gerstner, Karl. The Forms of Color. MIT Press, 1986. Kandinsky, Wassily. Concerning the Spiritual in Art. Dover, 1977. Kandinsky, Wassily. Point and Line to Plane. Dover, 1979. Kepes, Gyorgy. Language of Vision. Paul Theobald, 1944. Kepes, Gyorgy et al. Language + Vision. George Braziller, 1966 Kudielka, Robert. Bridget Riley. British Council, 1978. Maeda, John. Maeda and Media. Thames & Hudson, 2000. Munari, Bruno. Design e Comunicao Visual. Martins Fontes, 1968. Nachmanovitch, Stephen. Free Play. Tarcher, 1991. Toben, Bob. Space Time and Beyond. Bantam, 1983. Wardrip-Fruin, Noah and Nick Norton. The New Media Reader. MIT Press, 2003. Wurman, Richard Saul. Information Anxiety I. Bantam, 1990.

125

127

The following is my rst interactive object, developed at age of 5. The procedure is simple: 1) turn off all the lights 2) close your eyes and position a camera ash right in front of them 3) spin your boy, a lot 4) when its getting almost impossible to spin, re the ash as many times as possible. I can describe the effects: colored circles, patterns, oating around. Theres a scientic explanation, but it doesnt really matter.