Musical Time in Visual Space

Brian Evans
Department of Art, University of Alabama
brian.evans@ua.edu

The laws of harmony are the same for painting and music.
Maurice Ravel

Abstract
There is a renewed interest in composing visual music. A
time-based, visual art, using abstract material, can look to
fundamentals from traditional Western music practice as a
place to start. These fundamentals map into the visual art
foundations of design and color. Visual equivalents of the
musical ideas of consonance, dissonance, tension, and
release, can be easily understood and applied to the
composition of abstract animation. With the current state of
digital technology the medium of time-based sonic/visual art
is now available to any composer interested in expanding
their artistic expression into the visual domain.

Introduction

Music composers create coherent temporal structures

using the materials of soundpitch, timbre, rhythm, etc.
They organize sonic events. The hope is that the unfolding
of these events has a quality that would be described as
musical. There is no broad agreement on what musical
means, but there is a wide array of sonic art that most call
music and so consider musical, from plainsong to
Mississippi blues, North Indian ragas to the extended drones
of the Aboriginal digeridoo.
Musical motion (at least in most Western music) is
created through tension and resolution of tension through
controlled dissonance resolving to consonance. This tension
and release is expressed through harmonic pitch
relationships and rhythmic patterns that together develop as
chords, motives, phrases and cadences. These are the basis
of a musical syntax and provide a framework upon which
expressive, musical ideas are articulated.
Like music, the fundamental dimension of abstract
animation is time. Many animators of non-representational
images seek a visual unfolding of events that they describe
as musical. Examples are many, from the work of Viking
Eggling (1923) and his visual counterpoint to the digital
harmony of John Whitney (1980), to recent time-based
visual works by formally trained music composers. (Evans,
2003, Miller 2002) Hence many experimental animators call
what they do visual music. Can a truly musical expression

be made using non-narrative visual space? Are there visual

counterparts for the traditional building blocks of musical
time?
Focusing on motion and the materials of the traditional
art foundations, design, pictorial composition, and color
harmony, a time-based grammar of visual music can be
developed. The construction of this grammar starts by
defining visual consonance and dissonance. As in musical
harmony, controlling movement from dissonance to
consonance supplies a means of moving through time
musically. With a grammar established a visual music
theory can be developed. Visual music can be composed
and abstract animation, in truth any time-based visual
expression, can be seen as musical.

Music Foundations

Stravinskys statement, Music means nothing outside

itself, illustrates a common mindset, held by music
composers for centuries. (Stravinsky, 1956) Music
composition was a formalist activity. This was also a
modernist view. Modernist ideas have been somewhat
tattered over the past few decades, but they can still present
a viable basis for discussing musical time.
We can start with a simple definition of music as the
structuring of time with the materials of sound patterns.
(Lets exclude literary forms such as song, opera and
theatre.) The development of musical instruments has a
history measured in millennia. These instruments were
designed as generators of abstract sounds, without a referent
in the real world beyond the instruments themselves. These
abstract sounds were used to create temporal structures such
as sonatas, fugues and symphoniesabsolute music.
In general Stravinsky is talking about art for arts
sake. From this we can define musical as an aesthetic
response to the perception of sonic pattern t h e
appreciation of significant form, a primary focus of
modernism and its formalist leanings. (Bell 1914)
In music these patterns are built on foundations of
repetition, contrast and variation. There is no structure, no
pattern, without repetition. Repetition by itself can of course
become boring, so contrast is useful in keeping the listener
engaged. As any utterance is multi-dimensional, it is
possible to repeat in one dimension while contrasting

Proceedings ICMC 2004

another. For example playing the same notes on a different

instrument repeats the pitches while contrasting the timbre.
This is variation, a more subtle but effective technique for
creating and developing musical pattern.
The fundamental dimension of music is time. To make
music is to move coherently through time. This is evident in
Western tonal music where the music moves the listener
through time by first establishing a tonal center (balance and
harmonic stability). From this stability the listener is moved
to a sonic dissonance that builds tension. Tension is
resolved through resolution back to tonal consonance and
stability. The music returns to the comfort of familiarity.
Any activity that moves us through time, including dance,
poetry, theatre, etc., uses the idea of tension/release. Boy
gets girl Boy loses girl. Boy gets girl.
From here we can construct a syntax, a consensus on the
componentspatterns that repeat, vary and contrast, and the
combining and ordering of these patterns to create tension
and resolution. Combinations of sound materials become
musical statements. The motif, the minimal material needed
to expresses a musical idea, becomes the building block of
the phrase, a musical line. Phrases are combined and
articulated in time by cadences or punctuation points.

A Visual Music

Gauguin in describing his painting Manao Tupapau said,

The musical part: undulating horizontal lines; harmonies of
orange and blue, united by yellows and purples (their
derivatives) lit by greenish sparks. The literary part; the
spirit of a living person linked to the spirit of the dead.
Night and Day. (Chipp, 1968)
Here we get an insight into a visual artists separation of
form and content. Gauguin used the word musical to
describe the formal aspects of the work. When visual artists
talk about composition they are focused on the design, the
organization of materials in 2D or 3D space. Composition is
traditionally only one aspect of a visual artwork. The
formalist explorations of the 20th century illustrate the desire
by visual artists to achieve a purely musical expression, to
create a visual music by bringing form and content together.
Kandinsky, considered by many to be the father of
modernist abstraction, wrote, A painter, who finds no
satisfaction in mere representation, however artistic, in his
longing to express his inner life, cannot but envy the ease
with which music, the most non-material of arts today
achieves this end. He naturally seeks to apply the methods
of music to his own art. And from this results that modern
desire for rhythm in painting, for mathematical, abstract
construction, for repeated notes of color, for setting color in
motion. (Kandinsky, 1914)
This became truly possible with film. To follow the path
established by traditional music practice, visual music was
formal, abstract and temporal. In the early years of film,
abstract animation was developed by visual artists who
desired to compose visual music. (Russet and Starr, 1976)

Figure 1. Illustration of visual rightness from the book

Composition by Arthur Wesley Dow, 1899. Dow defines
the image on the left as a motif, and the other four as
variations.
The impulse was quickly lost (but never abandoned) in the
Hollywood deluge of character animation and big screen
storytelling. With developments in technology over the past
decade there is a renewed and growing interest in visual
music composition.
Art theorist Rudolph Arnheim says, One of the basic
visual experiences is that of right and wrong. This idea is
the basis of design foundations as taught in most art schools.
(Arnheim, 1966)
Visual rightness is visual consonance. Any
introductory design book talks about how to achieve visual
balance and harmony and good visual composition. (from
Dow, 1899, see Figure 1, to Lauer and Pentak, 1999) From
this premise composing visual music is a simple process. If
rightness is codified and understood, wrongness is easily
created by not being right. Movement from visually wrong
to visually right is a construction of tension/release.
Film and video artists have developed their own
grammars over the past century, and movement from
tension to release is a part of those grammars. The practice
of storyboarding is an example. (Begleiter, 2001) A director
plans a scene as a series of moments, composed as a series
of drawings that guide the eventual motion, framing, and
composition of the picture plane as it changes in time. A
scene is built on phrases that are delineated and cadenced
with moments of visual rightness.
The documentary films of Ken Burns are good examples
of this. (Burns 1990) The camera, under strict motion
control and compositional planning, pans over still
photographs. Motion and picture composition, structured in
phrases, punctuate time with cadences of visual rightness.
The viewer moves through time musically.
Color is another visual dimension available to the visual
music composer. Ideas of color harmony have developed
over the past centuries based on the premise that there is an

Proceedings ICMC 2004

experience of balance when color combinations reduce to an

experience of neutral. (Cheuvral, 1854 and Albers, 1963)
For example combining a primary and its opposing
secondary color is considered harmonious. When visually
mixed these colors reduce to grey. Grey is the most neutral,
balanced color experience. Moving from weighted to
balanced color spaces can also be effective in moving us
through time in visual space. (Evans 1990)
Consider the 1939 film of The Wizard of Oz. When
Dorothy is safe in the stability of home the film is in black
and white. When she is off on her adventures in Oz the film
is in color. The film returns to neutral black and white at the
end, when Dorothy returns to the comfort and safety of
home. (Baum, 1939)
Codified art foundations in design and color theory are
useful as entry points for those interested in composing
visual music. These foundations were developed empirically
over centuries by practicing visual artists. Over the past
decade there has been significant research in the
neurobiology of vision. This research validates and expands
on these art fundamentals. Our knowledge of how the eyes
and the brain process and understand visual information is
growing rapidly. This research offers a fertile field upon
which to continue exploring time-based visual art, and
indicates promising directions for future work. (Zeki, 2000,
and Livingstone, 2002)

After Formalism

John Cage had a different view of musical time. In the

book Silence he wrote, Music means nothing as a thing.
(Cage, 1958) With his music composition 4' 33" he removes
himself from the modernist viewpoint and loosens up the
idea of musical. He simply frames time by specifying
duration, claiming that all sounds heard in the specified
duration. whatever they may be, are the musical materials of
the piece.
Again, the fundamental dimension of music (sonic or
visual) is time. In truth, the most fundamental dimension of
experience is time. (It is odd that some art schools are
introducing time into their foundation courses, calling it 4D,
as an afterthought to two and three dimensional work. Of
course for human experience, time is the first dimension as
none of the other dimensions exist for us without it.)
We understand the world through signals received
through the senses. We are able to make sense of the world
because the signals are structured. Perceptions of sound and
light are built from the reception of waves (patterns) of
changing air pressure and electro-magnetic energy. Hence
pattern is axiomatic for experience, for life. For us, time
passes as the experience of patterns. Experienced time is by
definition musical. The set of possibilities is infinitely large
for time-based art.
Formalist ideas of music composition give us a basis for
entering visual space with musical intent. We can learn
much from the traditions of all art forms, as in the end they

are all time-based. Western music tradition is a wellcodified and understood practice and can provide guidance
in the creation of visual music work. Whether formalism
can succeed as an end in itself continues to be debated. It
does continue to be of value as a technique, a device of
construction, and a basis for exploring visual music
composition.
Technology has opened visual space to composers
interested in expanding their musical ideas into new media.
Fundamentals of music composition can be easily mapped
into time-based visual design and new avenues of musical
expression are possible. As Morton Feldman once
commented to Cage, on realizing the new possibilities that
had opened up for composers in the early post WWII years,
Now that things are so simple, theres so much to do.

References
Arnheim, R. (1966). A review of proportion, Module,
Albers, J. (1963) Interaction of Color. New Haven, CT: Yale
Baum, L. F. (1939). The Wizard of Oz. Warner Studios. (Motion
Picture)
Begleiter, M. (2001). From Word to Image, Storyboarding and the
Filmmaking Process. Studio City, California: Michael Wiese
Productions.
Bell, C. (1914). Art. London.
Burns, K. (1990). A Civil WarA Film by Ken Burns. PBS Home
Video.
Cage, J. (1980) Silence. Wesleyan, CT: Wesleyan University
Press.
Cheuvral, M. E. (1854). The Principles of Harmony and Contrast
of Colors and their Applications to the Artst. West Chester,
Pennsylvania: Schiffer Publishing Ltd.
Chipp Herschel B. (1968). Theories of Modern Art. Berkely and
Los Angeles, California: University of California Press.
Dow, A. W. (1899) Composition. Garden City, NY: Doubleday.
Eggling, V. (1923) Diagonal Symphony: Rhythm 21. Germany.
(Motion Picture).
Evans, B. (1990). Temporal Coherence with Digital Color.
Digital ImageDigital Cinema, SIGGRAPH 90, Art Show
Catalog, LEONARDO, Supplemental Issue. London: Pergamon
Press
Evans, B. (2002). limosa. Tuscaloosa, AL. (Music Animation).
Kandinsky, W. (1914). The Art of Spiritual Harmony. London:
Constable and Company, Ltd.
Livingstone, M. (2002). Vision and Art: The Biology of Seeing.
New York: Henry N. Abrams.
Lauer, D. and Pentak S. (1999). Design Basics. Orlando: Harcourt
Brace and Company.
Miller, D. (2002). Vis a Vis. Boston, MA. (Music Animation).
Russett, R. and Starr C. (1976). Experimental Animation. New
York: Van Nostrand Reinhold Company.
Stravinsky, I. (1956). The Poetics of Music in the Form of Six
Lessons. New York: Vintage Books.
Whitney, J. (1980) Digital Harmony. Peterborough, NH: McGrawHill.
Zeki, S. (2000) Inner Vision, An Exploration of Art and the Brain,
Oxford: Oxford Press.

Proceedings ICMC 2004