Professional Documents
Culture Documents
Robotic Shadow
Body motion-based interactive installation, 2014
In order to design robots that will coexist with human beings and
enrich human lives, a sensor technology for “feeling” human will and
gestures becomes necessary. This work is an exhibition designed for
the audience to experience a sensor technology that “feels” human
gestures and movements. Using 3D sensor technology, a surreal and
strange world is displayed on the large screen, in which a group of
cubes stick onto one’s body like magnets, and circular balls fly and
crash into each other. By having the ability to “feel” human gestures,
one is able to experience a fun and strange imaginary world.
AP P R O AC H : Ap p l y i n g t h e l a t e s t t e c h n o l o g y t o e x p e r i e n c e - b a s e d
entertainment attraction
Tel: 03-6658-5888
URL:http://www.it-chiba.ac.jp/skytree/
CREDIT
FILTER CONTENTS
All articles tagged: kinect
Fearful Symmetry:
Ruairi Glynn’s
mesmerising
installation comes
back to life
Published by Bakul Patki
c0
openFrameworks / Robotics
Perspection –
Anamorphic image
and sound synthesis
across multiple
screens
Published by Greg J. Smith
c0
c1
Members / openFrameworks
Connected Worlds –
Interactive ecosystem
for NYSCI by Design
I/O
Published by Filip Visnjic
c1
Members / openFrameworks
EGO – Between the
real and symbolic
Published by Filip Visnjic
c0
MaxMSP / Members
Momentum –
Interactive framework
for creating visuals
from body movement
Published by Filip Visnjic
c0
Members / News
Breaking the Surface –
Responsive ‘ocean’ of
acrylic actuators
Published by Filip Visnjic
c0
c1
c0
Members / TouchDesigner
May The Force Be
With You – Teo Park
Published by Filip Visnjic
c2
c0
c0
Members / vvvv
PREVIO US PO STS
Featured Jobs
Developer at B-Reel
B-Reel LA, located in Venice, is looking for an innovative developer who likes to push
the limits of technology.
W2 Professorship in Interaction Design at KISD – Köln International School of
Design
Köln International School of Design are looking for an outstanding, highly qualified
candidate who will represent the area of expertise both project- and practice-based in
the realm of teaching and research.
Tech Special Projects Architect at Meow Wolf
Meow Wolf is seeking a Tech Special Projects Architect to work directly with the CTO,
Corvas Brinkerhoff, in development of high-level strategies and frameworks for new
projects and opportunities.
General Manager of Technology at Meow Wolf
Meow Wolf is looking for a General Manager of Technology who will be responsible for
coordinating all divisions of the Meow Wolf tech team associated with the assigned
project(s).
Senior Software Engineer / Creative Coder at Wildbytes
Wildbytes is looking for a senior software engineer / creative coder to join our Barcelona
office to work on groundbreaking interactive audiovisual experiences.
Social
Newsletter
Subscribe
Shop
About
© 2017 CreativeApplications.Net
Licensed under a Creative Commons License.
Sections
Blog
Magazine
Events
Shop
Jobs
Apps
Quick Links
About
Advertise
Contact
Submit Project
Membership
Tools
Follow
Twitter
Instagram
Facebook
Vimeo
Kickstarter
GitHub
RSS Feed
Thank you
MW2013:
Museums
and the Web
2013
The annual conference of
Museums and the Web |
April 17-20, 2013 |
Portland, OR, USA
Search
Main menu
Skip to primary content
Skip to secondary content
About
News
Attending
Exhibiting
Conference
People
Publications
Abstract
Figure 1: The Beacon, as seen through the entrance to Gallery One (photo
courtesy of Local Projects)
In the main section of Gallery One, top quality works of art—including
many visitor favorites—are organized into thematic groupings that cross
culture, chronology, and media. Multi-touch screens embedded in the
gallery space invite close examination of the objects on view. Placed 14
feet in front of the groupings of art objects, the screens offer
interpretation and digital investigation of the art. In these interactives,
each artwork in the installation is interpreted through storytelling
hotspots with opportunities to explore artworks visually through
magnification and rotation, and to discover their original context and
location. Each interface has a series of “games” that invite visitors to
engage with the art on view through questions and experiences.
For example, one of the first art installations that visitors encounter is
titled “How Do Our Bodies Inspire Art?” It offers a broad look at
CMA’s encyclopedic collections of sculpture, including an ancient
Roman marble athlete, a ceramic Japanese Haniwa figure, a wooden
African sculpture, and a bronze head by Rodin. The games in the
interactive that interprets this installation encourage visitors to connect
actively with the collection and see themselves in the art on view.
“Strike a Pose” invites visitors to explore figurative sculpture by asking
them to match the pose of a sculpture they see on the screen. A motion
sensor records their pose, and the interactive determines how closely the
visitor has approached the artist’s sculpted body. “Make a Face” offers
visitors the chance to investigate the museum’s collection of portraits
through face-recognition software. A webcam records their facial
expressions and matches them to works in CMA’s collections. “Build in
Clay” encourages visitors to make a sculpture in clay by virtually
kneading, rolling, coiling, cutting, and assembling. Visitors can share all
of their creations through email, Facebook, and Twitter.
Figure 2: How Do our Bodies Inspire Art? – View of the Sculpture Lens. Photo
courtesy of Local Projects
Studio Play is a dedicated space within Gallery One that allows families
to explore the museum’s collections and create art together through
hands-on activities and interactive technology stations. Kids can use
easels to create a colorful drawing, and parents can place it in a frame
for all to see their work on the walls of CMA. Families can create a
dramatic production with shadow puppets, based on works from CMA’s
collection. They can also discover an interactive, multi-touch screen that
allows them to make simple lines or squiggles. The interactive then
reveals works of art in our collection that incorporate the same lines and
squiggles. It’s a fun way for families to become visually familiar with
the art they will see in CMA’s permanent collection galleries.
Figure 3: Line and Shape Interactive – Kids search the collection by drawing in
Studio Play. Photo courtesy of Local Projects
At the section of Gallery One closest to the museum’s new atrium, a 40-
foot multi-touch MicroTile Collection Wall dramatically visualizes all
the works currently on view in CMA’s permanent collection galleries,
plus some that are in storage— over 3,800 works of art. A few seconds
later, the wall changes views, allowing visitors to see a series of focused
groupings of objects from CMA’s collection, organized around curated
themes like “Love and Lust,” “Funerary Art,” and “Dance and Music.”
They will also see objects grouped by medium or geographical region,
drawn dynamically from CMA’s digital asset management system. This
huge interactive tool allows visitors to see the permanent collection as a
living organism, changing depending on the prism through which you
view it. The Collection Wall further functions as a giant group and
individual touchscreen interactive, and allows visitors to touch the
objects represented on the wall to make discoveries. Visitors follow their
curiosity through a visual interface that links each artwork to a series of
associated artworks, giving visitors the opportunity to browse and
explore relationships from object to object. Visitors can favorite works
of art and discover tours that then launch them on a pathway through the
museum’s galleries.
Visitors may also use ArtLens, CMA’s new iPad app, to deepen their
experience. Visitors can download it for free to their iPads, or pre-loaded
iPad 4’s are available to rent on location for a nominal fee of five
dollars. The app is designed for three visitor behaviors:
1. The “Near You Now” function allows visitors to browse and find
digital interpretation of works of art they like based on proximity.
Content is designed in short segments of audio and video, allowing
visitors to choose what they want rather than committing to a long,
linear narrative. Visitors can hear from curators, educators, and
community members to discover the continuing traditions that bring
art to life.
2. The “Tours” function allows visitors to have a more structured
experience in the galleries, taking a tour curated for the block of
time they have available. They can walk through the galleries with
CMA’s director to discover his favorites, or they can follow a
theme that carves a focused path through the museum’s galleries.
The two hundred most recently saved Visitor-created tours are also
available.
3. The “Scan” function uses image recognition to allow visitors to
scan two-dimensional art objects to trigger texts or videos to pop up
on the iPad screen. The immediate delivery of this additional
interpretive content enables visitors to delve more deeply into the
app to learn more about a work of art.
Figure 4: Using ArtLens, visitors can take a tour of their own creation, by
favoriting artworks in ArtLens and on the Collection Wall. Photo courtesy of
Local Projects
In an unprecedented combination of technology interfaces, borrowed or
visitor-owned iPads can be docked at the Collection Wall, where visitors
can save objects from the wall to the ArtLens app, creating a playlist of
favorites. Visitors can author a custom tour from their list of favorites,
saving their tour in ArtLens and on the Collection Wall for other visitors
to discover. Through this feature, ArtLens provides an iPad experience
that allows visitors to navigate throughout the museum, both physically
and virtually from off site, providing far-reaching access to media-rich
stories for CMA’s treasured works of art. Taken together, this suite of
new interfaces transforms the visitor experience by extending the access
and creative agency of each individual visitor.
Figure 8: ArtLens app, Near You Now. Photo courtesy of Local Projects
4. Specifications of the interactives: Hardware and software details
The Collection Wall is a dramatic 40-foot interface displaying over
3,800 artworks from CMA’s collection at once, most of which can be
viewed in the galleries. The Collection Wall also presents thirty-two
curated themes, which can be changed by museum staff on an ongoing
basis. These views are looped in a 40-second cycle. Standing 5 feet by
40 feet, the wall is composed of 150 Christie MicroTiles and displays
more than 23 million pixels, which is the equivalent of more than
twenty-three 720p HDTVs. Every 10 minutes, an application content
management system updates the wall with high-resolution artwork
images, metadata, and the frequency each artwork has been “favorited”
on the wall and within in the ArtLens iPad app. Users can save favorites
to their iPad from the wall by setting their device in one of eight docking
stations, which identify an iPad by detecting an RFID chip on the back
of its case. The Christie iKit multi-touch system allows multiple users to
interact with the wall, simultaneously opening as many as twenty
separate interfaces across the Collection Wall to explore the collection.
Software was written using open Frameworks and runs on two Windows
7 workstations supported by four Linux servers processing the video
across the wall, and an RFID server managing the iPad station
connectivity.
The visitor’s favoriting and sharing activity creates metrics that enable
museum staff to understand what artworks visitors are engaging with,
creating a feedback loop with the museum. Visitors can also queue
curated themes to display on the Collection Wall, playing them like a
jukebox that changes every 40 seconds. These themes can be changed
dynamically by the museum, creating another mode of expression for
staff, and connecting with temporary exhibitions or creating new ideas
for the permanent collection.
Figure 9: The Beacon, encouraging visitors to engage within Gallery One. Photo
courtesy of Local Projects
The Lenses
In all, there are six interactive lenses in Gallery One, each composed of a
large-format interactive 46-inch touch screen that interprets clusters of
related artworks. Each lens is a 1080p HD display with a 32-point,
optically-driven multi-touch overlay. The modular design of the lens
housing provides for easy maintenance and minimizes down time. To
give the lenses a small footprint in the gallery, the Windows PCs
running the software for each lens are located in a remote server room.
Audio is supported from a hidden overhead speaker system that utilizes
an extremely narrow audio beam to isolate the audio to a 2-meter area
where the user would be standing. The sculpture lens uses a Microsoft
Kinect to track user skeletons for the “Strike a Pose” game, and a
webcam to track faces for the “Make a Face” interactive. The software
was written with a mixture of ActionScript 3/Adobe AIR and
C++/openFrameworks.
The “Lion” installation was the first grouping of artworks that we chose
and interpreted, and represents a significant break-through in conceptual
planning for the entire team. We knew we wanted to develop a thematic
arrangement that focused on animals, ever popular with a wide cross-
section of visitors. We further wanted the grouping to allow visitors to
investigate modes of artistic representation, from realism to various
forms of stylization in artworks across time and cultures. We chose the
lion as a familiar animal that visitors could readily call to mind,
recognize when realistically represented, and be surprised by when
rendered in an expressionistic or abstracted way. We also have a strong
selection of works that reference lions from a variety of perspectives,
across the collection. Local Projects was charged to develop a
technology interface to engage visitors with these objects and concepts,
using an interrogative and conversational style rather than didactic. They
developed an innovative approach, conceiving of the fixed technology
kiosk as a transparent lens onto the installation of artworks. They
animated the top level of the interface with a question: What does a lion
look like? The CMA team loved the simplicity of that question, which
was a call to discovery, recognition, and surprise, and became the
visitor’s entry point into the interpretive technology.
Sculpture Lens
Make a Face. In real time, facial recognition software matches a
visitor’s facial expression to artworks within CMA’s collection. The
visitor’s expression is captured and the system measures nodal points on
the face, distance between eyes, shape of the cheekbones and other
distinguishable features. These nodal points are then compared to the
nodal points computed from a database of 189 artwork pictures in order
to find a match. The matched faces are collected into photo-booth-style
strips that are then displayed on the Beacon near the gallery’s entry. The
visitor is also able to email their ‘photo strip’ to themselves and share
with others.
Figure 10: Search the collection by making different faces. Photo courtesy of
Local Projects
Figure 11: Visitors captured at the “Make a Face” game in the sculpture lens.
Photo courtesy of Local Projects
Strike a Pose. The visitor is shown an image of a sculpture in a unique
pose and asked to imitate that physical position. A Kinect sensor
measures how closely their pose matches the original and assigns a
percentage to indicate how well the visitor embodied the sculpture’s
pose. The better the match, the higher the percentage achieved. The
skeleton matching software uses a library of human-generated skeleton
data captured via the Kinect data to quantify the match between the
poses of a museum visitor and each sculpture. Visitors can email their
image capture, see other visitors’ images, and try another pose.
Figure 12: Visitors captured at the “gesture” game in the Sculpture Lens. Photo
courtesy of Local Projects
Figure 13: Visitors playing Expression game in Gallery One’s sculpture lens.
Photo by Local Projects
Build with Clay. Visitors can construct their own clay sculptures in the
same form as the Japanese Haniwa sculpture on display nearby. The
process of creation is presented through an interactive, multi-touch, stop-
motion video.
Lions Lens
Cast a Vote. This activity explores the ideas of realism in representation
and of art as a visual language. Polling questions present a progression
of thought about what a lion looks like and what a lion means, and
aggregates answers in a cumulative interactive infographic.
Epic Stories Lens
Find the Origin. Visitors are asked to match historical and contemporary
popular culture examples to three narrative archetypes that are
represented within the artworks in front of the visitor. Epic stories are
thereby understood as being retold across different eras and cultures.
After five matches, the visitor can watch a chronological sequencing of
all the examples within each archetype.
Tell a Story. The main plot points of the Perseus myth are extracted from
the tapestry hanging in front of the visitor, who is then allowed to put
them in any linear order to form a story arc within a comic book film.
Within the comic book style, visitors can rearrange each narrative plot
point within the cells of a typical comic book layout. They are then able
to add thought and speech bubbles and add their own text or a provided
sample. The visitor can then email to themselves the Perseus comic they
created. Within the film style, the visitor sequences the plot points to
select a soundtrack before watching their film in an animation.
Figure 14: Remix a tapestry into a comic book in the Tell a Story Lens. Photo
courtesy of Local Projects
Globalism Lens
Global Influences. The visitor is presented with an artwork and asked to
guess which two countries on the map influenced the artwork in
question. An introductory animation explores the hybridity present
within many artworks and design objects, thus calling attention to
specific examples of cultural cross-pollination.
Create a Vase. Images and text introduce the vase trade between Europe
and Asia. The visitor is able to make a vase by progressively building
upon chosen options (shape, materials, patterns, and techniques), each of
which is assigned a unique price estimate. The final product is
showcased alongside a similar vase within the CMA collection to
illustrate how techniques and origins affect the object’s market value.
Thirties’ Lens
Draw a Line. After a visitor draws a line across the screen, the
interactive calls up and displays one of 442 artworks from CMA’s
collection, which contains a similar line. All the artwork within this
game was created in the 1930s and contains additional information.
(This is an adaptation of the “Line and Shape” game located within
Studio Play; see below.)
Explore the 1930s. What was the world like then?: A narrative montage
of imagery from the 1930s depression era tells the story of the Great
Depression and Cleveland’s role in this period. The information
presented gives the visitor a context with which to approach the artwork
in the lens and form a deeper understanding of how these artworks fit
into the general themes of that era. The film is coupled with quotes from
Cleveland artists and an accompanying soundtrack.
Painting Lens
Choose a Reason. The visitor is presented with a large image of an
artwork from CMA’s collection (from a pool of 89 artworks in total) and
asked to select one of five reasons they think the painting was created.
Once selected, a visualization shows how other visitors in the museum
voted, along with a short caption giving further information about that
painting and the artist.
Make Your Mark. The visitor is presented with three abstract painting
techniques, represented by different objects from CMA’s collection. The
activity invites users to paint in the style of an abstract artist, exploring
the techniques of pour, drip, and gesture as paint color palettes are
generated from sample artworks. Visitors can contribute their painting to
a collection of visitor-created art on view in the lens.
Figure 15: Understanding abstract painting styles in Gallery One’s painting lens.
Photo courtesy of Local Projects
Remix Picasso. Introducing concepts of multiple and flattened
perspective, and fragmented forms, the visitor is invited to rearrange
abstracted elements or “pieces” of the composition in any way he or she
likes, exploring the interplay between flatness and depth. The pieces
may be manipulated through multi-touch zoom and rotate gestures.
Change Perspective. One-, two-, and three-point perspective is visually
presented via animated, morphing perspectival overlays as they are
applied to artworks within the collection. The visitor is then able to
manipulate a three-dimensional shape, shifting perspective according to
touch points across the horizon lines.
Discover Tempera. The tempera panel painting by Sano di Pietro
directly in front of the visitor inspires a demonstration of the tempera
process within the interactive. This visually captivating interactive
shows each of the five stages of the tempera painting process at a
zoomed-in scale. Each step in the process layers over another, and as a
process is completed, the visitor is able to slide the next step over the
last, seeing the highly detailed transition and effect of each stage in high
resolution. Each stage has an accompanying process video as well.
Studio Play: Two interactives
The “Line and Shape” interactive is located in the Studio Play area of
Gallery One and is oriented toward young visitors. Using multi-touch, it
allows users to “draw” lines across a small wall (twelve Christie
Microtiles) and matches those lines to those found within an artwork
within CMA’s collection. The software searches through over 10,000
annotated lines within 7,000 of the museum’s artworks. From those lines
and images, it chooses the artwork that includes a line that closely
resembles the drawn line and composites the artwork under the drawing,
so that the connection between the two becomes apparent. The
application is written in C++ and uses the openFrameworks library.
“Line and Shape” is unique in that it is the only exhibit running on the
Apple OSX platform. This is the first exhibit in the world to use the
Christie iKit on a MAC. The Line and Shape wall uses two linux
computers to process the video across the twelve MicroTiles.
The size of the initial software download is ~25MB. On first start up,
there is an additional download of data and featured image assets to
ensure app responsiveness. The size of this initial package is ~400MB
and generally takes about 5 minutes on site and 15 to 20 minutes off site,
the latter depending on internet connection. Other image assets are
downloaded on demand, with video assets being served via progressive
download. All images are cached by the app, but video is not.
Near You Now: On site, the app integrates the museum’s Navizon
service. This service, specifically installed for ArtLens, uses the nearest
wireless access points to triangulate the device’s position. Using this
indoor wayfinding technology, the visitor is alerted of nearby artworks
featured on ArtLens. These “Featured Artworks” have interpretive
media (including film, comparative images, text, and audio) and
scanning image recognition functionality, and are featured within a tour
or have related artworks associated with them for additional guided
looking.
Scanning: The scanning feature incorporates the device’s camera and
Qualcomm’s Vuforia image-recognition SDK to provide an augmented
reality experience for users on site. When a user scans artwork marked
with the ArtLens icon, the app will recognize the object and provide
context-sensitive content about the work. This content is anchored
within the app screen to the relevant regions of the physical artwork.
Figure 16: Using the scanning function of the ArtLens experience. Photo courtesy
of Local Projects
Tours: Visitors can select from both museum-curated and visitor-created
thematic tours, with artwork locations specified on an interactive map
that senses a visitor’s current position. Tours provide access to all
interpretive media. Because the tours are directly linked to CMA’s
Piction collection software, the museum can create new tours and
additionally moderate visitor tours.
Today: This modular popup displays CMA’s daily schedule of events
and exhibitions. The app ingests this content from the museum’s website
via a RESTful web service.
Favorites: Visitors can favorite their preferred artworks to share via
social media and can also create a personalized tour for other visitors to
take, which will appear in both the iPad and Collection Wall.
5. Technology integral to design process
Our guiding philosophy for Gallery One was based on collaboration,
teamwork, and an immersion in content to foster the best process of
realizing an ambitious project in record time. Our interactive design
firm, Local Projects, worked in deep collaboration with museum staff in
multiple, extended group brainstorms to translate creative content into
innovative visitor experiences. Many digital experiences were created
and workshopped, and then the best were chosen for final execution.
This helped offer flexibility to align the project budget and scope and
timeline into a final approach that was optimized for each part of the
team.
Modular design was important so that spare parts can be on site and all
exhibits can be repaired on site in less than an hour. The hardware was
also designed early on so that a software or hardware failure could be
mitigated by having the exhibit perform in a limited way. Even if the
actual interactive is not working, there can be engaging signage on the
screen rather than a dark monitor. Remote IP-based power switches are
also integrated into all of the devices to allow for remote reboot of any
exhibit to restore it in the event of a software problem that requires reset
of the touch interface or display.
CMA looked at a few different solutions for indoor wayfinding and
selected the Navizon system since it offered the greatest flexibility to be
retrofitted into an existing environment. The Navizon system uses a
series of small nodes that create a meshed environment that allows the
system to know where a visitor is located. Once the nodes were installed
and calibrated in the galleries, our application developer could tap into
the Navizon API so that its location data could be incorporated into the
iPad app. CMA deployed Navizon’s Indoor Triangulation System (ITS):
Over 100 Navizon ITS nodes were placed throughout the museum to
locate, in real time, the iPads as visitors carry them through the museum.
Even though the app is currently designed for iPad only, we chose this
ITS because it tracks active Wi-Fi devices including Android, iPad,
iPhone, laptops, and Wi-Fi tags with an accuracy of 2 or 3 meters,
pinpointing floor and room. Though no application is actually required
on the devices tracked by ITS, mobile apps like Artlens can leverage
Navizon’s API and be aware of the device’s position anywhere
throughout the monitored area. In addition, the ITS will enable CMA to
measure Gallery One’s success via indicators such as dwell time versus
time interacting with the iPad — i.e., we know how much time a visitor
spends in a gallery and when they are interacting with the iPad screen.
Figure 17. Exploring Dutch Painting on the collection wall. Photo courtesy of
Local Projects
Digitizing the collection
In 1996 the museum’s strategic plan outlined a commitment to
becoming a national leader in the use of new and emerging technologies.
A collections management database and digital scanners were
purchased. By 1998, CMA was able to submit 1000 digital images with
artwork metadata to the AMICO project. In 2003 CMA launched a
massive expansion and renovation project which became an even greater
force behind CMA’s image capture and inventory control. A dynamic
inventory management module within the catalog database became a
necessity, as the building project required the entire collection to be
moved several times. At the same time, the initiatives to create digital
assets for the collection for use in Collections Online received highest
priority, as the administration saw the website as a way to continue to
share the collection with visitors. As the collection was deinstalled and
moved to storage, photographers were able to efficiently schedule large
digital capture projects that would not be possible if the works were on
view. Three photographers divided up the photography by type so that
continuous steady progress is made across collections. CMA is well over
75% in its digital capture initiative and will have 100% digital capture
within the next several months.
References
Adams, M., et al. (2009). Cleveland Museum of Art Permanent
Collection Reinstallation Formative Evaluation Study. Unpublished 75-
page report; PDF available through the Cleveland Museum of Art.
Cite as:
J. Alexander, J. Barton and C. Goeser, Transforming the Art Museum
Experience: Gallery One. In Museums and the Web 2013, N. Proctor &
R. Cherry (eds). Silver Spring, MD: Museums and the Web. Published
February 5, 2013. Consulted November 24, 2017 .
http://mw2013.museumsandtheweb.com/paper/transforming-the-art-
museum-experience-gallery-one-2/
2 THOUGHTS ON “TRANSFORMING THE ART MUSEUM EXPERIENCE: GALLERY ONE”
What did it cost to create and install the interactives? We’re thinking of
doing something similar with a MUCH small visitor center.
Log in to Reply
Great Job….Cheers,
Linda
Log in to Reply
Leave a Reply
You must be logged in to post a comment.
Founded by Archives & Museum Informatics
www.archimuse.com