Pioneers and Performers
|Experts in the field offer their opinions about the state of the
art and the technology of performance animation.
by Barbara Robertson
It`s been nearly a decade since the first 3D computer graphics
(CG) characters animated with motion data rather than keyframe
animation appeared onscreen: In 1988, the Kleiser-Walczak
Construction Company introduced Dozo, a CG rock star who sang "Don`t
Touch Me" as she danced using motion captured from a human dancer.
That same year, deGraf/Wahrman brought "Mike the Talking Head," the
first interactive 3D CG character, to SIGGRAPH.
How successful is this data-driven animation? Although 3D
characters created with keyframe animation have been populating
nearly every type of entertainment medium during the past few years,
with some--Woody and Buzz, Draco, Casper, T Rex--even becoming film
stars and household words, their data-driven cousins have rarely
seen the limelight. Thus it seemed time to check in with some
pioneers and practitioners in the field of performance animation to
find out just what`s happening in this corner of the computer
We talked with Brad deGraf, CEO and founder of Protozoa (San
Francisco); Michael Wahrman, pioneer and computer animation director
(New York City); Scott Dyer, cofounder of Windlight Studios
(Minneapolis), an animation studio; Larry Lamb, founder of Lamb
& Company, an animation studio, and of Lambsoft, a software
company (Minneapolis); Gregory Panos and Richard Cray, a technology
developer and a stage performer and musician, who are cofounders of
the Performance Animation Society (www.pasociety.com); Gary Jesch,
president of Chops, a performance animation company (Incline
Village, CA); David Sturman, director of R&D at Medialab, an
animation studio in Hollywood and Paris; Chris Walker, founder of
two animation studios, Mr. Film and Modern Cartoons (Venice, CA);
Jeff Kleiser, founder of Kleiser-Walczak Construction Company, an
animation studio in North Adams (MA), New York City, and Los
Angeles; and Dave Verso, president of SimGraphics, a performance
animation company (South Pasadena, CA).
CGW: Why isn`t performance animation taking off any faster--or do
you think it`s evolving at a reasonable pace?
LAMB: I think putting body English into animation was a Holy
Grail. It has been and still is a great idea. But it has been a very
difficult thing to do.
VERSO: We`re maybe a third of the way toward becoming a mature
technology. We started about five years ago in terms of having
talking characters. It will probably take another 10 years for the
processes and tools to evolve and for enough people to have
experience. You can`t just buy a tool and get an animation; it`s not
a magic button. It`s complicated. It`s a process.
WAHRMAN: Some people thought this technology would make animation
easy. But it`s not about equipment, it`s about art. And nothing
about art is easy. Jim Henson with a green sock on his hand was
Kermit. Michael Wahrman with a green sock on his hand is Michael
Wahrman with a green sock on his hand.
CRAY: When the prices come down and more talented performing
artists can work with the technology at their own pace and on their
own terms, it will become more creative.
STURMAN: The Jurassic Park of performance animation hasn`t
arrived yet, but it will. The technology is mature enough to have
something important happen.
CGW: Do you draw a distinction between performance animation and
JESCH: Motion capture is a component of performance animation;
one of the performance animation tools. It`s particularly good for
lip sync. But we also use joysticks, midi sliders, and other
devices. You could do a performance animation without a lick of
DYER: The difference is in application. Motion capture is
applying motion to a model. Performance animation is acting a
character. At Windlight, many of the characters we animate are not
humanoids, yet we use actors to drive the performances of these
characters. In our system, the actor sees the character as he`s
CRAY: Performance animation is really live theater.
DEGRAF: Think of the difference between performing Moxy live and
capturing Michael Jackson dancing. You can capture Michael Jackson`s
motion without seeing the character it will be applied to. But
imagine trying to puppet a character without seeing the puppet.
[Moxy, a real-time 3D character developed at Protozoa, appears on
the Cartoon Network; data from a dancing Michael Jackson was
captured at Digital Domain and used to help make a skeleton dance in
a music video.]
STURMAN: For the most part, with motion capture you see the
animation a day to a week later, but you can get high-quality images
as in Andre Agassi [a TV commercial created at Digital Domain] and
Michael Jackson, and the work being done at Windlight. At Protozoa,
Medialab, Modern Cartoons, and SimGraphics, a performer sees what
the character is doing. Most of our characters receive no
post-treatment. It`s more like puppetry than acting. They`re driving
a character, and everyone sees the result; the director sees what he
WAHRMAN: Part of what makes performance animation so great is
that it`s alive. It can be improvised.
WALKER: When a performer or director can see the character
onscreen, rendered in real time, talking to them, the difference
really hits home. That`s the distinction.
CGW: Why do you think some animators have such a negative opinion
of motion capture and performance animation?
STURMAN: They`ve missed the point. We`re not trying to replace
the animator. It`s not about that. It`s digital puppetry. Because
it`s on a screen and there is image after image, it`s called
animation, but in process it`s more like puppetry. Unfortunately,
many of the performance characters are stiff, the shapes are blocky,
the movements aren`t smooth, their facial expressions are rigid, and
they don`t come off as alive. But that`s not a question of
technology, it`s a question of experience and of designers.
VERSO: Some people have been burned. They`ve been promised things
by people who didn`t understand the issues. Some have been trained
another way and this is new to them. Sometimes it has been sold as a
replacement. And some people compare this evolving form of animation
to more mature techniques.
LAMB: For keyframe animators, it`s been an either/or situation
that`s left them out in the cold. They couldn`t adjust the captured
motion data without losing some or all of the subtleties. Animators
just weren`t part of the party. Plus, it`s often used for
low-resolution animation; not to capture acting but to do low-end
animation. And, well, you get low-end animation. So unfortunately,
there are many bad examples.
DYER: The more bad examples there are, the more people will think
badly of it. There`s always a certain purity about traditional
animation, and when motion capture is used, there`s a view that the
animator`s art has been subverted. But this view is often held by
people who haven`t experienced what they can do.
WALKER: We`re really creating another art form that`s a cross
between cartoons and puppets. People haven`t accepted it. We`re
still creating it. We have yet to really see it stretch its muscles.
CGW: Give us some examples of successful performance animation.
WAHRMAN: One of the most creative successes I`ve seen is Fabrique
the villainous robot created by Medialab in a test for Starwatcher.
I thought the Fabrique test was brilliant--ominous, mysterious,
evil, genius. Another thing that blew me away was our second-take
rehearsal for Mike the Talking Head. One of the most commercially
successful real-time characters is without doubt Bert the Fish
[created by Medialab for Nickelodeon (UK)].
WALKER: We just finished CyberLucy for CBS`s Wheel 2000 on
Saturday mornings. The show is a children`s version of Wheel of
Fortune and CyberLucy is that show`s Vanna White. She`s on a 9x12
screen in the set; the performer is off stage. The children go nuts
when they see her. I think Digits `n Art (Montreal) has done some
really good work. SimGraphics has really great technology.
DEGRAF: I think of Bert the Fish and Donkey Kong Country [both
created by Medialab] as reaching a new level in terms of commercial
awareness. In terms of return on investment, I think of our own Dev
Null. I`m biased, but it`s been on the air for a year now. It`s a
branded character...maybe not Ren and Stempy, but...
Also, to me, our Moxy was a big leap. We had Class A writers, a
good performer, and state of the art software. Beyond that, we`ve
all done characters all over the place.
LAMB: Brad`s Moxy. The demo of how Moxy worked is one of the best
pieces of performance animation I`ve seen. The Boxer from Digits `N
Art. It`s immodest, but I think our Huzzah showed a range of
expression and emotion that was successful. We discovered that it`s
really important to capture the entire performance. There is a
continuity between facial twitches and finger twitches and when we
tried to do faces separately, there was a discontinuity. We captured
facial motion with an optical system, hand motion with a glove, and
body motion with a magnetic system.
CRAY: Probably the silliest and simplest was a quick interaction
between Sinbad and Flea on Protozoa`s demo tape. It was completely
off the cuff and improvised. It wasn`t profound, but it made me
think, "This is going to work."
VERSO: We just did a project in Malaysia. The client had just
started up a facility and the people weren`t trained in a particular
mindset. We were able to come in and they could quickly do an
animation that was out of their own culture and their own aesthetic.
In terms of technical success, at SIGGRAPH we showed multiple
characters in a scene. Even as crude as it was, the characters were
STURMAN: I`m pretty interested in what Protozoa is doing on the
Internet. The idea of an ongoing cartoon in VRML sounds like fun.
KLEISER: Digital Domain`s work on Andre Agassi. Our most
successful work was on Judge Dredd. We captured the motion of people
on motorcycles and applied it to digital stunt people that were
flying [digital] motorcycles.
JESCH: I think one of the biggest commercial successes I`ve seen
was Mario for Nintendo. They took a system and a performer on the
road to WalMarts. [Mario was a real-time 3D character created by
SimGraphics and performed by Charles Martinet.] I thought Suzy the
Sex Therapist from Mr. Film was successful at suspending disbelief.
CGW: What is the primary market for performance animation or
motion-capture animation, and how do you see that changing?
JESCH: I decided to design and perform my own character, so for
me, it`s tradeshows and special events. I`m not a Hollywood studio,
but I`m making money. It would never have worked, though, without
the help of Motion Analysis [optical motion-capture technology],
Quantum Works [Geppetto performance software], and 3Dfx [3D graphics
chip]. It`s a thrill to go back to them and tell them this is a real
KLEISER: I think the technology is very handy for human-like
motion. The most likely market is for a TV series with lots and lots
of footage of characters walking around that`s on a tight schedule
and has a tight budget.
STURMAN: Broadcast TV fits really well. It`s also very good for
line tests. If you could block out a scene using live actors and use
it as a reference, you could easily try 10 scenes with your
characters, your snake, your rabbit, your lion, and use them as a
reference for an animation. You won`t get fine-tuning, but you can
get a rough idea in a day.
VERSO: Long-form animation--when time-to-market and cost are
important and there are volumes of animation. We also have a lot of
interest in location-based entertainment--not just trade-shows, but
also putting interactive characters in places like Treasure Island
in Las Vegas.
LAMB: Today, captured motion is just another tool along with
keyframe animation for creating moving 3D objects. I believe motion
capture opens the door for animation in corporate videos and
DYER: Today, probably games, especially sports or action games.
Secondarily, TV shows with performed characters. It`s working into
features and commercials.
DEGRAF: Small broadcasters around the world who need
differentiated content. A good application is a Muppet-style
multiple-character show. I think there`s a type of show that could
take the writing and show design of a puppet show and marry it with
a 3D animation look that would be interesting and allow for great
stories. Sort of Jim Henson meets Toy Story.
On the Internet the bang for the buck is really significant
compared to any other way of livening up Web pages. We`ve done 65
episodes of Floops [for Silicon Graphics] and we`re doing 50 of
Dilbert, sponsored by Intel. The medium is a good fit.
WALKER: For us, the primary market is television. We`ve been
working on a new adult series, a political satire. We`re excited
about it, but our instinct is that children`s animation is a better
fit. We`re certainly moving into movies and want to do a full-on
WAHRMAN: My biggest disappointment has been that the Henson
organization hasn`t used performance animation. They have the
properties, the talent, the organization. They could do it.
CGW: What tools--hardware or software--need to be developed?
JESCH: I`d like to see face tracking under $5000. I`d also like
to have spline-based tracking. Here`s what I mean: If you were to
draw a line around your lips, you`d have a curve. When you talked,
the curve would change shape.
WALKER: Faster computers. Faster render speeds at more affordable
prices. As for the sensors--all the systems are good, and none is
the best. Each has its advantages and problems. Optical systems have
less spatial limitations than magnetic, but you don`t get real time
with more than one character. The mechanical systems look beautiful,
but they aren`t as accurate. We use a hybrid of all the
technologies. Our focus is on production ergonomics, on making our
performers feel comfortable.
DYER: The translation tools that can turn a doughboy into a
dinosaur are not well developed or understood. We also need tools
and interfaces for manipulating, filtering, and storing data.
DEGRAF: Bodysuits and face trackers are still too expensive. We
still don`t have a good glove. They`re better, but quirky. The Holy
Grail is to do unlimited characters in an untethered way in a large
space with no occlusion, no noise.
STURMAN: Capture needs to be lighter, less intrusive, less
susceptible to environmental problems. Hardware needs to be faster.
We`re just starting to do multiple characters. We have an Infinite
Reality with a gazillion processors and we can get two good-quality
worms doing a lot together. And software needs to evolve. The
software doesn`t support characters touching each other yet.
LAMB: The recording equipment is fairly well along. We`re
finishing up two pieces of software that I think will address some
fundamental problems. One is the ability to take motion from varying
sizes and shapes of performers and apply it to varying sizes and
shapes of characters. Another is making it easier for keyframe
animators to adjust the captured motion.
CRAY: I have never believed that a performer should have to wear
computerized clothing. I want control over motion without wearing
something that`s pieced together like Frankenstein`s monster.
PANOS: The toolmakers have been working separately from the
talent; performers haven`t been involved in the development of the
tools. The sensors could be much less invasive. You could have
body-tracking sensors that would be more like jewelry than clothing,
and develop sophisticated gestural recognition software. It`s funny
that people haven`t married virtual reality and performance
animation, but they will.
CGW: Do you think this will ever make it into the mainstream or
DYER: More quickly than you can imagine. You`ll put on a suit and
you`ll get to kick and fight with a character or do a work-out tape
or analyze a golf swing. We just need less expensive, less
restrictive capture equipment. If you could get a bodysuit for $100,
there would be an awful lot of applications.
||Performance-Animation pioneer, Chris
Walker of Modern Cartoons, turns game character Crash Bandicoot into an interactive character for Sony Computer Entertainment.
|These Extreme Dinosaurs were created at Windlight
Studios for a Foote, Cone, and Belding TV commercial advertising
The animation was created by capturing data from
performers who acted the part of dinosaurs.
|Donkey Kong and his virtual pals have
gone from Nintendo video game to TV stars, thanks to the
Medialab Real-Time Performance Animation System
used to animate these characters in real time as they co-host a new
magazine show for kids on French TV.
Richard Cray, shown here animating the "Sunny" character
in an AOA optical facial expression capture helmet for
"A Musical Performance Animation" to introduce performing
artists to the new medium via the special interest group: the Performance Animation Society,
which he co-founded with VR guru Greg Panos.
||At SIGGRAPH `97, real-time characters could be found dancing in
the booths and walking the floor. SimGraphics demonstrated a
multiple-character version of its real-time software with two
Meanwhile, Protozoa`s Alive! software helped
digital puppeteers control Reginold the roving bug reporter as he
interviewed attendees (right).
|Short Stack, the star of this TV spot created for the Louis
London agency by Windlight Studios, is one of three
delicious-looking characters animated using data captured from
||Meet Dr. Dexter Finnery Klaus, an evil geneticist who lives in an
in-house project titled "Meat" at Protozoa.
Dr. Klaus is one of
several real-time characters created at Protozoa for television
(Moxy) and for the Internet (Floops and Dilbert).
|Gary Jesch is a performer whose real-time animated character
Chops makes regular appearances at trade shows and business
To animate Chops and other characters, Jesch uses a
PC-based system with Geppetto software and Motion Analysis optical
Computer Graphics World