If the experimental work going on in RISD's "g-speak
studio" feels a lot like sci-fi, it's probably because students
are working with a groundbreaking technology that's not yet on the market.
Anyone who remembers Steven Spielberg's futuristic thriller Minority
Report would recognize it, though - the ability to operate a computer
through simple hand gestures, much like a conductor leading an orchestra.
At RISD, graduate students are now exploring potential uses
for the g-speak spatial operating environment (SOE) currently under development
by Oblong
Industries. Representing
the first major breakthrough in computer interface design in the past 25 years,
g-speak (the "g" stands for "gestural") enables people
to use computers through gestures - by moving their hands and arms in
the air instead of using a keyboard, touch-screen or mouse. Oblong has installed
the system at MIT and the University of Southern California, but chose RISD
as its third test site because of their interest in "design and humanist
principles" and their expectation that RISD students "will come
up with forms, systems and designs beyond what Oblong alone can imagine," notes
John Underkoffler, the company's chief scientist and cofounder - and
as it so happens, a science advisor for Minority Report when he was still a
student at the MIT Media Lab earlier this decade.
Taught by Digital + Media faculty member Amber Frid-Jimenez
and research assistant Kate Hollenbach MFA '11 GD, the first g-speak
studio - called Embodied Computation: Design for
Fashion, Information and the Body - ran during the spring 2009 semester and is being offered
again in the fall of 2009. Graduate students from Architecture, Furniture Design,
Graphic Design and Textiles took part in the first iteration of the studio,
learning to work with g-speak's sensor-studded gloves, cameras and projectors,
and the developing wearable props and other objects designed to help artists
and designers work more effectively in the next-generation environment.
spring semester focus
Although Oblong's own user "metaphor" for g-speak is that
of a pilot who is essentially "flying" through the SOE, Frid-Jiminez
says her group felt that "it would be more relevant to use a different
metaphor for artists and designers." As one of their first experiments,
students developed individualized login gestures - by moving their hands,
arms and/or fingers in distinctive, replicable ways. As they learned to master
their personal gestures to sign on to the computer, they quickly moved on to
experimenting with the language of gesture to produce amazing imagery and visual
effects.
"We investigated all dimensions of interacting with machines," Hollenbach
says, "and quickly realized that gesturing creates a whole different
set of reactions that go way beyond what you can do by using a mouse or through
verbal commands." In her experiments with digital drawing, Er-ti Chen
BArch '09 first used her own sweeping hand gestures and then tried pairing
up with a partner, but quickly discovered that having certain props - a
straight stick or attached sleeves that limit the span between two arms - helps
control the precision of an artist's gestures.
Hollenbach and Ruth Fore MFA '09 FD developed other props for the system
that allow users to pull, stretch and otherwise manipulate words or images
on the screen in response to the users' own movements in the environment.
As a 3D artist, Mariel Tavares MFA '09 FD chose to experiment with what
happens when gesturing to make computer images appear on an uneven surface
placed on the floor instead of on a flat wall. And eager to introduce a tactile "softness" into
this high-tech environment, Nami Minaki MFA '10 TX collaborated with
Marcos Ojeda MFA '10 GD and others in the studio to create a series of
hanging scrims that are sensitive to gestures within the system. She also began
work on a weaving application that allows users to generate and manipulate
digital patterns through gestures.
"Putting color and material parameters on a project allows students
to focus on form, construction and knit structure," Collins explains. "When
you're forced to solve design problems within set limitations, it can
actually be very liberating and lead to more expansive thinking."
"Oblong realizes that its technology has the potential of becoming
more accessible, emotional and usable in the hands of artists and designers," notes
RISD President John Maeda, a former classmate of Underkoffler at the MIT
Media Lab. "The typical way to develop applications for a system like
this is through programming algorithms. But RISD students are developing
applications that involve beautiful fabrics, dance-like movements, rhythm
and other things that people can relate to. G-speak is a system that is perfectly
suited to how RISD students manipulate the world - by using their hands."
