Christoph Bregler - US grants

EIA-9986010
Christoph Bregler
Stanford University

CISE Research Instrumentation: High-Speed Motion Acquisition

The Departments of Computer Science and Mechanical Engineering will purchase 2 high-speed multi-camera setups, a RAID videodisk, and 2 Pentium PC's and an animation software package. This equipment will be dedicated to the support of research in computer and information sciences and engineering. The equipment will be used for several research areas including, in particular, projects addressing:

Anterior Cruciate Ligament-Functional Biocmechanics
Human Motion Models for Tracking and Activity Interpretation
3D Motion Acquisition for the Stanford Immersive Television Project
Animation of realistic and believable characters

The general thrust of these efforts is to develop high-quality acquisition and analysis techniques of human movements.

This project is about the study of representations and algorithms for the computational modeling of human movements. The ultimate goal is to devise a general representation of human motion structure that potentially covers a wide range of domains: from subtle lip motions during speaking and facial expressions, to complex hand and full-body gestures. Possible applications include improved visual tracking algorithms, gesture classification tasks, face and body animation tasks, and new tools for bio-medical studies. Corner-stones will be the use of large example motion databases and appropriate statistical estimation algorithms. In this proposal, we specifically focus on techniques that can acquire kinematic models from video sequences. These models and measurements will be a building block for further studies on dynamics and higher-level human activity representations and will aid motion studies for research in bio-mechanical and medical projects.

The PIs propose to establish a five-year ITR program that will lead the development of the next generation distributed video sensing systems for understanding human movements. Novel models of human movement and structure will be used for modeling the movements of singe-joint and whole bodies with applications to animation, biomotion, and gait analysis for diagnosing and treating movement-related disorders. The interdisciplinary team includes leading researchers from three core institutions - the University of Maryland (lead institution), Stanford University and New York University. The researchers cover a broad spectrum of interests, including biomechanics, computer science and engineering, electrical engineering, and kinesiology. The proposed research efforts will enable novel approaches for realistic animation and the detection of subtle variations in movement, leading to better diagnostic tools and personalized programs for rehabilitation of movement disorders. Strong educational and industrial outreach programs will also enhance our research program.

This project's goal is to develop new motion capture and movement representations inspired by Laban Movement Analysis (LMA). Consider a statement such as: "Hey, that's John striding across the quad, I'd recognize his bold confident walk anywhere!" Every human being has his/her own unique perceivable movement style-repeated recognizable movement elements that can be notated-which a Laban movement analyst would call a "movement signature." These elements in their combinations and phrasing capture the liveliness of a person's movement-his or her dynamic expressiveness. Despite advances in the motion capture process, many important subtleties still get diminished or lost. If applied to computer animation, current motion capture results lack the original dynamic life-quality; the person's movement signature is not preserved, and the animations look rather artificial and robotic. So it is unlikely that John would look like himself in an animation done using current motion capture techniques. The PI will collaborate with artists and scientists that are movement specialists, to study current motion capture shortcomings and to develop LMA filters that explicitly capture perceptually important features and amplify them as necessary to achieve more expressive animations. To this end, the PI plans to exploit new factorization-based capture techniques, and new representations of dynamics that are estimated from example LMA data. The PI will also develop new motion retargeting techniques to transfer and modify the measured dynamic LMA features to other character models. . The PI will apply his techniques to medical analysis and rehabilitation, gesture recognition, and archiving and annotation of movement databases.

Broader Impacts: This project will lead to a better understanding of human movement analysis, and to better computer tools that can be used for human movement in applications related to HCI, medicine, psychology, education, entertainment and the arts.

This project will pioneer a technical, social, and aesthetic grammar for fully immersive interactive installations. This project will identify breakthroughs in the grammar of interactivity analogous to the development of cross-cutting in cinema, which transformed cinema from a diversion to a powerful communication medium; or the introduction of hyperlinking to digital texts on the Internet, which transformed reading from a passive linear activity to an interactive nonlinear one. The intellectual advances will include:
? New systems and technologies for full-body interaction with projected interactive installations that go beyond simple physical cause and effect, including real-time computer vision analysis of viewers? body language, culture and mood; and analysis of large crowds? movements.
? High-level principles of this medium for effective communication. Communication will be explored in the educational (science themes) and the cultural (art and humanity themes).
? Human-Computer Interface analysis of immersive interactive media, including a comparison with passive media and studies with psychologists for independent analysis (pending human subjects review).

The broader impact is the development of a new medium which is as powerful as cinema, and yet where the viewers remain aware of themselves and the people around them as active participants in a mutually created narrative story. This medium may become the dominant means for education and communication in public institutions as the internet subsumes traditional "kiosk" and "wall text" installations.