Marcia K. O'Malley - US grants

Engineering-Mechanical (56) This Adaptation and Implementation project is improving the effectiveness of laboratory exercises by adapting the use of a haptic paddle (a device that allows users to interact via a sense of touch with virtual environments) in a mechanical engineering modeling dynamic systems course. The project is using the successful Haptic Paddle Laboratory series developed at Stanford University and currently used in undergraduate courses at John Hopkins University.

This project is: (1) Improving cohesiveness of course and lab content to deepen student conceptual understanding of system dynamics concepts; (2) Demonstrating that haptic virtual learning increases student's ability to apply conceptual knowledge to real-world systems, and (3) Improving understanding of critical system dynamics topics in a cost-effective way.

Students are investigating how the haptic paddle can serve as a real electromechanical system with known parameters and how to use the haptic paddle as a tool to interact with virtual mechanical systems using LabView, Matlab simulations, and system interfacing.

The project is demonstrating how the materials can be applied to a required course that is present in most undergraduate mechanical engineering curricula. Collaboration with colleagues, including faculty at a neighboring institution at the University of Houston that serves a substantial number of Hispanic students, is demonstrating the effectiveness of the lab series in teaching system modeling and system dynamics concepts to underrepresented classes of students. The dissemination plan includes presentation of a method that demonstrates how to adapt and improve the Haptic Paddle Laboratory series.

These multi-sensory labs are closely tied to course concepts. Assessment of student learning, and the effectiveness of the outreach activities to the University of Houston and prospective Rice students are embedded throughout the project.

The primary goal of this research effort is to improve the effectiveness of skill transfer, rehabilitation, and collaboration via haptic devices. To do so, the PI will formulate requirements for shared control between humans and robots in haptic systems designed for training, rehabilitation, and collaboration. An experimental test-bed comprised of two commercial haptic devices with force sensing capabilities will be employed throughout the project. The PI will study two shared control system architectures for skill transfer. In the first, the human acts as the novice or patient, and the robot serves as the expert. Control schemes for the expert system (the haptic device) will be designed and analyzed theoretically and experimentally. The second phase of the research effort will explore human-robot-human interfaces. Here, the PI will focus on expert-novice and therapist-patient teams, with a robotic system acting as the mediator between the two.

The intellectual merit of this research and education plan is that human factors and control system design will be used to develop effective human-robot systems for skill transfer and rehabilitation. A set of fundamental human-robot interaction issues will be addressed theoretically and experimentally: (1) the design of system architectures for effective skill transfer between humans and robots, (2) skill transfer between human-human teams as a model for human-robot interaction, and (3) the design and implementation of control algorithms that promote efficient skill transfer by effectively controlling feedback to each member of the system.

The broader impacts of this work are improved health care through more efficient and effective robot assisted rehabilitation practices, enhanced skill training for defense and surgical applications, and improved understanding of human-robot interactions for skill transfer. Specific education goals are to create innovative laboratory modules using haptic devices to enhance student learning of important engineering concepts, reinvigorate an introductory robotics course to include hands-on experiments with haptics and robotics, and high school outreach that includes internships, days-on-campus, and demonstrations using haptics to encourage students to pursue careers in science and engineering. Each of these educational goals ties closely to the PI's existing NSF support for curriculum enhancement, and further supports her commitment to combining research and education into a cohesive and integrated career plan

Some of the most exciting cyber technologies on the research horizon involve sophisticated digital systems that interact with the physical world. Examples include remote surgery, physical manipulation of nano-structures, autonomous (ground and air) vehicular travel, and space and terrestrial exploration. Because such applications interact directly with the physical world, it is imperative their physical safety be assured. This project is developing a comprehensive formal framework for producing controllers for cyber-physical systems, with machine checkable proofs of their physical safety. The project brings together ideas from control theory, language design, program verification, program generation, software engineering, and real-time and embedded systems to build a framework that can be applied to challenging applications. The framework promotes an efficient, rigorous engineering process for producing embedded controllers, incorporating explicit models not only of the controller itself, but also of the physical context in which it operates, the required stability conditions, the platform on which it will run, and the associated real-time constraints. The results of the project are being demonstrated and evaluated in the context of a tele-surgery application. This application is currently being developed at the Mechatronics and Haptic Interfaces Lab in the Mechanical Engineering Department at Rice University.

Engineering - Materials Science (57)

This project is developing Smart Vibration Platforms (SVP) for distribution and utilization at several diverse types of institutions and undergraduate degree programs. The SVP is being utilized to teach mechanical engineering dynamics concepts such as damping and structural vibration controls. The "Smart" part of the platform is based on the utilization of two smart materials: a shape memory alloy, which changes stiffness with temperature, and a magneto-rheological fluid, which provides adjustable damping based on use of an electromagnet.

The SVP is a creative and easy to use device, and is advancing student knowledge and understanding by helping students experiment with smart materials, vibrations, and controls. This project is developing and deploying SVPs at six different institutions and extending its implementation to civil, electrical, and materials engineering as well as engineering technology courses. Using the SVP as a tool, the PIs are creating innovative teaching materials, including demonstrations, hands-on experiments, course modules, and new courses.

Theoretical psychology has not, until recently, been in position to have much to say about how humans learn in dynamic, multidegree-of-freedom manual control domains. While applied concerns have led to successful training programs for manual control tasks (e.g., most people learn to steer automobiles in reasonable time), we currently lack the ability to predict how well people will do in these domains or how rapidly they will learn. This limits our ability to train people in these domains, where we presently rely on expensive one-on-one tutoring or similar intensive methods.

The project studies human performance and acquisition of sensorimotor tasks in real and virtual environments. Human motion data and performance of various skills by high performers and low performers, who exhibit linear performance gains, will be analyzed and compared to data for subjects who rapidly acquire skill and exhibit nonlinear performance gains. This data will inform the development of more accurate models of sensorimotor skill acquisition that can be expressed in ACT-R, and doing this should lead to improved understanding of training methods in human motor learning domains.

In many important situations, analytically predicting the behavior of physical systems is not possible. For example, the three dimensional nature of physical systems makes it provably impossible to express closed-form analytical solutions for even the simplest systems. This has made experimentation the primary modality for designing new cyber-physical systems (CPS). Since physical prototyping and experiments are typically costly and hard to conduct, "virtual experiments" in the form of modeling and simulation can dramatically accelerate innovation in CPS. Unfortunately, major technical challenges often impede the effectiveness of modeling and simulation. This project develops foundations and tools for overcoming these challenges. The project focuses on robotics as an important, archetypical class of CPS, and consists of four key tasks: 1) Compiling and analyzing a benchmark suite for modeling and simulating robots, 2) Developing a meta-theory for relating cyber-physical models, as well as tools and a test bed for robot modeling and simulation, 3) Validating the research results of the project using two state-of-the-art robot platforms that incorporate novel control technologies and will require novel programming techniques to fully realize their potential 4) Developing course materials incorporating the project's research results and test bed.

With the aim of accelerating innovation in a wide range of domains including stroke rehabilitation and prosthetic limbs, the project is developing new control concepts and modeling and simulation technologies for robotics. In addition to new mathematical foundations, models, and validation methods, the project will also develop software tools and systematic methods for using them. The project trains four doctoral students; develops a new course on modeling and simulation for cyber-physical systems that balances both control and programming concepts; and includes an outreach component to the public and to minority-serving K-12 programs.

This research involves collaboration among investigators at four institutions. Recent advances in motor behavior have uncovered structure in the supporting neural control architecture, including distinctions between feed-forward and feedback control functions and learning. While the neural code has not yet been cracked, much is now known about how its foundations for sensorimotor control differ from those of even the most modern computer-based algorithms. For example, neural function must accommodate transmission and processing delays, so feedback control is subservient to feed-forward and anticipatory control. The nervous system produces exquisite, constantly and widely available predictions concerning body and environment interactions. These predictive models (also called internal models) are constructed by learning the invariants in the mapping from motor commands to sensory feedback (and inverses thereof). The PIs have developed a unique approach based upon readings from a scalp array of EEG electrodes for the construction of algorithms (decoders) which predict motor behavior (control signals) as a weighted sum of the EEG data from all electrodes at multiple time lags. The team has demonstrated two-axis control over a screen cursor using only 10 minutes of EEG and motion training data, a feat far surpassing any brain-computer interface (BCI) available to date. In the current project, the team will build upon this prior work to design and validate noninvasive neural decoders that generate agile control in upper limb prosthetics. To this end, they will investigate neural correlates of brain adaptation to multiple sources of feedback using EEG and functional near infrared spectroscopy (fNIR). An important challenge will be to provide sensory feedback appropriate to contact tasks performed with a prosthesis. Existing BCIs and neuro-prosthetic devices rely at best on vibrotactile feedback and often only on visual feedback. The PIs will add haptic and proprioceptive feedback in concert with a novel adaptation of vibrotactile, skin stretch, and arm squeeze technologies in the prosthesis interface, to provide intuitive control over contact tasks and to strengthen the motor imagery whose neural correlates are processed by the EEG decoder. To establish baseline measures, the team will compare prosthetic performance under direct brain control to myoelectric prosthetic control and direct manual control. Experiments will be performed involving both able-bodied individuals and amputees, in which real-time decoding (EEG) and analysis (EEG/fNIR) of sensorimotor control and cognitive load will be combined.

Broader Impacts: This research will revolutionize the control and interface of upper limb prosthetics. The work will lead to a better understanding of the role of sensory feedback in brain-computer interfaces and will lay the foundation for restoration of motor and sensory function for amputees and individuals with neurological disease. The project will create a unique interdisciplinary environment enabling education, training, co-advising and exchange of graduate students, course development, and involvement of undergraduates in research. The PIs will also participate in outreach activities on their various campuses, targeting underrepresented groups in science and engineering.

Robotic devices are excellent candidates for delivering repetitive and intensive practice that can restore functional use of the upper limbs, even years after a stroke. Rehabilitation of the wrist and hand in particular are critical for recovery of function, since hands are the primary interface with the world. However, robotic devices that focus on hand rehabilitation are limited due to excessive cost, complexity, or limited functionality. A design and control strategy for such devices that bridges this gap is critical. The goals of the research effort are to analyze the properties and role of passive dynamics, defined by joint stiffness and damping, in the human hand and wrist during grasping and manipulation, and then mimic such properties in a wrist-hand exoskeleton for stroke rehabilitation. The project will culminate with device testing in collaboration with rehabilitation clinicians.

A significant problem in robotic rehabilitation is how to provide assisted movement to the multiple degrees of freedom of the hand in order to restore motor coordination and function, with a system that is practical for deployment in a clinical environment. Armed with a clearer understanding of the mechanisms underlying passive dynamics and control of systems exhibiting such behavior, this project will inform the design of more effective wrist/hand rehabilitation devices that are feasible for clinical use. In addition, the proposed project will create a unique interdisciplinary environment enabling education, training, and co-advising of graduate students, undergraduate research, and significant and targeted outreach activities to underrepresented groups in science and engineering.

DESCRIPTION (provided by applicant): This research aims to accelerate the development, efficacy and use of robotic rehabilitation after stroke by capitalizing on the benefits of patient intent and real-time assessment of impairment. Validation will occur using the MAHI EXO-II exoskeleton robot at The Institute for Rehabilitation and Research (TIRR) in Houston, Texas. Robotic rehabilitation is an effective platform for sensorimotor training in stroke patients. A robotic device enables accurate positioning of the impaired limb while simultaneously providing assistance & resistance forces and collection of motion data that can be used to characterize the quality of the patient's movements. The MAHI EXO-II, a physical human-robot interface, will be augmented with a non-invasive brain-machine interface (BMI) to include the patient in the control loop, thereby making the therapy 'active' and engaging patients across a broad spectrum of impairment severity in the rehabilitation tasks. This approach capitalizes on the known benefits of patient intent in movement initiation observed in other clinical studies of robotic rehabilitation and on the beneficial effects of BMI use on cortical plasticity. Robotic measures of motor impairment, derived from real-time data acquired from sensors on the MAHI EXO-II and from the BMI, will drive patient-specific therapy sessions adapted to the capabilities ofthe individual, with the robot providing assistance or challenging the participant as appropriate, in order to maximize rehabilitation outcomes. Assist-as-needed paradigms in robotic rehabilitation have been shown to be efficacious; however, such paradigms are passive and driven by performance metrics that have not been sufficiently validated and verified. Additionally, intense practice and continual 'challenge' during therapy is known to improve rehabilitation outcomes. Key contributions: 1) Adapting most advanced EEG- BMI methods to stroke patients and developing a BMI for the control of the MAHI EXO-II that will a) increase upper limb function, b) advance understanding of brain plasticity, and c) innovate rehabilitation; 2) Determining appropriate robotic and electrophysiological measures of motor impairment and associated control algorithms for patient-specific therapy; and 3) Clinical validation in pilot studies to evaluate the proposed approach.

This Integrative Graduate Education and Research Traineeship (IGERT) award provides Ph.D. students at Rice University with innovative training in neuroengineering, spanning the disciplines of neuroscience, electrical engineering, mechanical engineering, and bioengineering. In collaboration with Baylor College of Medicine and seven other universities and participating organizations, Rice University trainees are developing the tools to understand, interface with, model, and manipulate the nervous system.

Intellectual Merit: Due to improved technologies that enable neuroscientists to interact with brain cells, and due to the increasing types of neuroscientific data collected through electrical and optical methods, the neuroengineers who create and work with these complex data sets require highly specialized training. This program trains students in three specific areas: (1) cellular systems neuroengineering, which studies the nervous system?s signaling processes at the molecular and cellular levels; (2) engineering multi-neuron circuits, which involves collecting and analyzing data from groups of brain cells and devising methods to induce them to produce new functional responses; and (3) translational neuroengineering, which develops systematic approaches to improve clinical devices such as prosthetics and deep brain stimulators. Trainees in this program are learning to be technologically innovative; to be aware of social, cultural, and ethical aspects of neuroengineering; to communicate their work effectively to a wide variety of audiences; and to understand the pathways to commercialize their discoveries.

Broader Impacts: As this program trains neuroengineers to develop advanced solutions to functional and structural problems in the brain, a new problem-based learning curriculum will result and will be shared with the public through open education resources. By cultivating relationships with biomedical device companies, international researchers, and collaborators within two minority-serving institutions in Texas, students in this program are expanding the applications of their training and increasing participation in their research.

IGERT is an NSF-wide program intended to meet the challenges of educating U.S. Ph.D. scientists and engineers with the interdisciplinary background, deep knowledge in a chosen discipline, and the technical, professional, and leadership skills needed for the career demands of the future. The program is intended to establish new models for graduate education and training in a fertile environment for collaborative research that transcends traditional disciplinary boundaries, and to engage students in understanding the processes by which research is translated into innovations that benefit society.

This proposal will support U.S. Ph.D. students working in robotics the opportunity to share their knowledge and interact with each other and more senior researchers, to learn about different sub-fields within robotics, to meet potential employers, and to become apprised of new technology that will be demonstrated by vendors. This goal will be accomplished by partially supporting the travel costs for U.S. Ph.D. students to attend the International Conference on Intelligent Robots and Systems (IROS), which is one of two major international robotics conferences co-sponsored by the IEEE Robotics and Automation Society. The conference, which in 2016 will be held in Daejeon DCC, South Korea, attracts an international crowd that includes academics, industry workers, entrepreneurs, and funding agency leaders.

Virtual reality has enabled surgeons to train on procedures without risk to patients. However, the feedback surgical trainees receive is delayed, subjective, and qualitative, thereby lacking support for rapid acquisition of skill. This project will enhance surgical performance and training by providing performance feedback using touch-based cues that convey movement quality and strategies rather than task outcomes like procedure time. This will allow trainees to get feedback that is immediate and quantitative, and result in improved performance in difficult-to-train motor skills. Should this research prove successful, there is an opportunity to make meaningful changes in how surgeons are trained.

Co-robots in the endovascular surgical domain can take several forms, from a virtual reality simulator that mimics patient-specific procedures, enabling high-fidelity procedural rehearsal, to telesurgical systems that are used to navigate flexible robotic tools to anatomical locations. Such systems require significant skill to use and, in turn, a rigorous training protocol before certification to operate with the robot is granted. To date, training protocols rely on practice and subjective feedback by a skilled observer, and acquisition of skill on these systems can be inefficient. Furthermore, there is a lack of objective metrics of skill acquisition. This project will investigate the use of haptic feedback to trainees based on motion-based performance metrics, and will evaluate the potential role of this feedback modality during performance and training for simulated and robotic endovascular surgical tasks. In particular, the project will evaluate the benefit of providing specific performance feedback that highlights task strategies in the motion space, and the effect of providing such directed feedback on learning and retention of the task.

This National Robotics Initiative (NRI) project will promote the progress of science and beneficially impact human health and quality of life by developing wearable soft robotic devices with distributed tactile stimulation that enable new forms of communication. Human-robot interactions will be commonplace in the near future. In applications such as self-driving cars and physically assistive devices, interaction will require effective and intuitive bidirectional communication. Transferring information through vision and sound can be slow and inappropriate in many circumstances. This project focuses on haptic (touch-based) robotics to enable communication in a salient but private manner that alleviates demands on other sensory channels. This project serves the national interest by advancing knowledge in the fields of human perception, psychology and neuroscience, while developing novel, convergent technology that integrates concepts across the fields of robotics, haptics, and control engineering. Project results will be disseminated through tactile haptic devices for education and publicly available software and data. The project aims to broaden participation of underrepresented groups in engineering through outreach programs, public lab tours, and the mentoring of female and minority graduate students, undergraduates, and high school students.

Wearable haptic systems have the potential to enable private, salient communication between humans and intelligent systems through an underutilized sensory channel (somatosensation). In this research, information will be transmitted through the haptic channel via wearable, ubiquitous, soft robotic devices that provide both passive and active touch interactions with the human user. This research is comprised of four main objectives. First, a characterization of human perception of the forearm will set the requirements for the frequency, amplitude, directions, spacing, and temporal actuation patterns for a two-dimensional array of haptic stimulators that are able to convey a range of haptic cues. Second, the project will develop a wearable, soft, haptic device able to stimulate the skin of one forearm, while also providing mechanical stimuli that are intended to be explored by the fingertips of the other hand. Third, the project will develop rendering algorithms for the haptic device that take into consideration human perceptual abilities for passive stimulation of the arm and active exploration by the fingertips. Fourth, the project team will create application scenarios to evaluate and refine the system. Wearable haptic systems have potential to improve human health and well-being through a variety of applications including: physical cueing for rehabilitation/movement therapy; explosive ordnance defusing; feedback from assistive devices including mobile robots in the home; tactile communication to enable design and e-commerce; immersion in virtual worlds for education; and the facilitation of remote interaction between people.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.