Lucy Pao - US grants

This project investigates combined visual and haptic (touch) interfaces to display multi-dimensional data generated by computer models of physical systems. The focus is on data features that are difficult to convey visually, such as scalar data which fills volumes, vector fields, and tensor fields. First, tests are conducted to better understand the perception of haptic rendering elements: virtual surfaces and constraints, forces and torques, and mechanical impedances. Second, new haptic rendering modes are developed which can convey multi- dimensional data via combinations of haptic rendering elements. These modes are tested using representative visualization problems in fluid dynamics, electromagnetics, and solid mechanics. Third, visualization puzzles are developed which quantify the perceptual added value of haptic rendering modes. Both cooperative and complementary visual/haptic rendering are explored. The project seeks to discover synergistic rendering modes which produce understanding more easily than visual rendering alone. An improved visual/haptic interface enables intuitive debugging of computer models, efficient visualization of modeling results, and improved physical understanding. Such a capability benefits the conduct of scientific research in many fields, the design and analysis of engineered systems, and the education of future scientists and engineers.

Proposal # HRD-0095944
Institution: University of Colorado at Boulder
Principal Investigators: Lucy Y. Pao, Dale A. Lawrence, and
Howard Kramer
Title: "Haptic Interfaces for Spatial Learning"

ABSTRACT

This project will explore the use of haptic (touch) interfaces, in concert with conventional visual and audio interfaces, to enhance communication and learning of spatial concepts in science and engineering. Graphical means of expressing spatial concepts provide the most clear and concrete representation of spatial ideas, but are often the most difficult for people to use. In contrast to existing approaches that use only vision, the project will seek non-visual means of expressing and communicating spatial ideas and data. The approach also differs from recent attempts to reproduce 2D visual graphs or pictures as 2D haptic or tactile artifacts for the visually impaired. Such approaches depend on projections of 3D objects onto viewing planes, a technique that is only marginally accessible to blind people.

Technology exists that can enable people to draw effectively in 3D without depending on vision or vision-like projections of the 3D object or idea. The project will explore the integration of a 6 degree of freedom (DOF) haptic interface with new software tools that produce a variety of direct 3D drawing capabilities, including the capability to instantly review and correct the concept as it is created. Investigators will explore the benefits of non-visual (haptic and audio) feedback for drawing. We believe non-visual interaction with drawing tools can make graphical representations of spatial constructs, relationships, and ideas much easier to generate and share, promoting clearer discourse in fields that depend on spatial concepts. The ability to create precise 3D drawings would provide a mode of communication for visually impaired people opening new opportunities in fields that require an ability to communicate using spatial representations.

The technology to be developed and test consists of a desktop workstation that provides capability for visual, audio, and haptic interaction with computer-generated spatial constructs. The tools will consist of software programs that allow users to easily draw in 3-dimensions with visual, haptic, and audio feedback. A suite of rendering/drawing modes will also be developed to enable users to create and interpret 3-dimensional objects or drawings.

The existing visual/haptic interface facility at the University of Colorado will be augmented with audio capabilities similar to those currently used in the University of Colorado Assistive Technology Lab. This augmented workstation will be used as a testbed during years 1 and 2 of the project, where work will focus on the development and testing of particular modes of drawing and rendering spatial objects and data, and of particular pedagogic approaches to learning spatial concepts. The resulting rendering modes will be evaluated by students with learning and/or visual impairments as well as non-impaired students who are interested in science and engineering.

In this project, controllers are being developed for maneuvering flexible structures, with particular application to disk drive systems, where the flexibility of the read/write arm are being addressed. The control approach separates the non-flexible and flexible dynamics of the system. Because of the desire for fast read/write access times in disk drives, the approach starts with the time-optimal control for the non-flexible dynamics. Depending on how complex these non-flexible dynamics are, this time-optimal control may be solved analytically or numerically. The notions of input shaping are then used to address the flexible dynamics of the system, where a technique is being developed that incorporates the ideas of input shaping and time-optimal control to derive a feedback control law that causes flexible systems to track desired projected phase-plane trajectories that are constructed based upon state responses to shaped time-optimal control inputs. The research ideas proposed provide a clear methodology for deriving a complete theory for designing feedback controllers for multi-modal flexible structures, including non-ideal effects such as back EMF, inductance, and slew rate limitations. The PI's involvement with the Colorado Center for Information Storage (CCIS) ensures interaction with disk drive companies (such as Maxtor Corporation and Seagate Corporation) so that the techniques developed can be transferred successfully to industry. The results of this research are expected to make an impact in the disk drive industry and hence ultimately an impact on every computer user. Further, because several aspects of the basic approach are more general in nature, it is expected that the proposed research will impact other fields where structural flexibilities must be controlled.

While Atomic Force Microscopes (AFMs) can provide high-resolution images with atomic scale resolution, the speed and actual quality of AFM images depend upon the overall dynamics of the AFM system. AFMs exhibit coupling effects across the axes of motion and the full multi-input, multi-output (MIMO) nature of the system should be taken into account in order to achieve higher speed and higher quality images. In this project, a few control architectures, with varying combinations of feedforward and feedback controllers, will be explored to determine the advantages and disadvantages of each architecture with respect to model uncertainty characteristics and model inversion control design techniques. Based upon these results, adaptive model-inverse based controllers will be developed for some architectures to enable good performance across all "plants" within a set of similar plants. The results of this research will demonstrate advanced control methods for nonminimum phase MIMO systems, with a particular focus towards enabling AFMs to be fast and behave like dependable instruments.

The Atomic Force Microscope (AFM) is one of the most versatile methods of imaging nanoscale structures. It is becoming a driving technology in nanomanipulation and nanoassembly and is playing a burgeoning role in the field of molecular biology. The behavior of current commercial AFMs varies considerably across AFM probes as well as changes in samples and environmental conditions. The variability causes commercial AFMs to not behave like reliable instruments, and this slows down and frustrates AFM users. The overall dynamics of AFMs can be improved by improving either the dynamics of the actuators themselves or by improving the control system. This project focuses on investigating, analyzing, and deriving control systems to increase the speed and image quality in AFMs, while working in close collaboration with a team of scientists and engineers at Agilent Technologies who will be simultaneously developing better actuators for AFMs.

A number of outreach programs are planned at the University of Colorado at Boulder, including several K-12 programs that seek to educate and interest students in the field of engineering, where AFMs will be highlighted as an example of an engineering tool used in many different applications.

The primary research objective of this proposal is to improve the temporal resolution of atomic force microscopy (AFM) through non-raster sampling schemes based on compressed sensing (CS). While AFM continues to be used heavily for the study of systems with nanometer-scale features, its temporal resolution limits its applicability to the study of dynamics. The research approach progresses from non-raster sampling of a single image, including robust time-optimal control techniques to move the tip of the microscope as rapidly as possible between measurement locations, to CS driven schemes for acquisition of image sequences. The methods developed will be implemented and tested on AFMs to demonstrate their capabilities.

If successful, the results of this research will extend the utility of AFMs by increasing the imaging rate while also decreasing the interaction with the sample, limiting any damage caused by the imaging process. While focused on AFMs, the techniques to be developed will be directly applicable to other scanning probe methodologies, such as scanning tunneling microscopy and near-field scanning optical microscopy, as well as more broadly to scenarios in which a short-range sensor is acquiring information in a large area. Examples of such scenarios include environmental monitoring by autonomous robots, large-scale data collection in ocean environments, and weather sampling. Further, the robust time-optimal control results will be broadly applicable in industries that include disk drives, tape drives, wafer scanning systems, electronic manufacturing, and more. Graduate and undergraduate students will benefit through participation in the research while outreach activities will engage middle and high school students in the Boston metro area, with a focus on students from low-income families, and in the Boulder metro area, with a focus on female students.

The Planning Grants for Engineering Research Centers competition was run as a pilot solicitation within the ERC program. Planning grants are not required as part of the full ERC competition, but intended to build capacity among teams to plan for convergent, center-scale engineering research.

The overarching vision of the WiPOWER ERC is to create a "Cordless World" by providing static and dynamic power charging technologies for transportation vehicles, devices in homes and hospitals, and devices on/in the human body. Research thrusts within the center will investigate foundational technologies required for wireless, tether-free power for daily use devices. Recent advances accomplished in tunable electronic design offer the opportunity to develop wireless power transfer (WPT) topologies with the potential to meet the necessary efficiency, power, function and distance requirements for widespread implementation. These advances include tunable passive elements (e.g., capacitors, transformers, and inductors), metamaterial based enhanced coupling and routing of wireless power, broad bandwidth current sensors and circuit architectures that exploit tunability, as well as high-efficiency active components (switches, filters). Utilizing these tunable electronics based topologies, wearable devices and sensors (such as a blood glucose monitor and an insulin pump) can be powered through RF wireless power transmission when the wearer is inside a house/hospital or in its vicinity. The team will investigate approaches such as a phase array to direct the power-carrying RF beam toward one or multiple wearable devices/sensors. Common electronic devices in the home and offices will be able to operate without cords and plugs. The proposed wireless power infrastructure will also enable the coming electric and autonomous vehicle revolution by providing parked charging and dynamic moving charging technology that meet the high efficiency, safety and reliability requirements. The workshops organized during the planning phase will consider how to best coordinate with the industrial, educational and entrepreneurial community on integrating WPT technology to serve the community at large. The WiPOWER ERC planning team will engage with broader impacts stakeholders during the workshops to develop diversity and inclusion plan, technology roadmaps and workforce development for the center. Planning phase will leverage that experience by including representatives from the participating institutions broader impacts community and beyond. The workshops organized during the planning phase will consider how to best coordinate with the industrial, educational and entrepreneurial community on integrating WPT technology to serve the community at large. The WiPOWER ERC planning team will engage with broader impacts stakeholders during the workshops to develop strategic objectives and milestones for the center.

Combination of active and passive components operating at different power levels are required in order to implement WPT systems. Each component has its own set of performance metrics, operational guidelines, testing standard and pricing strategy. Complete understanding of these variables for all the components needed for WPT systems is an essential part of the investigations to be conducted during the planning phase. Reliability and accelerated lifetime testing protocols will be established to ensure robust deployment in dynamic charging conditions. During the planning phase, the goal will be to utilize workshops, industrial visits, conference calls, and exchange programs to develop a consistent summary of the current technology status, technology barriers, intellectual property, industrial roadmaps, regulations and standards, and commercialization partners. One of the emphasis areas in this investigation will be passive components with added-tunability that have adaptability to different circuit operating conditions. Tunable capacitors and inductors will help a critical challenge in WPT - the loss of received power as the gap between transmitting and receiving coils/plates increases or the coils/plates are misaligned. In addition to low-loss tunable passive components, state-of-the-art in the area of high-efficiency power transmitter and receiver active components will be investigated to achieve cost-effective high-efficiency WPT. Advanced circuit designs using wide bandgap semiconductors, such as GaN and SiC, will be identified through extensive interactions with industry partners. Commercialization pathways for ultra-low power applications requiring specialized low-power IC designs in SOI and in CMOS will be identified and incorporated in the targeted testbeds. High power, efficient, and long-life battery systems integrated with WPT will be identified to provide opportunities to bridge the gaps between charging events. Together these investigations will advance an ERC designed to support the foundational requirements around research, engineering workforce development, diversity and a culture of inclusion, and an innovation ecosystem.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

Wind energy represents the largest growing energy source nationally and globally in terms of total added capacity; with an expected 8 GW coming online nationally in 2019 alone according to the U.S. Energy Information Administration (EIA). As the net penetration of wind energy increases, its random delivery of electricity makes it difficult to use as a demand-focused dispatchable resource within the electric power grid. The proposed solution of the NSF ERC entitled Wind Integrated with Storage for Energy Resilience (WISER) is to integrate energy storage into the wind farm infrastructure to facilitate the economic delivery of energy on demand. The opportunity to address this challenge is enabled by recent emergent technologies, including the growing number of larger wind turbines, advances in autonomous control of wind energy, and, most importantly, new breakthroughs in chemistry, materials, thermodynamics, and energy economic analysis. Developing and translating fundamental science in all these areas will allow more than 24 hours of average power to be stored in a single turbine. Achieving this will require highly integrated multidisciplinary academic research and education as well as strong engagement with a wide host of industry partners, all of which will be considered in the planning grant for a new NSF ERC entitled Wind Integrated with Storage for Energy Resilience (WISER).

The ERC planning grant for high penetration of wind energy integrated with storage will seek to uncover the key fundamental technical, economic, and societal questions using cross-pollination of ideas and concepts across these different fields. These questions will set the stage for an ERC to develop a new generation of cost-effective wind storage by investigating compressed air energy storage, liquid metal batteries, and flow batteries directly integrated into extreme-scale turbines (10+ MW), wind farms, and the grid itself. Achieving this will require new knowledge and breakthroughs in: a) the efficiency, formatting, and durability of storage and wind technology at extreme-scales, b) strategies to integrate and leverage storage with wind energy generation and grid infrastructure, c) effective use of autonomous power control for the integrated system dynamics, and d) market-based revenue design and adaptivity to optimize these systems for grid resilience and power demand. The WISER planning grant will develop a globally- competitive diverse workforce through novel student internships, coursework, and pervasive engagement with the key industry alliances that will commercialize, manufacture, install and operate this technology. Through these profound impacts, the WISER ERC will provide a new generation of "renewable energy on demand" that will power increases in renewable energy penetration and enhance the resilience of the power grid. More broadly, this will directly enhance the USA's energy independence, environment, infrastructure, workforce, and global manufacturing leadership.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.