2011 — 2015 |
Macleod, Donald (co-PI) [⬀] Nguyen, Truong Anstis, Stuart (co-PI) [⬀] |
N/AActivity Code Description: No activity code was retrieved: click on the grant title for more information |
Cif: Medium: Understanding Quality of 3d Video With Applications in 3d Video Processing and Communications @ University of California-San Diego
Research on 3D image/video perception in the light of general principles of stereo processing in the human visual system is being used to derive 3D quality metrics for 3D video applications in order to deliver the best 3D experience.
Human observers can detect differences in depth with high sensitivity, but limited precision. Moreover, while the visual system can represent fine details of the 2D image that are carried by high spatial frequency components (even when the image is rapidly changing), it can not track variations in depth with comparably high resolution in space or time. Thus the representation of stereoscopic depth is restricted both in bandwidth and in bit depth. Because of those limitations, some deviations from accuracy in the representation of depth at the retinal level are perceptually salient, and others less so. Measurements of perceived image fidelity across a range of spatial and temporal profiles for the depth signal are being used to guide the development of optimal video processing techniques, and to allow evaluation of the advantages and limitations of alternative 3D video coding algorithms such as multiview versus video+depth).
Vision experiments investigate both perceived fidelity and perceived image quality in 3D video generated using a variety of encoding schemes. From those results quality metrics are developed and integrated into video processing and communications applications. A human-centric disparity estimation and view synthesis algorithm is being developed for video processing and communications applications; this can also be used to improve the performance of object detection, classification and tracking, and to generate multi views for autostereoscopic display, which finds applications in 3D enabled diagnostic medical imaging and surgical systems.
|
0.915 |
2012 — 2016 |
Nguyen, Truong Dey, Sujit (co-PI) [⬀] Cosman, Pamela [⬀] Milstein, Laurence (co-PI) [⬀] |
N/AActivity Code Description: No activity code was retrieved: click on the grant title for more information |
Cif: Medium: Mobile Multiview Video: Compression, Rendering, and Transmission @ University of California-San Diego
The prominence of 3D video technology has skyrocketed. The 2009 movie Avatar in 3D became the highest grossing movie of all time. Such a movie requires left and right views of a scene. Many video games which provide a 3D experience require multiple views of a scene. Such data is costly to store and to transmit.
This research studies how to efficiently compress multiple-view video data, how to allow the scene to be viewed from any angle at different levels of precision, and how to reliably transmit the data over mobile wireless channels. This research has important applications in science education, traffic monitoring, and surveillance and security. This research studies efficient encoding, rendering, and transmission of multi-view video, aiming for robust performance at arbitrary speeds of mobile units. The research is applicable both to videos of the real world taken with multiple cameras, and to rendered videos. The investigators study left/right view coding such as in the H.264 MVC standard, and view+depth coding. The latter approach is enhanced by encoding the error signal between the original view and its decoder-synthesized version. To optimally design the system, the techniques use cross-layer optimization, in which physical-layer channel-state information and application-layer distortion-rate or slice-priority information are exploited. Whenever multiple views are rendered from an underlying 3D virtual world, the application's bit requirements can be hugely altered by rendering parameters which affect the content and level of detail of the scene. The investigators study user-experience models to quantify the relationship between rendering parameters and user satisfaction, and develop a channel-aware adaptive encoding and rendering algorithm to account for fluctuations caused by transmission over a mobile channel.
|
0.915 |
2015 — 2019 |
Nguyen, Truong Cosman, Pamela [⬀] Dey, Sujit (co-PI) [⬀] Coleman, Todd |
N/AActivity Code Description: No activity code was retrieved: click on the grant title for more information |
Sch: Int: Collaborative Research: Replicating Clinic Physical Therapy At Home: Touch, Depth, and Epidermal Electronics in An Interactive Avatar System @ University of California-San Diego
Physical therapy is often hampered by lack of access to therapists, and lack of adherence to home therapy regimens. This research develops a physical therapy assistance system for home use, with emphasis on stroke rehabilitation. As a person exercises, inexpensive cameras observe color and depth, and unobtrusive tattoo sensors monitor detailed muscle activity. The 3D movement trajectory is derived and compared against the exercise done with an expert therapist. The patient watches a screen avatar where arrows and color coding guide the patient to move correctly. In addition to advancing fields such as movement tracking, skin sensors, and assistive systems, the project has the potential for broad impact by attracting women and under-represented minorities to engineering through health-related engineering coursework and projects, and because home physical therapy assistance can especially help rural and under-served populations.
This project uses bio-electronics, computer vision, computer gaming, high-dimensional machine learning, and human factors to develop a home physical therapy assistance system. During home exercises, patient kinematics and physiology are monitored with a Kinect color/depth camera and wireless epidermal electronics transferable to the skin with a temporary tattoo. The project involves optimization of electrode design and wireless signaling for epidermal electronics to monitor spatiotemporal aspects of muscle recruitment, hand and body pose estimation and tracking algorithms that are robust to rapid motion and occlusions, and development of machine learning and avatar rendering algorithms for multi-modal sensor fusion and expert-trained optimal control guidance logic, for both cloud and local usage. The system aims to provide real-time feedback to make home sessions as effective as office visits with an expert therapist, reducing the time and money required for full recovery.
|
0.915 |
2021 |
An, Cheolhong (co-PI) [⬀] Bartsch, Dirk-Uwe G Freeman, William R. Nguyen, Truong Q |
R01Activity Code Description: To support a discrete, specified, circumscribed project to be performed by the named investigator(s) in an area representing his or her specific interest and competencies. |
Sch: Multimodal Retina Image Alignment and Applications @ University of California, San Diego
RELEVANCE (See instructions):
|
1 |