Pamela Cosman - US grants

TITLE: Image Coding with Constraints

ABSTRACT

This project studies image and video coding schemes for emerging and future
networks. The study concentrates on certain specific types of image coding
including facsimile, still imagery, and video. Networks of the future will
consist of hybrid combinations of wired and wireless components. Commercial
products such as digital cameras, cell phones, and PDAs will send and/or
receive images of various types over a wide range of network channel
conditions. The networks impose constraints such as: low data rate and
high/unpredictable packet losses. The devices impose constraints including
small batteries, small displays, low delay, limited memory, and limited
processing power. Together, these constraints make the design of inexpensive
and efficient image transmission devices of the future a very challenging task.
In addition to designing such systems, a solid theoretical understanding of the
achievable qualities and limitations is important to know.

The main objectives of this research are to achieve deep theoretical
understanding of source and channel coding for images transmitted on lossy
networks and to develop practical algorithms that can be effectively used in
real applications. The work exploits the diverse backgrounds of the PIs in
image and video coding and combined source/channel coding. The investigation
involves code design, theoretical analysis, and computer simulation.
The main topics investigated for constrained image coding include:

(1) Robust facsimile transmission,
(2) Robust low rate video source coding,
(3) Error correction, resilience, and concealment.

To achieve more efficient spectrum utilization, the FCC has been revisiting traditional license-based policies and moving toward the increased use of unlicensed, rule-based, strategies. In this scenario, attention is being given to cognitive radio, which uses spectrum on an opportunistic basis.
The investigators study the design and performance evaluation of algorithms for the transmission of real-time video over cognitive radio channels. Since this technology involves sharing common spectrum, one key question is how much interference its deployment will impose upon the primary users' signals occupying the band. The investigation involves designing specific compression algorithms for video transmission over an opportunistically-used channel, and determining end-to-end performance in terms of video quality.

The goal of a cognitive radio is to increase the spectral efficiency of allocated spectral bands by opportunistically making use of spectrum that is temporarily free of traffic. This is accomplished by sensing the channel, and adapting parameters of the transmit waveform such as modulation format, power, bandwidth, frequency location, and code rate. This research takes a cross layer approach, involving physical layer waveform design, receiver design, and channel state information, and application layer considerations, such as scalable video coding, multiple frame prediction, and end-to-end distortion measures.
System components consume bandwidth and delay, and the research involves allocating a fixed bit budget and/or delay budget across layers. For example, a delay constraint necessitates tradeoffs between system components such as interleaver size and source encoder output buffer size. The main topics investigated are: (a) Adaptive receiver design and performance analysis for video transmission over cognitive radio mobile channels, (b) Optimal design of scalable video encoding for a cognitive radio environment, and (c) Optimal delay budget allocation.

Scalable Multimedia with Unequal Error Protection
Pamela C. Cosman Laurence B. Milstein
Abstract
In today?s communications environment, it is important to have rich media content (images, audio, video) that can scale up or down depending on availability of resources. In quality scalable multimedia, some portions of a bit stream contain information that allows a moderate quality reconstruction of the image or video, and additional portions of the bit stream allow the source to be reconstructed at progressively higher quality. We consider the transmission of scalable multimedia data (image and video) through variable types of channels, with a focus on providing different levels of unequal error protection (UEP) appropriate for different levels of information importance and suitable for the channel conditions.
There are many techniques for providing protection against errors, including forward error protection (FEC), hierarchical modulation, and leaky and partial prediction in video coding. Our research involves two new techniques for combining hierarchical modulation with either image or video to produce enhanced performance. We consider a MIMO-based technique, in which MIMO space-time coding is used to increase reliability for the most important information in the scalable image or video data, whereas MIMO spatial multiplexing is used to increase data rate for the less important information. This is combined and optimized with existing techniques where unequal error protection is achieved by transmitting different power levels on multiple antennas. The hierarchical approaches for UEP, as well as the MIMO techniques for UEP, are considered in conjunction with FEC and with leaky/partial prediction mechanisms for scalable video. We also consider UEP for cooperative communications, where a virtual MIMO array is formed out of cooperating nodes. Lastly, we investigate the effects of delay considerations in UEP.

The prominence of 3D video technology has skyrocketed. The 2009 movie Avatar in 3D became the highest grossing movie of all time. Such a movie requires left and right views of a scene. Many video games which provide a 3D experience require multiple views of a scene. Such data is costly to store and to transmit.

This research studies how to efficiently compress multiple-view video data, how to allow the scene to be viewed from any angle at different levels of precision, and how to reliably transmit the data over mobile wireless channels. This research has important applications in science education, traffic monitoring, and surveillance and security.
This research studies efficient encoding, rendering, and transmission of multi-view video, aiming for robust performance at arbitrary speeds of mobile units. The research is applicable both to videos of the real world taken with multiple cameras, and to rendered videos. The investigators study left/right view coding such as in the H.264 MVC standard, and view+depth coding. The latter approach is enhanced by encoding the error signal between the original view and its decoder-synthesized version. To optimally design the system, the techniques use cross-layer optimization, in which physical-layer channel-state information and application-layer distortion-rate or slice-priority information are exploited. Whenever multiple views are rendered from an underlying 3D virtual world, the application's bit requirements can be hugely altered by rendering parameters which affect the content and level of detail of the scene. The investigators study user-experience models to quantify the relationship between rendering parameters and user satisfaction, and develop a channel-aware adaptive encoding and rendering algorithm to account for fluctuations caused by transmission over a mobile channel.

Physical therapy is often hampered by lack of access to therapists, and lack of adherence to home therapy regimens. This research develops a physical therapy assistance system for home use, with emphasis on stroke rehabilitation. As a person exercises, inexpensive cameras observe color and depth, and unobtrusive tattoo sensors monitor detailed muscle activity. The 3D movement trajectory is derived and compared against the exercise done with an expert therapist. The patient watches a screen avatar where arrows and color coding guide the patient to move correctly. In addition to advancing fields such as movement tracking, skin sensors, and assistive systems, the project has the potential for broad impact by attracting women and under-represented minorities to engineering through health-related engineering coursework and projects, and because home physical therapy assistance can especially help rural and under-served populations.

This project uses bio-electronics, computer vision, computer gaming, high-dimensional machine learning, and human factors to develop a home physical therapy assistance system. During home exercises, patient kinematics and physiology are monitored with a Kinect color/depth camera and wireless epidermal electronics transferable to the skin with a temporary tattoo. The project involves optimization of electrode design and wireless signaling for epidermal electronics to monitor spatiotemporal aspects of muscle recruitment, hand and body pose estimation and tracking algorithms that are robust to rapid motion and occlusions, and development of machine learning and avatar rendering algorithms for multi-modal sensor fusion and expert-trained optimal control guidance logic, for both cloud and local usage. The system aims to provide real-time feedback to make home sessions as effective as office visits with an expert therapist, reducing the time and money required for full recovery.

Low-income students are underrepresented in engineering and more likely to struggle in engineering programs. Research has found that increasing first and second-year retention enhances the ability of academically talented low-income students to successfully graduate with engineering degrees. In this collaborative research project six institutions will replicate, improve, and test a model of student success originally developed at the University of Colorado, Boulder. The model is designed to increase the retention, success, and graduation of low-income (Pell-eligible) academically talented students from underserved populations. The project will make scholarship awards to 800 students across a consortium of the six partner institutions. To support the students, the project will adapt and implement an ecosystem of high quality evidence-based curricular and co-curricular activities. Members of the consortium are: the University of Colorado, Boulder; the University of Washington; Washington State University; Boise State University; the University of Illinois, Urbana-Champaign; and the University of California, San Diego.

The Redshirt in Engineering Consortium is committed to propagating the "Redshirt" model, which focuses primarily on the first-year of college and consists of intrusive academic advising, an innovative first-year academic curriculum, community building, and career awareness. The term "redshirt" refers to the idea of providing an extra year of preparation for the rigors of engineering curricula. A quantitative and qualitative mixed methods research study will examine the implementation of the model under different conditions and with different student populations. A comparative longitudinal study will examine differences in expected student outcomes between scholarship recipients and similar students who are pursuing engineering degrees. The primary analytic approach for the ethnographic research will be the Constant Comparative Analysis, which will involve concurrent engagement in data collection and data analysis.

The underrepresentation of women faculty in STEM limits the potential for scientific creativity and reduces available role models for female undergraduate and graduate students pursuing STEM careers. Research on implicit bias indicates that women candidates for faculty positions are less likely to be selected than men. It is critical to understand the experiences of hiring committee members, and of candidates for faculty positions, in order to advance STEM academic hiring practices. Having a more diverse STEM academic workforce contributes to promoting the progress of science.

The project team will investigate whether and, if so, how bias against women creeps into the process of selecting and interviewing tenure-track faculty candidates at a highly ranked public research university. There are three studies: Study 1 analyzes the deployment and outcomes of faculty using evaluation rubrics in the processes of rating candidates and recommending them for interviews and for hire. The research team will compare rubric ratings and outcomes to its own assessment of scholarly production, impact and teaching quality, extracted from candidates' CVs. Also, video-recordings of the candidates' formal job talks will be analyzed to determine social interactions, such as the number and tone of questions and interruptions, which may vary depending on whether the candidate is male or female. Study 2 surveys advanced graduate students, postdoctoral fellows and newly hired faculty to examine if men and women self-report being treated differently during academic STEM interviews, and to determine if men and women react differently to similar interview events such as interrupting. Study 3 surveys early career scholars to determine their perceptions of discipline-specific understandings and stereotypes about merit and how these perceptions may or may not affect how they navigate their own interview processes.

This project is supported by NSF's EHR Core Research (ECR) program. The ECR program emphasizes fundamental STEM education research that generates foundational knowledge in the field. Investments are made in critical areas that are essential, broad and enduring: STEM learning and STEM learning environments, broadening participation in STEM, and STEM workforce development. The program supports the accumulation of robust evidence to inform efforts to understand, build theory to explain, and suggest intervention and innovations to address persistent challenges in STEM interest, education, learning and participation.

This award is a planning grant for the Spectrum Innovation Initiative: National Center for Wireless Spectrum Research (SII-Center). The focus of a spectrum research SII-Center goes beyond 5G, IoT, and other existing or forthcoming systems and technologies to chart out a trajectory to ensure United States leadership in future wireless technologies, systems, and applications in science and engineering through the efficient use and sharing of the radio spectrum. Over the past four decades, worldwide growth of wireless systems has provided tremendous societal benefit, enabling increased digital connectivity, telemedicine, social media, improved weather forecasting and more. It has also put significant and competing demands on many parts of the radio spectrum. To meet these challenges, this project supports the Wireless Innovation and Spectrum Evolution (WISE); which convenes former members of the Senior Executive Service in the FCC and NTIA and university faculty from the University of Colorado Boulder, the University of California San Diego, the University of Pittsburgh, and the University of Puerto Rico Mayaguez. The WISE group will develop a research agenda and partnerships through workshops and discussions. Students will be supported to participate in these workshops and to develop spectrum science modules for K-12 students.

Through the center development process, stakeholders from academia, the federal government, and industry will be engaged to identify key technology, policies and workforce challenges to be addressed by the center. Six core research focus areas will include: Spectrum Efficiency and Coexistence; Next Generation Radio Devices; Spectrum Science; Modeling and Measurement; Wireless Systems; and Policy, Economics and Privacy. The process of bringing researchers, policy makers, and key stakeholders will result in the development of a vision and road map for an integrated and successful Wireless Innovation and Spectrum Evolution Center.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.