David J. Kriegman - US grants

Consider creating (a) a computer system to help physicians make a diagnosis using all of a patient's medical data and images along with millions of case histories; (b) intelligent buildings and cars that are aware of their occupants activities; (c) personal digital assistants that watch and learn your habits -- not only gathering information from the web but recalling where you had left your keys; or (d) a computer tutor that watches a child as she performs a science experiment. Each of these scenarios requires machines that can see and learn, and while there have been tremendous advances in computer vision and computational learning, current computer vision and learning systems for many applications (such as face recognition) are still inferior to the visual and learning capabilities of a toddler. Meanwhile, great strides in understanding visual recognition and learning in humans have been made with psychophysical and neurophysiological experiments. The intellectual merit of this proposal is its focus on creating novel interactions between the four areas of: computer and human vision, and human and machine learning. We believe these areas are intimately intertwined, and that the synergy of their simultaneous study will lead to breakthroughs in all four domains.

Our goal in this IGERT is therefore to train a new generation of scientists and engineers who are as versed in the mathematical and physical foundations of computer vision and computational learning as they are in the biological and psychological basis of natural vision and learning. On the one hand, students will be trained to propose a computational model for some aspect of biological vision and then design experiments (fMRI, single cell recordings, psychophysics) to validate this model. On the other hand, they will be ready to expand the frontiers of learning theory and embed the resulting techniques in real-world machine vision applications. The broader impact of this program will be the development of a generation of scholars who will bring new tools to bear upon fundamental problems in human and computer vision, and human and machine learning.

We will develop a new curriculum that introduces new cross-disciplinary courses to complement the current offerings. In addition, students accepted to the program will go through a two-week boot camp, before classes start, where they will receive intensive training in machine learning and vision using MatLab, perceptual psychophysics, and brain imaging. We will balance on-campus training with summer internships in industry.

IGERT is an NSF-wide program intended to meet the challenges of educating U.S. Ph.D. scientists and engineers with the interdisciplinary background, deep knowledge in a chosen discipline, and the technical, professional, and personal skills needed for the career demands of the future. The program is intended to catalyze a cultural change in graduate education by establishing innovative new models for graduate education and training in a fertile environment for collaborative research that transcends traditional disciplinary boundaries. In this sixth year of the program, awards are being made to institutions for programs that collectively span the areas of science and engineering supported by NSF

In the future, in the digital fabric, computing elements will be distinguished by their networked bandwidth, compute and data capabilities, and of course their particular input/output capabilities (e.g. graphical displays, cameras, GPS, laser range-finding, etc.). These elements will be knit into large scale distributed applications and resource pools - together an entity increasingly known as the "computational grid". The key capabilities of a "Grid" element will be determined largely by their communication and input/output capabilities.

We propose a research infrastructure (FWGrid) that enables research in innovative and radically new applications, systems and system architectures, and the emerging technical and even social use of systems and services built for technology environment of the future. The key aspects of this infrastructure are:
o highly capable mobile image/video capture and display devices (to interact with the world)
o high bandwidth wireless 100-500 Mbps (to tightly couple mobile elements to high capability resources),
o rich wired networks of 10-100Gbps (move and aggregate data and computation without limit), and
o distributed clusters with large compute (teraflops) and data (10's of terabytes) capabilities (to power the infrastructure).

Broader Impact: FWGrid will enable us to explore with real systems and users, radical new applications, novel application structures, system architecture, resource management policies, innovative algorithms, as well as system management. It will fundamentally enhance undergraduate and graduate education and all activities of the department. Because FWGrid will be a "living laboratory", we will gain access to real users and actual workloads. FWGrid models the world of five years hence, where widespread high bandwidth wireless, extreme wired (fiber) bandwidth, and plentiful wired computing and data resource are the norm. FWGrid will be used to support a wide variety of research, ranging from low-level network measurement and analysis, to grid middleware and modeling, to application-oriented middleware and new distributed application architecture and finally to higher level applications using rich image and video - both off and on line.

FWGrid will have a transformative impact on the department, accelerating our growth into experimental computer science, continuing the transform and broaden both undergraduate and graduate education, providing a modern Grid infrastructure on which to perform research experiments, but one that also extends to wireless and peer-to-peer computing/storage. Because of the fortuitous timing with respect to our new CSE building, the infrastructure will have a deep impact on the department's educational, research, and social environment. It will support experimental research, multi-faculty and multi-disciplinary collaborative research and education. FWGrid will form a nexus for systems research within the department, and couple all of us to researchers on the campus, to SDSC and CalIT2, the metropolitan area, and wide area at high bandwidth. This will enable innumerable shared experiments and collaboration. In short, it will couple us to the future at high speed.

This award is funded under the American Recovery and Reinvestment Act of 2009 (Public Law 111-5).

Anthropogenic stresses to reefs coupled with climate change impacts are causing unprecedented declines in coral reefs globally yet automated technology to monitor the health of coral reefs at large temporal and spatial scales does not exist. This project will develop and deploy automated technology to monitor the health of coral reefs at large temporal and spatial scales. It will create a cyber-enabled Computer Vision Coral Ecology Portal that will allow upload, and expert classification remotely via an interactive web-based server paradigm where the external user need not manage the hardware or software. To achieve this goal new computer vision technology is needed. The effort has the potential to not only advance computer vision technology but also transform the way that coral reef ecology is performed. Fundamental computer vision problems in underwater imaging and illumination modeling, registration of RGB and fluorescence images from a moving camera rig will be addressed. Progressive improvement in the computer vision system combined with new data collected with these enabling technologies, will allow for transformational ecological outcomes including: 1) creation of baselines of coral reef health at diverse locations essential for understanding impacts of climate change; 2) development of rapid methods for quantitative surveying in support of ecology research or management; 3) estimates of growth, mortality, and recruitment rates and competitive abilities of the key coral reef species and the changes associated with warming and acidification; 4) improved knowledge of deep coral reefs for which very little is known; 5) rapid and spatially extensive in situ classification in support work by our collaborators on using airborne and satellite remote sensing and 6) creation of novel methods for multi-spectral image acquisition and processing to improve cyber-based ecological classification of coral reef communities.

The project includes participation and education for undergraduate students, graduate students and postdoctoral researchers. The project also includes an interdisciplinary education and out-reach program to involve high school students and teachers, using an innovative web-based "Coral Quiz" that will communicate details of ecology, computer science and the biology of coral reefs.