Andrew A. Ng - US grants

This project, trying to break the very high barrier to entry of mobile manipulation, aims at integrating in a robot platform methods drawn from all areas of AI, including machine learning, vision, navigation, manipulation, planning, reasoning, and speech/natural language processing (NLP). The project contemplates building a robot that can navigate home and office environments, that can pick up and interact with objects and tools, and that can intelligently converse with and help people in these environments. Over the long tern, a single robot will perform such tasks as:

-Fetching a book or a person from an office, in response to a verbal request.
-Tidying up a space after a party, including picking up and throwing away trash, and placing dirty dishes, and glasses in the dishwasher.
-Using multiple tools necessary to assemble, say, a bookshelf.
-Showing guest around an active research lab (where things change daily), answering questions, and keeping track of an entire group.

A robot capable of these tasks will revolutionize home and office automation and have important applications ranging from elderly care to machine shop assistants. To realize this vision, the PIs carry out an integrated research program on learning, manipulation, perception, spoken dialog, and reasoning, all in the context of applying them to STAIR (Stanford AI Robot). The proposed computing platform would provide a unified testbed for developing machine learning, vision, navigation, manipulation, planning, reasoning, and NLP.

Broader Impact: The application (robots performing tasks for home and office) itself exhibits broad impact. Additionally, the STAIR project will be used to train more graduate and undergraduate students. A course is being developed exposing students to participate in teams. Women and underrepresented students are actively being sought.

The impact of Big Data is all around us and is enabling a plethora of commercial services. Further it is establishing the fourth paradigm of scientific investigation where discovery is based on mining data rather than from theories verified by observation. Big Data has established a new discipline (Data Science) with vibrant research activities across several areas of computer science. This ?Rapid Python Deep Learning Infrastructure? (RaPyDLI) project advances Deep Learning (DL) which is a novel exciting artificial intelligence approach to Big Data problems, which also involves a sophisticated model and a corresponding ?big compute? needing high end supercomputer architectures. DL has already seen success in areas like speech recognition, drug discovery and computer vision where self-driving cars are an early target. DL uses a very general unbiased way of analyzing large data sets inspired by the brain as a set of connected neurons. As with the brain, the artificial neurons learn from experience corresponding to a ?training dataset? and the ?trained network? can be used to make decisions. Trained on voices, the DL network can enhance voice recognition and trained on images, the DL network can recognize objects in the image. A recent study by the Stanford participants in this project trained 10 billion connections on 10 million images to recognize objects in an image. This study involved a dataset that was approximately 0.1% the size of data ?learnt? by an adult human in their lifetime and one billionth of the total digital data stored in the world today. Note the 1.5 billion images uploaded to social media sites every day emphasize the staggering size of big data. The project aims to enhance by DL by allowing it to use large supercomputers efficiently and by providing a convenient DL computing environment that enables rapid prototyping i.e. interactive experimentation with new algorithms. This will enable DL to be applied to much larger datasets such as those ?seen? by a human in their lifetime. The RaPyDLI partnership of Indiana University, University of Tennessee, and Stanford enables this with expertise in parallel computing algorithms and run times, big data, clouds, and DL itself.
RaPyDLI will reach out to DL practitioners with workshops both to gather requirements for and feedback on its software. Further it will proactively reach out to under-represented communities with summer experiences and DL curriculum modules that include demonstrations built as ?Deep Learning as a Service?.
RaPyDLI will be built as a set of open source modules that can be accessed from a Python user interface but executed interoperably in a C/C++ or Java environment on the largest supercomputers or clouds with interactive analysis and visualization. RaPyDLI will support GPU accelerators and Intel Phi coprocessors and a broad range of storage approaches including files, NoSQL, HDFS and databases. RaPyDLI will include benchmarks as well as software and will offer a repository so users can contribute the high level code for a range of neural networks with benefits to research and education.