Perry Cook - US grants

This is the first year of funding of a 4 year continuing award. This project will investigate new approaches to the classification, analysis, and resynthesis of sound. The research plan is composed of three elements: The majority of the effort will focus on the development of algorithms and techniques for sound parameterization and synthesis. The PI will explore ways to sort and match sounds with algorithms, voice algorithm parameters, and test human subjects to determine success of synthesis. He will develop classification and analysis software to automate existing algorithms, eliminating the human guidance andjudgment currently required. He will develop new algorithms for classes of sounds that do not yield to existing parametric synthesis algorithms. He will also work on novel interfaces for controlling sound, and create systems that can take advantage of flexible sound synthesis and control. By the end of the project, machine algorithms will have been developed that can take certain (large) classes of sounds as input, select the appropriate synthesis algorithm, extract parameters for resynthesis, and predict human responses as to the quality of the synthesized sound; simple systems will have been constructed which demonstrate the utility of the synthesis algorithms. This research will yield new tools at both the input and output ends of the interactive sound chain to benefit diverse applications such as portable computers, virtual reality, augmented reality, awareness systems, teleconferencing and remote collaboration applications.

Storage capacity and data volume have been doubling every 18 months during the past
two decades. A key challenging issue in building next-generation storage systems is to
manage massive amounts of feature-rich (non-text) data, which has dominated the
increasing volume of digital information. Comparing noisy, feature-rich data requires fast similarity match instead of exact match, and thus exploring such data requires similarity
search instead of exact search. Current file systems are designed for named text files;
they do not have mechanisms to manage feature-rich data. To date, there is no practical
storage system with the ability to do similarity search for noisy, high-dimensional data
and there is no index engine design for efficient similarity search. This research
addresses this problem by studying how to design and implement a content-addressable
and -searchable storage (CASS) system to manage and explore diverse feature-rich data.
The system includes a built-in similarity search engine for general-purpose, noisy, highdimensional
metadata using compact data structures and novel indexing methods. The
research will also develop segmentation methods and feature extraction methods for
audio, image and genomic data, and develop similarity search benchmarks and to
evaluate the CASS system.

This research will advance knowledge and understanding in the area of storage system
designs such as data structures, mechanisms, and APIs for managing, searching and
exploring noisy, high-dimensional feature-rich data. The research will accelerate the
development of next-generation storage systems which will revolutionize how to access,
search, explore and manage massive amounts of feature-rich data in many disciplines.

The New Curriculum to Teach Computer Science Principles to Students in Digital Media Arts at California Institute of the Arts (CalArts) is developing a replicable model for teaching computer science to undergraduate students in arts schools, art departments, and arts programs. A two-semester course sequence, Applied Introduction to Programming and Algorithms, has been designed and is being offered as part of the core Media Arts curriculum. Using powerful, real-time, open source programming languages, ChucK and Processing, Bachelor of Fine Arts (BFA) students with little or no computer science background are acquiring foundational programming skills that are immediately applied to their digital arts practice. Students generate code for real-time music compositions and live image synthesis/processing, creating technology-driven art while simultaneously gaining proficiencies in core computer science concepts and Digital Signal Processing (DSP). The project's approach is to teach computer science in a non-traditional computer science context. By providing skills and tools in programming, networking, and basic robotic control through a course designed specifically for artists as a means of furthering their creative work, a model is created for providing computer science curricula specifically, and STEM curricula more generally, to students from a diverse background and degree trajectory. All course material, code examples, syllabus and assignments are being made publicly available online.