Robert Jacobs - US grants

This project is purchasing a network of six graphics workstations that, together with existing Macintosh and DEC computers, serves as the basis for new course developments in the area of computational chemistry within the undergraduate curriculum in the chemistry department and the Pharmacology Program of the biological sciences department. The new computers are being used in courses in Freshman chemistry, organic and advanced organic chemistry, biochemistry, polymer chemistry, computational chemistry, and upper-division pharmacology courses and are being used to support undergraduate research projects in the academic year and the summers. The capabilities of this new instrumentation makes it possible to communicate to students the reality and excitement of modern developments in chemistry and to involve these students in exercises and projects that will give them "hands-on" access to these computational techniques as an integral part of our undergraduate curriculum. Thus far, experimental courses in this area have been very enthusiastically accepted by students and give students a set of skills that serve them in graduate and professional schools and careers in the bulk chemical and pharmaceutical industries.

Nearly all of the research investigating the modular nature of the mind/brain in general, and the acquisition of functional specializations in particular, is behavioral or neuroscientific in character. The progress of this research effort has been impeded by a lack of computational studies that attempt to relate functional properties with underlying cognitive and neural mechanisms. Jacobs's research program has developed a family of computational architectures, referred to as mixtures-of- experts architectures, that acquire functional specializations by combining associative learning mechanisms and competitive learning mechanisms. Within the mixtures-of-experts computational framework, a surprising result is that the mechanisms underlying the acquisition of functional specializations and the mechanisms underlying the acquisition of modular integrations are identical. The goal of the proposed research is to develop these mechanisms for the purpose of acquiring modular integrations. Studies one through three investigate bootstrap learning; that is, the ability to use solutions to simpler tasks as "building blocks" for more difficult tasks. One aim of these studies is to investigate the role of context-dependent visual object representations in the acquisition of invariant visual representations. Studies four and five use the mixtures-of-experts architecture to investigate the aggregation of multiple expert opinions. In particular, they examine the issue of whether modular integrations during visual processing should be based on selection or on combination.

Both humans and non-human primates show remarkable learning abilities. However, these abilities are often limited to certain domains, developmental periods, or behavioral contexts. For example, nearly all humans acquire one or more complex linguistic systems-that is, languages -- but not all humans acquire complex musical systems. Similarly, non-human primates are exceptionally adept at learning to forage for and categorize different types of food, but are severely limited in acquiring complex communication systems. Also, both humans and non-human primates appear to learn best in several domains during early periods of development. Thus, learning is nearly always characterized by specializations, rather than by general-purpose mechanisms. Understanding the constraints on learning will contribute to basic research, by accounting for domain- and species-specializations, and to applied research, by refining our understanding of which domains, ages, and contexts are optimal for human
learners.

The goal of the present research project is to explore the ability of human adults, children, infants, and non-human primates (Tamarins) to learn rapid sequential events. A prime example of a rapid sequential event is language, in which sounds are combined to form words, and words are combined to form sentences. Recent findings have demonstrated that human adults and infants can rapidly extract and remember very detailed 'statistics' of linguistic input, such as the frequency and probability that one syllable will follow another. In our proposed research, we will employ miniature artificial 'languages' which simulate some of the structural properties of natural languages, but which can be built with equivalent structures across different domains (speech sequences, tone sequences, visual sequences, motor sequences). At issue is the facility humans and non-human primates show for the extraction of statistical structure from these different learning materials. Are they equally sensitive to the distributions of elements and higher order structure in the materials? Do learning abilities differ across learners of different ages and species, and across different structures and domains?

In addition to behavioral experiments with humans and non-human primates, a series of computational studies will allow us to investigate the formal properties of learning mechanisms, in order to ask what architectural and neural differences might underlie such differences in learning abilities. What kinds of computational architectures can learn the types of regularities and patterns that human infants learn? Is the inability of adults or non-human primates to learn some types of complex sequential events due to the absence of a learning device specialized for that domain, or could small differences in computation and/or memory lead to large differences in learning outcome?

The representations used by humans for motor control can be readily studied by imitation learning. Subjects imitate the movements of an instructor under different conditions. During these trials eye and hand movements are recorded. Preliminary results show that the representations of copied and copied from memory movements are different implying different internal systems.

DESCRIPTION (Adapted from applicant's abstract): The human visual system obtains information about object depth from a large number of distinct cues. A full understanding of visual depth perception requires an understanding of how information provided by these cues is combined by our visual systems. Although nearly all theories of visual depth perception use the concept of cue reliability, we lack a good understanding of what this concept means, of how observers can measure cue reliability, and of what observers can do once they have measured it. The proposed research program places much importance on the need to understand observers' estimates of cue reliabilities, and on the need to understand how observers use these reliabilities during visual reliability judgments, and the roles that these factors play in experience-dependent adaptation of visual depth perception. Two types of experience-dependent adaptation are considered. Cue combination learning refers to the adaptation of the integration process that combines depth estimates based on individual cues into a single, composite depth estimate. Cue recalibration refers to the adaptation of depth interpretations of individual visual cues, such as adaptation of depth-from-motion estimates or adaptation of depth-from-texture estimates. The research program hypothesizes that observers regard a depth cue as reliable when: (i) depth estimates based on that cue are less variable than estimates based on other cues; or when (ii) depth estimates based on that cue are positively correlated with estimates based on other cues. It also hypothesizes that observers adapt their depth perception strategies so as to: (i) rely more heavily on reliable cues during cue integration; and to (ii) recalibrate depth judgments based on unreliable cues so that they more closely match those based on reliable cues.

Marine organisms have proven to be a prolific source of novel, biologically active natural
products. A growing number of these are entering clinical trials and many more are in preclinical
development. In general, the production of these compounds through synthesis is not feasible on
a commercial scale and consequently, the supply of such compounds presents an escalating
problem.

This project is directed at the development of seco-pseudopterosins and related metabolites,
anti-inflammatory agents from the soft coral Pseudopterogorgia elisabethae.
These compounds have been shown to be anti-inflammatory and analgesic agents in mouse ear
models with potencies superior to that of pseudopterosins which are in clinical use, and existing
drugs such as indomethacin. A biotechnological production method of these compounds will be developed based on the cloning of a key biosynthetic enzyme. In addition to addressing the supply issue of these specific metabolites, completion of these goals will provide useful models for the development of a growing number of anti-inflammatory terpenes being discovered from marine invertebrates. Since all of the pseudopterosin used commercially is obtained through collections from nature, success with this project will provide a sustainable source of these valuable compounds and save delicate reef environments from the effects of large scale collecting.

The synthesis of the targeted anti-inflammatory agents will be approached as follows. Firstly, the first biosynthetic intermediate will be generated by sequencing the coral's diterpene cyclase and cloning this into E. coli. via the construction of a P. elisabethae cDNA library. The cyclase product will serve as a key intermediate in the syntheses and will be modified to the seco-pseudopterosin aglycone by chemical methods. This would then set the state for the ultimate glycosylation to complete the synthesis of the seco-pseudopterosins. ( This last step is not a goal of the present project.)

This research represents a collaboration between two labs with complementary expertise. R.
Kerr (PI, FAU) will conduct the natural products chemistry, enzymology and some of the molecular work
J. Lopez (Co-PI, HBOI) will compete the cDNA library construction and work with Kerr on the cloning
experiments.

The proposed research program consists of experimental and computational studies of human visual learning. The project focuses on the information processing mechanisms mediating the perceptual learning that underlies expertise in a variety of STEM fields, such as biology, astronomy, and geoscience. In particular, the investigators attempt to take advantage of insights from the field of Machine Learning (e.g., its formalisms for conceptualizing the properties of different learning environments, its powerful sets of statistical learning algorithms for each environment, and its numerous mathematical and empirical findings about the advantages and disadvantages of these algorithms). The studies look at learning performance on lower-level and higher-level discrimination tasks in four types of learning environments: supervised, unsupervised, semi-supervised, and reinforcement learning environments. The project also explores visual learning based on correlated perceptual signals in multisensory or multi-cue environments, such as when a person both sees and touches surfaces. The computational studies compare people's learning performances with the statistically optimal performances of "ideal learners", and also with the performances of on-line learning algorithms from the Machine Learning literature. A key hypothesis is that people can visually learn with "unlabeled" data items (i.e., items that are not labeled by an instructor as examples of a particular category of interest) by transferring knowledge gained with "labeled" data items or by transferring knowledge gained from other sensory modalities. The work has important implications for the design of STEM training environments.

Field experience is a fundamental part of the training of student geologists, but practical considerations limit the numbers of students who can take part in extensive field programs. Moreover, little is known about how novice geologists acquire the visual skills of experts, raising questions about how best to develop teaching interventions. The 5-year project investigates differences between expert and novice geoscientists in the field and in a virtual semi-immersive display environment. The research team is composed of scientists and educators with expertise in perceptual learning, geology and geophysics, the recording and analyzing of eye movements, and large-field-of-view image capture of natural environments. They hypothesize that there are large differences between the eye-movement sequences of experts and novices, and that novices will show improvement during a field trip. The researchers will study similar groups in a virtual environment, hoping to gain additional insight into learning through comparisons of the data collected in the two environments. Their ultimate goal is to design a virtual semi-immersive environment that replicates the salient aspects of the field learning experience.

People can perceive the shape of objects accurately and reliably but how this occurs is not yet understood. This ability may stem, at least in part, from our use of both visual information and haptic information (information obtained when an object is touched or grasped). Moreover, if we learn to recognize an object based on visual information, we can often recognize the same object when our eyes are closed but we are allowed to grasp it. Similarly, if we learn to recognize an object based on haptic information, we can often recognize the object when we see it but cannot touch it. In other words, we exhibit cross-modal transfer of object shape information. How does information from the eyes and hands link up in the brain to yield a coherent representation of object shape? Insights obtained from this research can contribute both to our understanding of how humans perceive object shape using vision and/or touch and to development of improved robotic and other artificial intelligence systems operating in multi-modal settings in industrial, medical, military, and other applications.

The present project develops a theory of visual-haptic object shape perception in which people's notions of object similarity are not based on sensory features but rather on latent or hidden variables that represent object parts and their spatial relations in an abstract, modality-independent format. Object representations are formalized using a probabilistic "shape grammar" with Bayesian inference used to infer grammar-based object representations when an object is viewed, when it is grasped, or both. The model is tested using data obtained from behavioral studies of visual, haptic, and visual-haptic object shape perception by humans. The investigators will explore the types of representational change that underlie the transition from perceptual novice to expert (e.g, radiologists) and will assess whether perceptual expertise is well characterized as category learning, grammar learning, both, or neither. The research program will also develop a large public database of code for re-creating both visual and haptic features of complex objects. This will allow other researchers to fabricate the objects using a 3D printer, enhancing complementarity and comparison across research sites. Finally, training undergraduate and graduate students in the emerging field of computational cognitive science will contribute to a new generation of multidisciplinary scientists working across traditional boundaries between cognitive science and computer science.

Understanding the cognitive and neural basis of human behavior is one of the most fundamental areas of scientific inquiry for the 21st Century. It will impact almost every facet of human existence, including commerce, education, health care, and national security, as well as basic science. This National Science Foundation Research Traineeship (NRT) award prepares Ph.D. students at the University of Rochester to advance discoveries at the intersection of computer science, brain and cognitive sciences, and neuroscience. Trainees will be prepared to harness the burgeoning power of data science to make novel inroads into understanding the neural foundations of human behavior. Trainees will learn to be equally comfortable applying these skills in industrial and academic settings. By emphasizing both practical applications and basic science, this program will prepare trainees to develop research solutions relevant to today?s societal needs as well as develop research approaches of critical importance to future needs.

Focusing on understanding the nature of intelligence, this program will provide students with skills to blend expertise in data science and computer science with a deep understanding of experimental approaches to collecting and analyzing neural and behavioral data. The program will use theories and methods from data science including machine learning and statistics to provide students with a foundation for theory development, computational modeling, and data analysis. This foundation will serve as a conceptual and methodological framework unifying their studies of artificial and biological intelligence. The hands-on, project-oriented nature of this program will provide students with the capabilities needed to conceptualize, design, and implement large-scale research projects from beginning to end. This traineeship provides a novel model for structuring interdisciplinary education, based on a modular cross-training course followed by an interdisciplinary practicum course, which can be replicated in many fields and universities.

This project will identify ways to improve education and training in the geosciences, building on fundamental research in cognitive science. Geoscience is a STEM discipline that is of growing importance to several national and global issues, including climate change, energy resources, and understanding earthquake activity. Expertise in geoscience depends heavily upon unconscious perceptual skills that are difficult or impossible to impart through traditional classroom education. Consequently there is a great need to improve or develop new methods for perceptual training (i.e., training students to visually detect, identify, and interpret geologic processes) and to study how educational practices in this area may be optimized. This project will apply a cognitive science approach to meet this goal, by conducting behavioral experiments involving geoscience students, and developing computational models of human cognition. This project is supported by NSF's EHR Core Research (ECR) program. The ECR program emphasizes fundamental STEM education research that generates foundational knowledge in the field.

The proposed research will focus on the role of visual working memory (VWM) in perceptual learning and perceptual expertise in the geosciences. VWM is a cognitive system that is critical to the geosciences. For example, when a geologist views or studies a landscape, he or she acquires visual information from multiple points in the scene by making numerous eye movements. The geologist can then (unconsciously) use his or her VWM to integrate information across eye movements to develop a coherent representation of the landscape. While there is much existing research on VWM, very little is known about the relationship between visual working memory on the one hand, and fundamental changes in perceptual ability in specific domains such as geoscience on the other hand. The proposed research will build upon, and extend a recently developed model of human perception that has the potential to connect these two domains. This model is based upon information theory, or the mathematical study of how physical systems can efficiently communicate information. This model suggests that training and expertise fundamentally change a person's perceptual system in a way that leads to more efficient use of cognitive resources. In order to test this model, a series of behavioral experiments will be conducted involving novice and experienced students in the geosciences. These experiments and computational modeling efforts will contribute fundamental knowledge that can be used to improve geoscience and education.

Similarity is fundamental to nearly all aspects of human cognition. Perception uses similarity: when viewing a person's face, we (unconsciously) calculate its similarity to the faces of people we know in order to recognize who we are looking at. Categorization uses similarity: when judging whether a building was designed by the architect Frank Lloyd Wright, we calculate its similarity to buildings known to have been designed by Wright in order to make our best estimate. Reasoning and problem solving use similarity: when attempting to solve a calculus problem, we calculate its similarity to previous problems that we have encountered in order to determine a good solution strategy. However, how people calculate the similarity of two items is not yet understood. Which features of items do people use to calculate similarity? And how are the feature values of items compared in order to calculate similarity? This research project will use human experimentation and computational modeling to address these questions when items are viewed or grasped. A long-term benefit of the project is that a greater understanding of people's perceptual similarity judgments will provide a foundation for understanding how people calculate and use similarity in other areas of cognition. While conducting the research, undergraduate and graduate students will be mentored in the cross-disciplinary approach embodied in our investigation through participation in both experimental and computational aspects of the research project.

This project focuses on developing a new empirical and theoretical foundation for understanding people's notions of similarity, particularly in the domain of perceptual similarity. The field of cognitive science is well aware that understanding similarity is essential to understanding human cognition. Despite this, the primary motivation for this project is the belief that, to date, cognitive science's approach to the study of similarity judgments is much too simple---the restricted class of similarity metrics considered by cognitive scientists is unlikely to scale to large, realistic settings. The primary hypothesis of this project is that the field of machine learning---especially the study of metric learning---can supply cognitive science with a rich array of complex and sophisticated models, models that will be necessary to accurately characterize people's similarity notions in large, realistic domains. Machine learning has pioneered the study of mathematically rigorous linear and nonlinear similarity metrics. We believe that the time is ripe for the field of cognitive science to make use of machine learning's recent advances. Machine learning's metric learning framework extends and elaborates the cognitive science approach in principled and innovative new directions. Indeed, this framework presents an unparalleled opportunity for cognitive science with the potential for transforming this field. Using the empirical and theoretical findings from machine learning, cognitive scientists can now begin to explore human notions of similarity in more complex and sophisticated ways---and in more realistic domains---than has ever been possible. We regard the research project as an early step for cognitive science towards a more sophisticated understanding of people's notions of similarity. Because the project cannot study similarity in all domains of human cognition, it concentrates on perception. Future work will need to develop further the models proposed and evaluated here. If successful, the program will establish an empirical and theoretical foundation that can subsequently be extended to many other areas of human cognition.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.