Rick Dale - US grants

Traditional theories in cognitive science often see mental processing and motor execution as separate domains of psychological investigation. In opposition to this, the past half-century has seen a resurgence of continuous and dynamical approaches to the mind, arguing for a smooth transition from cognitive into motor subsystems. This predicts that the dynamic characteristics of observable action will directly reflect underlying thought processes. In support of this, much recent research has shown that continuous tracking of motor movement can provide rich information about the mental processing giving rise to it.

With support of the National Science Foundation, Drs. Rick Dale and Michael Spivey will explore how action dynamics can shed light on human learning. By tracking computer-mouse trajectories, Dale and Spivey aim to uncover how action dynamics evolve as people learn. Several multi-choice, computer-based experiments will employ tasks of varying complexity, from simple associative learning (e.g., pairing symbols) to acquiring knowledge from text. Such measures would supplement traditional performance variables based on discrete answers. These studies can also provide a further understanding of the intimate connection between thought and action. In addition, the work can provide insights about how best to adapt computer systems for education: Through automatic analysis of computer-mouse movements, these adapted systems could detect such states as lack of learner confidence or deep understanding, and could appropriately respond to these potential deficits in real time, and thereby enhance learning.

When two humans interact, they often mirror each other's behavior. While discussing a work of art, their eye movements and gestures may become similar or even synchronize. When conversing face to face, they may adjust to facial expressions and gestures of their conversational partner. In addition, they may carefully adapt their sentences to the shared world that each can see. Therefore when two people interact, to some extent they become one "coupled" system. Like a couple dancing, they adjust their behavior in response to their partner. This research will explore how and when two people can be conceived as one dynamic, coupled system. An interdisciplinary team, consisting of two psychologists, a computer scientist, and a philosopher, will develop experiments and computer simulations to find new ways of understanding this "mirroring" and "adapting" that people seem to do while communicating.

Theories of human communication often emphasize strong coupling or, as a default, its absence. This work will supplement such theories by seeking a systematic middle ground based in cognitive mechanisms. How does facilitating memory and attention foster coupling between speakers? Humans can be made to adapt effectively to each other if the conditions are just right, and less often otherwise. The findings from this research could suggest ways of redesigning task environments to facilitate communication between partners. These tasks are common in everyday life, from sharing a computer screen during interaction, navigation, and even educational contexts in which tutors and students discuss visual or other information. The project could help two people couple more strongly, thus helping task partners avoid vagueness and ambiguity, and enhance the efficiency of their communication.

Communication is an act of coordination. Conversants must coordinate their mental activities to achieve the common ground required for effective communication. But how is this coordination achieved? This project tests the hypothesis that coordination of body movements and eye movements helps minds work together. The objective is to evaluate whether coordinated movement that occurs during conversation embodies the coordinated mental activities that make communication possible. The research employs a highly innovative approach to understanding these processes of mental and bodily coordination. The approach capitalizes on methods derived from nonlinear dynamical systems theory, a sub-domain of complexity science. The investigators will record patterns of body movements, such as gestures and postures, and patterns of eye movements. They will then apply time series analyses designed to capture the complexity of nonlinear dynamical systems to quantify the coordination of body and eye movements between people who are conversing.

Research on interpersonal coordination during conversation is a theoretically important and far-reaching issue because it serves as a hub topic for interdisciplinary studies of cognition, perception-action, language and communication, social psychology, and neuroscience. The proposed research offers innovative methods for studying joint action and strategies for understanding the relation between coordinated movement and coordinated cognition during joint action. These new measures of coordination provide quantitative tools for studying the complex and covert processes of conversational interaction. As part of this project, the investigators will establish a freely available, online, Joint Action Research Database (JARD). This will enhance the research infrastructure in this interdisciplinary field by serving as a repository for eye and body movement time series and speech data collected in this project and from other interested labs. In parallel with JARD the researchers will make available the software analysis tools appropriate for studying the complex dynamics of interpersonal coordination.

This INSPIRE award is partially funded by the Perception, Action, and Cognition Program in the Division of Behavioral and Cognitive Sciences in the Directorate for Social, Behavioral, and Economic Sciences and the Mathematical Biology Program in the Division of Mathematical Sciences in the Directorate for Mathematical and Physical Sciences.

This work explores the role of selection and adaptation in two radically different domains, 1) molecules and 2) languages. Consider, for example, human languages. The 6,000-7,000 languages spoken by people display a dazzling variety of sounds, words, and grammatical forms. This diversity is typically explained as a product of random drift: As a single population splits and drifts apart, the accumulation of small random changes eventually produces mutually unintelligible languages. An alternative is that some of the variation we see among human languages is due to selection. In this account, languages adapt to some extent to the different social and ecological environments in which they are spoken. Similarly, researchers have only recently considered the role of selection and adaptation in the study of prions (self-replicating proteins)and how they propagate within and across generations of cells.

Though biological structures and human languages are radically different domains, they share properties that suggest they may be described by a common mathematical framework. Specifically, (i) they are both epigenetically inherited, (ii) they both capitalize upon a pre-existing biological substrate, and (iii) they both propagate in a system of agents (cells, people). A highly interdisciplinary team of cognitive scientists, linguists, biologists, and mathematicians seek to connect and inform these domains by using mathematical models and large-scale behavioral experiments to understand the selective processes. They will assess convergent tests of the idea that selection acts as an organizing principle of systems at different scales. This work has important implications for issues such as the role of environmental context in the spread of structures such as prions or linguistic elements. For example, results could help explain how a structure newly infects populations, such as when a new word "invades" a linguistic environment, or when a prion structure successfully propagates and infects cells in its environment. The interdisciplinary nature of this award will provide a unique training experience for graduate students and will include outreach efforts to local schools.

People need to coordinate their actions in order to carry out a wide range of daily activities. For example, cooking a meal together, moving a large couch, and working as a team in sport or business, all require coordination among the participants. Despite evidence that collaborating partners in these joint tasks adapt their behavior to one another in real time, it is unclear what features of coordination lead to the best outcomes. The aim of this project is to develop a framework for predicting optimal coordination among people across diverse settings and roles. Using a combination of empirical studies and dynamical computational modeling, the investigators will unpack issues surrounding the roles of dialogue, perspective-taking, and collaboration in successfully performing joint tasks. Understanding the principles that underlie successful coordination among people engaged in a joint task has potential commercial and societal impact in a wide variety of settings, including health care, education, aviation, military operations, and search-and-rescue scenarios. The project will also contribute to the development of an interdisciplinary workforce by increasing opportunities to engage undergraduate and graduate students in cognitive science research at two academic institutions.

A prominent view of how interpersonal coordination influences task performance is that when task partners align or match their behavior, their joint task performance improves. Alignment of, for example, pronunciation, word choices, sentence structure, or movement has been documented in tasks that require partners to monitor each other’s perspective. However, it is unknown how different tasks may increase or decrease alignment for successful performance. In this project, the investigators will manipulate a number of task features: the goals of the task, the symmetry of the partners’ roles, and the number of perspectives that partners have to adopt. Eye movement measurements and dialogue transcripts will be analyzed in order to quantify how aligned partners are in their attention and language use. The aim is to evaluate and model how task performance is predicted by task features and by the degree of interpersonal coordination. For example, interpersonal alignment might increase and be more useful in tasks that require partners to monitor one another closely (e.g., when partners have asymmetrical roles because they hold distinct information). In tasks that require less monitoring, excessive alignment might be detrimental to task outcomes. A dynamical model of collaborative performance, which has parameters for task constraints and interpersonal coordination, will be evaluated against the experimental data and will be used to generate predictions about performance in new tasks.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

This award was provided as part of NSF's Social, Behavioral and Economic Sciences Postdoctoral Research Fellowships (SPRF) program. The goal of the SPRF program is to prepare promising, early career doctoral-level scientists for scientific careers in academia, industry or private sector, and government. SPRF awards involve two years of training under the sponsorship of established scientists and encourage Postdoctoral Fellows to perform independent research. NSF seeks to promote the participation of scientists from all segments of the scientific community, including those from underrepresented groups, in its research programs and activities; the postdoctoral period is considered to be an important level of professional development in attaining this goal. Each Postdoctoral Fellow must address important scientific questions that advance their respective disciplinary fields. Under the sponsorship of Dr. Carolyn Parkinson at the University of California, Los Angeles, this postdoctoral fellowship award supports an early career scientist investigating how mental representation of social agents affect decision processes involving these others. Defined as decisions that have direct or indirect social consequences, social decision-making has received much scientific attention over the past two decades. However, existing social decision-making research is impoverished insofar that very little is known about how the features of social targets implicated in social decision-making influence decision processes. Humans represent a wealth of information about others that profoundly shapes thought and behavior, yet it is unknown how these representations—dynamic mental models of others—impact decision preferences. Indeed, classic work from social cognitive psychology and neuroscience indicates that representations guide behavior in context, suggesting that mental representations likely play an important role in shaping social decision preferences. The current study aims to use computational methods to analyze functional magnetic resonance images and text data to (i) probe mental representations of common social decision-making targets (parents, friends) and (ii) relate structural features of these representations to social decision behavior. This project stands to make important theoretical contributions towards social decision-making research and contribute to ongoing efforts to build unifying and generalizable models of social decision-making. Notably, because social decision-making behavior has widespread impacts—ranging from individual well-being to aggregate societal phenomena—this project could inform future efforts to promote individually and societally adaptive social decision behavior.

This project will employ eminent computational methodologies to test how mental representations of real-life social partners shape social decision preferences as a function of the motivational goals and needs fulfilled by said agents. By combining multivariate pattern analysis of functional magnetic resonance imaging (fMRI) data with natural language processing (NLP) and computational models of behavior, this project will comprehensively measure mental representations of everyday social partners and identify their relationship to social decision-making preferences. Concretely, this will involving (i) probing the neural representations of different kinds of social agents using fMRI, (ii) determining how neural representational overlap shapes social decision preferences, and (iii) use NLP tools on written content of social agents to help identify the representational content that drives differences in social decision preferences across social partners. This research will hopefully help lay the groundwork for the gradual establishment and refinement of unifying quantitative models of social decision-making, aiding both basic and applied scientific endeavors in the future.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.