Ivon M. Arroyo, Ed.D. - US grants

The primary objective of this project is to develop new methods for optimizing an automated pedagogical agent to improve its teaching efficiency through customization to individual students based on information about their responses to individual problems, student individual differences such as level of cognitive development, spatial ability, memory retrieval speed, long-term retention, effectiveness of alternative teaching strategies (such as visual vs. computational solution strategies), and degree of engagement with the tutor. An emphasis will be placed on using machine learning and computational optimization methods to automate the process of developing efficient Intelligent Tutoring Systems (ITS) for new subject domains. The approach is threefold.
First, a methodology based on hierarchical graphical models and machine learning will be developed and evaluated for automating the creation of student models with rich representations of student state based on data collected from populations of students over multiple tutoring episodes. Second, methods will be developed and evaluated for deriving pedagogical decision strategies that are effective and efficient not just over the short-term (from one math problem to the next one), but over the long-term where retention over a period of at least one month is the objective. Third, a systematic study will be conducted of the role that known and powerful latent and instructional variables can have on performance through their inclusion in student models. Research in cognitive and educational psychology clearly shows the critical role that latent variables such as short-term memory and engagement play in learning, and that instructional variables such as over-learning and review, and massed and distributed practice have on the rate at which material is learned. The investigators jointly have strengths in the areas of intelligent tutoring, machine learning and optimization, and cognitive, mathematical and educational psychology, strengths that are needed in order to make the synergistic advances that are being proposed. Our preliminary simulations and classroom experiments suggest that we can significantly reduce the time it takes students to learn new material based on improved pedagogical decisions.
For intellectual merit, he proposed research should advance fundamental knowledge of the learning and teaching of basic mathematics and more advanced algebra and geometry. It should add to the set of growing statistical and computational techniques that are available to estimate the complex hidden hierarchical structures that govern human behavior. The research should also significantly broaden the capabilities of machine learning systems by addressing learning scenarios that are grounded on the real and challenging problem of mathematics education than the abstract scenarios typically studied at present. For broader impact, this foundational educational research will lead to the broadening of participation of underrepresented groups, especially women, in a variety of science, technology, engineering and mathematics (STEM) disciplines. It will advance discovery and understanding of learning and engagement as predictors of individual differences in learning and will result in intelligent tutors that are more sensitive to individual differences. It will unveil the extent to which students of different genders and cognitive abilities learn more efficiently with different forms of teaching. This research will benefit society as machine learning methods, which provide a core technology for building complex systems, will be applicable to a variety of teaching systems.

One promise of the Internet is educational "mass customization." Thanks to the National Science Digital Library (NSDL; http://nsdl.org), the opportunity exists to filter millions of resources and customize them for individual learners. This NSDL targeted research project is organizing digital libraries by dimensions that are important to teachers and learners--specifically, cognitive characteristics (cognitive development, spatial and math retrieval skills, reading level), affective characteristics (self-efficacy, motivation, beliefs/attitudes toward the subject), and social characteristics (gender, main language, ethnicity). The investigators hypothesize that customization of resources will result in visitors spending more time in NSDL and students achieving more in-depth learning.

As a testbed, the investigators are creating a customized learning environment, "Customized MathForum," within the Math Forum Digital Library (MFDL; http://mathforum.org), which is one of the most popular instructional digital libraries and has one of the largest communities of users (over a million individuals). The investigators are indexing the digital library according to cognitive, affective, and social dimensions and are evaluating whether such indexing helps stakeholders (teachers, students, and contributors/authors) find effective and efficient material and whether such indexing results in more effective learning than when resources are chosen randomly. Project activities include:

* designing a customized learning environment for middle school and high school teachers and students within MFDL;
* generating a portal to a special library of 750 arithmetic and geometry problems individualized for specific cognitive and behavioral skills;
* developing smart search tools and intelligent agents that will search the digital library for appropriate resources;
* integrating an enhanced metadata system in MFDL along dimensions that are important to teachers and learners--e.g., relation to state educational standards, and cognitive, affective, and social characteristics;
* evaluating the impact of providing customized problems for students and teachers; and
* disseminating tools for customized services to other digital library service providers.

Though described in terms of MFDL, this research is general and the methodology is applicable to many NSDL collections.

This project builds on tools and technologies that have evolved from several NSF-supported projects in three domains: intelligent tutoring systems, digital libraries, and instructional networks. The research directly addresses computational issues (advances in machine agents in distributed environments and the integration of intelligent tutors and digital libraries) and cognitive and affective issues (human learning characteristics and student models that improve online instruction). The research should result in sensitive instruction that is responsive to individual differences, especially among underrepresented minorities and women, and should unveil the extent to which students of different cognitive abilities learn with different forms of teaching.

Intellectual Merit. This project analyzes whether specific software interventions produce motivational and mathematics achievement gains for girls within real K12-level educational settings, at two crucial moments of girls? development of attitudes towards STEM, grades 5-6 and 10-11. Randomized controlled evaluations are used to analyze the impact of strategies that improve girls' and minorities' performance in mathematics and motivation to pursue mathematics coursework. The study uncovers empirically-supported guidelines for the design of math software that benefit girls' and minorities' motivation and achievement in mathematics. This research furthers research into computation techniques (intelligent agents/learning companions, user modeling and tools), educational psychology (rigorous analysis of the impact of interventions on motivation and self-efficacy, student characteristics and on-line instruction) and developmental psychology (gender differences across several ages).

Broader Impact. The project provides Internet environments for students in poorly performing school districts and those who might be home-schooled as a result of a disability. It advances the understanding of students who find potential failure in math to be threatening (most often, females and students from traditionally under represented minority groups), promoting interest in mathematics among generally underrepresented students. It improves the quality of on-line courseware and reduces the barrier for entry to STEM. The project will produce instruction that is responsive to individuals, lays the groundwork for more innovative curricula and creates new understandings of the complexities of taught materials.

Emotion and motivation are fundamental to learning; students with high intrinsic motivation often outperform students with low motivation. Yet affect and emotion are often ignored or marginalized with respect to classroom practice. This project will help redress the emotion versus cognition imbalance. The researchers will develop Affective Learning Companions, real-time computational agents that infer emotions and leverage this knowledge to increase student performance. The goal is to determine the affective state of a student, at any point in time, and to provide appropriate support to improve student learning in the long term. Emotion recognition methods include using hardware sensors and machine learning software to identify a student's state. Five independent affective variables are targeted (frustration, motivation, self-confidence, boredom and fatigue) within a research platform consisting of four sensors (skin conductance glove, pressure mouse, face recognition camera and posture sensing devices). Emotion feedback methods include using a variety of interventions (encouraging comments, graphics of past performance) varied according to type (explanation, hints, worked examples) and timing (immediately following an answer, after some elapsed time). The interventions will be evaluated as to which best increase performance and in which contexts. Machine learning optimization algorithms search for policies that further engage individual students who are involved in different affective and cognitive states. Animated agents are enhanced with appropriate gestures and empathetic feedback in relation to student achievement level and task complexity. Approximately 500 ethnically and economically diverse students in Massachusetts and Arizona will participate.

The broader impact of this research is its potential for developing computer-based tutors that better address student diversity, including underrepresented minorities and disabled students. The solution proposed here provides alternative representations of scientific content, alternative paths through material and alternative means of interaction; thus, potentially leading to highly individualized science learning. Further, the project has the potential to advance our understanding of emotion as a predictor of individual differences in learning, unveiling the extent to which emotion, cognitive ability and gender impact different forms of learning.

The Preparing for College: Using Technology to Support Achievement for Students with Learning Disabilities in Mathematics project will advance knowledge about improved learning, motivation and achievement of undergraduate students with mathematics learning disabilities when using digital interventions. This demonstration research project will result in pilot-tested interventions, which will serve as the basis for more advanced studies of how students with learning disabilities learn math in a cyber-enabled environment.

The primary intervention tool for this project is a cyber-enabled mathematics tutor, "Wayang Outpost," that helps students solve challenging test problems, teaches explicitly, and uses visual representations to help students learn. Modifications to the tutoring system will be piloted that address cognitive, metacognitive, and affective dimensions of the system, corresponding to the representation/expression and engagement strands underlying universal design for learning.

This demonstration research pilot project will include testing a series of interventions with undergraduates in developmental mathematics classes at the University of Massachusetts and at Arizona State University. The project will target approximately 200 undergraduates, with and without learning disabilities, across four classrooms in experimental and control conditions.

The purpose of this proposal is to explore the extent to which timely emotional, cognitive, and metacognitive interventions in tutoring software will have positive effects on students' emotions, attitudes, and achievements in mathematics. The intelligent tutor, Wayang Outpost, a high school mathematics tutoring system, is being enhanced to leverage automatic detection of emotions to guide cognitive, metacognitive, and affective forms of learning support.

The PIs are conducting a set of experiments to understand the interplay of observed emotional states, emotion assessments, student behavior within tutors, and student achievement. In particular, the experiments are testing the effects of the tutoring system when it assesses the emotions of a student and then responds with instructional support appropriate to that student's affect and content knowledge.

In this project, an interdisciplinary team of researchers in learning technologies and mathematics education are working together to investigate issues related to motivation in learning mathematics. They are taking the results of lab-based studies into classrooms. The novel technology and approaches developed in the lab were tested with a small population of learners; in their classroom-based investigations, they are testing feasibility of the approach with a more diverse population and refining the technology for use in a broad range of classroom learning environments. This translational research project will not only make significant contributions to the field of learning technologies, but will also contribute to our understanding of issues related to motivation in mathematics learning.

This PFI: AIR Technology Translation (TT) project addresses the high failure rate of K12 students to learn mathematics. This project focuses on technology translation of an intelligent online tutor, named MathSpring, which is important because it provides adaptive and personalized responses to students and teaches by matching the learning needs of individual students with effective teaching approaches. It applies theoretical understanding of cognitive, metacognitive and affective student characteristics to each tutor response. The MathSpring Tutor is also important because no online tutor today responds by analyzing both student knowledge and behavior. This PFI:AIR-TT project will result in a scale-up of the MathSpring Tutor and provide advantages in the marketplace by capitalizing on the general appeal of animation and humanoid characters that talk to students about the importance of perseverance and effort. The project will also provide low-cost, quality solutions for a wide range of students, adaptive tutoring based on student models, just-in-time verbal and animated interactions designed to move students away from boredom or disengagement, and the capability to select from among potentially 700 problems in the system.

These features of the MathSpring tutor provide improved performance, efficiency and efficacy when compared to classroom teaching or to the leading competing technology, primarily drill and practice problems, videos of lectures or games in this market space. The potential economic impact of translating this technology to the market place will positively contribute to the growth rate of eLearning within the next 5 years and to the U.S. competitiveness in the eLearning domain. Since the annual U.S. education expenditure for K-12 is approximately $625 billion, a large potential exists for making both a commercial and social impact in this space. Potential outcomes include: personalized tutors that guide students into their own zone or state of ?flow?; identification of target educational markets; and reaching any student with access to a computer and an Internet connection.

This PFI project addresses the following technology gaps as the software is translated from research discovery toward commercial application: identification of tutor responses that are effective for students in distress (e.g., bored, unmotivated); building sufficient content so the tutor can be used through an entire semester in Grades 5-9; and providing tools that enable teachers to select math problem based on the Common Core curriculum. The project work also includes hardening the tutor, porting it to two platforms (e.g., Android, IOS) and identifying consortia of schools (e.g., linked by geography, or pedagogy) for long-term partnerships.

Personnel involved in this project, e.g., graduate students and programmers, will receive innovation and technology translation experiences through efforts to identify paths through the idiosyncratic school procurement process and the communication of the efficacy studies arising from credible evaluation of MathSpring. The project engages CarneyLabs to guide commercial aspects of the translation and Virginia Advanced Studies Strategies (VASS), a non-profit company that works with the Virginia Department of Education (DoED), to provide a test environment in this technology translation effort from research discovery toward commercial reality.

The Cyberlearning and Future Learning Technologies Program funds efforts that support envisioning the future of learning technologies and advance what we know about how people learn in technology-rich environments. Integration (INT) projects refine and study emerging genres of learning technologies that have already undergone several years of iterative refinement in the context of rigorous research on how people learn with such technologies; INT projects contribute to our understanding of how the prototype tools might generalize to a larger category of learning technologies. This INT project integrates prior work from two well-developed NSF-sponsored projects on (i) advanced computer vision and (ii) affect detection in intelligent tutoring systems. The latter work in particular developed instruments to detect student emotion (interest, confusion, frustration and boredom) and showed that when a computer tutor responded to negative student affect, learning performance improved. The current project will expand this focus beyond emotion to attempt to also detect persistence, self-efficacy, and the trait called 'grit.' The project will measure the impact of these constructs on student learning and explore whether the grit trait (a persistent tendency towards sustained initiative and interest) can be improved and whether and how it depends on other recently experienced emotions. The technological innovation enabling this research into the genre of broadly affectively aware instruction is Smartutors, a tool that uses advanced computer vision techniques to view a student's gaze, hand gestures, head, and face to increase the "bandwidth" for automatically detecting their affect. One goal is to reorient students to more productive attitudes once waning attention is recognized.

This research team brings together a unique blend of leading interdisciplinary researchers in computer vision; adaptive education technology and computer science; mathematics education; learning companions; and meta-cognition, emotion, self-efficacy and motivation. Nine experiments will provide valuable data to extend and validate existing models of grit and emotion. In particular, the team will gather fine-grained data on grit, assess the impact of tutor interventions in real-time, and contribute thereby to a theory of grit. Visual data of student behavior will be integrated with advanced analytics of log data of students' actions based on the behavior of over 10,000 prior students (e.g., hint requests, topic mastery) to provide individualized guidance and tutor responses in a timely fashion. This will allow the researchers to measure the impact of interventions on student performance and attitude, and it will uncover how grit levels relate to emotion and what impact emotions and grit combined have on overall student initiative. By identifying interventions that are sensitive to individual differences, this research will refine theories of motivation and emotion and will reveal principles about how to respond to student grit and affect, especially when attention and persistence begin to wane. To ensure classroom success, the PIs will evaluate Smartutors with 1,600 students and explore its transferability by testing it in a more difficult mathematics domain with older students.

The role of embodiment (e.g., the use of both fine-grained and gross motor gestures) in mathematics learning for young students is well-established. With the recent success of mobile games, there is more public awareness of augmented reality - a real-time view of a physical, real-world environment that has elements that are augmented by computer-generated sensory input such as sound, video, graphics or GPS data. This project brings together these concepts of augmented reality and embodied learning to develop a new innovation: mobile-phone-based augmented reality math activities. This research project investigates these activities in public elementary-schools, where students solve math related challenges, and are encouraged to find mathematics in the real objects of classrooms, gyms and playgrounds. Students carry a cell phone-based intelligent system (or a SmartWatch) that supports them as needed, giving hints and feedback as they play multiplayer team-based mathematics games that require them to measure, estimate, compute, discern and discard objects in their environments, both in their minds and using their bodies. This project advances scientific knowledge on how people learn with active physical activities, investigating the role of hand and full-body motions, as well as the role of team-based cooperation in the mathematics classrooms of our public schools.

This project involves a new genre of Cyberlearning Technologies called Multi-Modal Embodied Intelligent Learning and Tutoring Environments that supports learning through (1) Intelligent Tutoring Systems (ITS), which can trace students' knowledge and affect in real time, and personalize instruction through moment-by-moment tracking of students' mental states (i.e. knowledge and affective tracing); and, (2) Embodied Cognition and multi-modal interaction (physically acting on the environment in meaningful ways for more solid understanding and encoding of ideas). This project establishes a new genre of learning technologies that blends well with classroom culture, including hands-on manipulatives and educational games, while still retaining the benefits of intelligent tutoring systems, which consist of smart pedagogical decisions based on a moment-to-moment automatic assessment and understanding of the student at various levels. The project includes a series of experimental interventions that investigate the role of fine-grained gestures and gross motor actions during mathematical problem solving. Specifically, the project investigates (1) the impact of these problem-solving activities on cognitive and affective outcomes; (2) the role of personalized feedback and face-to-face social interactions; and, (3) the elements of the embodied activities that are most effective.

This NSF Convergence Accelerator Workshop will create smart and integrated platforms, devices and processes for education technology. Current educational technologies often do not sufficiently leverage basic research on learning, nor have a culture of continuous improvement, nor meet the needs of diverse learners, nor leverage the growing power of computers. One issue is the disconnected, fragmented, and often closed nature of different sectors: educators, parents, commercial ventures, not-for-profit organizations, stakeholders, researchers, and communities. A concerted effort among diverse stakeholders is needed now to create real and immediate solutions. Significant technology exists for digital instruction. Experts in academia (learning science, AI, human-computer interaction, education, psychology), industry and government will identify barriers and solutions to the delivery of high-quality online education; they will inform best practices in design, generate future development and testing, and leverage technology and new modes of platform design. This workshop will support communities to reason about fruitful near-term approaches for scaling up innovative pedagogies. We will increase the number of trials of new products; test more often and fail faster; identify promising interventions, and evaluate the conditions and circumstances that increase the probability of successful products. <br/>The scientific agenda will investigate and augment human learning at large scale in authentic education settings (online, hybrid, and on-the-job). The workshop will establish a framework for new AI, learning science, and education theory and technologies to understand, model, infer, and respond to learning. It will explore new theory, algorithms, big data, and systems that optimize every point in the education process to understand students, organize what they can learn, and optimize how they learn. The workshop will couple use-inspired AI research with foundational AI and learning science research in a virtuous cycle and forge new partnerships among diverse stakeholders as they bridge the divide with novel tools from engineers, makers, technologists, and designers. It will make a laser focus on projects that present well-defined deliverables within 3-years.<br/><br/>This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.