Anirban Mandal, Ph.D. - US grants

The backbone of IT infrastructure is evolving towards a service-oriented model, in which distributed resources (software services, virtualized hardware infrastructure, data repositories, sensors, and network overlays) can be composed as a customized IT service on demand. In particular, cloud computing infrastructure services manage a shared ``cloud'' of servers as a unified hosting substrate for diverse scientific applications, using various technologies to provision servers and orchestrate their operation. At the same time, high-speed networks increasingly offer dynamic provisioning services at multiple layers. Network-connected clouds offer a general, flexible, and powerful model to scale up computing power for data-intensive science applications running at multiple cloud sites. The software produced in this project offers interfaces and control policies for application-driven orchestration of federated clouds interconnected by advanced networks.

The project develops software to link cloud computing clusters to other cyberinfrastructure resources through dynamically provisioned networks. A principal focus is to extend popular cloud infrastructure software with hooks to connect provisioned machine instances running in the cloud to external resources through dynamic circuit networks. The project enables cloud applications to dynamically request compute resources at multiple points in the network, together with bandwidth-provisioned network pipes to interconnect them and link them with other services and data repositories. The orchestration framework is based on the Open Resource Control Architecture (ORCA), an extensible platform for dynamic leasing of resources in a shared networked infrastructure. The resource allocation policies are enabled through semantic resource descriptions and extended intelligent SPARQL queries. Driving applications for this project are MotifNetwork, IMG/JGI and Supernova Factory and sensor networks linked to cloud resources (CASA). Development and demonstrations leverage the Breakable Experimental Network (BEN, a multi-layer optical network testbed located in North Carolina), NLR and ESNet.

Computational science today depends on many complex, data-intensive applications operating on datasets that originate from a variety of scientific instruments and data stores. A major challenge for data-driven science applications is the integration of data into the scientist's workflow. Recent advances in dynamic, networked cloud infrastructures provide the building blocks to construct integrated, reconfigurable, end-to-end infrastructure that has the potential to increase scientific productivity. However, applications and workflows have seldom taken advantage of these advanced capabilities.
Dynamo will allow atmospheric scientists and hydrologists to improve short- and long-term weather forecasts, and aid the oceanographic community to improve key scientific processes like ocean-atmosphere exchange, turbulent mixing etc., both of which have direct impact on our society.

The Dynamo project will develop innovative network-centric algorithms, policies and mechanisms to enable programmable, on-demand access to high-bandwidth, configurable network paths from scientific data repositories to national CyberInfrastructure facilities, and help satisfy data, computational and storage requirements of science workflows. This will enable researchers to test new algorithms and models in real time with live streaming data, which is currently not possible in many scientific domains. Through enhanced interactions between Pegasus, the network-centric platform, and new network-aware workflow scheduling algorithms, science workflows will benefit from workflow automation and data management over dynamically provisioned infrastructure. The system will transparently map application-level, network Quality of Service expectations to actions on programmable software defined infrastructure.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

Scientists use computer systems to analyze and store their scientific data, sometimes in a complex process across multiple machines in different geographical locations. It has been observed that sometimes during this complex process, scientific data is unintentionally modified or accidentally tampered with, with errors going undetected and corrupt data becoming part of the scientific record. The IRIS project tackles the problem of detecting and diagnosing these unintentional data errors that might occur during the scientific processing workflow. The approach is to collect data relevant to the correctness and integrity of the scientific data from various parts of the computing and network system involved in the processing, and to analyze the collected data using machine learning techniques to uncover errors in the scientific data processing. The solutions are integrated into Pegasus, a popular "workflow management system" - a software used to describe the complex process in a user-friendly way and that handles the details of processing for the scientists. The research methods will be validated on national computing resources with exemplar scientific applications from gravitational-wave physics, earthquake science, and bioinformatics. These solutions will allow scientists, and our society, to be more confident of scientific findings based on collected data.

Data-driven science workflows often suffer from unintentional data integrity errors when executing on distributed national cyberinfrastructure (CI). However, today, there is a lack of tools that can collect and analyze integrity-relevant data from workflows and thus, many of these errors go undetected jeopardizing the validity of scientific results. The goal of the IRIS project is to automatically detect, diagnose, and pinpoint the source of unintentional integrity anomalies in scientific workflows executing on distributed CI. The approach is to develop an appropriate threat model and incorporate it in an integrity analysis framework that collects workflow and infrastructure data and uses machine learning (ML) algorithms to perform the needed analysis. The framework is powered by novel ML-based methods developed through experimentation in a controlled testbed and validated in and made broadly available on NSF production CI. The solutions will be integrated into the Pegasus workflow management system, which is used by a wide variety of scientific domains. An important part of the project is the engagement with science application partners in gravitational-wave physics, earthquake science, and bioinformatics to deploy the analysis framework for their workflows, and to iteratively fine tune the threat models, ML model training, and ML model validation in a feedback loop.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

NSF's major multi-user research facilities (large facilities) are sophisticated research instruments and platforms - such as large telescopes, interferometers and distributed sensor arrays - that serve diverse scientific disciplines from astronomy and physics to geoscience and biological science. Large facilities are increasingly dependent on advanced cyberinfrastructure (CI) - computing, data and software systems, networking, and associated human capital - to enable broad delivery and analysis of facility-generated data. As a result of these cyber infrastructure tools, scientists and the public gain new insights into fundamental questions about the structure and history of the universe, the world we live in today, and how our plants and animals may change in the coming decades. The goal of this pilot project is to develop a model for a Cyberinfrastructure Center of Excellence (CI CoE) that facilitates community building and sharing and applies knowledge of best practices and innovative solutions for facility CI.

The pilot project will explore how such a center would facilitate CI improvements for existing facilities and for the design of new facilities that exploit advanced CI architecture designs and leverage establish tools and solutions. The pilot project will also catalyze a key function of an eventual CI CoE - to provide a forum for exchange of experience and knowledge among CI experts. The project will also gather best practices for large facilities, with the aim of enhancing individual facility CI efforts in the broader CI context. The discussion forum and planning effort for a future CI CoE will also address training and workforce development by expanding the pool of skilled facility CI experts and forging career paths for CI professionals. The result of this work will be a strategic plan for a CI CoE that will be evaluated and refined through community interactions: workshops and direct engagement with the facilities and the broader CI community.

This project is being supported by the Office of Advanced Cyberinfrastructure in the Directorate for Computer and Information Science and Engineering and the Division of Emerging Frontiers in the Directorate for Biological Sciences.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

Unmanned Aerial Vehicles (also known as drones) are becoming popular in the sky. Their application reaches from hobby drones for leisurely activities to life-critical drones for organ transport to commercial applications such as air taxis. The safe, efficient, and economic operation of such drones poses a variety of challenges that have to be addressed by the science community. For example, drones need very detailed, close to the ground weather information for safe operations, and data processing and energy consumption of drones need to be intelligently handled. This project will provide tools that will allow researchers and drone application developers to address operational drone challenges by using advanced computer and network technologies.

This project will provide an architecture and tools that will enable scientists to include edge computing devices in their computational workflows. This capability is critical for low latency and ultra-low latency applications like drone video analytics and route planning for drones. The proposed work will include four major tasks. First, cutting edge network and compute infrastructure will be integrated into the overall architecture to make them available as part of scientific workflows. Second, in-network processing at the network edge and core will be made available through new programming abstractions. Third, enhanced end-to-end monitoring capabilities will be offered. Finally, the architecture will leverage the Pegasus Workflow Management System to integrate in-network and edge processing capabilities.

Providing best practices and tools that enable the use of advanced cyberinfrastructure for scientific workflows will have a broad impact on society in the long term. The science drivers that will be supported by this project have the potential to increase the safety and efficiency of drone applications, an area that will grow in significance in the foreseeable future. The project team will enable access to a rich set of resources for researchers and educators from a diverse set of institutions (historically black colleges and universities (HBCU), community colleges, women?s colleges) to further democratize research. In addition, collaboration with the NSF REU (Research Experience for Undergraduates) Site in Consumer Networking will promote participation of under-served/under-represented students in project activities.

Information about the project will be available at http://www.flynet-ci.org to provide information on overall project activities, outreach activities, publications, tools and software, and the project?s team members. The project website will be preserved for at least three years after the project ends.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

Innovative and robust Cyberinfrastructure (CI) is critical to the science missions of the NSF Major Facilities (MFs), which are at the forefront of science and engineering innovations, enabling pathbreaking discoveries across a broad spectrum of scientific areas. The MFs serve scientists, researchers and the public at large by capturing, curating, and serving data from a variety of scientific instruments (from telescopes to sensors). The amount of data collected and disseminated by the MFs is continuously growing in complexity and size and new software solutions are being developed at an increasing pace. MFs do not always have all the expertise, human resources, or budget to take advantage of the new capabilities or to solve every technological issue themselves. The proposed NSF Cyberinfrastructure Center of Excellence, CI Compass, brings together experts from multiple disciplines, with a common passion for scientific CI, into a problem-solving team that curates the best of what the community knows; shares expertise and experiences; connects communities in response to emerging challenges; and builds on and innovates within the emerging technology landscape. By supporting MFs to enhance and evolve the underlying CI, the proposed CI Compass will amplify the largest of NSF’s science investments, and have a transformative, broad societal impact on a multitude of MF science and engineering areas and the community of scientists, engineers, and educators MFs serve. CI Compass will also impact the broader NSF CI ecosystem through dissemination of CI Compass outcomes, which can be adapted and adopted by other large-scale CI projects and thus empower them to more efficiently serve their user communities.

The goal of the proposed CI Compass is to enhance the CI underlying the MF data lifecycle (DLC) that represents the transformation of raw data captured by state-of-the-art scientific MF instruments into interoperable and integration-ready data products that can be visualized, disseminated, and converted into insights and knowledge. CI Compass will engage with MFs and contribute knowledge and expertise to the MF DLC CI by offering a collection of services that includes evaluating CI plans, helping design new architectures and solutions, developing proofs of concept, and assessing applicability and performance of existing CI solutions. CI Compass will also enable knowledge-sharing across MFs and the CI community, by brokering connections between MF CI professionals, facilitating topical working groups, and organizing community meetings. CI Compass will also disseminate the best practices and lessons learned via online channels, publications, and community events.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.