Charles C. Weems - US grants

EIA-9977540
Weinberg, Martin D.
Auerbach, Scott M.
University of Massachusetts Amherst

MRI: Acquisition of a High-Performance Parallel Computer to Support an Interdisciplinary Computational Science Center for Physical Science, Mathematics and Engineering

This is a proposal to construct a high-performance computing platform to enhance the state of scientific computing at the University of Massachusetts-Amherst. The computing platform will consist of 48 Pentium II processors. The purpose of this parallel computing platform is to: (i) facilitate the research of faculty members whose work involves computational simulation or massive data processing, (ii) encourage collaboration among faculty located in diverse departments and schools, (iii) provide a medium where new compiler technology and parallel processing research in the Computer Science Department can be tested and incorporated into state-of-the-art scientific applications, (iv) provide a testbed for new parallel algorithms and approaches and (v) allow graduate students quick access to parallel processing capabilities for use in classes or directly in research.

This research will explore which high-level information the architecture can use and the compiler can provide to exploit, predict, and react to program behavior. The compiler will give the architecture control- and data-dependence information in aggregate forms or using new high-level instructions and the architecture will further augment this information with run-time information to improve perfor-mance. To exploit this information, this research will also explore new architectural functionality, flexibility, caching policies, etc.

This research will also explore using symbolic data volumes and reuse distances parameter-ized with run-time values, and the architecture will decide the policies; how much and which data to prefetch, stream, or cache and with which replacement policies. This investigation will use the Scale compilation system to produce program "road maps," and will use simulation to explore advanced architecture features that employ navigation of the maps. The benefit of this research will be to sig-nificantly improve scientific and next generation software performance through a synergy of compiler information and new architectural mechanisms.

The CoGenT project aims to make it much easier to sketch new
architectural ideas at the instruction set architecture and
micro-architecture ("block diagram") levels. We have designed languages
for expressing instruction set semantics and encoding (CISL, the CoGenT
Instruction Set Language), and for describing micro-architecture
components, connections, and timing (CASL, the CoGenT Architecture
Specification Language). CoGenT is now completing the language
processors and run-time environments so that we can generate, from CISL
and CASL machine descriptions, functional and timing simulators, and
competent (but not automatically innovative) compilers. Among other
things we are solving the problem of generating provably correct code
generators and register allocators automatically, for the compiler side,
and generating similarly correct simulators automatically on the
simulator side. (Of course correctness is relative to correctness of the
CISL and CASL descriptions.) CoGenT also aims to provide a capable
simulator environment and tools for setting up and running simulation
experiments. Our hope is to revolutionize architecture design
exploration and simulation, with research, industrial, and teaching
impact. We are focusing on getting the maximum functionality implemented
under this award.

This is an EAGER project that addresses a highly exploratory investigation into key elements needed to specify the characteristics of an operating system (OS) in a way that permits an architectural model to be created that interacts fully with a suite of simulation tools.

The suite of tools, CoGenT (CoGeneration of Tools), include specification languages to allow researchers to express novel instruction sets and micro-architectures and the infrastructure for automatic generation of corresponding functional and timing co-simulators, compilers, linkers, loaders, debuggers, assemblers, disassemblers, and a fully integrated instrumentation facility to enable meaningful experimentation within this new design space. CoGenT?s ability to automatically generate a functional simulator from a specification, and other related elements, will be released this year.

This EAGER addresses the problem that, in simulating complex architectures, it is important to be able to specify OS support, not just as a set of external calls, but as a specific model that integrates with the rest of the architecture. Current architectures rely on the services and policies of the operating system, and the operating system itself needs to evolve with the radical shifts in architecture and applications that are anticipated in the next decade.

With this project, this team develops an approach that enables simultaneous research into novel hardware and software paradigms, with great flexibility, and without the heretofore prohibitive cost of manually building a complete hardware and software simulation infrastructure with a tailored OS implementation. Traditional system simulation approaches either ignored OS impact on performance or resorted to costly and inflexible full system simulation where an actual OS implementation is executed directly. The former provides unrealistic results, and the latter does not admit the kind of exploration needed for transformative paradigm shifts.

The goal of this project is to extend the relatively recent approach of functional and timing co-simulation for hardware architectures into "pseudo-full system simulation", where the OS becomes a first-class element in the simulation modeling and instrumentation framework. Simulating an OS model derived from a specification will also enable sensitivity and significance analyses, often neglected in current simulation-based research even though they are essential to understanding the real impact of new approaches.

Computing increasingly permeates daily lives, yet few appreciate the growing presence of Parallel and Distributed Computing (PDC) in common computing activities; e.g., modern laptops' processors contain multiple cores and special-purpose devices such as graphics processors (GPUs). With increasing availability of powerful PDC technology, familiarity with single-processor computers and sequential computing no longer constitutes computer literacy. Technological developments point to the need for a broad-based skill set in PDC at all levels of higher education in disciplines such as Computer Science, Computer Engineering, and the related computational disciplines. The rapid changes in technology challenge educators to decide what to teach and how to teach it. Students and employers face similar challenges in characterizing "basic" expertise in computing. The PIs are addressing these challenges via a project devoted to creating and sustaining curricular and educational infrastructure to facilitate the teaching of PDC topics in undergraduate computer-related curricula. The goal is for every graduating student to become skilled in PDC technology, hence be prepared to enter tomorrow's workforce.

The project embodies multiple synergistic activities that develop: flexible PDC curricula for a spectrum of academic programs and institutions; mechanisms that help individuals maintain currency; instructional materials for PDC-related topics; experience-based guidelines for injecting PDC into curricula. A signature activity is competitions for early adopters of PDC curricula (winners receive seed funds, equipment donations from industry) and workshops and training sessions to foster awareness and adoption of PDC curricula. Feedback from early adopters and coordination with the ACM/IEEE 2013 CS Curriculum Taskforce steers future development of both the PDC curricular guidelines and of strategies for deploying PDC material within computing curricula at a larger scale.

This project is supported by CISE, OCI, and EHR/DUE.

Computers directly support arithmetic that is typically limited to 64 bits (about 19 decimal digits) of precision. Applications that need more precision must implement arithmetic through computationally expensive software. Beyond about 256 bits of precision, such calculations become quite costly. The RSA encryption algorithm, for example, can require arithmetic with up to 4096 bits of precision. Applications in areas such as experimental mathematics and number theory can require millions of bits of precision. One multiplication with 10 million bits of precision can take a tenth of a second to compute on a modern processor, which means that matrix arithmetic using such large values can take days to weeks to execute. In previous work the investigators have shown that it is possible to obtain a factor of 20 improvement in performance by utilizing the parallel processing capabilities of a commodity graphics processing unit (GPU) in place of the traditional CPU. However, programming a GPU to achieve this level of performance is quite difficult, and the resulting code requires considerable hand-tuning to move it to new generations of GPU and gain the advantage of their performance, which is scaling up at a rate that exceeds CPU performance scaling.

This project is working to develop a framework that automatically generates and tunes multi-precision arithmetic libraries to execute on successive generations of GPUs. The libraries include both scalar and basic matrix arithmetic routines. They support scaling in precision as well as matrix size. The problem is challenging because different parallel algorithms must be automatically selected for different levels of precision, which must be balanced with the exploitation of the alternate dimension of parallelism inherent in matrix arithmetic. In addition, the work seeks to employ distributed parallelism across a cluster of computers enhanced with GPUs, so that the libraries can be used on a new generation of GPU-based supercomputers that is beginning to be deployed at national laboratories.

The work is significant because it enables easier exploitation of low-cost commodity graphics processors to achieve more than an order of magnitude increase in performance for multi-precision scalar and matrix arithmetic. One important application is enhancing performance of RSA encryption to support longer, more secure keys, at greater data rates, so that it becomes feasible to encrypt greater volumes of internet traffic. Another important use is experimental mathematics, where computationally expensive functions (e.g., integrals, infinite series) are computed at high precision and compared to other functions and high precision constants to help identify more efficient closed-form solutions. Results from experimental mathematics have found applications in particle physics, chaos theory, and calculation of fundamental constants. The resulting software framework offers a significant performance enhancement for multi-precision arithmetic to systems that range from individual researcher workstations to large supercomputers.

This project is initiating design of a Parallel and Distributed Computing (PDC) curriculum enhancement for the Computer Science Principles (CS0) course that is also applicable to the corresponding high school advanced placement (AP) course.
Modern computing systems routinely employ high levels of parallelism (e.g., multiple cores, graphics processors) and distributed computing (e.g., cloud services). PDC significantly affects daily lives, and its importance in solving meaningful societal problems can inspire students to take a deeper interest in applying computing in STEM fields and beyond. However, most CS0 curricula are founded predominantly on older, sequential models of computing. CS0 has multiple purposes that address many audiences, and is now seen as a vehicle for increasing interest among students, especially from underrepresented groups, for studying computing in depth or to major in computer science. It has thus been the subject of an effort to create a new AP exam, with associated high school courses. Since the AP exam is still in a pilot phase, there is a brief window for amending the curriculum to reflect 21st century computing. The potentially vast impact of the AP course could quickly settle into educating students under the old sequential paradigm, acquiring an inertia that makes it difficult to change. This collaborative effort is developing PDC curriculum guidelines for CS0 and AP courses through gathering expert input and involving necessary stakeholders to ensure that an appropriate curriculum is developed and adopted. The goal is for students to gain an understanding of how modern computing technology actually functions, rather than being taught an obsolete operational model.
The effort is following a process similar to the proposers' highly successful prior effort to design a PDC guideline for undergraduate curricula, which was incorporated into the ACM standard curriculum. The approach begins with a thorough survey of the state of practice in covering PDC topics in CS0. The survey is followed by a workshop attended by experts and stakeholders to draw upon their knowledge and experience. Then a select steering committee is formed that advises the investigators on the design of a curriculum and helps them identify means to bring it to fruition. The effort seeks the broadest possible dissemination of the results to have the greatest possible impact.
Broad impact is the core of this effort, which seeks to expand awareness and appreciation of the important concepts at the foundation of PDC, to students in many different disciplines and in high school. Making these interesting and exciting topics more accessible to a wider range of students should generate deeper interest in the study of computer science and the application of concurrent computation in science, engineering, and commerce. Advancing PDC education further enables advances in science and engineering, which depend ever more on high performance computing, by providing the next generation of practitioners and researchers with the necessary skills and knowledge to routinely recognize how and where PDC concepts may be applied.

This project extends the PI's prior research into achieving high performance for multiprecision arithmetic utilizing commodity graphics processors (GPUs). Multiprecision (MP) arithmetic has important applications in science, engineering, and mathematics when computations require greater numerical precision than standard computer systems support. It is also an important part of cryptography used in secure internet communication. GPUs can accelerate MP arithmetic by more than two orders of magnitude. However, achieving this performance requires novel algorithms and software tools. The world-record performance for exponentiation achieved under the prior grant will be extended to include floating point vector arithmetic. A new code generation model will enable handling a wider range of precisions across newer generations of graphics processors. Support for clusters of GPUs to work together on larger problems, and practical demonstrations of the effectiveness of MP library such as showing how one GPU can offload decryption work from more than a hundred servers, with higher levels of security than are currently in common use, is being developed.

Each generation of GPU architecture requires extensive experimentation and reworking of multiprecision code to obtain a new optimum. Yet the potential benefits of a portable and scalable package could be transformational in certain application areas. This effort extends PI's prior work to include floating point and vectors, and begin the transition to GPU clusters. The result will be a publicly available multi-precision arithmetic package and implementation toolset that enables the scientific community to easily take full advantage of GPU scaling to obtain at least an order of magnitude improvement in performance per dollar and performance per watt over CPUs at the same technology step. The approach relies on a novel set of models for GPU storage that provide a higher level of abstraction over which the code generation tools can search for optimal combinations of algorithm, register/memory layout, and kernel launch geometry for a given precision size and GPU architectural generation to achieve maximum resource utilization.

Introducing parallelism early into undergraduate education is very important to broader society goals to create knowledgeable workforce. This proposal is seeking support for a meeting to gather community input to inform further efforts for the ongoing initiative of developing and implementing of Parallel and Distributed Computing (PDC) undergraduate curricula nationally and internationally.
Meeting attendees include domain experts, educators, and representatives from government and industry. The discussion is focused around needs of undergraduate curricula, both in introductory and upper level courses, given changes in the parallel and distributed computing landscape, as well as the challenges and opportunities for broadening adoption and participation.

This effort will develop a cadre of college faculty to pioneer a shift in the computer science curriculum to ensure students are educated in the use of 21st century platforms that pervasively incorporate parallel and distributed computing (PDC). Twentieth century computers were mostly built on the principle of a single processor, executing a sequence of operations. That model is tightly bound into curricula, even though the last decade has seen widespread deployment of multi-core processors, graphics processors, online servers, and the internet of things, all of which depend on the much different PDC mindset for problem solving and programming. Financial, technical, scientific, engineering and medical companies, government laboratories, the department of defense, the intelligence community, and many other sectors are desperately seeking employees who can exploit PDC systems, because the existing workforce was heavily steeped in the older sequential model. Even new graduates continue to learn the old approach because of the considerable inertia in the educational system. Thus, by turning the tide toward incorporation of PDC into the early stages of computer science education, through teaching the teachers, this project will strategically serve the national interest, as stated by NSF's mission: to promote the progress of science; to advance the national health, prosperity and welfare; and to secure the national defense. It will be a significant step toward modernizing the emerging workforce to have the computing skills needed for the United States to maintain leadership in all of these areas.

The Center for Parallel and Distributed Computing Curriculum Development and Educational Resources (CDER) with the IEEE Computer Society Technical Committee on Parallel Processing, developed curriculum guidelines for parallel and distributed computing (PDC) that guided PDC aspects of the ACM Computer Science Curricula 2013. The guideline sought to shift courses in the first two years from the sequential model toward the PDC paradigm. In the modern world, students must see PDC as an aspect of computational problem solving from the very beginning.
This project will update the curriculum guideline, with special foci on big data, energy, distributed computing, and exemplars. It will offer grants (encouraging participation by institutions serving underrepresented groups) for faculty to work on course development and attend a workshop where they will be trained in the use of PDC, and in experimental course design and evaluation. The workshops, offered each summer of the grant period, will also support attendance by industrial and government stakeholders to help build a network of relationships. Additional instructors will be offered travel support to attend the workshops, beyond the course development grants.

This collaborative project represents a multi-faceted effort to shift computer science and engineering education toward ensuring that students can use 21st century platforms that pervasively incorporate parallel and distributed computing (PDC). Twentieth century computers were mostly designed around a single processor, executing a sequence of operations. But this century is characterized by widespread deployment of multi-core, graphics, and AI tensor processors, as well as a shift to cloud servers, and the internet of things, all of which depend on the much different PDC approach to problem solving and programming. Financial, technical, scientific, engineering and medical companies, government labs, the department of defense, the intelligence community, and many other sectors are desperately seeking employees who can exploit PDC systems, because the existing workforce was heavily steeped in the old model. Yet most students continue to learn the old approach due to significant inertia in academia. To turn the tide toward infusing PDC into the early stages of computer science and engineering education, this project will guide curricula and accreditation standards, prepare teachers, and foster a strong PDC education community. It will thus strategically serve the national interest, as stated by NSF's mission: to promote the progress of science; to advance the national health, prosperity and welfare; and to secure the national defense. It will be a significant step toward modernizing the emerging workforce to have the computing skills needed for the United States to maintain leadership in all of these areas.

The Center for Parallel and Distributed Computing Curriculum Development and Educational Resources (CDER) is preparing the 2020 update of their 2013 curriculum guidelines for introducing parallel and distributed computing (PDC) into early undergraduate courses. This project will engage in four areas of activity to foster adoption of the curriculum, and extend it, with the goal of modernizing computer science and engineering workforce development.
One major thrust is running summer training workshops for teachers, to learn both PDC concepts and experimental course evaluation methodology. The discipline is still in a phase of discovery with respect to PDC education approaches, and must encourage a diverse set of well-designed experiments to test and evaluate a broad range of pedagogical hypotheses. The workshop participants will be drawn from a diverse pool of educators, and given curriculum development grants in support of experimental course offerings and evaluation, leading to conference or journal publications, as well as contributions of exemplars to the CDER online course materials repository.
A second effort is to help ABET/CSAB to formulate core PDC requirements and to inform/train ABET/CSAB evaluators (CSAB is the lead society within ABET for accreditation of degree programs in computer science).
A third effort is expanding the curriculum guidelines to explicitly address adding PDC to computer engineering programs, which present novel curricular opportunities.
Lastly, it will continue CDER's successes in organizing PDC education workshops in conjunction with major conferences, publishing PDC education books and journal special issues, maintaining and curating an online repository of PDC education resources, and providing free access to a publicly available PDC education cluster system.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.