Christopher G. Tully - US grants

The scale of data storage and analysis for the high energy physics experiments at the Large Hadron Collider (LHC) at CERN, Switzerland, provides new challenges and necessitates research into new concepts of CPU utilization. The quantity of data produced is too large to store at any single university. Rather, the data will be archived in a central location. This project will develop techniques and software packages to enable the efficient distributed analysis of these data by researchers at remote sites. This proposal would support the investigation into how to send a user's request for a particular analysis and some code to the locations where the data are stored and run the code there. The system would then collect and assemble the results and present them to the user. The framework would be the BlueOx system and Java will be the implementation language for the BlueOx framework so as to take advantage of its cross-platform compatibility.

The most important challenge facing the framework is that of scalability. This project plans to study and improve the scalability by testing it with large dummy datasets of simulated data on a moderate-sized computing cluster. The aim is to keep the generality that can make it useful to many large scientific projects.

Scientific researchers on university campuses create, analyze, visualize, and share large and diverse datasets from experimental devices like brain scanners, particle colliders, and genome sequencers. However, these "big data" applications place strain on traditional campus networks, due to rapidly increasing volumes of data, the need for either predictably low latency (to adapt experiments in real time) or high throughput (to transfer large data sets between locations), and sophisticated access-control policies (to protect the privacy of human subjects). To enable the next wave of scientific advances, university campuses must find effective ways to meet these challenging demands, at reasonable cost. The emerging technology of Software-Defined Networking (SDN) lowers the barrier to innovation in network management, and can substantially reduce cost through (i) inexpensive commodity network switches, (ii) greater automation of network configuration, and (iii) novel network-management applications that optimize bandwidth usage. Yet, existing innovation in SDN focuses primarily on the needs of commercial cloud providers, rather than the unique requirements of university campuses and scientific researchers. Princeton University is creating a software-defined campus network that can enable the next generation of data-driven scientific research. The initiative brings together big-data science researchers, computer scientists who are experts in SDN, and the campus Office of Information Technology. Princeton is deploying an open-source SDN platform for monitoring and configuring the network, conducting trials of new ways to support big-data applications, and bridging with the larger community, on and off campus, to support the sharing of scientific data, SDN software, and operational experiences.