“The purpose of computing is insight, not numbers.” — Richard W. Hamming

Petascale computing refers to both petaflops, a million-billion calculations per second, and petabytes, a million-billion bytes of data. This level of computing power will enable the study of scientific problems at an unprecedented level of detail. For example, current models allow scientists to design materials with thousands of atoms, while petascale computing will allow models with millions of atoms, yielding more accurate simulations of the properties of these materials.

Supercomputers
Over the last 2 decades, there have been many calls for investment in supercomputing
Figure 1. Snapshot of the magnetic field simulated with the Glatzmaier-Roberts geodynamo model using a spatial resolution of 145 radial levels, 144 latitudinal levels, and 144 longitudinal levels. The field is illustrated with magnetic lines of force that extend out to two Earth radii. The field is intense and complicated inside the model’s fluid core, where convection and rotation generate it. Outside the core, the field structure is dominated by the dipole. The evolution was simulated for roughly one million numerical time steps, which took almost a year of computing on 24 processors of a Linux cluster. Simulations have been run with resolution of roughly 300 by 300 by 300 on 128 or 256 processors in order to specify somewhat more-realistic parameters. These higher- resolution runs have been done at NSF supercomputing centers, but far fewer time steps can be simulated because of the time the 12-hr incremental jobs sit in the queue waiting to run.
resources in support of US science and engineering research. The National Science Foundation (NSF) has responded by establishing a network of supercomputing centers and associated infrastructure that is open to the broad academic research community. Despite this investment, recent assessments of cyber-infrastructure requirements have indicated that research at the frontiers of the geosciences in the United States is being impeded by an acute shortage of capability-class (the most powerful supercomputers used to solve the largest and most demanding problems) computing resources.

An ad hoc committee of scientists working on behalf of the atmospheric, solid Earth, ocean, and space science communities was formed with the NSF’s encouragement to address the gap between the scientific requirements for, and the availability of, high-end computational resources. The committee’s members are from universities, research centers and national laboratories.

Their report, Establishing a Petascale Collaboratory for the Geosciences: Scientific Frontiers, is a landmark in petascale computing literature. It presents an overview of the scientific frontiers that would be opened by a national investment in leadership-class computing systems dedicated to geosciences research.

The report recommends an additional, complementary strategy for NSF investment in computational resources: deployment of the most powerful leadership-class systems available in the form of a “collaboratory,” a community-specific computational environment for research and education that provides high-performance computing services; data, information and knowledge management services; human interface and visualization services; and collaboration services.

The collaboratory concept is inspired by the report’s observation that in the past decade the geosciences have progressed beyond traditional disciplinary organization that focused separately on atmospheric, oceanic, solid Earth and space science problems. The challenges today demand integration across the disciplinary science because the individual systems and the planet’s biota (including humans) interact to form a much larger system of systems whose components must ultimately be studied and understood together. The challenges today also demand interaction among the disciplinary scientists, because there is a great deal of commonality in the way the various disciplines approach their respective problems. A principal commonality is that laws of hydrodynamics, magneto-hydrodynamics, and thermodynamics govern many central problems in the geosciences.

Feasibility
Based on the analysis in a companion report (Technical Working Group and Ad Hoc Committee for a Petascale Collaboratory for the Geosciences, 2005), it appears to be technically feasible to construct a highly capable petascale computing collaboratory that can deliver 200 teraflops (TFLOPS) aggregate peak in 2007 and achieve 1 petaflop (PFLOP) peak by 2010.

Outcomes
Scientific discovery. In recent years, the geosciences research community has made
Figure 2. T170 simulation: An experimental high-resolution integration of the National Center for Atmospheric Research (NCAR) Community Climate Model ver. 3 (CCM3) was made for one year of simulated time with specified observed sea surface temperature as lower boundary conditions. The model includes 170 global wave numbers in the horizontal, equivalent to resolving features as small as 75 km across. The figure shows the instantaneous distribution of total precipitable water (white shading) and precipitation (orange shading). The fine structure evident in the figure, including the narrow moisture flux convergence zones near the equator and in bands oriented from northwest to southeast in the south Indian, Pacific, and Atlantic Oceans, is much more representative of the scale of moisture distribution in the real atmosphere than could be achieved with typical climate models in use today. (NCAR)
significant progress in its ability to monitor and simulate Earth system components and is poised to make significant breakthroughs in understanding the system as a whole. Scientific applications described in this report indicate that significant breakthroughs will be made possible by establishing the PCG equipped with a system capable of sustaining 100-200 TFLOPS.

Programmatic coordination. The PCG will fertilize the interdisciplinary nature of geosciences research, strengthening existing ties and building new relationships among disciplines, facilitate the sharing of advances in computational methods, and thereby accelerate progress in the development of a comprehensive understanding of the Earth system. The subdisciplines of the geosciences community are prepared to enter this endeavor collectively. This community, among all the branches of science, represents a unique balance between the breadth of the disciplinary research and commonality of purpose to justify and cost-effectively direct a national investment of this order.

The PCG will make possible new and extraordinary capabilities. For example, it will:
• Permit the computational geosciences community to dramatically expand the range of processes and spatial and temporal scales represented in simulations, making it possible to understand how the many Earth-system elements interact to produce the complex phenomena that shape Earth’s past, present, and future;
• Enable exploration of new phenomena and more accurate formulations that are inaccessible with current computational resources;
• Enable the simulation of the full spectrum of interactions among physical, chemical, and biological processes in coupled Earth system models;
• Challenge and improve understanding of Earth-system processes via synthesis of state-of-the-art models and modern Earth observations; and
• Permit the geosciences research community to assist in the evaluation of risk and design of mitigation strategies for natural hazards.

Numerous areas of investigation will be enabled by petascale computing and the PCG. For example:
Global seismology. A petascale capability will facilitate the simulation of global seismic wave propagation at periods of one second and longer. Observational seismologists routinely analyze signals in this period range. Reaching this parameter regime numerically will finally enable us to probe Earth’s deep interior with sufficient resolution to analyze the complex three-dimensional properties and structure of the core-mantle interface (the D” layer) and the enigmatic inner core.

Multiscale modeling in mineral and rock physics. A petascale facility will allow integration
Figure 3. Remotely sensed, multispectral image of the Potomac River, 65 to 80 km up-river from the Chesapeake Bay, illustrating the range of features that could be resolved explicitly using a petascale facility. (T. Donato, Naval Research Laboratory)
of spatial information from the atomic scale (10-10 m) to the Earth process scale (103 m), and time scales from femtoseconds (10-15) for molecular dynamics to millions of years by coupling simulations of the properties of minerals to geodynamics simulations through multiscale modeling frameworks.

Seismic tomography. The challenge in modern seismic tomography lies in harnessing new found three-dimensional modeling capabilities to enhance the quality of tomographic images of the Earth’s interior in conjunction with improving models of the rupture process during an earthquake. Using modern data-assimilation techniques, a petascale facility would finally enable seismologists to tackle this fundamental problem.

Conclusion
The report observes that one of the most compelling and challenging intellectual frontiers facing humankind is the comprehensive and predictive understanding of Earth’s structure, dynamics, and metabolism. The Earth is a complex, nonlinear system dynamically coupled across a vast range of spatial and temporal scales. In addition, Earth-system processes are coupled across its physical, chemical and biological components, all of which interact with human society. Recent progress in understanding the atmosphere, ocean, solid Earth, and geospace environment, and the rapid expansion in our ability to observe our planet, present the geosciences community with new and expanded opportunities to advance understanding of the Earth as a coupled system. This improved understanding will provide a basis for meeting urgent needs to reduce the uncertainty in predictions of Earth system processes, provide more accurate and comprehensive tools to decision-makers, and foster technological development in the geosciences-related spheres of the public and private sectors. Along with theory and observation, computational simulation has become established as a cornerstone of this progress in Earth system science. Increasingly, numerical simulations are not only tested by observations, but provide the first glimpses of new phenomena and quantitative characterization of complex processes. In turn, these simulations inspire new theoretical investigations and observational strategies.

EDITOR’S NOTE: Material in this article was taken from the Ad Hoc Committee and Technical Working Group for a Petascale Collaboratory for the Geosciences. 2005. Establishing a Petascale Collaboratory for the Geosciences: Scientific Frontiers. A Report to the Geosciences Community. UCAR/JOSS. 80 pp. Additional material was taken from the Advanced Scientific Computing Advisory Committee Petascale Metrics Report, February 28, 2007 by the Petascale Metrics Panel, a subcommittee of the Department of Energy Office of Science, Advanced Scientific Computing Advisory Committee.