Petaflop Essay Contest

Peta flop E ssay Contest

Simulating the Birth of the Universe on a Petaflop By Thomas Luu, Ron Soltz, and Pavlos Vranas

I

n the beginning, when the universe was less than one microsecond old and more than one trillion degrees hot, it transformed from a plasma of quarks

and gluons into bound states of quarks we refer to as protons and neutrons, the fundamental building blocks of nuclear matter that make up most of the visible universe. We believe this happened because the theory of quantum chromodynamics (QCD), which governs the interactions of the strong nuclear force, predicts it should happen when such conditions occur. Recent experiments at the Relativistic Heavy Ion Collider at Brookhaven National Laboratory have provided direct evidence of the existence of this phase transition in collisions between gold nuclei at the highest attainable energies. But calculating the properties of this phase transition has been notoriously difficult and computationally challenging. What we’ve been able to discern via this theory has come by situating space and time onto a 4D grid of lattice points, on a volume no bigger than the size of a large nucleus. This discrete formulation of QCD known as lattice QCD (LQCD) can be tamed in a way that’s numerically amenable to massively parallel machines. Indeed, our LQCD code scaled with perfect speedup all the way to the 131,072 CPU cores on the world’s fastest supercomputer, the BlueGene/L (BG/L) system at Lawrence Livermore National Laboratory (LLNL). Efforts are currently under way to calculate this phase transition’s properties using tens of teraflops spread across many of the world’s fastest computers. However, the lattice discretization used in our calculations of space–time breaks one of the most crucial properties of the underlying theory: the basic symmetry of chiral rotations. Chiral symmetry can’t be fully restored on the lattice, but it can be restored and controlled to a high degree by using the method of domain wall fermions (DWF). DWF introduces an extra fifth dimension, in which chiral symmetry is restored with only a small amount of breaking, which decreases as the fifth dimension’s size increases. In particular, researchers have estimated that a fifth dimension of roughly 64 lattice points is sufficient to restore symmetry with a systematic error of only a few percent,

November/December 2007

even around the transition temperature. Studies have also shown that DWF exhibits excellent fidelity in incorporating the transition’s basic driving forces. But the computational cost of this calculation is roughly 2 × 64 = 128 times higher than the cost of current methods that don’t preserve this symmetry on the lattice, and thus it falls outside the realm of current terascale supercomputing. With petascale supercomputing, however, this restriction is lifted. If we had petascale supercomputing for our calculation, we could use computational techniques similar to those used on BG/L to calculate the thermodynamic properties—or more precisely, the equation of state—of the quark–gluon plasma just as it begins to form protons and neutrons as it did in the very early universe. The basic level of parallelism again comes from the division of the 4D

Petaflop Contest for Supercomputing 2007

N

ew levels of computing power open up new vistas for science. Earlier this summer, former CiSE editor in chief Francis Sullivan launched an essay contest to see what people would choose to compute, assuming they had a petaflop machine on their desktops. The contest generated many compelling essays about using computers to study the deepest and most exciting questions in science. Our contest winners chose the birth of the universe, and the runner-up focused on the evolution of life. In an additional essay, Sullivan comments on the likelihood of access to petaflop computing for large numbers of scientists. —eds.

Copublished by the IEEE CS and the AIP

1521-9615/07/$25.00 ©2007 IEEE

55

Petaflop Essay Contest

grid into subgrids assigned to nodes on a massively parallel machine, thereby providing perfect speedup. DWF’s fifth dimension provides yet another level of parallelism: given that the nodes of a massively parallel petascale supercomputer will likely consist of several CPU cores per chip, it’s natural to fully map the fifth dimension along the node chip’s CPU cores. On a 16-CPU core chip, for example, we could assign four fifth-dimension “slices” per CPU core for a total of 64. In this way, the new level of parallelism in the application matches the new level of parallelism in the hardware. The communication patterns along the fifth dimension are nearest-lattice-neighbor only and can be handled via shared memory or any other on-chip data transfer mechanism, thus alleviating off-chip data transfers while exploiting the hardware’s native capability. Based on current calculations using tens of teraflops on the LLNL BG/L supercomputer, our proposed simulation would require 128x more computing power—in other words, several petaflops for approximately three months. This level of computing could provide a glimpse of the early universe as we’ve never seen it before and lead to a dramatic improvement in our understanding of the interactions of nature’s smallest particles—quarks—which also comprise its largest visible structures. Acknowledgments This work is supported under the auspices of the US Department of Energy by the University of California, Lawrence Livermore National Laboratory, under contract W-7405-Eng-48. Thomas Luu is a staff physicist at Lawrence Livermore National Laboratory. His research interests include many-body nuclear physics, cold-atom physics, and lattice quantum chromodynamics applied to few-body nuclear systems. Luu has a PhD in physics from the University of Washington. Contact him at [email protected]. Ron Soltz is a staff physicist at Lawrence Livermore National Laboratory. His research interests include relativistic heavy ion collisions, lattice quantum chromodynamics, and investigating methods for detecting nuclear materials. Soltz has a PhD in physics from MIT. Contact him at [email protected]. Pavlos Vranas is a staff physicist at Lawrence Livermore National Laboratory. His research interests include lattice quantum chromodynamics and physics beyond the standard model and supercomputing. Vranas has a PhD in physics from the University of California, Davis. Contact him at [email protected].

56

Petaflop Essay Contest Runner-up

Petaflop-Enabled Simulation of Molecular Evolution at an Atomic Scale By Christopher M. Frenz

O

ne of the most prominent uses of computational methods in the

biophysical sciences is to help us understand the function of biological molecules at the atomic level. Researchers often use techniques that employ varying degrees of classical and quantum mechanical methods to predict molecular motions and other physical characteristics from atomic interactions.1 Although these simulations often provide insights into the structural basis for molecular functionality, they’re extremely computationally intensive. Simulations of large protein systems that comprise multiple proteins, and hence tens of thousands of atoms and atomic interactions, can therefore only be conducted in finite time by the most powerful computing environments. Computational time can be a limiting factor even for smaller protein systems, making it difficult to detect characteristics such as largescale molecular motions in a protein.2 The availability of a computing environment with peta­ flop calculation capabilities, however, would make it feasible not only to perform atomic-scale calculations of protein systems but would let us simulate an entire population of cells at an atomic level. The large increase in computational power would likely even make it feasible to observe large-scale molecular interactions and catalytic reactions in silico. Such simulation capabilities would provide an interesting environment for studying molecular evolution as well. The introduction of mutations into an organism’s genome might be a random event, but recent evidence suggests that the physics of the proteins expressed from those

Computing in Science & Engineering

genes plays a strong role in whether mutations remain. Recent studies have demonstrated correlations between amino acid residue conservation and protein physical properties such as electrostatics, energetics, and residue packing.3–5 Computational methods exist for computing each of these properties, so it’s computationally feasible that petascale computing could help us develop a way to predict a mutation’s viability over evolutionary time. Moreover, we could use other computational methods such as docking to assess the actual functional impact a given mutation would have on a biological system, such as by predicting a change in ligand binding or in the potential for molecular interactions.6 Conducting an atomic-level simulation of a cell would be a major breakthrough for the biological sciences in and of itself because it would let us observe the metabolic pathways and all the biochemical steps and the interactions that comprise them with an unprecedented level of detail. Yet, by simulating a population of such cells and bestowing on them a reasonable degree of genetic diversity and the ability to replicate, the possibilities are even more profound. We could expose such a simulation system to some type of environmental stressor and then monitor it to see what types of mutations enabled certain cells to survive and breed while others died out. These findings, in turn, could help scientists come closer to answering some of the most pressing questions humans have faced, including how we got here and where we come from. The evolutionary insights gained from developing a simulation of this nature are only a fraction of its utility, however—such a simulation system would also have practical applications to medicine and biotechnology. A common problem that the healthcare field faces is that many viruses and bacteria have developed drug-resistant strains that curtail the effect certain pharmaceuticals such as antibiotics pose to them.7 With petascale-enabled simulations, we could expose a simulated population of cells or viruses to a pharmaceutical stressor and try to predict in silico what types of drug resistance will likely develop, the biological effects of the resistance mutations, and how long it might take for such strains to appear. Moreover, much of the biotechnology industry is concerned with developing enzymes or metabolic systems that can carry out novel metabolic reactions, such as biodegrading oil spills.8 With a petascale-based simulation methodology, we could expose a cell population to the novel metabolite as an environmental stressor. The cells

November/December 2007

that evolved the ability to process the metabolite would be the ones fit enough to survive the simulation, allowing for the potential introduction of this mutation into actual biological cells and thereby bestowing the “evolved” capabilities on them. Thus, the simulations could serve as an extremely sophisticated method of bioengineering via directed evolution. This type of cellular evolution simulation at an atomic level of detail would be one of the most beneficial uses for a petaflop computing system. It would not only have an insurmountable value for deepening our understanding of one of biology’s greatest phenomena—evolution—but it would also be of great practical value for both medicine and biotechnology. References 1.

D.A. Case et al., “The Amber Biomolecular Simulation Program,” J. Computational Chemistry, vol. 26, no. 16, 2005, pp. 1668–1688.

2.

C.M. Frenz, “Possibilities and Limitations of Computer Simulation,” IEEE Potentials, vol. 26, no. 2, 2007, pp. 30–33.

3.

C.M. Frenz, “Interrelationship between Protein Electrostatics and Evolution in HCV and HIV Replicative Proteins,” Proc. 2007 Int’l Conf. Bioinformatics and Computational Biology (Biocomp 07), CSREA Press, 2007, pp. 91–98.

4.

D.R. Livesay et al., “Conservation of Electrostatic Properties within Enzyme Families and Superfamilies,” Biochemistry, vol. 42, no. 12, 2003, pp. 3464–3473.

5.

H. Liao et al., “Protein Sequence Entropy Is Closely Related to Packing Density and Hydrophobicity,” Protein Eng. Design and Selection, vol. 18, no. 2, 2005, pp. 59–64.

6.

D.S. Goodsell and A.J. Olson, “Automated Docking of Substrates to Protein by Simulated Annealing,” Proteins: Structure, Function, Genetics, vol. 8, no. 3, 1990, pp. 195–202.

7.

C.F. Higgins, “Multiple Molecular Mechanisms for Multidrug Resistant Transporters,” Nature, vol. 446, no. 7137, 2007, pp. 749–757.

8.

A. Yamatsu et al., “Isolation and Characterization of a Novel Poly(vinyl alcohol)-Degrading Bacterium, Sphyngopyxis sp. PVA3,” Applied Microbiological Biotechnology, vol. 72, no. 4, 2006, pp. 804–811.

Christopher M. Frenz is an instructor in the Department of Computer Engineering Technology at the New York City College of Technology (CUNY) and author of the books Visual Basic and Visual Basic .NET for Scientists and Engineers (Apress, 2002) and Pro Perl Parsing (Apress, 2005). His research interests include modeling protein structure and function relationships as well as the development of machine learning and artificial intelligence approaches to protein engineering. Contact him at [email protected].

Visit our Web site at www.computer.org/cise/ for more articles about supercomputing.

57

Petaflop Essay Contest

Let’s Get with the Program

Roughly 20 years ago, I wrote an essay for one of the many, many conferences about using vector supercomputers going on at the time. Here’s an excerpt: The availability of supercomputers has encouraged scientists to attempt computations which would have been considered impractical only a few years ago. In some cases, success has been achieved and extremely significant results have been obtained with the help of more powerful computers. In almost all cases, researchers have obtained some benefit from the use of fast machines. However, in many cases users have experienced inadequate speed-up in moving codes to vector machines.

By Francis Sullivan

I

n his most recent book, A Many­­Colored Glass: Reflections on the

Place of Life in the Universe, Freeman Dyson ­

comments

on

the

radical

changes in the world due to the invention of programmable computers. He notes that very powerful computational resources are now available even to individual researchers who aren’t members of large organizations with significant central computing facilities. This is in dramatic contrast to a very early prediction by John von Neumann that only 18 computers would suffice for the entire US. However, for individual researchers, things aren’t nearly as good as they could or should be. Although they do in principle have a lot of computational capability readily available, what they can actually get is a very small fraction of what they ought to get from the easy-to-afford machines currently sitting on their desks. Like everyone who started writing programs in 1960 or earlier, my first experience with computers was lifealtering. Suddenly, it was possible to do “impossible” computations, but the real revelation was how interesting computation became once I no longer had to do arithmetic by hand (or with a slide rule) but instead got to write a program. Of course, compared to modern machines, the IBM 650 I used was very simple. I wrote in SOAP, and when speaking this low-level language, I felt that I could control everything happening on the machine and make full use of every capability it had. But that was then, and this is now. Computers are very, very good at computing very, very quickly, and to me, this is the single most remarkable thing they do. It’s also the reason their effect on science and engineering has been and continues to be revolutionary. However, pure computation isn’t the main reason why people use computers—in fact, computing is probably pretty far down on the list. Computation for the sake of computational results must surely rank below word processing, email, and displaying Web pages, and I’m absolutely certain it ranks way below computer games. 58

... no programming language processor can reorganize and restructure applications programs in a fundamental way; yet major rethinking of algorithms is often required in order to actually attain the speeds of which supercomputers are capable.

In a sense, not much has changed since then. Or, to put it differently, things are as they were, only more so. The announced capability of machines is vastly greater than it was when I wrote that essay, but the difficulty of using them fully is also vastly greater. And that difficulty is about to increase because of multicore and related innovations. Unfortunately, scientific users who aren’t associated with large projects are drifting in limbo someplace between the paradise of big science and the inferno of role-playing games. There’s no point in grumbling about simple economic facts. Most computers aren’t sold for the purpose of computation, and giving general scientific users the ability to use a computer’s full power isn’t the main part of a computer manufacturer’s business plan. However, we should give some thought about how the computational science community should respond to this reality. Here are three ideas I think could help and should be encouraged: 1.

Develop simple, realistic, and descriptive measures of computer performance. Scientists using desktop machines need reliable sources of information about how a machine actually performs—something like Motor Week but for computers instead of cars. The Linpack benchmarks are interesting and useful in a limited way, but they aren’t designed to give information about real application performance. Projects aimed at developing the most advanced computing capabilities do, of course, have benchmarks keyed to real problems, but these aren’t useful to general us-

Computing in Science & Engineering

2.

3.

ers of desktop machines or clusters assembled by a small team. Include more training in algorithms for all disciplines that use scientific computing. In my essay 20 years ago, I made two claims about designing codes for highperformance computers: one, computer architecture determines algorithm design (it would be beneficial if this simple fact of life could be made more explicit in our thinking and teaching), and two, the real problem is often combinatorial. The most famous example of the latter is the fast Fourier transform, which appears to be a mere programming trick for reorganizing a calculation, but, in fact, no mathematical property of the Fourier transform itself is changed by reordering computation steps. The mathematics of complex roots of unity allows a specific order that reduces the computation from O(n2) to O(n ln n). Give the title “programmer” the same respect afforded to titles like “engineer,” “physicist,” “artist,” and “computer scientist.” The world of science would be an even better place if universities gave degrees in computer programming, but it’s usually viewed as an add-on skill. In many discussions about the qualifications of potential postdocs, for example, you hear phrases like, “Her published papers are excellent, and she’s also a good programmer,” or, “We didn’t extend his appointment because he spent all his time improving his program. (But we sure do wish he was still around!)”

The first is an R&D challenge—in fact, the issue has Advertiser Index November/December 2007 Advertiser

Page number

Palisade Corp. SIAM

Cover 2 Cover 3

Advertising Personnel Marion Delaney IEEE Media, Advertising Dir. Phone: +1 415 863 4717 Email: [email protected] Marian Anderson Advertising Coordinator Phone: +1 714 821 8380 Fax: +1 714 821 4010 Email: [email protected] Sandy Brown | IEEE Computer Society Business Development Mgr. Phone: +1 714 821 8380 Fax: +1 714 821 4010 Email: [email protected]

November/December 2007

been around for so long that I’d call it a grand challenge. The second, a suggestion about education, is probably easy to do for an individual instructor, but a large change is much more difficult to effect. The last is hardest of all because it’s an issue of social status. Many people can write programs but only a few of them are really what I’d call “programmers.” Becoming a real programmer requires training, experience, hard work, and that most elusive thing, talent. It’s time our community recognized that this special set of skills is central to everything we do. During a period when inspiration had fled, the poet most famous for his works in German, Rainer Maria Rilke, was advised to visit the zoo and look at an animal until he really saw it. The result was his celebrated and powerful verse “The Panther.” It seems that nearly every English speaker who’s read this poem has had a go at translating it, so I won’t try your patience with my version. It’s enough to say that in three amazing stanzas, Rilke describes a panther who’s grown so weary of looking at bars that he can see nothing but bars. He just prowls his cage in tight circles and when—on that rare occasion—an image does pass through the curtain over his eyes and reach his pupils, it enters his heart and dies there. In a few years, the largest institutions will have exaflop machines, and so desktop machines will be capable of petaflop speeds. Unless things change, these desktop machines won’t exhibit anything near petaflop performance for the vast majority of users. Then we’ll all be panthers, staring at spam while visions of tremendous computations die in our hearts. 

Advertising Sales Representatives Will Hamilton Phone: +1 269 381 2156 Fax: +1 269 381 2556 Mid Atlantic (product/recruitment) Email: [email protected] Dawn Becker Joe DiNardo Phone: +1 732 772 0160 Phone: +1 440 248 2456 Fax: +1 732 772 0164 Fax: +1 440 248 2594 Email: [email protected] Email: [email protected] New England (product) Southeast (recruitment) Jody Estabrook Thomas M. Flynn Phone: +1 978 244 0192 Phone: +1 770 645 2944 Fax: +1 978 244 0103 Fax: +1 770 993 4423 Email: [email protected] Email: [email protected] New England (recruitment) Southeast (product) John Restchack Bill Holland Phone: +1 212 419 7578 Phone: +1 770 435 6549 Fax: +1 212 419 7589 Fax: +1 770 435 0243 Email: [email protected] Email: [email protected] Connecticut (product) Midwest/Southwest (recruitment) Stan Greenfield Darcy Giovingo Phone: +1 203 938 2418 Phone: +1 847 498-4520 Fax: +1 203 938 3211 Fax: +1 847 498-5911 Email: [email protected] Email: [email protected] Midwest (product) Southwest (product) Dave Jones Steve Loerch Phone: +1 708 442 5633 Phone: +1 847 498 4520 Fax: +1 708 442 7620 Fax: +1 847 498 5911 Email: [email protected]

Email: [email protected] Northwest (product) Lori Kehoe Phone: +1 650 458 3051 Fax: +1 650 458 3052 Email: [email protected] Southern CA (product) Marshall Rubin Phone: +1 818 888 2407 Fax: +1 818 888 4907 Email: [email protected] Northwest/Southern CA (recruitment) Tim Matteson Phone: +1 310 836 4064 Fax: +1 310 836 4067 Email: [email protected] Japan Tim Matteson Phone: +1 310 836 4064 Fax: +1 310 836 4067 Email: [email protected] Europe (product) Hilary Turnbull Phone: +44 1875 825700 Fax: +44 1875 825701 Email: [email protected]

59