CERN plans for even more intense year of LHC physics

February 13, 2012 | 9:40 am

CERN scientists will begin running the Large Hadron Collider at a higher energy than ever before when this winter’s technical stop comes to a close in mid-March, the laboratory announced in a press release today.

Scientists and CERN management came to the decision to raise the LHC’s energy from 7 to 8 TeV after a week-long meeting in Chamonix, France.

Greater energy leads to a higher rate of collisions between protons in the LHC. The operation crew predicts that, with this increase in energy, the LHC experiments should gather more than twice as much data in 2012 as they did in 2011. This data increase is significant for Higgs searches and should give sufficient data by the end of 2012 to show or exclude the existence of a Higgs.

Raising the collision rate will cause the number of interesting physics events in general to go up, though scientists will have to sift them out from an even larger pile of non-interesting ones than before. Last year, for example, physicists on the ATLAS experiment saw 15 extra events for every one deemed worth studying. That number will likely double this year.

Scientists decided to run the LHC up until now at half the energy the machine was designed to handle. They made the decision to run at a lower energy after an accident that occurred when it first started in 2008. The accelerator’s magnets operate in a superconducting state and are cooled by liquid helium. Problems with an interconnect between magnets caused heating and a rapid expansion of helium, displacing about 50 magnets. To avoid risking another year-long shutdown for repairs, scientists restarted in 2010 at 7 TeV.

Operators are comfortable turning the machine up after running the LHC successfully at 7 TeV in 2011, improving their understanding of the interconnects and completing further testing, said Steve Myers, director of accelerators and technology at CERN. They will not raise the energy to 14 TeV until sometime after a more extensive shutdown of about 20 months at the end of 2012.

For the machine operators, the main challenge of going to 8 TeV and running with higher collision rates will be reducing the size of the particle beams at the collision points inside the detectors, said Mike Lamont, operations group leader for the LHC and its injectors. Squeezing beams at these places takes extra finesse and care.

The plethora of collision events will provide another challenge for the experimentalists, data-hungry as they may be. In order to process everything, the collaborations each use their own elaborate software simulations, called Monte Carlos after the locale famous for its games of chance. The simulations use statistics to predict the types of particles that will be created in collisions with different characteristics in the LHC. The scientists are constantly revising the algorithms to keep up with changes in machine operations.

Although the simulations are necessary, rewriting them and ensuring their accuracy can be a huge challenge for the experiments, said ATLAS physicist Bill Murray. Last September his collaboration began using a new set of Monte Carlos. In three and a half months, they were able to fully simulate a record-breaking 1.5 billion events. But the work was exhausting and just barely finished in time for the December Higgs update.

Now, with the imminent energy increase to 8 TeV, the scientists will have to wrangle with the same issue of adjusting software and finishing analysis for the summer conferences as well. Debugging and simulating events shouldn’t be a problem; it’s a matter of how much time the work will take. With the Higgs race on and thoughts that this could be a Nobel-worthy year, no particle physicist wants to slow down. “It’s going to be extremely painful in May and June,” Murray said. “But it should be entertaining.”

The software used by CMS, the other collaboration contending for a discovery, takes only about half the amount of processing power as the software used by ATLAS, Murray said. It may be easier for CMS to get Higgs results sooner, he said — not that it will stop his collaboration from trying harder than ever.

“By 2014 the downsides [to having to redo the collaboration’s Monte Carlos] should disappear,” Murray said, “So for me, the downsides are short-term things. In the end we should have more data at higher energy, so a greater physics reach.”

Beams will commence around March 15 and first collisions at full energy should be seen after about three weeks. Then the hunt for Higgs and other physics will shift into the highest gear yet.

Amy Dusto

No Comments »

CERN hosts Swiss semi-finals for international science communication competition

February 9, 2012 | 9:00 am

Science doesn’t happen in a bubble, but sometimes new scientific knowledge seems to remain in one. International science communications competition FameLab is working to burst that bubble for the benefit of the public.

CERN hosted the Swiss semi-finals for FameLab, run jointly by the Cheltenham Festivals and the British Council, in Geneva on Saturday, Feb. 4. Of the 21 researchers and students who braved the frigid weather to take a turn sharing their knowledge on stage, five will advance to the national finals in Zurich at the end of March. Twenty-two countries are hosting regional competitions this year to select a representative for the ultimate showdown. The televised finals will take place this summer in Cheltenham.

To compete, entrants from scientific backgrounds must explain a complex topic to a panel of judges and a live audience in three minutes or less.

“Now I’ll know how to explain my research to my parents,” said entrant and University of Geneva immunology postdoc Susan Johnson.

As well as awarding prizes and bestowing glory, FameLab is also designed as a workshop event to help train the next generation of science communicators.

Before the first set of auditions last Saturday, the participants went through blitz training sessions with experienced communicators and FameLab aficionados. They practiced their speeches in front of cameras while two communicators from CERN offered critique. They also did body and intonation exercises with a nonverbal communications specialist from Geneva, Branca Zei. Finally, two previous national winners, Tom Whyntie from the U.K. and Venelin Kozhuharov from Bulgaria, were available throughout the day to lend insights and encouragement to all.

Sharing one’s work is a responsibility, said Diana Marek, a geneticist from nearby Lausanne; researchers must give something back to the taxpayers who fund them. She enjoys the task, she said. “I find it very rewarding to see in the eyes of people that they got something out of what you said.”

FameLab’s judging criteria are based on “the three C’s”: content, clarity and charisma. Speakers may use a limited number of props but no PowerPoint or other electronic presentation tools. Immediately after each audition, the four judges commented aloud about what did and didn’t work.

“We were skeptical about giving feedback right after, but actually it worked quite well,” said judge Muriel Brouet of the Geneva Tribune. Another judge, Didier Raboud of the University of Geneva communications department, said the speakers clearly learned a lot in the course of the day. Indeed, in closing comments all the judges noted a marked improvement between the morning and afternoon performances.

The participants also remarked on the instruction’s benefits. “I think everyone should do this — especially professors,” said cosmologist Christian Byrnes, whose audition explained Olber’s paradox about why the night sky is dark. CERN physicist and event judge John Ellis said he would steal Byrnes’ idea in the future when explaining the same concept to the public.

“I got so many great ideas from everyone,” said another participant, Philippe Kobel. “It’s inspiring and gives you the enthusiasm to keep on trying.”

Along with the five finalists to compete in the Zurich nationals — which will be preceded by a weekend-long communications workshop free of charge — the panel chose one grand winner of the semi-finals for that day. Physicist Boris Lemmer, of the ATLAS experiment, took the iPad prize after his talk tilted “There is a particle physicist in every one of us.”

In it, he made an analogy between the way LHC detectors select interesting events and the way someone might search for a girlfriend in downtown Geneva. As Ellis noted on behalf of the judges, the thing that tipped the jury was “the way he talked about particle physics in the language of every day life.”

Lemmer said he was ecstatic to have won, but he had a thoughtful take on his experience at the end of the day. The best part, he said, was meeting and learning from the other scientists who attended. “All these people want to talk about their science,” he said, “And that’s the way it should be.”

Watch a video of the afternoon session, including all ten finalist presentations: http://cdsweb.cern.ch/record/1421673

Amy Dusto

No Comments »

Reader challenge: My physical romance

February 7, 2012 | 5:00 am

Chemistry gets all the press around Valentine’s Day, but for some, real romance springs from physics.

Before next week’s holiday, we at symmetry breaking want to know about your affair with physics. Send us a love letter (or “Dear John” letter) about your research, a playful pun about a physical concept, or a story about a connection you’ve made with a fellow scientist. Post your comments here or send them to scharley@fnal.gov. We will publish our favorites on Feb. 14.

What is it about physics that ignites our passions and enchants our hearts? Perhaps it’s the inherent romantic nature of the subject. For example, even though there is no “physical” relationship between love and entanglement, the figurative metaphor is enticing: two kindred spirits with entangled hearts, their two quantum spin states forever interlocked. Of course their two quantum spin states might be opposite once a measurement is made, but perhaps that’s just how love works.

You could make a similar conclusion about quantum coupling and dating. Or even conjecture that the plot of every romantic comedy is merely a dramatic interpretation of the strong nuclear force (particles pulled together despite their initial repulsions for one another—sound familiar?)

But maybe our love for physics goes deeper than the superficial puns we can make about various classical concepts. Perhaps deep down, all physicists are hopeless romantics, continually seeking that ideal and perfect relationship that will bring meaning to their lives (and data).

Tell us your story (or give us your best joke). Share the love.

Sarah Charley

3 Comments »

Introducing LHC Lunch

February 2, 2012 | 8:30 am

Editor’s note: This article comes from US LHC intern Amy Dusto, who is currently working as a communicator at CERN. She is introducing LHC Lunch, a series of articles and videos she created while getting to know some of the members of experiments at the Large Hadron Collider from U.S. institutions.

The busy cafeteria known as Restaurant 1 is generally The Place to meet anyone at CERN. So, in order to gather stories from U.S.-based researchers working here, I met with eight of them over lunch.

The people I interviewed are experimentalists, students, professors and career changers. They represent each of the four main experiments: ALICE, ATLAS, CMS and LHCb. They followed different paths to CERN but share similar goals in science.

I’ve written a story for each of them, captured their voices on camera and – why not? – also documented what they ate. All this is set to appear on a new page of the US LHC website called LHC Lunch.

Every Tuesday and Thursday this month, LHC Lunch will add another physicist to the page. Today we begin with University of Michigan graduate student Shannon Walch, who has been energized by the mysteries of physics since her high school days in Highland, Utah.

The series will also feature:

• Ricardo Vasquez Sierra, a CMS physicist who was intimately involved in a one-of-a-kind installation deep underground before the LHC first started running.
• Anna Phan, a postdoc from LHCb who has flown around the world for physics.
• Mauro Cosentino, a former banker who has found happiness in research at ALICE.
• Helena Malbouisson, a CMS researcher from Brazil who grew up in a family of physicists and artists before picking her own path in the study of particles.
• Sheldon Stone, a professor who balances teaching at Syracuse with research at CERN in the LHCb experiment.
• Jacob Searcy, a graduate student on ATLAS who is inspired to do what he does by considering the big picture.
• Peter Jacobs, an ALICE researcher who has watched CERN’s rise over his decades of work in nuclear and particle physics.

You can find them all here as they appear. See you at lunch!

Amy Dusto

1 Comment »

Calculating the Universe

February 1, 2012 | 9:42 am

This story appeared today in isgtw.

Since 2000, the three Sloan Digital Sky Surveys (SDSS I, II, and III) have surveyed well over a quarter of the night sky, producing the biggest 3-D color map of the Universe ever made. Now, scientists have used this visual information for the most accurate computation yet of how matter clumped together – from a time when the universe was only half its present age until now.

“The way galaxies cluster together over vast expanses of the sky tells us how both ordinary visible matter and underlying invisible dark matter are distributed, across space and back in time,” said Shirley Ho, an astrophysicist at Lawrence Berkeley National Laboratory and Carnegie Mellon University who led the work. “The distribution gives us cosmic rulers to measure how the universe has expanded, and a basis for calculating what’s in it: how much dark matter, how much dark energy, even the mass of the hard-to-see neutrinos it contains. What’s left over is the ordinary matter and energy we’re familiar with.”

For the present study, Ho and her colleagues first selected 900,000 luminous galaxies from among over 1.5 million such galaxies gathered by the Baryon Oscillation Spectrographic Survey, or BOSS, the largest component of the still-ongoing SDSS III. Most of these are ancient red galaxies, which contain only red stars because all their faster-burning stars are long gone, and which are exceptionally bright and visible at great distances. The galaxies chosen for this study populate the largest volume of space ever used for galaxy clustering measurements. Their brightness was measured in five different colors, allowing the redshift of each to be estimated.

“By covering such a large area of sky and working at such large distances, these measurements are able to probe the clustering of galaxies on incredibly vast scales, giving us unprecedented constraints on the expansion history, contents, and evolution of the universe,” said Berkeley Lab’s Martin White, chair of the BOSS science survey teams. “The clustering we’re now measuring on the largest scales also contains vital information about the origin of the structure we see in our maps, all the way back to the epoch of inflation, and it helps us to constrain – or rule out – models of the very early universe.”

After augmenting their study with information from other data sets, the team derived a number of such cosmological constraints (measurements of the universe’s contents based on different cosmological models). Among the results: in the most widely accepted model, the researchers found – to less than two percent uncertainty – that dark energy accounts for 73 % of the density of the universe.

The team’s results are presented 11 January at the annual meeting of the American Astronomical Society in Austin, Texas, and have been submitted to the Astrophysical Journal. They are currently available online at http://arxiv.org/abs/1201.2137.

Read on at isgtw.org.

- Paul Preuss

Guest author

No Comments »

Fermilab sounds debut in “Alternative Energy”

January 31, 2012 | 9:33 am

This story appeared today in Fermilab Today.

Most Fermilab personnel have learned to ignore the ubiquitous booms, hums, growls and crackles of Fermilab machinery. But composer Mason Bates places these sounds center stage in his new piece “Alternative Energy.”

“Alternative Energy” is an imaginative musical narrative that follows the evolution of energy and technology. The Chicago Symphony Orchestra will perform the piece at 8 p.m. from Feb. 2 through 4 and Feb. 7.

“The idea was that each movement would be separated by a hundred years, starting with old energy and moving to present and future energy,” Bates said about his newest symphony. The first movement in the show uses scrap metal to evoke a junkyard.

Bates came to Fermilab last spring seeking inspiration for the present-day movement of his piece. He was not disappointed. Bates found a variety of sounds to give a modern-twist to his dynamic orchestral symphony.

“Fermilab exists at the intersection of technological power and human curiosity, and I wanted the symphony to include an example of massive energy used in a positive way. When we hear a surround-sound recreation of the Tevatron booting up — a massive machine spins around the orchestra — it is as if the crank on an old Model T suddenly grew to be several acres in size,” Bates said.

Physicist Todd Johnson gave Bates a behind-the-scenes tour of Fermilab and revved different machinery while Bates eagerly listened and recorded.

“He was very enthusiastic and asked a lot of questions. He was obviously really happy to be here,” Johnson said.

Bates not only enjoyed his visit to Fermilab, but found exactly what he was looking for sonically.

“I was blown away by the beautiful architecture of the main building and the sculptures scattered around. On a sonic level, I was astounded at the variety of noises that jostle out of this huge facility. I had hoped to find the sounds of massive machines, and I found that in one of the refrigeration units,” Bates said. “And when Todd told me about the mysterious, capricious ‘quench,’ I told him that we needed to find a way to capture that. He contacted Derek Plant who, unbelievably, was able to set up various recording devices in just the right places. We got it!”

Bates features the sounds he recorded at Fermilab in the present day movement of his symphony. He called “Alternative Energy” his biggest piece to date. For tickets, visit the Chicago Symphony Orchestra’s website.

Sarah Charley

No Comments »

Fermilab plans for a future of discovery

January 26, 2012 | 10:58 am

The only laboratory in the United States dedicated entirely to particle physics recently released its plan for the next two decades.

According to the document:

The keys to Fermilab’s long-term future are two facilities that could be operating in the 2020s: the Long-Baseline Neutrino Experiment and Project X.

LBNE will take the next major step in the quest to measure and understand the properties of neutrinos and determine their connection to the observed excess of matter over antimatter in the universe.

The Project X accelerator complex will be unique in the world in its ability to simultaneously deliver high-intensity proton beams in different formats to multiple experimental areas. Project X experiments using neutrinos, muons, kaons and nuclei will provide new windows on phenomena not accessible at particle colliders, and will be essential to break through to a deeper understanding of nature and the origins of matter.

Fermilab has proposed building detectors for LBNE at Sanford Underground Laboratory in Lead, South Dakota. The laboratory hopes to construct Project X on its campus.

In the near future, Fermilab will upgrade its accelerator complex to double the intensity of its proton beams, which scientists use to create beams of other particles such as neutrinos.

Read the full document, “A Plan for Discovery.”

Kathryn Grim

No Comments »

Scientists finish installation of 80-ton ‘particle thermometer’ at ALICE detector

January 24, 2012 | 8:50 am

Scientists on the ALICE experiment at the Large Hadron Collider just completed the installation of a crucial component for tracking high-energy particle jets. Without it, physicists would be lacking critical tools to select which events out of billions to store and analyze.

Engineers and physicists around the world worked intensively over five years to complete the electromagnetic calorimeter, or EMCal. The United States, supported by the Department of Energy’s Nuclear Physics Office, contributed 70 percent of the project costs. Scientists installed the last two pieces of the 80-ton device on Jan. 18.

The EMCal’s heft comes from its many sheets of lead absorbers, which it needs to stop particles coming from collisions in the detector in order to measure their energy. “The calorimeter measures the energy of individual photons and electrons,” said ALICE physicist Peter Jacobs. “It’s a sort of particle thermometer.”

The ALICE detector’s calorimeter was specifically designed to study the most complex collisions at the LHC, those created using beams of heavy ions. These collisions recreate big-bang-like conditions and produce events with many more particles than the Large Hadron Collider’s usual collisions using beams of protons.

CERN typically smashes lead ions together each November. These collisions produce a goopy mixture, known as the quark-gluon plasma, in the center of ALICE. Occasionally, a very energetic quark or gluon, called a jet, will also be created in the collision. When this happens, the QGP gets in its way, and that interaction is important for researchers seeking to understand material which first existed in the earliest moments of the universe. The EMCal allows ALICE to select and record the rare events containing such jets, and to measure their properties precisely.

A second arm of the EMCal will be added to ALICE during the long LHC shutdown in 2013.

The two pieces of the EMCal scientists installed this year were small; they add only about 10 percent to the calorimeter’s overall coverage, Jacobs said. However, all the small parts do add up — every new measurement gets us a little closer to the heart of the matter.

Symmetry caught up with researcher Peter Jacobs underground during the end of the installation.

Amy Dusto

No Comments »

Cutting-edge accelerator design gets results 60 years later

January 20, 2012 | 11:33 am

At Daresbury Laboratory in England, a team of scientists recently completed the first successful runs on a new prototype that may change the way accelerators speed up particles. Its novel design, scientists said, is capable of energies beyond the reach of current cyclotrons, with acceleration rates exceeding those of the most powerful synchrotrons – all within a compact, cost-effective and operationally simple package.

Daresbury’s accelerator, called EMMA, gains its technological edge through a high-intensity accelerator concept nearly abandoned a half century ago.

The accelerator’s magnet configuration incorporates a fixed field, rather than a pulsed field like a cyclotron, but includes alternating gradients like a synchrotron. The first versions of these fixed-field alternating-gradient machines were invented independently in Japan, Russia and the U.S. in the early 1950s.

The U.S. FFAG came about through the Midwest Universities Research Association, a group of 15 universities dedicated to developing a machine capable of accelerating particles beyond a billion electron volts, or 1 GeV. This unique institution soon built the first FFAG prototypes, albeit low-energy models.

“It was an exceedingly clever group of people. They had lots of ideas,” said Alvin Tollestrup, a long-time Fermilab physicist who attended MURA’s FFAG workshop.

For nearly 10 years, the MURA team pushed for a high-energy, high-intensity FFAG accelerator. One of several designs the group developed included colliding beams that essentially doubled the collision energy. They submitted a number of their proposals for machines in the 10-20 GeV range to the Atomic Energy Commission. None were approved.

Meanwhile, the invention of the storage ring and cascaded synchrotrons significantly reduced the cost of high-energy accelerators. As a consequence, a 1963 report, chaired by Norman Ramsey, to the Atomic Energy Commission ranked the FFAG project lower in priority than the competing proposals for high-energy particle beams.

Under rising excitement over a new strong-focusing synchrotron design that could achieve 300 GeV, and in the heat of an increasingly contentious political climate, the new president, Lyndon B. Johnson, rejected the MURA proposal on the basis of the report. MURA then disbanded, with the scientists shifting to accelerator laboratories like Fermilab, Berkeley and Brookhaven.

Although the technological advances and heavy influence of the MURA team did in part lead to the construction of Fermilab in the Midwest, the FFAG fell into obscurity.

Decades later, in a 1997 UCLA workshop on muon colliders, Fermilab physicist Carol Johnstone, while collaborating with one of the original MURA physicists, Fred Mills, proposed a new type of FFAG, termed non-scaling.

The FFAGs of the ‘50s followed a strict scaling method for the magnetic field to confine and accelerate beam. In the quest for a rapid acceleration scheme for unstable particles began a project that would for the first time ever combine FFAG with a new non-scaling method of beam confinement that resulted in smaller, simpler magnets.

The U.K.’s drive for an affordable collider that would accelerate and collide heavy elementary particles called muons set the stage for the prototype to be built at Daresbury Laboratory by a volunteer team of international scientists. Applied to a muon collider, the new non-scaling FFAG design would rapidly accelerate and inject the short-lived muon particles into storage or a collision before they decayed away.

“Emma was a bandwagon effect. It attracted accelerator physicists from across the world,” Johnstone said. “It’s the new technology that attracts the best.”

This FFAG format has the potential to quickly reach energies higher than 1 GeV, though the EMMA electron prototype tested at a moderate 20 million electron volts. The success of the preliminary runs put to rest decades of skepticism over FFAG technology. At a time when laboratories are pushing for more high-intensity experiments, the new approach is piquing interest among investors.

“Now that EMMA works, it’s considered a breakthrough,” Johnstone said. “And it’s a proof of principle for industry, too.”

Eventually EMMA’s method could be applied to a broad spectrum of accelerators. In medicine, the lower cost, versatile design and higher performance would enhance and expand proton and ion cancer therapy. For nuclear power, this accelerator-based approach could generate energy more safely while reusing old waste stockpiles. Along with muon colliders, a more cost-efficient generation of accelerators would greatly enhance experiments requiring high-energy neutrino beams.

“It’s interesting, I think, that something that was cooked up and kicked around 50 years ago all of a sudden could become quite interesting,” said Alvin Tollestrup. “There’s a number of interesting ideas for accelerators that haven’t been explored yet that could really change the way we accelerate particles.”

The journal Nature Physics recently covered the science behind EMMA in detail. For more history on FFAG, MURA and Fermilab, read “Fermilab: Physics, the Frontier, and Megascience,” by Lillian Hoddeson, Adrienne Kolb and Catherine Westfall.

Brad Hooker

1 Comment »

The Tevatron’s enduring computing legacy

January 18, 2012 | 3:53 pm

This story appeared Dec. 21 in iSGTW.

This is the first part of a two-part series on the contribution Tevatron-related computing has made to the world of computing. This part begins in 1981, when the Tevatron was under construction, and brings us up to recent times. The second part will focus on the most recent years, and look ahead to future analysis.

Few laypeople think of computing innovation in connection with the Tevatron particle accelerator, which shut down earlier this year. Mention of the Tevatron inspires images of majestic machinery, or thoughts of immense energies and groundbreaking physics research, not circuit boards, hardware, networks, and software.

Yet over the course of more than three decades of planning and operation, a tremendous amount of computing innovation was necessary to keep the data flowing and physics results coming. In fact, computing continues to do its work. Although the proton and antiproton beams no longer brighten the Tevatron’s tunnel, physicists expect to be using computing to continue analyzing a vast quantity of collected data for several years to come.

When all that data is analyzed, when all the physics results are published, the Tevatron will leave behind an enduring legacy. Not just a physics legacy, but also a computing legacy.

In the beginning: The fixed-target experiments

1981. The first Indiana Jones movie is released. Ronald Reagan is the U.S. President. Prince Charles makes Diana a Princess. And the first personal computers are introduced by IBM, setting the stage for a burst of computing innovation.

Meanwhile, at the Fermi National Accelerator Laboratory in Batavia, Illinois, the Tevatron has been under development for two years. And in 1982, the Advanced Computer Program formed to confront key particle physics computing problems. ACP tried something new in high performance computing: building custom systems using commercial components, which were rapidly dropping in price thanks to the introduction of personal computers. For a fraction of the cost, the resulting 100-node system doubled the processing power of Fermilab’s contemporary mainframe-style supercomputers.

“The use of farms of parallel computers based upon commercially available processors is largely an invention of the ACP,” said Mark Fischler, a Fermilab researcher who was part of the ACP. “This is an innovation which laid the philosophical foundation for the rise of high throughput computing, which is an industry standard in our field.”

The Tevatron fixed-target program, in which protons were accelerated to record-setting speeds before striking a stationary target, launched in 1983 with five separate experiments. When ACP’s system went online in 1986, the experiments were able to rapidly work through an accumulated three years of data in a fraction of that time.

Entering the collider era: Protons and antiprotons and run one

1985. NSFNET (National Science Foundation Network), one of the precursors to the modern Internet, is launched. And the Tevatron’s CDF detector sees its first proton-antiproton collisions, although the Tevatron’s official collider run one won’t begin until 1992.

The experiment’s central computing architecture filtered incoming data by running Fortran-77 algorithms on ACP’s 32-bit processors. But for run one, they needed more powerful computing systems.

By that time, commercial workstation prices had dropped so low that networking them together was simply more cost-effective than a new ACP system. ACP had one more major contribution to make, however: the Cooperative Processes Software.

CPS divided a computational task into a set of processes and distributed them across a processor farm – a collection of networked workstations. Although the term “high throughput computing” was not coined until 1996, CPS fits the HTC mold. As with modern HTC, farms using CPS are not supercomputer replacements. They are designed to be cost-effective platforms for solving specific compute-intensive problems in which each byte of data read requires 500-2000 machine instructions.

CPS went into production-level use at Fermilab in 1989; by 1992 it was being used by nine Fermilab experiments as well as a number of other groups worldwide.

1992 was also the year that the Tevatron’s second detector experiment, DZero, saw its first collisions. DZero launched with 50 traditional compute nodes running in parallel, connected to the detector electronics; the nodes executed filtering software written in Fortran, E-Pascal, and C.

Gearing up for run two

1990. CERN’s Tim Berners-Lee launches the first publicly accessible World Wide Web server using his URL and HTML standards. One year later, Linus Torvalds releases Linux to several Usenet newsgroups. And both DZero and CDF begin planning for the Tevatron’s collider run two.

Between the end of collider run one in 1996 and the beginning of run two in 2001, the accelerator and detectors were scheduled for substantial upgrades. Physicists anticipated more particle collisions at higher energies, and multiple interactions that were difficult to analyze and untangle. That translated into managing and storing 20 times the data from run one, and a growing need for computing resources for data analysis.

Enter the Run Two Computing Project (R2CP), in which representatives from both experiments collaborated with Fermilab’s Computing Division to find common solutions in areas ranging from visualization and physics analysis software to data access and storage management.

R2CP officially launched in 1996. It was the early days of the dot com era. eBay had existed for a year, and Google was still under development. IBM’s Deep Blue defeated chess master Garry Kasparov. And Linux was well-established as a reliable open-source operating system. The stage is set for experiments to get wired and start transferring their irreplaceable data to storage via Ethernet.

“It was a big leap of faith that it could be done over the network rather than putting tapes in a car and driving them from one location to another on the site,” said Stephen Wolbers, head of the scientific computing facilities in Fermilab’s computing sector. He added ruefully, “It seems obvious now.”

The R2CP’s philosophy was to use commercial technologies wherever possible. In the realm of data storage and management, however, none of the existing commercial software met their needs. To fill the gap, teams within the R2CP created Enstore and the Sequential Access Model (SAM, which later stood for Sequential Access through Meta-data). Enstore interfaces with the data tapes stored in automated tape robots, while SAM provides distributed data access and flexible dataset history and management.

By the time the Tevatron’s run two began in 2001, DZero was using both Enstore and SAM, and by 2003, CDF was also up and running on both systems.

Linux comes into play

The R2CP’s PC Farm Project targeted the issue of computing power for data analysis. Between 1997 and 1998, the project team successfully ported CPS and CDF’s analysis software to Linux. To take the next step and deploy the system more widely for CDF, however, they needed their own version of Red Hat Enterprise Linux. Fermi Linux was born, offering improved security and a customized installer; CDF migrated to the PC Farm model in 1998.

Fermi Linux enjoyed limited adoption outside of Fermilab, until 2003, when Red Hat Enterprise Linux ceased to be free. The Fermi Linux team rebuilt Red Hat Enterprise Linux into the prototype of Scientific Linux, and formed partnerships with colleagues at CERN in Geneva, Switzerland, as well as a number of other institutions; Scientific Linux was designed for site customizations, so that in supporting it they also supported Scientific Linux Fermi and Scientific Linux CERN.

Today, Scientific Linux is ranked 16th among open source operating systems; the latest version was downloaded over 3.5 million times in the first month following its release. It is used at government laboratories, universities, and even corporations all over the world.

“When we started Scientific Linux, we didn’t anticipate such widespread success,” said Connie Sieh, a Fermilab researcher and one of the leads on the Scientific Linux project. “We’re proud, though, that our work allows researchers across so many fields of study to keep on doing their science.”

Grid computing takes over

As both CDF and DZero datasets grew, so did the need for computing power. Dedicated computing farms reconstructed data, and users analyzed it using separate computing systems.

“As we moved into run two, people realized that we just couldn’t scale the system up to larger sizes,” Wolbers said. “We realized that there was really an opportunity here to use the same computer farms that we were using for reconstructing data, for user analysis.”

Today, the concept of opportunistic computing is closely linked to grid computing. But in 1996 the term “grid computing” had yet to be coined. The Condor Project had been developing tools for opportunistic computing since 1988. In 1998, the first Globus Toolkit was released. Experimental grid infrastructures were popping up everywhere, and in 2003, Fermilab researchers, led by DZero, partnered with the US Particle Physics Data Grid, the UK’s GridPP, CDF, the Condor team, the Globus team, and others to create the Job and Information Management system, JIM. Combining JIM with SAM resulted in a grid-enabled version of SAM: SAMgrid.

“A pioneering idea of SAMGrid was to use the Condor Match-Making service as a decision making broker for routing of jobs, a concept that was later adopted by other grids,” said Fermilab-based DZero scientist Adam Lyon. “This is an example of the DZero experiment contributing to the development of the core Grid technologies.”

By April 2003, the SAMGrid prototype was running on six clusters across two continents, setting the stage for the transition to the Open Science Grid in 2006.

From the Tevatron to the LHC – and beyond

Throughout run two, researchers continued to improve the computing infrastructure for both experiments. A number of computing innovations emerged before the run ended in September 2011. Among these was CDF’s GlideCAF, a system that used the Condor glide-in system and Generic Connection Brokering to provide an avenue through which CDF could submit jobs to the Open Science Grid. GlideCAF served as the starting point for the subsequent development of a more generic glidein Work Management System. Today glideinWMS is used by a wide variety of research projects across diverse research disciplines.

Another notable contribution was the Frontier system, which was originally designed by CDF to distribute data from central databases to numerous clients around the world. Frontier is optimized for applications where there are large numbers of widely distributed clients that read the same data at about the same time. Today, Frontier is used by CMS and ATLAS at the LHC.

“By the time the Tevatron shut down, DZero was processing collision events in near real-time and CDF was not far behind,” said Patricia McBride, the head of scientific programs in Fermilab’s computing sector. “We’ve come a long way; a few decades ago the fixed-target experiments would wait months before they could conduct the most basic data analysis.”

One of the key outcomes of computing at the Tevatron was the expertise developed at Fermilab over the years. Today, the Fermilab computing sector has become a worldwide leader in scientific computing for particle physics, astrophysics, and other related fields. Some of the field’s top experts worked on computing for the Tevatron. Some of those experts have moved on to work elsewhere, while others remain at Fermilab where work continues on Tevatron data analysis, a variety of Fermilab experiments, and of course the LHC.

The accomplishments of the many contributors to Tevatron-related computing are noteworthy. But there is a larger picture here.

“Whether in the form of concepts, or software, over the years the Tevatron has exerted an undeniable influence on the field of scientific computing,” said Ruth Pordes, Fermilab’s head of grids and outreach. “We’re very proud of the computing legacy we’ve left behind for the broader world of science.”

Miriam Boon

No Comments »