What to do with 15 million gigabytes of data

November 3, 2008

When it is fully up and running, the four massive detectors on the new Large Hadron Collider (LHC) at the CERN particle-physics lab near Geneva are expected to produce up to 15 million gigabytes, aka 15 petabytes, of data every year. Andreas Hirstius, manager of CERN Openlab and the CERN School of Computing, explains in November's Physics World how computer scientists have risen to the challenge of dealing with this unprecedented volume of data.

When CERN staff first considered how they might deal with the large volume of data that the huge collider would produce when its two beams of protons collide, in the mid-1990s, a single gigabyte of disk space still cost a few hundred dollars and CERN's total external connectivity was equivalent to just one of today's broadband connections.

It quickly became clear that computing power at CERN, even taking Moore's Law into account, would be significantly less than that required to analyse LHC data. The solution, it transpired during the 1990s, was to turn to "high-throughput computing" where the focus is not on shifting data as quickly as possible from A to B but rather from shifting as much information as possible between those two points.

High-performance computing is ideal for particle physics because the data produced in the millions of proton-proton collisions are all independent of one another - and can therefore be handled independently. So, rather than using a massive all-in-one mainframe supercomputer to analyse the results, the data can be sent to separate computers, all connected via a network.

From here sprung the LHC Grid. The Grid, which was officially inaugurated last month, is a tiered structure centred on CERN (Tier-0), which is connected by superfast fibre links to 11 Tier-1 centres at places like the Rutherford Appleton Laboratory (RAL) in the UK and Fermilab in the US. More than one CD's worth of data (about 700 MB) can be sent down these fibres to each of the Tier-1 centres every second.

Tier 1 centres then feed down to another 250 regional Tier-2 centres that are in turn accessed by individual researchers through university computer clusters and desktops and laptops (Tier-3).

As Andreas Hirstius writes, "The LHC challenge presented to CERN's computer scientists was as big as the challenges to its engineers and physicists. The computer scientists managed to develop a computing infrastructure that can handle huge amounts of data, thereby fulfilling all of the physicists' requirements and in some cases even going beyond them."

Source: Institute of Physics

4.4 /5 (26 votes)  

Filter


Move the slider to adjust rank threshold, so that you can hide some of the comments.


Display comments: newest first

kerry
Nov 03, 2008

Rank: 4.8 / 5 (4)
Awesome! It's great to see scientists and engineers from all different fields coming together to work towards a single goal. This makes me happy =)
physpuppy
Nov 03, 2008

Rank: 5 / 5 (1)
Science has become very inter-disciplinary. While it is possible to study your own little niche and stay there, to be successful you need to branch out (or collaborate with other fields). Some of the most successful scientists are those who are excellent in their field but look at other fields for ideas.
QubitTamer
Nov 03, 2008

Rank: 5 / 5 (3)
Sounds like a very smart plan and a very cool network they built.

Too bad the whole place will be destroyed soon by strangelets or stable quantum black holes that don't evaporate via Hawking radiation.

I am kidding... I hope!
tkjtkj
Nov 04, 2008

Rank: 2 / 5 (1)
This story is a year old.
It is , no doubt,even older for
the people working on the issue.

weewilly
Nov 04, 2008

Rank: not rated yet
As a tree reaches up and out to intertwine with the atmosphere and it's root system grows a strong anchor into the earth so will it be with the sciences, growing, helping, supporting one another.
Rank 4.4 /5 (26 votes)
Related Stories
Relevant PhysicsForums posts

More news stories

Explained: Sigma

It's a question that arises with virtually every major new finding in science or medicine: What makes a result reliable enough to be taken seriously? The answer has to do with statistical significance -- but ...

Physics / General Physics

created Feb 09, 2012 | popularity 5 / 5 (20) | comments 76

Quantum physicist explains $100K offer for proof scaled-up quantum computing is impossible

(PhysOrg.com) -- MIT researcher Scott Aaronson has certainly riled the physics community with his offer this past Friday, of $100,000 to anyone who can prove that scaled-up quantum computing is impossible. ...

Physics / Quantum Physics

created Feb 08, 2012 | popularity 4.3 / 5 (14) | comments 36 | with audio podcast weblog

Diamond light, brighter than the sun

It’s the size of five football pitches and generates light 10 billion times brighter than the sun. As the Diamond Light Source celebrates its tenth anniversary this year, Penny Bailey visits one of the ...

Physics / General Physics

created Feb 07, 2012 | popularity 4.4 / 5 (9) | comments 18 | with audio podcast

Physicists 'record' magnetic breakthrough

An international team of scientists has demonstrated a revolutionary new way of magnetic recording which will allow information to be processed hundreds of times faster than by current hard drive technology.

Physics / General Physics

created Feb 07, 2012 | popularity 4.6 / 5 (43) | comments 14 | with audio podcast

Hints of the Higgs - papers are submitted

Back in December 2011, the ATLAS and CMS experiments at CERN presented some exciting results that provided tantalising hints of the Higgs boson.

Physics / General Physics

created Feb 08, 2012 | popularity 4.3 / 5 (8) | comments 10


Scientists discover molecular secrets of 2,000-year-old Chinese herbal remedy

For roughly two thousand years, Chinese herbalists have treated Malaria using a root extract, commonly known as Chang Shan, from a type of hydrangea that grows in Tibet and Nepal. More recent studies suggest that halofuginone, ...

New method to examine batteries -- MRI from the inside

There is an ever-increasing need for advanced batteries for portable electronics, such as phones, cameras, and music players, but also to power electric vehicles and to facilitate the distribution and storage of energy derived ...

Lab study raises questions over nano-particle impact

Tests involving chickens have raised questions about the impact on health from engineered nano-particles, the ultra-fine grains commonly used in drugs and processed foods, scientists said on Sunday.

Google might launch Drive for cloud storage soon

(PhysOrg.com) -- Google's next big move, according to the Wall Street Journal, is a cloud storage service called Drive. Hardly first to the plate, Google is simply catching up to introducing its cloud reposi ...

A mitosis mystery solved: How chromosomes align perfectly in a dividing cell

Although the process of mitotic cell division has been studied intensely for more than 50 years, Whitehead Institute researchers have only now solved the mystery of how cells correctly align their chromosomes during symmetric ...

Starve a virus, feed a cure? Findings show how some cells protect themselves against HIV

A protein that protects some of our immune cells from the most common and virulent form of HIV works by starving the virus of the molecular building blocks that it needs to replicate, according to research published online ...