Software to unlock the power of grids

July 13, 2009

(PhysOrg.com) -- A huge amount of computing power sits idle most of the time, and new technologies enabling the sharing of resources aim to capitalise on that. Now European researchers have developed software to simultaneously run applications on very different IT infrastructures.

Unlocking the power of grid computing has become one of the top priorities in the IT world. Optimising the use of existing resources rather than deploying new infrastructure is becoming increasingly important in the current economic climate.

At the same time, applications are taking longer to execute using traditional computing methods, with the amount of data being processed increasing at a rapid pace. This presents a problem because processing power is not keeping up with demand.

Moore’s Law predicted that the processing power of integrated circuits, or chips, would roughly double every 18 months. This has held true from the late 1950s until recently - the exponential growth has finally slowed down as the miniaturisation of components is nearing its limit.

An EU-funded project, GridCOMP, has been working around this conundrum by developing some clever software and middleware which allows applications to run on several, or even many, computers at the same time regardless of different infrastructures and architectures.

Parallel computing the way forward

The project’s scientific coordinator Denis Caromel says: “We have been developing an open source suite of products to allow parallel, distributed, multi-core computing. What this means is that an application can be run on a series of machines, or a cluster of servers, or an enterprise grid comprising many machines and even all the way up to cloud computing, or any combination of these.”

An example of this would be the heavy-duty enterprise data processing used by financial institutions or telecoms operators. These huge applications typically take a long time to run and make often-inefficient use of a large amount of internal computing power.

What the project’s ProActive Parallel Software Suite does is to allow an organisation to use all of its computing hardware - wherever in the world it is based, and whatever manufacturer it is made by - to run an application simultaneously. If there is not enough computing power inhouse, then outside resources, such as cloud computing where processing power is rented from other companies, can also be deployed.

What is particularly clever about the software, though, is its automatic ability to monitor quality of service (QOS) on a real-time basis, and add or remove processing power as required. “If there is an expected QOS of (for example) several seconds, and the system sees this degrading to, say, a minute, then it will automatically add new resources, be they internal or external, for as long as they are needed,” Caromel explains.

Developing new standards

The project worked alongside the European standards body, ETSI, while developing the components of GridCOMP. During the course of the project, four standards were developed and approved by ETSI and are now officially available on the body’s standards sheet. The project was also supported by many of the big names in IT and telecoms, including IBM, Sun, Oracle, HP, NEC, Alcatel Lucent, top Chinese equipment maker Huawei, Nokia Siemens, British Telecom, France Telecom/Orange and Telefonica/O2.

Field trials of the software were also conducted to develop different use cases. One of these was with a major telco in the field of extended data record (EDR) processing. Information from the millions of calls made by customers is processed following a set of rules and, from the processed data, useful conclusions can be drawn and valuable business knowledge obtained.

Prior to GridCOMP, the transformation took a lot of time and required a considerable amount of computing resources. But with GridCOMP the telco was able to use multi-platform, low-cost hardware instead of single platform, expensive hardware.

GridCOMP also replaced high management costs with automatic management providing built-in redundancy and fault tolerance and made a process which was difficult and expensive into one which was cheap and easy to scale. And expensive, commercial, closed-source software was replaced by a free open-soft framework.

Cutting down on costs

A second use case was at IBM where a biometric identification system (BIS), based on fingerprint matching, was built using the project’s software. It was proved to work on a large user population in real time while being easily scalable and cost efficient.

The third use case involved keeping tables on unpaid invoices in a large organisation, while a fourth aimed to largely replace expensive wind tunnels in aircraft design with a computerised simulation. Both proved highly successful.

On the back of this success, a commercial entity - a professional open source company called ActiveEon - has also been spun off from the project to support users of the ProActive Parallel Software Suite and be their partners in accelerating and scaling applications.

Caromel expects it to be a success, as more and more companies look at ways to improve efficiency, cut costs and utilise existing resources to the full.

GridCOMP has been funded under the ICT strand of the EU’s Sixth Framework Programme for research.

More information: Watch videos of GridCOMP protoypes and use cases.