Finding patterns of importance in a deluge of data

January 9, 2007

Dartmouth engineers George Cybenko and Vincent Berk think that PQS, or process query systems, are the way to go to make sense of the huge volume of data we collect each day from computer network monitors, video surveillance cameras, financial transaction records, databases of email exchanges, etc. The duo present their case in a paper published in this month's IEEE Computer, the flagship magazine of the Institute of Electrical and Electronics Engineers' Computer Society.

"PQS closes the gap between gathering a tremendous amount of valuable data and figuring out what the data means," says Cybenko, the Dorothy and Walter Gramm Professor of Engineering at Dartmouth's Thayer School of Engineering.

PQS has been an evolving algorithmic and software framework for the past few years. Cybenko and Berk say that PQS is a useful and incredibly powerful tool to quickly analyze credit reports for ID theft, discover attacks on computer networks, and measure activity at, say, national borders, mall parking lots, or wildlife refuge areas. According to Cybenko, "PQS can do for discrete, categorical data analysis problems what classical times series analysis did for finance and control systems where the data are numerical."

It is based on the premise that sensed environments, be they computer networks, email traffic, or high-security buildings, all consist of processes with distinct states, dynamics, and observables. PQS works to detect and understand the changes or irregularities in these processes. The PQS software is easily installed with the sensor equipment to collect, monitor, and sort out a great deal of data.

"I think the most interesting application of PQS to date is in network security monitoring," says Vincent Berk, research associate and lecturer at Dartmouth's Thayer School. "Network administrators have many options when it comes to monitoring tools, however none of them are integrated; and, while all of them produce useful information, it's often in hugely impractical quantities. PQS brings together the information, and effectively focuses on the most important issues first. To my knowledge there has not been a new software technology that is this versatile since the introduction of relational databases."

Source: Dartmouth College

3.7 /5 (3 votes)  

Rank 3.7 /5 (3 votes)
Tags

Relevant PhysicsForums posts

More news stories

Expat French get Internet vote for first time

French citizens will for the first time this year be able to vote in a parliamentary election over the Internet, an experiment that could be extended to other elections if successful.

Technology / Internet

created 1 hour ago | popularity not rated yet | comments 0

"Twisted Metal" gamers get shot at real gunplay

Fans of "Twisted Metal" will get to welcome a long-awaited sequel of the car-battle videogame with a real-world bang by blasting an ice cream truck to bits with a machine gun.

Technology / Software

created 56 minutes ago | popularity not rated yet | comments 0

New error-correcting codes guarantee the fastest possible rate of data transmission

Error-correcting codes are one of the triumphs of the digital age. They’re a way of encoding information so that it can be transmitted across a communication channel — such as an optical fiber o ...

Technology / Computer Sciences

created 7 hours ago | popularity 5 / 5 (3) | comments 4 | with audio podcast

Small modular reactor design could be a 'SUPERSTAR'

(PhysOrg.com) -- Though most of today's nuclear reactors are cooled by water, we've long known that there are alternatives; in fact, the world's first nuclear-powered electricity in 1951 came from a reactor ...

Technology / Energy & Green Tech

created 6 hours ago | popularity 4.2 / 5 (10) | comments 14 | with audio podcast

Advanced power-grid model finds low-cost, low-carbon future in West

(PhysOrg.com) -- The least expensive way for the Western U.S. to reduce greenhouse gas emissions enough to help prevent the worst consequences of global warming is to replace coal with renewable and other ...

Technology / Energy & Green Tech

created 6 hours ago | popularity 5 / 5 (2) | comments 6 | with audio podcast


Putting the squeeze on planets outside our solar system

(PhysOrg.com) -- Using high-powered lasers, scientists at Lawrence Livermore National Laboratory and collaborators discovered that molten magnesium silicate undergoes a phase change in the liquid state, abruptly ...

Employers feel no love for unscrupulous practice of 'service sweethearting'

A new study led by two Florida State University marketing professors finds that some frontline service employees who are rewarded for hikes in customer loyalty and satisfaction also may engage in "service ...

Human cognitive performance suffers following natural disasters, researchers find

Not surprisingly, victims of a natural disaster can experience stress and anxiety, but a new study indicates that it might also cause them to make more errors - some serious - in their daily lives. In their upcoming Human Fa ...

The power of estrogen -- male snakes attract other males

A new study has shown that boosting the estrogen levels of male garter snakes causes them to secrete the same pheromones that females use to attract suitors, and turned the males into just about the sexiest ...

Curry spice component may help slow prostate tumor growth

Curcumin, an active component of the Indian curry spice turmeric, may help slow down tumor growth in castration-resistant prostate cancer patients on androgen deprivation therapy (ADT), a study from researchers ...

Fool's gold may prove an unlikely alternative to overexploited catalytic materials

Catalytic materials, which lower the energy barriers for chemical reactions, are used in everything from the commercial production of chemicals to catalytic converters in car engines. However, with current catalytic materials ...