Finding patterns of importance in a deluge of data

January 9, 2007

Dartmouth engineers George Cybenko and Vincent Berk think that PQS, or process query systems, are the way to go to make sense of the huge volume of data we collect each day from computer network monitors, video surveillance cameras, financial transaction records, databases of email exchanges, etc. The duo present their case in a paper published in this month's IEEE Computer, the flagship magazine of the Institute of Electrical and Electronics Engineers' Computer Society.

"PQS closes the gap between gathering a tremendous amount of valuable data and figuring out what the data means," says Cybenko, the Dorothy and Walter Gramm Professor of Engineering at Dartmouth's Thayer School of Engineering.

PQS has been an evolving algorithmic and software framework for the past few years. Cybenko and Berk say that PQS is a useful and incredibly powerful tool to quickly analyze credit reports for ID theft, discover attacks on computer networks, and measure activity at, say, national borders, mall parking lots, or wildlife refuge areas. According to Cybenko, "PQS can do for discrete, categorical data analysis problems what classical times series analysis did for finance and control systems where the data are numerical."

It is based on the premise that sensed environments, be they computer networks, email traffic, or high-security buildings, all consist of processes with distinct states, dynamics, and observables. PQS works to detect and understand the changes or irregularities in these processes. The PQS software is easily installed with the sensor equipment to collect, monitor, and sort out a great deal of data.

"I think the most interesting application of PQS to date is in network security monitoring," says Vincent Berk, research associate and lecturer at Dartmouth's Thayer School. "Network administrators have many options when it comes to monitoring tools, however none of them are integrated; and, while all of them produce useful information, it's often in hugely impractical quantities. PQS brings together the information, and effectively focuses on the most important issues first. To my knowledge there has not been a new software technology that is this versatile since the introduction of relational databases."

Source: Dartmouth College


   
Rate this story - 3.7 /5 (3 votes)


January 9, 2007 all stories

Comments: 0

3.7 /5 (3 votes)



  • hide
  • Relevant PhysicsForums posts

  • Computer 5V or 0V output to Sensaphone Express II
    created Feb 04, 2010
  • Ti-89 ROM Image
    created Jan 29, 2010
  • TV ads
    created Jan 29, 2010
  • Apple introduces latest iNonsense
    created Jan 27, 2010
  • More from Physics Forums - Computing & Technology

Other News

Handling emergencies online

Technology / Internet

created 43 minutes ago | popularity not rated yet | comments 0

Online social networking sites could solve many problems plaguing information dissemination and communications when disaster strikes, according to a report from US researchers in a recent issue of the International Journal of ...


A general view of the arrival area of the Whistler Creek Alpine Skiing venue

Google Maps climbs to Olympic peaks

Technology / Internet

created 1hour ago | popularity not rated yet | comments 0

Google sent snowmobiles rigged with cameras into Canadian mountains so folks snug and warm at home will get views of slopes at the Winter Olympic Games kicking off on Friday.


Warner CEO sees e-book 'fracas' as helping music

Technology / Business

created 2 hours ago | popularity not rated yet | comments 0

(AP) -- The head of Warner Music Group expressed hope on Tuesday that the recent "fracas" over the price of e-books would help give content creators such as his company more pricing power over device makers.


Spanish Minister of industry Miguel Sebastian (C) sits in an electric car with Jean Pierre Laurent

EU ministers call for common electric car strategy

Technology / Energy

created 2 hours ago | popularity 3 / 5 (1) | comments 1

EU industry ministers on Tuesday pressed the European Commission to establish a common strategy to develop electric cars.


'Revolutionary' water treatment units on their way to Afghanistan

Technology / Engineering

created 4 hours ago | popularity 5 / 5 (2) | comments 0 | with audio podcast

The United States Army has taken delivery of the first two units of a "revolutionary" waste-water treatment system that will clean putrid water within 24 hours and leave no toxic by-products, according to scientists at Sam ...