Search engines return similar results for e-commerce comparison shopping

In a recent study, Dr. Jim Jansen discovered no significant difference in the effectiveness of five popular search engines in helping consumers find what they were looking for.

"What we learned is that there is little benefit for consumers who occasionally shop online to visit various search engines for product comparisons as those engines basically return nearly identical results in terms of effectiveness," said Jansen, assistant professor in the College of Information Sciences and Technology (IST).

The study also indicated that sponsored links-those that are paid for by businesses-- are no more relevant to consumers than organic or non-sponsored links -- those returned automatically by a search engine's algorithms.

But Jansen said some advantage to using niche search engines designed for e-commerce can be gained for those consumers who primarily do online shopping.

"Basically, e-commerce search engines will give consumers one more relevant result in a group of 40 links," Jansen said. "This isn't a significant increase for consumers conducting a single search, but for people doing numerous searches every day, the performance increase could be considerable."

The study, described in "The Effectiveness of Web Search Engines for Retrieving Relevant Ecommerce Links" will be published in July's Information Processing & Management, Vol. 44, Issue 4, involved 100 e-commerce queries, five major search engines and more than 3,540 returned Web links. The study is available online at http://dx.doi.org/ … 2005.09.003.

Independent evaluators analyzed each link for relevance, comparing the original query with the returned results. From their analysis, Jansen and co-author Paulo Molina determined the five search engines returned nearly identical results in terms of relevance. At the time of the study, Molina was a student in the Smeal College of Business at Penn State.

The search engines represented the five general types of searching services that can support e-commerce searching. The individual engines selected and the categories in which each belongs are Excite, a meta search engine; Froogle, an e-commerce search engine; Google, a general-purpose search engine; Yahoo!MarketingServices, a pay-per-click engine; and Yahoo!Directory, a directory of categorized Web sites.

While it is estimated that 80 percent of Web users start with search engines, little is known about their effectiveness in helping consumers locate relevant e-commerce information. Niche search engines, such as those designed for e-commerce, are assumed to perform better in returning relevant results than all-purpose search engines.

Not so, Jansen concluded.

"E-commerce search engines or directory services do return slightly more relevant results per query, but for most consumers, the difference is minimal," the Penn State researcher said.

Niche search engines targeted to specific markets such as technology, retail clothes or sports equipment may be the way to increase the number of relevant results, Jansen said.

The study also reports that evaluators don't perceive sponsored results as more relevant than organic links. In fact, the reverse is true as the evaluators judged the organic links as more relevant, he noted.

"That finding has implications for Web search engines supporting e-commerce searching as well as for businesses desiring to sell on the Web," Jansen said.

Future research will focus on a more detailed study concerning ranking of sponsored and non-sponsored links to evaluate their effectiveness for consumers.

Source: Penn State