Scientists describe technique for extracting hierarchical structure of networks

In a May 1 Nature paper, “Hierarchical structure and the prediction of missing links in networks,” Santa Fe Institute (SFI) researchers Aaron Clauset, Cristopher Moore, and Mark Newman show that many real-world networks can be understood as a hierarchy of modules, where nodes cluster together to form modules, which themselves cluster into larger modules -- arrangements similar to the organization of sports players into teams, teams into conferences, and conferences into leagues, for example.

This hierarchical organization, the researchers show, can simultaneously explain a number of patterns previously discovered in networks, such as the surprising heterogeneity in the number of connections some nodes have, or the prevalence of triangles in a network diagram. Their discovery suggests that hierarchy may, in fact, be a fundamental organizational principle for complex networks.

Unlike much previous work in this area, Clauset, Moore, and Newman propose a direct but flexible model of hierarchical structure, which they apply to networks using the tools of statistical physics and machine learning.

To demonstrate the practical utility of their model, they analyze networks from three disparate fields: the metabolic network of the spirochete Treponema pallidum (the bacteria that causes syphilis), a network of associations between terrorists, and a food web of grassland species. Even when only half of the connections in these networks were shown to their algorithm, the researchers found that hierarchical structure can predict missing connections with an accuracy of up to 80 percent.

“Many networks, particularly those in the biological sciences, are not well understood,” says Clauset, an SFI Postdoctoral Fellow. "But hierarchy offers a way to understand their large-scale organization and, from this, predict what interactions we might have missed.”

Source: Santa Fe Institute