Comparison of Communities Detection Algorithms for Multiplex

Chuan Wen, Loe1 and Henrik Jeldtoft Jensen2
Abstract

Multiplex is a set of graphs on the same vertex set, i.e. {G(V,E1),,G(V,Em)}. It is a generalized graph to model multiple relationships with parallel edges between vertices. This paper is a literature review of existing communities detection algorithms for multiplex and a comparative analysis of them.

pacs:

I Introduction

Complexity Science studies the collective behaviour of a system of interacting agents, and a graph (network) is often an apt representation to visualize such systems. Traditionally the agents are expressed as the vertices, and an edge between a vertex pair implies that there are interactions between them.

However the modern outlook in Network Science is to generalize the edges to encapsulate the multiple type of relationships between agents. For example in a social network, people are acquainted through work, school, family, etc. This is to preserve the richness of the data and to reveal deeper perspectives of the system. This is known as a multiplex.

Multiplex is a natural transition from graphs and many disciplines independently studied this mathematical model for various applications like communities detection. A community refers to a set of vertices that behaves differently from the rest of the system. This is to modularize a complex system into simpler representations to form the bigger picture of the information flow.

The first half of this paper is a literature review of the different communities detection algorithms and some theoretical bounds on graph cutting. Next we propose a suite of benchmark multiplexes and similarity metrics to determine the similarity of various communities detection algorithms. Finally we present the empirical results for this paper.

II Preliminaries

Definition II.1.

(Multiplex) A multiplex is a finite set of m graphs, 𝒢={G1,,Gm}, where every graph Gi=(V,Ei) has a distinct edge set EiV×V.

There are many synonymous names for multiplex and occasionally they simultaneously used in the same paper. This is to assist readers to visualize the system through the different descriptions. For instance a “multi-layer network” describes the multiplex as layers of graphs, i.e. layer i implies Gi𝒢. The following list is the synonymous names for multiplex and will be avoided for the rest of the paper: Multigraph Cormode and Muthukrishnan (2005); Gjoka et al. (2011), MultiDimensional Network Berlingerio et al. (2011, 2012); Hossmann et al. (2012); Kazienko et al. (2011); Lerch et al. (2006); Rossetti et al. (2011); Tang et al. , Multi-Relational Network Cai et al. (2005); Davis et al. (2011); Rodriguez and Shinavier (2010); Szell et al. (2010); Bry et al. (2012); Louati et al. (2012); Skvoretz and Agneessens (2007), MultiLayer Network Bródka et al. (2012, 2011, 2013); Li et al. (2012); Cozzo et al. (2012); Mucha et al. (2010), PolySocial Networks Applin and Fischer , Multi-Modal Network Ambrosino and Sciomachen (2012); LeBlanc (1988); Nagurney (2000), Heterogeneous Networks Dong et al. (2012) and Multiple Networks Magnani and Rossi (2013). However we will use the ith layer, relationship or dimension to refer to the graph Gi𝒢. This is to help us to express certain ideas in a more concrete manner.

For example when layers of graphs are stacked on top of each other, there will be vertex pairs with edges “overlapping” each other (Def. II.2). The distribution to the number of the overlapping edges is an important characteristic of a multiplex, where it is used to classify different multiplex ensembles Cellai et al. (2013); Lee et al. (2012); Chen et al. (2012); Ranola et al. (2010); Bianconi (2013).

Definition II.2.

(Overlapping Edges) Let two edges from two different graphs in 𝒢 be e(u,v)Ei and e(u,v)Ej, where ij. The edges e and e overlap if and only if e=e, i.e. u=u and v=v.

If there arise ambiguity to the context, we will distinguish the “community” between a multiplex and a monoplex (a graph in a multiplex) as multiplex-community and monoplex-community respectively. This will avoid confusion when we review the different multiplex communities detection algorithms.

Many of these multiplex-algorithms divide the multiplex problem into independent communities detection problems on the monoplexes. The solutions for these monoplexes, known as auxiliary-partitions, provide the supplementary information for the multiplex-algorithm to aggregate. The principal solution from the aggregation forms the multiplex-partition, which defines the communities in the multiplex.

III Definitions of a Multiplex-Community

A community is vaguely described as a set of interacting agents that collectively behaves differently from its neighboring agents. However there is no universally accepted formal definition, since the construct of a community depends on the problem domain. Fortunato (2010) categorized this diversity by the communities’ Local Definitions, Global Definitions and Vertex Similarity.

III.1 Local Definition

From the assumption that a community has weak interactions with their neighboring vertices, the evaluation of a community can be isolated from the rest of the network. Thus it is sufficient to establish a community from the perspective of the members in the community.

Consider each graphs in a multiplex as an independent mode of communication between the members, e.g. email, telephone, postal, etc. A high quality community should resume high information flow amongst its members when one of the communication modes fails (1 less graph). Hence Berlingerio et al. proposed the redundancy of the communities Berlingerio et al. (2011) as a measure to the quality of a multiplex-community.

Definition III.1.

(Redundancy) Let WV be the set of vertices in a multiplex-community and PW×W be the set of vertex pairs in W that are adjacent in 1 relationship. The set of redundant vertex pairs are PP where vertex pairs in W that are adjacent in 2 relationships. The redundancy of W is determined by:

1|𝒢|×|P|Gi𝒢{u,v}Pδ(u,v,Ei), (1)

where δ(u,v,Ei)=1 (zero otherwise) if {u,v}Ei.

Eq. 1 counts the number of edges in the multiplex-community where their corresponding vertex pairs are adjacent in two or more graphs. The sum is normalized by the theoretical maximum number of edges between all adjacent vertex pairs, i.e. (|𝒢|×|P|). The quality of a multiplex-community is determined by how identical the subgraphs (induced by the vertices of the multiplex-community) are across the graphs in the multiplex.

Thus the redundancy does not depends on the number of edges in the multiplex-community, i.e. not a necessary condition to its quality. This can lead to an unusual idea that a community can be low in density. For instance a cycle of overlapping edges form a “community” of equal quality as a complete clique of overlapping edges.

III.2 Global Definition

The global measure of a partition considers the quality of the communities and their interactions among the communities. For example the modularity function by Newman and Girvan measures how different a monoplex-communities are from a random graph (Def. III.2) Newman (2006).

Definition III.2.

(Modularity) Let Aij be the adjacency matrix of a graph with |E| edges and ki is the degree of vertex i. δ(vi,vj)=1 if vi and vj are in the same community, otherwise δ(vi,vj)=0. The Modularity function measures how far the communities differs from a random graph:

Q=12|E|ij(Aijkikj2|E|)δ(vi,vj). (2)

Given a fixed partition on the vertex set, the modularity on each of the m graphs in the multiplex differs. Therefore a good multiplex-communities suggests that all the monoplex-communites in the graphs have high modularity.

To quantify this concept, Tang et. al claims that if there exists latent communities in the multiplex, a subset of the graphs in the multiplex, 𝒢𝒢 has sufficient information to find these communities Tang et al. . If the hypothesis is true, then the communities detected from 𝒢 should reflect high modularity on the rest of the graphs in the multiplex, i.e. 𝒢𝒢.

In the language of machine learning, pick a random graph G𝒢 as the test data and let 𝒢=𝒢G be the training data. The multiplex-partition P yielded from a communities detection algorithm on 𝒢 is evaluated with the modularity function on the test data G. P is a good multiplex-partition if the modularity of partition P on the graph G is maximized. This extends the modularity metric for multiplex.

III.3 Vertex Similarity

Two vertices belongs to a Vertex Similarity community if they are similar by some measure. For example the Edge Clustering Coefficient Radicchi et al. (2004) of a vertex pair in a graph measures the (normalized) number of common neighbors between them. A high Edge Clustering Coefficient implies that there are many common neighbors between the vertex pair, thus suggesting that the two vertices should belong in the same community. In the extension for multiplex, Brodka et. al introduced Cross-Layer Edge Clustering Coefficient (CLECC) Bródka et al. (2012).

Definition III.3.

(Cross-Layer Edge Clustering Coefficient) Given a parameter α, the MIN-Neighbors of vertex v, N(v,α) are the set of vertices that are adjacent to v in at least α graphs. The Cross-Layer Edge Clustering Coefficient of two vertices u,vV measures the ratio of their common neighbors to all their neighbors.

CLECC(u,v,α)=|N(u,α)N(v,α)||N(u,α)N(v,α){u,v}|. (3)

A pair of vertices in a multiplex of social networks with low CLECC suggests that the individuals do not share a common clique of friends through at least α social networks. Therefore it is unlikely that they form a community together.

III.4 Densely Connected Community Cores

Communities detection is the process to partition the vertices such that every vertex belongs in at least a community. However there are cases where one just desires some substructures in a multiplex with certain properties. For example a Dense Connected Community Core is the set of vertices such that they are in the same community for all the auxiliary-partitions Yin and Khaing (2013).

IV Theoretical Bounds

The Max-Cut problem finds a partition of a graph such that the number of edges induced across the clusters are maximized. Thus the same partition over the complement graph minimizes the number of edges between the clusters. This is known as the Balanced-Min-Cut problem and it is closely related to our communities detection problem, where the number of edges induced between the communities are minimized.

Therefore to extend our understanding for multiplex, we begin with a known result for the Max-Cut problem. It allows us to prove a corollary for the Balanced-Min-Cut problem on multiplex.

IV.1 Maximum Cut Problem on Multiplex

Theorem IV.1.

Consider graph G1,,Gm on the same vertex set V. There exists a k-partition of V into k2 communities C1,,Ck such that for all i=1,,m:

# edges cut in Gi(k1)|Ei|k2m|Ei|/k. (4)
Proof.

Sketch Kühn and Osthus (2007): The proof for k3 is similar to the case for k=2, hence we will only prove the latter. WLOG, the vertex set of G1 is partitioned into 2 equal sets, A and B. Define Xi be an indicator function where Xi=1 if the ith edge is induced from A to B, Xi=0 if otherwise. Since Pr(Xi)=1/2 and with linearity of expectation, we get 𝔼[Xi]=|Ei|/2 and 𝔼[Xi2]=|Ei||Ei+1|/4. We can now by use of Chebyshev’s Inequality get the probability that a given partition fail the inequality 4 for G1. Since the probability sum of all graphs to fail is less than 1, thus there exists a partition such that all the graphs fulfill inequality 4.

Further developments on the bounds were made by imposing additional conditions where the maximum degree is bounded Kühn and Osthus (2007) and in cases where k=m=2 Patel (2008). Since the solution for Max-Cut on graphs is NP-complete, the extension to simultaneously Max-Cut all the graphs in a multiplex is naturally NP-complete too. Thus this also implies that balanced minimum bisection is NP-complete too Garey and Johnson (1990).

IV.2 Balanced Minimum Cut Problem on Multiplex

Corollary IV.1.

Consider graph G1,,Gm on the same vertex set V. There exists a k-partition of V into k2 equal-sized communities C1,,Ck (i.e. |Ci||Cj|) such that for all i=1,,m:

# edges cut in Gi(k1)|E¯|k2m|E¯|/k, (5)

where |E¯|=(n2)|Ei|.

Proof.

Let G¯i be the complement graph of Gi, and its edge set is denoted by E¯i. Since the maximum number of edges in a graph is (n2), hence |E¯i|=(n2)|Ei|. Apply (Max-Cut) Theorem IV.1 on the set of complement graphs G¯i and substitutes |E¯|i into the result, the expression in the corollary follows. The proof in Theorem IV.1 ensures that the communities are equal in size.

A partition that fulfills Eq. 5 is not necessary a good community defined in Section III, vice versa. However the edges induced between partition classes are often perceived as bottlenecks when information flows through the network/multiplex. They are similar to the bridges between cities and communities. Therefore Communities Detection Algorithms tend to minimize the number of edges between different communities.

V Communities Detection Algorithms for Multiplex

The general strategy for existing communities detection algorithm for multiplex is to extract features from the multiplex and reduce the problem to a familiar representation. In solving the reduced representation, the multiplex-communities are then deduced from the auxiliary solutions of the reduced problems.

Therefore many multiplex algorithms rely on existing monoplex-communities detection algorithms to get the auxiliary-partitions for the interim steps. The choice of algorithms is often independent of their extension for multiplex, and hence any communities detection algorithm in theory can be chosen to solve the interim steps. In this paper our experiments used Louvain Algorithm to generate the auxiliary-partitions.

V.1 Projection

The naive method is to projected the multiplex into a weighted graph. I.e. let Ai be the adjacency matrix of Gi𝒢. The adjacency matrix of the weighted projection of 𝒢 is given by A¯=1mi=1mAi. We will call this the “Projection-Average” of multiplex.

It was been independently proposed as a baseline for more sophisticated multiplex algorithms as the performance is often “sub-par” Berlingerio et al. (2011); Kazienko et al. (2011); Nefedov (2011); Davis et al. (2011); Rossetti et al. (2011). In our experiments we will compare this with the unweighted variant, that is the “Projection-Binary” of a multiplex, i.e. G(V,E1Em).

An alternative weight assignment between vertex pair is to consider the connectivity of their neighbors, where a high ratio of common neighbors implies stronger ties Berlingerio et al. (2011). This is based on the idea that members of the same community tend to interact over the same subset of relations, which was independently proposed by Brodka et. al in Def. III.3 Bródka et al. (2012). This alternative will be known as “Projection-Neighbors”.

V.2 Consensus Clustering

The previous strategy aggregates the graphs first, and then it performs the communities detection algorithm over the resultant graph. It is a poor strategy as it neglects the rich information of the dimensions Tang et al. . Therefore the Consensus Clustering strategy is to first apply the communities detection algorithm on the graphs separately as auxiliary partitions, and then the principal clustering (multiplex communities) is derived by aggregating these auxiliary partitions in a meaningful manner.

The key concept behind consensus clustering is to measure the frequency with which two vertices are found in the same community among the auxiliary partitions. Vertices that are frequently in the same monoplex-community are more likely to be in the same multiplex-community. Therefore the communities detection algorithm on the individual graphs on the multiplex determines the structural properties of multiplex-communities, whereas Consensus Clustering determines the relational properties of the multiplex-communities.

V.2.1 Frequent Closed Itemsets Mining

Data-mining is to find a set of items that occurs frequently together in a series of transactions. For example items like milk, cereal and fruits are frequently bought together in supermarkets based on a series of sales transactions. These sets are known as itemsets. Berlingerio et. al translates the Consensus Clustering of the auxiliary-partitions as a data-mining problem to discover multiplex-communities Berlingerio et al. (2013).

The vertices in the multiplex defines the |V| transactions for the data-mining, and the items are tuples (c,d) where the respective vertex belongs in monoplex-community c in dimension d. For example suppose vertex vi belongs to monoplex-communities c1,c5 and c2 in dimensions d1,d2 and d3 respectively. The ith transaction is the set of items {(c1,d1),(c5,d2),(c2,d3)}. Therefore when we use data-mining methods like Frequent Closed Itemsets Mining, we are able to identify the frequent (relative to a predefined threshold) itemsets as multiplex-communities.

For example each vertex is a customer’s transaction in a supermarket, and a community is a target market that the supermarket wants to discover. It is only meaningful if the target market is sufficiently large, and thus we need to defined the minimum community size (e.g. 10). In this case a customer’s transaction is his auxiliary-communities membership. Therefore Frequent Closed Itemsets Mining will extract a multiplex-community on at least e.g. 10 vertices with which each customer’s transaction is a subset of the target market’s itemsets.

V.2.2 Cluster-based Similarity Partitioning Algorithm

Cluster-based Similarity Partitioning Algorithm averages the number of instances vertex pairs are in the same auxiliary-communities. For example in a multiplex with 5 dimensions, if there are 3 instances where vertices vi and vj are in the same auxiliary-community, then the similarity value of vertex pair (vi,vj) is 3/5.

Once the similarity is measured for all the vertex pairs, the principal cluster is determined with k-means clustering — vertices with the closest similarity at each iteration are grouped together. Therefore vertex pairs that are frequently in the same auxiliary-communities will have high similarity value, and hence more likely to be clustered together in the same principal-community. This is known as Partition Integration by Tang et. al Tang et al. .

V.2.3 Generalized Canonical Correlations

Each of the auxiliary-partitions maps the vertices as points in a l-dimensional (this dimension is independent to the dimensions of a multiplex) Euclidean space. The points are positioned in a way that the shorter the shortest path between two vertices are, the closer they are in the Euclidean space. One of such mapping can be achieved by concatenating the top eigenvectors of the adjacency matrix. Thus given d graphs in a multiplex, there are d structural feature matrices Si of size l×n where the column in each matrix is the position of a vertex in the l-dimensional Euclidean space.

Tang et. al wants to aggregate the structural feature matrices to a principal structural feature matrix S¯ such that the principal partition can be determined from S¯ Tang et al. . The “average” S¯=1di=1dS(i) however does not result in sensible principal structural feature matrix since the matrix elements between S(i) and S(j) are independent.

A solution to fix this problem is to transform the S(i) such that they are in the same space and their “average” is sensible. That is the same vertex in the d different Euclidean spaces are aligned in the same point in a Euclidean space. Specifically we need a set of linear transformations wi such that they maximize the pairwise correlations of the S(i), and Generalized Canonical Correlations Analysis is one of such standard statistical tools Kettenring (1971). This allows us to “average” the structures in a more sensible way:

S¯=1di=1dS(i)wi. (6)

Finally the principal partition is determined via k-mean clustering of the principal feature matrix S¯.

V.3 Bridge Detection

A bridge in a graph refers to an edge with high information flow, like the busy roads between two cities, where the absence of these roads separates the cities into isolated communities. One way to do this is to project the multiplex 𝒢 to a weighted network and determine the bridges from the projection. Alternatively one can remove them by the definition of a multiplex-bridge to get the desired partitions.

In social networks, strong edge ties are desirable within the communities. Hence to identify weak ties between vertex pairs, Brodka et. al proposed CLECC (Eq. 3) as a measure. At each iteration, all connected vertex pairs are recomputed and the pair with the lowest CLECC score will be disconnected in all the graphs. The algorithm halts when the desired number of communities (components) are yielded greedily Bródka et al. (2012). This is the same strategy presented by Girvan and Newman, where the bridges of a graph were identified by their betweenness centrality score Girvan and Newman (2002). In the experiments, we set α=|𝒢|/2 for the CLECC score in Eq. 3.

V.4 Tensor Decomposition

Algebraic Graph Theory is a branch of Graph Theory where algebraic methods like linear algebra are used to solve problems on graph. Hence the natural representation for a multiplex is a 3rd-order tensor (as a multidimensional array) instead of a matrix (2nd-order tensor). The set of m graphs in a multiplex is a set of m n×n adjacency matrices, which can be represented as a m×n×n multidimensional array (tensor) Rodriguez and Shinavier (2010). This allows us to leverage on the available tensor arithmetics like tensor decomposition.

Tensor decompositions are analogues to the singular value decomposition and ’Lower Upper’ decomposition in matrices, where they express the tensor into simpler components. For example a PARAFAC tensor decomposition Harshman (1970) is the rank-k approximation of a tensor 𝒯 as a sum of rank-one tensors (vectors u¯(i), v¯(i) and w¯(i)), i.e.:

𝒯i=1ku¯(i)v¯(i)w¯(i). (7)

where a¯b¯ denotes the vector outer product. The components in the ith factor, u¯(i), v¯(i) and w¯(i), suggest that there are strong ties (possibly a cluster/community) between the elements in u¯(i) and v¯(i) via the dimension in the top component in w¯(i).

For instance suppose the jth element in w¯(i) is the most largest element. This suggests that in the ith community, the top 10 (or any predefined threshold) elements in u¯(i) are in the same cluster as the top 10 elements in v¯(i) via the jth dimension/relationship Dunlavy et al. (2011); Leginus et al. (2012); Mirzal and Furukawa (2010); Papalexakis et al. (2013).

VI Benchmark Multiplex

A Erdős-Rényi Graph is a graph where vertex pairs are connected with a fixed probability Erdös and Rényi (1960). The random nature of this construction usually does not have any meaningful communities structures in them. Hence it is not ideal to use as a benchmark graph for Communities Detection Algorithms.

Therefore a benchmark graph should be similar to the Girvan and Newman Model where some random edges are induced between a set of dense subgraphs (as communities) to form a single connected component/graph. The set of dense subgraphs acts as “ground-truth” communities of the graph for a Communities Detection Algorithm to discover and hence a benchmark for Communities Detection Algorithms. The goal of this section is to design similar benchmarks for multiplex.

The main challenge is that there is not yet a universally accepted definition of a good multiplex community. Hence there is no methodology for us to construct a benchmark such that it does not favor certain algorithms. Therefore the objective of the following benchmark graphs is to study the relations between these multiplex-communities detection algorithms. This allows us to use a collection of highly uncorrelated algorithms to study different perspectives of a multiplex-community.

VI.1 Unstructured Synthetic Random Multiplex

The simplest construction of a random multiplex is to generate a set of independent graphs on the same vertex set. However many communities detection algorithms are based on some observations of real world multiplexes and these algorithms do not yield interesting results on such random construction. We denote such random multiplexes as Unstructured Synthetic Random Multiplex (USRM), and they are analogous to Erdős-Rényi Graphs where Communities Detection Algorithms should not find any meaningful communities in them.

For simplicity in the numerical experiments there are only two dimensions in the multiplexes such that we can easily generate all six combinations of Erdős-Rényi, Watts Strogatz Watts and Strogatz (1998) and Barabási-Albert graphs Albert and Barabási (2002). The statistical properties of such construction can be found in Loe and Jensen (2013).

Name Graph 1 Graph 2
USRM1 Erdős-Rényi Erdős-Rényi
USRM2 Erdős-Rényi Watts Strogatz
USRM3 Erdős-Rényi Barabási-Albert
USRM4 Watts Strogatz Watts Strogatz
USRM5 Watts Strogatz Barabási-Albert
USRM6 Barabási-Albert Barabási-Albert

In the experiments the size of the vertex set is 128. It is important that the number of edges in both graphs are equal so that neither dominates the interactions of the multiplex. To ensure the Erdős-Rényi graph is connected with high probability, the probability that a vertex pair is connected has to be >ln|V|/|V|. Therefore the number of edges in the Erdős-Rényi graph (as well as the other graphs in the multiplex) is (1282)2ln128/128616.

VI.2 Structured Synthetic Random Multiplex

This construction is similar to the Girvan and Newman Model where independently generated communities are connected in a way such that the “ground-truth” communities remains. However we saw above that there are different perspective of multiplex communities and we want to encapsulate these ideas into a multiplex benchmark.

Structured Synthetic Random Multiplex (SSRM) is a construction where different multiplex-partitions give high quality multiplex-communities defined by different definitions. However at the same time the partition has to be of “less-than-ideal” quality from the perspective of other definitions of multiplex-communities.

We begin by generating multiplex-communities, each of which are of high quality with respect to different independent definitions of communities. Next we modify these multiplex-communities such that it remains good in one of the three multiplex-communities definitions and poor quality by the other definitions. The final step is to combine these multiplex-communities into a single multiplex as our SSRM benchmark (Fig. 1).

Fig. 1 shows {[c1,c2],[c3,c4]} as a partition with 2 communities where clusters 1 and 2 form a community and cluster 3 and 4 form the second community. This partition has high redundancy, low modularity and low CLECC multiplex-communities. There is another partition {[c1,c3],[c2,c4]} where the communities have high modularity, but low in the remaining metrics. Lastly {[c2,c3],[c1,c4]} is the final partition where communities have high CLECC vertex pairs. This synthetic multiplex expresses the multi-perspective nature of a multiplex communities.

c1c2c3c4IIIIII
Figure 1: To aid visualization, the edges in this two dimensional SSRM are drawn with solid and dashed lines. Let clusters c1 and c3 be dense subgraphs where there are more solid edges than dashed edges. Similarly c2 and c4 are dense subgraphs with more dashed edges than solid edges. I: The solid edges between c1 & c4 implies that there are only solid edges between them. This applies the same to the dashed edges between c2 & c3. II: All the edges between c1 and c2 overlaps. III: None of the edges between c1 & c3 (or c2 & c4) overlaps. We denote {[c1,c2],[c3,c4]} as a partition with 2 communities where clusters 1 and 2 form a community and cluster 3 and 4 form the second community. This partition has high redundancy, low modularity and low CLECC multiplex-communities.

VI.2.1 High Modularity, Low Redundancy & Low CLECC Multiplex-Communities

To create a high modularity multiplex-community, we begin with cluster 1 and cluster 3 as a clique-like structure in both dimensions. For these clusters to be low in redundancy, we have to remove some edges in both clusters such that there are very few overlapping edges in the clusters while maintaining high modularity. This can be done by making the first dimension graph in both clusters to be the complement of the graph in the second dimension.

The next step is to add edges between cluster 1 and 3 such that the resultant cluster is a single connected component. We denote [c1,c3] as a component connected by cluster 1 and cluster 3. To maintain a low redundancy, the new edges cannot overlap.

Finally we need to tweak the clusters such that the CLECC score is low between a significant number of the vertex pairs in the combined component of cluster 1 and 3. Specifically we want the vertex pairs connected by the new edges in the previous step to have low CLECC scores. This is possible if cluster 1 has more edges in the first dimension whereas cluster 3 has more edges in the second dimensions. In doing so the neighbors of the vertex in cluster 1 will be significantly different from the neighbors of vertex in cluster 3, thus a low CLECC score.

The same construction applies to cluster 2 and cluster 4, where cluster 2 is similar to cluster 3 and cluster 4 is similar to cluster 1.

VI.2.2 High CLECC, Low Modularity & Low Redundancy Multiplex-Communities

Since all 4 clusters do not have overlapping edges, the redundancy remains low for any partition on the multiplex. Therefore the first step is to make cluster 1 and 4 to be high in CLECC score, and apply the same construction for cluster 2 and 3.

Since cluster 1 and 4 are similarly based on the construction from the previous subsection, the neighbors of any given vertex in each cluster will be similar too. Therefore by adding new edges between cluster 1 and 4 will not affect the CLECC score. However these new edges should only be drawn in the first dimension, since it is the dominant dimension in [c1,c4]. This simultaneously reduces the modularity of [c1,c4] in the second dimension, since the clusters are not connected and the graph in the second dimension is sparse. This gives a low modularity for multiplex communities while maintaining the high CLECC score.

The construction is similar for cluster 2 and 3, expect that only edges in the second dimension connects both clusters together.

VI.2.3 High Redundancy, Low Modularity & CLECC Multiplex-Communities

Given [c1,c3] have low CLECC score, the same score should apply for [c1,c2] since cluster 2 and cluster 3 are similar. The main goal is to connect cluster 1 and cluster 2 such that the redundancy of [c1,c2] is high. Redundancy is measured by Eq. 1, where it counts the number of edges that overlaps. Since there is no overlapping edges at this point of the construction, the simplest way to increase the redundancy is to add new overlapping edges between clusters to form the components [c1,c2] and [c3,c4].

Although [c1,c2] and [c3,c4] have relatively high redundancy as compare to other partition in the multiplex, it can still have lower redundancy than a random community in USRM1. To nudge the redundancy higher, it is necessary to add new edges such that there are overlaps in the four clusters. However this might increase the modularity of [c1,c2] which we want to avoid. Therefore this final step has to be done incrementally.

VI.2.4 Evaluation of the different ground truth partitions

There are 3 different “ground-truth” partitions {[c1,c2],[c3,c4]}, {[c2,c3],[c1,c4]} and {[c1,c3],[c2,c4]}, where they each represents a different “ideal-partition”. However simultaneously they are “less-than-ideal” from the perspective of the other metrics. Although some of these metrics are correlated (from our experiments) in general, these ground-truth partitions of SSRM show that these metrics are each able to capture essential aspects of the community structure.

Redundancy *CLECC **Modularity
[c1,c2] 0.0492 0.1142 -0.0287
[c3,c4] 0.0537 0.1087 -0.0332
[c2,c3] 0 0.1541 0.007
[c1,c4] 0 0.1642 0.012
[c1,c3] 0 0.1113 0.0317
[c2,c4] 0 0.1083 0.0245
Random 0.0217 0.1056 0.0033
Table 1: The rows (except the last row) are paired up such that they implies a common partition. For example the first two rows are communities of the partition {[c1,c2],[c3,c4]}. The redundancy and CLECC score of community [c1,c2] are 0.0492 and 0.1142 respectively. *CLECC: The CLECC score is the average CLECC score between all vertex pairs in the community. **Modularity: The two values in the partition refers to the modularity of the two graphs in the multiplex. E.g. partition {[c1,c2],[c3,c4]} has modularity -0.0287 and -0.0332 for the first and second dimensions of SSRM respectively. A partition is “less-than-ideal” if its measurement is closer to a random partition (last row) than the maximum (values in bold).

For example Table 1 shows that the partition {[c1,c2],[c3,c4]} has communities with the maximum redundancy. However it has multi-modularity and CLECC scores similar to a random partition, suggesting that it is “less-than-ideal” relative to other metrics.

VI.3 Real World Multiplex

The issue with synthetic multiplex is that it does not reflect the real-world systems correctly. The relationships between vertices are artificially imposed such that we can distinguish the different communities detection algorithms. However real-world communities do not behave in such a systematic manner and might approach similar structure and relationships even if they are created by different dynamics. Therefore we compare the multiplex-communities detection algorithms with real-world multiplexes from open dataset.

VI.3.1 Youtube Social Network

Youtube is a video sharing website that allows interactions between the video creators and their viewers. Tang et al. collected 15,088 active users to form a multiplex with 5 relationships where two users are connected if Tang et al. :

  1. 1.

    they are in the contact list of each other;

  2. 2.

    their contact list overlaps;

  3. 3.

    they subscribe to the same user’s channel;

  4. 4.

    they have subscription from a common user;

  5. 5.

    they share common favorite videos.

VI.3.2 Transportation Network

Generalized graphs in transportation networks are known as multimodal network, where bus stops, train stations and terminals are indistinguishable locations (vertices) to transit to a different transport. Cardillo et al. constructed an air traffic multiplex from the data of European Air Transportation (EAT) Network with 450 airports as vertices Cardillo et al. (2013). An edge is drawn between two vertices if there is a direct flight between them. Each of the 37 distinct airlines in the EAT Network forms a relationship between airports.

Given that the base of airlines are often in older and major airports, hence there is the “rich-gets-richer” phenomenon where new airports tends to have direct flight to major airports. Therefore Cardillo et al. verified the phenomenon by ensuring that the degree distribution follows a power-law distribution, implying a scale-free system.

VII Comparing Partitions

Normalized Mutual Information (NMI) Manning et al. (2008) is a popular similarity metric for the network partition, where a real-value score between [0,1] with score 1 implying identical partition. However we are not able to simply apply the NMI metric to all pairwise comparisons of all the algorithms.

Unlike the rest of the algorithms, Frequent Closed Itemsets Mining and Tensor Decomposition yield overlapping communities, which NMI was not designed to measure. Furthermore these algorithms also do not ensure that all vertices belong in at least one community, i.e. there might exists “isolated” vertices with no community membership. Therefore even variants of NMI Lancichinetti et al. (2009) for overlapping communities are not applicable in our study.

Adjusted Rand Index is an alternative metric to NMI in classification research Hubert and Arabie (1985). Its extension to measure overlapping communities is known as Omega Index Collins and Dent (1988) and is able to measure partitions with isolated vertices. Unlike overlapping variants of NMI, the comparison of two non-overlapping partitions with Omega Index reduces to Adjusted Rand Index.

VII.1 Normalized Mutual Information

Mutual Information I(𝒜;) measures the information of the communities-membership of all vertex-pairs in A given the communities-membership in B, vice versa. Roughly speaking given , how well can we guess that a vertex pair is in the same community in 𝒜? Formally this is defined as I(𝒜;)=H(𝒜)H(𝒜|) where H(𝒜) is the Shannon entropy:

H(𝒜)=kP(ak)logP(ak). (8)

where P(ak) is the probability of a random vertex is in the kth community of partition 𝒜, i.e. the ratio of the size of the kth community to the total number of vertices. Similarly P(bk) denotes the case for partition . The Mutual Information can also be expressed as:

I(𝒜;)=jkP(akbj)logP(akbj)P(ak)P(bj), (9)

where P(akbj) is the probability that a random vertex is both in kth and jth communities. Basically the larger the intersection of the kth and jth communities of 𝒜 and respectively is, the higher the Mutual Information. Finally to normalize the Mutual Information score:

NMI(𝒜,)=I(𝒜;)[H(𝒜)+H()]/2. (10)

VII.2 Omega Index

The unadjusted Omega Index averages the number of vertex pairs that are in the same number of communities. Such vertex pairs are known to be in agreement. Consider the case with two partitions 𝒜 and , and the number of communities in them are |𝒜| and || respectively. The function tj(𝒜) returns the set of vertex pairs that appears exactly in j0 overlapping communities in 𝒜. Thus the unadjusted Omega Index:

ωu(𝒜;)=1(n2)j=0max(|𝒜|,||)|tj(𝒜)tj()|. (11)

To account for vertex pairs that are allocated into the same communities by chance, we have to subtract it from expected omega index of a null model:

ωe(𝒜;)=1(n2)2j=0max(|𝒜|,||)|tj(𝒜)||tj()|. (12)

Finally the Omega Index is given by:

ω(𝒜;)=ωu(𝒜;)ωe(𝒜;)1ωe(𝒜;). (13)

It is possible for the Omega Index to be negative, where there are less agreement than pure stochastic coincidence would expect. It is regarded as uninteresting as it does not suggests anything more than the fact that the partitions are not similar. Identical partitions have Omega Index of 1.

VII.3 Notations For Empirical Results

To simplify the plots exhibiting the results from our experiments, we will use some shorthands to denote the algorithms and partitions. For communities detection algorithms, we use 𝒜 and 𝒫 to denote “Algorithm” and “SSRM Partition” respectively (Table 2).

Notation Description
𝒜1 Projection-Binary
𝒜2 Projection-Average
𝒜3 Projection-Neighbors
𝒜4 Cluster-based Similarity Partition Algorithm
𝒜5 Generalized Canonical Correlations
𝒜6 CLECC Bridge Detection
𝒜7 Frequent Closed Itemsets Mining
𝒜8 PARAFAC, Tensor Decomposition
𝒫1 {[c1,c2],[c3,c4]} of SSRM
𝒫2 {[c2,c3],[c1,c4]} of SSRM
𝒫3 {[c1,c3],[c2,c4]} of SSRM
Table 2: Shorthands for the different algorithms and “ground-truth” communities.

VIII Empirical Comparison of Multiplex-Communities Detection Algorithms

VIII.1 Unstructured Synthetic Random Multiplex

Figure 2 shows the Omega Index of all pairwise multiplex communities detection algorithms for the USRM benchmarks. The last 13 boxplots on the right are the pairwise comparisons with overlapping-communities algorithms, i.e. 𝒜7 and 𝒜8.

The first observation is that 𝒜8 (PARAFAC) is not similar to any of the algorithms. One of the reasons is that it is hard to choose the right parameters for PARAFAC, i.e the k approximation for Eq. 7 and the predefined threshold for the top few elements of the rank-one tensors. There is no systematic method besides manual experimentation for a given graph Dunlavy et al. (2011); Leginus et al. (2012); Mirzal and Furukawa (2010); Papalexakis et al. (2013).

However overlapping-communities algorithm 𝒜7 (Frequent Closed Itemsets Mining) is similar to a class of non-overlapping algorithms, i.e. 𝒜1 to 𝒜4. Specifically Frequent Closed Itemsets Mining is similar to the class of Projection algorithms 𝒜1 to 𝒜3. In fact the Omega Index for all pairwise comparisons of the algorithm family ={𝒜1,𝒜2,𝒜3,𝒜4,𝒜7} is generally greater than the other algorithm pairs. This is supported by their NMI scores in Fig. 3.

However the family of algorithms has low pairwise Omega Index and NMI scores for USRM 1, 3 and 6. There is no fundamental reasons to why the class of projection algorithms (𝒜1 to 𝒜3) should produce non similar communities, therefore we can deduce that USRM 1, 3 and 6 are not good multiplexes for benchmarks.

The reason is that USRM1 is the combination of two Erdős-Rényi graphs, thus there is no community structure. Whereas USRM 3 and 6 are the combinations of Barabási-Albert graph with Erdős-Rényi and Barabási-Albert respectively. Although Barabási-Albert has structural properties, it has low Clustering Coefficient (similar to Erdős-Rényi), which measures the tendency that vertices cluster together. Therefore the vertices in USRM 3 and 6 do not form communities.

To increase the clustering coefficient of a multiplex, we can introduce a Watts Strogatz graph to the system. We can approximate the clustering coefficient of the projection of such multiplexes Loe and Jensen (2013) and deduce the tendency that communities exists. This allows us to observe interesting relationships between the algorithms from benchmark multiplexes like USRM 2, 4 and 5.

For example USRM 5 in Fig. 3 shows that the similarity range of with 𝒜6 (CLECC Bridge Detection) is wide. This is more apparent when we study their relationship with SSRM benchmark.

Refer to caption
Figure 2: The Omega Index of all pairwise multiplex-communities detection algorithms for the different benchmark USRM. The tuple (i,j) on the x-axis refers to the pairwise comparison of 𝒜i and 𝒜j. The tuples are arranged such that the comparisons with overlapping-communities algorithms (13 tuples) are placed on the right.
Refer to caption
Figure 3: The NMI scores of all pairwise non-overlapping multiplex-communities detection algorithms for USRM benchmarks. The tuples are arranged such that pairwise comparisons of {𝒜1,𝒜2,𝒜3,𝒜4} are grouped to the left of the boxplots. The “interesting” figures USRM 2, 4 and 5 are placed to the right for comparison.

VIII.2 Structured Synthetic Random Multiplex

In the previous section, USRM benchmark suggests that {𝒜1,𝒜2,𝒜3,𝒜4} yields similar communities. However Fig. 4 shows that the SSRM communities from 𝒜4 (Cluster-based Similarity Partition Algorithm) are distinct from the class of projection algorithms (𝒜1 to 𝒜3). Therefore we can conclude that Cluster-based Similarity Partition Algorithm provides an alternative perspective for multiplex-communities, and it is not reducible to the class of projection algorithms.

Recall from the last observation in the previous section that the similarity range of 𝒜6 (CLECC Bridge Detection) with projection algorithms is wide. Fig. 4 shows that there are cases where they are very different (close to NMI = 0), and there are cases where the NMI >0.8. In this way our experiments highlights the weakness of the CLECC Bridge Detection algorithm.

CLECC occasionally yields separate components prematurely, where one of the components is a small cluster of vertices or even a single vertex as a community. Therefore it appears that CLECC yields significantly different communities since the rest of the algorithms tend to produce balanced-size communities. Hence if we exclude such cases for CLECC, 𝒜6 is similar to projection algorithms for SSRM.

Finally we will compare the algorithms with the “ground-truth” partitions 𝒫1, 𝒫2 and 𝒫3. Table 3 shows that none of the algorithms were able to capture 𝒫1, which is the partition with high redundancy. 𝒜3 (Projection-Neighbors) was proposed to extract high redundancy communities Berlingerio et al. (2011).

In contrast, 𝒜6 (CLECC Bridge Detection) was able to find the high CLECC partition 𝒫2. However it does not have any advantage over any of the projection algorithms (𝒜1 to 𝒜3). This further supports that 𝒜6 is similar to the class of projection algorithms.

The results by Tang et. al shows that 𝒜5 (Generalized Canonical Correlations) tends to be better than 𝒜4 (Cluster-based Similarity Partition Algorithm) to capture high-modularity communities like 𝒫3 Tang et al. . This was also observed in this experiment.

Refer to caption
Figure 4: The NMI scores of all pairwise non-overlapping multiplex-communities detection algorithms for SSRM benchmarks. The tuples are arranged in the same way as Fig. 3.
𝒫1 𝒫2 𝒫3
𝒜1 0 0.983 0
𝒜2 0.002 0.94 0.017
𝒜3 0 0.978 0
𝒜4 0.019 0.14 0.083
𝒜5 0.004 0.002 0.158
𝒜6 0.006 0.964 0.006
Table 3: The NMI scores between the algorithms and the different ground-truth partitions. The entries in bold represents the algorithms that are “close” to the ground-truth partition.

VIII.3 Real World Multiplex

The results for the European Air Transportation Network is similar to the Youtube Social Network, hence Fig. 5 is sufficient for this discussion. The general observation is similar to USRM 2, 4 and 5 in Fig. 2, where the set of algorithms {𝒜1,𝒜2,𝒜3,𝒜4,𝒜7} are relatively similar with pairwise NMI score of 0.55.

In addition, Fig. 5 highlights that overlapping-communities detection algorithm 𝒜8 (PARAFAC) is relatively more similar to 𝒜5 (Generalized Canonical Correlations) than the other non-overlapping communities detection algorithms. This observation was less apparent in Fig. 2.

Unfortunately 𝒜6 (CLECC Bridge Detection) tends to halt prematurely despite different parameter choices. Hence we did not managed to get any insight for CLECC Bridge Detection in this experiment.

Refer to caption
Figure 5: The Omega Index heatmap of all pairwise algorithms for the European Air Transportation Network. This result is similar to the Youtube Social Network.

VIII.4 Summary

The parameters in algorithms 𝒜6, 𝒜7 and 𝒜8 require manual fine-tuning to yield meaningful communities for comparisons. Hence it is not practical to exhaustively test for all configurations for these algorithms. However the out come of the analysis doesn’t change in any essential way when for different choice of parameters.

From USRM benchmarks and real world multiplexes, algorithms in the set {𝒜1,𝒜2,𝒜3,𝒜4,𝒜7} tends to generate relatively similar partitions. However SSRM benchmark demonstrate that 𝒜4 is able to capture high redundancy communities and performed differently from the class of projection algorithms.

Our experiments with USRM and SSRM benchmarks support that 𝒜6 (CLECC Bridge Detection) is similar to the class of projection algorithms {𝒜1,𝒜2,𝒜3} when the algorithm does not infer small clusters of vertices as communities. Therefore CLECC Bridge Detection is not very stable without careful analysis.

Finally we have to emphasize that the absolute Omega Index and NMI index are generally low (<0.5) for most pairwise comparisons. This implies that the algorithms technically perceive less-than-ideal multiplex-communities differently. However our results shows that there are some vertex clusters within communities that are more likely to be captured by certain algorithms than the others.

IX Discussions And Conclusion

The literature review and SSRM benchmark highlight the additional complexity to define a multiplex-community. It is a balance between the structural topology of the communities and their relationships. SSRM is a useful benchmark to bring out the fundamental differences between these algorithms. Therefore further research is done to improve SSRM such that the different definitions of communities are more apparent.

The main contribution of this paper is to consolidate and compare all the existing multiplex-communities detection algorithms. The long list of synonymous names for multiplex causes many researchers to be unaware of related efforts for multiplex, and hence has a tendency to make this research some what diffuse and fragmented Mikko Kivela (2013). Therefore we hope that this paper brings awareness for further developments on multiplex-communities detection algorithm such that researchers can build on to existing ideas.

References

  • Cormode and Muthukrishnan (2005) G. Cormode and S. Muthukrishnan, in Proceedings of the twenty-fourth ACM SIGMOD-SIGACT-SIGART symposium on Principles of database systems, PODS ’05 (ACM, New York, NY, USA, 2005) pp. 271–282.
  • Gjoka et al. (2011) M. Gjoka, C. T. Butts, M. Kurant, and A. Markopoulou, IEEE J. Sel. Areas Commun. on Measurement of Internet Topologies (2011).
  • Berlingerio et al. (2011) M. Berlingerio, M. Coscia, and F. Giannotti, in ASONAM (IEEE Computer Society, 2011) pp. 490–494.
  • Berlingerio et al. (2012) M. Berlingerio, M. Coscia, F. Giannotti, A. Monreale, and D. Pedreschi, World Wide Web , 1 (2012).
  • Hossmann et al. (2012) T. Hossmann, G. Nomikos, T. Spyropoulos, and F. Legendre, Computer Communications 35, 1613 (2012).
  • Kazienko et al. (2011) P. Kazienko, K. Musial, E. Kukla, T. Kajdanowicz, and P. Bródka, in Computational Collective Intelligence. Technologies and Applications (Springer, 2011) pp. 378–387.
  • Lerch et al. (2006) F. Lerch, J. Sydow, and K. G. Provan, in annual colloquium of the European Group for Organizational Studies, Bergen, Norway (2006).
  • Rossetti et al. (2011) G. Rossetti, M. Berlingerio, and F. Giannotti, in ICDM Workshops, edited by M. Spiliopoulou, H. Wang, D. J. Cook, J. Pei, W. W. 0010, O. R. Zaïane, and X. Wu (IEEE, 2011) pp. 979–986.
  • (9) L. Tang, X. Wang, and H. Liu, in ICDM ’09: Proceedings of IEEE International Conference on Data Mining.
  • Cai et al. (2005) D. Cai, Z. Shao, X. He, X. Yan, and J. Han, in Proceedings of the 9th European Conference on Principles and Practice of Knowledge Discovery in Databases (PKDD) (Porto, Portugal, 2005).
  • Davis et al. (2011) D. A. Davis, R. Lichtenwalter, and N. V. Chawla, in ASONAM (IEEE Computer Society, 2011) pp. 281–288.
  • Rodriguez and Shinavier (2010) M. A. Rodriguez and J. Shinavier, Journal of Informetrics 4, 29 (2010).
  • Szell et al. (2010) M. Szell, R. Lambiotte, and S. Thurner, Proceedings of the National Academy of Sciences 107, 13636 (2010), http://www.pnas.org/content/107/31/13636.full.pdf+html .
  • Bry et al. (2012) F. Bry, F. Kneißl, K. A. Weiand, and T. Furche, IJSNM 1, 141 (2012).
  • Louati et al. (2012) A. Louati, J. E. Haddad, and S. Pinson, in ICSOC, Lecture Notes in Computer Science, Vol. 7636, edited by C. Liu, H. Ludwig, F. Toumani, and Q. Yu (Springer, 2012) pp. 664–671.
  • Skvoretz and Agneessens (2007) J. Skvoretz and F. Agneessens, Quality & Quantity 41, 341 (2007).
  • Bródka et al. (2012) P. Bródka, T. Filipowski, and P. Kazienko, CoRR abs/1209.6050 (2012).
  • Bródka et al. (2011) P. Bródka, K. Skibicki, P. Kazienko, and K. Musial, in Computational Aspects of Social Networks (CASoN), 2011 International Conference on (IEEE, 2011) pp. 237–242.
  • Bródka et al. (2013) P. Bródka, T. Filipowski, and P. Kazienko, in Information Systems, E-learning, and Knowledge Management Research (Springer, 2013) pp. 185–190.
  • Li et al. (2012) C. Li, J. Luo, J. Huang, and J. Fan, in Intelligence and Security Informatics, Lecture Notes in Computer Science, Vol. 7299, edited by M. Chau, G. Wang, W. Yue, and H. Chen (Springer Berlin Heidelberg, 2012) pp. 60–72.
  • Cozzo et al. (2012) E. Cozzo, A. Arenas, and Y. Moreno, Physical Review E 86 (2012), 10.1103/physreve.86.036115, arXiv:1205.3111 .
  • Mucha et al. (2010) P. J. Mucha, T. Richardson, K. Macon, M. A. Porter, and J.-P. Onnela, Science 328, 876 (2010).
  • (23) S. A. Applin and M. Fischer, The 111th Annual Meeting of the American Anthropological Association (AAA) .
  • Ambrosino and Sciomachen (2012) D. Ambrosino and A. Sciomachen, European Transport\ Trasporti Europei (2012).
  • LeBlanc (1988) L. J. LeBlanc, Transportation Research Part B: Methodological 22, 383 (1988).
  • Nagurney (2000) A. Nagurney, Mathematical and Computer Modelling 32, 393 (2000).
  • Dong et al. (2012) Y. Dong, J. Tang, S. Wu, J. Tian, N. V. Chawla, J. Rao, and H. Cao, in Data Mining (ICDM), 2012 IEEE 12th International Conference on (IEEE, 2012) pp. 181–190.
  • Magnani and Rossi (2013) M. Magnani and L. Rossi, in Social Computing, Behavioral-Cultural Modeling and Prediction (Springer, 2013) pp. 257–264.
  • Cellai et al. (2013) D. Cellai, E. López, J. Zhou, J. P. Gleeson, and G. Bianconi, arXiv preprint arXiv:1307.6359 (2013).
  • Lee et al. (2012) K.-M. Lee, J. Y. Kim, W. kuk Cho, K.-I. Goh, and I.-M. Kim, New Journal of Physics 14, 033027 (2012).
  • Chen et al. (2012) A. L. Chen, F. J. Zhang, and H. Li, Acta Mathematica Sinica, English Series 28, 941 (2012).
  • Ranola et al. (2010) J. M. O. Ranola, S. Ahn, M. E. Sehl, D. J. Smith, and K. Lange, Bioinformatics 26, 2004 (2010).
  • Bianconi (2013) G. Bianconi, Phys. Rev. E 87, 062806 (2013).
  • Fortunato (2010) S. Fortunato, Physics Reports 486, 75 (2010).
  • Newman (2006) M. E. Newman, Proceedings of the National Academy of Sciences 103, 8577 (2006).
  • Radicchi et al. (2004) F. Radicchi, C. Castellano, F. Cecconi, V. Loreto, and D. Parisi, Proceedings of the National Academy of Sciences 101, 2658 (2004).
  • Yin and Khaing (2013) Z. M. Yin and S. S. Khaing, International Journal of Advanced Research in Computer Engineering & Technology 2 (2013).
  • Kühn and Osthus (2007) D. Kühn and D. Osthus, Combinatorics, Probability & Computing 16, 277 (2007).
  • Patel (2008) V. Patel, Journal of Graph Theory 57, 19 (2008).
  • Garey and Johnson (1990) M. R. Garey and D. S. Johnson, Computers and Intractability; A Guide to the Theory of NP-Completeness (W. H. Freeman & Co., New York, NY, USA, 1990).
  • Nefedov (2011) N. Nefedov, in WIMS, edited by R. Akerkar (ACM, 2011) p. 64.
  • Berlingerio et al. (2013) M. Berlingerio, F. Pinelli, and F. Calabrese, Data Mining and Knowledge Discovery 27, 294 (2013).
  • Kettenring (1971) J. R. Kettenring, Biometrika 58, pp. 433 (1971).
  • Girvan and Newman (2002) M. Girvan and M. E. Newman, Proceedings of the National Academy of Sciences 99, 7821 (2002).
  • Harshman (1970) R. A. Harshman, UCLA Working Papers in Phonetics 16, 84 (1970).
  • Dunlavy et al. (2011) D. M. Dunlavy, T. G. Kolda, and W. P. Kegelmeyer, Graph Algorithms in the Language of Linear Algebra, J. Kepner and J. Gilbert, eds., Fundamentals of Algorithms, SIAM, Philadelphia , 85 (2011).
  • Leginus et al. (2012) M. Leginus, P. Dolog, and V. Žemaitis, in User Modeling, Adaptation, and Personalization, Lecture Notes in Computer Science, Vol. 7379, edited by J. Masthoff, B. Mobasher, M. Desmarais, and R. Nkambou (Springer Berlin Heidelberg, 2012) pp. 151–163.
  • Mirzal and Furukawa (2010) A. Mirzal and M. Furukawa, “Node-Context Network Clustering using PARAFAC Tensor Decomposition,” (2010), arXiv:1005.0268 .
  • Papalexakis et al. (2013) E. E. Papalexakis, L. Akoglu, and D. Ienco, in 16th International Conference on Information Fusion (2013).
  • Erdös and Rényi (1960) P. Erdös and A. Rényi, in PUBLICATION OF THE MATHEMATICAL INSTITUTE OF THE HUNGARIAN ACADEMY OF SCIENCES (1960) pp. 17–61.
  • Watts and Strogatz (1998) D. J. Watts and S. H. Strogatz, nature 393, 440 (1998).
  • Albert and Barabási (2002) R. Albert and A.-L. Barabási, Reviews of modern physics 74, 47 (2002).
  • Loe and Jensen (2013) C. W. Loe and H. J. Jensen, Journal of Physics A: Mathematical and Theoretical 46, 245002 (2013).
  • Cardillo et al. (2013) A. Cardillo, J. Gómez-Gardeñes, M. Zanin, M. Romance, D. Papo, F. del Pozo, and S. Boccaletti, Scientific reports 3 (2013).
  • Manning et al. (2008) C. D. Manning, P. Raghavan, and H. Schütze, Introduction to information retrieval, Vol. 1 (Cambridge university press Cambridge, 2008).
  • Lancichinetti et al. (2009) A. Lancichinetti, S. Fortunato, and J. Kertész, New Journal of Physics 11, 033015 (2009).
  • Hubert and Arabie (1985) L. Hubert and P. Arabie, Journal of classification 2, 193 (1985).
  • Collins and Dent (1988) L. M. Collins and C. W. Dent, Multivariate Behavioral Research 23, 231 (1988).
  • Mikko Kivela (2013) M. B. J. P. G. Y. M. M. A. P. Mikko Kivela, Alexandre Arenas, arXiv:1309.7233 (2013).