Search

Big graphs are so large that their analysis often rely on the cloud for parallel processing. Data placement, as a key pre-processing step, has a profound impact on the performance of parallel processing. Traditional placement methods fail to preserve graph topologies, leading to poor performance. As the community is the most common structure of big graphs, in this work, we present a two-phase community-aware placement algorithm to place big graphs into the cloud for parallel processing. It can obtain a placement scheme that preserves the community structure well by maximizing the modularity density of the scheme under memory capacity constraints of computational nodes of the cloud in two phases. In the first phase, we design a streaming partitioning heuristic to detect communities based on partial and incomplete graph information. They form an initial placement scheme with relatively high modularity density. To improve it further, in the second phase, we put forward a scale-constrained kernel k-means algorithm. It takes as input the initial placement scheme and iteratively redistributes graph vertices across computational nodes under scale constraints until the modularity density cannot be improved any further. Finally, experiments show that our algorithm can preserve graph topologies well and greatly support parallel processing of big graphs in the cloud.

The power communication network can be abstracted as a graph based on its topology. In this paper, we propose an approach to conduct simulations of power communication network based on its graph representation. In particular, the nodes and edges in the graph refer to the ports and channels in the grid topology. Different applications on the grid can be transformed into queries over the graph. Hence, in this paper, we build our grid simulation model based on the Neo4j graph database. We also propose a fault extension algorithm based on predicate calculus. Our experiment evaluations show that the proposed approach can effectively improve the efficiency of the power grid.

Big graph data is different from traditional data and they usually contain complex relationships and multiple attributes. With the help of graph pattern matching, a pattern graph can be designed, satisfying special personal requirements and locate the subgraphs which match the required pattern. Then, how to locate a graph pattern with better attribute values in the big graph effectively and efficiently becomes a key problem to analyze and deal with big graph data, especially for a specific domain. This article introduces fuzziness into graph pattern matching. Then, a genetic algorithm, specifically an NSGA-II algorithm, and a particle swarm optimization algorithm are adopted for multi-fuzzy-objective optimization. Experimental results show that the proposed approaches outperform the existing approaches effectively.

Graph data analysis is a hot topic in recent research area. Graph classification is one of the most important graph data analysis problems, which choose the most probable class labels of graphs using models based on the training dataset. It has wildly applications in protein group identification, chemical compounds classification and so on. Many existing research of graph learning suffer from high computation cost as large scale graph data are dramatically increased. In order to realize big graph classification with real-time learning ability and good scalability, efficient feature extraction approaches and ELM variants are utilized in this paper. To be specific, we present three frameworks of big graph classification based on ELMs: (1) a framework with a compression-based frequent subgraph mining method to reduce graph size; (2) an incremental framework to handle dynamic graphs; (3) a distributed framework with distributed ELMs to provide good scalability and easy implementation on cloud platforms. Extensive experiments are conducted on clusters with large real-world graph datasets. The experimental results demonstrate that our frameworks are efficient in big graph classification applications, and well suitable for dynamic networks. The results also validate that ELM and its variants have good classification performance on large-scale graphs.

How can we effectively detect fake reviews or fraudulent connections on a website? How can we spot communities that suddenly appear based on users’ interaction? And how can we efficiently find the minimum cut in a big graph? All of these are related to the problem of finding dense subgraphs, an important primitive problem in graph data analysis with extensive applications across various domains.

Graphs are, by nature, "unifying abstractions" that can leverage interconnectedness to represent, explore, predict, and explain real- and digital-world phenomena. although real users and consumers of graph instances and graph workloads understand these abstractions, future problems will require new abstractions and systems. What needs to happen in the next decade for big graph processing to continue to succeed?, Communications of the ACM, 64 (9), ISSN:1557-7317, ISSN:0001-0782

Relations among data items can be modeled with graphs in most of big data sets such as social networks’ data. This modeling creates big graphs with many vertices and edges. Balanced k-way graph partitioning is a common problem with big graphs. It has many applications in several fields. There are many approximate solutions for this problem; however, most of them do not have enough scalability for big graph partitioning and cannot be executed in a distributed manner. Vertex-centric model has been introduced recently as a scalable distributed processing method for big graphs. There are a few methods for graph partitioning based on this model. Existing approaches only consider one-step neighbors of vertices for graph partitioning and do not consider neighbors with higher steps. In this paper, a distributed method is introduced based on vertex-centric model for balanced k-way graph partitioning. This method applies the personalized PageRank vectors of vertices and partitions to decide how vertices are joined partitions. This method has been implemented in the Giraph system. The proposed method has been evaluated with several synthetic and real graphs. Experimental results have shown that this method has scalability for partitioning big graphs. It was also found that this method produces partitions with higher quality compared to the state-of-the-art stream-based methods and distributed methods based on vertex-centric programming model. Its result is close to the results of Metis method.

As a special type of big data, the big graph has wide applications. The remote data integrity checking (RDIC) scheme enables public clouds to efficiently convince the clients that their graph data are stored properly, without the need of retrieving the actual data contents. The existing schemes support verifiable update often by adopting the authentication data structures (ADSs), e.g., Merkel hash tree (MHT). The main obstacle for applying the existing RDIC schemes to big graphs is due to the lack of support for verifiable sub-graph operations with high efficiency. In this paper, we propose a new RDIC scheme for big graphs, called GVD-RDIC, to support public auditing and verifiable dynamic graph updates with high efficiency. We have designed novel ADS based on the graph Voronoi diagram (GVD) and enhanced MHT to address the integrity of graph structure and verifiable sub-graph updates. In addition, our ADS can be applied to any type of graphs. Moreover, our GVD-RDIC scheme adopts a new construction for the homomorphic authenticator to enable index verifications for public auditors. A server response constructed with unchallenged block will be rejected. The proposed scheme is proven secure under a random oracle model. Both the theoretical analysis and the simulation results show that our scheme is practicable for real-world big graphs.

In this paper, an efficient page rank (PR) exact algorithm is proposed, which can improve the computation efficiency without sacrificing results accuracy. The existing exact algorithms are generally based on the original power method (PM). In order to reduce the number of I/Os required to improve efficiency, they partition the big graph into multiple smaller ones that can be totally fitted in memory. The algorithm proposed in this paper can further reduce the required number of I/Os. Instead of partitioning the graph into the general subgraphs, our algorithm partitions graph into a special kind of subgraphs: SCCs (strongly connected components), the nodes in which are reachable to each other. By exploiting the property of SCC, some theories are proposed, based on which the computation iterations can be constrained on these SCC subgraphs. Our algorithm can reduce lots of I/Os and save a large amount of computations, as well as keeping the results accuracy. In a word, our algorithm is more efficient among the existing exact algorithms. The experiments demonstrate that the algorithms proposed in this paper can make an obvious efficiency improvement and can attain high accurate results.

Graph pattern matching is widely used in big data applications. However, real-world graphs are usually huge and dynamic. A small change in the data graph or pattern graph could cause serious computing cost. Incremental graph matching algorithms can avoid recomputing on the whole graph and reduce the computing cost when the data graph or the pattern graph is updated. The existing incremental algorithm PGC_IncGPM can effectively reduce matching time when no more than half edges of the pattern graph are updated. However, as the number of changed edges increases, the improvement of PGC_IncGPM gradually decreases. To solve this problem, an improved algorithm iDeltaP_IncGPM is developed in this paper. For multiple insertions (resp., deletions) on pattern graphs, iDeltaP_IncGPM determines the nodes’ matching state detection sequence and processes them together. Experimental results show that iDeltaP_IncGPM has higher efficiency and wider application range than PGC_IncGPM.

Privacy protection is one of the most challenging problems of social networks. Simple removal of the labels is unable to protect the privacy of social networks because the information of graph structures can be utilized to deanonymize target nodes. Previous related proposals mostly assume that attacker knows only the target's neighborhood graph, but ignoring of signed edge attribute. The graph structure with signed edge attributes could cause serious privacy leakage of social networks. In this paper, we take the signed attribute of edges into account when achieving k-anonymity privacy protection for social networks. We propose a signed k-anonymity scheme to protect the privacy of key entities in social networks. With signed k-anonymity protection, these targets cannot be re-identified by attackers with confidence higher than 1 / k. The proposed scheme minimizes the modification, which preserves high utility of the original data. Extensive experiments on real data sets and synthetic graph illustrate the effectiveness of the proposed scheme.

Graph computing is widely applied in a large number of big data applications. Despite its importance, high performance graph computing remains a challenge, especially for large-scale graphs. In this paper, by analyzing from the architectural perspective, we study computational behaviors of graph computing in real-world use cases. We benchmark a set of representative graph algorithms implemented on a unified framework and conduct experiments to analyze comprehensive performance characteristics. In the characterization, we observed multiple insights, including irregular memory patterns, significant diverse behavior across different computations, highly data dependent behaviors, etc., using large-scale synthetic and real-world graphs. To the best of our knowledge, this is the first comprehensive architectural study on the full-scope of graph computing. It can improve our understanding on graph computing and help high performance computing research for graph-based big data applications.

In analysis of ubiquitous networks, one critical work is to understand the internal structure of the networks. Many approaches are proposed to address the structure mining in the graph. Unfortunately, these approaches cannot cope with the big graph generated by large-scale streaming data. Aiming to mine the structure from the big graph, this paper presents an efficient mining framework, namely cSketch, which is used for capturing the cliques from big graph quickly. Two illustrative examples are conducted for demonstrating the feasibility and effectiveness of the proposed framework.

Driven by a multitude of use cases, graph data analytics has become a hot topic in research and industry. Particularly on big graphs, performing complex analytical queries efficiently to derive new insights is a challenging task. Systems that aim at solving the technical part of this challenge are often referred to as graph processing systems. They allow expressing and executing analytic algorithms and queries, while hiding most of the technical details related to efficiently storing and processing graph data. Since 2010, work on graph processing systems for distributed systems as well as shared memory systems has virtually exploded. In this article, we give an overview of this work with the particular focus on graph processing systems for large multiprocessor machines. We describe the state of the art established in recent years and outline trends and challenges in research and development that point towards the future of graph processing systems.

Several models have been proposed to cope with the rapidly increasing size of data, such as Anchor Graph Regularization (AGR). The AGR approach significantly accelerates graph-based learning by exploring a set of anchors. However, when a dataset becomes much larger, AGR still faces a big graph which brings dramatically increasing computational costs. To overcome this issue, we propose a novel Hierarchical Anchor Graph Regularization (HAGR) approach by exploring multiple-layer anchors with a pyramid-style structure. In HAGR, the labels of datapoints are inferred from the coarsest anchors layer by layer in a coarse-to-fine manner. The label smoothness regularization is performed on all datapoints, and we demonstrate that the optimization process only involves a small-size reduced Laplacian matrix. We also introduce a fast approach to construct our hierarchical anchor graph based on an approximate nearest neighbor search technique. Experiments on million-scale datasets demonstrate the effectiveness and efficiency of the proposed HAGR approach over existing methods. Results show that the HAGR approach is even able to achieve a good performance within 3 minutes in an 8-million-example classification task.

We propose a new design of universal designated verifier transitive signatures, to authenticate dynamically growing big graph data. The scheme is built on the classical RSA signature and possesses several desirable properties. It supports edge-signature composition as transitive signatures, i.e., with the signatures of two adjacent edges ( i , j ) and ( j , k ) , one can obtain a valid signature of the edge ( i , k ) . Additionally, a signature holder can convince only one designated verifier about the existence of an edge. Our design can efficiently achieve a tradeoff between data authenticity (when publishing dynamically growing big graph data) and data privacy (when disseminating big graph data).

Recently, graph edge partitioning has shown better partitioning quality than the vertex graph partitioning for the skewed degree distribution of real-world graph data. Graph edge partitioning can be classified as stream and offline. The stream edge partitioning approach supports a big graph partitioning; however, it has lower partitioning quality, is affected by stream order, and it has taken much time to make partitioning compared with the offline edge partitioning. Conversely, the offline edge partitioning approach has better partitioning quality than stream edge partitioning; however, it does not support big graph partitioning. In this study, we propose partial stream hybrid graph edge partitioning OffStreamNG, which leverages the advantage of both offline and stream edge partitioning approaches by interconnecting via saved partition state layer. The OffStreamNG holds vertex and load states as partition state, while the offline component is partitioning using neighborhood expansion heuristic. And it is transferring this partition state to the online component of Greedy heuristic with minor modification of both algorithms. Experimental results show that OffStreamNG achieves attractive results in terms of replication factor, load balance, and total partitioning time.

Due to popularity of social networking in the current era of big data, many social networking sites (e.g., Facebook) has been generating huge volumes of social data. In this paper, we aim to discover interesting relationships in a undirected (social) graph via a theoretical approach. Specifically, we examine the graph theory and linear algebra approaches to find mutual friendships from social networks represented in the form of big graphs or big graph databases.

This article presents a comparison of the computing performance of the MapReduce tool Hadoop and Giraph on large-scale graphs. The main ideas of MapReduce and bulk synchronous parallel (BSP) are reviewed as big data computing approaches to highlight their applicability in large-scale graph processing. This paper reviews the execution performance of Hadoop and Giraph on the PageRank algorithm to classify web pages according to their relevance, and on a few other algorithms to find the minimum spanning tree in a graph with the primary goal of finding the most efficient computing approach to work on large-scale graphs. Experimental results show that the use of Giraph for processing large-size graphs reduces the execution time by 25% in comparison with the results obtained using the Hadoop for the same experiments. Giraph represents the optimal option thanks to its in-memory computing approach that avoids secondary memory direct interaction.

The vertex-centric programming model, known as “think like a vertex”, is being used more and more to support various big graph processing methods through iterative supersteps that execute in parallel a user-defined vertex program over each vertex of a graph. However, the imperative and message-passing style of existing systems makes defining a vertex program unintuitive. In this paper, we show that one can benefit more from “Thinking like a vertex” by “Behaving like a function” rather than “Acting like a procedure” with full use of side effects and explicit control of message passing, state, and termination. We propose a functional approach to vertex-centric graph processing in which the computation at every vertex is abstracted as a higher-order function and present Fregel, a new domain-specific language. Fregel has clear functional semantics, supports declarative description of vertex computation, and can be automatically translated into Pregel, an emerging imperative-style distributed graph processing framework, and thereby achieve promising performance. Experimental results for several typical examples show the promise of this functional approach.

The increasing size of Big Data is often heralded but how data are transformed and represented is also profoundly important to knowledge discovery, and this is exemplified in Big Graph analytics. Much attention has been placed on the scale of the input graph but the product of a graph algorithm can be many times larger than the input. This is true for many graph problems, such as listing all triangles in a graph. Enabling scalable graph exploration for Big Graphs requires new approaches to algorithms, architectures, and visual analytics. A brief tutorial is given to aid the argument for thoughtful representation of data in the context of graph analysis. Then a new algebraic method to reduce the arithmetic operations in counting and listing triangles in graphs is introduced. Additionally, a scalable triangle listing algorithm in the MapReduce model will be presented followed by a description of the experiments with that algorithm that led to the current largest and fastest triangle listing benchmarks to date. Finally, a method for identifying triangles in new visual graph exploration technologies is proposed.

With the growth of the inter-connectivity of the world, Big Graph has become a popular emerging technology. For instance, social media (Facebook, Twitter). Prominent examples of Big Graph include social networks, biological network, graph mining, big knowledge graph, big web graphs and scholarly citation networks. A Big Graph consists of millions of nodes and trillion of edges. Big Graphs are growing exponentially and requires large computing machinery. Big Graph is posing many issues such as storage, scalability, processing and many more. This paper gives a brief overview of in-memory Big Graph Systems and some key challenges. Also, sheds some light on future research agendas of in-memory systems.

Subgraph matching query is to find out the sub-graphs of data graph G which match a given query graph Q. Traditional methods can not deal with big data graphs due to their high computational complex. In this paper, we propose a distributed top-k subgraph search method over big graphs. The proposed method is designed at the level of single vertex and all vertices obtain their matching state separately without requiring global graph information. Therefore, it can be easily deployed in distributed platform like Hadoop. The evaluations of running time, number of messages and supersteps show the efficiency and scalability of the proposed method.

The OpenAIRE infrastructure populates a scholarly communication big graph interlinking metadata objects of publications, datasets, software, organizations, funders, and projects.In order to de-duplicate this graph, OpenAIRE has developed GDup , an integrated, scalable, general-purpose system for entity deduplication over big information graphs. GDup offers functionalities to realize a fully-fledged entity deduplication workflow over a generic input graph, inclusive of Ground Truth support, end-user feedback, and strategies for identifying and merging duplicates to obtain an output disambiguated graph.

Unlike the traditional methods, such as the Independent Cascade Model and Liner Threshold Model. In this paper, we propose a kind of new Degree-priority heuristic IM algorithm in a big graph. We prove this algorithm. We do some experiments and make some comaparision between the Degree-priority heuristic IM algorithm and traditional IM algorithm.

Spectral sparsification is a general technique developed by Spielman et al. to reduce the number of edges in a graph while retaining its structural properties. We investigate the use of spectral sparsification to produce good visual representations of big graphs. We evaluate spectral sparsification approaches on real-world and synthetic graphs. We show that spectral sparsifiers are more effective than random edge sampling. Our results lead to guidelines for using spectral sparsification in big graph visualization.

Given a complex graph of large network, people always want to capture its desirable information of interest. In order to dig the underlying structure of the extremely large graph dataset, compression is necessary, which also helps in further communication and computation. In this paper, by invoking a newly proposed node centrality metric named relative node importance (RNI), an effective compressing scheme is presented for complex graph datasets. Besides measuring the distance and connectivity distribution of the graph structure, we firstly take k-core distribution into consideration. Compared with the existing schemes, the proposed one has lower computational complexity and fits different kinds of networks, e.g., social network, the World Wide Web (WWW) and autonomous systems (AS) network. Numerical results show that our RNI-based method outperforms other schemes and well preserving the basic features of a graph.

We now live surrounded by sensors, we create information continuously and we leave constantly computer traces of our activities. The processing and analysis of this huge volume data, so called Big Data, offer innumerable and still largely unexplored: health (epidemiology, genomics complex energy networks, intelligent cities, forecasting and management of environmental risks, etc. Big Data has, and will increasingly, a very significant impact at the societal economic and commercial levels. Many interesting Big Data problems can be modeled as problems on graphs/networks.

Data sampling has been extensively studied for large scale graph mining. Many analyses and tasks become more efficient when performed on graph samples of much smaller size. The use of proxy objects is common in software engineering for analysis and interaction with heavy objects or systems. In this paper, we coin the term ’proxy graph’ and empirically investigate how well a proxy graph visualization can represent a big graph. Our investigation focuses on proxy graphs obtained by sampling ; this is one of the most common proxy approaches. Despite the plethora of data sampling studies, this is the first evaluation of sampling in the context of graph visualization. For an objective evaluation, we propose a new family of quality metrics for visual quality of proxy graphs. Our experiments cover popular sampling techniques. Our experimental results lead to guidelines for using sampling-based proxy graphs in visualization.

This position paper provides an overview of our recent advances in the study of big graphs, from theory to systems to applications. We introduce a theory of bounded evaluability, to query big graphs by accessing a bounded amount of the data. Based on this, we propose a framework to query big graphs with constrained resources. Beyond queries, we propose functional dependencies for graphs, to detect inconsistencies in knowledge bases and catch spams in social networks. As an example application of big graph analyses, we extend association rules from itemsets to graphs for social media marketing. We also identify open problems in connection with querying, cleaning and mining big graphs.

Many big data applications in business and science require the management and analysis of huge amounts of graph data. Suitable systems to manage and to analyze such graph data should meet a number of challenging requirements including support for an expressive graph data model with heterogeneous vertices and edges, powerful query and graph mining capabilities, ease of use as well as high performance and scalability. In this chapter, we survey current system approaches for management and analysis of “big graph data”. We discuss graph database systems, distributed graph processing systems such as Google Pregel and its variations, and graph dataflow approaches based on Apache Spark and Flink. We further outline a recent research framework called Gradoop that is build on the so-called Extended Property Graph Data Model with dedicated support for analyzing not only single graphs but also collections of graphs. Finally, we discuss current and future research challenges.

Louvain algorithm is known to be fast to deliver good results in detecting communities of big graphs. This paper further speeds up Louvain algorithm by restricting the internal search rules and early pruning the non-promising candidates. Experimental results of the modified algorithm on various sized data show outstanding speedups of up to 40.5 and 4.7 times for weighted graphs and unweighted graphs, respectively.

Big graph mining is an important research area and it has attracted considerable attention. It allows to process, analyze, and extract meaningful information from large amounts of graph data. Big graph mining has been highly motivated not only by the tremendously increasing size of graphs but also by its huge number of applications. Such applications include bioinformatics, chemoinformatics and social networks. One of the most challenging tasks in big graph mining is pattern mining in big graphs. This task consists on using data mining algorithms to discover interesting, unexpected and useful patterns in large amounts of graph data. It aims also to provide deeper understanding of graph data. In this context, several graph processing frameworks and scaling data mining/pattern mining techniques have been proposed to deal with very big graphs. This paper gives an overview of existing data mining and graph processing frameworks that deal with very big graphs. Then it presents a survey of current researches in the field of data mining / pattern mining in big graphs and discusses the main research issues related to this field. It also gives a categorization of both distributed data mining and machine learning techniques, graph processing frameworks and large scale pattern mining approaches., Submitted to Big Data Research, Elsevier

This special issue of Information Visualization explores the technical challenges and technology development opportunities of graph visual analytics arising from the trend of big data. Big graph visual analytics is about applying visualization and analytics techniques to gather, analyze, and understand big graphs and the knowledge behind them.

Big Graph Mining" is a continuously developing research that was started in 2009 until now. After 7 years, there are many researches that put this topic as the main concern. However, there is no mapping or summary concerning the important issues and solutions to explain this topic. This paper contains a summary of researches that have been conducted since 2009. The result is grouped based on the algorithms, built system and also preprocess techniques that have been developed. Based on survey, there are 11 algorithms and 6 distributed systems to analyse the Big Graph have been improved. While improved pre-process algorithm only covers: sampling and compression technique. These improving algorithms are usually aimed to frequent sub graphs discovery, whereas slightly those of is aimed to cluster Big Graph, and there is no algorithm to classify Big Graph. As a conclusion of this survey, there is a need for more researches to be conducted to improve a comprehensive Graph Mining System, especially for very big Graph.

On one hand, compared with traditional relational and XML models, graphs have more expressive power and are widely used today. On the other hand, various applications of social computing trigger the pressing need of a new search paradigm. In this article, we argue that big graph search is the one filling this gap. To show this, we first introduce the application of graph search in various scenarios. We then formalize the graph search problem, and give an analysis of graph search from an evolutionary point of view, followed by the evidences from both the industry and academia. After that, we analyze the difficulties and challenges of big graph search. Finally, we present three classes of techniques towards big graph search: query techniques, data techniques and distributed computing techniques.