CAREER: Scalable Techniques for Visualizing Very Large Graphs

Award Number: 1652846
Project Duration (expected): 09/01/2017 - 08/31/2025

Principal Investigator: Hongfeng Yu
University of Nebraska-Lincoln

Abstract

Graphs, also called networks, are important data structures used to represent structural relationships between different entities. Graph models have been ubiquitously employed in scientific applications (e.g., computational molecular biology and ecology) and industrial applications (e.g., world wide web and social network services). With advanced computing techniques, real-world applications can generate graph datasets of unprecedented scales, such as a worldwide social network, where a large graph can contain billions or trillions of vertices (identities) and edges (relationships). Graph visualization, creating visual or diagrammatic representations of graphs, has been commonly used as an effective means to facilitate users to gain meaningful overviews of graph structures and capture regions of interest. However, there is still a lack of scalable visualization solutions that are efficient and practical for very large graph datasets and allow users to explore and discover possible insights in a timely manner. Such solutions can only be obtained with a holistic coordination of processing, organization, and visualization of large graphs, which however has not been fully investigated in most of the previous graph visualization work. This project seeks techniques to ensure the scalability and the usability of large graph visualization by tackling an end-to-end graph visualization pipeline including graph processing, organization, and visualization. Since graphs exist in many scientific and industrial applications as a critical data model, through the outreach activities and the collaboration with domain experts, graph visualization tools and optimization techniques developed in this research can greatly benefit a broader class of fields and communities. The education objective of this project is to leverage real-world large graph visualization to effectively promote students' engagement and learning efficiency in science and engineering studies. In particular, the project aims to leverage interdisciplinary synergies to develop and renovate undergraduate and graduate courses to facilitate the learning of both major and non-major students. Interdisciplinary graph applications will be used to enhance outreach, student recruitment, and research opportunities. Teaching effectiveness will be assessed with an involvement of educators from different disciplines.

The project will develop scalable visualization techniques for very large graphs by exploiting graph structure properties and computer systems optimization. To accomplish this goal, the project has the following objectives. First, scalable parallel clustering methods will be developed to extract sub-graphs with dense intra-connections from large graphs. Second, new algorithms will be designed to address the fundamental data locality problem by identifying and organizing sub-graphs according to their potential access patterns in support of graph processing and visualization. Third, structure-aware visualization techniques, adopting a hierarchical manner, will be developed to provide users an efficient and effective visual guide for large graph exploration. These results will transform conventional visualization methods, which were not ready for handling graphs with billions or trillions of vertices and edges, to efficient and practical techniques for real-world large-scale graph applications. The research and education results from this project will be disseminated in premier conferences and journals and other forms.

Collaborators

Berthe Y. ChoueiryUNL
Yufeng GeUNL
Bonita SharifUNL
Harkamal WaliaUNL
Kwo-Sen KuoNASA
Qi ZhangUNL

Students

Tian GaoGraduate Student
Ian HowellGraduate Student
Yu PanGraduate Student
Jianxin SunGraduate Student
Yves TuyishimeGraduate Student
Jieting WuGraduate Student
Xinyan XieGraduate Student
Jianping ZengGraduate Student
Li ZhangGraduate Student
Feiyu ZhuGraduate Student
Mina Shijie WangUndergraduate Student
Winston HouUndergraduate Student
Tong WuUndergraduate Student

Publications

RmdnCache: Dual-Space Prefetching Neural Network for Large-Scale Volume Visualization
Jianxin Sun, Xinyan Xie, Hongfeng Yu.
IEEE Transactions on Visualization and Computer Graphics. 2024.
DOI:10.1109/TVCG.2024.3410091
[PAPER][VIDEO]
 
Quantitatively Evaluating the Validity of Contrastive Generators for Recourse
Ian Howell, Eleanor Quint, Hongfeng Yu.
Proceedings of 2023 IEEE International Conference on Big Data (BigData), Sorrento, Italy, 2023.
DOI:10.1109/BigData59044.2023.10386609
[PAPER]
 
LM-DiskANN: Low Memory Footprint in Disk-Native Dynamic Graph-Based ANN Indexing
Yu Pan, Jianxin Sun, Hongfeng Yu.
Proceedings of 2023 IEEE International Conference on Big Data (BigData), Sorrento, Italy, 2023.
DOI:10.1109/BigData59044.2023.10386517
[PAPER]
 
Accelerating Web-based Graph Visualization with Pixel-Based Edge Bundling
Jieting Wu, Jianxin Sun, Xinyan Xie, Tian Gao, Yu Pan, Hongfeng Yu.
Proceedings of 2023 IEEE International Conference on Big Data (BigData), Sorrento, Italy, 2023.
DOI:10.1109/BigData59044.2023.10386295
[PAPER]
 
Visualization of 3D Hyperspectral Soil Mapping Data via Autoencoder-based Clustering
Jianxin Sun, Xinyan Xie, Yu Pan, Yakub Islamov, Yufeng Ge, Hongfeng Yu.
Proceedings of 2023 IEEE International Conference on Big Data (BigData), Sorrento, Italy, 2023.
DOI:10.1109/BigData59044.2023.10386924
[PAPER]
 
Partition and Visualization of Earth Network Data by Distributed Edge Bundling
Xinyan Xie, Hongfeng Yu.
Proceedings of 2023 The Nebraska Academy of Sciences, April 21, 2023.
 
Assessing Deep Neural Networks as Probability Estimators
Yu Pan, Kwo-Sen Kuo, Michael L. Rilee, Hongfeng Yu.
Proceedings of 2021 IEEE International Conference on Big Data (Big Data), December 15-18, 2021.
DOI:10.1109/BigData52589.2021.9671328
[PAPER]
 
Interactive Visualization of Hyperspectral Images based on Neural Networks
Feiyu Zhu, Yu Pan, Tian Gao, Harkamal Walia, Hongfeng Yu.
IEEE Computer Graphics and Applications, vol. 41, no. 5, pp. 57-66, 1 Sept.-Oct. 2021.
DOI: 10.1109/MCG.2021.3097730
[PAPER][VIDEO]
 
A Distributed Algorithm for Force Directed Edge Bundling
Yves Tuyishime, Yu Pan, Hongfeng Yu.
Proceedings of IEEE Symposium on Large Data Analysis and Visualization (LDAV), October, 2020.
DOI: 10.1109/LDAV51489.2020.00013
[PAPER]
 
Visualizations to Summarize Search Behavior
Ian Howell, Berthe Choueiry, Hongfeng Yu.
Principles and Practice of Constraint Programming. CP 2020. Lecture Notes in Computer Science, vol 12333. Springer, Cham.
DOI: https://doi.org/10.1007/978-3-030-58475-7_23
[PAPER]
 
Effectively Unified optimization for Large-scale Graph Community Detection
Jianping, Hongfeng Yu.
Proceedings of 2019 IEEE International Conference on Big Data (Big Data), Los Angeles, CA, December 9-12, 2019.
DOI: 10.1109/BigData47090.2019.9005481
[PAPER]
 
Eirene: Improving Short Job Latency Performance with Coordinated Cold Data Migration and Scheduler-Aware Task Cloning
Wei Zhou, K. Preston White, Hongfeng Yu.
Proceedings of 2019 IEEE International Conference on Big Data (Big Data), Los Angeles, CA, December 9-12, 2019.
DOI: 10.1109/BigData47090.2019.9006575
[PAPER]
 
Prediction Approach for Ising Model Estimation
Jinyu Li, Yu Pan, Hongfeng Yu, Qi Zhang.
Proceedings of 2019 International Conference on Data Mining Workshops (ICDMW), Beijing, China, November 8-11, 2019.
DOI: 10.1109/ICDMW.2019.00106
[PAPER]
 
Exploring Eye Tracking Data on Source Code via Dual Space Analysis
Li Zhang, Jianxin Sun, Cole Peterson, Bonita Sharif, Hongfeng Yu.
Proceedings of 2019 Working Conference on Software Visualization (VISSOFT), Cleveland, OH, September 30-October 1, 2019.
DOI: 10.1109/VISSOFT.2019.00016
[PAPER]
 
Distributed Edge Bundling for Large Graphs
Yves Tuyishime.
Master Thesis, University of Nebraska-Lincoln, August, 2019.
 
Exploring Eye Tracking Data on Source Code via Dual Space Analysis
Li Zhang.
Master Thesis, University of Nebraska-Lincoln, May, 2019.
Deploying, Improving and Evaluating Edge Bundling Methods For Visualizing Large Graphs
Jieting Wu.
Ph.D. Dissertation, University of Nebraska-Lincoln, December, 2018.
 
A Scalable Distributed Louvain Algorithm for Large-scale Graph Community Detection
Jianping Zeng, Hongfeng Yu.
Proceedings of IEEE Cluster (CLUSTER), September, 2018.
DOI: 10.1109/CLUSTER.2018.00044
[PAPER]
 
An Information-Theoretic Framework for Evaluating Edge Bundling Visualization
Jieting Wu, Feiyu Zhu, Xin Liu, Hongfeng Yu.
Entropy, 20(9), 625, 2018.
DOI: 10.3390/e20090625
[PAPER]
 
A Distributed Infomap Algorithm for Scalable and High-Quality Community Detection
Jianping Zeng, Hongfeng Yu.
Proceedings of 47th International Conference on Parallel Processing Proceedings (ICPP), August, 2018.
DOI:10.1145/3225058.3225137
[PAPER]
 
A Qualitative Analysis of Search Behavior: A Visual Approach
Ian Howell, Robert Woodward, Berthe Choueiry, Hongfeng Yu.
IJCAI/ECAI 2018 Workshop on Explainable Artificial Intelligence (XAI), July, 2018.
 
Scalable Parallel Community Detection for Large-Scale Graphs
Jianping Zeng.
Ph.D. Dissertation, University of Nebraska-Lincoln, December, 2017.
 
MLSEB: Edge Bundling Using Moving Least Squares Approximation
Jieting Wu, Jianping Zeng, Feiyu Zhu, Hongfeng Yu.
Graph Drawing and Network Visualization. GD 2017. Lecture Notes in Computer Science, vol 10692. Springer, Cham.
DOI: 10.1007/978-3-319-73915-1_30
[PAPER]

Resources - Software and Dataset

Outreach

 
Our visualization lab participated in Lincoln's 2023 Hour of Code & Tech Fair, December, 2023   Girls Code Lincoln's visit at our visualizaiton lab, November, 2023  
 
Girls Code Lincoln's visit at our visualizaiton lab, November, 2023   Xinyan's presentation at Nebraska Academy of Sciences, April, 2023  
 
The first in-person hour of code event in over two years was a big success, December 12, 2022   Xinyan's presentation at IEEE VIS, October, 2022  
 
LPS (Lincoln Public School) Science Connector, August, 2022   We increamentally resume in-person outreach activities, June, 2021  
 
Our booth at the third annual Introduce a Girl to Engineering day, January, 2020  
 
Our Lab Attracted Young Students Again at UNL & LPS Annual Hour of Code Event, December, 2019  
 
Our Visualization Booth at the UNL Annual "Introduce a Girl to Engineering Day" in May 2019  
 
Jianping's presentation at IEEE CLUSTER, September, 2018   Our Visualization Demo at UNL & LPS Annual Hour of Code Event, December, 2018  
 
Jieting presented our MLSEB paper at Graph Drawing & Network Visualization (GD), September, 2017   Jianping's Presentation at the 47th International Conference on Parallel Processing (ICPP), August, 2018  

Highlights

04/29/2024 Jianxin successfully defended his Ph.D. disseration.
12/02/2023 Our visualization lab participated in Lincoln's 2023 Hour of Code & Tech Fair event at Nebraska Innovation Campus, which had 570 people register for the event and checked in 600 attendees. A media report on the event can be found at here.
11/12/2023 Girls Code Lincoln visited our visualization lab, and girls had hands-on activities on interactive visualization.
12/12/2022 The first in-person hour of code event in over two years was a big success! We have employed and demonstrated our visualization results. It was estimated that 425-450 people attended the event. A media report on the event can be found at here.
08/10/2022 We participated in LPS (Lincoln Public School) Science Connector to provide LPS teachers with information about our research and explore possibilities for science education in public school.
08/30/2021 Congratulations! Yu joined the faculty member in the Department of Biological System Engineering (BSE) at the University of Nebraska-Lincoln.
08/05/2021 Yu successfully defended his Ph.D. dissertation.
07/23/2020 Feiyu successfully defended his Ph.D. dissertation.
01/25/2020 Our lab hosted a booth with visualization demonstrations at the third annual Introduce a Girl to Engineering day.
12/07/2019 Our lab attracted young students again at UNL & LPS Annual Hour of Code Event, December, 2019.
07/23/2019 Yves successfully defended his Master thesis on distributed edge bundling.
05/26/2019 Li successfully defended her Master thesis on eye tracking data visualization.
05/11/2019 We hosted a visualization booth at the UNL Annual "Introduce a Girl to Engineering Day" in May 2019. The event geared towards girls in grades three through high school. There were over 500 registered.
12/08/2018 We had an interactive scientific visualization booth at Hour of Code 2018, an annual event co-hosted by University of Nebraska-Lincoln (UNL) and Lincoln Public Schools (LPS) for promoting interdisciplinary computer technology. There were 980 people registered for this event, including 459 adults and 521 students (most k-12 students).
11/27/2018 Jieting successfully defended his Ph.D. dissertation.
05/14/2018 Jieting gained an internship opportunity at NovuMind Inc.
04/27/2018 CSCE486/488 showcase was held at Nebraska Innovation Campus.
[NEWS] | [REPORT]


This material is based upon work supported by the National Science Foundation under Grant No. 1652846. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation.

The page is created on 3/1/2017, and last updated on 09/03/2024.