Our adaptive volume compression tool is designed to reduce time-varying volume data generated from scientific simulations. The tool leverages recent advances in deep learning architectures and simultaneously trains an encoder and a decoder. The encoder extracts features from each temporal-spatial chunk using several layers of convolutional neural network (CNN) followed by a discrete encoding layer that conducts quantization, and tries to spread the information evenly into each bit. The encoding binary representation then goes through a symmetric structure as the decoder for reconstructing the raw volume data. In this way, this autoencoder structure can learn a compact data representation while preserving most of the temporal-spatial structures of the original data. The following figure summarizes the pipeline of our approach.

The overview of the volume data compression pipeline.

Here is the source code, volume_compression_src.zip, of the adaptive volume compression tool.

The tool supports regular volume data generated from simulations. The tool has been tested using Isabel Hurricane simulation data. The description and the download link of the dataset can be found here.

The tool requires tensorflow, argparse, numpy, and matplotlib.

The code could be run as following steps:

upload to google drive
right click open with: google colaboratory
run each steps

or run locally with:

start jupyter notebook
a web based file system will pop up
search for and open the .ipynb file using the web page
run the .ipynb file

Adaptive Deep Learning based Time-Varying Volume Compression Yu Pan, Feiyu Zhu, Tian Gao, Hongfeng Yu. Proceedings of 2019 IEEE International Conference on Big Data (Big Data), Los Angeles, CA, December 9-12, 2019.
DOI: 10.1109/BigData47090.2019.9006146 [PAPER]

@inproceedings{pan2019adaptive,
  title={Adaptive Deep Learning based Time-Varying Volume Compression},
  author={Pan, Yu and Zhu, Feiyu and Gao, Tian and Yu, Hongfeng},
  booktitle={2019 IEEE International Conference on Big Data (Big Data)},
  pages={1187--1194},
  year={2019},
  organization={IEEE}
}