Identifying Anomalies in GridFTP transfers for Data-Intensive Science through Application-Awareness

Abstract

Network anomaly detection systems can be used to identify anomalous transfers or threats, which, when undetected, can trigger large-scale malicious events. Data-intensive science projects rely on high-throughput computing and high-speed networking resources for data analysis and processing. In this paper, we propose an anomaly detection framework and architecture for identifying anomalies in GridFTP transfers. Application-awareness plays an important role in our proposed architecture and is used to communicate GridFTP application metadata to the machine learning and anomaly detection system. We demonstrate the effectiveness of our architecture by evaluating the framework with a real-world, large-scale dataset of GridFTP transfers. Preliminary results show that our framework can be used to develop novel anomaly detection services with diverse feature sets for distributed and data-intensive projects.