A Comparative Study on Storage Solutions for Analysis of Streaming Telemetry Data

Streaming telemetry refers to the collection of real-time data from various devices, servers and applications. It’s a highly data-intensive process and requires new storage methods. In recent years, cloud storage has become a popular solution for storing large amounts of unstructured data. This paper explores the benefits of storing streaming telemetry data over cloud services and reviews the key considerations for selecting a cloud storage solution that can provide scalability, flexibility, accessibility, security, and cost-effectiveness. It compares various time-series databases and cloud storage services and also contrasts storing telemetry data in a time-series database (TSDB) versus a cloud storage solution. While the current norm is to store telemetry data in a TSDB for faster real-time data analysis, moving data to the cloud could drastically reduce the cost of storage. Based on this inference, a novel methodology for the storage of routing data is proposed, where the archival data can be moved to cloud and the audit data can be stored in a TSDB, utilizing the benefits of both approaches. This approach can be implemented in organizations using Telemetry data for analysis of networks and servers. ...