Integrated data collection and analysis frameworks: project deliverable D4.4

Abstract

CactoScale is a CACTOS component which provides monitoring and data analysis functionality. This deliverable presents the integration of the algorithmic data analysis framework and the data collection tool in a production-mode setup. For the setup we utilised the testbed provided by the industrial partner Flexiant and we examined CactoScale under the workload provided by DataPlay application from Playgen. In this deliverable we measure the performance of CactoScale when this is integrated with Flexiant Cloud Orchestrator (FCO) and the University of Ulm (UULM) testbeds under the impact of Playgen’s enterprise application (DataPlay) workload. We assess the scaling capability of the monitoring framework by measuring the overall latency under increasing numbers of VMs on the platform’s nodes. Furthermore, we assess any overhead induced by CactoScale on the integrated use case of Playgen when deployed on FCO. CactoScale also provides data analysis functionality to CACTOS. We demonstrate the performance of the analysis framework when integrated on the FCO-testbed and the UULM testbed. We use data collected from monitoring the deployed DataPlay application to perform correlation analysis using a Lightweight Anomaly Detection Tool (LADT). LADT utilises data correlation analysis to indicate any potential anomalies on a cloud compute node. To experiment on anomaly detection analysis we employ the DICE-fault-injection tool which allows fault injection on the Flexiant’s platform. We use DataPlay’s workload measurements to further estimate the scalability of the analytics tool by comparing the performance when different numbers of CPU cores are used for the analysis. Further CactoScale is integrated with the rest of the CACTOS ecosystem through the development of a prediction tool which is used by CactoOpt and CactoSim to enable cloud operators to make informed choices for deploying workloads. This integration has been evaluated using a computational quantum chemistry tool named MOLPRO. The evaluation considers stress testing the monitoring infrastructure both on single and multiple nodes. The experimental evaluations highlight that CactoScale offers scalable and low overhead monitoring and data analysis in a cloud environment.