Performance evaluation of the CACTOS toolkit on a small cloud testbed: project deliverable D5.5

Abstract

This document describes the performance and scalability evaluation for the CACTOS Runtime Toolkit. The evaluation is conducted on the basis of the evaluation methodology documented in (D5.4 Evaluation Methology for the CACTOS Runtime and Prediction Toolkits). It concludes the integration of the CACTOS components into the CACTOS Runtime Toolkit. The evaluation investigates performance scalability for the final iteration of the CACTOS Runtime Toolkit prototype (D5.2.2 CACTOS Toolkit Version 2) in the two CACTOS testbeds. The two CACTOS testbeds use the OpenStack and Flexiant Cloud Orchestrator (FCO) variants of the CACTOS Runtime Toolkit as documented in (D7.3.2 Validation Goals and Metrics). For both testbeds a set of scenarios representative of the CACTOS use cases Business Analytics (FLEXIANT), Scientific Computing (UULM) and DataPlay (PlayGen) are conducted. The evaluation investigates whether the scenarios meet the use case specific requirements regarding performance and scalability. For the investigated scenarios, CACTOS meets the large majority of all performance and scalability requirements. All remaining performance and scalability requirements are partially achieved, and can be achieved by reconfiguring the CACTOS Runtime Toolkit. The evaluation of performance and scalability was conducted in coordination with the final validation of functionality and robustness of the CACTOS Runtime Toolkit that will be presented in (D7.4.2 Validation and Result Analysis). In addition to the scenario-specific evaluation on the CACTOS testbeds, the performance and scalability of the framework technologies used by CACTOS are evaluated beyond the scope and size supported by the testbed infrastructure. The two core technologies evaluated are Chukwa and Connected Data Objects (CDO). Chukwa is used by CactoScale to collect monitoring data and propagate it to the Historic Database as explained in (D4.4 Integrated Data Collection and Analysis Framework). The scalability and performance of CDO is shown using a set of load drivers that replicate the load caused by the components found in CACTOS Runtime Toolkit. Across all evaluated model sizes representing data centres with up to 1000 nodes and 50 000 VMs, no notable performance degradation or performance bottlenecks could be identified. The results of the presented evaluation showcase that the CACTOS Runtime Toolkit and the underlying technology stack are fit for handling the use cases Business Analytics, Scientific Computing and White Box Application management by the example of DataPlay.