Validation goals and metrics: project deliverable D7.3.2

Abstract

The document at hand describes the second and final iteration of validation goals and metrics of the CACTOS software components. This deliverable D7.3.2 updates the first iteration (D7.3.1 Validation Goals and Metrics) and takes recent changes in tooling and a new use case scenario, but also results from the first validation cycle (D7.4.1 Validation and Result Analysis) into account. Global validation goals have been derived in (D7.3.1 Validation Goals and Metrics) based on scenario requirements and objectives from the description of work (DoW). Further and more detailed validation goals and metrics is presented in a split view, first from the CACTOS scenarios, and second from the CACTOS tools perspective. The scenarios are namely Business Analytics (by Flexiant), Scientific Computing (by University of Ulm), and Enterprise Applications (by PlayGen). The tools considered are CACTOS’ three main components CactoScale, CactoOpt and CactoSim. This document outlines at least one validation scenario per tool. A validation scenario is declared as successful when all of its acceptance criteria are met. The validation scenarios described in the document at hand for business analytics are #BC.1 NODE LOAD DISTRIBUTION and #BC.2 FAULT TOLERANCE. Both will be mostly validated on the Flexiant testbed. The validation scenarios for scientific computation are #SC.1 Running Molpro in a virtualised environment, #SC.2 Deploying Molpro through CACTOS, #SC.3 Monitoring Molpro instances, #SC.4 Mine Molpro traces, #SC.5 Prediction of execution time and execution phases, #SC.6 On-line Phase Detection of Running Applications, #SC.7 Failure Detection and Snapshots, #SC.8 Application Restart from Snapshots in Case of Failures, #SC.9 Concurrent Usage of Resources, #SC.10 Phase-aware scheduling, #SC.11 Detect Lack of Resources, #SC.12 Simulate Spare Resources, #SC.13 Simulate Change of Resource Allocation, and #SC.14 Control Power State of Physical Resources. The validation scenarios for enterprise application are Use Case I – Initial System Setup, Use Case II – User-Driven Optimisation, Use Case III – Automatic Optimisation, and Use Case IV – Scaling. The validation for CactoSim is delivered in (D6.5 Final results from optimization algorithms validation and experimentation) and hence not listed here. CactoOpt defines in the document at hand the validation scenarios Periodic Optimisation, Event driven Optimisation, and Previous Optimisation Plan in Progress. CactoScale defines the validation scenarios Cloud Platform Data Collection, Infrastructure Model Generation, and Offline Log Analysis. Concluding, the set of validation scenarios are the fundamental guide for the second and final validation of the CACTOS software in (D7.4.2 Validation and Result Analysis) at the end of the project. The validation scenarios together cover the global validation goals and requirements, requested by the use case scenarios and the DoW’s objectives.