Show simple item record

AuthorPapazachos, Zafeiriosdc.contributor.author
AuthorBharbuiya, Sakildc.contributor.author
AuthorIbidunmoye, Olumuyiwadc.contributor.author
AuthorMehta, Amardeepdc.contributor.author
AuthorRezaei, Alidc.contributor.author
AuthorTsitsipas, Athanasiosdc.contributor.author
AuthorCastañé, Gabriel Gonzálezdc.contributor.author
AuthorAli-Eldin, Ahmeddc.contributor.author
AuthorNikolopoulos, Dimitrios S.dc.contributor.author
Date of accession2017-04-20T13:36:45Zdc.date.accessioned
Available in OPARU since2017-04-20T13:36:45Zdc.date.available
Year of creation2015-06-30dc.date.created
Date of first publication2017-04-20dc.date.issued
AbstractCactoScale provides monitoring and data analysis functionality to CACTOS. This deliverable presents the framework and algorithms used by CactoScale for parallel trace analysis. We describe different CactoScale framework extensions which enable the implementation of parallel correlation analysis of system utilisation metric traces and cloud data logs. We also present the implementation of Lambda Architecture into CactoScale which parallelises several aspects of monitoring and exchanging information in CACTOS. CactoScale trace analysis tackles parallelism on various dimensions. We describe a hierarchical log analysis and anomaly detection framework. The anomaly detection utilises parallel data analysis frameworks such as Spark and mapreduce framework for parallel analysis of workload traces and system logs, coupled with HDFS for in-memory processing of the data. The trace analysis also involves the pre-processing of raw data logs for storage in HDFS. It allows executing anomaly detection algorithms hierarchically, both utilising the compute nodes in situ and the parallel HDFS monitoring facility. This is feasible by pairing the CactoScale agents with in situ analytics modules to cover the cases such as workload spike detection, but also to filter the data that flows to the database for post-processing. An in situ analytic module is a process designed to run locally in a node. This tactic provides the advantage of data locality. The data are pre-processed by the local node before being collected by a remote distributed service for further processing. In this way, the hierarchical design of data analysis allows for an additional level of real-time processing which is much closer to the data source. CactoScale has different features and capabilities for parallel trace analysis which are demonstrated in this deliverable by using different algorithms for anomaly detection. Anomaly detection involves the use of trace analysis algorithms that detects outliers (numerical, textual, or correlation based) in data traces. Detecting outliers can trigger actions in resource management and for this reason we focus in anomaly detection as a use case. We demonstrate a Lightweight Anomaly Detection Tool based on correlation analysis. This tool utilises a monitoring cluster to perform parallel trace analysis using Spark and mapreduce. The online data analysis modules that we demonstrate include a log analysis module and several spike detection methods. Workload spikes are one of the main causes of QoS degradation in cloud applications. The log analysis demonstrates how information on cloud platform can contribute in reducing any false positive alerts.dc.description.abstract
Languageendc.language.iso
PublisherUniversität Ulmdc.publisher
LicenseCC BY-ND 4.0 Internationaldc.rights
Link to license texthttps://creativecommons.org/licenses/by-nd/4.0/dc.rights.uri
KeywordAnalyticsdc.subject
KeywordAnalysisdc.subject
KeywordClouddc.subject
KeywordFrameworkdc.subject
KeywordOptimisationdc.subject
KeywordSimulationdc.subject
KeywordCactos Projektdc.subject
KeywordData managementdc.subject
KeywordParallel tracedc.subject
KeywordContext-aware cloud topologydc.subject
KeywordCloud servicesdc.subject
Dewey Decimal GroupDDC 004 / Data processing & computer sciencedc.subject.ddc
LCSHCloud computingdc.subject.lcsh
LCSHElectric network topologydc.subject.lcsh
TitleParallel trace analysis: project deliverable D4.3dc.title
Resource typeBerichtdc.type
DOIhttp://dx.doi.org/10.18725/OPARU-4309dc.identifier.doi
URNhttp://nbn-resolving.de/urn:nbn:de:bsz:289-oparu-4348-7dc.identifier.urn
GNDDatenmanagementdc.subject.gnd
GNDCloud Computingdc.subject.gnd
FacultyFakultät für Ingenieurwissenschaften, Informatik und Psychologieuulm.affiliationGeneral
InstitutionInstitut für Organisation und Management von Informationssystemenuulm.affiliationSpecific
DCMI TypeTextuulm.typeDCMI
TypeErstveröffentlichunguulm.veroeffentlichung
CategoryPublikationenuulm.category
In cooperation withQueen’s University of Belfastuulm.cooperation
In cooperation withUmeå Universitetuulm.cooperation
In cooperation withDublin City Universityuulm.cooperation
EU projectCACTOS / Context-Aware Cloud Topology Optimisation and Simulation / EC / FP7 / 610711uulm.projectEU
FundingEC / FP7uulm.funding
University Bibliographyjauulm.unibibliographie


Files in this item

Thumbnail

This item appears in the following Collection(s)

Show simple item record

CC BY-ND 4.0 International
Except where otherwise noted, this item's license is described as CC BY-ND 4.0 International