An End-to-end deep learning framework for acoustic emotion recognition

Abstract

This work examines acoustic emotion recognition by a system using only an audio file containing a person’s speech. This can be applied in several areas of human-computer-interaction (HCI). In this context, a system consisting of only a microphone and a processor is less expensive than other methods that use additional variants for emotion recognition. This thesis explores how automatic emotion recognition by a system using only speech as input works. This work provides insight into different technologies that can be applied for emotion recognition of a system. Feature extraction and computation of Mel spectrograms from audio files was investigated and their usage as input to a machine learning algorithm or a neural network. A framework was developed using a model, which is well suited to classify an emotion for a short or longer audio clip. The experiments of this thesis show the success that can be achieved using convolutional neural networks and Mel spectrograms as its input to classify an emotion based only on an audio file. This in combination with different training methods has results in a framework that can classify emotions from short but also longer audio files in a very short time. In summary, automatic classification of emotions from a system is a very interesting topic, where multiple methods can lead to different successes. With the utilization of the results of this work, a further step towards improved HCI can be taken, as systems can adapt their action to the emotion state of the user.