KuralHub lists 3 speech emotion recognition datasets for English, 2 of them openly available. Each entry below gives the emotion labels, size, recording setup, licence, where to get it and how to cite it. English is also part of the benchmark. The best of 11 models reached 82.0% test accuracy (hubert-base-ls960).
Language family
Indo-European (Germanic)
ISO code
en
Spoken mainly in
United States, United Kingdom, Canada, Australia, New Zealand
English is one of the 29 languages in the KuralHub benchmark. Each of the 11 frozen speech models was trained with a small classification head on the shared emotions and scored on a held-out test split. Over all 22 runs the mean test accuracy was 51.0% (median 57.0%). The best run reached 82.0% with hubert-base-ls960. Compare with other languages.
IESC (run ID IESC). Scores on the test split, best run highlighted.
IEMOCAP is a multimodal, multispeaker dataset containing ~12 hours of audiovisual data, including speech, video, facial motion capture, and text transcriptions. It features dyadic sessions with scripted and improvised interactions designed to elicit emotions. The dataset is annotated with categorical (anger, happiness, sadness, neutrality) and dimensional (valence, activation, dominance) labels, making it valuable for studying expressive human communication.
Citation
@article{busso2008iemocap,
title={IEMOCAP: Interactive emotional dyadic motion capture database},
author={Busso, Carlos and Bulut, Murtaza and Lee, Chi-Chun and Kazemzadeh, Abe and Mower, Emily and Kim, Samuel and Chang, Jeannette N and Lee, Sungbok and Narayanan, Shrikanth S},
journal={Language resources and evaluation},
volume={42},
pages={335--359},
year={2008},
publisher={Springer}
}
The Indian Emotional Speech Corpora (IESC) is a dataset containing 600 emotional speech recordings from 8 North Indian speakers (5 males, 3 females). The audio files, recorded in a noise-free environment using a mobile phone and headset, cover five emotions: neutral, happy, angry, sad, and fearful. Each .wav file follows a structured naming convention indicating emotion, repetition, speaker, and sentence ID.
Used in the KuralHub benchmark (run ID IESC). See the results above.
Citation
@article{singh2023lightweight,
title={A lightweight 2D CNN based approach for speaker-independent emotion recognition from speech with new Indian Emotional Speech Corpora},
author={Singh, Youddha Beer and Goel, Shivani},
journal={Multimedia Tools and Applications},
volume={82},
number={15},
pages={23055--23073},
year={2023},
publisher={Springer}
}
The RAVDESS (Ryerson Audio-Visual Database of Emotional Speech and Song) is a dataset containing 1440 speech-only audio files (16-bit, 48kHz WAV) recorded by 24 professional actors (12 female, 12 male). The dataset includes seven emotions—calm, happy, sad, angry, fearful, surprise, and disgust—each performed at two intensity levels (normal and strong), along with a neutral expression. The recordings feature two lexically-matched statements in a neutral North American accent. The full dataset, including speech and song, is available on Zenodo (24.8 GB).
Used in the KuralHub benchmark (run ID RAVDESS). See the results above.
Citation
@article{livingstone2018ryerson,
title={The Ryerson Audio-Visual Database of Emotional Speech and Song (RAVDESS): A dynamic, multimodal set of facial and vocal expressions in North American English},
author={Livingstone, Steven R and Russo, Frank A},
journal={PloS one},
volume={13},
number={5},
pages={e0196391},
year={2018},
publisher={Public Library of Science San Francisco, CA USA}
}
Know another English speech emotion dataset, or a newer version of one listed here? Suggest it or open a pull request.
This page is generated from datasets/English/ in the KuralHub repository.
If you use a dataset, cite its authors. If KuralHub helped you find it, please also cite the KuralHub paper.