KuralHub lists 8 speech emotion recognition datasets for Chinese, 5 of them openly available. Each entry below gives the emotion labels, size, recording setup, licence, where to get it and how to cite it. Chinese is also part of the benchmark. The best of 11 models reached 98.0% test accuracy (whisper-large).
Chinese is one of the 29 languages in the KuralHub benchmark. Each of the 11 frozen speech models was trained with a small classification head on the shared emotions and scored on a held-out test split. Over all 11 runs the mean test accuracy was 75.5% (median 90.0%). The best run reached 98.0% with whisper-large. Compare with other languages.
ESD (run ID ESD). Scores on the test split, best run highlighted.
The CASIA Chinese Emotion Corpus was developed by the Institute of Automation, Chinese Academy of Sciences. It contains emotional speech recordings from professional speakers expressing six different emotions in Mandarin Chinese. The recordings were conducted in a noise-free environment with approximately 35 dB SNR.
Note: Dataset is publicly available through Kaggle.
Citation
@inproceedings{zhang2008design,
title={Design of speech corpus for mandarin text to speech},
author={Zhang, JTFLM and Jia, Huibin},
booktitle={The blizzard challenge 2008 workshop},
year={2008}
}
The CHEAVD dataset (Chinese Natural Emotional Audio-Visual Database) is a comprehensive collection of emotional expressions in Mandarin Chinese. It contains 2322 emotional segments extracted from Chinese movies, TV shows, and interviews, representing natural emotional expressions.
The dataset is annotated with 26 distinct emotion categories: neutral, angry, happy, sad, worried, anxious, disgust, surprise, blamed, sarcastic, aggrieved, curious, fearful, embarrassing, nervous, confused, proud, helpless, hesitant, contemptuous, frustrated, serious, anticipated, shy and guilty. Each segment is labeled with a primary emotion and potentially a secondary emotion label.
CHEAVD uniquely captures the complexity of human emotions by including:
Non-prototypical emotions (such as shy, nervous, proud, frustrated)
Multi-emotion labels where emotions co-occur (e.g., anger accompanied by disgust)
Fake/suppressed emotions where internal and external expressions differ
The annotation was performed by four annotators with moderate consistency (kappa coefficient around 0.5), with neutral being the most represented emotion, reflecting real-world emotion distribution. The dataset includes demographic information about speakers, including gender and age range.
Citation
@article{li2017cheavd,
title={CHEAVD: a Chinese natural emotional audio--visual database},
author={Li, Ya and Tao, Jianhua and Chao, Linlin and Bao, Wei and Liu, Yazhu},
journal={Journal of Ambient Intelligence and Humanized Computing},
volume={8},
pages={913--924},
year={2017},
publisher={Springer}
}
CHEAVD 2.0 is an extended version of the original CHEAVD dataset released in MEC 2016, adding 4178 new samples. The dataset consists of emotional segments selected from Chinese movies, soap operas, and TV shows, containing background noise to simulate real-world conditions. It comprises 474 minutes of spontaneous emotional content from 527 speakers of diverse ages (from children to elderly), with a gender distribution of 58.4% male and 41.6% female subjects.
The emotional segments range from 1 to 19 seconds in duration, with an average length of 3.3 seconds. Four experienced taggers labeled each sample using a discrete emotion annotation strategy, with the final label determined by majority vote. The dataset contains 7030 samples across eight emotion categories: happiness, sadness, worry, anger, anxiety, surprise, disgust, and neutral.
For machine learning purposes, the dataset is divided into training (4917 samples), validation (707 samples), and testing (1406 samples) sets, making it suitable for developing and evaluating emotion recognition models in Mandarin Chinese speech.
Citation
@inproceedings{li2018mec,
title={Mec 2017: Multimodal emotion recognition challenge},
author={Li, Ya and Tao, Jianhua and Schuller, Bj{\"o}rn and Shan, Shiguang and Jiang, Dongmei and Jia, Jia},
booktitle={2018 First Asian Conference on Affective Computing and Intelligent Interaction (ACII Asia)},
pages={1--5},
year={2018},
organization={IEEE}
}
The Chinese Elderly Emotion Database (EESDB) is a specialized speech corpus focusing on emotional expressions of elderly Chinese speakers. The database consists of speech samples collected from Chinese television broadcasts, featuring 16 individuals (8 male and 8 female) all over 60 years of age. Each utterance is categorized into one of seven emotional states: anger, disgust, fear, happiness, neutral, sadness, and surprise. To ensure quality and reliability, the database underwent a rigorous perception test conducted by independent evaluators who were not involved in the voice collection or annotation process. Only utterances that achieved over 60% agreement on emotional labels were included in the final database, ensuring high consistency in emotional expression representation.
Citation
@inproceedings{wang2014database,
title={A database of elderly emotional speech},
author={Wang, KX and Zhang, QL and Liao, SY},
booktitle={Proc. Int. Symp. Signal Process. Biomed. Eng Informat},
pages={549--553},
year={2014}
}
EMOVIE is a Mandarin Chinese emotional speech dataset containing 9,724 utterances extracted from seven Mandarin movies. The dataset provides both audio and text transcriptions with emotion polarity annotations on a 5-point scale. The audio is extracted from the front center channel of 5.1/7.1 movie audio using ffmpeg, with a sampling rate of 22.05 kHz. The average utterance length is 1.78 seconds with an average of 8.93 characters per utterance.
Citation
@article{cui2021emovie,
title={Emovie: A mandarin emotion speech dataset with a simple emotional text-to-speech model},
author={Cui, Chenye and Ren, Yi and Liu, Jinglin and Chen, Feiyang and Huang, Rongjie and Lei, Ming and Zhao, Zhou},
journal={arXiv preprint arXiv:2106.09317},
year={2021}
}
The Emotional Speech Database (ESD) is a comprehensive resource developed to address the growing research needs in emotional voice conversion. The Mandarin portion of ESD features 350 parallel utterances spoken by 10 native Mandarin speakers expressing 5 distinct emotion categories: neutral, happiness, anger, sadness, and surprise. The recordings were conducted in a controlled acoustic environment to ensure high-quality audio samples.
Part of a larger dataset that also includes English speakers, ESD was specifically designed to facilitate multi-speaker and cross-lingual emotional voice conversion studies. With over 29 hours of speech data in total, this dataset provides researchers with robust material to develop and test emotional voice conversion systems across different languages and emotional expressions.
The parallel nature of the utterances (same content expressed with different emotions) makes it particularly valuable for studying the acoustic characteristics that distinguish various emotional states in speech.
Used in the KuralHub benchmark (run ID ESD). See the results above.
Citation
@inproceedings{zhou2021seen,
title={Seen and unseen emotional style transfer for voice conversion with a new emotional speech dataset},
author={Zhou, Kun and Sisman, Berrak and Liu, Rui and Li, Haizhou},
booktitle={ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)},
pages={920--924},
year={2021},
organization={IEEE}
}
@article{zhou2021emotional,
title = {Emotional voice conversion: Theory, databases and ESD},
journal = {Speech Communication},
volume = {137},
pages = {1-18},
year = {2022},
issn = {0167-6393}
}
M3ED is a comprehensive multimodal emotional dialogue dataset created by Renmin University of China in 2022. The dataset contains dialogues extracted from 56 different TV series, featuring 990 dialogues with 24,449 utterances across 9,082 conversation turns. It provides text and audio features with rich emotional annotations including 7 primary emotions and captures both inter-turn and intra-turn emotional patterns.
Citation
@article{zhao2022rucm3ed,
title={RUCM3ED: A Multimodal Multi-Scene Multi-Label Emotional Dialogue Dataset},
author={Zhao, W. and Zhang, Z. et al.},
journal={},
year={2022}
}
The SECEPDB dataset addresses a significant limitation in emotion interaction studies by providing a specialized database focused on elderly individuals. Collected from 16 elderly actors (8 female and 8 male) performing in daily conversations from TV series, this multimodal database contains both audio and video recordings.
The database comprises 810 speech-video excerpts extracted from 118 conversations, covering seven distinct emotion categories: anger, boredom, happiness, sadness, surprise, neutrality, and disgust. To ensure quality and reliability, emotion annotations were evaluated by a panel of 18 participants (10 men and 8 women).
A unique feature of SECEPDB is its documentation of emotional transitions during conversations, making it particularly valuable for researchers studying the dynamics of emotional interactions in elderly populations. This resource contributes significantly to the development of more natural and responsive human-computer interaction systems tailored for older adults.
Citation
@inproceedings{wang2016database,
title={A database for emotional interactions of the elderly},
author={Wang, Kunxia and Zhu, ZongBao and Wang, Shidong and Sun, Xiao and Li, Lian},
booktitle={2016 IEEE/ACIS 15th International Conference on Computer and Information Science (ICIS)},
pages={1--6},
year={2016},
organization={IEEE}
}
Know another Chinese speech emotion dataset, or a newer version of one listed here? Suggest it or open a pull request.
This page is generated from datasets/Chinese/ in the KuralHub repository.
If you use a dataset, cite its authors. If KuralHub helped you find it, please also cite the KuralHub paper.