VoxSRC challenge

The VoxCeleb Speaker Recognition Challenge (VoxSRC) was a series of challenges and workshops that ran annually from 2019 to 2023. The challenges primarily evaluated the tasks of speaker recognition and diarisation under various settings including: closed and open training data; supervised and self-supervised training, and semi-supervised training for domain adaptation. The challenges also provided publicly available training and evaluation datasets for each task and setting, with new test sets released each year.

This website provides an overview of the VoxSRC challenges, links to the previous challenges and ground truth of all test pairs from 2019 to 2023.

The series of VoxSRC challenges has now officially finished. We are grateful to the hundreds of people who have taken part in the challenges over the years.

Publication

If you want to look at a retrospective review of the VoxCeleb challenge series over the course of 5 years, please refer to our paper below.

The VoxCeleb Speaker Recognition Challenge: A Retrospective

IEEE/ACM Transactions on Audio, Speech, and Language Processing, 2024.

[arXiv] [IEEE Xplore]

@article{huh2024vox,
  title={The VoxCeleb Speaker Recognition Challenge: A Retrospective},
  author={Huh, Jaesung and Chung, Joon Son and Nagrani, Arsha and Brown, Andrew and Jung, Jee-weon and Garcia-Romero, Daniel and Zisserman, Andrew},
  journal={IEEE/ACM Transactions on Audio, Speech, and Language Processing},
  year={2024},
  publisher={IEEE}
}

Overview

An overview of the tasks, new features, and statistics across VoxSRC workshops.
Year	Tracks	New developments	Statistics of test set
2019	Speaker verification (Closed) Speaker verification (Open)	VoxCeleb2 dev is used for train set and VoxCeleb1 test is used for validation set.	208,008 pairs from 19,154 utterances.
2020	Speaker verification (Closed) Speaker verification (Open) Self-supervised speaker verification Speaker Diarisation	We introduce out-of-domain data for some of the identities in VoxCeleb. Two new tracks are introduced in this year, self-supervised speaker verification and speaker diarisation.	Verification : 1,695,248 pairs from 118,439 utterances. Diarisation : 232 audio files.
2021	Speaker verification (Closed) Speaker verification (Open) Self-supervised speaker verification Speaker diarisation	This year the verification tracks have a special focus on multi-lingual verification. For diarisation track, we start to use a customised test set and use the entire VoxConverse for validation set.	Verification : 476,224 pairs from 116,984 utterances. Diarisation : 264 audio files.
2022	Speaker verification (Closed) Speaker verification (Open) Semi-supervised domain adaptation Speaker diarisation	We include hard positive pairs using an age classifier and hard negative pairs from same background noise. Semi-supervised domain adaptation is newly introduced.	Supervised verification : 317,973 pairs from 34,684 utterances. Semi-supervised domain adaptation : 30,000 pairs from 18,377 utterances. Diarisation : 360 audio files.
2023	Speaker Verification (Closed) Speaker verification (Open) Semi-supervised domain adaptation Speaker diarisation	Mega test set is introduced including the subsets of VoxSRC 2019-2022 test sets.	Supervised verification : 825,437 pairs from 256,547 utterances. Semi-supervised domain adaptation : 80,000 pairs from 21,997 utterances. Diarisation : 413 audio files.

Previous Challenges

Details of the previous challenges can be found below. You can also find the slides and presentation videos of the winners on the workshop websites.

Challenge	Links
VoxSRC-19	challenge / workshop / report
VoxSRC-20	challenge / workshop / report
VoxSRC-21	challenge / workshop / report
VoxSRC-22	challenge / workshop / report
VoxSRC-23	challenge / workshop / report

VoxCeleb	CNCeleb	VoxMovies	VoxConverse

Test set labels in previous challenges

The test set labels of previous challenges are available through the following links. Please visit the challenge websites to download test data. We found out some erroneous labels in the test sets. For those who want to analyse the performance with the fixed labels, use Test labels(fixed)

File	Test labels	Test labels(fixed)
VoxSRC 2019 track 1 and 2 test trial pairs & labels	Download	Download
VoxSRC 2020 track 1, 2 and 3 test trial pairs & labels	Download	Download
VoxSRC 2020 track 4 rttm files	We used VoxConverse test set. Visit here.
VoxSRC 2021 track 1, 2 and 3 test trial pairs & labels	Download	Download
VoxSRC 2021 track 4 rttm files	Download
VoxSRC 2022 track 1 and 2 test trial pairs & labels	Download	Download
VoxSRC 2022 track 3 test trial pairs & labels	Download
VoxSRC 2022 track 4 rttm files	Download
VoxSRC 2023 track 1 and 2 test trial pairs & labels	Download	Download
VoxSRC 2023 track 3 test trial pairs & labels	Download
VoxSRC 2023 track 4 rttm files	Download

Organisers

Arsha Nagrani, Google Research
Joon Son Chung, KAIST
Andrew Brown, Facebook AI Research
Jaesung Huh, VGG, University of Oxford
Jee-weon Jung, Carnegie Mellon University
Daniel Garcia-Romero AWS AI
Andrew Zisserman, VGG, University of Oxford

Advisors

Mitchell McLaren, Speech Technology and Research Laboratory, SRI International, CA,
Douglas A Reynolds, Lincoln Laboratory, MIT.

Please contact jaesung[at]robots[dot]ox[dot]ac[dot]uk if you have any queries.

The VoxCeleb Speaker Recognition Challenge