Which datasets should torchaudio have?

[vincentqb]

Which new datasets should we offer and prioritize in torchaudio?

I want to follow-up on #31 and a few of the recent PRs. Instead of aiming to have an exhaustive list of datasets, we should focus on a few important/common/representative dataset that can serve as templates for users to easily implement datasets of their choosing. All datasets should already be free/accessible/online/common with license permitting linking to them.

torchaudio currently has:

1. commonvoice
2. librispeech
3. ljspeech
4. speechcommands
5. vctk
6. yesno

Current open proposals:

• Free Universal Sound Separation (FUSS) Add Free Universal Sound Separation (FUSS) Dataset #534
• CMU_ARCTIC Add CMU_ARCTIC dataset #512

[cyrta]

I would recommend to review list:

• https://github.com/JRMeyer/open-speech-corpora
• https://github.com/jim-schwoebel/voice_datasets
• https://wiki.inria.fr/rosp/Datasets
• https://www.ismir.net/resources/datasets/
• http://www.audiocontentanalysis.org/data-sets/

My suggestion are
for speech

• TED-LIUM 1-3 http://www.openslr.org/51/
• Spoken Wikipedia Corpus https://nats.gitlab.io/swc/
• CHiME 5 http://spandh.dcs.shef.ac.uk/chime_challenge/data.html
• AMI
• ICSI Meeting Corpus
• LibriTTS http://www.openslr.org/60/

from noise

• NOIZEUS
• MUSAN
• ESC-50 https://github.com/karolpiczak/ESC-50

from music

• AcousticBrainz-Genre
• GTZAN

All of them are considered onces you do benchmark evaluation for many tasks.

[cHemingway]

For Noise, some thoughts:

• NOIZEUS is already noisy speech, ideally a "pure" noise dataset should be used instead? However it is very popular, so possibly worth including for that reason.

• MUSAN looks good, has proper licensing, and is well cited, but I am unsure if it is sufficiently "varied", there is not much description of the contents of its noise in the paper, or how much care was made to make it representatives.

• ESC-50 seems to meet all the requirements quite nicely, and is directly accessible as a GitHub zip download. It is aimed at Environmental Sound Classification however, and not speech processing.

In the field I (briefly) studied of speech denoising and seperation, NOISEX 92 is very popular. Although it is quite an old noise database, it was created specifically for systems that need to deal with noisy speech, and steps to ensure proper statistical variation of the noise within it (PCA performed on a spectrograms) so that a test using this noise database could be considered somewhat representative of most potential "real world" noise. I am fairly sure the licensing is not suitable though.

I think DEMAND was created to be suitably diverse like NoiseX but with multiple channels instead of mono. Unlike other noise datasets, it is all fresh recordings and not a selection of Freesound samples, and aims to be as diverse as possible. The license is Creative Commons Attribution-ShareAlike 3.0 Unported which I think should be compatible? I am unsure if it is popular enough though, it has 50 citations on google scholar, compared to the 1458 for NOISEX 92, but if licensing prohibits NOISEX an alternative might be worth including even if it is not super popular.

[mmxgn]

Hello,

Regarding GTZAN, I just finished a Dataset class, linted and with the unit tests and all. I needed for personal use but I went the extra mile to bring it to format because why not. Is it ok to submit a PR for this? I understand that there might be issues regarding download traffic and GTZAN is already very popular. If its not, is it ok when called with download=True to instead give a warning that the dataset needs to be manually downloaded and fail quietly?

[vincentqb]

cc tedlium #765

[lcances]

Hello, For my Ph.D. I am using the following datasets. I am more and more following a PyTorch like implementation for my Dataset class and it would be better to contribute to torchaudio with my implementation of these datasets.

• UrbanSound8k
• ESC-10 / 50 / US
• SpeechCommand
• GTZAN

However, my UrbanSound8k and ESC implementation need some modification to be compliant with the PyTorch code of conduct. For SpeechCommand and GTZAN I will propose some modifications to make them easier to use.

I will be happy to contribute to the community.