Consistency between torchvision/torchaudio

[bchamand]

When we load a sound file with torchaudio, we get an output Tensor of size (L x C) (L the number of audio frames and C the number of channels).
It's not a better idea to get a Tensor of shape (C x L) ?

Because, with torchvision, when I load an image and I use the function ToTensor, the dimension of the output tensor is (C x H x W) with the channel in first dimension. Is not it more coherent if the output of the load function in torchaudio use a similar output shape to the ToTensor function ?

In this case, the functions LC2CL and BLC2CBL are no longer necessary.

[0x00b1]

I have mixed feelings about this.

On one hand, consistency is good and a possible future of many signal processing problems is decomposing into multi-channel two-dimensional images (e.g. spectrograms) and relying on computer vision infrastructure (e.g. standard CNNs like ResNet or VGG) for classification. On the other hand, (channels × frames) is—just like (channels × r × c) with vision—inconsistent with the entirety of the Python scientific (and signal processing) community.

[jph00]

It would be nice to have it internally consistent. Currently MEL() assumes CxL but loading gives LxC.

(FWIW I do agree with @bchamand that being inconsistent with pytorch is confusing and inefficient. All the pytorch conv functions expect channel first. Since this project is part of the pytorch gh org, it seems like it should follow the same specs as the pytorch project, which has already made a call on this issue.)

[jph00]

Ah also just discovered that the input and output dim order for MEL2 and MEL are inconsistent, as are the output formats (MEL2 scales to DB, but MEL doesn't).

I just did a %timeit check for a 16k frame audio file, and MEL2 was 654 µs, whereas MEL was 2.42 ms (this is on a V100, single precision). Since it's quite a bit faster, perhaps you can remove the "maybe it's faster" from the docstring, and replace MEL with MEL2? (Maybe even remove the librosa version?)

[jph00]

Sorry a correction - the above timing is for CPU, not GPU. I just submitted #77 to make it work on GPU. On GPU the timing is 949 µs.

[vincentqb]

Thank you for the feedback. We introduced changes around the standardization that should have addressed the initial topic: please see the release notes or #152. Please feel free to re-open if appropriate or create a new issue to discuss follow-ups.

[jph00]

Many thanks for the followup @vincentqb