run lowpass_biquad in gpu is much slower that run in cpu

[ChenPaulYu]

🐛 Bug

Hi everyone, I am doing some experiments via lowpass_biquad filter in torchaudio, but I was trying to use GPU to speed up by code. I found that running the lowpass_biquad filter in GPU is much slower than running in CPU (x 200 slower). I am not sure whether it is a bug, but I am curious about whether anyone has the same situation as me?

To Reproduce

import time
import torch
from torchaudio.functional import lowpass_biquad

gpu_device = torch.device('cuda:0')
cpu_device = torch.device('cpu')

sample_rate = 44100
cutoff_freq = 1000.
Q           = .7

# Run in cpu
x     = torch.rand(sample_rate * 10, device=cpu_device)
begin = time.time()
y     = lowpass_biquad(x, sample_rate, cutoff_freq, Q)

print(f'Run in cpu: {time.time() - begin}')

# Run in gpu
x     = torch.rand(sample_rate * 10, device=gpu_device)
begin = time.time()
y     = lowpass_biquad(x, sample_rate, cutoff_freq, Q)

print(f'Run in gpu: {time.time() - begin}')

Run in cpu: 0.01553034782409668
Run in gpu: 21.124146461486816

Expected behavior

Run in GPU should much faster that run in CPU

Environment

• What commands did you used to install torchaudio (conda/pip/build from source)? pip

• If you are building from source, which commit is it? no

• What does torchaudio.__version__ print? (If applicable) 0.8.0

• PyTorch Version (e.g., 1.0): 1.8.0

• OS (e.g., Linux): Linux Ubuntu 18.04.3 LTS (Bionic Beaver)

• How you installed PyTorch (conda, pip, source): pip

• Build command you used (if compiling from source): no

• Python version: 3.6.12

• CUDA/cuDNN version: 10.0.130

Versions of relevant libraries:

[pip3] python==3.6.12
[pip3] conda==4.9.2
[pip3] torch==1.8.0
[pip3] torchaudio==0.8.0

[yoyolicoris]

In torchaudio, a highly optimized implementation of lfilter is used when input tensor is on cpu.

The lowpass_biquad function use lfilter internally, so I think it's possible to have performance degradation when running on gpu, depends on what type of gpu you are using.

Also, cuda needs some warm up time, try a second run on gpu and you should get a better and more accurate result.

[cpuhrsch]

I'd expect this behavior, because we don't have an optimized CUDA kernel for lfilter. I don't expect it'll be easy to write one for this particular kind of computation, but I haven't done a lot of research into it.

[ChenPaulYu]

Hi @yoyololicon and @cpuhrsch ,

Thanks you for your reply, after doing some investigation. I realize that the run small for-loop via gpu is expensive, and may lead the performance degradation, which is happened in the current lfilter implementation.

rapidsai/cusignal#171
https://gist.github.com/coderforlife/17bf471a519a7d092e82f95f10e8f6e4

I found useful resource in the cusignal discussion. They trying to make a efficient lfilter via cupy. I try it, it work and really fast. Even faster than the CPU implementation in torchaudio. However, I can not read the Raw CUDA Kernel.

Do you guys have any idea how to integrated it to the torchaudio ?

[yoyolicoris]

Hi @yoyololicon and @cpuhrsch ,

Thanks you for your reply, after doing some investigation. I realize that the run small for-loop via gpu is expensive, and may lead the performance degradation, which is happened in the current lfilter implementation.

rapidsai/cusignal#171
https://gist.github.com/coderforlife/17bf471a519a7d092e82f95f10e8f6e4

I found useful resource in the cusignal discussion. They trying to make a efficient lfilter via cupy. I try it, it work and really fast. Even faster than the CPU implementation in torchaudio. However, I can not read the Raw CUDA Kernel.

Do you guys have any idea how to integrated it to the torchaudio ?

@ChenPaulYu
Have just took a brief look about the source code, seems like they wrote their own CUDA kernel.
If their license is compatible with us, maybe we can just borrow it and compiled as cpp extensions described in https://pytorch.org/tutorials/advanced/cpp_extension.html.

[mthrok]

Note: @carolineechen is adding a custom CUDA code and making changes to build/release processes. Once that part is done, adding custom CUDA kernel should be simple. Since we are not CUDA experts or do not have a decent experience there, it would be faster if someone with experience can contribute. If not I am curious to learn CUDA, so I will try to port it, but it will take some time to meet the performance criteria.

[yoyolicoris]

Since #1638 has merged, I think it's time to improve the running time on CUDA.

@mthrok I can also help look into it and try some CUDA code if you don't have time.

[mthrok]

@mthrok I can also help look into it and try some CUDA code if you don't have time.

@yoyololicon That will be great! Thanks for the offer.

[yoyolicoris]

@mthrok
I finally got some time to look into this.
With a custom cuda kernel, the speedup of IIR against torchaudio's version is huge. I will make a PR later in January.

[mthrok]

Hi @yoyololicon

Thanks for looking into this, it will be indeed great if we can make it more efficient.
Looking forward to your PR.
Let me know if you have any question.

cc @nateanl