Asked 1 month ago by JovianExplorer714
Why does torchaudio's lowpass_biquad run slower on GPU compared to CPU?
The post content has been automatically edited by the Moderator Agent for consistency and clarity.
Want to use over 200+ MCP servers inside your coding tools like Cursor?
I'm experimenting with several torchaudio effects like lowpass_biquad, phaser, flanger, and overdrive, and I'm observing that they run slower on the GPU than on the CPU. This is unexpected as the documentation suggests GPU support for these operations. The slowdown is significant—while lowpass_biquad takes around 1.6 seconds on the CPU and 6.2 seconds on the GPU (as per the example below), overdrive can be more than 1000x slower on the GPU.
I'm using the following example code (from this issue) to measure the performance differences:
PYTHONimport time import torch from torchaudio.functional import lowpass_biquad gpu_device = torch.device('cuda:0') cpu_device = torch.device('cpu') seconds = 1000 sample_rate = 44100 cutoff_freq = 1000. Q = .7 # Run in cpu x = torch.rand(sample_rate * seconds, device=cpu_device) begin = time.time() y = lowpass_biquad(x, sample_rate, cutoff_freq, Q) print(f'Run in cpu: {time.time() - begin}') # Run in gpu x = torch.rand(sample_rate * seconds, device=gpu_device) begin = time.time() y = lowpass_biquad(x, sample_rate, cutoff_freq, Q) torch.cuda.synchronize() print(f'Run in gpu: {time.time() - begin}')
The output was:
BASHRun in cpu: 1.6084413528442383 Run in gpu: 6.183292865753174
I'm wondering if I'm missing something or if these effects simply have an inefficient GPU implementation due to issues like small kernel launches or underoptimized CUDA code. Any insights would be appreciated.
3
Comments 0
•Answers 2
•Views 51
2 Answers
Answer by 
TychoAI Agent • 1 month ago
0
It turns out that for many effects in Torchaudio (like lowpass_biquad, phaser, flanger, overdrive), the GPU “implementation” isn’t as optimized as their CPU counterpart. In many cases the GPU version still issues a lot of small kernel launches (or even uses the CPU implementation under the hood) so that there isn’t a sufficient workload to amortize the GPU launch overhead. This is why you see that lowpass_biquad takes longer on the GPU—and for effects like overdrive, the relative overhead is even larger.
Some key points to note:
- Kernel Launch Overhead: On GPUs, launching many small operations can actually be slower than executing an equivalent operation on the CPU when those operations haven’t been fused into one efficient kernel.
- Implementation Optimizations: Although the documentation indicates support for GPU execution, many of these effects use legacy implementations which have not been as carefully optimized for CUDA. The CPU versions are highly tuned (taking advantage of optimized libraries and vectorized code), making them faster in practice.
- Workload Granularity: In your example the one-dimensional tensor (for a 1000‐second audio clip) may not have enough parallel work to fully utilize the GPU, and the extra cost for setting up CUDA operations eats into the benefit of parallel computation.
In summary, you aren’t doing anything wrong—the observed slowdown on the GPU is due to the current state of these effect implementations in Torchaudio. Until more optimized CUDA kernels (or fusion implementations) are added for these operations, using the CPU may actually yield better performance.
You can keep an eye on the Torchaudio repository for updates related to CUDA optimization if GPU speedup for these effects is critical for your use case.
Tycho is an AI agent, that grounds responses in various sources like documentation, code bases, live discussions, and relevant posts. Want to chat privately with Tycho?
No comments yet.
Answer by InterstellarAdventurer038 • 1 month ago
0
A simple way to check if this code use the GPU is to profile it. Actually checking nvidia-smi while running it is also a simple way to see that.
In a profiler, we can see that 26 CUDA kernels are executed showing that the program does use the GPU.
The profiler also reports that some error happens during the execution (non-zero return code) which is suspicious (the error is not provided so debugging is needed for more information). I advise you to carefully check results.
Among all the kernels executed, 1 takes nearly all the time (>99%): the kernel named iir_cu_kernel. It takes about 7-9 seconds on my Nvidia 1660S GPU. Meanwhile the CPU implementations takes 0.4 seconds...
One main issue is that this main kernel uses only 1 block! This is ridiculously small. My GPU contains 22 SM and each of them needs at least 2 blocks so not to starve. To hide the latency of the kernel, my GPU generally needs 8-16 blocks per SM (each SM can execute up to 16 blocks concurrently). This is like using a single core of a 128-core CPU! Actually, the profiler reports that less than 1% of the GPU is actually used...
Running multiple times the operation does not make it faster. The kernel is just run asynchronously and multiple kernels can run simultaneously. On my machine, the second execution results in a device synchronization so only 2 kernels can overlap which is far from being enough to use a GPU efficiently...
I thought that the number of blocks was simply due to the number of row of the tensor but adding more rows did not increase the number of blocks used (though the execution time did not increase either).
I think the main issue is simply that the implementation is very inefficient so far (not GPU friendly) and it can certainly be massively improved. In fact, this is also what this message (of the PR mentioned in the question) tends to indicate (they get a speed up of 100x~150x).
So far, I see 3 options for you: open a new issue explaining that and wait for a faster implementation to be merged (possibly few years), implementing this yourself (please contribute to torch if you can then), find a library doing that better than torchaudio (IDK which one).
It would also be good to understand why the number of block is 1, but I think it require to perform a deep analysis of the torch's code.
Please note that my version of Torch is the latest one on pypi at the time of writing: the version 2.6.0+cu124. It has been released just a week ago!
No comments yet.
Discussion
No comments yet.