[BUG] CUDA failed with error CUDA driver version is insufficient for CUDA runtime version

[alex-vyverman]

Is there an existing issue for this?

• I have searched the existing issues

Current Behavior

When running the gpu tagged image, I get an error in the container that the CUDA driver version is insufficient for CUDA runtime version.
But unless I am mistaken, the image uses cudnn-cu12 and cublas-cu12, which should be compatible with NVIDIA-SMI 550.120 Driver Version: 550.120 CUDA Version: 12.4 that I run
Any ideas?

Expected Behavior

No response

Steps To Reproduce

1. run the image in the environment described below

Environment

- OS:Ubuntu 24.04.1 LTS
- How docker service was installed: docker repo
- GPU: Nvidia RTX A4000
- nvcc --version
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2023 NVIDIA Corporation
Built on Fri_Jan__6_16:45:21_PST_2023
Cuda compilation tools, release 12.0, V12.0.140
Build cuda_12.0.r12.0/compiler.32267302_0
- nvidia-smi: NVIDIA-SMI 550.120 Driver Version: 550.120 CUDA Version: 12.4

CPU architecture

x86-64

Docker creation

services:
  faster-whisper:
    image: lscr.io/linuxserver/faster-whisper:gpu
    container_name: faster-whisper
    environment:
      - PUID=1000
      - PGID=1000
      - TZ=Etc/UTC
      - WHISPER_MODEL=tiny-int8
      - WHISPER_BEAM=1 #optional
      - WHISPER_LANG=nl #optional
    volumes:
      - /mnt/local/container_data/whisper/data2:/config
    ports:
      - 10300:10300
    deploy:
      resources:
          reservations:
              generic_resources:
              - discrete_resource_spec:
                  kind: "NVIDIA-GPU"
                  value: 1

Container logs

Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/lsiopy/lib/python3.12/site-packages/wyoming_faster_whisper/__main__.py", line 149, in <module>
    run()
  File "/lsiopy/lib/python3.12/site-packages/wyoming_faster_whisper/__main__.py", line 144, in run
    asyncio.run(main())
  File "/usr/lib/python3.12/asyncio/runners.py", line 194, in run
    return runner.run(main)
           ^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/asyncio/runners.py", line 118, in run
    return self._loop.run_until_complete(task)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/asyncio/base_events.py", line 687, in run_until_complete
    return future.result()
           ^^^^^^^^^^^^^^^
  File "/lsiopy/lib/python3.12/site-packages/wyoming_faster_whisper/__main__.py", line 119, in main
    whisper_model = faster_whisper.WhisperModel(
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/lsiopy/lib/python3.12/site-packages/faster_whisper/transcribe.py", line 133, in __init__
    self.model = ctranslate2.models.Whisper(
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: CUDA failed with error CUDA driver version is insufficient for CUDA runtime version

[github-actions]

Thanks for opening your first issue here! Be sure to follow the relevant issue templates, or risk having this issue marked as invalid.

[davidcampos]

I have a similar issue. Any ideas?

[gartensofa]

same here.

[mreilaender]

Had the same error. In my case I started the docker container with a

services:
  faster-whisper:
    runtime: nvidia

but I was not exposing the nvidia gpu. Adding the following section fixed the issue for me:

services:
  faster-whisper:
    runtime: nvidia
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: 1
              capabilities: [gpu]

[wills106]

On Unraid on the Docker container page I toggled Basic View to Advance View, then next to 'Extra Parameters:' I added:
--gpus=all

[mobster1940]

This fixed it for me, thanks!

[Phantom-Glass]

I am getting the same issue with Unraid. I attempted the "--gpus=all" parameter, but still no luck.

[LinuxServer-CI]

This issue has been automatically marked as stale because it has not had recent activity. This might be due to missing feedback from OP. It will be closed if no further activity occurs. Thank you for your contributions.

[klepptor]

Are you sure @Phantom-Glass ? I just fixed the "error" with adding --gpus all on Unraid 7.0

[Phantom-Glass]

Are you sure @Phantom-Glass ? I just fixed the "error" with adding --gpus all on Unraid 7.0

After updating to 7.0, adding that into extra parameters did start working for me.

[jokerigno]

hey. Just tried with the extra parameter and the issue is gone.

But it seems not to work because I found this in the logs:

INFO:faster_whisper:Processing audio with duration 00:02.820
INFO:wyoming_faster_whisper.handler:!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

Any insight?