mistralai/Voxtral-Mini-3B-2507 · Could not load symbol cudnnGetLibConfig

Hi guys,

Thanks for the release of this package!

I just wanted to share with you the reason for the error mentioned in the title, looking something like this:

Could not load symbol cudnnGetLibConfig. Error: /usr/local/lib/python3.9/site-packages/nvidia/cudnn/lib/libcudnn_graph.so.9: undefined symbol: cudnnGetLibConfig
...
RuntimeError: Engine core initialization failed. See root cause above. Failed core proc(s): {}
/root/.pyenv/versions/3.12.8/lib/python3.12/multiprocessing/resource_tracker.py:254: UserWarning: resource_tracker: There appear to be 1 leaked semaphore objects to clean up at shutdown
  warnings.warn('resource_tracker: There appear to be %d '

The reason for the error was that I already had LD_LIBRARY_PATH variable pointed to other cudnn & cublas in other Python version than the one used to start vllm.

After I set the variable to vllm's python version, e.g.:

export LD_LIBRARY_PATH=`python3  -c 'import os; import nvidia.cublas.lib; import nvidia.cudnn.lib; print(os.path.dirname(nvidia.cublas.lib.__file__) + ":" + os.path.dirname(nvidia.cudnn.lib.__file__))'`

Starting the model with vllm worked great.

Hope it unlocks some of you encountering the problem!