Using it across multiple gpus
#4
by
Ashvini08
- opened
Is it possible to do inferencing of this model across multiple GPUs. If yes, could you please share instructions for the same.
Thank you:)
I also am interested in this, I have a 3060 12gb and a 5070 12gb
When running i get:
python qwen_image.py --cpu_offload
/.../Qwen/qwen/lib/python3.13/site-packages/dfloat11/dfloat11.py:19: UserWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html. The pkg_resources package is slated for removal as early as 2025-11-30. Refrain from using this package or pin to Setuptools<81.
import pkg_resources
The config attributes {'pooled_projection_dim': 768} were passed to QwenImageTransformer2DModel, but are not expected and will be ignored. Please verify your config.json configuration file.
Killed
Currently multi-GPU is not supported.
I also am interested in this, I have a 3060 12gb and a 5070 12gb
When running i get:python qwen_image.py --cpu_offload /.../Qwen/qwen/lib/python3.13/site-packages/dfloat11/dfloat11.py:19: UserWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html. The pkg_resources package is slated for removal as early as 2025-11-30. Refrain from using this package or pin to Setuptools<81. import pkg_resources The config attributes {'pooled_projection_dim': 768} were passed to QwenImageTransformer2DModel, but are not expected and will be ignored. Please verify your config.json configuration file. Killed
You have insufficient CPU memory, so the process is "killed" by the OS. Can you try with this:
python qwen_image.py --cpu_offload --no_pin_memory --cpu_offload_blocks 40
You may need to increase the number of cpu_offload_blocks
if you run into out-of-GPU-memory issues.