Using it across multiple gpus

#4
by Ashvini08 - opened

Is it possible to do inferencing of this model across multiple GPUs. If yes, could you please share instructions for the same.
Thank you:)

I also am interested in this, I have a 3060 12gb and a 5070 12gb
When running i get:

python qwen_image.py --cpu_offload
/.../Qwen/qwen/lib/python3.13/site-packages/dfloat11/dfloat11.py:19: UserWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html. The pkg_resources package is slated for removal as early as 2025-11-30. Refrain from using this package or pin to Setuptools<81.
  import pkg_resources
The config attributes {'pooled_projection_dim': 768} were passed to QwenImageTransformer2DModel, but are not expected and will be ignored. Please verify your config.json configuration file.
Killed
Dynamic-length Float (DFloat11) org

Currently multi-GPU is not supported.

I also am interested in this, I have a 3060 12gb and a 5070 12gb
When running i get:

python qwen_image.py --cpu_offload
/.../Qwen/qwen/lib/python3.13/site-packages/dfloat11/dfloat11.py:19: UserWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html. The pkg_resources package is slated for removal as early as 2025-11-30. Refrain from using this package or pin to Setuptools<81.
  import pkg_resources
The config attributes {'pooled_projection_dim': 768} were passed to QwenImageTransformer2DModel, but are not expected and will be ignored. Please verify your config.json configuration file.
Killed

You have insufficient CPU memory, so the process is "killed" by the OS. Can you try with this:

python qwen_image.py --cpu_offload --no_pin_memory --cpu_offload_blocks 40

You may need to increase the number of cpu_offload_blocks if you run into out-of-GPU-memory issues.

Sign up or log in to comment