Added all files including vyro_workflows

Imagine/Readme.md

@@ -0,0 +1,147 @@
+---
+language:
+- en
+library_name: diffusers
+---
+# Imagine V5 Model Card
+
+![row01](v5.jpg)
+
+## Model Details
+
+### Model Description
+The Imagine V5, developed by Vyro AI, represents the pinnacle of photorealism in AI art generation. Specializing in photographs and portraits, V5 is known for its exceptional ability to create images that closely mimic reality.
+
+V5 boasts an impressive ability to recognize a wide array of prompts and handle multiple subjects effortlessly. It's important to note that V5, with its vast capabilities, also demands significant computational resources and may exhibit slower processing times. This model is best suited for users with a good understanding of prompt composition, as it offers high-quality outputs for those who can navigate its complexities.
+
+- **Developed by:** Vyro AI
+- **Model type:** Generative text-to-image model
+  
+## Key Features
+- Photorealistic Portraits and Landscapes: Specializes in creating highly realistic images.
+- Large Dataset Training: Inherits a broad understanding of prompts from Stable Diffusion XL Model.
+- Color Characteristics: Tends to produce slightly saturated imagery.
+- Resource Intensive: Requires significant computational power.
+## Ideal Uses
+- Digital Art Creation: Ideal for artists seeking to create photorealistic portraits and landscapes.
+- Graphic Design: Useful for designers who require high-fidelity images.
+- Creative Experimentation: A valuable tool for exploring new artistic concepts, especially in realistic styles.
+- Professional Projects: Suitable for advanced users in fields like advertising, where photorealism is key.
+  
+For more ways to use V5 and to explore its full potential, visit [ImagineV5 Use Cases](https://www.imagine.art/blogs/10-awesome-ways-to-use-imagine-s-new-v5-model)
+## Limitations
+- Factual Representations: Not intended for creating accurate depictions of real-world events or people.
+- Sensitive Content: Must not be used for generating offensive or explicit material.
+- Identity Misrepresentation and Deepfakes: Prohibited from creating deceptive images of real individuals.
+- Legal and Ethical Compliance: Users must adhere to copyright, privacy, and ethical standards.
+
+## Get Started with Using V5
+Ready to dive into the world of AI-generated art with Imagine V5? 
+Begin your journey into photorealistic art creation today. Visit [imagine.art](https://www.imagine.art/) to access a user-friendly platform designed to help you harness the full capabilities of V5. Whether you're an experienced artist or just starting out, you'll find the tools and guidance you need to transform your artistic visions into stunning digital realities.
+
+Explore detailed tutorials, creative tips, and a supportive community that will guide you through the exciting process of AI art generation. Start crafting your unique art pieces with Imagine V5 now at [imagine.art](https://www.imagine.art/)
+
+# Setup and Usage
+
+We offer two workflows:
+- One for **Imagine V5**
+- One for **Imagine V5 Ultra**
+
+---
+
+### Initial Setup
+
+Clone this repository 
+```bash
+git clone https://huggingface.co/vyroAI/ImagineV5
+```
+Create a conda environment
+```bash
+conda create -n imagineservices python==3.11 -y
+```
+Install the requirements
+```bash
+pip install -r requirements.txt
+```
+
+---
+
+## Imagine V5 Setup
+### Step 1: Navigate into the imagine-v5 folder
+```bash
+cd imagine-v5
+```
+
+### Step 2: Install PyTorch Nightly
+```bash
+pip install --pre torch torchvision torchaudio --index-url https://download.pytorch.org/whl/nightly/cu128 
+```
+
+### Step 3: Clone ComfyUI
+```bash
+git clone https://github.com/comfyanonymous/ComfyUI.git
+```
+
+### Step 4: Replace & Configure
+- Replace the ComfyUI/comfy and ComfyUI/models folders with the ones provided in this repo in imagine-v5 folder.
+- Copy the vyro_workflows folder into ComfyUI/custom_nodes/.
+
+### Step 5: Run the Application
+```bash
+cd ComfyUI
+python3.11 main.py
+```
+
+### Step 6: Load the Workflow
+- Load the workflow in the ComfyUI interface.
+- Use the following workflow file:
+```text
+https://huggingface.co/vyroAI/ImagineV5/blob/main/Imaginev5-Workflow.json
+```
+
+---
+
+### Imagine V5 Ultra
+
+## Imagine V5 Ultra Setup
+### Step 1: Navigate into the imagine-v5-ultra folder
+```bash
+cd imagine-v5-ultra
+```
+### Step 2: Install PyTorch Nightly
+```bash
+pip install --pre torch torchvision torchaudio --index-url https://download.pytorch.org/whl/nightly/cu128 
+```
+
+### Step 3: Clone ComfyUI
+```bash
+git clone https://github.com/comfyanonymous/ComfyUI.git
+```
+
+### Step 4: Replace & Configure
+- Replace the ComfyUI/comfy and ComfyUI/models folders with the ones provided in this repo in imgaine-v5-ultra folder.
+- Copy the vyro_workflows folder into ComfyUI/custom_nodes/.
+
+### Step 5: Run the Application
+```bash
+cd ComfyUI
+python3.11 main.py
+```
+
+### Step 6: Load the Workflow
+- Load the workflow in the ComfyUI interface.
+- Use the following workflow file provided in Workflows folder:
+
+```text
+https://huggingface.co/vyroAI/ImagineV5/blob/main/Imaginev5-ultra-Workflow.json
+```
+⚠️ If you are facing any issues, make sure to use ComfyUI version: "0.3.27"
+
+---
+## Privacy Policy
+For detailed information on data handling and privacy, refer to the [Imagine V5 Privacy Policy](https://drive.google.com/file/d/1odKfNRoJmwD3sg8dl4zGXjC65zzf8Ejm/view) document.
+
+## Conclusion
+Imagine V5 stands as a significant advancement in the realm of AI art generation, especially in the domain of photorealism. It presents a unique opportunity for artists, designers, and creatives to push the boundaries of digital art. While V5 demands a certain level of proficiency and computational resources, the quality of its output makes it a worthy tool for those seeking to explore the forefront of AI-generated art.
+
+Explore the capabilities of Imagine V5 on various platforms including web browsers, Android, and iOS devices. Join the Imagine AI Art community, participate in the Affiliate Program, or delve into technical integrations via the APIs page. Embrace the fusion of art and technology with Imagine AI Art.

Imagine/Workflows/Imaginev5-Workflow.json

@@ -0,0 +1,2307 @@
+{
+  "id": "a353eeec-1dc4-41d6-b5d3-846916cc12bb",
+  "revision": 0,
+  "last_node_id": 394,
+  "last_link_id": 1209,
+  "nodes": [
+    {
+      "id": 71,
+      "type": "Reroute",
+      "pos": [
+        -706,
+        -1007
+      ],
+      "size": [
+        75,
+        26
+      ],
+      "flags": {},
+      "order": 7,
+      "mode": 0,
+      "inputs": [
+        {
+          "name": "",
+          "type": "*",
+          "link": 209
+        }
+      ],
+      "outputs": [
+        {
+          "name": "",
+          "type": "VAE",
+          "slot_index": 0,
+          "links": [
+            210
+          ]
+        }
+      ],
+      "properties": {
+        "showOutputText": false,
+        "horizontal": false
+      }
+    },
+    {
+      "id": 147,
+      "type": "Reroute",
+      "pos": [
+        3679,
+        -85
+      ],
+      "size": [
+        140.8000030517578,
+        26
+      ],
+      "flags": {},
+      "order": 25,
+      "mode": 0,
+      "inputs": [
+        {
+          "name": "",
+          "type": "*",
+          "link": 435
+        }
+      ],
+      "outputs": [
+        {
+          "name": "CONDITIONING",
+          "type": "CONDITIONING",
+          "slot_index": 0,
+          "links": [
+            439,
+            445
+          ]
+        }
+      ],
+      "properties": {
+        "showOutputText": true,
+        "horizontal": false
+      }
+    },
+    {
+      "id": 148,
+      "type": "Reroute",
+      "pos": [
+        3681,
+        -44
+      ],
+      "size": [
+        140.8000030517578,
+        26
+      ],
+      "flags": {},
+      "order": 26,
+      "mode": 0,
+      "inputs": [
+        {
+          "name": "",
+          "type": "*",
+          "link": 436
+        }
+      ],
+      "outputs": [
+        {
+          "name": "CONDITIONING",
+          "type": "CONDITIONING",
+          "slot_index": 0,
+          "links": [
+            440,
+            446
+          ]
+        }
+      ],
+      "properties": {
+        "showOutputText": true,
+        "horizontal": false
+      }
+    },
+    {
+      "id": 150,
+      "type": "Reroute",
+      "pos": [
+        3683.89599609375,
+        15.209931373596191
+      ],
+      "size": [
+        75,
+        26
+      ],
+      "flags": {},
+      "order": 22,
+      "mode": 0,
+      "inputs": [
+        {
+          "name": "",
+          "type": "*",
+          "widget": {
+            "name": "value"
+          },
+          "link": 442
+        }
+      ],
+      "outputs": [
+        {
+          "name": "INT",
+          "type": "INT",
+          "slot_index": 0,
+          "links": [
+            443,
+            448
+          ]
+        }
+      ],
+      "properties": {
+        "showOutputText": true,
+        "horizontal": false
+      }
+    },
+    {
+      "id": 63,
+      "type": "KSamplerAdvanced",
+      "pos": [
+        4274,
+        -795
+      ],
+      "size": [
+        315,
+        518
+      ],
+      "flags": {},
+      "order": 30,
+      "mode": 0,
+      "inputs": [
+        {
+          "name": "model",
+          "type": "MODEL",
+          "link": 734
+        },
+        {
+          "name": "positive",
+          "type": "CONDITIONING",
+          "link": 211
+        },
+        {
+          "name": "negative",
+          "type": "CONDITIONING",
+          "link": 212
+        },
+        {
+          "name": "latent_image",
+          "type": "LATENT",
+          "link": 176
+        },
+        {
+          "name": "noise_seed",
+          "type": "INT",
+          "widget": {
+            "name": "noise_seed"
+          },
+          "link": 430
+        }
+      ],
+      "outputs": [
+        {
+          "name": "LATENT",
+          "shape": 3,
+          "type": "LATENT",
+          "slot_index": 0,
+          "links": [
+            441
+          ]
+        }
+      ],
+      "properties": {
+        "cnr_id": "comfy-core",
+        "ver": "0.3.27",
+        "Node name for S&R": "KSamplerAdvanced"
+      },
+      "widgets_values": [
+        "enable",
+        95094378581456,
+        "randomize",
+        40,
+        7,
+        "dpmpp_3m_sde_gpu",
+        "simple",
+        20,
+        40,
+        "disable"
+      ]
+    },
+    {
+      "id": 149,
+      "type": "Reroute",
+      "pos": [
+        3730,
+        -125
+      ],
+      "size": [
+        82,
+        26
+      ],
+      "flags": {},
+      "order": 24,
+      "mode": 0,
+      "inputs": [
+        {
+          "name": "",
+          "type": "*",
+          "link": 437
+        }
+      ],
+      "outputs": [
+        {
+          "name": "MODEL",
+          "type": "MODEL",
+          "slot_index": 0,
+          "links": [
+            444,
+            734
+          ]
+        }
+      ],
+      "properties": {
+        "showOutputText": true,
+        "horizontal": false
+      }
+    },
+    {
+      "id": 106,
+      "type": "Note",
+      "pos": [
+        -990,
+        -936
+      ],
+      "size": [
+        210,
+        140.22134399414062
+      ],
+      "flags": {},
+      "order": 0,
+      "mode": 0,
+      "inputs": [],
+      "outputs": [],
+      "properties": {
+        "text": ""
+      },
+      "widgets_values": [
+        "To simuulate image coming via API, right click on pipe input and click 'convert init_img to input'.\n(or face swap)\n\nDrag output from Vyro Image to String to init_img connector"
+      ],
+      "color": "#432",
+      "bgcolor": "#653"
+    },
+    {
+      "id": 103,
+      "type": "Vyro Image to String",
+      "pos": [
+        -932,
+        -711
+      ],
+      "size": [
+        168,
+        26
+      ],
+      "flags": {},
+      "order": 8,
+      "mode": 0,
+      "inputs": [
+        {
+          "name": "image",
+          "type": "IMAGE",
+          "link": 326
+        }
+      ],
+      "outputs": [
+        {
+          "name": "string",
+          "shape": 3,
+          "type": "STRING",
+          "slot_index": 0,
+          "links": [
+            950
+          ]
+        }
+      ],
+      "properties": {
+        "aux_id": "Vyro-ai/vyro-workflows",
+        "ver": "bf85eeb45327c24b3fa1c946e86a28fae2056e80",
+        "Node name for S&R": "Vyro Image to String"
+      },
+      "widgets_values": []
+    },
+    {
+      "id": 68,
+      "type": "Note",
+      "pos": [
+        111,
+        -1016
+      ],
+      "size": [
+        228.4390106201172,
+        416.8971862792969
+      ],
+      "flags": {},
+      "order": 1,
+      "mode": 0,
+      "inputs": [],
+      "outputs": [],
+      "properties": {
+        "text": ""
+      },
+      "widgets_values": [
+        "640 x 1536\n768 x 1344\n832 x 1216\n896 x 1152\n1024 x 1024\n1152 x 896\n1216 x 832\n1344 x 768\n1536 x 640\n\n\"3d render\",\n        \"abstract art\",\n        \"anime\",\n        \"architecture\",\n        \"cinematic\",\n        \"conceptual art\",\n        \"dark fantasy\",\n        \"fantasy realism\",\n        \"fashion\",\n        \"graffiti\",\n        \"illustration\",\n        \"interior design\",\n        \"logo\",\n        \"painting\",\n        \"photography:1.2\",\n        \"portrait photography\",\n        \"poster\",\n        \"product:0.75\",\n        \"sticker\",\n        \"surrealism\",\n        \"typography\",\n        \"ukiyo-e\",\n        \"vector design\",\n        \"vibrant digital artwork:0.1\",\n        \"watercolor\",\n        \"wildlife photography\""
+      ],
+      "color": "#432",
+      "bgcolor": "#653"
+    },
+    {
+      "id": 2,
+      "type": "VAELoader",
+      "pos": [
+        -1066,
+        -1026
+      ],
+      "size": [
+        315,
+        58
+      ],
+      "flags": {},
+      "order": 2,
+      "mode": 0,
+      "inputs": [],
+      "outputs": [
+        {
+          "name": "VAE",
+          "shape": 3,
+          "type": "VAE",
+          "slot_index": 0,
+          "links": [
+            209
+          ]
+        }
+      ],
+      "properties": {
+        "cnr_id": "comfy-core",
+        "ver": "0.3.27",
+        "Node name for S&R": "VAELoader"
+      },
+      "widgets_values": [
+        "sdxl_vae.safetensors"
+      ]
+    },
+    {
+      "id": 27,
+      "type": "Vyro Prompt Analyzer",
+      "pos": [
+        761,
+        -525
+      ],
+      "size": [
+        315,
+        142
+      ],
+      "flags": {},
+      "order": 12,
+      "mode": 0,
+      "inputs": [
+        {
+          "name": "vyro_params",
+          "type": "VYRO_PARAMS",
+          "link": 48
+        },
+        {
+          "name": "styles",
+          "type": "LIST",
+          "link": 75
+        },
+        {
+          "name": "prompt_tree",
+          "type": "DICT",
+          "link": 50
+        },
+        {
+          "name": "classifier",
+          "type": "TRANSFORMER",
+          "link": 51
+        }
+      ],
+      "outputs": [
+        {
+          "name": "vyro_params",
+          "shape": 3,
+          "type": "VYRO_PARAMS",
+          "slot_index": 0,
+          "links": [
+            147,
+            1205
+          ]
+        },
+        {
+          "name": "style",
+          "shape": 3,
+          "type": "STYLE",
+          "slot_index": 1,
+          "links": [
+            1147
+          ]
+        }
+      ],
+      "properties": {
+        "aux_id": "Vyro-ai/vyro-workflows",
+        "ver": "bf85eeb45327c24b3fa1c946e86a28fae2056e80",
+        "Node name for S&R": "Vyro Prompt Analyzer"
+      },
+      "widgets_values": [
+        "enabled",
+        "disabled"
+      ]
+    },
+    {
+      "id": 385,
+      "type": "Reroute",
+      "pos": [
+        643,
+        -895
+      ],
+      "size": [
+        82,
+        26
+      ],
+      "flags": {},
+      "order": 14,
+      "mode": 0,
+      "inputs": [
+        {
+          "name": "",
+          "type": "*",
+          "link": 1147
+        }
+      ],
+      "outputs": [
+        {
+          "name": "STYLE",
+          "type": "STYLE",
+          "slot_index": 0,
+          "links": [
+            1152,
+            1153
+          ]
+        }
+      ],
+      "properties": {
+        "showOutputText": true,
+        "horizontal": false
+      }
+    },
+    {
+      "id": 386,
+      "type": "Reroute",
+      "pos": [
+        620,
+        -1042
+      ],
+      "size": [
+        75,
+        26
+      ],
+      "flags": {},
+      "order": 9,
+      "mode": 0,
+      "inputs": [
+        {
+          "name": "",
+          "type": "*",
+          "link": 1149
+        }
+      ],
+      "outputs": [
+        {
+          "name": "DICT",
+          "type": "DICT",
+          "slot_index": 0,
+          "links": [
+            1150,
+            1155
+          ]
+        }
+      ],
+      "properties": {
+        "showOutputText": true,
+        "horizontal": false
+      }
+    },
+    {
+      "id": 387,
+      "type": "Reroute",
+      "pos": [
+        639,
+        -968
+      ],
+      "size": [
+        75,
+        26
+      ],
+      "flags": {},
+      "order": 10,
+      "mode": 0,
+      "inputs": [
+        {
+          "name": "",
+          "type": "*",
+          "link": 1148
+        }
+      ],
+      "outputs": [
+        {
+          "name": "DICT",
+          "type": "DICT",
+          "slot_index": 0,
+          "links": [
+            1151,
+            1156
+          ]
+        }
+      ],
+      "properties": {
+        "showOutputText": true,
+        "horizontal": false
+      }
+    },
+    {
+      "id": 384,
+      "type": "Vyro Oneflow Refiner Model Loader",
+      "pos": [
+        871,
+        -826
+      ],
+      "size": [
+        330,
+        66
+      ],
+      "flags": {},
+      "order": 17,
+      "mode": 0,
+      "inputs": [
+        {
+          "name": "style",
+          "type": "STYLE",
+          "link": 1153
+        },
+        {
+          "name": "prompt_tree",
+          "type": "DICT",
+          "link": 1155
+        },
+        {
+          "name": "model_config",
+          "type": "DICT",
+          "link": 1156
+        }
+      ],
+      "outputs": [
+        {
+          "name": "refiner_model",
+          "shape": 3,
+          "type": "MODEL",
+          "slot_index": 0,
+          "links": [
+            1165
+          ]
+        },
+        {
+          "name": "refiner_clip",
+          "shape": 3,
+          "type": "CLIP",
+          "slot_index": 1,
+          "links": [
+            1158
+          ]
+        }
+      ],
+      "properties": {
+        "aux_id": "Vyro-ai/vyro-workflows",
+        "ver": "bf85eeb45327c24b3fa1c946e86a28fae2056e80",
+        "Node name for S&R": "Vyro Oneflow Refiner Model Loader"
+      },
+      "widgets_values": []
+    },
+    {
+      "id": 57,
+      "type": "VAEDecode",
+      "pos": [
+        5441,
+        -641
+      ],
+      "size": [
+        210,
+        46
+      ],
+      "flags": {},
+      "order": 33,
+      "mode": 0,
+      "inputs": [
+        {
+          "name": "samples",
+          "type": "LATENT",
+          "link": 728
+        },
+        {
+          "name": "vae",
+          "type": "VAE",
+          "link": 323
+        }
+      ],
+      "outputs": [
+        {
+          "name": "IMAGE",
+          "shape": 3,
+          "type": "IMAGE",
+          "slot_index": 0,
+          "links": [
+            1142
+          ]
+        }
+      ],
+      "properties": {
+        "cnr_id": "comfy-core",
+        "ver": "0.3.27",
+        "Node name for S&R": "VAEDecode"
+      },
+      "widgets_values": []
+    },
+    {
+      "id": 84,
+      "type": "PreviewImage",
+      "pos": [
+        5746,
+        -719
+      ],
+      "size": [
+        511.7968444824219,
+        426.3374938964844
+      ],
+      "flags": {},
+      "order": 34,
+      "mode": 0,
+      "inputs": [
+        {
+          "name": "images",
+          "type": "IMAGE",
+          "link": 1142
+        }
+      ],
+      "outputs": [],
+      "properties": {
+        "cnr_id": "comfy-core",
+        "ver": "0.3.27",
+        "Node name for S&R": "PreviewImage"
+      },
+      "widgets_values": [
+        ""
+      ]
+    },
+    {
+      "id": 145,
+      "type": "KSamplerAdvanced",
+      "pos": [
+        4694,
+        -293
+      ],
+      "size": [
+        315,
+        518
+      ],
+      "flags": {},
+      "order": 31,
+      "mode": 0,
+      "inputs": [
+        {
+          "name": "model",
+          "type": "MODEL",
+          "link": 732
+        },
+        {
+          "name": "positive",
+          "type": "CONDITIONING",
+          "link": 439
+        },
+        {
+          "name": "negative",
+          "type": "CONDITIONING",
+          "link": 440
+        },
+        {
+          "name": "latent_image",
+          "type": "LATENT",
+          "link": 441
+        },
+        {
+          "name": "noise_seed",
+          "type": "INT",
+          "widget": {
+            "name": "noise_seed"
+          },
+          "link": 443
+        }
+      ],
+      "outputs": [
+        {
+          "name": "LATENT",
+          "shape": 3,
+          "type": "LATENT",
+          "slot_index": 0,
+          "links": [
+            447
+          ]
+        }
+      ],
+      "title": "Step1",
+      "properties": {
+        "cnr_id": "comfy-core",
+        "ver": "0.3.27",
+        "Node name for S&R": "KSamplerAdvanced"
+      },
+      "widgets_values": [
+        "enable",
+        1114398158696762,
+        "randomize",
+        30,
+        7,
+        "ddim",
+        "karras",
+        25,
+        27,
+        "enable"
+      ]
+    },
+    {
+      "id": 151,
+      "type": "KSamplerAdvanced",
+      "pos": [
+        5085,
+        -268
+      ],
+      "size": [
+        315,
+        518
+      ],
+      "flags": {},
+      "order": 32,
+      "mode": 0,
+      "inputs": [
+        {
+          "name": "model",
+          "type": "MODEL",
+          "link": 444
+        },
+        {
+          "name": "positive",
+          "type": "CONDITIONING",
+          "link": 445
+        },
+        {
+          "name": "negative",
+          "type": "CONDITIONING",
+          "link": 446
+        },
+        {
+          "name": "latent_image",
+          "type": "LATENT",
+          "link": 447
+        },
+        {
+          "name": "noise_seed",
+          "type": "INT",
+          "widget": {
+            "name": "noise_seed"
+          },
+          "link": 448
+        }
+      ],
+      "outputs": [
+        {
+          "name": "LATENT",
+          "shape": 3,
+          "type": "LATENT",
+          "slot_index": 0,
+          "links": [
+            728
+          ]
+        }
+      ],
+      "title": "Step2",
+      "properties": {
+        "cnr_id": "comfy-core",
+        "ver": "0.3.27",
+        "Node name for S&R": "KSamplerAdvanced"
+      },
+      "widgets_values": [
+        "disable",
+        1089159239373161,
+        "randomize",
+        30,
+        7,
+        "ddim",
+        "karras",
+        27,
+        30,
+        "enable"
+      ]
+    },
+    {
+      "id": 254,
+      "type": "Reroute",
+      "pos": [
+        1837,
+        -1031
+      ],
+      "size": [
+        82,
+        26
+      ],
+      "flags": {},
+      "order": 19,
+      "mode": 0,
+      "inputs": [
+        {
+          "name": "",
+          "type": "*",
+          "link": 1164
+        }
+      ],
+      "outputs": [
+        {
+          "name": "MODEL",
+          "type": "MODEL",
+          "slot_index": 0,
+          "links": [
+            1195
+          ]
+        }
+      ],
+      "properties": {
+        "showOutputText": true,
+        "horizontal": false
+      }
+    },
+    {
+      "id": 28,
+      "type": "Vyro Prompt Encoder",
+      "pos": [
+        1908,
+        -959
+      ],
+      "size": [
+        367.79998779296875,
+        118
+      ],
+      "flags": {},
+      "order": 21,
+      "mode": 0,
+      "inputs": [
+        {
+          "name": "base_clip",
+          "type": "CLIP",
+          "link": 1144
+        },
+        {
+          "name": "refiner_clip",
+          "type": "CLIP",
+          "link": 1158
+        },
+        {
+          "name": "params",
+          "type": "VYRO_PARAMS",
+          "link": 1205
+        }
+      ],
+      "outputs": [
+        {
+          "name": "base_positive",
+          "shape": 3,
+          "type": "CONDITIONING",
+          "slot_index": 0,
+          "links": [
+            1166
+          ]
+        },
+        {
+          "name": "base_negative",
+          "shape": 3,
+          "type": "CONDITIONING",
+          "slot_index": 1,
+          "links": [
+            1167
+          ]
+        },
+        {
+          "name": "refiner_positive",
+          "shape": 3,
+          "type": "CONDITIONING",
+          "slot_index": 2,
+          "links": [
+            211,
+            435
+          ]
+        },
+        {
+          "name": "refiner_negative",
+          "shape": 3,
+          "type": "CONDITIONING",
+          "slot_index": 3,
+          "links": [
+            212,
+            436
+          ]
+        }
+      ],
+      "properties": {
+        "aux_id": "Vyro-ai/vyro-workflows",
+        "ver": "bf85eeb45327c24b3fa1c946e86a28fae2056e80",
+        "Node name for S&R": "Vyro Prompt Encoder"
+      },
+      "widgets_values": [
+        0
+      ]
+    },
+    {
+      "id": 55,
+      "type": "Reroute",
+      "pos": [
+        2227,
+        -688
+      ],
+      "size": [
+        75,
+        26
+      ],
+      "flags": {},
+      "order": 13,
+      "mode": 0,
+      "inputs": [
+        {
+          "name": "",
+          "type": "*",
+          "link": 147
+        }
+      ],
+      "outputs": [
+        {
+          "name": "",
+          "type": "VYRO_PARAMS",
+          "slot_index": 0,
+          "links": [
+            624
+          ]
+        }
+      ],
+      "properties": {
+        "showOutputText": false,
+        "horizontal": false
+      }
+    },
+    {
+      "id": 135,
+      "type": "Vyro Mode Filter",
+      "pos": [
+        2738,
+        -899
+      ],
+      "size": [
+        315,
+        250
+      ],
+      "flags": {},
+      "order": 15,
+      "mode": 0,
+      "inputs": [
+        {
+          "name": "vyro_params",
+          "type": "VYRO_PARAMS",
+          "link": 624
+        }
+      ],
+      "outputs": [
+        {
+          "name": "vyro_params",
+          "shape": 3,
+          "type": "VYRO_PARAMS",
+          "slot_index": 0,
+          "links": [
+            429
+          ]
+        }
+      ],
+      "properties": {
+        "aux_id": "Vyro-ai/vyro-workflows",
+        "ver": "bf85eeb45327c24b3fa1c946e86a28fae2056e80",
+        "Node name for S&R": "Vyro Mode Filter"
+      },
+      "widgets_values": [
+        "allow",
+        "allow",
+        "allow"
+      ]
+    },
+    {
+      "id": 101,
+      "type": "Reroute",
+      "pos": [
+        2229,
+        -766
+      ],
+      "size": [
+        75,
+        26
+      ],
+      "flags": {},
+      "order": 20,
+      "mode": 0,
+      "inputs": [
+        {
+          "name": "",
+          "type": "*",
+          "link": 1165
+        }
+      ],
+      "outputs": [
+        {
+          "name": "",
+          "type": "MODEL",
+          "slot_index": 0,
+          "links": [
+            437,
+            732
+          ]
+        }
+      ],
+      "properties": {
+        "showOutputText": false,
+        "horizontal": false
+      }
+    },
+    {
+      "id": 390,
+      "type": "KSamplerAdvanced",
+      "pos": [
+        3448,
+        -1001
+      ],
+      "size": [
+        315,
+        334
+      ],
+      "flags": {},
+      "order": 27,
+      "mode": 0,
+      "inputs": [
+        {
+          "name": "model",
+          "type": "MODEL",
+          "link": 1196
+        },
+        {
+          "name": "positive",
+          "type": "CONDITIONING",
+          "link": 1166
+        },
+        {
+          "name": "negative",
+          "type": "CONDITIONING",
+          "link": 1167
+        },
+        {
+          "name": "latent_image",
+          "type": "LATENT",
+          "link": 1168
+        },
+        {
+          "name": "noise_seed",
+          "type": "INT",
+          "widget": {
+            "name": "noise_seed"
+          },
+          "link": 1169
+        }
+      ],
+      "outputs": [
+        {
+          "name": "LATENT",
+          "shape": 3,
+          "type": "LATENT",
+          "slot_index": 0,
+          "links": [
+            1204
+          ]
+        }
+      ],
+      "properties": {
+        "cnr_id": "comfy-core",
+        "ver": "0.3.27",
+        "Node name for S&R": "KSamplerAdvanced"
+      },
+      "widgets_values": [
+        "enable",
+        144870654803597,
+        "randomize",
+        20,
+        8,
+        "euler",
+        "normal",
+        0,
+        10000,
+        "disable"
+      ]
+    },
+    {
+      "id": 100,
+      "type": "VAELoader",
+      "pos": [
+        1540,
+        143
+      ],
+      "size": [
+        412.548095703125,
+        58
+      ],
+      "flags": {},
+      "order": 3,
+      "mode": 0,
+      "inputs": [],
+      "outputs": [
+        {
+          "name": "VAE",
+          "shape": 3,
+          "type": "VAE",
+          "slot_index": 0,
+          "links": [
+            323
+          ]
+        }
+      ],
+      "properties": {
+        "cnr_id": "comfy-core",
+        "ver": "0.3.27",
+        "Node name for S&R": "VAELoader"
+      },
+      "widgets_values": [
+        "vae-ft-mse-840000-ema-pruned.safetensors"
+      ]
+    },
+    {
+      "id": 48,
+      "type": "VyroLatentInterposer",
+      "pos": [
+        3861,
+        -1029
+      ],
+      "size": [
+        315,
+        82
+      ],
+      "flags": {},
+      "order": 28,
+      "mode": 0,
+      "inputs": [
+        {
+          "name": "samples",
+          "type": "LATENT",
+          "link": 1204
+        }
+      ],
+      "outputs": [
+        {
+          "name": "LATENT",
+          "shape": 3,
+          "type": "LATENT",
+          "slot_index": 0,
+          "links": [
+            140
+          ]
+        }
+      ],
+      "properties": {
+        "aux_id": "Vyro-ai/vyro-workflows",
+        "ver": "bf85eeb45327c24b3fa1c946e86a28fae2056e80",
+        "Node name for S&R": "VyroLatentInterposer"
+      },
+      "widgets_values": [
+        "xl",
+        "v1"
+      ]
+    },
+    {
+      "id": 393,
+      "type": "Reroute",
+      "pos": [
+        3095,
+        -1056
+      ],
+      "size": [
+        75,
+        26
+      ],
+      "flags": {},
+      "order": 23,
+      "mode": 0,
+      "inputs": [
+        {
+          "name": "",
+          "type": "*",
+          "link": 1195
+        }
+      ],
+      "outputs": [
+        {
+          "name": "",
+          "type": "MODEL",
+          "slot_index": 0,
+          "links": [
+            1196
+          ]
+        }
+      ],
+      "properties": {
+        "showOutputText": false,
+        "horizontal": false
+      }
+    },
+    {
+      "id": 102,
+      "type": "LoadImage",
+      "pos": [
+        -1527,
+        -880
+      ],
+      "size": [
+        315,
+        314
+      ],
+      "flags": {},
+      "order": 4,
+      "mode": 0,
+      "inputs": [],
+      "outputs": [
+        {
+          "name": "IMAGE",
+          "shape": 3,
+          "type": "IMAGE",
+          "slot_index": 0,
+          "links": [
+            326
+          ]
+        },
+        {
+          "name": "MASK",
+          "shape": 3,
+          "type": "MASK",
+          "links": null
+        }
+      ],
+      "properties": {
+        "cnr_id": "comfy-core",
+        "ver": "0.3.27",
+        "Node name for S&R": "LoadImage"
+      },
+      "widgets_values": [
+        "example.png",
+        "image",
+        ""
+      ]
+    },
+    {
+      "id": 25,
+      "type": "Vyro Config Loader",
+      "pos": [
+        125,
+        -526
+      ],
+      "size": [
+        315,
+        162
+      ],
+      "flags": {},
+      "order": 5,
+      "mode": 0,
+      "inputs": [],
+      "outputs": [
+        {
+          "name": "styles",
+          "shape": 3,
+          "type": "LIST",
+          "slot_index": 0,
+          "links": [
+            75
+          ]
+        },
+        {
+          "name": "prompt_tree",
+          "shape": 3,
+          "type": "DICT",
+          "slot_index": 1,
+          "links": [
+            50,
+            1149
+          ]
+        },
+        {
+          "name": "model_config",
+          "shape": 3,
+          "type": "DICT",
+          "slot_index": 2,
+          "links": [
+            1148
+          ]
+        },
+        {
+          "name": "classifier",
+          "shape": 3,
+          "type": "TRANSFORMER",
+          "slot_index": 3,
+          "links": [
+            51
+          ]
+        },
+        {
+          "name": "unweighted_styles",
+          "shape": 3,
+          "type": "LIST",
+          "slot_index": 4,
+          "links": []
+        }
+      ],
+      "properties": {
+        "aux_id": "Vyro-ai/vyro-workflows",
+        "ver": "bf85eeb45327c24b3fa1c946e86a28fae2056e80",
+        "Node name for S&R": "Vyro Config Loader"
+      },
+      "widgets_values": [
+        "v5.json",
+        "en_core_web_trf-3.8.0"
+      ]
+    },
+    {
+      "id": 382,
+      "type": "Vyro Oneflow Base Model Loader",
+      "pos": [
+        644.8353881835938,
+        -1211.888671875
+      ],
+      "size": [
+        315,
+        78
+      ],
+      "flags": {},
+      "order": 6,
+      "mode": 0,
+      "inputs": [],
+      "outputs": [
+        {
+          "name": "base_model",
+          "shape": 3,
+          "type": "MODEL",
+          "slot_index": 0,
+          "links": [
+            1206
+          ]
+        },
+        {
+          "name": "base_clip",
+          "shape": 3,
+          "type": "CLIP",
+          "slot_index": 1,
+          "links": [
+            1146
+          ]
+        }
+      ],
+      "properties": {
+        "aux_id": "Vyro-ai/vyro-workflows",
+        "ver": "bf85eeb45327c24b3fa1c946e86a28fae2056e80",
+        "Node name for S&R": "Vyro Oneflow Base Model Loader"
+      },
+      "widgets_values": [
+        "sd_xl_base_1.0.safetensors"
+      ]
+    },
+    {
+      "id": 383,
+      "type": "Vyro LoRa Loader",
+      "pos": [
+        1137.228515625,
+        -1234.281005859375
+      ],
+      "size": [
+        292.20001220703125,
+        106
+      ],
+      "flags": {},
+      "order": 16,
+      "mode": 0,
+      "inputs": [
+        {
+          "name": "base_model",
+          "type": "MODEL",
+          "link": 1206
+        },
+        {
+          "name": "base_clip",
+          "type": "CLIP",
+          "link": 1146
+        },
+        {
+          "name": "style",
+          "type": "STYLE",
+          "link": 1152
+        },
+        {
+          "name": "prompt_tree",
+          "type": "DICT",
+          "link": 1150
+        },
+        {
+          "name": "model_config",
+          "type": "DICT",
+          "link": 1151
+        }
+      ],
+      "outputs": [
+        {
+          "name": "base_model",
+          "shape": 3,
+          "type": "MODEL",
+          "slot_index": 0,
+          "links": [
+            1164
+          ]
+        },
+        {
+          "name": "base_clip",
+          "shape": 3,
+          "type": "CLIP",
+          "slot_index": 1,
+          "links": [
+            1144
+          ]
+        }
+      ],
+      "properties": {
+        "aux_id": "Vyro-ai/vyro-workflows",
+        "ver": "bf85eeb45327c24b3fa1c946e86a28fae2056e80",
+        "Node name for S&R": "Vyro LoRa Loader"
+      },
+      "widgets_values": []
+    },
+    {
+      "id": 144,
+      "type": "Vyro Param Extractor",
+      "pos": [
+        2857.13623046875,
+        -499.512939453125
+      ],
+      "size": [
+        418.1999816894531,
+        466
+      ],
+      "flags": {},
+      "order": 18,
+      "mode": 0,
+      "inputs": [
+        {
+          "name": "vyro_params",
+          "type": "VYRO_PARAMS",
+          "link": 429
+        }
+      ],
+      "outputs": [
+        {
+          "name": "latents",
+          "shape": 3,
+          "type": "LATENT",
+          "slot_index": 0,
+          "links": [
+            1168
+          ]
+        },
+        {
+          "name": "user_prompt",
+          "shape": 3,
+          "type": "STRING",
+          "links": null
+        },
+        {
+          "name": "user_neg_prompt",
+          "shape": 3,
+          "type": "STRING",
+          "links": null
+        },
+        {
+          "name": "mode",
+          "shape": 3,
+          "type": "STRING",
+          "links": null
+        },
+        {
+          "name": "cfg",
+          "shape": 3,
+          "type": "FLOAT",
+          "links": null
+        },
+        {
+          "name": "batch_size",
+          "shape": 3,
+          "type": "INT",
+          "links": null
+        },
+        {
+          "name": "steps",
+          "shape": 3,
+          "type": "INT",
+          "links": null
+        },
+        {
+          "name": "width",
+          "shape": 3,
+          "type": "INT",
+          "links": null
+        },
+        {
+          "name": "height",
+          "shape": 3,
+          "type": "INT",
+          "links": null
+        },
+        {
+          "name": "seed",
+          "shape": 3,
+          "type": "INT",
+          "slot_index": 9,
+          "links": [
+            430,
+            442,
+            1169
+          ]
+        },
+        {
+          "name": "denoise",
+          "shape": 3,
+          "type": "FLOAT",
+          "slot_index": 10,
+          "links": []
+        },
+        {
+          "name": "stage1_strength",
+          "shape": 3,
+          "type": "FLOAT",
+          "links": null
+        },
+        {
+          "name": "stage2_strength",
+          "shape": 3,
+          "type": "FLOAT",
+          "links": []
+        },
+        {
+          "name": "efficiency_multiplier",
+          "shape": 3,
+          "type": "FLOAT",
+          "links": [
+            1209
+          ]
+        },
+        {
+          "name": "style",
+          "shape": 3,
+          "type": "STRING",
+          "links": null
+        },
+        {
+          "name": "final_positive_prompt",
+          "shape": 3,
+          "type": "STRING",
+          "slot_index": 15,
+          "links": []
+        },
+        {
+          "name": "final_negative_prompt",
+          "shape": 3,
+          "type": "STRING",
+          "links": null
+        },
+        {
+          "name": "is_raw",
+          "shape": 3,
+          "type": "BOOLEAN",
+          "links": null
+        },
+        {
+          "name": "final_negative_prompt",
+          "shape": 3,
+          "type": "STRING",
+          "links": null
+        },
+        {
+          "name": "is_raw",
+          "shape": 3,
+          "type": "BOOLEAN",
+          "links": null
+        },
+        {
+          "name": "face_swap_img",
+          "shape": 3,
+          "type": "IMAGE",
+          "links": null
+        },
+        {
+          "name": "image_prompt_weights",
+          "shape": 3,
+          "type": "STRING",
+          "links": null
+        },
+        {
+          "name": "control_net_input_img",
+          "shape": 3,
+          "type": "IMAGE",
+          "links": null
+        }
+      ],
+      "properties": {
+        "aux_id": "Vyro-ai/vyro-workflows",
+        "ver": "bf85eeb45327c24b3fa1c946e86a28fae2056e80",
+        "Node name for S&R": "Vyro Param Extractor"
+      },
+      "widgets_values": []
+    },
+    {
+      "id": 49,
+      "type": "LatentUpscaleBy",
+      "pos": [
+        4250,
+        -1046
+      ],
+      "size": [
+        315,
+        82
+      ],
+      "flags": {},
+      "order": 29,
+      "mode": 0,
+      "inputs": [
+        {
+          "name": "samples",
+          "type": "LATENT",
+          "link": 140
+        },
+        {
+          "name": "scale_by",
+          "type": "FLOAT",
+          "widget": {
+            "name": "scale_by"
+          },
+          "link": 1209
+        }
+      ],
+      "outputs": [
+        {
+          "name": "LATENT",
+          "shape": 3,
+          "type": "LATENT",
+          "slot_index": 0,
+          "links": [
+            176
+          ]
+        }
+      ],
+      "properties": {
+        "cnr_id": "comfy-core",
+        "ver": "0.3.27",
+        "Node name for S&R": "LatentUpscaleBy"
+      },
+      "widgets_values": [
+        "nearest-exact",
+        1.333
+      ]
+    },
+    {
+      "id": 16,
+      "type": "Vyro Pipe Input V2",
+      "pos": [
+        -584,
+        -1007
+      ],
+      "size": [
+        400,
+        732
+      ],
+      "flags": {},
+      "order": 11,
+      "mode": 0,
+      "inputs": [
+        {
+          "name": "vae",
+          "type": "VAE",
+          "link": 210
+        },
+        {
+          "name": "init_img",
+          "type": "STRING",
+          "widget": {
+            "name": "init_img"
+          },
+          "link": 950
+        }
+      ],
+      "outputs": [
+        {
+          "name": "vyro_params",
+          "shape": 3,
+          "type": "VYRO_PARAMS",
+          "slot_index": 0,
+          "links": [
+            48
+          ]
+        }
+      ],
+      "properties": {
+        "aux_id": "Vyro-ai/vyro-workflows",
+        "ver": "bf85eeb45327c24b3fa1c946e86a28fae2056e80",
+        "Node name for S&R": "Vyro Pipe Input V2"
+      },
+      "widgets_values": [
+        "professor working in lab",
+        "t2i",
+        "perlin1",
+        "",
+        1,
+        8,
+        10,
+        1024,
+        1024,
+        2890257064,
+        "randomize",
+        "",
+        0.8,
+        1,
+        1,
+        1
+      ]
+    }
+  ],
+  "links": [
+    [
+      48,
+      16,
+      0,
+      27,
+      0,
+      "VYRO_PARAMS"
+    ],
+    [
+      50,
+      25,
+      1,
+      27,
+      2,
+      "DICT"
+    ],
+    [
+      51,
+      25,
+      3,
+      27,
+      3,
+      "TRANSFORMER"
+    ],
+    [
+      75,
+      25,
+      0,
+      27,
+      1,
+      "LIST"
+    ],
+    [
+      140,
+      48,
+      0,
+      49,
+      0,
+      "LATENT"
+    ],
+    [
+      147,
+      27,
+      0,
+      55,
+      0,
+      "*"
+    ],
+    [
+      176,
+      49,
+      0,
+      63,
+      3,
+      "LATENT"
+    ],
+    [
+      209,
+      2,
+      0,
+      71,
+      0,
+      "*"
+    ],
+    [
+      210,
+      71,
+      0,
+      16,
+      0,
+      "VAE"
+    ],
+    [
+      211,
+      28,
+      2,
+      63,
+      1,
+      "CONDITIONING"
+    ],
+    [
+      212,
+      28,
+      3,
+      63,
+      2,
+      "CONDITIONING"
+    ],
+    [
+      323,
+      100,
+      0,
+      57,
+      1,
+      "VAE"
+    ],
+    [
+      326,
+      102,
+      0,
+      103,
+      0,
+      "IMAGE"
+    ],
+    [
+      429,
+      135,
+      0,
+      144,
+      0,
+      "VYRO_PARAMS"
+    ],
+    [
+      430,
+      144,
+      9,
+      63,
+      4,
+      "INT"
+    ],
+    [
+      435,
+      28,
+      2,
+      147,
+      0,
+      "*"
+    ],
+    [
+      436,
+      28,
+      3,
+      148,
+      0,
+      "*"
+    ],
+    [
+      437,
+      101,
+      0,
+      149,
+      0,
+      "*"
+    ],
+    [
+      439,
+      147,
+      0,
+      145,
+      1,
+      "CONDITIONING"
+    ],
+    [
+      440,
+      148,
+      0,
+      145,
+      2,
+      "CONDITIONING"
+    ],
+    [
+      441,
+      63,
+      0,
+      145,
+      3,
+      "LATENT"
+    ],
+    [
+      442,
+      144,
+      9,
+      150,
+      0,
+      "*"
+    ],
+    [
+      443,
+      150,
+      0,
+      145,
+      4,
+      "INT"
+    ],
+    [
+      444,
+      149,
+      0,
+      151,
+      0,
+      "MODEL"
+    ],
+    [
+      445,
+      147,
+      0,
+      151,
+      1,
+      "CONDITIONING"
+    ],
+    [
+      446,
+      148,
+      0,
+      151,
+      2,
+      "CONDITIONING"
+    ],
+    [
+      447,
+      145,
+      0,
+      151,
+      3,
+      "LATENT"
+    ],
+    [
+      448,
+      150,
+      0,
+      151,
+      4,
+      "INT"
+    ],
+    [
+      624,
+      55,
+      0,
+      135,
+      0,
+      "VYRO_PARAMS"
+    ],
+    [
+      728,
+      151,
+      0,
+      57,
+      0,
+      "LATENT"
+    ],
+    [
+      732,
+      101,
+      0,
+      145,
+      0,
+      "MODEL"
+    ],
+    [
+      734,
+      149,
+      0,
+      63,
+      0,
+      "MODEL"
+    ],
+    [
+      950,
+      103,
+      0,
+      16,
+      1,
+      "STRING"
+    ],
+    [
+      1142,
+      57,
+      0,
+      84,
+      0,
+      "IMAGE"
+    ],
+    [
+      1144,
+      383,
+      1,
+      28,
+      0,
+      "CLIP"
+    ],
+    [
+      1146,
+      382,
+      1,
+      383,
+      1,
+      "CLIP"
+    ],
+    [
+      1147,
+      27,
+      1,
+      385,
+      0,
+      "*"
+    ],
+    [
+      1148,
+      25,
+      2,
+      387,
+      0,
+      "*"
+    ],
+    [
+      1149,
+      25,
+      1,
+      386,
+      0,
+      "*"
+    ],
+    [
+      1150,
+      386,
+      0,
+      383,
+      3,
+      "DICT"
+    ],
+    [
+      1151,
+      387,
+      0,
+      383,
+      4,
+      "DICT"
+    ],
+    [
+      1152,
+      385,
+      0,
+      383,
+      2,
+      "STYLE"
+    ],
+    [
+      1153,
+      385,
+      0,
+      384,
+      0,
+      "STYLE"
+    ],
+    [
+      1155,
+      386,
+      0,
+      384,
+      1,
+      "DICT"
+    ],
+    [
+      1156,
+      387,
+      0,
+      384,
+      2,
+      "DICT"
+    ],
+    [
+      1158,
+      384,
+      1,
+      28,
+      1,
+      "CLIP"
+    ],
+    [
+      1164,
+      383,
+      0,
+      254,
+      0,
+      "*"
+    ],
+    [
+      1165,
+      384,
+      0,
+      101,
+      0,
+      "*"
+    ],
+    [
+      1166,
+      28,
+      0,
+      390,
+      1,
+      "CONDITIONING"
+    ],
+    [
+      1167,
+      28,
+      1,
+      390,
+      2,
+      "CONDITIONING"
+    ],
+    [
+      1168,
+      144,
+      0,
+      390,
+      3,
+      "LATENT"
+    ],
+    [
+      1169,
+      144,
+      9,
+      390,
+      4,
+      "INT"
+    ],
+    [
+      1195,
+      254,
+      0,
+      393,
+      0,
+      "*"
+    ],
+    [
+      1196,
+      393,
+      0,
+      390,
+      0,
+      "MODEL"
+    ],
+    [
+      1204,
+      390,
+      0,
+      48,
+      0,
+      "LATENT"
+    ],
+    [
+      1205,
+      27,
+      0,
+      28,
+      2,
+      "VYRO_PARAMS"
+    ],
+    [
+      1206,
+      382,
+      0,
+      383,
+      0,
+      "MODEL"
+    ],
+    [
+      1209,
+      144,
+      13,
+      49,
+      1,
+      "FLOAT"
+    ]
+  ],
+  "groups": [
+    {
+      "id": 1,
+      "title": "I/O",
+      "bounding": [
+        -673,
+        -1108,
+        1094,
+        787
+      ],
+      "color": "#3f789e",
+      "font_size": 24,
+      "flags": {}
+    },
+    {
+      "id": 2,
+      "title": "Analysis/Encoding",
+      "bounding": [
+        842,
+        -1110,
+        1491,
+        810
+      ],
+      "color": "#8AA",
+      "font_size": 24,
+      "flags": {}
+    },
+    {
+      "id": 3,
+      "title": "T2I/I2I",
+      "bounding": [
+        2661,
+        -1160,
+        3644,
+        1695
+      ],
+      "color": "#3f789e",
+      "font_size": 24,
+      "flags": {}
+    }
+  ],
+  "config": {},
+  "extra": {
+    "ds": {
+      "scale": 0.2853116706110015,
+      "offset": [
+        2577.217082241404,
+        2131.7449438691942
+      ]
+    }
+  },
+  "version": 0.4
+}

Imagine/Workflows/Imaginev5-ultra-Workflow.json

@@ -0,0 +1,1433 @@
+{
+  "id": "f2d9da4d-a1e8-47f3-b08e-e6bc0ea46feb",
+  "revision": 0,
+  "last_node_id": 99,
+  "last_link_id": 263,
+  "nodes": [
+    {
+      "id": 35,
+      "type": "Reroute",
+      "pos": [
+        530,
+        770
+      ],
+      "size": [
+        75,
+        26
+      ],
+      "flags": {},
+      "order": 0,
+      "mode": 0,
+      "inputs": [
+        {
+          "name": "",
+          "type": "*",
+          "link": null
+        }
+      ],
+      "outputs": [
+        {
+          "name": "",
+          "type": "*",
+          "links": []
+        }
+      ],
+      "properties": {
+        "showOutputText": false,
+        "horizontal": false
+      }
+    },
+    {
+      "id": 36,
+      "type": "Reroute",
+      "pos": [
+        530,
+        820
+      ],
+      "size": [
+        75,
+        26
+      ],
+      "flags": {},
+      "order": 1,
+      "mode": 0,
+      "inputs": [
+        {
+          "name": "",
+          "type": "*",
+          "link": null
+        }
+      ],
+      "outputs": [
+        {
+          "name": "",
+          "type": "*",
+          "links": []
+        }
+      ],
+      "properties": {
+        "showOutputText": false,
+        "horizontal": false
+      }
+    },
+    {
+      "id": 45,
+      "type": "Reroute",
+      "pos": [
+        840,
+        50
+      ],
+      "size": [
+        75,
+        26
+      ],
+      "flags": {},
+      "order": 5,
+      "mode": 0,
+      "inputs": [
+        {
+          "name": "",
+          "type": "*",
+          "link": 82
+        }
+      ],
+      "outputs": [
+        {
+          "name": "",
+          "type": "CLIP",
+          "slot_index": 0,
+          "links": [
+            83,
+            84
+          ]
+        }
+      ],
+      "properties": {
+        "showOutputText": false,
+        "horizontal": false
+      }
+    },
+    {
+      "id": 66,
+      "type": "Reroute",
+      "pos": [
+        830,
+        110
+      ],
+      "size": [
+        75,
+        26
+      ],
+      "flags": {},
+      "order": 6,
+      "mode": 0,
+      "inputs": [
+        {
+          "name": "",
+          "type": "*",
+          "link": 150
+        }
+      ],
+      "outputs": [
+        {
+          "name": "",
+          "type": "VAE",
+          "slot_index": 0,
+          "links": [
+            188
+          ]
+        }
+      ],
+      "properties": {
+        "showOutputText": false,
+        "horizontal": false
+      }
+    },
+    {
+      "id": 87,
+      "type": "Reroute",
+      "pos": [
+        85,
+        253
+      ],
+      "size": [
+        75,
+        26
+      ],
+      "flags": {},
+      "order": 9,
+      "mode": 0,
+      "inputs": [
+        {
+          "name": "",
+          "type": "*",
+          "widget": {
+            "name": "value"
+          },
+          "link": 227
+        }
+      ],
+      "outputs": [
+        {
+          "name": "",
+          "type": "STRING",
+          "slot_index": 0,
+          "links": [
+            228,
+            229
+          ]
+        }
+      ],
+      "properties": {
+        "showOutputText": false,
+        "horizontal": false
+      }
+    },
+    {
+      "id": 92,
+      "type": "EmptyLatentImage",
+      "pos": [
+        1001,
+        1057
+      ],
+      "size": [
+        315,
+        106
+      ],
+      "flags": {},
+      "order": 11,
+      "mode": 0,
+      "inputs": [
+        {
+          "name": "width",
+          "type": "INT",
+          "widget": {
+            "name": "width"
+          },
+          "link": 241
+        },
+        {
+          "name": "height",
+          "type": "INT",
+          "widget": {
+            "name": "height"
+          },
+          "link": 242
+        }
+      ],
+      "outputs": [
+        {
+          "name": "LATENT",
+          "shape": 3,
+          "type": "LATENT",
+          "slot_index": 0,
+          "links": [
+            243
+          ]
+        }
+      ],
+      "properties": {
+        "cnr_id": "comfy-core",
+        "ver": "0.3.27",
+        "Node name for S&R": "EmptyLatentImage"
+      },
+      "widgets_values": [
+        512,
+        512,
+        1
+      ]
+    },
+    {
+      "id": 86,
+      "type": "Vyro Param Extractor",
+      "pos": [
+        -379,
+        228
+      ],
+      "size": [
+        418.1999816894531,
+        466
+      ],
+      "flags": {},
+      "order": 8,
+      "mode": 0,
+      "inputs": [
+        {
+          "name": "vyro_params",
+          "type": "VYRO_PARAMS",
+          "link": 226
+        }
+      ],
+      "outputs": [
+        {
+          "name": "latents",
+          "shape": 3,
+          "type": "LATENT",
+          "slot_index": 0,
+          "links": null
+        },
+        {
+          "name": "user_prompt",
+          "shape": 3,
+          "type": "STRING",
+          "slot_index": 1,
+          "links": [
+            227
+          ]
+        },
+        {
+          "name": "user_neg_prompt",
+          "shape": 3,
+          "type": "STRING",
+          "slot_index": 2,
+          "links": [
+            238
+          ]
+        },
+        {
+          "name": "mode",
+          "shape": 3,
+          "type": "STRING",
+          "links": null
+        },
+        {
+          "name": "cfg",
+          "shape": 3,
+          "type": "FLOAT",
+          "links": null
+        },
+        {
+          "name": "batch_size",
+          "shape": 3,
+          "type": "INT",
+          "links": null
+        },
+        {
+          "name": "steps",
+          "shape": 3,
+          "type": "INT",
+          "links": null
+        },
+        {
+          "name": "width",
+          "shape": 3,
+          "type": "INT",
+          "slot_index": 7,
+          "links": [
+            241
+          ]
+        },
+        {
+          "name": "height",
+          "shape": 3,
+          "type": "INT",
+          "slot_index": 8,
+          "links": [
+            242
+          ]
+        },
+        {
+          "name": "seed",
+          "shape": 3,
+          "type": "INT",
+          "links": [
+            247,
+            248
+          ]
+        },
+        {
+          "name": "denoise",
+          "shape": 3,
+          "type": "FLOAT",
+          "links": null
+        },
+        {
+          "name": "stage1_strength",
+          "shape": 3,
+          "type": "FLOAT",
+          "links": null
+        },
+        {
+          "name": "stage2_strength",
+          "shape": 3,
+          "type": "FLOAT",
+          "links": null
+        },
+        {
+          "name": "efficiency_multiplier",
+          "shape": 3,
+          "type": "FLOAT",
+          "links": null
+        },
+        {
+          "name": "style",
+          "shape": 3,
+          "type": "STRING",
+          "links": null
+        },
+        {
+          "name": "final_positive_prompt",
+          "shape": 3,
+          "type": "STRING",
+          "links": null
+        },
+        {
+          "name": "final_negative_prompt",
+          "shape": 3,
+          "type": "STRING",
+          "links": null
+        },
+        {
+          "name": "is_raw",
+          "shape": 3,
+          "type": "BOOLEAN",
+          "links": null
+        },
+        {
+          "name": "final_negative_prompt",
+          "shape": 3,
+          "type": "STRING",
+          "links": null
+        },
+        {
+          "name": "is_raw",
+          "shape": 3,
+          "type": "BOOLEAN",
+          "links": null
+        },
+        {
+          "name": "face_swap_img",
+          "shape": 3,
+          "type": "IMAGE",
+          "links": null
+        },
+        {
+          "name": "image_prompt_weights",
+          "shape": 3,
+          "type": "STRING",
+          "links": null
+        },
+        {
+          "name": "control_net_input_img",
+          "shape": 3,
+          "type": "IMAGE",
+          "links": null
+        }
+      ],
+      "properties": {
+        "aux_id": "Vyro-ai/vyro-workflows",
+        "ver": "987bd627ca63ee6815b42082eccf1b2199bf53ed",
+        "Node name for S&R": "Vyro Param Extractor"
+      },
+      "widgets_values": []
+    },
+    {
+      "id": 89,
+      "type": "Reroute",
+      "pos": [
+        141,
+        329
+      ],
+      "size": [
+        75,
+        26
+      ],
+      "flags": {},
+      "order": 10,
+      "mode": 0,
+      "inputs": [
+        {
+          "name": "",
+          "type": "*",
+          "widget": {
+            "name": "value"
+          },
+          "link": 238
+        }
+      ],
+      "outputs": [
+        {
+          "name": "",
+          "type": "STRING",
+          "slot_index": 0,
+          "links": [
+            239,
+            240
+          ]
+        }
+      ],
+      "properties": {
+        "showOutputText": false,
+        "horizontal": false
+      }
+    },
+    {
+      "id": 30,
+      "type": "CLIPTextEncodeSDXL",
+      "pos": [
+        1009,
+        728
+      ],
+      "size": [
+        400,
+        270.0000305175781
+      ],
+      "flags": {},
+      "order": 13,
+      "mode": 0,
+      "inputs": [
+        {
+          "name": "clip",
+          "type": "CLIP",
+          "link": 84
+        },
+        {
+          "name": "text_g",
+          "type": "STRING",
+          "widget": {
+            "name": "text_g"
+          },
+          "link": 239
+        },
+        {
+          "name": "text_l",
+          "type": "STRING",
+          "widget": {
+            "name": "text_l"
+          },
+          "link": 240
+        },
+        {
+          "name": "width",
+          "type": "INT",
+          "widget": {
+            "name": "width"
+          },
+          "link": 234
+        },
+        {
+          "name": "height",
+          "type": "INT",
+          "widget": {
+            "name": "height"
+          },
+          "link": 235
+        },
+        {
+          "name": "target_width",
+          "type": "INT",
+          "widget": {
+            "name": "target_width"
+          },
+          "link": 236
+        },
+        {
+          "name": "target_height",
+          "type": "INT",
+          "widget": {
+            "name": "target_height"
+          },
+          "link": 237
+        }
+      ],
+      "outputs": [
+        {
+          "name": "CONDITIONING",
+          "shape": 3,
+          "type": "CONDITIONING",
+          "slot_index": 0,
+          "links": [
+            159,
+            163
+          ]
+        }
+      ],
+      "title": "negativePromt_sdxl1Base",
+      "properties": {
+        "cnr_id": "comfy-core",
+        "ver": "0.3.27",
+        "Node name for S&R": "CLIPTextEncodeSDXL"
+      },
+      "widgets_values": [
+        4096,
+        4096,
+        0,
+        0,
+        4096,
+        4096,
+        "blurry, bokeh,",
+        "blurry, bokeh,"
+      ],
+      "color": "#322",
+      "bgcolor": "#533"
+    },
+    {
+      "id": 29,
+      "type": "CLIPTextEncodeSDXL",
+      "pos": [
+        1001,
+        383
+      ],
+      "size": [
+        400,
+        270.0000305175781
+      ],
+      "flags": {},
+      "order": 12,
+      "mode": 0,
+      "inputs": [
+        {
+          "name": "clip",
+          "type": "CLIP",
+          "link": 83
+        },
+        {
+          "name": "text_g",
+          "type": "STRING",
+          "widget": {
+            "name": "text_g"
+          },
+          "link": 228
+        },
+        {
+          "name": "text_l",
+          "type": "STRING",
+          "widget": {
+            "name": "text_l"
+          },
+          "link": 229
+        },
+        {
+          "name": "width",
+          "type": "INT",
+          "widget": {
+            "name": "width"
+          },
+          "link": 230
+        },
+        {
+          "name": "height",
+          "type": "INT",
+          "widget": {
+            "name": "height"
+          },
+          "link": 231
+        },
+        {
+          "name": "target_width",
+          "type": "INT",
+          "widget": {
+            "name": "target_width"
+          },
+          "link": 232
+        },
+        {
+          "name": "target_height",
+          "type": "INT",
+          "widget": {
+            "name": "target_height"
+          },
+          "link": 233
+        }
+      ],
+      "outputs": [
+        {
+          "name": "CONDITIONING",
+          "shape": 3,
+          "type": "CONDITIONING",
+          "slot_index": 0,
+          "links": [
+            158,
+            162
+          ]
+        }
+      ],
+      "title": "positivePromt_sdxl1Base",
+      "properties": {
+        "cnr_id": "comfy-core",
+        "ver": "0.3.27",
+        "Node name for S&R": "CLIPTextEncodeSDXL"
+      },
+      "widgets_values": [
+        4096,
+        4096,
+        0,
+        0,
+        4096,
+        4096,
+        "photo of beautiful 24 years woman, closeup, peach fuzz, skin pores, teal punk rocker hair style, sitting on a couch",
+        "photo of beautiful 24 years woman, closeup, peach fuzz, skin pores, teal punk rocker hair style, sitting on a couch"
+      ],
+      "color": "#232",
+      "bgcolor": "#353"
+    },
+    {
+      "id": 67,
+      "type": "KSamplerAdvanced",
+      "pos": [
+        1571,
+        209
+      ],
+      "size": [
+        315,
+        546
+      ],
+      "flags": {
+        "collapsed": false
+      },
+      "order": 14,
+      "mode": 0,
+      "inputs": [
+        {
+          "name": "model",
+          "type": "MODEL",
+          "link": 156
+        },
+        {
+          "name": "positive",
+          "type": "CONDITIONING",
+          "link": 158
+        },
+        {
+          "name": "negative",
+          "type": "CONDITIONING",
+          "link": 159
+        },
+        {
+          "name": "latent_image",
+          "type": "LATENT",
+          "link": 243
+        },
+        {
+          "name": "noise_seed",
+          "type": "INT",
+          "widget": {
+            "name": "noise_seed"
+          },
+          "link": 247
+        }
+      ],
+      "outputs": [
+        {
+          "name": "LATENT",
+          "shape": 3,
+          "type": "LATENT",
+          "slot_index": 0,
+          "links": [
+            261
+          ]
+        }
+      ],
+      "properties": {
+        "cnr_id": "comfy-core",
+        "ver": "0.3.27",
+        "Node name for S&R": "KSamplerAdvanced"
+      },
+      "widgets_values": [
+        "enable",
+        987654321357988,
+        "increment",
+        6,
+        1.1,
+        "dpmpp_2m_sde",
+        "karras",
+        0,
+        12,
+        "enable"
+      ]
+    },
+    {
+      "id": 68,
+      "type": "KSamplerAdvanced",
+      "pos": [
+        2263,
+        212
+      ],
+      "size": [
+        315,
+        546
+      ],
+      "flags": {},
+      "order": 16,
+      "mode": 0,
+      "inputs": [
+        {
+          "name": "model",
+          "type": "MODEL",
+          "link": 161
+        },
+        {
+          "name": "positive",
+          "type": "CONDITIONING",
+          "link": 162
+        },
+        {
+          "name": "negative",
+          "type": "CONDITIONING",
+          "link": 163
+        },
+        {
+          "name": "latent_image",
+          "type": "LATENT",
+          "link": 246
+        },
+        {
+          "name": "noise_seed",
+          "type": "INT",
+          "widget": {
+            "name": "noise_seed"
+          },
+          "link": 248
+        }
+      ],
+      "outputs": [
+        {
+          "name": "LATENT",
+          "shape": 3,
+          "type": "LATENT",
+          "slot_index": 0,
+          "links": [
+            189
+          ]
+        }
+      ],
+      "properties": {
+        "cnr_id": "comfy-core",
+        "ver": "0.3.27",
+        "Node name for S&R": "KSamplerAdvanced"
+      },
+      "widgets_values": [
+        "enable",
+        987654321357988,
+        "increment",
+        12,
+        1.1,
+        "dpmpp_2m_sde",
+        "karras",
+        5,
+        10000,
+        "disable"
+      ]
+    },
+    {
+      "id": 88,
+      "type": "PrimitiveNode",
+      "pos": [
+        655,
+        488
+      ],
+      "size": [
+        210,
+        82
+      ],
+      "flags": {},
+      "order": 2,
+      "mode": 0,
+      "inputs": [],
+      "outputs": [
+        {
+          "name": "INT",
+          "type": "INT",
+          "widget": {
+            "name": "width"
+          },
+          "slot_index": 0,
+          "links": [
+            230,
+            231,
+            232,
+            233,
+            234,
+            235,
+            236,
+            237
+          ]
+        }
+      ],
+      "properties": {
+        "Run widget replace on values": false
+      },
+      "widgets_values": [
+        4096,
+        "fixed"
+      ]
+    },
+    {
+      "id": 93,
+      "type": "LatentUpscaleBy",
+      "pos": [
+        1920,
+        87
+      ],
+      "size": [
+        315,
+        82
+      ],
+      "flags": {},
+      "order": 15,
+      "mode": 0,
+      "inputs": [
+        {
+          "name": "samples",
+          "type": "LATENT",
+          "link": 261
+        }
+      ],
+      "outputs": [
+        {
+          "name": "LATENT",
+          "shape": 3,
+          "type": "LATENT",
+          "slot_index": 0,
+          "links": [
+            246
+          ]
+        }
+      ],
+      "properties": {
+        "cnr_id": "comfy-core",
+        "ver": "0.3.27",
+        "Node name for S&R": "LatentUpscaleBy"
+      },
+      "widgets_values": [
+        "nearest-exact",
+        2
+      ]
+    },
+    {
+      "id": 46,
+      "type": "Reroute",
+      "pos": [
+        840,
+        10
+      ],
+      "size": [
+        75,
+        26
+      ],
+      "flags": {},
+      "order": 4,
+      "mode": 0,
+      "inputs": [
+        {
+          "name": "",
+          "type": "*",
+          "link": 262
+        }
+      ],
+      "outputs": [
+        {
+          "name": "",
+          "type": "MODEL",
+          "slot_index": 0,
+          "links": [
+            156,
+            161
+          ]
+        }
+      ],
+      "properties": {
+        "showOutputText": false,
+        "horizontal": false
+      }
+    },
+    {
+      "id": 20,
+      "type": "CheckpointLoaderSimple",
+      "pos": [
+        -474,
+        -165
+      ],
+      "size": [
+        645.7987060546875,
+        98
+      ],
+      "flags": {},
+      "order": 3,
+      "mode": 0,
+      "inputs": [],
+      "outputs": [
+        {
+          "name": "MODEL",
+          "shape": 3,
+          "type": "MODEL",
+          "slot_index": 0,
+          "links": [
+            262
+          ]
+        },
+        {
+          "name": "CLIP",
+          "shape": 3,
+          "type": "CLIP",
+          "slot_index": 1,
+          "links": [
+            82
+          ]
+        },
+        {
+          "name": "VAE",
+          "shape": 3,
+          "type": "VAE",
+          "slot_index": 2,
+          "links": [
+            150,
+            244
+          ]
+        }
+      ],
+      "properties": {
+        "cnr_id": "comfy-core",
+        "ver": "0.3.27",
+        "Node name for S&R": "CheckpointLoaderSimple"
+      },
+      "widgets_values": [
+        "turbovisionxlSuperFastXLBasedOnNew_alphaV0101Bakedvae.safetensors"
+      ]
+    },
+    {
+      "id": 77,
+      "type": "VAEDecode",
+      "pos": [
+        2645.012939453125,
+        381.77679443359375
+      ],
+      "size": [
+        210,
+        46
+      ],
+      "flags": {
+        "collapsed": true
+      },
+      "order": 17,
+      "mode": 0,
+      "inputs": [
+        {
+          "name": "samples",
+          "type": "LATENT",
+          "link": 189
+        },
+        {
+          "name": "vae",
+          "type": "VAE",
+          "link": 188
+        }
+      ],
+      "outputs": [
+        {
+          "name": "IMAGE",
+          "shape": 3,
+          "type": "IMAGE",
+          "slot_index": 0,
+          "links": [
+            190,
+            263
+          ]
+        }
+      ],
+      "properties": {
+        "cnr_id": "comfy-core",
+        "ver": "0.3.27",
+        "Node name for S&R": "VAEDecode"
+      },
+      "widgets_values": []
+    },
+    {
+      "id": 99,
+      "type": "PreviewImage",
+      "pos": [
+        2965.96533203125,
+        326.4072570800781
+      ],
+      "size": [
+        584.3117065429688,
+        613.8179931640625
+      ],
+      "flags": {},
+      "order": 18,
+      "mode": 0,
+      "inputs": [
+        {
+          "name": "images",
+          "type": "IMAGE",
+          "link": 263
+        }
+      ],
+      "outputs": [],
+      "properties": {
+        "cnr_id": "comfy-core",
+        "ver": "0.3.27",
+        "Node name for S&R": "PreviewImage"
+      },
+      "widgets_values": [
+        ""
+      ]
+    },
+    {
+      "id": 85,
+      "type": "Vyro Pipe Input V2",
+      "pos": [
+        -811,
+        200
+      ],
+      "size": [
+        400,
+        748
+      ],
+      "flags": {},
+      "order": 7,
+      "mode": 0,
+      "inputs": [
+        {
+          "name": "vae",
+          "type": "VAE",
+          "link": 244
+        }
+      ],
+      "outputs": [
+        {
+          "name": "vyro_params",
+          "shape": 3,
+          "type": "VYRO_PARAMS",
+          "slot_index": 0,
+          "links": [
+            226
+          ]
+        }
+      ],
+      "properties": {
+        "aux_id": "Vyro-ai/vyro-workflows",
+        "ver": "987bd627ca63ee6815b42082eccf1b2199bf53ed",
+        "Node name for S&R": "Vyro Pipe Input V2"
+      },
+      "widgets_values": [
+        "digital drawing of cyberpunk skull with armor, maximalist detailing, colorful, vibrant, --ar 9:16 --chaos 30",
+        "t2i",
+        "perlin1",
+        "trees",
+        1,
+        7.5,
+        10,
+        512,
+        512,
+        3465171079,
+        "randomize",
+        "",
+        1,
+        0.25000000000000006,
+        1,
+        1
+      ]
+    }
+  ],
+  "links": [
+    [
+      82,
+      20,
+      1,
+      45,
+      0,
+      "*"
+    ],
+    [
+      83,
+      45,
+      0,
+      29,
+      0,
+      "CLIP"
+    ],
+    [
+      84,
+      45,
+      0,
+      30,
+      0,
+      "CLIP"
+    ],
+    [
+      150,
+      20,
+      2,
+      66,
+      0,
+      "*"
+    ],
+    [
+      156,
+      46,
+      0,
+      67,
+      0,
+      "MODEL"
+    ],
+    [
+      158,
+      29,
+      0,
+      67,
+      1,
+      "CONDITIONING"
+    ],
+    [
+      159,
+      30,
+      0,
+      67,
+      2,
+      "CONDITIONING"
+    ],
+    [
+      161,
+      46,
+      0,
+      68,
+      0,
+      "MODEL"
+    ],
+    [
+      162,
+      29,
+      0,
+      68,
+      1,
+      "CONDITIONING"
+    ],
+    [
+      163,
+      30,
+      0,
+      68,
+      2,
+      "CONDITIONING"
+    ],
+    [
+      188,
+      66,
+      0,
+      77,
+      1,
+      "VAE"
+    ],
+    [
+      189,
+      68,
+      0,
+      77,
+      0,
+      "LATENT"
+    ],
+    [
+      226,
+      85,
+      0,
+      86,
+      0,
+      "VYRO_PARAMS"
+    ],
+    [
+      227,
+      86,
+      1,
+      87,
+      0,
+      "*"
+    ],
+    [
+      228,
+      87,
+      0,
+      29,
+      1,
+      "STRING"
+    ],
+    [
+      229,
+      87,
+      0,
+      29,
+      2,
+      "STRING"
+    ],
+    [
+      230,
+      88,
+      0,
+      29,
+      3,
+      "INT"
+    ],
+    [
+      231,
+      88,
+      0,
+      29,
+      4,
+      "INT"
+    ],
+    [
+      232,
+      88,
+      0,
+      29,
+      5,
+      "INT"
+    ],
+    [
+      233,
+      88,
+      0,
+      29,
+      6,
+      "INT"
+    ],
+    [
+      234,
+      88,
+      0,
+      30,
+      3,
+      "INT"
+    ],
+    [
+      235,
+      88,
+      0,
+      30,
+      4,
+      "INT"
+    ],
+    [
+      236,
+      88,
+      0,
+      30,
+      5,
+      "INT"
+    ],
+    [
+      237,
+      88,
+      0,
+      30,
+      6,
+      "INT"
+    ],
+    [
+      238,
+      86,
+      2,
+      89,
+      0,
+      "*"
+    ],
+    [
+      239,
+      89,
+      0,
+      30,
+      1,
+      "STRING"
+    ],
+    [
+      240,
+      89,
+      0,
+      30,
+      2,
+      "STRING"
+    ],
+    [
+      241,
+      86,
+      7,
+      92,
+      0,
+      "INT"
+    ],
+    [
+      242,
+      86,
+      8,
+      92,
+      1,
+      "INT"
+    ],
+    [
+      243,
+      92,
+      0,
+      67,
+      3,
+      "LATENT"
+    ],
+    [
+      244,
+      20,
+      2,
+      85,
+      0,
+      "VAE"
+    ],
+    [
+      246,
+      93,
+      0,
+      68,
+      3,
+      "LATENT"
+    ],
+    [
+      247,
+      86,
+      9,
+      67,
+      4,
+      "INT"
+    ],
+    [
+      248,
+      86,
+      9,
+      68,
+      4,
+      "INT"
+    ],
+    [
+      261,
+      67,
+      0,
+      93,
+      0,
+      "LATENT"
+    ],
+    [
+      262,
+      20,
+      0,
+      46,
+      0,
+      "*"
+    ],
+    [
+      263,
+      77,
+      0,
+      99,
+      0,
+      "IMAGE"
+    ]
+  ],
+  "groups": [],
+  "config": {},
+  "extra": {
+    "ds": {
+      "scale": 0.15030096025614706,
+      "offset": [
+        6414.9972594019255,
+        3091.1902356429177
+      ]
+    }
+  },
+  "version": 0.4
+}

Imagine/imagine-v5-ultra/comfy/checkpoint_pickle.py

@@ -0,0 +1,13 @@
+import pickle
+
+load = pickle.load
+
+class Empty:
+    pass
+
+class Unpickler(pickle.Unpickler):
+    def find_class(self, module, name):
+        #TODO: safe unpickle
+        if module.startswith("pytorch_lightning"):
+            return Empty
+        return super().find_class(module, name)

Imagine/imagine-v5-ultra/comfy/cldm/cldm.py

@@ -0,0 +1,433 @@
+#taken from: https://github.com/lllyasviel/ControlNet
+#and modified
+
+import torch
+import torch.nn as nn
+
+from ..ldm.modules.diffusionmodules.util import (
+    timestep_embedding,
+)
+
+from ..ldm.modules.attention import SpatialTransformer
+from ..ldm.modules.diffusionmodules.openaimodel import UNetModel, TimestepEmbedSequential, ResBlock, Downsample
+from ..ldm.util import exists
+from .control_types import UNION_CONTROLNET_TYPES
+from collections import OrderedDict
+import comfy.ops
+from comfy.ldm.modules.attention import optimized_attention
+
+class OptimizedAttention(nn.Module):
+    def __init__(self, c, nhead, dropout=0.0, dtype=None, device=None, operations=None):
+        super().__init__()
+        self.heads = nhead
+        self.c = c
+
+        self.in_proj = operations.Linear(c, c * 3, bias=True, dtype=dtype, device=device)
+        self.out_proj = operations.Linear(c, c, bias=True, dtype=dtype, device=device)
+
+    def forward(self, x):
+        x = self.in_proj(x)
+        q, k, v = x.split(self.c, dim=2)
+        out = optimized_attention(q, k, v, self.heads)
+        return self.out_proj(out)
+
+class QuickGELU(nn.Module):
+    def forward(self, x: torch.Tensor):
+        return x * torch.sigmoid(1.702 * x)
+
+class ResBlockUnionControlnet(nn.Module):
+    def __init__(self, dim, nhead, dtype=None, device=None, operations=None):
+        super().__init__()
+        self.attn = OptimizedAttention(dim, nhead, dtype=dtype, device=device, operations=operations)
+        self.ln_1 = operations.LayerNorm(dim, dtype=dtype, device=device)
+        self.mlp = nn.Sequential(
+            OrderedDict([("c_fc", operations.Linear(dim, dim * 4, dtype=dtype, device=device)), ("gelu", QuickGELU()),
+                         ("c_proj", operations.Linear(dim * 4, dim, dtype=dtype, device=device))]))
+        self.ln_2 = operations.LayerNorm(dim, dtype=dtype, device=device)
+
+    def attention(self, x: torch.Tensor):
+        return self.attn(x)
+
+    def forward(self, x: torch.Tensor):
+        x = x + self.attention(self.ln_1(x))
+        x = x + self.mlp(self.ln_2(x))
+        return x
+
+class ControlledUnetModel(UNetModel):
+    #implemented in the ldm unet
+    pass
+
+class ControlNet(nn.Module):
+    def __init__(
+        self,
+        image_size,
+        in_channels,
+        model_channels,
+        hint_channels,
+        num_res_blocks,
+        dropout=0,
+        channel_mult=(1, 2, 4, 8),
+        conv_resample=True,
+        dims=2,
+        num_classes=None,
+        use_checkpoint=False,
+        dtype=torch.float32,
+        num_heads=-1,
+        num_head_channels=-1,
+        num_heads_upsample=-1,
+        use_scale_shift_norm=False,
+        resblock_updown=False,
+        use_new_attention_order=False,
+        use_spatial_transformer=False,    # custom transformer support
+        transformer_depth=1,              # custom transformer support
+        context_dim=None,                 # custom transformer support
+        n_embed=None,                     # custom support for prediction of discrete ids into codebook of first stage vq model
+        legacy=True,
+        disable_self_attentions=None,
+        num_attention_blocks=None,
+        disable_middle_self_attn=False,
+        use_linear_in_transformer=False,
+        adm_in_channels=None,
+        transformer_depth_middle=None,
+        transformer_depth_output=None,
+        attn_precision=None,
+        union_controlnet_num_control_type=None,
+        device=None,
+        operations=comfy.ops.disable_weight_init,
+        **kwargs,
+    ):
+        super().__init__()
+        assert use_spatial_transformer == True, "use_spatial_transformer has to be true"
+        if use_spatial_transformer:
+            assert context_dim is not None, 'Fool!! You forgot to include the dimension of your cross-attention conditioning...'
+
+        if context_dim is not None:
+            assert use_spatial_transformer, 'Fool!! You forgot to use the spatial transformer for your cross-attention conditioning...'
+            # from omegaconf.listconfig import ListConfig
+            # if type(context_dim) == ListConfig:
+            #     context_dim = list(context_dim)
+
+        if num_heads_upsample == -1:
+            num_heads_upsample = num_heads
+
+        if num_heads == -1:
+            assert num_head_channels != -1, 'Either num_heads or num_head_channels has to be set'
+
+        if num_head_channels == -1:
+            assert num_heads != -1, 'Either num_heads or num_head_channels has to be set'
+
+        self.dims = dims
+        self.image_size = image_size
+        self.in_channels = in_channels
+        self.model_channels = model_channels
+
+        if isinstance(num_res_blocks, int):
+            self.num_res_blocks = len(channel_mult) * [num_res_blocks]
+        else:
+            if len(num_res_blocks) != len(channel_mult):
+                raise ValueError("provide num_res_blocks either as an int (globally constant) or "
+                                 "as a list/tuple (per-level) with the same length as channel_mult")
+            self.num_res_blocks = num_res_blocks
+
+        if disable_self_attentions is not None:
+            # should be a list of booleans, indicating whether to disable self-attention in TransformerBlocks or not
+            assert len(disable_self_attentions) == len(channel_mult)
+        if num_attention_blocks is not None:
+            assert len(num_attention_blocks) == len(self.num_res_blocks)
+            assert all(map(lambda i: self.num_res_blocks[i] >= num_attention_blocks[i], range(len(num_attention_blocks))))
+
+        transformer_depth = transformer_depth[:]
+
+        self.dropout = dropout
+        self.channel_mult = channel_mult
+        self.conv_resample = conv_resample
+        self.num_classes = num_classes
+        self.use_checkpoint = use_checkpoint
+        self.dtype = dtype
+        self.num_heads = num_heads
+        self.num_head_channels = num_head_channels
+        self.num_heads_upsample = num_heads_upsample
+        self.predict_codebook_ids = n_embed is not None
+
+        time_embed_dim = model_channels * 4
+        self.time_embed = nn.Sequential(
+            operations.Linear(model_channels, time_embed_dim, dtype=self.dtype, device=device),
+            nn.SiLU(),
+            operations.Linear(time_embed_dim, time_embed_dim, dtype=self.dtype, device=device),
+        )
+
+        if self.num_classes is not None:
+            if isinstance(self.num_classes, int):
+                self.label_emb = nn.Embedding(num_classes, time_embed_dim)
+            elif self.num_classes == "continuous":
+                self.label_emb = nn.Linear(1, time_embed_dim)
+            elif self.num_classes == "sequential":
+                assert adm_in_channels is not None
+                self.label_emb = nn.Sequential(
+                    nn.Sequential(
+                        operations.Linear(adm_in_channels, time_embed_dim, dtype=self.dtype, device=device),
+                        nn.SiLU(),
+                        operations.Linear(time_embed_dim, time_embed_dim, dtype=self.dtype, device=device),
+                    )
+                )
+            else:
+                raise ValueError()
+
+        self.input_blocks = nn.ModuleList(
+            [
+                TimestepEmbedSequential(
+                    operations.conv_nd(dims, in_channels, model_channels, 3, padding=1, dtype=self.dtype, device=device)
+                )
+            ]
+        )
+        self.zero_convs = nn.ModuleList([self.make_zero_conv(model_channels, operations=operations, dtype=self.dtype, device=device)])
+
+        self.input_hint_block = TimestepEmbedSequential(
+                    operations.conv_nd(dims, hint_channels, 16, 3, padding=1, dtype=self.dtype, device=device),
+                    nn.SiLU(),
+                    operations.conv_nd(dims, 16, 16, 3, padding=1, dtype=self.dtype, device=device),
+                    nn.SiLU(),
+                    operations.conv_nd(dims, 16, 32, 3, padding=1, stride=2, dtype=self.dtype, device=device),
+                    nn.SiLU(),
+                    operations.conv_nd(dims, 32, 32, 3, padding=1, dtype=self.dtype, device=device),
+                    nn.SiLU(),
+                    operations.conv_nd(dims, 32, 96, 3, padding=1, stride=2, dtype=self.dtype, device=device),
+                    nn.SiLU(),
+                    operations.conv_nd(dims, 96, 96, 3, padding=1, dtype=self.dtype, device=device),
+                    nn.SiLU(),
+                    operations.conv_nd(dims, 96, 256, 3, padding=1, stride=2, dtype=self.dtype, device=device),
+                    nn.SiLU(),
+                    operations.conv_nd(dims, 256, model_channels, 3, padding=1, dtype=self.dtype, device=device)
+        )
+
+        self._feature_size = model_channels
+        input_block_chans = [model_channels]
+        ch = model_channels
+        ds = 1
+        for level, mult in enumerate(channel_mult):
+            for nr in range(self.num_res_blocks[level]):
+                layers = [
+                    ResBlock(
+                        ch,
+                        time_embed_dim,
+                        dropout,
+                        out_channels=mult * model_channels,
+                        dims=dims,
+                        use_checkpoint=use_checkpoint,
+                        use_scale_shift_norm=use_scale_shift_norm,
+                        dtype=self.dtype,
+                        device=device,
+                        operations=operations,
+                    )
+                ]
+                ch = mult * model_channels
+                num_transformers = transformer_depth.pop(0)
+                if num_transformers > 0:
+                    if num_head_channels == -1:
+                        dim_head = ch // num_heads
+                    else:
+                        num_heads = ch // num_head_channels
+                        dim_head = num_head_channels
+                    if legacy:
+                        #num_heads = 1
+                        dim_head = ch // num_heads if use_spatial_transformer else num_head_channels
+                    if exists(disable_self_attentions):
+                        disabled_sa = disable_self_attentions[level]
+                    else:
+                        disabled_sa = False
+
+                    if not exists(num_attention_blocks) or nr < num_attention_blocks[level]:
+                        layers.append(
+                            SpatialTransformer(
+                                ch, num_heads, dim_head, depth=num_transformers, context_dim=context_dim,
+                                disable_self_attn=disabled_sa, use_linear=use_linear_in_transformer,
+                                use_checkpoint=use_checkpoint, attn_precision=attn_precision, dtype=self.dtype, device=device, operations=operations
+                            )
+                        )
+                self.input_blocks.append(TimestepEmbedSequential(*layers))
+                self.zero_convs.append(self.make_zero_conv(ch, operations=operations, dtype=self.dtype, device=device))
+                self._feature_size += ch
+                input_block_chans.append(ch)
+            if level != len(channel_mult) - 1:
+                out_ch = ch
+                self.input_blocks.append(
+                    TimestepEmbedSequential(
+                        ResBlock(
+                            ch,
+                            time_embed_dim,
+                            dropout,
+                            out_channels=out_ch,
+                            dims=dims,
+                            use_checkpoint=use_checkpoint,
+                            use_scale_shift_norm=use_scale_shift_norm,
+                            down=True,
+                            dtype=self.dtype,
+                            device=device,
+                            operations=operations
+                        )
+                        if resblock_updown
+                        else Downsample(
+                            ch, conv_resample, dims=dims, out_channels=out_ch, dtype=self.dtype, device=device, operations=operations
+                        )
+                    )
+                )
+                ch = out_ch
+                input_block_chans.append(ch)
+                self.zero_convs.append(self.make_zero_conv(ch, operations=operations, dtype=self.dtype, device=device))
+                ds *= 2
+                self._feature_size += ch
+
+        if num_head_channels == -1:
+            dim_head = ch // num_heads
+        else:
+            num_heads = ch // num_head_channels
+            dim_head = num_head_channels
+        if legacy:
+            #num_heads = 1
+            dim_head = ch // num_heads if use_spatial_transformer else num_head_channels
+        mid_block = [
+            ResBlock(
+                ch,
+                time_embed_dim,
+                dropout,
+                dims=dims,
+                use_checkpoint=use_checkpoint,
+                use_scale_shift_norm=use_scale_shift_norm,
+                dtype=self.dtype,
+                device=device,
+                operations=operations
+            )]
+        if transformer_depth_middle >= 0:
+            mid_block += [SpatialTransformer(  # always uses a self-attn
+                            ch, num_heads, dim_head, depth=transformer_depth_middle, context_dim=context_dim,
+                            disable_self_attn=disable_middle_self_attn, use_linear=use_linear_in_transformer,
+                            use_checkpoint=use_checkpoint, attn_precision=attn_precision, dtype=self.dtype, device=device, operations=operations
+                        ),
+            ResBlock(
+                ch,
+                time_embed_dim,
+                dropout,
+                dims=dims,
+                use_checkpoint=use_checkpoint,
+                use_scale_shift_norm=use_scale_shift_norm,
+                dtype=self.dtype,
+                device=device,
+                operations=operations
+            )]
+        self.middle_block = TimestepEmbedSequential(*mid_block)
+        self.middle_block_out = self.make_zero_conv(ch, operations=operations, dtype=self.dtype, device=device)
+        self._feature_size += ch
+
+        if union_controlnet_num_control_type is not None:
+            self.num_control_type = union_controlnet_num_control_type
+            num_trans_channel = 320
+            num_trans_head = 8
+            num_trans_layer = 1
+            num_proj_channel = 320
+            # task_scale_factor = num_trans_channel ** 0.5
+            self.task_embedding = nn.Parameter(torch.empty(self.num_control_type, num_trans_channel, dtype=self.dtype, device=device))
+
+            self.transformer_layes = nn.Sequential(*[ResBlockUnionControlnet(num_trans_channel, num_trans_head, dtype=self.dtype, device=device, operations=operations) for _ in range(num_trans_layer)])
+            self.spatial_ch_projs = operations.Linear(num_trans_channel, num_proj_channel, dtype=self.dtype, device=device)
+            #-----------------------------------------------------------------------------------------------------
+
+            control_add_embed_dim = 256
+            class ControlAddEmbedding(nn.Module):
+                def __init__(self, in_dim, out_dim, num_control_type, dtype=None, device=None, operations=None):
+                    super().__init__()
+                    self.num_control_type = num_control_type
+                    self.in_dim = in_dim
+                    self.linear_1 = operations.Linear(in_dim * num_control_type, out_dim, dtype=dtype, device=device)
+                    self.linear_2 = operations.Linear(out_dim, out_dim, dtype=dtype, device=device)
+                def forward(self, control_type, dtype, device):
+                    c_type = torch.zeros((self.num_control_type,), device=device)
+                    c_type[control_type] = 1.0
+                    c_type = timestep_embedding(c_type.flatten(), self.in_dim, repeat_only=False).to(dtype).reshape((-1, self.num_control_type * self.in_dim))
+                    return self.linear_2(torch.nn.functional.silu(self.linear_1(c_type)))
+
+            self.control_add_embedding = ControlAddEmbedding(control_add_embed_dim, time_embed_dim, self.num_control_type, dtype=self.dtype, device=device, operations=operations)
+        else:
+            self.task_embedding = None
+            self.control_add_embedding = None
+
+    def union_controlnet_merge(self, hint, control_type, emb, context):
+        # Equivalent to: https://github.com/xinsir6/ControlNetPlus/tree/main
+        inputs = []
+        condition_list = []
+
+        for idx in range(min(1, len(control_type))):
+            controlnet_cond = self.input_hint_block(hint[idx], emb, context)
+            feat_seq = torch.mean(controlnet_cond, dim=(2, 3))
+            if idx < len(control_type):
+                feat_seq += self.task_embedding[control_type[idx]].to(dtype=feat_seq.dtype, device=feat_seq.device)
+
+            inputs.append(feat_seq.unsqueeze(1))
+            condition_list.append(controlnet_cond)
+
+        x = torch.cat(inputs, dim=1)
+        x = self.transformer_layes(x)
+        controlnet_cond_fuser = None
+        for idx in range(len(control_type)):
+            alpha = self.spatial_ch_projs(x[:, idx])
+            alpha = alpha.unsqueeze(-1).unsqueeze(-1)
+            o = condition_list[idx] + alpha
+            if controlnet_cond_fuser is None:
+                controlnet_cond_fuser = o
+            else:
+                controlnet_cond_fuser += o
+        return controlnet_cond_fuser
+
+    def make_zero_conv(self, channels, operations=None, dtype=None, device=None):
+        return TimestepEmbedSequential(operations.conv_nd(self.dims, channels, channels, 1, padding=0, dtype=dtype, device=device))
+
+    def forward(self, x, hint, timesteps, context, y=None, **kwargs):
+        t_emb = timestep_embedding(timesteps, self.model_channels, repeat_only=False).to(x.dtype)
+        emb = self.time_embed(t_emb)
+
+        guided_hint = None
+        if self.control_add_embedding is not None: #Union Controlnet
+            control_type = kwargs.get("control_type", [])
+
+            if any([c >= self.num_control_type for c in control_type]):
+                max_type = max(control_type)
+                max_type_name = {
+                    v: k for k, v in UNION_CONTROLNET_TYPES.items()
+                }[max_type]
+                raise ValueError(
+                    f"Control type {max_type_name}({max_type}) is out of range for the number of control types" +
+                    f"({self.num_control_type}) supported.\n" +
+                    "Please consider using the ProMax ControlNet Union model.\n" +
+                    "https://huggingface.co/xinsir/controlnet-union-sdxl-1.0/tree/main"
+                )
+
+            emb += self.control_add_embedding(control_type, emb.dtype, emb.device)
+            if len(control_type) > 0:
+                if len(hint.shape) < 5:
+                    hint = hint.unsqueeze(dim=0)
+                guided_hint = self.union_controlnet_merge(hint, control_type, emb, context)
+
+        if guided_hint is None:
+            guided_hint = self.input_hint_block(hint, emb, context)
+
+        out_output = []
+        out_middle = []
+
+        if self.num_classes is not None:
+            assert y.shape[0] == x.shape[0]
+            emb = emb + self.label_emb(y)
+
+        h = x
+        for module, zero_conv in zip(self.input_blocks, self.zero_convs):
+            if guided_hint is not None:
+                h = module(h, emb, context)
+                h += guided_hint
+                guided_hint = None
+            else:
+                h = module(h, emb, context)
+            out_output.append(zero_conv(h, emb, context))
+
+        h = self.middle_block(h, emb, context)
+        out_middle.append(self.middle_block_out(h, emb, context))
+
+        return {"middle": out_middle, "output": out_output}
+

Imagine/imagine-v5-ultra/comfy/cldm/control_types.py

@@ -0,0 +1,10 @@
+UNION_CONTROLNET_TYPES = {
+    "openpose": 0,
+    "depth": 1,
+    "hed/pidi/scribble/ted": 2,
+    "canny/lineart/anime_lineart/mlsd": 3,
+    "normal": 4,
+    "segment": 5,
+    "tile": 6,
+    "repaint": 7,
+}

Imagine/imagine-v5-ultra/comfy/cldm/dit_embedder.py

@@ -0,0 +1,120 @@
+import math
+from typing import List, Optional, Tuple
+
+import torch
+import torch.nn as nn
+from torch import Tensor
+
+from comfy.ldm.modules.diffusionmodules.mmdit import DismantledBlock, PatchEmbed, VectorEmbedder, TimestepEmbedder, get_2d_sincos_pos_embed_torch
+
+
+class ControlNetEmbedder(nn.Module):
+
+    def __init__(
+        self,
+        img_size: int,
+        patch_size: int,
+        in_chans: int,
+        attention_head_dim: int,
+        num_attention_heads: int,
+        adm_in_channels: int,
+        num_layers: int,
+        main_model_double: int,
+        double_y_emb: bool,
+        device: torch.device,
+        dtype: torch.dtype,
+        pos_embed_max_size: Optional[int] = None,
+        operations = None,
+    ):
+        super().__init__()
+        self.main_model_double = main_model_double
+        self.dtype = dtype
+        self.hidden_size = num_attention_heads * attention_head_dim
+        self.patch_size = patch_size
+        self.x_embedder = PatchEmbed(
+            img_size=img_size,
+            patch_size=patch_size,
+            in_chans=in_chans,
+            embed_dim=self.hidden_size,
+            strict_img_size=pos_embed_max_size is None,
+            device=device,
+            dtype=dtype,
+            operations=operations,
+        )
+
+        self.t_embedder = TimestepEmbedder(self.hidden_size, dtype=dtype, device=device, operations=operations)
+
+        self.double_y_emb = double_y_emb
+        if self.double_y_emb:
+            self.orig_y_embedder = VectorEmbedder(
+                adm_in_channels, self.hidden_size, dtype, device, operations=operations
+            )
+            self.y_embedder = VectorEmbedder(
+                self.hidden_size, self.hidden_size, dtype, device, operations=operations
+            )
+        else:
+            self.y_embedder = VectorEmbedder(
+                adm_in_channels, self.hidden_size, dtype, device, operations=operations
+            )
+
+        self.transformer_blocks = nn.ModuleList(
+            DismantledBlock(
+                hidden_size=self.hidden_size, num_heads=num_attention_heads, qkv_bias=True,
+                dtype=dtype, device=device, operations=operations
+            )
+            for _ in range(num_layers)
+        )
+
+        # self.use_y_embedder = pooled_projection_dim != self.time_text_embed.text_embedder.linear_1.in_features
+        # TODO double check this logic when 8b
+        self.use_y_embedder = True
+
+        self.controlnet_blocks = nn.ModuleList([])
+        for _ in range(len(self.transformer_blocks)):
+            controlnet_block = operations.Linear(self.hidden_size, self.hidden_size, dtype=dtype, device=device)
+            self.controlnet_blocks.append(controlnet_block)
+
+        self.pos_embed_input = PatchEmbed(
+            img_size=img_size,
+            patch_size=patch_size,
+            in_chans=in_chans,
+            embed_dim=self.hidden_size,
+            strict_img_size=False,
+            device=device,
+            dtype=dtype,
+            operations=operations,
+        )
+
+    def forward(
+        self,
+        x: torch.Tensor,
+        timesteps: torch.Tensor,
+        y: Optional[torch.Tensor] = None,
+        context: Optional[torch.Tensor] = None,
+        hint = None,
+    ) -> Tuple[Tensor, List[Tensor]]:
+        x_shape = list(x.shape)
+        x = self.x_embedder(x)
+        if not self.double_y_emb:
+            h = (x_shape[-2] + 1) // self.patch_size
+            w = (x_shape[-1] + 1) // self.patch_size
+            x += get_2d_sincos_pos_embed_torch(self.hidden_size, w, h, device=x.device)
+        c = self.t_embedder(timesteps, dtype=x.dtype)
+        if y is not None and self.y_embedder is not None:
+            if self.double_y_emb:
+                y = self.orig_y_embedder(y)
+            y = self.y_embedder(y)
+            c = c + y
+
+        x = x + self.pos_embed_input(hint)
+
+        block_out = ()
+
+        repeat = math.ceil(self.main_model_double / len(self.transformer_blocks))
+        for i in range(len(self.transformer_blocks)):
+            out = self.transformer_blocks[i](x, c)
+            if not self.double_y_emb:
+                x = out
+            block_out += (self.controlnet_blocks[i](out),) * repeat
+
+        return {"output": block_out}

Imagine/imagine-v5-ultra/comfy/cldm/mmdit.py

@@ -0,0 +1,81 @@
+import torch
+from typing import Optional
+import comfy.ldm.modules.diffusionmodules.mmdit
+
+class ControlNet(comfy.ldm.modules.diffusionmodules.mmdit.MMDiT):
+    def __init__(
+        self,
+        num_blocks = None,
+        control_latent_channels = None,
+        dtype = None,
+        device = None,
+        operations = None,
+        **kwargs,
+    ):
+        super().__init__(dtype=dtype, device=device, operations=operations, final_layer=False, num_blocks=num_blocks, **kwargs)
+        # controlnet_blocks
+        self.controlnet_blocks = torch.nn.ModuleList([])
+        for _ in range(len(self.joint_blocks)):
+            self.controlnet_blocks.append(operations.Linear(self.hidden_size, self.hidden_size, device=device, dtype=dtype))
+
+        if control_latent_channels is None:
+            control_latent_channels = self.in_channels
+
+        self.pos_embed_input = comfy.ldm.modules.diffusionmodules.mmdit.PatchEmbed(
+            None,
+            self.patch_size,
+            control_latent_channels,
+            self.hidden_size,
+            bias=True,
+            strict_img_size=False,
+            dtype=dtype,
+            device=device,
+            operations=operations
+        )
+
+    def forward(
+        self,
+        x: torch.Tensor,
+        timesteps: torch.Tensor,
+        y: Optional[torch.Tensor] = None,
+        context: Optional[torch.Tensor] = None,
+        hint = None,
+    ) -> torch.Tensor:
+
+        #weird sd3 controlnet specific stuff
+        y = torch.zeros_like(y)
+
+        if self.context_processor is not None:
+            context = self.context_processor(context)
+
+        hw = x.shape[-2:]
+        x = self.x_embedder(x) + self.cropped_pos_embed(hw, device=x.device).to(dtype=x.dtype, device=x.device)
+        x += self.pos_embed_input(hint)
+
+        c = self.t_embedder(timesteps, dtype=x.dtype)
+        if y is not None and self.y_embedder is not None:
+            y = self.y_embedder(y)
+            c = c + y
+
+        if context is not None:
+            context = self.context_embedder(context)
+
+        output = []
+
+        blocks = len(self.joint_blocks)
+        for i in range(blocks):
+            context, x = self.joint_blocks[i](
+                context,
+                x,
+                c=c,
+                use_checkpoint=self.use_checkpoint,
+            )
+
+            out = self.controlnet_blocks[i](x)
+            count = self.depth // blocks
+            if i == blocks - 1:
+                count -= 1
+            for j in range(count):
+                output.append(out)
+
+        return {"output": output}

Imagine/imagine-v5-ultra/comfy/cli_args.py

@@ -0,0 +1,214 @@
+import argparse
+import enum
+import os
+import comfy.options
+
+
+class EnumAction(argparse.Action):
+    """
+    Argparse action for handling Enums
+    """
+    def __init__(self, **kwargs):
+        # Pop off the type value
+        enum_type = kwargs.pop("type", None)
+
+        # Ensure an Enum subclass is provided
+        if enum_type is None:
+            raise ValueError("type must be assigned an Enum when using EnumAction")
+        if not issubclass(enum_type, enum.Enum):
+            raise TypeError("type must be an Enum when using EnumAction")
+
+        # Generate choices from the Enum
+        choices = tuple(e.value for e in enum_type)
+        kwargs.setdefault("choices", choices)
+        kwargs.setdefault("metavar", f"[{','.join(list(choices))}]")
+
+        super(EnumAction, self).__init__(**kwargs)
+
+        self._enum = enum_type
+
+    def __call__(self, parser, namespace, values, option_string=None):
+        # Convert value back into an Enum
+        value = self._enum(values)
+        setattr(namespace, self.dest, value)
+
+
+parser = argparse.ArgumentParser()
+
+parser.add_argument("--listen", type=str, default="127.0.0.1", metavar="IP", nargs="?", const="0.0.0.0,::", help="Specify the IP address to listen on (default: 127.0.0.1). You can give a list of ip addresses by separating them with a comma like: 127.2.2.2,127.3.3.3 If --listen is provided without an argument, it defaults to 0.0.0.0,:: (listens on all ipv4 and ipv6)")
+parser.add_argument("--port", type=int, default=8188, help="Set the listen port.")
+parser.add_argument("--tls-keyfile", type=str, help="Path to TLS (SSL) key file. Enables TLS, makes app accessible at https://... requires --tls-certfile to function")
+parser.add_argument("--tls-certfile", type=str, help="Path to TLS (SSL) certificate file. Enables TLS, makes app accessible at https://... requires --tls-keyfile to function")
+parser.add_argument("--enable-cors-header", type=str, default=None, metavar="ORIGIN", nargs="?", const="*", help="Enable CORS (Cross-Origin Resource Sharing) with optional origin or allow all with default '*'.")
+parser.add_argument("--max-upload-size", type=float, default=100, help="Set the maximum upload size in MB.")
+
+parser.add_argument("--base-directory", type=str, default=None, help="Set the ComfyUI base directory for models, custom_nodes, input, output, temp, and user directories.")
+parser.add_argument("--extra-model-paths-config", type=str, default=None, metavar="PATH", nargs='+', action='append', help="Load one or more extra_model_paths.yaml files.")
+parser.add_argument("--output-directory", type=str, default=None, help="Set the ComfyUI output directory. Overrides --base-directory.")
+parser.add_argument("--temp-directory", type=str, default=None, help="Set the ComfyUI temp directory (default is in the ComfyUI directory). Overrides --base-directory.")
+parser.add_argument("--input-directory", type=str, default=None, help="Set the ComfyUI input directory. Overrides --base-directory.")
+parser.add_argument("--auto-launch", action="store_true", help="Automatically launch ComfyUI in the default browser.")
+parser.add_argument("--disable-auto-launch", action="store_true", help="Disable auto launching the browser.")
+parser.add_argument("--cuda-device", type=int, default=None, metavar="DEVICE_ID", help="Set the id of the cuda device this instance will use.")
+cm_group = parser.add_mutually_exclusive_group()
+cm_group.add_argument("--cuda-malloc", action="store_true", help="Enable cudaMallocAsync (enabled by default for torch 2.0 and up).")
+cm_group.add_argument("--disable-cuda-malloc", action="store_true", help="Disable cudaMallocAsync.")
+
+
+fp_group = parser.add_mutually_exclusive_group()
+fp_group.add_argument("--force-fp32", action="store_true", help="Force fp32 (If this makes your GPU work better please report it).")
+fp_group.add_argument("--force-fp16", action="store_true", help="Force fp16.")
+
+fpunet_group = parser.add_mutually_exclusive_group()
+fpunet_group.add_argument("--fp32-unet", action="store_true", help="Run the diffusion model in fp32.")
+fpunet_group.add_argument("--fp64-unet", action="store_true", help="Run the diffusion model in fp64.")
+fpunet_group.add_argument("--bf16-unet", action="store_true", help="Run the diffusion model in bf16.")
+fpunet_group.add_argument("--fp16-unet", action="store_true", help="Run the diffusion model in fp16")
+fpunet_group.add_argument("--fp8_e4m3fn-unet", action="store_true", help="Store unet weights in fp8_e4m3fn.")
+fpunet_group.add_argument("--fp8_e5m2-unet", action="store_true", help="Store unet weights in fp8_e5m2.")
+
+fpvae_group = parser.add_mutually_exclusive_group()
+fpvae_group.add_argument("--fp16-vae", action="store_true", help="Run the VAE in fp16, might cause black images.")
+fpvae_group.add_argument("--fp32-vae", action="store_true", help="Run the VAE in full precision fp32.")
+fpvae_group.add_argument("--bf16-vae", action="store_true", help="Run the VAE in bf16.")
+
+parser.add_argument("--cpu-vae", action="store_true", help="Run the VAE on the CPU.")
+
+fpte_group = parser.add_mutually_exclusive_group()
+fpte_group.add_argument("--fp8_e4m3fn-text-enc", action="store_true", help="Store text encoder weights in fp8 (e4m3fn variant).")
+fpte_group.add_argument("--fp8_e5m2-text-enc", action="store_true", help="Store text encoder weights in fp8 (e5m2 variant).")
+fpte_group.add_argument("--fp16-text-enc", action="store_true", help="Store text encoder weights in fp16.")
+fpte_group.add_argument("--fp32-text-enc", action="store_true", help="Store text encoder weights in fp32.")
+fpte_group.add_argument("--bf16-text-enc", action="store_true", help="Store text encoder weights in bf16.")
+
+parser.add_argument("--force-channels-last", action="store_true", help="Force channels last format when inferencing the models.")
+
+parser.add_argument("--directml", type=int, nargs="?", metavar="DIRECTML_DEVICE", const=-1, help="Use torch-directml.")
+
+parser.add_argument("--oneapi-device-selector", type=str, default=None, metavar="SELECTOR_STRING", help="Sets the oneAPI device(s) this instance will use.")
+parser.add_argument("--disable-ipex-optimize", action="store_true", help="Disables ipex.optimize default when loading models with Intel's Extension for Pytorch.")
+
+class LatentPreviewMethod(enum.Enum):
+    NoPreviews = "none"
+    Auto = "auto"
+    Latent2RGB = "latent2rgb"
+    TAESD = "taesd"
+
+parser.add_argument("--preview-method", type=LatentPreviewMethod, default=LatentPreviewMethod.NoPreviews, help="Default preview method for sampler nodes.", action=EnumAction)
+
+parser.add_argument("--preview-size", type=int, default=512, help="Sets the maximum preview size for sampler nodes.")
+
+cache_group = parser.add_mutually_exclusive_group()
+cache_group.add_argument("--cache-classic", action="store_true", help="Use the old style (aggressive) caching.")
+cache_group.add_argument("--cache-lru", type=int, default=0, help="Use LRU caching with a maximum of N node results cached. May use more RAM/VRAM.")
+
+attn_group = parser.add_mutually_exclusive_group()
+attn_group.add_argument("--use-split-cross-attention", action="store_true", help="Use the split cross attention optimization. Ignored when xformers is used.")
+attn_group.add_argument("--use-quad-cross-attention", action="store_true", help="Use the sub-quadratic cross attention optimization . Ignored when xformers is used.")
+attn_group.add_argument("--use-pytorch-cross-attention", action="store_true", help="Use the new pytorch 2.0 cross attention function.")
+attn_group.add_argument("--use-sage-attention", action="store_true", help="Use sage attention.")
+attn_group.add_argument("--use-flash-attention", action="store_true", help="Use FlashAttention.")
+
+parser.add_argument("--disable-xformers", action="store_true", help="Disable xformers.")
+
+upcast = parser.add_mutually_exclusive_group()
+upcast.add_argument("--force-upcast-attention", action="store_true", help="Force enable attention upcasting, please report if it fixes black images.")
+upcast.add_argument("--dont-upcast-attention", action="store_true", help="Disable all upcasting of attention. Should be unnecessary except for debugging.")
+
+
+vram_group = parser.add_mutually_exclusive_group()
+vram_group.add_argument("--gpu-only", action="store_true", help="Store and run everything (text encoders/CLIP models, etc... on the GPU).")
+vram_group.add_argument("--highvram", action="store_true", help="By default models will be unloaded to CPU memory after being used. This option keeps them in GPU memory.")
+vram_group.add_argument("--normalvram", action="store_true", help="Used to force normal vram use if lowvram gets automatically enabled.")
+vram_group.add_argument("--lowvram", action="store_true", help="Split the unet in parts to use less vram.")
+vram_group.add_argument("--novram", action="store_true", help="When lowvram isn't enough.")
+vram_group.add_argument("--cpu", action="store_true", help="To use the CPU for everything (slow).")
+
+parser.add_argument("--reserve-vram", type=float, default=None, help="Set the amount of vram in GB you want to reserve for use by your OS/other software. By default some amount is reserved depending on your OS.")
+
+
+parser.add_argument("--default-hashing-function", type=str, choices=['md5', 'sha1', 'sha256', 'sha512'], default='sha256', help="Allows you to choose the hash function to use for duplicate filename / contents comparison. Default is sha256.")
+
+parser.add_argument("--disable-smart-memory", action="store_true", help="Force ComfyUI to agressively offload to regular ram instead of keeping models in vram when it can.")
+parser.add_argument("--deterministic", action="store_true", help="Make pytorch use slower deterministic algorithms when it can. Note that this might not make images deterministic in all cases.")
+
+class PerformanceFeature(enum.Enum):
+    Fp16Accumulation = "fp16_accumulation"
+    Fp8MatrixMultiplication = "fp8_matrix_mult"
+
+parser.add_argument("--fast", nargs="*", type=PerformanceFeature, help="Enable some untested and potentially quality deteriorating optimizations. --fast with no arguments enables everything. You can pass a list specific optimizations if you only want to enable specific ones. Current valid optimizations: fp16_accumulation fp8_matrix_mult")
+
+parser.add_argument("--dont-print-server", action="store_true", help="Don't print server output.")
+parser.add_argument("--quick-test-for-ci", action="store_true", help="Quick test for CI.")
+parser.add_argument("--windows-standalone-build", action="store_true", help="Windows standalone build: Enable convenient things that most people using the standalone windows build will probably enjoy (like auto opening the page on startup).")
+
+parser.add_argument("--disable-metadata", action="store_true", help="Disable saving prompt metadata in files.")
+parser.add_argument("--disable-all-custom-nodes", action="store_true", help="Disable loading all custom nodes.")
+
+parser.add_argument("--multi-user", action="store_true", help="Enables per-user storage.")
+
+parser.add_argument("--verbose", default='INFO', const='DEBUG', nargs="?", choices=['DEBUG', 'INFO', 'WARNING', 'ERROR', 'CRITICAL'], help='Set the logging level')
+parser.add_argument("--log-stdout", action="store_true", help="Send normal process output to stdout instead of stderr (default).")
+
+# The default built-in provider hosted under web/
+DEFAULT_VERSION_STRING = "comfyanonymous/ComfyUI@latest"
+
+parser.add_argument(
+    "--front-end-version",
+    type=str,
+    default=DEFAULT_VERSION_STRING,
+    help="""
+    Specifies the version of the frontend to be used. This command needs internet connectivity to query and
+    download available frontend implementations from GitHub releases.
+
+    The version string should be in the format of:
+    [repoOwner]/[repoName]@[version]
+    where version is one of: "latest" or a valid version number (e.g. "1.0.0")
+    """,
+)
+
+def is_valid_directory(path: str) -> str:
+    """Validate if the given path is a directory, and check permissions."""
+    if not os.path.exists(path):
+        raise argparse.ArgumentTypeError(f"The path '{path}' does not exist.")
+    if not os.path.isdir(path):
+        raise argparse.ArgumentTypeError(f"'{path}' is not a directory.")
+    if not os.access(path, os.R_OK):
+        raise argparse.ArgumentTypeError(f"You do not have read permissions for '{path}'.")
+    return path
+
+parser.add_argument(
+    "--front-end-root",
+    type=is_valid_directory,
+    default=None,
+    help="The local filesystem path to the directory where the frontend is located. Overrides --front-end-version.",
+)
+
+parser.add_argument("--user-directory", type=is_valid_directory, default=None, help="Set the ComfyUI user directory with an absolute path. Overrides --base-directory.")
+
+parser.add_argument("--enable-compress-response-body", action="store_true", help="Enable compressing response body.")
+
+if comfy.options.args_parsing:
+    args = parser.parse_args()
+else:
+    args = parser.parse_args([])
+
+if args.windows_standalone_build:
+    args.auto_launch = True
+
+if args.disable_auto_launch:
+    args.auto_launch = False
+
+if args.force_fp16:
+    args.fp16_unet = True
+
+
+# '--fast' is not provided, use an empty set
+if args.fast is None:
+    args.fast = set()
+# '--fast' is provided with an empty list, enable all optimizations
+elif args.fast == []:
+    args.fast = set(PerformanceFeature)
+# '--fast' is provided with a list of performance features, use that list
+else:
+    args.fast = set(args.fast)

Imagine/imagine-v5-ultra/comfy/clip_config_bigg.json

@@ -0,0 +1,23 @@
+{
+  "architectures": [
+    "CLIPTextModel"
+  ],
+  "attention_dropout": 0.0,
+  "bos_token_id": 0,
+  "dropout": 0.0,
+  "eos_token_id": 49407,
+  "hidden_act": "gelu",
+  "hidden_size": 1280,
+  "initializer_factor": 1.0,
+  "initializer_range": 0.02,
+  "intermediate_size": 5120,
+  "layer_norm_eps": 1e-05,
+  "max_position_embeddings": 77,
+  "model_type": "clip_text_model",
+  "num_attention_heads": 20,
+  "num_hidden_layers": 32,
+  "pad_token_id": 1,
+  "projection_dim": 1280,
+  "torch_dtype": "float32",
+  "vocab_size": 49408
+}

Imagine/imagine-v5-ultra/comfy/clip_model.py

@@ -0,0 +1,244 @@
+import torch
+from comfy.ldm.modules.attention import optimized_attention_for_device
+import comfy.ops
+
+class CLIPAttention(torch.nn.Module):
+    def __init__(self, embed_dim, heads, dtype, device, operations):
+        super().__init__()
+
+        self.heads = heads
+        self.q_proj = operations.Linear(embed_dim, embed_dim, bias=True, dtype=dtype, device=device)
+        self.k_proj = operations.Linear(embed_dim, embed_dim, bias=True, dtype=dtype, device=device)
+        self.v_proj = operations.Linear(embed_dim, embed_dim, bias=True, dtype=dtype, device=device)
+
+        self.out_proj = operations.Linear(embed_dim, embed_dim, bias=True, dtype=dtype, device=device)
+
+    def forward(self, x, mask=None, optimized_attention=None):
+        q = self.q_proj(x)
+        k = self.k_proj(x)
+        v = self.v_proj(x)
+
+        out = optimized_attention(q, k, v, self.heads, mask)
+        return self.out_proj(out)
+
+ACTIVATIONS = {"quick_gelu": lambda a: a * torch.sigmoid(1.702 * a),
+               "gelu": torch.nn.functional.gelu,
+               "gelu_pytorch_tanh": lambda a: torch.nn.functional.gelu(a, approximate="tanh"),
+}
+
+class CLIPMLP(torch.nn.Module):
+    def __init__(self, embed_dim, intermediate_size, activation, dtype, device, operations):
+        super().__init__()
+        self.fc1 = operations.Linear(embed_dim, intermediate_size, bias=True, dtype=dtype, device=device)
+        self.activation = ACTIVATIONS[activation]
+        self.fc2 = operations.Linear(intermediate_size, embed_dim, bias=True, dtype=dtype, device=device)
+
+    def forward(self, x):
+        x = self.fc1(x)
+        x = self.activation(x)
+        x = self.fc2(x)
+        return x
+
+class CLIPLayer(torch.nn.Module):
+    def __init__(self, embed_dim, heads, intermediate_size, intermediate_activation, dtype, device, operations):
+        super().__init__()
+        self.layer_norm1 = operations.LayerNorm(embed_dim, dtype=dtype, device=device)
+        self.self_attn = CLIPAttention(embed_dim, heads, dtype, device, operations)
+        self.layer_norm2 = operations.LayerNorm(embed_dim, dtype=dtype, device=device)
+        self.mlp = CLIPMLP(embed_dim, intermediate_size, intermediate_activation, dtype, device, operations)
+
+    def forward(self, x, mask=None, optimized_attention=None):
+        x += self.self_attn(self.layer_norm1(x), mask, optimized_attention)
+        x += self.mlp(self.layer_norm2(x))
+        return x
+
+
+class CLIPEncoder(torch.nn.Module):
+    def __init__(self, num_layers, embed_dim, heads, intermediate_size, intermediate_activation, dtype, device, operations):
+        super().__init__()
+        self.layers = torch.nn.ModuleList([CLIPLayer(embed_dim, heads, intermediate_size, intermediate_activation, dtype, device, operations) for i in range(num_layers)])
+
+    def forward(self, x, mask=None, intermediate_output=None):
+        optimized_attention = optimized_attention_for_device(x.device, mask=mask is not None, small_input=True)
+
+        if intermediate_output is not None:
+            if intermediate_output < 0:
+                intermediate_output = len(self.layers) + intermediate_output
+
+        intermediate = None
+        for i, l in enumerate(self.layers):
+            x = l(x, mask, optimized_attention)
+            if i == intermediate_output:
+                intermediate = x.clone()
+        return x, intermediate
+
+class CLIPEmbeddings(torch.nn.Module):
+    def __init__(self, embed_dim, vocab_size=49408, num_positions=77, dtype=None, device=None, operations=None):
+        super().__init__()
+        self.token_embedding = operations.Embedding(vocab_size, embed_dim, dtype=dtype, device=device)
+        self.position_embedding = operations.Embedding(num_positions, embed_dim, dtype=dtype, device=device)
+
+    def forward(self, input_tokens, dtype=torch.float32):
+        return self.token_embedding(input_tokens, out_dtype=dtype) + comfy.ops.cast_to(self.position_embedding.weight, dtype=dtype, device=input_tokens.device)
+
+
+class CLIPTextModel_(torch.nn.Module):
+    def __init__(self, config_dict, dtype, device, operations):
+        num_layers = config_dict["num_hidden_layers"]
+        embed_dim = config_dict["hidden_size"]
+        heads = config_dict["num_attention_heads"]
+        intermediate_size = config_dict["intermediate_size"]
+        intermediate_activation = config_dict["hidden_act"]
+        num_positions = config_dict["max_position_embeddings"]
+        self.eos_token_id = config_dict["eos_token_id"]
+
+        super().__init__()
+        self.embeddings = CLIPEmbeddings(embed_dim, num_positions=num_positions, dtype=dtype, device=device, operations=operations)
+        self.encoder = CLIPEncoder(num_layers, embed_dim, heads, intermediate_size, intermediate_activation, dtype, device, operations)
+        self.final_layer_norm = operations.LayerNorm(embed_dim, dtype=dtype, device=device)
+
+    def forward(self, input_tokens=None, attention_mask=None, embeds=None, num_tokens=None, intermediate_output=None, final_layer_norm_intermediate=True, dtype=torch.float32):
+        if embeds is not None:
+            x = embeds + comfy.ops.cast_to(self.embeddings.position_embedding.weight, dtype=dtype, device=embeds.device)
+        else:
+            x = self.embeddings(input_tokens, dtype=dtype)
+
+        mask = None
+        if attention_mask is not None:
+            mask = 1.0 - attention_mask.to(x.dtype).reshape((attention_mask.shape[0], 1, -1, attention_mask.shape[-1])).expand(attention_mask.shape[0], 1, attention_mask.shape[-1], attention_mask.shape[-1])
+            mask = mask.masked_fill(mask.to(torch.bool), -torch.finfo(x.dtype).max)
+
+        causal_mask = torch.full((x.shape[1], x.shape[1]), -torch.finfo(x.dtype).max, dtype=x.dtype, device=x.device).triu_(1)
+
+        if mask is not None:
+            mask += causal_mask
+        else:
+            mask = causal_mask
+
+        x, i = self.encoder(x, mask=mask, intermediate_output=intermediate_output)
+        x = self.final_layer_norm(x)
+        if i is not None and final_layer_norm_intermediate:
+            i = self.final_layer_norm(i)
+
+        if num_tokens is not None:
+            pooled_output = x[list(range(x.shape[0])), list(map(lambda a: a - 1, num_tokens))]
+        else:
+            pooled_output = x[torch.arange(x.shape[0], device=x.device), (torch.round(input_tokens).to(dtype=torch.int, device=x.device) == self.eos_token_id).int().argmax(dim=-1),]
+        return x, i, pooled_output
+
+class CLIPTextModel(torch.nn.Module):
+    def __init__(self, config_dict, dtype, device, operations):
+        super().__init__()
+        self.num_layers = config_dict["num_hidden_layers"]
+        self.text_model = CLIPTextModel_(config_dict, dtype, device, operations)
+        embed_dim = config_dict["hidden_size"]
+        self.text_projection = operations.Linear(embed_dim, embed_dim, bias=False, dtype=dtype, device=device)
+        self.dtype = dtype
+
+    def get_input_embeddings(self):
+        return self.text_model.embeddings.token_embedding
+
+    def set_input_embeddings(self, embeddings):
+        self.text_model.embeddings.token_embedding = embeddings
+
+    def forward(self, *args, **kwargs):
+        x = self.text_model(*args, **kwargs)
+        out = self.text_projection(x[2])
+        return (x[0], x[1], out, x[2])
+
+
+class CLIPVisionEmbeddings(torch.nn.Module):
+    def __init__(self, embed_dim, num_channels=3, patch_size=14, image_size=224, model_type="", dtype=None, device=None, operations=None):
+        super().__init__()
+
+        num_patches = (image_size // patch_size) ** 2
+        if model_type == "siglip_vision_model":
+            self.class_embedding = None
+            patch_bias = True
+        else:
+            num_patches = num_patches + 1
+            self.class_embedding = torch.nn.Parameter(torch.empty(embed_dim, dtype=dtype, device=device))
+            patch_bias = False
+
+        self.patch_embedding = operations.Conv2d(
+            in_channels=num_channels,
+            out_channels=embed_dim,
+            kernel_size=patch_size,
+            stride=patch_size,
+            bias=patch_bias,
+            dtype=dtype,
+            device=device
+        )
+
+        self.position_embedding = operations.Embedding(num_patches, embed_dim, dtype=dtype, device=device)
+
+    def forward(self, pixel_values):
+        embeds = self.patch_embedding(pixel_values).flatten(2).transpose(1, 2)
+        if self.class_embedding is not None:
+            embeds = torch.cat([comfy.ops.cast_to_input(self.class_embedding, embeds).expand(pixel_values.shape[0], 1, -1), embeds], dim=1)
+        return embeds + comfy.ops.cast_to_input(self.position_embedding.weight, embeds)
+
+
+class CLIPVision(torch.nn.Module):
+    def __init__(self, config_dict, dtype, device, operations):
+        super().__init__()
+        num_layers = config_dict["num_hidden_layers"]
+        embed_dim = config_dict["hidden_size"]
+        heads = config_dict["num_attention_heads"]
+        intermediate_size = config_dict["intermediate_size"]
+        intermediate_activation = config_dict["hidden_act"]
+        model_type = config_dict["model_type"]
+
+        self.embeddings = CLIPVisionEmbeddings(embed_dim, config_dict["num_channels"], config_dict["patch_size"], config_dict["image_size"], model_type=model_type, dtype=dtype, device=device, operations=operations)
+        if model_type == "siglip_vision_model":
+            self.pre_layrnorm = lambda a: a
+            self.output_layernorm = True
+        else:
+            self.pre_layrnorm = operations.LayerNorm(embed_dim)
+            self.output_layernorm = False
+        self.encoder = CLIPEncoder(num_layers, embed_dim, heads, intermediate_size, intermediate_activation, dtype, device, operations)
+        self.post_layernorm = operations.LayerNorm(embed_dim)
+
+    def forward(self, pixel_values, attention_mask=None, intermediate_output=None):
+        x = self.embeddings(pixel_values)
+        x = self.pre_layrnorm(x)
+        #TODO: attention_mask?
+        x, i = self.encoder(x, mask=None, intermediate_output=intermediate_output)
+        if self.output_layernorm:
+            x = self.post_layernorm(x)
+            pooled_output = x
+        else:
+            pooled_output = self.post_layernorm(x[:, 0, :])
+        return x, i, pooled_output
+
+class LlavaProjector(torch.nn.Module):
+    def __init__(self, in_dim, out_dim, dtype, device, operations):
+        super().__init__()
+        self.linear_1 = operations.Linear(in_dim, out_dim, bias=True, device=device, dtype=dtype)
+        self.linear_2 = operations.Linear(out_dim, out_dim, bias=True, device=device, dtype=dtype)
+
+    def forward(self, x):
+        return self.linear_2(torch.nn.functional.gelu(self.linear_1(x[:, 1:])))
+
+class CLIPVisionModelProjection(torch.nn.Module):
+    def __init__(self, config_dict, dtype, device, operations):
+        super().__init__()
+        self.vision_model = CLIPVision(config_dict, dtype, device, operations)
+        if "projection_dim" in config_dict:
+            self.visual_projection = operations.Linear(config_dict["hidden_size"], config_dict["projection_dim"], bias=False)
+        else:
+            self.visual_projection = lambda a: a
+
+        if "llava3" == config_dict.get("projector_type", None):
+            self.multi_modal_projector = LlavaProjector(config_dict["hidden_size"], 4096, dtype, device, operations)
+        else:
+            self.multi_modal_projector = None
+
+    def forward(self, *args, **kwargs):
+        x = self.vision_model(*args, **kwargs)
+        out = self.visual_projection(x[2])
+        projected = None
+        if self.multi_modal_projector is not None:
+            projected = self.multi_modal_projector(x[1])
+
+        return (x[0], x[1], out, projected)

Imagine/imagine-v5-ultra/comfy/clip_vision.py

@@ -0,0 +1,143 @@
+from .utils import load_torch_file, transformers_convert, state_dict_prefix_replace
+import os
+import torch
+import json
+import logging
+
+import comfy.ops
+import comfy.model_patcher
+import comfy.model_management
+import comfy.utils
+import comfy.clip_model
+import comfy.image_encoders.dino2
+
+class Output:
+    def __getitem__(self, key):
+        return getattr(self, key)
+    def __setitem__(self, key, item):
+        setattr(self, key, item)
+
+def clip_preprocess(image, size=224, mean=[0.48145466, 0.4578275, 0.40821073], std=[0.26862954, 0.26130258, 0.27577711], crop=True):
+    mean = torch.tensor(mean, device=image.device, dtype=image.dtype)
+    std = torch.tensor(std, device=image.device, dtype=image.dtype)
+    image = image.movedim(-1, 1)
+    if not (image.shape[2] == size and image.shape[3] == size):
+        if crop:
+            scale = (size / min(image.shape[2], image.shape[3]))
+            scale_size = (round(scale * image.shape[2]), round(scale * image.shape[3]))
+        else:
+            scale_size = (size, size)
+
+        image = torch.nn.functional.interpolate(image, size=scale_size, mode="bicubic", antialias=True)
+        h = (image.shape[2] - size)//2
+        w = (image.shape[3] - size)//2
+        image = image[:,:,h:h+size,w:w+size]
+    image = torch.clip((255. * image), 0, 255).round() / 255.0
+    return (image - mean.view([3,1,1])) / std.view([3,1,1])
+
+IMAGE_ENCODERS = {
+    "clip_vision_model": comfy.clip_model.CLIPVisionModelProjection,
+    "siglip_vision_model": comfy.clip_model.CLIPVisionModelProjection,
+    "dinov2": comfy.image_encoders.dino2.Dinov2Model,
+}
+
+class ClipVisionModel():
+    def __init__(self, json_config):
+        with open(json_config) as f:
+            config = json.load(f)
+
+        self.image_size = config.get("image_size", 224)
+        self.image_mean = config.get("image_mean", [0.48145466, 0.4578275, 0.40821073])
+        self.image_std = config.get("image_std", [0.26862954, 0.26130258, 0.27577711])
+        model_class = IMAGE_ENCODERS.get(config.get("model_type", "clip_vision_model"))
+        self.load_device = comfy.model_management.text_encoder_device()
+        offload_device = comfy.model_management.text_encoder_offload_device()
+        self.dtype = comfy.model_management.text_encoder_dtype(self.load_device)
+        self.model = model_class(config, self.dtype, offload_device, comfy.ops.manual_cast)
+        self.model.eval()
+
+        self.patcher = comfy.model_patcher.ModelPatcher(self.model, load_device=self.load_device, offload_device=offload_device)
+
+    def load_sd(self, sd):
+        return self.model.load_state_dict(sd, strict=False)
+
+    def get_sd(self):
+        return self.model.state_dict()
+
+    def encode_image(self, image, crop=True):
+        comfy.model_management.load_model_gpu(self.patcher)
+        pixel_values = clip_preprocess(image.to(self.load_device), size=self.image_size, mean=self.image_mean, std=self.image_std, crop=crop).float()
+        out = self.model(pixel_values=pixel_values, intermediate_output=-2)
+
+        outputs = Output()
+        outputs["last_hidden_state"] = out[0].to(comfy.model_management.intermediate_device())
+        outputs["image_embeds"] = out[2].to(comfy.model_management.intermediate_device())
+        outputs["penultimate_hidden_states"] = out[1].to(comfy.model_management.intermediate_device())
+        outputs["mm_projected"] = out[3]
+        return outputs
+
+def convert_to_transformers(sd, prefix):
+    sd_k = sd.keys()
+    if "{}transformer.resblocks.0.attn.in_proj_weight".format(prefix) in sd_k:
+        keys_to_replace = {
+            "{}class_embedding".format(prefix): "vision_model.embeddings.class_embedding",
+            "{}conv1.weight".format(prefix): "vision_model.embeddings.patch_embedding.weight",
+            "{}positional_embedding".format(prefix): "vision_model.embeddings.position_embedding.weight",
+            "{}ln_post.bias".format(prefix): "vision_model.post_layernorm.bias",
+            "{}ln_post.weight".format(prefix): "vision_model.post_layernorm.weight",
+            "{}ln_pre.bias".format(prefix): "vision_model.pre_layrnorm.bias",
+            "{}ln_pre.weight".format(prefix): "vision_model.pre_layrnorm.weight",
+        }
+
+        for x in keys_to_replace:
+            if x in sd_k:
+                sd[keys_to_replace[x]] = sd.pop(x)
+
+        if "{}proj".format(prefix) in sd_k:
+            sd['visual_projection.weight'] = sd.pop("{}proj".format(prefix)).transpose(0, 1)
+
+        sd = transformers_convert(sd, prefix, "vision_model.", 48)
+    else:
+        replace_prefix = {prefix: ""}
+        sd = state_dict_prefix_replace(sd, replace_prefix)
+    return sd
+
+def load_clipvision_from_sd(sd, prefix="", convert_keys=False):
+    if convert_keys:
+        sd = convert_to_transformers(sd, prefix)
+    if "vision_model.encoder.layers.47.layer_norm1.weight" in sd:
+        json_config = os.path.join(os.path.dirname(os.path.realpath(__file__)), "clip_vision_config_g.json")
+    elif "vision_model.encoder.layers.30.layer_norm1.weight" in sd:
+        json_config = os.path.join(os.path.dirname(os.path.realpath(__file__)), "clip_vision_config_h.json")
+    elif "vision_model.encoder.layers.22.layer_norm1.weight" in sd:
+        if sd["vision_model.encoder.layers.0.layer_norm1.weight"].shape[0] == 1152:
+            json_config = os.path.join(os.path.dirname(os.path.realpath(__file__)), "clip_vision_siglip_384.json")
+        elif sd["vision_model.embeddings.position_embedding.weight"].shape[0] == 577:
+            if "multi_modal_projector.linear_1.bias" in sd:
+                json_config = os.path.join(os.path.dirname(os.path.realpath(__file__)), "clip_vision_config_vitl_336_llava.json")
+            else:
+                json_config = os.path.join(os.path.dirname(os.path.realpath(__file__)), "clip_vision_config_vitl_336.json")
+        else:
+            json_config = os.path.join(os.path.dirname(os.path.realpath(__file__)), "clip_vision_config_vitl.json")
+    elif "embeddings.patch_embeddings.projection.weight" in sd:
+        json_config = os.path.join(os.path.join(os.path.dirname(os.path.realpath(__file__)), "image_encoders"), "dino2_giant.json")
+    else:
+        return None
+
+    clip = ClipVisionModel(json_config)
+    m, u = clip.load_sd(sd)
+    if len(m) > 0:
+        logging.warning("missing clip vision: {}".format(m))
+    u = set(u)
+    keys = list(sd.keys())
+    for k in keys:
+        if k not in u:
+            sd.pop(k)
+    return clip
+
+def load(ckpt_path):
+    sd = load_torch_file(ckpt_path)
+    if "visual.transformer.resblocks.0.attn.in_proj_weight" in sd:
+        return load_clipvision_from_sd(sd, prefix="visual.", convert_keys=True)
+    else:
+        return load_clipvision_from_sd(sd)

Imagine/imagine-v5-ultra/comfy/clip_vision_config_g.json

@@ -0,0 +1,18 @@
+{
+  "attention_dropout": 0.0,
+  "dropout": 0.0,
+  "hidden_act": "gelu",
+  "hidden_size": 1664,
+  "image_size": 224,
+  "initializer_factor": 1.0,
+  "initializer_range": 0.02,
+  "intermediate_size": 8192,
+  "layer_norm_eps": 1e-05,
+  "model_type": "clip_vision_model",
+  "num_attention_heads": 16,
+  "num_channels": 3,
+  "num_hidden_layers": 48,
+  "patch_size": 14,
+  "projection_dim": 1280,
+  "torch_dtype": "float32"
+}

Imagine/imagine-v5-ultra/comfy/clip_vision_config_h.json

@@ -0,0 +1,18 @@
+{
+  "attention_dropout": 0.0,
+  "dropout": 0.0,
+  "hidden_act": "gelu",
+  "hidden_size": 1280,
+  "image_size": 224,
+  "initializer_factor": 1.0,
+  "initializer_range": 0.02,
+  "intermediate_size": 5120,
+  "layer_norm_eps": 1e-05,
+  "model_type": "clip_vision_model",
+  "num_attention_heads": 16,
+  "num_channels": 3,
+  "num_hidden_layers": 32,
+  "patch_size": 14,
+  "projection_dim": 1024,
+  "torch_dtype": "float32"
+}

Imagine/imagine-v5-ultra/comfy/clip_vision_config_vitl.json

@@ -0,0 +1,18 @@
+{
+  "attention_dropout": 0.0,
+  "dropout": 0.0,
+  "hidden_act": "quick_gelu",
+  "hidden_size": 1024,
+  "image_size": 224,
+  "initializer_factor": 1.0,
+  "initializer_range": 0.02,
+  "intermediate_size": 4096,
+  "layer_norm_eps": 1e-05,
+  "model_type": "clip_vision_model",
+  "num_attention_heads": 16,
+  "num_channels": 3,
+  "num_hidden_layers": 24,
+  "patch_size": 14,
+  "projection_dim": 768,
+  "torch_dtype": "float32"
+}