ariG23498/open-image-preferences-v1-sdxl-lora

Low Rank Adapted Supervised Fine Tuned Stable Diffusion XL

Comparison

Prompt	SDXL	Fine Tuned
a boat in the canals of Venice, painted in gouache with soft, flowing brushstrokes and vibrant, translucent colors, capturing the serene reflection on the water under a misty ambiance, with rich textures and a dynamic perspective
Grainy shot of a robot cooking in the kitchen, with soft shadows and nostalgic film texture.

Model description

These are ariG23498/open-image-preferences-v1-sdxl-lora LoRA adaption weights for stabilityai/stable-diffusion-xl-base-1.0.

The weights were trained using DreamBooth using the open-image-preferences-v1-binarized dataset.

Use with diffusers

from diffusers import AutoPipelineForText2Image
import torch

pipeline = AutoPipelineForText2Image.from_pretrained(
    "stabilityai/stable-diffusion-xl-base-1.0",
    torch_dtype=torch.bfloat16
).to('cuda')
pipeline.load_lora_weights('ariG23498/open-image-preferences-v1-sdxl-lora', weight_name='pytorch_lora_weights.safetensors')
prompt = "ENTER PROMPT"
image = pipeline(prompt).images[0]

Command to train the model

!accelerate launch examples/dreambooth/train_dreambooth_lora_sdxl.py \
    --pretrained_model_name_or_path "stabilityai/stable-diffusion-xl-base-1.0" \
    --dataset_name "data-is-better-together/open-image-preferences-v1-binarized" \
    --hub_model_id "ariG23498/open-image-preferences-v1-sdxl-lora" \
    --push_to_hub \
    --output_dir "open-image-preferences-v1-sdxl-lora" \
    --image_column "chosen" \
    --caption_column "prompt" \
    --mixed_precision="bf16" \
    --resolution=1024 \
    --train_batch_size=1 \
    --repeats=1 \
    --report_to="wandb"\
    --gradient_accumulation_steps=1 \
    --gradient_checkpointing \
    --learning_rate=1.0 \
    --text_encoder_lr=1.0 \
    --optimizer="prodigy"\
    --lr_scheduler="constant" \
    --lr_warmup_steps=0 \
    --rank=8 \
    --checkpointing_steps=2000 \
    --seed="0"