maryzhang/motif_lora

Chinese Porcelain Motif LoRA

Model Details

Model Description

This is a LoRA (Low-Rank Adaptation) fine-tuned version of runwayml/stable-diffusion-v1-5 specifically trained to generate Chinese porcelain motif patterns. The model specializes in creating intricate line-based decorative patterns characteristic of traditional Chinese porcelain artwork.

• Developed by: Mary Zhang
• Model type: LoRA fine-tuned Stable Diffusion v1.5
• Language(s): English prompts
• License: [Inherit from base model - CreativeML Open RAIL-M]
• Finetuned from model: runwayml/stable-diffusion-v1-5
• Repository: https://huggingface.co/maryzhang/motif_lora

Model Sources

• Repository: https://huggingface.co/maryzhang/motif_lora
• Base Model: https://huggingface.co/runwayml/stable-diffusion-v1-5 currently redirected to https://huggingface.co/stable-diffusion-v1-5/stable-diffusion-v1-5

Uses

Direct Use

This model is designed for generating Chinese porcelain motif patterns for:

• Digital art creation
• Pattern design for ceramics
• Educational materials about Chinese decorative arts
• Inspiration for traditional craft designs
• Cultural heritage visualization

Downstream Use

The model can be integrated into:

• Design software for ceramic artists
• Educational applications for art history
• Digital restoration projects for historical porcelain patterns
• Commercial design workflows for products inspired by Chinese motifs

Out-of-Scope Use

This model should not be used for:

• Generating photorealistic images unrelated to porcelain motifs
• Creating culturally insensitive or inappropriate content
• Authenticating or verifying historical artifacts
• Commercial reproduction of copyrighted porcelain designs

Bias, Risks, and Limitations

Biases

• Cultural Representation: The model is trained specifically on Chinese porcelain motifs and may not accurately represent other ceramic traditions
• Historical Period Bias: Depending on the training dataset, certain dynasties or periods may be overrepresented
• Style Bias: May favor certain decorative styles (e.g., blue and white porcelain) over others

Risks

• Cultural Appropriation: Users should be mindful of respectful use of traditional Chinese artistic elements
• Historical Accuracy: Generated patterns should not be presented as authentic historical designs
• Quality Variance: Output quality may vary based on prompt specificity

Limitations

• Limited to decorative motif generation, not full porcelain object rendering
• Best results with prompts including "porcelain motif" trigger phrase
• Resolution optimized for 512×512 pixels
• May struggle with highly specific regional or temporal style variations

Recommendations

Users should:

• Include cultural context when sharing generated images
• Verify historical accuracy if using for educational purposes
• Test various prompt formulations for optimal results
• Consider combining with other models for complete ceramic designs

How to Get Started with the Model

from diffusers import StableDiffusionPipeline
import torch

# Load the pipeline with LoRA weights
model_id = "runwayml/stable-diffusion-v1-5"
lora_model_id = "maryzhang/motif_lora"

pipe = StableDiffusionPipeline.from_pretrained(
    model_id,
    torch_dtype=torch.float16
)
pipe = pipe.to("cuda")

# Load LoRA weights
pipe.load_lora_weights(lora_model_id)

# Generate an image
prompt = "porcelain motif, intricate blue and white floral pattern, traditional Chinese design"
image = pipe(
    prompt,
    num_inference_steps=50,
    guidance_scale=7.5,
    width=512,
    height=512
).images[0]

image.save("porcelain_motif.png")

Training Details

Training Data

The model was fine-tuned on a curated dataset of Chinese porcelain motif images, focusing on:

• Traditional decorative patterns from various dynasties
• Line-based artistic elements
• Both geometric and naturalistic motifs
• Various color schemes with emphasis on traditional palettes

Training Procedure

Preprocessing

• Images resized to 512×512 pixels
• Normalization applied consistent with SD v1.5 preprocessing
• Augmentation: minimal to preserve pattern integrity

Training Hyperparameters

• Training regime: fp16 mixed precision
• Epochs: 3
• Base Learning Rate: 1e-4 (assumed standard for LoRA)
• LoRA Rank: 4-16 (typical range)
• LoRA Alpha: Same as rank or 2x rank
• Target Modules: Cross-attention layers (q, k, v, out projections)
• Batch Size: [Not specified - likely 1-4 given hardware constraints]
• Training Resolution: 512×512
• Trigger Phrase: "porcelain motif"

Development Process

AI-Assisted Development

Transparency Note: ChatGPT was utilized during the development process for:

• Training Script Generation: Initial boilerplate code for LoRA fine-tuning setup
• Data Preprocessing Pipeline: Code for image resizing and dataset preparation
• Hyperparameter Optimization: Suggestions for learning rate schedules and LoRA configurations
• Debugging Support: Troubleshooting CUDA memory issues and gradient accumulation
• Documentation: Assistance with code comments and README formatting

Example AI-assisted code components:

# Dataset preprocessing code structure suggested by ChatGPT
class PorcelainMotifDataset(Dataset):
    def __init__(self, image_paths, captions, transform=None):
        self.image_paths = image_paths
        self.captions = captions
        self.transform = transform
    
    def __getitem__(self, idx):
        # Implementation guided by AI assistance
        image = Image.open(self.image_paths[idx])
        if self.transform:
            image = self.transform(image)
        return image, self.captions[idx]

Evaluation

Testing Data

Evaluated on held-out set of porcelain motif images not seen during training, including:

• Historical museum photographs
• Contemporary porcelain designs
• Various regional styles

Metrics

Qualitative evaluation based on:

• Pattern Coherence: Logical flow and completeness of decorative elements
• Style Authenticity: Resemblance to traditional Chinese porcelain motifs
• Line Quality: Clarity and precision of line-based patterns
• Cultural Accuracy: Appropriate use of traditional symbols and elements

Results

• Successfully generates recognizable Chinese porcelain motif patterns
• Best performance with blue and white color schemes
• Strong reproduction of floral and geometric patterns
• Occasional blending of different historical periods in single outputs

Environmental Impact

• Hardware Type: [Likely consumer GPU - RTX 3090/4090 or similar]
• Hours used: Approximately 6-12 hours for 3 epochs
• Cloud Provider: [If applicable]
• Carbon Emitted: Estimated 2-5 kg CO2eq based on typical GPU training

Technical Specifications

Model Architecture and Objective

• Architecture: U-Net with LoRA adaptations on cross-attention layers
• Objective: Denoising diffusion probabilistic model with LoRA fine-tuning

Compute Infrastructure

Hardware

• GPU: [Specific model not provided - likely NVIDIA consumer/prosumer GPU]
• VRAM Requirements: Minimum 10GB recommended

Software

• Framework: PyTorch with Diffusers library
• LoRA Implementation: PEFT (Parameter-Efficient Fine-Tuning) library
• Base Dependencies:
  • diffusers >= 0.21.0
  • transformers >= 4.25.0
  • accelerate >= 0.20.0
  • peft >= 0.4.0

Citation

BibTeX:

@misc{zhang2024porcelainlora,
  author = {Zhang, Mary},
  title = {Chinese Porcelain Motif LoRA: Fine-Tuned Stable Diffusion for Traditional Pattern Generation},
  year = {2024},
  publisher = {HuggingFace},
  url = {https://huggingface.co/maryzhang/motif_lora}
}

APA: Zhang, M. (2024). Chinese Porcelain Motif LoRA: Fine-Tuned Stable Diffusion for Traditional Pattern Generation [Model]. HuggingFace. https://huggingface.co/maryzhang/motif_lora

Model Card Authors

Mary Zhang, Cassie Li

Acknowledgments

• Runway ML for the base Stable Diffusion v1.5 model
• The open-source community for LoRA implementation tools
• ChatGPT (OpenAI) for development assistance with code generation and debugging
• Museums and cultural institutions whose publicly available porcelain collections inspired this work

Version History

• v1.0 (Current): Initial release with 3-epoch training on Chinese porcelain motifs

---