|
|
--- |
|
|
title: DiffSketcher |
|
|
emoji: 🎨 |
|
|
colorFrom: blue |
|
|
colorTo: purple |
|
|
sdk: custom |
|
|
app_file: handler.py |
|
|
pinned: false |
|
|
license: mit |
|
|
tags: |
|
|
- text-to-svg |
|
|
- vector-graphics |
|
|
- diffusion |
|
|
- sketch |
|
|
- art |
|
|
pipeline_tag: text-to-image |
|
|
--- |
|
|
|
|
|
# DiffSketcher: Text Guided Vector Sketch Synthesis |
|
|
|
|
|
DiffSketcher is a novel method for generating high-quality vector sketches from text prompts using latent diffusion models. This model can create artistic SVG representations based on natural language descriptions. |
|
|
|
|
|
## Model Description |
|
|
|
|
|
DiffSketcher leverages the power of Stable Diffusion to guide the generation of vector graphics. The model optimizes SVG paths to match the semantic content described in the input text while maintaining the artistic quality of hand-drawn sketches. |
|
|
|
|
|
## Usage |
|
|
|
|
|
### Direct API Call |
|
|
|
|
|
```python |
|
|
import requests |
|
|
|
|
|
API_URL = "https://api-inference.huggingface.co/models/jree423/diffsketcher" |
|
|
headers = {"Authorization": "Bearer YOUR_HF_TOKEN"} |
|
|
|
|
|
def query(payload): |
|
|
response = requests.post(API_URL, headers=headers, json=payload) |
|
|
return response.json() |
|
|
|
|
|
output = query({ |
|
|
"inputs": "a beautiful mountain landscape", |
|
|
"parameters": { |
|
|
"num_paths": 96, |
|
|
"num_iter": 500, |
|
|
"guidance_scale": 7.5, |
|
|
"width": 224, |
|
|
"height": 224, |
|
|
"seed": 42 |
|
|
} |
|
|
}) |
|
|
``` |
|
|
|
|
|
### Using the Inference Client |
|
|
|
|
|
```python |
|
|
from huggingface_hub import InferenceClient |
|
|
|
|
|
client = InferenceClient("jree423/diffsketcher") |
|
|
result = client.post( |
|
|
json={ |
|
|
"inputs": "a cat sitting on a windowsill", |
|
|
"parameters": { |
|
|
"num_paths": 128, |
|
|
"guidance_scale": 8.0 |
|
|
} |
|
|
} |
|
|
) |
|
|
``` |
|
|
|
|
|
## Parameters |
|
|
|
|
|
- **num_paths** (int, default: 96): Number of SVG paths to generate. More paths create more detailed sketches. |
|
|
- **num_iter** (int, default: 500): Number of optimization iterations. More iterations improve quality but take longer. |
|
|
- **guidance_scale** (float, default: 7.5): Controls how closely the generation follows the text prompt. |
|
|
- **width** (int, default: 224): Output SVG width in pixels. |
|
|
- **height** (int, default: 224): Output SVG height in pixels. |
|
|
- **seed** (int, default: 42): Random seed for reproducible results. |
|
|
|
|
|
## Output Format |
|
|
|
|
|
The model returns a JSON object containing: |
|
|
- `svg`: The generated SVG content as a string |
|
|
- `svg_base64`: Base64 encoded SVG for easy transmission |
|
|
- `prompt`: The input text prompt |
|
|
- `parameters`: The parameters used for generation |
|
|
|
|
|
## Examples |
|
|
|
|
|
### Simple Objects |
|
|
- "a red apple" |
|
|
- "a flying bird" |
|
|
- "a vintage car" |
|
|
|
|
|
### Complex Scenes |
|
|
- "a mountain landscape with trees" |
|
|
- "a city skyline at sunset" |
|
|
- "a garden with flowers and butterflies" |
|
|
|
|
|
### Artistic Styles |
|
|
- "a portrait in the style of Van Gogh" |
|
|
- "minimalist line drawing of a face" |
|
|
- "abstract geometric patterns" |
|
|
|
|
|
## Technical Details |
|
|
|
|
|
- **Base Model**: Stable Diffusion 2.1 |
|
|
- **Framework**: PyTorch + Diffusers |
|
|
- **Vector Rendering**: DiffVG (differentiable vector graphics) |
|
|
- **Optimization**: Adam optimizer with custom learning rates for different SVG parameters |
|
|
|
|
|
## Citation |
|
|
|
|
|
```bibtex |
|
|
@inproceedings{xing2023diffsketcher, |
|
|
title={DiffSketcher: Text Guided Vector Sketch Synthesis through Latent Diffusion Models}, |
|
|
author={Xing, XiMing and others}, |
|
|
booktitle={NeurIPS}, |
|
|
year={2023} |
|
|
} |
|
|
``` |
|
|
|
|
|
## License |
|
|
|
|
|
This model is released under the MIT License. |
