HUNYUAN3D_SETUP.md

Hunyuan3D Direct Model Setup Guide

Overview

This guide explains how to use the Hunyuan3D-2.1 model directly in DigiPal, taking advantage of your available RAM/VRAM.

What Changed

Previous Implementation (Gradio API)

• Used external Gradio API calls to tencent/Hunyuan3D-2.1 space
• API calls were timing out or hanging
• Limited control over generation parameters

New Implementation (Direct Model)

• Downloads and uses Hunyuan3D model directly
• Full control over generation process
• Three-tier fallback system for robustness
• Optimized for systems with >12GB VRAM

Installation

1. Basic Requirements

pip install -r requirements.txt

2. Hunyuan3D Requirements

pip install -r requirements_hunyuan3d.txt

3. Optional: Full Hunyuan3D Setup

For the complete Hunyuan3D experience:

# Clone the Hunyuan3D repository
git clone https://huggingface.co/spaces/tencent/Hunyuan3D-2.1 hunyuan3d_repo

# Copy the required modules to your project
cp -r hunyuan3d_repo/hy3dshape ./
cp -r hunyuan3d_repo/hy3dpaint ./

How It Works

Three-Tier 3D Generation System

1. Direct Model Mode (Best Quality)

   • Uses full Hunyuan3D model if modules are available
   • Generates high-quality 3D models with textures
   • Takes 2-3 minutes per model

2. Simplified Mode (Faster)

   • Uses PyTorch-based depth estimation
   • Creates textured 3D models from 2D images
   • Takes 30-60 seconds per model
   • Good quality for most use cases

3. Fallback Mode (Always Works)

   • Simple heightmap-based 3D generation
   • Ensures pipeline never fails
   • Takes 5-10 seconds per model
   • Basic but functional 3D models

Configuration

The pipeline now uses these optimized settings:

# Pipeline configuration
'max_retries': 3,
'timeout': 180,  # 3 minutes for local generation
'enable_caching': True,
'low_vram_mode': False,  # Disabled since you have enough VRAM
'enable_rigging': False  # Disabled by default for speed

# 3D Generation parameters
'num_inference_steps': 30,  # Reduced from 50 for faster generation
'guidance_scale': 7.5,
'resolution': 256,
'generation_timeout': 180  # 3 minutes timeout

Memory Requirements

• Minimum: 8GB RAM + 6GB VRAM
• Recommended: 16GB RAM + 12GB VRAM
• Optimal: 32GB RAM + 24GB VRAM (your current setup)

Features

Enhanced 3D Generation

• Depth-based mesh generation: Creates 3D models from estimated depth maps
• Texture mapping: Applies original image colors to 3D model vertices
• Base stabilization: Adds a stable base to generated models
• Mesh smoothing: Applies smoothing for better visual quality

Robust Error Handling

• Timeout protection: Prevents infinite hangs
• Automatic fallbacks: Seamlessly switches to simpler methods if needed
• Clear logging: Detailed progress and error messages

Performance Optimizations

• Lazy model loading: Models loaded only when needed
• Memory management: Automatic cleanup after each stage
• Threading support: Non-blocking 3D generation

Usage

The pipeline automatically selects the best available method:

# Initialize pipeline
pipeline = MonsterGenerationPipeline(device="cuda")

# Generate with text input
result = pipeline.generate_monster(
    text_input="Create a fire dragon monster",
    user_id="user123"
)

# Generated 3D model will be in result['model_3d']

Troubleshooting

If 3D generation is slow:

1. Check VRAM usage with nvidia-smi
2. Reduce num_inference_steps to 20
3. Use simplified mode by not installing hy3dshape/hy3dpaint

If getting out of memory errors:

1. Enable low_vram_mode in pipeline config
2. Reduce batch size or resolution
3. Use CPU mode (slower but works)

If models look basic:

1. Ensure Hunyuan3D modules are properly installed
2. Check that background removal is working
3. Increase texture_resolution for better quality

Benefits of Direct Model Usage

1. No external dependencies: No reliance on external APIs
2. Faster generation: Local processing is typically faster
3. Full control: Adjust all parameters to your needs
4. Better reliability: No network timeouts or API limits
5. Privacy: All processing happens locally

Next Steps

1. Install the requirements
2. Optionally set up full Hunyuan3D modules
3. Run the pipeline and enjoy fast, local 3D generation!

The system will automatically use the best available method based on what's installed, ensuring you always get a 3D model output.