Instructions to use Quant-Cartel/MilkDropLM-32b-v0.3-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Quant-Cartel/MilkDropLM-32b-v0.3-GGUF with llama-cpp-python:

# !pip install llama-cpp-python

from llama_cpp import Llama

llm = Llama.from_pretrained(
	repo_id="Quant-Cartel/MilkDropLM-32b-v0.3-GGUF",
	filename="MilkDropLM-32b-v0.3-Q4_K_M.gguf",
)

llm.create_chat_completion(
	messages = [
		{
			"role": "user",
			"content": "What is the capital of France?"
		}
	]
)

Notebooks
Google Colab
Kaggle
Local Apps

llama.cpp

How to use Quant-Cartel/MilkDropLM-32b-v0.3-GGUF with llama.cpp:

Install from brew

brew install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf Quant-Cartel/MilkDropLM-32b-v0.3-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama-cli -hf Quant-Cartel/MilkDropLM-32b-v0.3-GGUF:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf Quant-Cartel/MilkDropLM-32b-v0.3-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama-cli -hf Quant-Cartel/MilkDropLM-32b-v0.3-GGUF:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf Quant-Cartel/MilkDropLM-32b-v0.3-GGUF:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf Quant-Cartel/MilkDropLM-32b-v0.3-GGUF:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf Quant-Cartel/MilkDropLM-32b-v0.3-GGUF:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf Quant-Cartel/MilkDropLM-32b-v0.3-GGUF:Q4_K_M

Use Docker

docker model run hf.co/Quant-Cartel/MilkDropLM-32b-v0.3-GGUF:Q4_K_M

LM Studio
Jan

vLLM

How to use Quant-Cartel/MilkDropLM-32b-v0.3-GGUF with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Quant-Cartel/MilkDropLM-32b-v0.3-GGUF"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Quant-Cartel/MilkDropLM-32b-v0.3-GGUF",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/Quant-Cartel/MilkDropLM-32b-v0.3-GGUF:Q4_K_M

Ollama
How to use Quant-Cartel/MilkDropLM-32b-v0.3-GGUF with Ollama:
```
ollama run hf.co/Quant-Cartel/MilkDropLM-32b-v0.3-GGUF:Q4_K_M
```

Unsloth Studio new

How to use Quant-Cartel/MilkDropLM-32b-v0.3-GGUF with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for Quant-Cartel/MilkDropLM-32b-v0.3-GGUF to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for Quant-Cartel/MilkDropLM-32b-v0.3-GGUF to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for Quant-Cartel/MilkDropLM-32b-v0.3-GGUF to start chatting

Pi new

How to use Quant-Cartel/MilkDropLM-32b-v0.3-GGUF with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama-server -hf Quant-Cartel/MilkDropLM-32b-v0.3-GGUF:Q4_K_M

Configure the model in Pi

# Install Pi:
npm install -g @mariozechner/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "Quant-Cartel/MilkDropLM-32b-v0.3-GGUF:Q4_K_M"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Hermes Agent new

How to use Quant-Cartel/MilkDropLM-32b-v0.3-GGUF with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama-server -hf Quant-Cartel/MilkDropLM-32b-v0.3-GGUF:Q4_K_M

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default Quant-Cartel/MilkDropLM-32b-v0.3-GGUF:Q4_K_M

Run Hermes

hermes

Docker Model Runner
How to use Quant-Cartel/MilkDropLM-32b-v0.3-GGUF with Docker Model Runner:
```
docker model run hf.co/Quant-Cartel/MilkDropLM-32b-v0.3-GGUF:Q4_K_M
```

Lemonade

How to use Quant-Cartel/MilkDropLM-32b-v0.3-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull Quant-Cartel/MilkDropLM-32b-v0.3-GGUF:Q4_K_M

Run and chat with the model

lemonade run user.MilkDropLM-32b-v0.3-GGUF-Q4_K_M

List all available models

lemonade list

  e88 88e                               d8     
 d888 888b  8888 8888  ,"Y88b 888 8e   d88     
C8888 8888D 8888 8888 "8" 888 888 88b d88888   
 Y888 888P  Y888 888P ,ee 888 888 888  888     
  "88 88"    "88 88"  "88 888 888 888  888     
      b                                        
      8b,                                      
 
  e88'Y88                  d8           888    
 d888  'Y  ,"Y88b 888,8,  d88    ,e e,  888    
C8888     "8" 888 888 "  d88888 d88 88b 888    
 Y888  ,d ,ee 888 888     888   888   , 888    
  "88,d88 "88 888 888     888    "YeeP" 888    
                                               
PROUDLY PRESENTS

⚠️ Epilepsy Warning: This model may generate visuals with flashing lights. Viewer discretion is advised for those with photosensitive epilepsy.

MilkDropLM-32b-v0.3

Building upon the success of our 7b model release, we're thrilled to introduce MilkDropLM-32b-v0.3, the latest iteration of our state-of-the-art language model designed specifically for creating visually stunning MilkDrop presets. This new 32b model is based on the Qwen2.5-Coder-32B-Instruct model and has been fine-tuned with the same tried and tested hyperparameters that made our 7b release function smoothly.

The future is here… so don’t be surprised if the next amazing concert visual you see was authored by AI!

Notable Enhancements

Unraveling the MilkDrop Genome: MilkDropLM-32b-v0.3 has a more nuanced grasp of the intricate relationships between different elements within presets, allowing for far more accurate and creative generations.
Seamless Enhancements: This new model can now “upgrade” the presets that were generated with the 7b model, breathing new life into your favorite visuals. Try prompting with any existing MilkDrop preset script and asking for variations (at least 16k context size minimum for this feature).
Fewer Loops, More Presets: We've made significant strides in reducing the likelihood of the model getting stuck in loops, ensuring that you can focus on what matters most – creating stunning visuals.
Smooth Operator: Engage in more natural-sounding conversations with MilkDropLM-32b-v0.3, as it responds to your requests in a more human-like way than ever before.

What's Changed Under the Hood?

While we've built upon the solid foundation of our hand curated and highly organized dataset containing 10,000+ best MilkDrop presets, we've made several key changes to the training process to address the increased requirements of the 32b model. In particular we've doubled down on the training time, pushing the model to its limits with 2 full epochs of training, coming out to approximately 48 hours on an A100 GPU. We also maintained the same generous maximum sequence length used in the 7b model during training to ensure that even the longest presets were accounted for in the training corpus.

Full Precision Weights (Not required for inference)

bf16

Get Started with MilkDropLM-32b-v0.3

Ready to unlock the full potential of MilkDropLM-32b-v0.3? Unlike the 7b model, the 32b edition works out of the box with a variety of temperature and sampler configs, even default settings.

For starters we find that the default temperature of 1.0 works great. But just like before you can lower the temperature to increase the output quality at the cost of creativity.

In terms of context length, we would recommend the default maximum (32,768). We understand that this model’s size is quite large, and so to ease VRAM requirements you can lower the context length to meet your compute.

Min: 8192 /// Minimum requirement. Enough to output most presets once
Regular: 16384 /// Allows for having up to 2 presets in ‘memory’
Max: 32768 /// Allows for 3-4 Presets in ‘memory’, recommended

Also make sure to max out your output length to prevent the model from stopping short and having to manually ‘continue’ the response.

After you've generated a MilkDrop preset, copy-and-paste it into a text file, and then save it using the .milk file format. Move the .milk file into the presets folder of your Milkdrop app. We recommend using the NestDrop Classic app, which is freely available. (Make sure to close and reopen NestDrop to see your newly added presets.)

Text Prompt Template

In terms of the approach to text prompting with this model, the classic “Give me a Glowsticks milkdrop preset” or “Make a milkdrop preset with [x], [y], [z]” still works very well. But feel free to experiment with brand new ways to ask the 32b model for presets to take advantage of its new conversational capabilities and the results may surprise you! Below is the full list of preset categories/subcategories that this model was trained on.

Dancer /// Aurora, Blobby Mirror, Blobby, Comet Mirror, Comet, Glowsticks Fast, Glowsticks Mirror, Glowsticks, Hatches Mirror, Hatches, Infect Mirror, Infect, Jello Mirror, Lasers, Murky Mirror, Murky, Nexus Mirror, Orbit Mirror, Orbit, Petals, Serpent, Shapes, Spinner Mirror, Spinner, Streamers, Swarm, Tree Branch, Wake Mirror, Wake, Whirl Mirror, Whirl, Wire
Drawing /// Dunes Mirror, Dunes, Explosions Mirror, Explosions, Feedback, Fractal Mirror, Fractal, Glimmer Mirror, Growth Mirror, Growth, Lasers, Liquid Mirror, Liquid, Maze, Rorschach Mirror, Rorschach, Trails Mirror, Trails, Viscera, Whirlpools Mirror, Whirlpools
Fractal /// Blobby, Core Mirror, Core Tunnel, Core, Grid, Horizon Mirror, Lattice Mirror, Lattice, Loops, Mandelbox, Nested Circle, Nested Dancer, Nested Ellipse, Nested Hexagon, Nested Pyramid, Nested Spiral Multiple, Nested Spiral, Nested Square, Nested Triangle, Pointy, Radial, Shine, Sierpinski, Trees, Wave Interference, Wings, Womb
Geometric /// Cathedral, Circles Nested, City, Cube Array, Cube Fly, Cube, Dots, Gears, Honeycomb, Landscape, Monster, Pyramids, Snowflakes, Sphere Array, Sphere Particles, Sphere Wild, Spiral Bounce, Squares Glass, Squares, Stripes Circle, Stripes Dance, Stripes Liquid, Stripes, Symbols, Torus Interior, Triangles Solo, Tunnel Fans, Tunnel Morph, Tunnel Spheres, Wire Circles, Wire Cube Trace, Wire Flower, Wire Grid, Wire Morph, Wire Orbits, Wire Parallel, Wire Sphere, Wire Spiral Flower, Wire Spiral, Wire Torus, Wire Trace Mirror, Wire Trace
Hypnotic /// Illusion Radiate, Illusion, Polar Closeup, Polar Mirror, Polar Rolling, Polar Static, Polar Warp, Radial Warp
Particles /// Blobby, Crystal, Grid, Orbit, Points Fast, Points Trails, Points, Swarm
Reaction /// Automata, Cloudy, Contagion, Crystalize, Dunes, Feedback, Growth, Liquid Blobby, Liquid Closeup, Liquid Gradient, Liquid Ripples, Liquid Simmering, Liquid Windy, Luma Mirror, Maze, Mountains, Rorschach, Viscera, Whirlpools, Windy
Sparkle /// Explosions, Glimmer Mirror, Glimmer Tunnel, Glimmer, Jewel, Mass Circles, Mass Squares, Mass Stars, Mass Triangles, Squares
Supernova /// Burst, Gas, Lasers, Orbits, Radiate, Shimmer, Stars
Waveform /// Spectrum, Wire Circular, Wire Flat Double, Wire Flat, Wire Flower, Wire Mirror, Wire Rising, Wire Spirograph, Wire Tangle, Wire Tunnel

Alpha Release Notice

As with any new model release, we want to emphasize that MilkDropLM-32b-v0.3 is still in the alpha stage of development and we’re actively running experiments to see what it’s capable of. While we're confident that this model represents a significant leap forward, we're not done yet. We encourage you to approach MilkDropLM-32b-v0.3 with a spirit of experimentation and discovery, and we can't wait to see what amazing visuals you'll create!

Acknowledgements

This project is the result of a collaboration between ISOSCELES and InferenceIllusionist. This was a unique meeting of minds since ISOSCELES brought his MilkDrop preset knowledge and experience in helping develop NestDrop for the VJ community, and InferenceIllusionist brought his vital experience in fine-tuning and quantizing LLMs. We stand on the shoulders of the many Milkdrop authors which have freely released their original presets for everyone to enjoy. Much respect!

We would like to express our deepest gratitude towards our growing community of alpha testers and feedback providers for their invaluable insights and support throughout this development process. We truly appreciate your pioneer spirit and courage in embracing this new family of Large Language Models.

Shoutout to Unsloth as well for providing the tools used for this fine-tune.