Instructions to use Nitral-AI/Captain-Eris_Violet-V0.420-12B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Nitral-AI/Captain-Eris_Violet-V0.420-12B with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="Nitral-AI/Captain-Eris_Violet-V0.420-12B")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("Nitral-AI/Captain-Eris_Violet-V0.420-12B")
model = AutoModelForCausalLM.from_pretrained("Nitral-AI/Captain-Eris_Violet-V0.420-12B")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Inference
Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use Nitral-AI/Captain-Eris_Violet-V0.420-12B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Nitral-AI/Captain-Eris_Violet-V0.420-12B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Nitral-AI/Captain-Eris_Violet-V0.420-12B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/Nitral-AI/Captain-Eris_Violet-V0.420-12B

SGLang

How to use Nitral-AI/Captain-Eris_Violet-V0.420-12B with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Nitral-AI/Captain-Eris_Violet-V0.420-12B" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Nitral-AI/Captain-Eris_Violet-V0.420-12B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Nitral-AI/Captain-Eris_Violet-V0.420-12B" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Nitral-AI/Captain-Eris_Violet-V0.420-12B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use Nitral-AI/Captain-Eris_Violet-V0.420-12B with Docker Model Runner:
```
docker model run hf.co/Nitral-AI/Captain-Eris_Violet-V0.420-12B
```

Smart and has emotional depth

by GhostGate - opened Dec 17, 2024

Discussion

GhostGate

Dec 17, 2024

Hi!

I tried the model and I have to say that it is smart as well as emotional when it needs to be. It adheres to the instructions really well (might be Violet DNA which does this), but also has a tendency to set boundaries when needed to. One thing I noticed is that it has some "forgetful moments". I had to enforce some clothing items into the notes, so that the model doesn't forget about them.
Overall, I think it's a really good addition and an improvement to BMO, which I liked to begin with.

Nitral-AI

Owner Dec 17, 2024

Hi!

I tried the model and I have to say that it is smart as well as emotional when it needs to be. It adheres to the instructions really well (might be Violet DNA which does this), but also has a tendency to set boundaries when needed to. One thing I noticed is that it has some "forgetful moments". I had to enforce some clothing items into the notes, so that the model doesn't forget about them.
Overall, I think it's a really good addition and an improvement to BMO, which I liked to begin with.

Appreciate the feedback, you might have some luck using the tracker extension for dealing with stuff like clothes/items/inventory if thats something it seems to struggle with on your end. I'm considering adding additional entries to some of the sets to improve on this behavior in the future.

GhostGate

Dec 17, 2024

Awesome! Looking forward to it. I think BMO and this are my favourite Nemo finetunes thus far. I like that Violet seems to kick the plot forward or add twists on it's own (BMO had a hard time with that).

Nitral-AI changed discussion status to closed Mar 21, 2025

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment