Instructions to use chargoddard/rpguild-chatml-13b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use chargoddard/rpguild-chatml-13b with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="chargoddard/rpguild-chatml-13b")

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("chargoddard/rpguild-chatml-13b")
model = AutoModelForCausalLM.from_pretrained("chargoddard/rpguild-chatml-13b")

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use chargoddard/rpguild-chatml-13b with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "chargoddard/rpguild-chatml-13b"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "chargoddard/rpguild-chatml-13b",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/chargoddard/rpguild-chatml-13b

SGLang

How to use chargoddard/rpguild-chatml-13b with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "chargoddard/rpguild-chatml-13b" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "chargoddard/rpguild-chatml-13b",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "chargoddard/rpguild-chatml-13b" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "chargoddard/rpguild-chatml-13b",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use chargoddard/rpguild-chatml-13b with Docker Model Runner:
```
docker model run hf.co/chargoddard/rpguild-chatml-13b
```
Browse Quantizations to use this model in llama.cpp, Ollama, LM Studio, or any compatible app.

It's a roleplay model! Trained on ~50k examples of human written roleplay sessions. Not a hint of gpt-flavor writing. Very creative and very able to roll with the punches.

Does tend to speak for you more than it should and has a very, very strong bias towards speech in quotes.

Uses ChatML prompt format, with optional length suffixes:

<|im_start|>system
Enter roleplay mode. You are Steve.

Steve is a nasty little man and solves all his problems by punching people in the face.<|im_end|>
<|im_start|>Gundalf, Gandalf's cousin with a gun
Gundalf runs frantically up to Steve and begins yelling. "We have a big problem, Steve!"<|im_end|>
<|im_start|>Steve (Length: short)
"What is it now Gundalf? Do you want me to punch you in the face?" Steve asked, looking down at him. He was pretty sure that this guy wasn't worth the trouble of punching him in the face, but he didn't like to leave loose ends hanging either.

"There's been an incident," Gundalf said quickly, "A dwarf has stolen some gold from Smaug."

"Well I don't care about any of those things," Steve responded, turning away. <|im_end|>

Downloads last month: 1,018

Model tree for chargoddard/rpguild-chatml-13b

Quantizations

4 models

Dataset used to train chargoddard/rpguild-chatml-13b

Collection including chargoddard/rpguild-chatml-13b

Usable Models

Collection

5 items • Updated Dec 10, 2023 • 2