chendren/deepseek-dnd-lora

DeepSeek-R1-Distill-Qwen-7B Fine-tuned for Dungeons & Dragons

This model is a fine-tuned version of deepseek-ai/DeepSeek-R1-Distill-Qwen-7B specifically trained on Dungeons & Dragons content. The model is designed to excel at creating D&D characters, adventures, and other D&D-related content.

Model Details

• Base Model: deepseek-ai/DeepSeek-R1-Distill-Qwen-7B
• Fine-tuning Method: LORA (Parameter-Efficient Fine-Tuning)
• LoRA Rank: 8
• LoRA Alpha: 16
• Target Modules: q_proj, k_proj, v_proj, o_proj
• Training Date: 2025-05-11
• Dataset Size: 500 examples from a curated D&D dataset

Usage

This is a LoRA adapter that needs to be combined with the base model to work. Here's how to use it:

Using the Transformers Library

from huggingface_hub import snapshot_download
import torch
from peft import PeftModel, PeftConfig
from transformers import AutoModelForCausalLM, AutoTokenizer

# Load the base model and tokenizer
base_model_id = "deepseek-ai/DeepSeek-R1-Distill-Qwen-7B"
model = AutoModelForCausalLM.from_pretrained(
    base_model_id,
    torch_dtype=torch.float16,
    device_map="auto",
    use_auth_token=True  # If you're using a private model
)
tokenizer = AutoTokenizer.from_pretrained(base_model_id)

# Load the LoRA adapter
adapter_model_id = "chendren/deepseek-dnd-lora"
model = PeftModel.from_pretrained(
    model, 
    adapter_model_id,
    use_auth_token=True  # If you're using a private model
)

# Test generation
prompt = "Create a D&D character with the following details: Race: Half-Elf, Class: Bard, Background: Entertainer"
inputs = tokenizer(f"You are a Dungeons & Dragons assistant. {prompt}", return_tensors="pt").to(model.device)

outputs = model.generate(
    input_ids=inputs["input_ids"],
    attention_mask=inputs["attention_mask"],
    max_new_tokens=500,
    temperature=0.7,
    top_p=0.9,
    top_k=50,
    repetition_penalty=1.1,
    do_sample=True
)

response = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(response)

Using the Hugging Face Inference API

You can also use this model directly with the Inference API:

import requests

API_URL = "https://api-inference.huggingface.co/models/chendren/deepseek-dnd-lora"
headers = {"Authorization": "Bearer YOUR_API_TOKEN"}

def query(payload):
    response = requests.post(API_URL, headers=headers, json=payload)
    return response.json()

output = query({
    "inputs": "You are a Dungeons & Dragons assistant. Create a D&D character with the following details: Race: Half-Elf, Class: Bard, Background: Entertainer",
    "parameters": {
        "max_new_tokens": 500,
        "temperature": 0.7,
        "top_p": 0.9,
        "top_k": 50,
        "repetition_penalty": 1.1,
        "do_sample": True
    }
})

Example Outputs

When prompted to create a D&D character with specific details, the model will generate a complete character sheet with attributes, skills, background story, and more. For example:

Prompt: Create a D&D character with the following details: Race: Half-Elf, Class: Bard, Background: Entertainer

Output: [The model will generate a detailed character sheet including attributes, skills, spells, personality traits, and background story for a Half-Elf Bard with the Entertainer background]

Training

This model was fine-tuned using the following hyperparameters:

• Learning rate: 5e-5
• Epochs: 1
• Batch size: 1
• Gradient accumulation steps: 4
• Maximum sequence length: 256

License

This model inherits the license of the base model, deepseek-ai/DeepSeek-R1-Distill-Qwen-7B.