Theta-35-mini

A lightweight, high-efficiency reasoning model distilled from Theta-35. Theta-35-Mini is a compact 3B parameter language model developed by SVECTOR, built on the Qwen architecture and trained using Group Relative Policy Optimization (GRPO). It is the smaller sibling of our flagship Theta-35 model (33B parameters), offering efficient performance for resource-constrained environments.

---

🔍 Overview

• Architecture: Based on Qwen2-style transformer blocks
• Training Objective: Autoregressive next-token prediction
• Technique: Trained using Group Relative Policy Optimization (GRPO) – a reinforcement learning optimization strategy enabling fine-grained control and alignment
• Size: 3 billion parameters
• Parent Model: Theta-35 (33B)

🚀 Model Highlights

• ✅ Compact and Capable: Achieves strong performance despite its small size
• ⚙️ GRPO-trained: Trained with Group Relative Policy Optimization for better alignment, coherence, and efficiency
• 💡 Low-latency Inference: Ideal for edge and on-device applications

📦 How to Use

Install dependencies:

pip install transformers

Run model in Python:

from transformers import AutoModelForCausalLM, AutoTokenizer

# Force use of the slow tokenizer to avoid tokenizer.json issues
tokenizer = AutoTokenizer.from_pretrained("SVECTOR-CORPORATION/Theta-35-Mini", use_fast=False)
model = AutoModelForCausalLM.from_pretrained("SVECTOR-CORPORATION/Theta-35-Mini")

inputs = tokenizer("Once upon a time", return_tensors="pt")

outputs = model.generate(**inputs, max_length=100, temperature=0.7)

print(tokenizer.decode(outputs[0], skip_special_tokens=True))

---

📄 License

This model is released under the MIT License.

---

🏢 About SVECTOR

🔗 Visit us at svector.co.in

---

🙌 Acknowledgements

• DeepSeek GRPO Paper
• Qwen2 Architecture

---