Create README.md
Browse files
README.md
ADDED
|
@@ -0,0 +1,25 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: apache-2.0
|
| 3 |
+
base_model:
|
| 4 |
+
- Qwen/QwQ-32B
|
| 5 |
+
---
|
| 6 |
+
# Like my work? Support me on patreon for only $5 a month and get to vote on what model's i make next as well as get access to this org's private repo's
|
| 7 |
+
Subscribe bellow:
|
| 8 |
+
- Patreon.com/Rombodawg
|
| 9 |
+
__________________________________________________
|
| 10 |
+
|
| 11 |
+
# Rombo-LLM-V3.1-QWQ-32b
|
| 12 |
+
|
| 13 |
+

|
| 14 |
+
|
| 15 |
+
Rombo-LLM-V3.1-QWQ-32b is a Continued Finetune model (Merge only) of (Qwen/QwQ-32B) and its base model (Qwen/Qwen2.5-32B). This merge is done to decrease catastrophic forgetting during finetuning, and increase overall performance of the model. The tokenizers are taken from the QwQ-32B for thinking capabilities.
|
| 16 |
+
|
| 17 |
+
You can find the Continous Finetuning Doc bellow for more details on the full method:
|
| 18 |
+
|
| 19 |
+
- https://docs.google.com/document/d/1OjbjU5AOz4Ftn9xHQrX3oFQGhQ6RDUuXQipnQ9gn6tU/edit?usp=sharing
|
| 20 |
+
|
| 21 |
+
Quantized model: (coming soon)
|
| 22 |
+
|
| 23 |
+
-
|
| 24 |
+
|
| 25 |
+
Benchmarks: (coming soon)
|