Upload llama_3.2_3b-lora-qlora-dart-llm GGUF quantized models

.gitattributes

@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+llama_3.2_3b-lora-qlora-dart-llm_q4_k_m.gguf filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,114 @@
+---
+license: llama3.1
+base_model: meta-llama/Llama-3.2-3B
+tags:
+- llama
+- gguf
+- quantized
+- robotics
+- task-planning
+- construction
+- dart-llm
+language:
+- en
+pipeline_tag: text-generation
+---
+
+# Llama 3.2 3B DART LLM - GGUF Quantized Models
+
+This repository contains GGUF quantized versions of the **Llama 3.2 3B DART LLM** model, fine-tuned for robot task planning in construction environments.
+
+## Model Details
+
+- **Base Model**: meta-llama/Llama-3.2-3B
+- **Fine-tuned Version**: Based on QLoRA fine-tuned model for robotics task planning
+- **Format**: GGUF (GPT-Generated Unified Format)
+- **Use Case**: Optimized for inference with llama.cpp and compatible frameworks
+
+## Available Files
+
+- **M**: `llama_3.2_3b-lora-qlora-dart-llm_q4_k_m.gguf` - m quantization
+
+## Usage with llama.cpp
+
+```bash
+# Clone llama.cpp repository
+git clone https://github.com/ggerganov/llama.cpp
+cd llama.cpp
+
+# Build llama.cpp
+make
+
+# Download a quantized model (example with q4_k_m)
+wget https://huggingface.co/YongdongWang/llama-3.2-3b-lora-qlora-dart-llm-gguf/resolve/main/{model_filename}_q4_k_m.gguf
+
+# Run inference
+./main -m {model_filename}_q4_k_m.gguf -p "### Instruction:\nDeploy Excavator 1 to Soil Area 1 for excavation\n\n### Response:\n" -n 512
+```
+
+## Usage with Python (llama-cpp-python)
+
+```python
+from llama_cpp import Llama
+
+# Load model
+llm = Llama(model_path="{model_filename}_q4_k_m.gguf", n_ctx=2048)
+
+# Generate response
+prompt = "### Instruction:\nDeploy Excavator 1 to Soil Area 1 for excavation\n\n### Response:\n"
+output = llm(prompt, max_tokens=512, stop=["</s>"], echo=False)
+
+print(output['choices'][0]['text'])
+```
+
+## Quantization Details
+
+Different quantization levels offer trade-offs between model size, inference speed, and quality:
+
+- **f16**: Full 16-bit precision (largest, highest quality)
+- **q8_0**: 8-bit quantization (good balance of size and quality)
+- **q5_k_m**: 5-bit quantization with mixed precision (recommended)
+- **q4_k_m**: 4-bit quantization (good for most use cases)
+- **q3_k_m**: 3-bit quantization (smaller, some quality loss)
+- **q2_k**: 2-bit quantization (smallest, significant quality loss)
+
+## Performance
+
+The model generates structured JSON task sequences for construction robotics:
+
+```json
+{
+  "tasks": [
+    {
+      "instruction_function": {
+        "dependencies": [],
+        "name": "target_area_for_specific_robots", 
+        "object_keywords": ["soil_area_1"],
+        "robot_ids": ["robot_excavator_01"],
+        "robot_type": null
+      },
+      "task": "target_area_for_specific_robots_1"
+    }
+  ]
+}
+```
+
+## Original Model
+
+This GGUF model is converted from: [YongdongWang/llama-3.2-3b-lora-qlora-dart-llm](https://huggingface.co/YongdongWang/llama-3.2-3b-lora-qlora-dart-llm)
+
+## License
+
+This model inherits the license from the base model (meta-llama/Llama-3.2-3B).
+
+## Citation
+
+```bibtex
+@misc{llama_3.2_3b_lora_qlora_dart_llm_gguf,
+  title={Llama 3.2 3B DART LLM - GGUF Quantized Models},
+  author={YongdongWang},
+  year={2024},
+  publisher={Hugging Face},
+  url={https://huggingface.co/YongdongWang/llama-3.2-3b-lora-qlora-dart-llm-gguf}
+}
+```

llama_3.2_3b-lora-qlora-dart-llm_q4_k_m.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1c00e0b7d9285316e92e0f7276320ff2078bbf4d8fc98a0dc5531f4027948407
+size 2019373248