🧠 LLaMA 3.2-1B - MedQA LoRA Merged (Chat v1)
A merged 🧬 LoRA fine-tuned version of meta-llama/Llama-3.2-1B, optimized for medical domain question answering using the MedQA dataset.
💡 Fine-tuned and merged by Team 9 for high-quality semantic answers in the healthcare domain.
📊 Evaluation — BERTScore (on MedQA test set, 100 samples)
| Metric | Score |
|---|---|
| Precision | 0.8436 |
| Recall | 0.8774 |
| F1 Score | 0.8598 |
🛠️ Model Details
- Base Model:
meta-llama/Llama-3.2-1B - Dataset:
ChakradharS/clean-medqa - Fine-tuning: LoRA (merged into base weights)
- Merged Model Size: Full Causal LM weights (no adapter needed)
- Tokenizer Padding: Right;
pad_token = eos_token - LoRA Targets:
q_proj,k_proj,v_proj,o_proj,gate_proj,up_proj,down_proj
🚀 Inference Example
from transformers import AutoTokenizer, AutoModelForCausalLM, pipeline
model_id = "ChakradharS/llama3-1b-medqa-lora-merged-chat-v1"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, device_map="auto", torch_dtype="auto")
pipe = pipeline(
"text-generation",
model=model,
tokenizer=tokenizer,
device_map="auto",
max_new_tokens=100,
do_sample=False,
temperature=0.7,
top_p=0.9,
top_k=50,
repetition_penalty=1.1,
no_repeat_ngram_size=3
)
prompt = "What are the symptoms of a mild heart attack?"
print(pipe(prompt)[0]["generated_text"])
- Downloads last month
- -