kirubel1738/biogpt-bioqa-lora-merged

Model Card for Model ID

This model, biogpt-bioqa-lora-merged, is a fine-tuned version of the large BioGPT model, specialized for high-accuracy Question Answering (QA) in the biomedical domain. It was trained using Low-Rank Adaptation (LoRA) to efficiently adapt the foundational BioGPT knowledge to complex, specific biomedical and clinical inquiries.

Model Details

Model Description

This model is a specialized derivative of the Microsoft BioGPT model. It excels at complex recall and synthesis tasks across biology, pharmacology, and clinical knowledge. The model combines the extensive pre-training of BioGPT with a custom, targeted dataset to improve QA performance, especially in handling multi-step biological processes and retrieving specific medical facts.

• Developed by: kirubel1738
• Model type: Causal Language Model (Text Generation / Question Answering)
• Language(s) (NLP): English
• License: Apache 2.0
• Finetuned from model : microsoft/BioGPT-Large

Training Details

• Finetuning Method: Low-Rank Adaptation (LoRA)
• Training Dataset: kirubel1738/biogpt-bioqa-combined
• Purpose of Finetuning: To improve factual accuracy, reduce verbosity, and enhance the model's ability to answer complex, high-stakes biomedical questions, including clinical scenarios.

Uses

Primary Use

This model is primarily intended for use in academic and research settings for:

• Biomedical Question Answering: Extracting facts and explanations from complex biological and medical queries.
• Scientific Literature Review: Assisting researchers in summarizing or synthesizing information related to genes, diseases, and drug mechanisms.
• Educational Tool: Providing accurate, detailed explanations of core concepts in biology and pharmacology.

How to Use

The model can be loaded and used via the Hugging Face transformers library for text generation. It works best when provided a clear question followed by a separator.

Python Code Example (Inference)

from transformers import AutoTokenizer, AutoModelForCausalLM

model_name = "kirubel1738/biogpt-bioqa-lora-merged" tokenizer = AutoTokenizer.from_pretrained(model_name) model = AutoModelForCausalLM.from_pretrained(model_name)

Example Prompt prompt = "Q: What specific amino acid substitution causes Sickle Cell Anemia?" Tokenize and generate inputs = tokenizer(prompt, return_tensors="pt") outputs = model.generate(**inputs, max_length=150, num_return_sequences=1, do_sample=False) Decode and print answer = tokenizer.decode(outputs[0], skip_special_tokens=True) print(answer) Expected output begins: Q: What specific amino acid substitution causes Sickle Cell Anemia? A: Glutamic acid to Valine at the sixth position of the beta-globin chain...

from transformers import AutoTokenizer, AutoModelForCausalLM
import torch

# Specify the model ID
model_id = "kirubel1738/biogpt-bioqa-lora-merged"

# Load tokenizer and model
tokenizer = AutoTokenizer.from_pretrained(model_id)
# Note: device_map="auto" loads the model efficiently across available devices (CPU/GPU)
model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype=torch.float16, device_map="auto")

# The prompt structure Q: [Question]? A: is often used in QA fine-tuning.
# Ensure your prompt ends in 'A:' to trigger the model to generate the answer.
prompt = "Q: What specific amino acid substitution causes Sickle Cell Anemia? A:"

# Tokenize and generate
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(
    **inputs,
    max_new_tokens=100,
    num_return_sequences=1,
    do_sample=False,
    pad_token_id=tokenizer.eos_token_id
)

# Decode and print the full output (input + generation)
generated_text = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(generated_text)
# Expected output begins: Q: What specific amino acid substitution causes Sickle Cell Anemia? A: Glutamic acid to Valine at the sixth position of the beta-globin chain...