Instructions to use medalpaca/medalpaca-lora-13b-8bit with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use medalpaca/medalpaca-lora-13b-8bit with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="medalpaca/medalpaca-lora-13b-8bit")

# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("medalpaca/medalpaca-lora-13b-8bit", dtype="auto")

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use medalpaca/medalpaca-lora-13b-8bit with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "medalpaca/medalpaca-lora-13b-8bit"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "medalpaca/medalpaca-lora-13b-8bit",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/medalpaca/medalpaca-lora-13b-8bit

SGLang

How to use medalpaca/medalpaca-lora-13b-8bit with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "medalpaca/medalpaca-lora-13b-8bit" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "medalpaca/medalpaca-lora-13b-8bit",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "medalpaca/medalpaca-lora-13b-8bit" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "medalpaca/medalpaca-lora-13b-8bit",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use medalpaca/medalpaca-lora-13b-8bit with Docker Model Runner:
```
docker model run hf.co/medalpaca/medalpaca-lora-13b-8bit
```

MedAlpaca LoRA 13b 8bit

Model Description

Architecture
Training Data
Model Usage
Limitations

Model Description

Architecture

medalpaca-lora-13b-8bit is a large language model specifically fine-tuned for medical domain tasks. It is based on LLaMA (Large Language Model Meta AI) and contains 7 billion parameters. The primary goal of this model is to improve question-answering and medical dialogue tasks. It was trained using LoRA and quantized, to reduce memory footprint.

Training Data

The training data for this project was sourced from various resources. Firstly, we used Anki flashcards to automatically generate questions, from the front of the cards and anwers from the back of the card. Secondly, we generated medical question-answer pairs from Wikidoc. We extracted paragraphs with relevant headings, and used Chat-GPT 3.5 to generate questions from the headings and using the corresponding paragraphs as answers. This dataset is still under development and we believe that approximately 70% of these question answer pairs are factual correct. Thirdly, we used StackExchange to extract question-answer pairs, taking the top-rated question from five categories: Academia, Bioinformatics, Biology, Fitness, and Health. Additionally, we used a dataset from ChatDoctor consisting of 200,000 question-answer pairs, available at https://github.com/Kent0n-Li/ChatDoctor.

Source	n items
ChatDoc large	200000
wikidoc	67704
Stackexchange academia	40865
Anki flashcards	33955
Stackexchange biology	27887
Stackexchange fitness	9833
Stackexchange health	7721
Wikidoc patient information	5942
Stackexchange bioinformatics	5407

Limitations

The model may not perform effectively outside the scope of the medical domain. The training data primarily targets the knowledge level of medical students, which may result in limitations when addressing the needs of board-certified physicians. The model has not been tested in real-world applications, so its efficacy and accuracy are currently unknown. It should never be used as a substitute for a doctor's opinion and must be treated as a research tool only.

Downloads last month: -; Downloads are not tracked for this model. How to track

Space using medalpaca/medalpaca-lora-13b-8bit 1

Papers for medalpaca/medalpaca-lora-13b-8bit

ChatDoctor: A Medical Chat Model Fine-tuned on LLaMA Model using Medical Domain Knowledge

Paper • 2303.14070 • Published Mar 24, 2023 • 10

LoRA: Low-Rank Adaptation of Large Language Models

Paper • 2106.09685 • Published Jun 17, 2021 • 61

medalpaca
/

medalpaca-lora-13b-8bit

MedAlpaca LoRA 13b 8bit

Table of Contents

Model Description

Architecture

Training Data

Limitations

Space using medalpaca/medalpaca-lora-13b-8bit 1

Papers for medalpaca/medalpaca-lora-13b-8bit

ChatDoctor: A Medical Chat Model Fine-tuned on LLaMA Model using Medical Domain Knowledge

LoRA: Low-Rank Adaptation of Large Language Models