doguilmak/inferencevision-pythia-1B

Model Card for InferenceVision QA Fine-Tuned Model

Model Description

This model is a fine-tuned variant of the EleutherAI/pythia-1b causal language model, specifically adapted to handle interactive question-answering over the InferenceVision documentation. By leveraging domain-specific question–answer pairs, the model has learned to produce precise, contextually relevant responses, making it an ideal backbone for developer assistants, chatbots, and documentation-driven interfaces.

Intended Use

• Primary Use: Provide accurate, documentation-based answers to user queries about InferenceVision.

• Use Cases: Integration into chat applications, developer portals, knowledge retrieval systems, and automated support bots.

For a hands-on guide on fine-tuning and using this model with InferenceVision, check out the interactive notebook.

Out-of-Scope:

• Legal, medical, or financial advice beyond the scope of InferenceVision documentation.

• Generating content unrelated to the provided training material.

Training Data

The model was fine-tuned on a custom dataset inferencevision_docs.jsonl, containing 760 high-quality question–answer pairs sourced directly from InferenceVision’s official documentation. These QA pairs span key areas such as:

• Installation & Setup: Commands, environment requirements, and troubleshooting guidelines.
• Core API Usage: Function parameters, input/output formats, and typical usage scenarios.
• Advanced Features: Batch processing workflows, performance optimization tips, and integration examples.
• Error Handling: Common error codes, explanations, and recommended solutions.

Preprocessing Steps:

1. Deduplication & Cleanup: Eliminated duplicate or near-duplicate entries to prevent bias.
2. Tokenization: Employed the EleutherAI/pythia-1b’s byte-pair encoding with a maximum sequence length of 2,048 tokens.
3. Context Windowing: For multipart questions, context segments were extracted to ensure both the query and relevant documentation snippet fit within the model’s context window.
4. Quality Validation: Automated checks and manual reviews removed any QA pairs with unclear or incomplete answers.

The dataset was split into an 80% training set (608 examples) and a 20% evaluation set (152 examples), using stratified sampling to preserve topic distribution across both splits.

Training Procedure & Hyperparameters

Fine-tuning was performed using Hugging Face’s Trainer API with the following TrainingArguments:

training_args = TrainingArguments(
    output_dir="./results",
    evaluation_strategy="epoch",
    save_strategy="epoch",
    learning_rate=1e-5,
    per_device_train_batch_size=8,
    per_device_eval_batch_size=8,
    num_train_epochs=16,
    weight_decay=0.01,
    logging_dir="./logs",
    load_best_model_at_end=True,
    save_total_limit=1,
    metric_for_best_model="eval_loss",
    greater_is_better=False
)

Training leveraged GPU acceleration when available. By saving only the best checkpoint (based on lowest eval_loss), storage requirements were minimized without sacrificing model quality.

Evaluation Results

After 16 epochs, the training process yielded the following key outcomes:

• Global Steps: 1,216
• Final Training Loss: 0.03725
• Epochs Completed: 16.0
• Training Runtime: 2,572.28 seconds (trained on an NVIDIA A100 40GB GPU and took ~42.9 minutes)
• Training Throughput: 3.78 samples/sec, 0.47 steps/sec
• Total FLOPs: 2.72×10¹⁶

Limitations & Biases

• Although highly accurate on InferenceVision topics, the model may generate plausible but incorrect or outdated information if presented with out-of-distribution queries.

• Context length is limited to 2,048 tokens; very long or multi-turn contexts may require special handling.

Users should validate critical outputs against official documentation.

Inference Provider

This section provides a simple way to run inference using the fine-tuned doguilmak/inferencevision-pythia-1B model. It uses Hugging Face Transformers to load the model and generate answers for InferenceVision-related questions. The model is optimized for domain-specific QA and works best when given clear queries or documentation snippets.

from transformers import AutoTokenizer, AutoModelForCausalLM
import torch

model_name = "doguilmak/inferencevision-pythia-1B"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name)
model.eval()

device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
model.to(device)

def ask_question(question, context=None, max_new_tokens=100):
    if context:
        prompt = f"Context: {context}\n\nQuestion: {question}\nAnswer:"
    else:
        prompt = f"Question: {question}\nAnswer:"
    
    inputs = tokenizer(prompt, return_tensors="pt").to(device)

    with torch.no_grad():
        outputs = model.generate(
            **inputs,
            max_new_tokens=max_new_tokens,
            temperature=0.7,
            top_p=0.95,
            do_sample=False,
            pad_token_id=tokenizer.eos_token_id
        )

    answer = tokenizer.decode(outputs[0], skip_special_tokens=True)
    return answer.replace(prompt, "").strip()

question = "What is InferenceVision?"
answer = ask_question(question)
print("Answer:", answer)

Reference

Biderman, S., Schoelkopf, H., Anthony, Q. G., Bradley, H., O’Brien, K., Hallahan, E., ... & Van Der Wal, O. (2023, July). Pythia: A suite for analyzing large language models across training and scaling. In International Conference on Machine Learning (pp. 2397-2430). PMLR. https://arxiv.org/abs/2304.01373

This paper introduces Pythia, a suite of 16 large language models (LLMs) trained on public data in the same order, ranging from 70M to 12B parameters. The suite provides 154 checkpoints per model and tools to reconstruct training dataloaders, facilitating research in areas such as memorization, term frequency effects on few-shot performance, and reducing gender bias.