finiarisab/tamgpt_1_0

TamGPT-1.0

Model Details

Model Description

TamGPT-1.0 is a fine-tuned instruction-following language model specialized in engineering and cost intelligence. Built upon Qwen3.5-9B, this model is optimized to provide expert advice on project management metrics, cost performance analysis, and engineering decision-making.

• Developed by: TamGPT Team
• Funded by: Open-source community contribution
• Shared by: finiarisab
• Model type: Causal Language Model (Autoregressive)
• Language(s): English (primary), Tamfis Nig. (limited)
• License: Apache 2.0
• Finetuned from model: Qwen/Qwen3.5-9B
• Quantization: 4-bit (NF4) with double quantization

Model Sources

• Repository: https://huggingface.co/finiarisab/TamGPT-1.0
• Space: https://huggingface.co/spaces/finiarisab/tamgpt-trainer
• Base Model: https://huggingface.co/Qwen/Qwen3.5-9B

Uses

Direct Use

TamGPT-1.0 is designed for:

• Engineering project management consultation
• Cost performance analysis (CPI, SPI interpretation)
• Budget variance analysis and recommendations
• Risk assessment for engineering projects
• Cost intelligence and forecasting
• Technical decision support

Out-of-Scope Use

• Medical diagnosis or healthcare advice
• Legal counsel or document interpretation
• Financial trading or investment recommendations
• Military or weapons systems
• Automated decision-making without human oversight
• Generation of harmful or malicious content

Bias, Risks, and Limitations

Known Limitations

1. Domain Expertise: While specialized in engineering metrics, the model may not have deep expertise in all engineering subfields
2. Training Data Size: Fine-tuned on only 300 examples, which may limit generalization
3. Hallucination Risk: May generate plausible-sounding but incorrect engineering advice
4. Quantization Effects: 4-bit quantization may reduce accuracy compared to full precision
5. Context Length: Limited to 1024 tokens

Recommendations

• Always verify critical engineering decisions with domain experts
• Use as a decision support tool, not as the sole decision maker
• Provide clear context and metrics when querying the model
• Implement human-in-the-loop for high-stakes decisions

How to Get Started with the Model

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

# Load base model and tokenizer
MODEL_NAME = "Qwen/Qwen3.5-9B"
ADAPTER_PATH = "finiarisab/TamGPT-1.0"

tokenizer = AutoTokenizer.from_pretrained(MODEL_NAME, trust_remote_code=True)
if tokenizer.pad_token is None:
    tokenizer.pad_token = tokenizer.eos_token

# Load base model
base_model = AutoModelForCausalLM.from_pretrained(
    MODEL_NAME,
    device_map="auto",
    torch_dtype=torch.float16,
    trust_remote_code=True
)

# Load LoRA adapter
model = PeftModel.from_pretrained(base_model, ADAPTER_PATH)
model.eval()

# Example usage
messages = [
    {
        "role": "system",
        "content": "You are TamGPT, an elite engineering and cost intelligence AI."
    },
    {
        "role": "user",
        "content": "CPI is 0.82 and SPI is 0.91. What should I do?"
    }
]

text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(text, return_tensors="pt").to(model.device)

with torch.no_grad():
    outputs = model.generate(
        **inputs,
        max_new_tokens=300,
        temperature=0.7,
        top_p=0.9,
        do_sample=True,
        pad_token_id=tokenizer.pad_token_id,
        eos_token_id=tokenizer.eos_token_id,
    )

response = tokenizer.decode(outputs[0], skip_special_tokens=True)
if "assistant" in response:
    response = response.split("assistant")[-1].strip()
print(response)
Training Details
Training Data
Dataset: Custom engineering and cost intelligence dataset

Format: JSONL with chat conversations

Size: 300 training examples

Content: Engineering scenarios, CPI/SPI analysis, cost management questions

Training Hyperparameters
Batch size: 1 (per device)

Gradient accumulation steps: 8 (effective batch size = 8)

Learning rate: 2e-4

Optimizer: paged_adamw_8bit

LR scheduler: Cosine with warmup ratio 0.03

Epochs: 3

Precision: bfloat16

Gradient checkpointing: Enabled

Max gradient norm: 0.3

Training Results
Initial loss: 1.994

Final loss: 0.105

Average loss: 0.2954

Training time: ~50 minutes

Trainable parameters: 29,097,984 (0.32% of total)

LoRA Configuration
Rank (r): 16

Alpha: 32

Dropout: 0.05

Target modules: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj

Bias: none

Evaluation
Performance Metrics
The model shows significant learning during training:

Loss decreased from 1.994 to 0.105

Gradient norms remained stable between 0.12-2.6

No signs of overfitting observed

Environmental Impact
Hardware Type: A10G GPU (Hugging Face Spaces)

Hours used: ~1 hour (training only)

Cloud Provider: Hugging Face Spaces

Carbon Emitted: ~0.12 kg CO2 (estimated)

Technical Specifications
Framework Versions
PEFT: 0.19.1

Transformers: 4.35.0+

PyTorch: 2.0.0+

Accelerate: 1.13.0

BitsAndBytes: 0.46.1+

Citation
BibTeX
bibtex
@misc{tamgpt2026,
  author = {TamGPT Team},
  title = {TamGPT-1.0: Engineering and Cost Intelligence Assistant},
  year = {2026},
  publisher = {Hugging Face},
  url = {https://huggingface.co/finiarisab/TamGPT-1.0}
}
APA
TamGPT Team. (2026). TamGPT-1.0: Engineering and Cost Intelligence Assistant. Hugging Face. https://huggingface.co/finiarisab/TamGPT-1.0

Model Card Authors
finiarisab

Model Card Contact
https://huggingface.co/finiarisab

Disclaimer: This model is provided as-is for research and assistive purposes. Always verify AI-generated recommendations with qualified professionals before making engineering or financial decisions.