nolanplatt/hvf-slm-v2-llama

HVF-SLM v2 (Llama): Improved Maritime/AIS LLM with Limitations

Second iteration in the HVF-SLM series, based on Llama 3.1 8B. Shows significant improvements over v1-magistral in coordinate extraction but suffers from hallucination issues.

Improvements over v1:

• 97% better vessel identification accuracy
• Successful 131k token context processing
• Lower training loss (0.009 → 0.0002)

Critical Issues:

• Repeats phrases 50+ times
• Invents vessel positions
• Extracts wrong vessels when similar names exist

Model Details

• Base Model: Llama-3.1-8B-Instruct
• Context Length: 131k tokens (already supported by llama)
• Training Dataset: ~22,000 synthetic maritime Q&A pairs
• Fine-tuning Method: QLoRA (4-bit) rank 128
• Status: Superseded by v2-llama and v3-qwen

Research Value

Despite not being our final SLM, this model does demonstrate the challenge of training SLMs on structured maritime data. Despite good training metrics, the model failed to generalize properly, leading to the development of v3-qwen with improvements.

Not Recommended For:

• Production maritime systems
• Safety-critical applications
• Real vessel tracking

Citation

Part of a larger, in-depth paper by HVF. Full citation available upon publication.