Update README.md

README.md

@@ -1,197 +1,55 @@
-
 ---
-
 license: apache-2.0
-
 language:
-
 - en
-
 tags:
-
 - maritime
-
 - AIS
-
 - vessel-tracking
-
 - navigation
-
 - fine-tuned
-
-- 131-context
-
+- experimental
+- research
 base_model: mistralai/Magistral-Small-2506
-
 datasets:
-
 - synthetic-maritime-ais-qa
-
 model-index:
-
-- name: nolanplatt/hvf-slm
-
+- name: hvf-slm-v1-magistral
   results: []
-
 ---
 
-# HVF-SLM: Maritime Domain-Specialized Language Model with 131k Context ⚓︎🛳️
-
+# HVF-SLM v1 (Magistral): Experimental Baseline for Maritime Domain LLM Research
 
 
-We present a small language model (SLM) with domain expertise in AIS/vessel data. We performed supervised fine tuning (SFT) on [Magistral Small](https://huggingface.co/mistralai/Magistral-Small-2506) with a customized dataset from publicly available AIS data in US coastal waters.
+This is the first experimental iteration (v1) in the HVF-SLM series, serving as a baseline for maritime domain-specialized language models. While it demonstrated 131k context capability during training, significant limitations were discovered during evaluation.
 
-Dataset creation and supervised fine-tuning (SFT) was performed by [Hitachi Vantara Federal](https://www.hitachivantarafederal.com/). 
-Cleaning and enrichment of the data was acomplished by leveraging [Pentaho+ Data Integration](https://pentaho.com/).
+## Model Status
 
+**This model is provided for research purposes only** as part of our iterative development process documented in [paper citation pending]. It has:
+- Successful 131k token context window during training
+- Poor coordinate extraction accuracy
+- Tendency to generate incorrect vessel positions
+- Limited understanding of maritime JSON structure
 
 ## Model Details
-
-
-
 - **Base Model**: Magistral-Small-2506 (24B parameters)
-
 - **Context Length**: 131k tokens (via RoPE scaling factor 3.2)
-
-- **Training Dataset**: ~22,000 synthetic maritime Q&A pairs with full AIS tracking data (random vessel context for each pair pulled from ~3.4B U.S. Coast Guard data). Differing linguistic variations, styles, focus areas.
-
+- **Training Dataset**: ~22,000 synthetic maritime Q&A pairs
 - **Fine-tuning Method**: QLoRA (4-bit) rank 128
+- **Status**: Superseded by v2-llama and v3-qwen
 
-- **Hardware**: NVIDIA H100 (80GB)
-
-- **Training Duration**: ~18 hours
-
-
-
-## Intended Use
-
-
-
-This model excels at:
-
-- AIS trajectory prediction and analysis
-
-- Maritime anomaly detection
-
-- Vessel behavior classification
-
-- Navigation compliance (COLREGs)
-
-- Route optimization with AIS constraints
-
-- Maritime domain Q&A
-
-
-
-## Technical Specifications
-
-
-
-- **Model Size**: 24B parameters (16-bit merged)
-
-- **Max Context**: 131,072 tokens
-
-- **RoPE Scaling**: Linear, factor 3.2
-
-- **Supported Tasks**: Text generation, maritime analysis
-
-- **Long Context Handling**: Successfully trained on sequences up to 131k tokens without truncation on a *singular GPU* via gradient checkpointing.
-
-- **Mixed Precision**: BFloat16 training with 4-bit base model quantization
-
-- **Cosine Warm Restarts**: 6 restart cycles to escape loss plateaus
+## Why This Model Failed
 
+Despite achieving low training loss (0.004), the model failed to generalize to real-world maritime queries:
+1. **Memorization over comprehension**: The model memorized training patterns rather than learning vessel relationships
+2. **JSON parsing failures**: Unable to reliably extract specific vessels from complex AIS data
+3. **Coordinate hallucination**: Generated plausible but incorrect lat/lon positions
 
 
-## Usage
-
-
-
-```python
-
-from transformers import AutoModelForCausalLM, AutoTokenizer
-
-
-
-model = AutoModelForCausalLM.from_pretrained("nolanplatt/hvf-slm")
-
-tokenizer = AutoTokenizer.from_pretrained("nolanplatt/hvf-slm")
-
-
-
-# Example: Analyze AIS data
-
-prompt = "Analyze the following AIS data and predict the vessel's next position..." # inject AIS data after prompt, formatted as JSON
-
-inputs = tokenizer(prompt, return_tensors="pt", max_length=131072, truncation=True)
-
-outputs = model.generate(**inputs, max_length=2000)
-
-response = tokenizer.decode(outputs[0], skip_special_tokens=True)
-
-```
-
-
-
-## Training Configuration
-
-# Through our extensive research, these hyperparameters enable 131k context on single H100
-
-```json
-
-{
-
-    "max_seq_length": 131072,
-
-    "per_device_train_batch_size": 1,
-
-    "gradient_accumulation_steps": 8,
-
-    "learning_rate": 3e-5, // inc. this if fall into loss plateau; helps escape
-
-    "warmup_steps": 300,
-
-    "lr_scheduler_type": "cosine_with_restarts", // also helps escape loss plateaus
-
-    "num_cycles": 6,
-
-    "optimizer": "paged_adamw_8bit",
-
-    "bf16": true,
-
-    "gradient_checkpointing": true 
-
-}
-
-```
-
-
-
-## Performance
-
-We are still performing evaluation's on HVF-SLM's success. We can say, preliminarily, that it successfully processes full AIS tracking sequences (90k+ tokens) and maintains  domain expertise while preserving general capabilities of the base Magistral model.
-
-
+This baseline informed critical improvements in subsequent versions:
+- v2-llama: Better extraction but with hallucination issues
+- v3-qwen: Architectural changes to address fundamental limitations
 
 ## Citation
 
-This model is open-source and free to use, permitted you cite the authors and do not claim it as your own.
-
-A full citation will be available here upon publication.
-
-```
-
-@misc{hvf-slm-2025,
-
-  title={HVF-SLM: Maritime Domain-Specialized Language Model with 131k Context},
-
-  author={Platt, Nolan and Nayak, Pragyansmita},
-
-  year={2025},
-
-  publisher={HuggingFace}
-
-}
-
-```
-
+Part of a larger, in-depth paper by HVF. Full citation available upon publication.