Update README.md

README.md

@@ -17,6 +17,15 @@ This model is designed for scalable training, long-context understanding, and ef
 
 ---
 
+
+**Key Modifications from the Original Paper:**
+
+1) Replaced the default positional encoding with Rotary Positional Embeddings (RoPE) ,
+2) Altered the attention mechanism to use Grouped Query Attention ,
+3) Customized the DataLoader to support sharded datasets and data parallelism ,
+4) Implemented Mixed Precision Training along with Distributed Data Parallel (DDP) support ,
+5) Tweaked several training and model hyperparameters for better adaptability .
+
 ## 🔬 Key Features
 
 - ✅ **Grouped Query Attention (GQA)** — Groups query heads to share key/value heads, saving memory and speeding up attention