Update README.md

README.md

@@ -38,117 +38,99 @@ inference:
     do_sample: true
 ---
 
-
-# ConflLlama: GTD-Finetuned Llama-3 8B
+# ConflLlama: Domain-Specific LLM for Conflict Event Classification
 
 \<p align="center"\>
 \<img src="images/logo.png" alt="Project Logo" width="300"/\>
 \</p\>
 
-  - **Model Type:** GGUF quantized (q4\_k\_m and q8\_0)
-  - **Base Model:** unsloth/llama-3-8b-bnb-4bit
-  - **Quantization Details:**
-      - Methods: q4\_k\_m, q8\_0, BF16
-      - q4\_k\_m uses Q6\_K for half of attention.wv and feed\_forward.w2 tensors
-      - Optimized for both speed (q8\_0) and quality (q4\_k\_m)
+**ConflLlama** is a large language model fine-tuned to classify conflict events from text descriptions. This repository contains the GGUF quantized models (q4\_k\_m, q8\_0, and BF16) based on **Llama-3.1 8B**, which have been adapted for the specialized domain of political violence research.
 
-### Training Data
+This model was developed as part of the research paper:
+**Meher, S., & Brandt, P. T. (2025). ConflLlama: Domain-specific adaptation of large language models for conflict event classification. *Research & Politics*, July-September 2025. [https://doi.org/10.1177/20531680251356282](https://doi.org/10.1177/20531680251356282)**
+
+-----
+
+### Key Contributions
+
+The ConflLlama project demonstrates how efficient fine-tuning of large language models can significantly advance the automated classification of political events. The key contributions are:
+
+  * **State-of-the-Art Performance**: Achieves a macro-averaged AUC of 0.791 and a weighted F1-score of 0.753, representing a 37.6% improvement over the base model.
+  * **Efficient Domain Adaptation**: Utilizes Quantized Low-Rank Adaptation (QLORA) to fine-tune the Llama-3.1 8B model, making it accessible for researchers with consumer-grade hardware.
+  * **Enhanced Classification**: Delivers accuracy gains of up to 1463% in challenging and rare event categories like "Unarmed Assault".
+  * **Robust Multi-Label Classification**: Effectively handles complex events with multiple concurrent attack types, achieving a Subset Accuracy of 0.724.
+
+-----
+
+### Model Performance
+
+ConflLlama variants substantially outperform the base Llama-3.1 model in zero-shot classification. The fine-tuned models show significant gains across all major metrics, demonstrating the effectiveness of domain-specific adaptation.
 
-  - **Dataset:** Global Terrorism Database (GTD)
-  - **Time Period:** Events before January 1, 2017
-  - **Format:** Event summaries with associated attack types
-  - **Labels:** Attack type classifications from GTD
-
-### Data Processing
-
-1.  **Date Filtering:**
-      - Filtered events occurring before 2017-01-01
-      - Handled missing dates by setting default month/day to 1
-2.  **Data Cleaning:**
-      - Removed entries with missing summaries
-      - Cleaned summary text by removing special characters and formatting
-3.  **Attack Type Processing:**
-      - Combined multiple attack types with separator '|'
-      - Included primary, secondary, and tertiary attack types when available
-4.  **Training Format:**
-      - Input: Processed event summaries
-      - Output: Combined attack types
-      - Used chat template:
-        ```
-        Below describes details about terrorist events.
-        >>> Event Details:
-        {summary}
-        >>> Attack Types:
-        {combined_attacks}
-        ```
-
-### Training Details
-
-  - **Framework:** QLoRA
-  - **Hardware:** NVIDIA A100-SXM4-40GB GPU on Delta Supercomputer
-  - **Training Configuration:**
-      - Batch Size: 1 per device
-      - Gradient Accumulation Steps: 8
-      - Learning Rate: 2e-4
-      - Max Steps: 1000
-      - Save Steps: 200
-      - Logging Steps: 10
-  - **LoRA Configuration:**
-      - Rank: 8
-      - Target Modules: q\_proj, k\_proj, v\_proj, o\_proj, gate\_proj, up\_proj, down\_proj
-      - Alpha: 16
-      - Dropout: 0
-  - **Optimizations:**
-      - Gradient Checkpointing: Enabled
-      - 4-bit Quantization: Enabled
-      - Max Sequence Length: 1024
-
-## Model Architecture
-
-The model uses a combination of efficient fine-tuning techniques and optimizations for handling conflict event classification:
+| Model          | Accuracy | Macro F1 | Weighted F1 | AUC   |
+| :------------- | :------- | :------- | :---------- | :---- |
+| **ConflLlama-Q8** | **0.765** | **0.582** | **0.758** | **0.791** |
+| ConflLlama-Q4  | 0.729    | 0.286    | 0.718       | 0.749 |
+| Base Llama-3.1 | 0.346    | 0.012    | 0.369       | 0.575 |
+
+The most significant improvements were observed in historically difficult-to-classify categories:
+
+  * **Unarmed Assault**: 1464% improvement (F1-score from 0.035 to 0.553).
+  * **Hostage Taking (Barricade)**: 692% improvement (F1-score from 0.045 to 0.353).
+  * **Hijacking**: 527% improvement (F1-score from 0.100 to 0.629).
+  * **Armed Assault**: 84% improvement (F1-score from 0.374 to 0.687).
+  * **Bombing/Explosion**: 65% improvement (F1-score from 0.549 to 0.908).
+
+-----
+
+### Model Architecture and Training
+
+  * **Base Model**: `unsloth/llama-3-8b-bnb-4bit`
+  * **Framework**: QLoRA (Quantized Low-Rank Adaptation).
+  * **Hardware**: NVIDIA A100-SXM4-40GB GPU on the Delta Supercomputer at NCSA.
+  * **Optimizations**: 4-bit quantization, gradient checkpointing, and other memory-saving techniques were used to ensure the model could be trained and run on consumer-grade hardware (under 6 GB of VRAM).
+  * **LoRA Configuration**:
+      * Rank (`r`): 8
+      * Alpha (`lora_alpha`): 16
+      * Target Modules: `q_proj`, `k_proj`, `v_proj`, `o_proj`, `gate_proj`, `up_proj`, `down_proj`
 
 \<p align="center"\>
 \<img src="images/model-arch.png" alt="Model Training Architecture" width="800"/\>
 \</p\>
 
-### Data Processing Pipeline
+### Training Data
 
-The preprocessing pipeline transforms raw GTD data into a format suitable for fine-tuning:
+  * **Dataset**: Global Terrorism Database (GTD). The GTD contains systematic data on over 200,000 terrorist incidents.
+  * **Time Period**: The training dataset consists of 171,514 events that occurred before January 1, 2017. The test set includes 38,192 events from 2017 onwards.
+  * **Preprocessing**: The pipeline filters data by date, cleans text summaries, and combines primary, secondary, and tertiary attack types into a single multi-label field.
 
 \<p align="center"\>
 \<img src="images/preprocessing.png" alt="Data Preprocessing Pipeline" width="800"/\>
 \</p\>
 
-### Memory Optimizations
+-----
 
-  - Used 4-bit quantization
-  - Gradient accumulation steps: 8
-  - Memory-efficient gradient checkpointing
-  - Reduced maximum sequence length to 1024
-  - Disabled dataloader pin memory
+### Intended Use
 
-## Intended Use
+This model is designed for academic and research purposes within the fields of political science, conflict studies, and security analysis.
 
-This model is designed for:
+1.  **Classification of terrorist events** based on narrative descriptions.
+2.  **Research** into patterns of political violence and terrorism.
+3.  **Automated coding** of event data for large-scale analysis.
 
-1.  Classification of terrorist events based on event descriptions
-2.  Research in conflict studies and terrorism analysis
-3.  Understanding attack type patterns in historical events
-4.  Academic research in security studies
+### Limitations
 
-## Limitations
+1.  **Temporal Scope**: The model is trained on events prior to 2017 and may not fully capture novel or evolving attack patterns that have emerged since.
+2.  **Task-Specific Focus**: The model is specialized for **attack type classification** and is not designed for identifying perpetrators, locations, or targets.
+3.  **Data Dependency**: Performance is dependent on the quality and detail of the input event descriptions.
+4.  **Semantic Ambiguity**: The model may occasionally struggle to distinguish between semantically close categories, such as 'Armed Assault' and 'Assassination,' when tactical details overlap.
 
-1.  Training data limited to pre-2017 events
-2.  Maximum sequence length limited to 1024 tokens
-3.  May not capture recent changes in attack patterns
-4.  Performance dependent on quality of event descriptions
+### Ethical Considerations
 
-## Ethical Considerations
+1.  The model is trained on sensitive data related to real-world terrorism and should be used responsibly for research purposes only.
+2.  It is intended for research and analysis, **not for operational security decisions** or prognostications.
+3.  Outputs should be interpreted with an understanding of the data's context and the model's limitations. Over-classification can lead to resource misallocation in real-world scenarios.
 
-1.  Model trained on sensitive terrorism-related data
-2.  Should be used responsibly for research purposes only
-3.  Not intended for operational security decisions
-4.  Results should be interpreted with appropriate context
+-----
 
 ## Training Logs
 
@@ -160,35 +142,26 @@ The training logs show a successful training run with healthy convergence patter
 
 **Loss & Learning Rate:**
 
-  - Loss decreases from 1.95 to \~0.90, with rapid initial improvement
-  - Learning rate uses warmup/decay schedule, peaking at \~1.5x10^-4
+  - Loss decreases from 1.95 to \~0.90, with rapid initial improvement. The final training loss reached 0.8843.
+  - Learning rate uses warmup/decay schedule, peaking at \~1.5x10^-4.
 
 **Training Stability:**
 
-  - Stable gradient norms (0.4-0.6 range)
-  - Consistent GPU memory usage (\~5800MB allocated, 7080MB reserved)
-  - Steady training speed (\~3.5s/step) with brief interruption at step 800
+  - Stable gradient norms (0.4-0.6 range).
+  - Consistent GPU memory usage (\~5800MB allocated, 7080MB reserved), staying under a 6 GB footprint.
+  - Steady training speed (\~3.5s/step) with brief interruption at step 800.
 
 The graphs indicate effective model training with good optimization dynamics and resource utilization. The loss vs. learning rate plot suggests optimal learning around 10^-4.
 
-## Citation
-
-```bibtex
-@misc{conflllama,
-  author = {Meher, Shreyas},
-  title = {ConflLlama: GTD-Finetuned LLaMA-3 8B},
-  year = {2024},
-  publisher = {HuggingFace},
-  note = {Based on Meta's LLaMA-3 8B and GTD Dataset}
-}
-```
-
-## Acknowledgments
-
-  - Unsloth for optimization framework and base model
-  - Hugging Face for transformers infrastructure
-  - Global Terrorism Database team
-  - This research was supported by NSF award 2311142
-  - This work used Delta at NCSA / University of Illinois through allocation CIS220162 from the Advanced Cyberinfrastructure Coordination Ecosystem: Services & Support (ACCESS) program, which is supported by NSF grants 2138259, 2138286, 2138307, 2137603, and 2138296
+-----
+
+### Acknowledgments
+
+  * This research was supported by **NSF award 2311142**.
+  * This work utilized the **Delta** system at the **NCSA (University of Illinois)** through ACCESS allocation **CIS220162**.
+  * This publication was made possible in part by a grant from the **Carnegie Corporation of New York**.
+  * Thanks to the **Unsloth** team for their optimization framework and base model.
+  * Thanks to **Hugging Face** for the model hosting and `transformers` infrastructure.
+  * Thanks to the **Global Terrorism Database** team at the University of Maryland.
 
 \<img src="[https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png](https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png)" width="200"/\>