Create README.md

README.md

@@ -0,0 +1,96 @@
+---
+language:
+- multilingual
+- bg
+- en
+- fr
+- de
+- ru
+- es
+- sw
+- tr
+- vi
+tags:
+- deberta
+- deberta-v3
+- mdeberta
+license: mit
+---
+
+# mdeberta-v3-base-lite
+
+This model was created through vocabulary pruning of the original [microsoft/mdeberta-v3-base](https://huggingface.co/microsoft/mdeberta-v3-base) model while maintaining full quality for Latin and Cyrillic-based languages.
+
+## Supported Languages
+- Bulgarian
+- English
+- French
+- German
+- Russian
+- Spanish
+- Swahili
+- Turkish
+- Vietnamese
+
+## Usage
+
+```python
+from transformers import AutoTokenizer, AutoModel
+
+tokenizer = AutoTokenizer.from_pretrained("rustemgareev/mdeberta-v3-base-lite")
+model = AutoModel.from_pretrained("rustemgareev/mdeberta-v3-base-lite")
+
+# Example usage
+text = "This is an example text in English."
+inputs = tokenizer(text, return_tensors="pt")
+outputs = model(**inputs)
+```
+
+## Performance Evaluation
+
+### Size Comparison
+| Metric | Original Model | Lite Model | Reduction |
+|--------|----------------|------------|-----------|
+| Vocabulary Size | 250,102 tokens | 163,211 tokens | 34.74% |
+| Disk Size | 1.06 GB | 817 MB | 23.23% |
+
+### VRAM Usage Comparison
+*Estimated using [Hugging Face Accelerate Model Estimator](https://huggingface.co/docs/accelerate/main/en/usage_guides/model_size_estimator).*
+
+| Metric | Original Model | Lite Model | Reduction |
+|--------|----------------|------------|-----------|
+| Largest Layer (float32) | 735.35 MB | 478.16 MB | 34.99% |
+| Total Size (float32) | 1.04 GB | 804.13 MB | 22.68% |
+| Training using Adam (Peak vRAM) | 4.15 GB | 3.14 GB | 24.34% |
+
+### Semantic Similarity Comparison
+
+**Evaluation Method**: Cosine similarity between embeddings of parallel sentences in different languages, using English as reference.
+
+**Test Phrases Used**:
+- English: "Artificial intelligence learns to understand human languages and helps people communicate."
+- Bulgarian: "Изкуственият интелект се учи да разбира човешките езици и помага на хората да общуват."
+- French: "L'intelligence artificielle apprend à comprendre les langages humains et aide les gens à communiquer."
+- German: "Künstliche Intelligenz lernt, menschliche Sprachen zu verstehen und hilft Menschen bei der Kommunikation."
+- Russian: "Искусственный интеллект учится понимать человеческие языки и помогает людям общаться."
+- Spanish: "La inteligencia artificial aprende a entender los idiomas humanos y ayuda a las personas a comunicarse."
+- Swahili: "Akili ya kisasa inajifunza kuelewa lugha za wanadamu na kusaidia watu kuwasiliana."
+- Turkish: "Yapay zeka, insan dillerini anlamayı öğrenir ve insanların iletişim kurmasına yardımcı olur."
+- Vietnamese: "Trí tuệ nhân tạo học cách hiểu ngôn ngữ con người và giúp mọi người giao tiếp."
+
+**Similarity Results**:
+
+| Language Pair | Original Similarity | Lite Similarity | Difference |
+|---------------|-----------------|-----------------|------------|
+| English-Bulgarian | 0.9276 | 0.9276 | 0.0000 |
+| English-French | 0.9322 | 0.9322 | 0.0000 |
+| English-German | 0.9178 | 0.9178 | 0.0000 |
+| English-Russian | 0.9335 | 0.9335 | 0.0000 |
+| English-Spanish | 0.9228 | 0.9228 | 0.0000 |
+| English-Swahili | 0.9591 | 0.9591 | 0.0000 |
+| English-Turkish | 0.9450 | 0.9450 | 0.0000 |
+| English-Vietnamese | 0.7955 | 0.7955 | 0.0000 |
+
+## License
+
+This model is distributed under the [MIT License](https://opensource.org/licenses/MIT).