---
license: mit
datasets:
- roskoN/dailydialog
language:
- en
tags:
- Conversation
- Transformers
- AI
- Model
---

**ConvoAI**

# Model Card for Model ID

- GBhaveshKumar/ConvoAI

- Try this in Spaces : https://huggingface.co/spaces/GBhaveshKumar/ConvoAI

## Model Details
- **Architecture:** Transformer decoder
- **Number of layers:** 32
- **Hidden size:** 1024
- **Number of attention heads:** 32
- **Context window:** 1024 tokens
- **Vocabulary size:** 50,257
- **Training objective:** Causal Conversation
- **Tokenizer:** GPT-2 Byte-Pair Encoding

### Model Description

ConvoAI is a fully custom conversational AI model trained from scratch on the DailyDialog dataset. Built using transformer architecture, this model does not rely on any pretrained weights, making it unique and tailored specifically for generating human-like dialogues. It learns to produce coherent and contextually appropriate responses in multi-turn conversations.

The model uses the GPT-2 tokenizer and was trained with a causal Conversation objective. It performs well on casual conversations and is suitable for chatbot applications, dialogue system research, and educational purposes. While powerful within the domain of DailyDialog, its general knowledge and open-domain capabilities are limited due to its focused training scope. It's not pre-trained well.


### Model Sources

https://huggingface.co/spaces/GBhaveshKumar/ConvoAI


## Limitations

- The model was trained only on the DailyDialog dataset and not on large-scale corpora like Wikipedia or Common Crawl, it lacks general world knowledge and factual accuracy on many topics.
  
- With a limited training sequence length, the model might struggle to maintain coherence in very long conversations or when asked to recall information from earlier in the dialogue.

- The model doesn’t learn from new interactions unless it is retrained. It cannot update its knowledge or remember prior conversations between sessions.

- On CPU-only systems, generation can be slow.

- The model is not fine tuned.


## How to Get Started with the Model

<pre>from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("GBhaveshKumar/ConvoAI")
tokenizer = AutoTokenizer.from_pretrained("GBhaveshKumar/ConvoAI")</pre>


### Training Data

Dataset: https://huggingface.co/datasets/roskoN/dailydialog


#### Training Hyperparameters

- Model type: Transformer

- Tokenizer: GPT-2 tokenizer

- Dataset: DailyDialog

- Number of Epochs: 15

- Maximum Sequence Length: 256

- Batch Size: 16

- Embedding Size (n_embd): 1024

- Number of Layers (n_layer): 32

- Number of Attention Heads (n_head): 32

- Sampling Strategy:

    Top-k: 50

    Top-p (nucleus sampling): 0.95

    Temperature: 0.8