---
license: mit
language:
- en
base_model:
- microsoft/deberta-v3-base
pipeline_tag: zero-shot-classification
tags:
- smart
- city
- classifier
- genai
---
# GenAI Smart City Classifier (DeBERTa v3 Base Fine-Tune)

Binary transformer classifier detecting whether a text describes a Generative AI (GenAI) application in a smart city context.

The full codebase can be found [here](https://github.com/les2feup/genai-smartcity/).

## Labels
- 0: GenAI used for smart city application
- 1: Not related

`id2label = {0: "GenAI used for smart city application", 1: "Not related"}`

## Model Card Summary
- Base: microsoft/deberta-v3-base
- Tokenizer: DebertaV2Tokenizer (same as base)
- Max length used in training batches: 512 (inference examples use 256)
- Loss: Custom focal loss (γ=2) + label smoothing (0.1)
- Scheduler: Cosine, warmup 10%
- Epochs: 8, batch size 8 (train) / 16 (eval)
- Calibration: Temperature scaling (optimal ≈ 0.602)

## Quick Start
```python
import torch
from transformers import DebertaV2Tokenizer, AutoModelForSequenceClassification

MODEL_ID = "joaocarlosnb/genai-smartcity-classifier"  # replace with actual repo id
TEMP = 0.602
id2label = {0: "GenAI used for smart city application", 1: "Not related"}

tokenizer = DebertaV2Tokenizer.from_pretrained(MODEL_ID)
model = AutoModelForSequenceClassification.from_pretrained(MODEL_ID)
model.eval()

def predict(text, max_length=256, apply_temp=True):
    inputs = tokenizer(text, truncation=True, padding="max_length",
                       max_length=max_length, return_tensors="pt")
    with torch.no_grad():
        logits = model(**inputs).logits
        if apply_temp:
            logits = logits / TEMP
        probs = torch.softmax(logits, dim=-1)[0]
    top = int(probs.argmax())
    return {
        "label": id2label[top],
        "probabilities": {id2label[i]: float(p) for i, p in enumerate(probs)}
    }

print(predict("We apply a diffusion model to simulate traffic for urban planning."))
```

## Installation
```bash
pip install transformers torch
```

## Input Guidance
Short technical sentences or abstract fragments (English). Truncate >512 tokens automatically.

## Limitations
- Binary only (no “mentioned, not used” middle class)
- English academic / technical domain bias
- Not evaluated for adversarial or multilingual robustness

## Intended Use
Research, corpus analysis, and exploratory filtering. Human review is recommended before operational deployment.

## Dataset
Training data hosted separately (same namespace). Contains augmented, adaptive, contrastive, and diagnostic subsets.

## Reproducibility Notes
Set `seed=42`. Use DebertaV2Tokenizer with max_length=512 for full retraining.

## Citation (Placeholder)
> Bittencourt, J. C. N., Flores, T. K. S., Jesus, T. C., & Costa, D. G. (2025). On the Role of AI in Building Generative Urban Intelligence. In Review. https://doi.org/10.21203/rs.3.rs-7131966/v1
> 
## License
See repository LICENSE (ensure compatibility with upstream model license).

## Security
Do not hard-code Hugging Face tokens. Use environment variable: `export