---
license: apache-2.0
language:
- en
pipeline_tag: text-classification
library_name: transformers
tags:
- text-classification
- intent-classification
- bert
- encoder
- pytorch
- banking77
- dl26
---

# Veyra-20M: Compact Many-Label Intent Classification

Veyra-20M is a small, encoder-only text classification model built by **Dl26**. It is designed for fast English banking-intent and topic-style classification using a BERT-style bidirectional Transformer encoder.

The released checkpoint, `Dl26/Veyra-20M`, is a 20M-parameter-class classifier trained from random initialization. It uses the standard Transformers `bert` model type, so it loads with `AutoModelForSequenceClassification` and does not require `trust_remote_code=True`.

## Why this model

- Compact encoder-only classifier
- Many-label classification setup with 77 labels
- Standard Hugging Face Transformers compatibility
- No custom architecture Python files
- Fast batch inference on CPU or GPU
- Built for intent routing, query classification, triage, and lightweight encoder research

## Model details

| Property | Value |
| --- | --- |
| Model name | `Veyra-20M` |
| Developer | Dl26 |
| Model type | Encoder-only sequence classifier |
| Transformers model type | `bert` |
| Architecture | `BertForSequenceClassification` |
| Parameters | 21,342,285 |
| Hidden size | 512 |
| Layers | 6 |
| Attention heads | 8 |
| Intermediate size | 2,048 |
| Max positions | 256 |
| Vocabulary size | 3,892 |
| Number of labels | 77 |
| Training objective | Supervised single-label classification |
| License | Apache 2.0 |

## Supported labels

| ID | Label |
| --- | --- |
| `0` | `activate_my_card` |
| `1` | `age_limit` |
| `2` | `apple_pay_or_google_pay` |
| `3` | `atm_support` |
| `4` | `automatic_top_up` |
| `5` | `balance_not_updated_after_bank_transfer` |
| `6` | `balance_not_updated_after_cheque_or_cash_deposit` |
| `7` | `beneficiary_not_allowed` |
| `8` | `cancel_transfer` |
| `9` | `card_about_to_expire` |
| `10` | `card_acceptance` |
| `11` | `card_arrival` |
| `12` | `card_delivery_estimate` |
| `13` | `card_linking` |
| `14` | `card_not_working` |
| `15` | `card_payment_fee_charged` |
| `16` | `card_payment_not_recognised` |
| `17` | `card_payment_wrong_exchange_rate` |
| `18` | `card_swallowed` |
| `19` | `cash_withdrawal_charge` |
| `20` | `cash_withdrawal_not_recognised` |
| `21` | `change_pin` |
| `22` | `compromised_card` |
| `23` | `contactless_not_working` |
| `24` | `country_support` |
| `25` | `declined_card_payment` |
| `26` | `declined_cash_withdrawal` |
| `27` | `declined_transfer` |
| `28` | `direct_debit_payment_not_recognised` |
| `29` | `disposable_card_limits` |
| `30` | `edit_personal_details` |
| `31` | `exchange_charge` |
| `32` | `exchange_rate` |
| `33` | `exchange_via_app` |
| `34` | `extra_charge_on_statement` |
| `35` | `failed_transfer` |
| `36` | `fiat_currency_support` |
| `37` | `get_disposable_virtual_card` |
| `38` | `get_physical_card` |
| `39` | `getting_spare_card` |
| `40` | `getting_virtual_card` |
| `41` | `lost_or_stolen_card` |
| `42` | `lost_or_stolen_phone` |
| `43` | `order_physical_card` |
| `44` | `passcode_forgotten` |
| `45` | `pending_card_payment` |
| `46` | `pending_cash_withdrawal` |
| `47` | `pending_top_up` |
| `48` | `pending_transfer` |
| `49` | `pin_blocked` |
| `50` | `receiving_money` |
| `51` | `Refund_not_showing_up` |
| `52` | `request_refund` |
| `53` | `reverted_card_payment?` |
| `54` | `supported_cards_and_currencies` |
| `55` | `terminate_account` |
| `56` | `top_up_by_bank_transfer_charge` |
| `57` | `top_up_by_card_charge` |
| `58` | `top_up_by_cash_or_cheque` |
| `59` | `top_up_failed` |
| `60` | `top_up_limits` |
| `61` | `top_up_reverted` |
| `62` | `topping_up_by_card` |
| `63` | `transaction_charged_twice` |
| `64` | `transfer_fee_charged` |
| `65` | `transfer_into_account` |
| `66` | `transfer_not_received_by_recipient` |
| `67` | `transfer_timing` |
| `68` | `unable_to_verify_identity` |
| `69` | `verify_my_identity` |
| `70` | `verify_source_of_funds` |
| `71` | `verify_top_up` |
| `72` | `virtual_card_not_working` |
| `73` | `visa_or_mastercard` |
| `74` | `why_verify_identity` |
| `75` | `wrong_amount_of_cash_received` |
| `76` | `wrong_exchange_rate_for_cash_withdrawal` |

## Installation

```bash
pip install -U transformers torch accelerate
```

## Quick start

```python
from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch

model_id = "Dl26/Veyra-20M"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForSequenceClassification.from_pretrained(model_id)
model.eval()

text = "Can you help me reset my password?"
inputs = tokenizer(text, return_tensors="pt", truncation=True, max_length=64)

with torch.no_grad():
    logits = model(**inputs).logits

label_id = int(logits.argmax(dim=-1))
print(model.config.id2label[label_id])
```

## Batch inference

```python
texts = [
    "Book me a flight to Zurich tomorrow morning.",
    "What is the weather like today?",
    "Please cancel my card.",
]

inputs = tokenizer(
    texts,
    return_tensors="pt",
    truncation=True,
    padding=True,
    max_length=64,
)

with torch.no_grad():
    logits = model(**inputs).logits

for text, label_id in zip(texts, logits.argmax(dim=-1).tolist()):
    print(model.config.id2label[label_id], "-", text)
```

## Evaluation highlights

| Evaluation | Result |
| --- | --- |
| Validation accuracy | 0.8890 |
| Test accuracy | 0.8976 |
| Remote code required | No |

## Intended use

Veyra-20M is intended for:

- intent classification
- query routing
- customer-support triage
- lightweight text classification
- many-class encoder experiments
- CPU-friendly classifier deployments

## Limitations

- The model is specialized for the supported intent labels.
- It is trained for English short utterances.
- It may fail on long documents or out-of-domain language.
- Confidence scores should be calibrated for production systems.
- It is not a generative model or a semantic embedding model.

## Citation

```bibtex
@misc{dl26_2026_veyra_20m,
  title        = {Veyra-20M: Compact Many-Label Intent Classification},
  author       = {Dl26},
  year         = {2026},
  url          = {https://huggingface.co/Dl26/Veyra-20M}
}
```