You need to agree to share your Hugging Face information to access this model

This repository is publicly accessible, but you have to accept the conditions to access its files and content.

This model is for non-commercial research only (CC BY-NC 4.0; see LICENSE and NOTICE). When you request access, Hugging Face shares your user name and e-mail address with the owner of this repository.

r2intent-zs-v1: zero-shot intent classification with intents you choose

Summary

r2intent-zs-v1 (zs: zero-shot; trained without any training data of the registered test benchmarks and without banking data) picks the matching intent for a customer message from a list of intents that you give at run time. Each intent is a name with an optional short description, and you need no training examples of your intents. What it was trained on is under Training data in the full report.

R2Intent v1 (https://huggingface.co/lukashm/r2intent-v1), the product model of this project, is released as a separate model, with a commercial license on request. It is not a zero-shot model: it was also trained on parts of the training splits of Banking77, HWU64, MASSIVE and CLINC150 banking and credit_cards.
r2intent-zs-v1 is released for research use only, including benchmarking and comparison. License: CC BY-NC 4.0, non-commercial use only (see below).
Only classify works with this model: this release ships no calibration file, so the anyintent library refuses check and scores for it.

Test results

Accuracy on the test sets of public benchmarks. r2intent-zs-v1 and the other zero-shot models were measured on 2026-10-08, in the project's single look at the test sets and in the baselines registered before it; R2Intent v1, the product model, was measured once on its frozen files after it was trained. The best number in each row is in bold.

Benchmark	r2intent-zs-v1	R2Intent v1	Qwen3-Embedding-8B	EmbeddingGemma 2	Laya
Banking77	0.711	0.810	0.766	0.933	0.452
HWU64	0.705	0.860	0.845	0.865	0.569
MASSIVE, 17 languages	0.587	0.774	0.832	0.660	0.311
MTOP, German, French and Spanish	0.547	0.633	0.728	0.677	0.301
xSID, Lithuanian	0.864	0.888	0.840	0.310	0.418
MInDS-14, 9 languages	0.895	0.896	0.737	0.814	0.615
SIB-200, 9 languages	0.714	0.715	0.795	0.722	0.761
Mean of the 7 benchmarks above	0.717	0.796	0.792	0.712	0.489
CLINC150 banking and credit cards	0.909	0.949	0.784	0.842	0.882
New topics (DEV, 8 topics in 5 languages)	0.779	0.781	0.641	0.546	0.482
Parameters	582 million	582 million	7.57 billion	744 million	421 million

Speed on a Kaggle T4 GPU, one message at a time with the project's measuring method: r2intent-zs-v1 26.56 ms per message (median; 90th percentile 28.06 ms), R2Intent v1 25.86 ms (median; 90th percentile 27.41 ms). The speed of the other models was not measured in this project.

r2intent-zs-v1 was trained without the training data of any of these benchmarks and without banking data; a counts-only audit found some of its training rows close to test messages, and they were not removed (the captions under Test results in the full report give the counts). R2Intent v1 was trained on public data that include the training splits of Banking77, HWU64, MASSIVE and CLINC150 banking and credit cards, so its numbers on those sets are in-domain results. The training data of Qwen3-Embedding-8B and EmbeddingGemma 2 are not disclosed, and Banking77, MASSIVE and MTOP are part of the public MTEB collection, so whether those models saw them is not known; both are shown with intent names, their better setting in these runs, and Laya with its default settings. New topics are 8 DEV topics that no public dataset contains, in English, German, Lithuanian, Polish and Slovak, generated by the same model that generated part of this project's synthetic training data, which favours this project's models. Parameter counts are those of the published model files.

Quick start

You need Python 3.9 or later, git and a Hugging Face account. On a Mac, the library needs Apple Silicon and uses the CPU. On Windows, use 64-bit x64 Python (3.12 and 3.13 are the safe choices) and, for an NVIDIA GPU, install PyTorch from the pytorch.org index before the library. The library and this model were tested on Linux, macOS and Windows, on the CPU. More details are in the library's Systems notes and its Windows page.

Install the anyintent library at the tag of this release:

pip install "git+https://github.com/lukamaga/anyintent@v0.1.0"

On this page, while logged in to Hugging Face, fill in the short access form (your name, your affiliation, the intended use and a box that confirms non-commercial research) and click Agree and request access. Hugging Face shares your user name and e-mail address with the owner of this repository. Access is granted automatically.
Create a read access token at https://huggingface.co/settings/tokens and log in with it once on this machine:

hf auth login

Or set the environment variable HF_TOKEN to the token.

Load the model and classify messages. The first run downloads the model into the Hugging Face cache: one model file of 1.19 GB and a few small files.

from anyintent import AnyIntent

clf = AnyIntent.from_pretrained("lukashm/r2intent-zs-v1", revision="main")  # or a commit hash of this repository, to pin its files
clf.set_labels([
    {"name": "lost card", "description": "The customer reports that a bank card was lost or stolen."},
    {"name": "refund status", "description": "The customer asks when a refund will arrive."},
    "close account",  # a name alone also works
])

p = clf.classify("I can't find my card anywhere")
print(p.label, p.score, p.margin)

preds = clf.classify(["Kur yra mano pinigai?", "Chcę zamknąć konto"])  # a list in, a list out

The model reads each intent as the text name: description (the name alone when there is no description). classify returns a Prediction (a list of them for a list of messages).

label is the chosen intent. score is its score (higher is better; not a probability).
margin is how far the best intent is ahead of the second best, in score units, so a small margin means a doubtful answer. It is None when only one intent was scored.
The model runs on a CUDA GPU (fp16) or on a CPU (fp32; no CPU speed was measured). On a CPU, a few answers where two intents score almost the same can differ from those on a GPU.

Details: what pip installs, and the margin

Install the anyintent library from its repository, https://github.com/lukamaga/anyintent, at the tag pinned for this release; pip also installs what it needs (PyTorch, transformers, safetensors, huggingface_hub and numpy). The library's code has its own terms (the LICENSE file of its repository); they give no rights to the model, whose license is below.

margin is the score of the best intent minus that of the second best among the intents that the retrieval step keeps (see How it works).

Details: test it on your own messages

To compare this model with others, or to study a first triage step, test it on messages that you have labeled with the right intent. Save them as messages.csv with two columns named text and intent, one message per row. In the script, replace the three example intents in DESCRIPTIONS with yours, each with a short, plain description, and spell each intent exactly as in the file. The script prints the accuracy, what three margin thresholds would do and every wrong answer.

import csv
from anyintent import AnyIntent

DESCRIPTIONS = {  # every intent in your file, with a short, plain description
    "lost card": "The customer reports that a bank card was lost or stolen.",
    "refund status": "The customer asks when a refund will arrive.",
    "close account": "The customer wants to close their account.",
}

with open("messages.csv", newline="", encoding="utf-8-sig") as f:  # columns: text, intent
    rows = list(csv.DictReader(f))
assert {r["intent"] for r in rows} <= DESCRIPTIONS.keys(), "every intent in the file needs a description"

clf = AnyIntent.from_pretrained("lukashm/r2intent-zs-v1", revision="main")
clf.set_labels([{"name": n, "description": d} for n, d in DESCRIPTIONS.items()])
preds = clf.classify([r["text"] for r in rows])  # a list in, a list out
right = [p.label == r["intent"] for p, r in zip(preds, rows)]
print(f"accuracy: {sum(right) / len(rows):.3f} on {len(rows)} messages")

# What a margin threshold would do: the answers below it go to a person
margins = sorted(p.margin for p in preds)
for share in (0.05, 0.1, 0.2):
    threshold = round(margins[int(share * len(margins))], 2)
    rest = [ok for ok, p in zip(right, preds) if p.margin >= threshold]
    print(f"threshold {threshold:.2f}: {1 - len(rest) / len(rows):.0%} doubtful, "
          f"accuracy of the rest {sum(rest) / max(len(rest), 1):.3f}")

for r, p, ok in zip(rows, preds, right):
    if not ok:
        print(f"wrong: {r['text']!r} is {r['intent']!r}, the model said {p.label!r} (margin {p.margin:.2f})")

Save the file in UTF-8: a file in another encoding can stop the script with a UnicodeDecodeError. If your spreadsheet program separates the columns with semicolons, change csv.DictReader(f) to csv.DictReader(f, delimiter=";").
On Windows, when you send the output to a file, Python can write it in the system's code page, and a letter that this code page lacks then stops the script. In cmd, with the script saved as, for example, check_messages.py, run python -X utf8 check_messages.py > out.txt. For PowerShell, see the Windows page linked above.
Each threshold line shows what share of answers falls below it and how accurate the rest would be. The library's HTTP server takes such a value as --doubt-margin and then marks those answers as doubtful. The margin is in score units, not a probability. Its scale depends on the model and its settings, so a threshold chosen for one model does not carry over to another. Compare models at the same share of doubtful answers instead.
With few messages, the three lines can show the same threshold, and a line can leave no answer at or above its threshold: its accuracy of the rest then shows as zero, which means that no answer is left. The HTTP server's --doubt-margin takes only a value above zero.
Tune the descriptions and the threshold on one part of your messages and keep another part, for example one in five, for a single final test. Numbers from the messages you tuned on usually look better than numbers from new messages.
Leave out messages that fit none of your intents, or count them separately: the model always picks one of your intents, so on such messages it is always wrong.
Do not paste messages from messages.csv into the descriptions. That makes the test look better than it is (see Limitations).

How it works

One XLM-RoBERTa large network in one file (model.safetensors), fine-tuned from BAAI/bge-reranker-v2-m3, makes two passes over each message. Size: 582 million parameters, one file of 1.19 GB.
Retrieval: with a small add-on (a LoRA adapter) switched on, it turns the message and every intent text into vectors and keeps the k intents closest to the message (k = 5 in this model). An intent this step misses cannot be chosen.
Reranking: with the add-on off, it reads the message together with each of those intent texts (the cross-encoder pass) and scores each pair. It reads at most a fixed number of tokens (pair_max_len in r2intent_meta.json) of the message and the intent text together; when a pair is longer, the longer of the two is cut first, so long messages and long descriptions can both be cut.
The final score is the cross-encoder score plus w times the retrieval cosine (w = 60 in this model); the highest score wins, so the model always picks one of the intents you give.
Load it with the anyintent library: it reads these settings from r2intent_meta.json and the add-on from model.safetensors; a plain transformers load uses neither.

Details: exact size

It has 581,911,553 parameters (567,755,777 in fp16; the embedding adapter's 14,155,776 in fp32), in one file of 1.19 GB (1,192,223,978 bytes).

Intended use

Research use only: this model is meant for research, benchmarking and comparison.

Research on zero-shot intent classification: routing short customer messages into a list of intents that you choose, also in domains it never saw in training.
Benchmarking and comparison: a zero-shot reference next to other models, on your own labeled messages. Its own test numbers (track T1 of the protocol) are under Test results above, with captions on the training rows that are close to test messages (Test results in the full report).
Studies of a first triage step before a person or a larger model: the model always picks one of your intents, so answers with a small margin (doubtful answers) can go to that person or model, with a threshold chosen on your own labeled messages.

Out-of-scope use

Decisions about people (for example about credit, insurance, jobs or access to a service) without review by a person.
Long documents: the model was not tested on them, and long text is cut.
Commercial use: no commercial license is offered for this model (see License and commercial use below).

Limitations

The model always picks one of the intents you give: it has no 'none of these fits' answer, so a message that fits none of them still gets one. A small margin marks a doubtful answer, which you can route elsewhere with a threshold of your own; the margin compares the best two of the intents it scored, so it does not show whether either fits. This card reports no measurement on messages that fit none of the given intents.
Its Lithuanian and Polish DEV numbers are measured on machine translations, not on messages written by Lithuanian or Polish customers.
It was trained once, with a single training seed.
No banking data was used in training (a finance blocklist; Training data in the full report), so banking intents are outside its training domains; some synthetic training label texts quote finance messages (captions under Test results in the full report).
Some of its training rows are close to messages of registered test sets (MTOP en, MInDS-14 en-US and the CLINC150 banking and credit_cards test set); the captions under Test results in the full report count them, and these rows were not removed from its training data.
Write intent descriptions like the training ones, which Qwen2.5-7B-Instruct wrote with the project's prompt (full report): a name and a short, plain description of what the customer wants. Do not paste example messages from your own test data into the descriptions: that makes the test numbers look better than the numbers on real traffic.
The margin is not a calibrated probability: it is in score units (the cross-encoder score plus w times the cosine), its scale depends on the settings, and a threshold chosen for one model or setting does not carry over to another.
Exposure of the base models to public test sets through web-crawl pre-training text cannot be excluded.

License and commercial use

CC BY-NC 4.0 (see LICENSE): you may use, share and adapt this model for non-commercial purposes, with attribution. No commercial license is offered for this model. For commercial use, see R2Intent v1: https://huggingface.co/lukashm/r2intent-v1. The base models' Apache-2.0 and MIT licenses and the third-party notices in NOTICE still apply to their parts: when you share this model or a model made from it, share LICENSE, NOTICE, LICENSE-APACHE-2.0.txt, LICENSE-MIT-FlagEmbedding.txt, LICENSE-MIT-fairseq.txt and hint3_alterations.json with it. Third-party notices: NOTICE. Copyright © 2026 Lukaš Patrik Magalinski.

Access: when you request access, Hugging Face shares your user name and e-mail address, and the answers of the access form, with the owner of this repository. They are used only to manage access and to see how the model is used in research.

Contact

Questions, feedback and collaboration: lukashcode@gmail.com or info@liriti.com. LinkedIn: https://www.linkedin.com/in/lukashm/

Citation

@misc{magalinski2026r2intentzs,
  author       = {Lukaš Patrik Magalinski},
  title        = {r2intent-zs-v1: zero-shot intent classification with intents you choose},
  year         = {2026},
  howpublished = {\url{https://huggingface.co/lukashm/r2intent-zs-v1}},
  url          = {https://huggingface.co/lukashm/r2intent-zs-v1}
}

Full report (PROTOCOL)

Everything below is the full report that the project's registered protocol (PROTOCOL) requires for this release, each statement with the record it comes from. A reference such as 'PROTOCOL 2026-10-13 00:06' names the protocol entry logged at that date and time; the protocol is published in the repository of the anyintent library, https://github.com/lukamaga/anyintent.

Track T1, zero-shot: no training data of these benchmarks and no banking data. Zero-shot intent classification: the labels are given at inference as name: description texts and no training example of them is needed. This model is the project's science headline (PROTOCOL E3); the public model P v1 (R2Intent v1, track T3) is released next to it under its own label, and no number of P appears in this card.

What this model is

configs.B of the stage-1.2 decision (sha256 0356061c7de71330f629d1039963c046f43a2ff4367458c5573aee5ba06e7ec2): the v1 (a) cross-encoder (BAAI/bge-reranker-v2-m3 fine-tuned on data v2 and ML v2, run 20261005T051315Z) with the ONE-0 embedding adapter (run 20261005T173137Z), in one file. Settings: w = 60, k = 5 (the K1 k_final), no cascade, v2 label texts. One backbone, two passes: the embedding pass with the LoRA adapter on retrieves the top-k labels by cosine, the cross-encoder pass with the adapter off scores the k pairs; final score = cross logit + w x cosine.

No few-shot block is packaged: the library uses this model zero-shot only.

Test results

Its test numbers are the T1 numbers of the single look (PROTOCOL entry 2026-10-07 22:08, headline configs.B; result entry 2026-10-08 01:35), published under the publication rule (PROTOCOL 2026-10-06 20:04) with the captions below; the numbers are under Test results above. Every T1 number carries the label above and these captions:

Near-duplicate caption V15 (PROTOCOL 2026-10-07 15:09, V15 result; record sha256 fee07f5012ec4d9d064a9ba7654ecd0b6ac85de881369f3dcde04eea314a729b): Counts-only audit V15 of the honest model's English training data (data v2, 75,310 rows) against MTOP en, MInDS-14 en-US and SIB-200 eng_Latn (all splits; exact match or bge-m3 cosine 0.90): 291 training rows hit at least one of the three sets (CLINC150 271, synthetic 18, Bitext 2); MTOP en 288 rows (train file 262, eval 105, test 147; exact 18, 1 and 6; mostly CLINC150 training rows, both sets being assistant commands); MInDS-14 en-US 3 rows (0 exact); SIB-200 eng_Latn 0. These rows were not removed from the honest model's training data.
Reading of the V15 caption: its MTOP en counts per file (train 262, eval 105, test 147) are rows near each file; a row near several files counts in each, so they sum to 514, more than the 288 rows.
Confirmatory-set caption (PROTOCOL 2026-10-06 21:47; counts only): of Z's 75,310 English training rows, 28 are within exact match or bge-m3 cosine 0.90 of the 900 CLINC150 banking and credit_cards test rows (0 exact), and 5 of its multilingual training rows; the model was trained before this set was registered and was not cleaned against it.
About 31 synthetic training label texts of the honest model quote finance messages (their generated descriptions carry financial phrasing).

Traceability to the evaluated model (plan correction 15)

Evaluated by the look: cross-encoder stage12_cross_b/model.safetensors sha256 32886d9e393ac51947ed46bb30daf2737c3b21fbcadb0bca3e70de18be65d5fb and adapter stage12_adapter_b.safetensors sha256 074f9512a79b59d48535251e228cdd68501cc7d7aa4cb019e7ad304a16e29eb4 (stored DEV frame sha256 3d137c5fab66181f5eb126c1d3d1af34c8b1db68e298665adfa5079a466c09af).
Name: r2intent-zs-v1 is model v1.1, Z, of the protocol; the PROTOCOL entries cited in this report use that name.
Source package: the one-file package of this model (model.safetensors sha256 302bfbf84e851091facbc6637ab62008764d13c3b03c9f02d59a5762daf87c70, meta sha256 5d853850713b64f93d2c9c422d8a02c0962cf87b57fbdfe8620128a977978b80; decision sha256 958428296e6322f183b12df625619f8ca875b3fbb543858e9443c911a262bca0).
This release: model.safetensors sha256 54b0fa0645d11162801f246e09b8b65b9631344c15a2e98046b7cfdb3b5f9fdb. Its tensors equal the source package's byte for byte (681 of 681; only the adapter prefix was renamed to r2intent_embed_lora.) and the look's evaluated files (681 of 681: backbone and head against the cross-encoder file, the adapter against the adapter file).
Changes of the metadata (no weight changed; the project's former internal working name is written as '(internal name)'): meta key '(internal name)' replaced by 'r2intent', with the value of the safetensors metadata key 'r2intent'; license 'apache-2.0' -> 'cc-by-nc-4.0' (PROTOCOL E3); adapter.prefix '(internal name)_embed_lora.' -> 'r2intent_embed_lora.'; merged_embedding_optional_dir and merged_embedding_optional_sha256 removed (that optional folder is not released); files_note names the R2Intent adapter prefix; keys product, model, copyright, track and release added; description_prompt names the project's prompt v2 without the former internal name; adapter.training.AD_LORA_IMPL names the EmbedLoRA class of the training code without the former internal name; latency.label_set names the run that measured this latency.

DEV results (no test set; PROTOCOL 2026-10-05 23:22)

Row-pooled accuracy on DEV decision rows (not flagged near_test), w 60, k 5: en13 0.862 (3498 rows), clinc_dev 0.818 (1716 rows), ltpl 0.747 (4305 rows), lt 0.750 (2188 rows), pl 0.744 (2117 rows), sk 0.739 (2153 rows), de 0.793 (2141 rows), clinc_dev@lt 0.739 (744 rows), clinc_dev@pl 0.729 (722 rows), clinc_dev@sk 0.734 (730 rows), clinc_dev@de 0.776 (713 rows), mixed_dev 0.914 (58 rows), synth_dev_unseen (8 pools) 0.788 (1237 rows). One training seed; seeds 1 and 2 of B's recipe exist as DEV information.

Latency

p50 single message 26.56 ms, p90 28.06 ms (Kaggle T4, method 3b, at k_final 5, CLINC-length DEV messages; the library's default engine, with torch.compile, which the anyintent library runs only with compile=True; K1, PROTOCOL 2026-10-07 15:11, record sha256 4a85c139c577c56569470844a81ff7e582c5037d82657e8dde234b0119c7002c).

Training data

Data v2 (manifest sha256 a71da5c924517df9970add9e5adab99a3814b711cddd934f6919bc64b7d6ad19: CLINC150 training domains, MultiWOZ 2.2, Bitext 8 verticals, HINT3 and synthetic data of Qwen2.5-7B-Instruct) and ML v2 (manifest sha256 559980c8c1ce5bbd78420f6f60b7c4eb36b63a0bf62a63534a7f66e445270be7: EuroLLM-9B-Instruct-2512 and OPUS-MT translations and native lt and pl messages). No training split of a registered benchmark and no banking data (the finance blocklist of data v2). Licenses and attributions: NOTICE; the HINT3 alterations: hint3_alterations.json.

Disclosures

Exposure of the base models to public test sets through web-crawl pre-training cannot be excluded; the training-data statements of BAAI/bge-m3 and BAAI/bge-reranker-v2-m3 do not mention Banking77, HWU64, MASSIVE, SLURP, SNIPS, xSID or MTOP.
The DEV lt and pl pools are machine translations; clinc_dev was used for many earlier choices.
The model needs label descriptions written like the training ones (Qwen2.5-7B-Instruct, the project's prompt v2).

License

CC BY-NC 4.0 (LICENSE); it replaces the Apache-2.0 files of the internal source package (PROTOCOL E3: Z is released next to P under CC BY-NC 4.0). No commercial license is offered for this model. Third-party notices: NOTICE. Copyright © 2026 Lukaš Patrik Magalinski.

Records

ML v2 manifest: sha256 559980c8c1ce5bbd78420f6f60b7c4eb36b63a0bf62a63534a7f66e445270be7
data 1.2 manifest (confirmatory caption): sha256 91993559ee1bac775f1d263ce340fb42034a213229170b1a1bf8ee4391941d3b
data v2 manifest: sha256 a71da5c924517df9970add9e5adab99a3814b711cddd934f6919bc64b7d6ad19
decision of the source package: sha256 958428296e6322f183b12df625619f8ca875b3fbb543858e9443c911a262bca0
ladder state (ladder_state.py): sha256 9e39991e8d3c4a6494308c558b5b54bb3a76a8a0df3e2f600106dc820c8d678d
source package model.safetensors: sha256 302bfbf84e851091facbc6637ab62008764d13c3b03c9f02d59a5762daf87c70
stage-1.2 decision (configs.B): sha256 0356061c7de71330f629d1039963c046f43a2ff4367458c5573aee5ba06e7ec2
z_t1_captions.json: sha256 e548ee219022478c5682efe6a29bb8db778a4b61163a6867ccf8a9bf8448ab51

Downloads last month: 2

Safetensors

Model size

0.6B params

Tensor type

F32

F16

Model tree for lukashm/r2intent-zs-v1

Base model

BAAI/bge-reranker-v2-m3

Finetuned

(122)

this model