Update README.md

README.md

@@ -296,4 +296,78 @@ datasets:
 deberta-v3-base with context length of 1280 fine-tuned on tasksource for 150k steps. I oversampled long NLI tasks (ConTRoL, doc-nli).
 Training data include helpsteer v1/v2, logical reasoning tasks (FOLIO, FOL-nli, LogicNLI...), OASST, hh/rlhf, linguistics oriented NLI tasks, tasksource-dpo, fact verification tasks.
 
-This model is suitable for long context NLI or and as a backbone for reward models or classifiers fine-tuning.
+This model is suitable for long context NLI or and as a backbone for reward models or classifiers fine-tuning.
+
+This checkpoint has strong zero-shot validation performance on many tasks (e.g. 70% on WNLI), and can be used for:
+- Zero-shot entailment-based classification for arbitrary labels [ZS].
+- Natural language inference [NLI]
+- Hundreds of previous tasks with tasksource-adapters [TA].
+- Further fine-tuning on a new task or tasksource task (classification, token classification or multiple-choice) [FT].
+
+# [ZS] Zero-shot classification pipeline
+```python
+from transformers import pipeline
+classifier = pipeline("zero-shot-classification",model="tasksource/deberta-base-long-nli")
+
+text = "one day I will see the world"
+candidate_labels = ['travel', 'cooking', 'dancing']
+classifier(text, candidate_labels)
+```
+NLI training data of this model includes [label-nli](https://huggingface.co/datasets/tasksource/zero-shot-label-nli), a NLI dataset specially constructed to improve this kind of zero-shot classification.
+
+# [NLI] Natural language inference pipeline
+
+```python
+from transformers import pipeline
+pipe = pipeline("text-classification",model="tasksource/deberta-base-long-nli")
+pipe([dict(text='there is a cat',
+  text_pair='there is a black cat')]) #list of (premise,hypothesis)
+# [{'label': 'neutral', 'score': 0.9952911138534546}]
+```
+
+# [TA] Tasksource-adapters: 1 line access to hundreds of tasks 
+
+```python
+# !pip install tasknet
+import tasknet as tn
+pipe = tn.load_pipeline('tasksource/deberta-base-long-nli','glue/sst2') # works for 500+ tasksource tasks
+pipe(['That movie was great !', 'Awful movie.'])
+# [{'label': 'positive', 'score': 0.9956}, {'label': 'negative', 'score': 0.9967}]
+```
+The list of tasks is available in model config.json.
+This is more efficient than ZS since it requires only one forward pass per example, but it is less flexible.
+
+
+# [FT] Tasknet: 3 lines fine-tuning
+
+```python
+# !pip install tasknet
+import tasknet as tn
+hparams=dict(model_name='tasksource/deberta-base-long-nli', learning_rate=2e-5)
+model, trainer = tn.Model_Trainer([tn.AutoTask("glue/rte")], hparams)
+trainer.train()
+```
+
+
+# Citation
+
+More details on this [article:](https://aclanthology.org/2024.lrec-main.1361/) 
+```
+@inproceedings{sileo-2024-tasksource,
+    title = "tasksource: A Large Collection of {NLP} tasks with a Structured Dataset Preprocessing Framework",
+    author = "Sileo, Damien",
+    editor = "Calzolari, Nicoletta  and
+      Kan, Min-Yen  and
+      Hoste, Veronique  and
+      Lenci, Alessandro  and
+      Sakti, Sakriani  and
+      Xue, Nianwen",
+    booktitle = "Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024)",
+    month = may,
+    year = "2024",
+    address = "Torino, Italia",
+    publisher = "ELRA and ICCL",
+    url = "https://aclanthology.org/2024.lrec-main.1361",
+    pages = "15655--15684",
+}
+```