# PreTrain RoBERTa for Kannada

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fpretrain-roberta-for-kannada%2F7873%3E%5C
**Category:** Flax/JAX Projects\
**Created:** [July 1, 2021, 9:39pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fpretrain-roberta-for-kannada%2F7873 "2021-07-01T21:39:37Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![SriharshaHatwar](https://avatars.discourse-cdn.com/v4/letter/s/3e96dc/32.png) [@SriharshaHatwar](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FSriharshaHatwar)%5C
**Post date:** [July 1, 2021, 9:39pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fpretrain-roberta-for-kannada%2F7873%2F1 "2021-07-01T21:39:37Z")

</div>

# RoBERTa for Kannada

Currently, there are only two models available for hate speech detection in the hugging face model hub. By pre-training a RoBERTa model, we wish to increase the accessibility to one of the oldest Indic languages.

## 2. Language

Kannada

## 3. Model

A randomly Initialized RoBERTa model.

## 4. Datasets

Here are some of the datasets containing Kannada sentences: preprocessing required.

1. [Metatext](https://metatext.io/datasets/cc100-kannada)

## 5. Training scripts

A masked language modeling script for Flax is available [here](https://github.com/huggingface/transformers/blob/master/examples/flax/language-modeling/run_mlm_flax.py). Probably the same can be used.

## 6. Desired project outcome

To use this model and fine-tune it for a sentiment analysis task for Kannada text sentences.

---

<div class="post-metadata">

**Author:** ![SriharshaHatwar](https://avatars.discourse-cdn.com/v4/letter/s/3e96dc/32.png) [@SriharshaHatwar](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FSriharshaHatwar)%5C
**Post date:** [July 1, 2021, 9:41pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fpretrain-roberta-for-kannada%2F7873%2F2 "2021-07-01T21:41:41Z")

</div>

More datasets containing Kannada sentences :

1. [Kannada News Dataset | Kaggle](https://www.kaggle.com/disisbig/kannada-news-dataset)
2. [Kannada Covid-19 Sentiment Analysis Dataset | Kaggle](https://www.kaggle.com/shravanc/kannada-covid19-sentiment-analysis-dataset)

---

<div class="post-metadata">

**Author:** ![SriharshaHatwar](https://avatars.discourse-cdn.com/v4/letter/s/3e96dc/32.png) [@SriharshaHatwar](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FSriharshaHatwar)%5C
**Post date:** [July 1, 2021, 9:42pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fpretrain-roberta-for-kannada%2F7873%2F3 "2021-07-01T21:42:33Z")

</div>

Some more datasets :

1. [Kannada Wikipedia Articles | Kaggle](https://www.kaggle.com/disisbig/kannada-wikipedia-articles)
2. [Samanantar | Kaggle](https://www.kaggle.com/mathurinache/samanantar)

---

<div class="post-metadata">

**Author:** ![patrickvonplaten](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/patrickvonplaten/32/2171_2.png) [@patrickvonplaten](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fpatrickvonplaten)%5C
**Post date:** [July 2, 2021, 3:34pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fpretrain-roberta-for-kannada%2F7873%2F4 "2021-07-02T15:34:12Z")

</div>

Let’s define it!