---
language: mn
license: mit
tags:
- mongolian
- tokenizer
- sentencepiece
---

# SentencePiece Tokenizer

This repository contains a fine-tuned SentencePiece tokenizer on Mongolian text.

## Files
- `tokenizer_config.json`: The tokenizer configuration file
- `mn_tokenizer.model`: The SentencePiece model file
- `mn_tokenizer.vocab`: The SentencePiece vocabulary file

## Usage

```python
from transformers import AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained("Namuun123/mn_sentencepiece_tokenizer")
```