# Convert mT5 to HF weights?
**URL:**
**Author:**  [@congcongwang](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fcongcongwang)%5C
**Post date:** [October 24, 2020, 7:32pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fconvert-mt5-to-hf-weights%2F1727%2F1 "2020-10-24T19:32:13Z")
Hi,
I have been attempting to convert the mT5 weights available [here](https://github.com/google-research/multilingual-t5) to the HF weights for TFT5ForConditionalGeneration or T5ForConditionalGeneration? Any ideas on how to do this?
---
**Author:**  [@Zack](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FZack)%5C
**Post date:** [October 24, 2020, 7:52pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fconvert-mt5-to-hf-weights%2F1727%2F2 "2020-10-24T19:52:28Z")
I have a question, can this new model be used for summarization on other languages other than English and without fine-tuning it ?
---
**Author:**  [@congcongwang](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fcongcongwang)%5C
**Post date:** [October 24, 2020, 8:57pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fconvert-mt5-to-hf-weights%2F1727%2F3 "2020-10-24T20:57:25Z")
If the model is pre-trained in a multi-task learning way, then the answer is yes. However, if further fine-tuning on a specific downstream task such as summarization, that may give better performance. Sure, as opposed to the original T5, mT5 supports more than 100 languages (for example, T5Tokenizer can not tokenize Chinese).
---
**Author:**  [@stykat](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fstykat)%5C
**Post date:** [October 26, 2020, 1:04pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fconvert-mt5-to-hf-weights%2F1727%2F4 "2020-10-26T13:04:30Z")
Hi @congcongwang!
How have you been trying to do it, if I may ask?
There is a way of doing it for other models, as shown [here](https://huggingface.co/transformers/converting_tensorflow_models.html) but T5 is not among them.
Thank you!
---
**Author:**  [@Jung](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FJung)%5C
**Post date:** [October 27, 2020, 4:42am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fconvert-mt5-to-hf-weights%2F1727%2F5 "2020-10-27T04:42:29Z")
Hi guys,
It seems mT5 employs the T5.1.1 architecture (not the original T5 arch), as you can see from the name T5-XL and T5-XXL instead of T5-3B and T5-11B.
In this case, HF still doesn’t have implementation on this T5.1.1 yet. Please see : [https://github.com/huggingface/transformers/issues/6285](https://github.com/huggingface/transformers/issues/6285)
**UPDATED (Nov 17, 2020)** : will be released soon by amazing Patrick – [https://github.com/huggingface/transformers/pull/8552](https://github.com/huggingface/transformers/pull/8552)
---
**Author:**  [@congcongwang](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fcongcongwang)%5C
**Post date:** [October 27, 2020, 9:18am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fconvert-mt5-to-hf-weights%2F1727%2F6 "2020-10-27T09:18:49Z")
Thanks for the info. They are similar but different in some places.
---
**Author:**  [@patrickvonplaten](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fpatrickvonplaten)%5C
**Post date:** [November 17, 2020, 2:41pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fconvert-mt5-to-hf-weights%2F1727%2F7 "2020-11-17T14:41:42Z")
Improved T5 models (small to large):
- [google/t5-v1\_1-small](https://huggingface.co/google/t5-v1_1-small)
- [google/t5-v1\_1-base](https://huggingface.co/google/t5-v1_1-base)
- [google/t5-v1\_1-large](https://huggingface.co/google/t5-v1_1-large)
and mT5 models (small to large):
- [google/mt5-small](https://huggingface.co/google/mt5-small)
- [google/mt5-base](https://huggingface.co/google/mt5-base)
- [google/mt5-large](https://huggingface.co/google/mt5-large)
are in the model hub Will upload the 3b and 11b versions in the coming days (modifié)