# Fine-tuning translator based on a single language

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ffine-tuning-translator-based-on-a-single-language%2F10224%3E%5C
**Category:** Intermediate\
**Created:** [September 22, 2021, 3:56pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ffine-tuning-translator-based-on-a-single-language%2F10224 "2021-09-22T15:56:41Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![DenMarC](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/denmarc/32/2762_2.png) [@DenMarC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FDenMarC)%5C
**Post date:** [September 22, 2021, 3:56pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ffine-tuning-translator-based-on-a-single-language%2F10224%2F1 "2021-09-22T15:56:41Z")

</div>

Hi,

I have to develop a translator where there are models trained only on the source language. For example, EN\_KLINGON, where only EN\_DE, EN\_FR and EN\_RU are available. I was wondering if I could take advantage of the trained (EN) encoder, while training the decoder from zero. Also, since such an encoder could be biased towards the target language, could I use a base language model (BERT, for example) for the source language as its encoder in the Seq2Seq training pipeline?