# How to use inputs\_embeds in generate()?
**URL:**
**Author:**  [@ymfa](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fymfa)%5C
**Post date:** [August 14, 2020, 5:17pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fhow-to-use-inputs-embeds-in-generate%2F713%2F1 "2020-08-14T17:17:58Z")
I have fine-tuned a T5 model to accept a sequence of custom embeddings as input. That is, I input `inputs_embeds` instead of `input_ids` to the model’s forward method.
However, I’m unable to use `inputs_embeds` with `T5ForConditionalGeneration.generate()`. It complains that bos\_token\_id has to be given if not inputting `input_ids`, but even if I provide a bos\_token\_id, it still doesn’t run. I considered running the encoder separately, but there is no way I can pass the encoder output to generate() either.
It will be very useful if generate() can accept inputs\_embeds or encoder output, so that we can use the decoding strategies provided in the GenerationMixin.
---
**Author:**  [@mengyahu](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fmengyahu)%5C
**Post date:** [August 16, 2020, 12:46am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fhow-to-use-inputs-embeds-in-generate%2F713%2F2 "2020-08-16T00:46:59Z")
Not an expert, but I just follow the example from this:[https://github.com/patil-suraj/exploring-T5/blob/master/t5\_fine\_tuning.ipynb](https://github.com/patil-suraj/exploring-T5/blob/master/t5_fine_tuning.ipynb), and it works for me.
---
**Author:**  [@anon3699016](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fanon3699016)%5C
**Post date:** [September 15, 2021, 2:28pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fhow-to-use-inputs-embeds-in-generate%2F713%2F3 "2021-09-15T14:28:05Z")
I guess you can use `model_kwargs` argument for `.generate()` function.
put `encoder_outputs` to `model_kwargs`, where `encoder_outputs` is encoder output of encoder-decoder model (like T5 and BART). I looked at the code and think this is the way.
---
**Author:**  [@kkkevinkkk](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fkkkevinkkk)%5C
**Post date:** [December 1, 2021, 2:17am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fhow-to-use-inputs-embeds-in-generate%2F713%2F4 "2021-12-01T02:17:38Z")
Hi, did you solve this problem? I have met the same one but it seems that there is still no solution
---
**Author:**  [@chaddechant](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fchaddechant)%5C
**Post date:** [December 30, 2021, 6:54pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fhow-to-use-inputs-embeds-in-generate%2F713%2F5 "2021-12-30T18:54:37Z")
Just encountered this problem too… anyone have a solution? thanks!
---
**Author:**  [@shiv6891](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fshiv6891)%5C
**Post date:** [July 8, 2023, 10:56am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fhow-to-use-inputs-embeds-in-generate%2F713%2F6 "2023-07-08T10:56:05Z")
I worked with something like this. It runs, but I’m yet to see some meaningful results via the “custom embedding–\>T5generate” route… would be great if someone has been able to train via input-embeds properly, and could share some advice…
```auto
output_sequences = model.t5_model.generate(inputs_embeds=i_embed,
num_beams=2,
min_length= 0,
max_length=128,
repetition_penalty=2.5,
length_penalty=1.0,
early_stopping=True)
```