# Using model.generate() in parrellel / faster?

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fusing-model-generate-in-parrellel-faster%2F58116%3E%5C
**Category:** Beginners\
**Created:** [October 11, 2023, 3:45am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fusing-model-generate-in-parrellel-faster%2F58116 "2023-10-11T03:45:53Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![jared312](https://avatars.discourse-cdn.com/v4/letter/j/db5fbb/32.png) [@jared312](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fjared312)%5C
**Post date:** [October 11, 2023, 3:45am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fusing-model-generate-in-parrellel-faster%2F58116%2F1 "2023-10-11T03:45:53Z")

</div>

Hey all,  
I have fine tuned a llama model and have loaded it into a model object with AutoPeftModelForCausalLM.

I now want to used this model for inference on a lot of queries. Currently I am looping (yes for looping) through each query and calling mymodel.generate, then tokenizer.decode to get each response.

Is there a smarter way I can do this in huggingface? Can you generate in parrellel or batches? I have 4 GPUs avaliable to me.

Thanks  
Jared