# Speed up the prediction in transformers models

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fspeed-up-the-prediction-in-transformers-models%2F12200%3E%5C
**Category:** 🤗Transformers\
**Created:** [November 23, 2021, 12:03pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fspeed-up-the-prediction-in-transformers-models%2F12200 "2021-11-23T12:03:43Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![devpy079](https://avatars.discourse-cdn.com/v4/letter/d/ee7513/32.png) [@devpy079](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fdevpy079)%5C
**Post date:** [November 23, 2021, 12:03pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fspeed-up-the-prediction-in-transformers-models%2F12200%2F1 "2021-11-23T12:03:43Z")

</div>

Hi I am using couple of model from transformers, they work good on GPU but the performance on CPU is not that great.

I am using the Google Pegasus-Xsum model for summarization and it takes around 15 seconds to process result. Also I am using the parrot paraphrase library that also uses the T5 model in the backend, it also very slow on CPU takes around 5-7 seconds to generate the result.  
Here is the link of both: [google/pegasus-xsum · Hugging Face](https://huggingface.co/google/pegasus-xsum)  
Parrot Paraphrase: [prithivida/parrot\_paraphraser\_on\_T5 · Hugging Face](https://huggingface.co/prithivida/parrot_paraphraser_on_T5)

Any tips and suggestion to speed up the prediction in CPU, as there is limitation on my server currently…