# Does fp16 training compromise accuracy?

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fdoes-fp16-training-compromise-accuracy%2F18014%3E%5C
**Category:** Models\
**Created:** [May 16, 2022, 3:37pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fdoes-fp16-training-compromise-accuracy%2F18014 "2022-05-16T15:37:37Z")\
**Posts on this page:** 1\
**Showing post:** 3

<div class="post-metadata">

**Author:** ![nbroad](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/nbroad/32/26465_2.png) [@nbroad](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fnbroad)%5C
**Post date:** [May 17, 2022, 9:49pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fdoes-fp16-training-compromise-accuracy%2F18014%2F3 "2022-05-17T21:49:45Z")

</div>

Mixed precision training (fp16) is only possible on certain hardware and in some cases results in [training instability](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fmixed-precision-for-bfloat16-pretrained-models%2F5315) depending on if the model was pre-trained using bfloat16.

For older GPUs (before Volta/Turing), fp16 provides no speed up and will require more memory because both the fp16 values and fp32 values will be stored in memory. I’d recommend reading this: [Performance and Scalability: How To Fit a Bigger Model and Train It Faster](https://huggingface.co/docs/transformers/performance#fp16-training)

---

_[View the full topic](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fdoes-fp16-training-compromise-accuracy%2F18014)._