# Finetuning GPT2 with user defined loss

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ffinetuning-gpt2-with-user-defined-loss%2F163%3E%5C
**Category:** Beginners\
**Created:** [July 9, 2020, 8:18pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ffinetuning-gpt2-with-user-defined-loss%2F163 "2020-07-09T20:18:56Z")\
**Posts on this page:** 1\
**Showing post:** 58

<div class="post-metadata">

**Author:** ![aclifton314](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/aclifton314/32/55465_2.png) [@aclifton314](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Faclifton314)%5C
**Post date:** [May 22, 2023, 10:21pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ffinetuning-gpt2-with-user-defined-loss%2F163%2F58 "2023-05-22T22:21:59Z")

</div>

@SUNM It is my understanding that if you train a model with the huggingface `Trainer` class (as you’ve shown in the post [Finetuning GPT2 using Multiple GPU and Trainer](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ffinetuning-gpt2-using-multiple-gpu-and-trainer%2F796)) and you have multiple GPUs available on your system, then `Trainer` will use all of the available GPUs unless explicitly told not to do so.

So, if you have 4 GPUs available but you set the environment variable `CUDA_VISIBLE_DEVICES=“1,2,3”`, then `Trainer` will only utilize those 3 GPU instead of all 4 of them.

---

_[View the full topic](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ffinetuning-gpt2-with-user-defined-loss%2F163)._