# Check how many TPU cores are using

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fcheck-how-many-tpu-cores-are-using%2F8821%3E%5C
**Category:** 🤗Accelerate\
**Created:** [July 30, 2021, 12:37am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fcheck-how-many-tpu-cores-are-using%2F8821 "2021-07-30T00:37:12Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![gabrielwong1991](https://avatars.discourse-cdn.com/v4/letter/g/d6d6ee/32.png) [@gabrielwong1991](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fgabrielwong1991)%5C
**Post date:** [July 30, 2021, 12:37am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fcheck-how-many-tpu-cores-are-using%2F8821%2F1 "2021-07-30T00:37:12Z")

</div>

I followed the instruction in [notebooks/simple\_nlp\_example.ipynb at master · huggingface/notebooks · GitHub](https://github.com/huggingface/notebooks/blob/master/examples/accelerate/simple_nlp_example.ipynb)

to setup the environment.

Is there a function to check how many TPU cores the model is using? Like XLA’s xm.xrt\_world\_size()?

Thanks

---

<div class="post-metadata">

**Author:** ![sgugger](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/sgugger/32/2291_2.png) [@sgugger](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fsgugger)%5C
**Post date:** [July 30, 2021, 1:02am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fcheck-how-many-tpu-cores-are-using%2F8821%2F2 "2021-07-30T01:02:52Z")

</div>

If you follow the notebook, you will see the launcher says: “Launching training on 8 TPUs”. You can print `accelerator.state` in your `training_function` if you want to be sure of that.

---

<div class="post-metadata">

**Author:** ![gabrielwong1991](https://avatars.discourse-cdn.com/v4/letter/g/d6d6ee/32.png) [@gabrielwong1991](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fgabrielwong1991)%5C
**Post date:** [July 30, 2021, 1:06am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fcheck-how-many-tpu-cores-are-using%2F8821%2F3 "2021-07-30T01:06:34Z")

</div>

Thanks! Yes I was adapting the code from the tutorial and in mine it doesn’t shows how many cores. I will add that and see if it shows.

---

<div class="post-metadata">

**Author:** ![gabrielwong1991](https://avatars.discourse-cdn.com/v4/letter/g/d6d6ee/32.png) [@gabrielwong1991](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fgabrielwong1991)%5C
**Post date:** [July 30, 2021, 1:14am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fcheck-how-many-tpu-cores-are-using%2F8821%2F4 "2021-07-30T01:14:39Z")

</div>

Interesting in the tutorial we don’t need to specify `device = accelerator.device` and does not need to push model to device `model = model.to(device)`. Perhaps this is the problem in my script that only uses 1 TPU core?

---

<div class="post-metadata">

**Author:** ![sgugger](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/sgugger/32/2291_2.png) [@sgugger](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fsgugger)%5C
**Post date:** [July 30, 2021, 1:32am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fcheck-how-many-tpu-cores-are-using%2F8821%2F5 "2021-07-30T01:32:23Z")

</div>

You should leave the device placement to the `Accelerate` library, yes.