# Reinforcement Learning Unity ML Agents

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Freinforcement-learning-unity-ml-agents%2F107249%3E%5C
**Category:** Course\
**Created:** [September 18, 2024, 4:14pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Freinforcement-learning-unity-ml-agents%2F107249 "2024-09-18T16:14:02Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![sakshamdura](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/sakshamdura/32/31757_2.png) [@sakshamdura](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fsakshamdura)%5C
**Post date:** [September 18, 2024, 4:14pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Freinforcement-learning-unity-ml-agents%2F107249%2F1 "2024-09-18T16:14:02Z")

</div>

> **[Hands-on - Hugging Face Deep RL Course](https://huggingface.co/learn/deep-rl-course/unit5/hands-on)**
>
> We’re on a journey to advance and democratize artificial intelligence through open source and open science.

TypeError: invalid dtype object: only floating-point types are supported as the default type  
I changed in ml-agents/ml-agents/mlagents/torch\_utils/torch.py  
if \_device.type == “cuda”:  
torch.set\_default\_device(\_device.type)  
torch.set\_default\_dtype(torch.float32) // changed  
else:  
torch.set\_default\_dtype(torch.float32)  
logger.debug(f"default Torch device: {\_device}")  
After changing I get this error:  
RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cpu! (when checking argument for argument mat1 in method wrapper\_CUDA\_addmm)

---

<div class="post-metadata">

**Author:** ![seangogo](https://avatars.discourse-cdn.com/v4/letter/s/bc8723/32.png) [@seangogo](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fseangogo)%5C
**Post date:** [October 8, 2024, 12:12am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Freinforcement-learning-unity-ml-agents%2F107249%2F2 "2024-10-08T00:12:04Z")

</div>

Seems setting `threaded: false` in the config solves the problem

```auto
behaviors:
  SnowballTarget:
    trainer_type: ppo
    summary_freq: 10000
    keep_checkpoints: 10
    checkpoint_interval: 50000
    max_steps: 200000
    time_horizon: 64
    threaded: false
    hyperparameters:
      learning_rate: 0.0003
      learning_rate_schedule: linear
      batch_size: 128
      buffer_size: 2048
      beta: 0.005
      epsilon: 0.2
      lambd: 0.95
      num_epoch: 3
    network_settings:
      normalize: false
      hidden_units: 256
      num_layers: 2
      vis_encode_type: simple
    reward_signals:
      extrinsic:
        gamma: 0.99
        strength: 1.0

```