---
license: mit
language:
- en
pipeline_tag: reinforcement-learning
tags:
- robotics
- reinforcement_learning
- humanoid
- soccer
- sai
- mujoco
---

## Model Details

### Model Description

This repository hosts the **Booster Soccer Controller Suite** — a collection of reinforcement learning policies and controllers powering humanoid agents in the [**Booster Soccer Showdown**](https://competesai.com/competitions/cmp_xnSCxcJXQclQ).

It contains:
1. **Low-Level Controller (robot/):**  
   A proprioceptive policy for the **Lower T1** humanoid that converts high-level commands (forward, lateral, and yaw velocities) into joint angle targets.
2. **Competition Policies (model/):**  
   High-level agents trained in SAI’s soccer environments that output those high-level commands for match-time play.

- **Developed by:** ArenaX Labs  
- **License:** MIT  
- **Frameworks:** PyTorch · MuJoCo · Stable-Baselines3  
- **Environments:** Booster Gym / SAI Soccer tasks  


## Testing Instructions

1. **Clone the repo**

```bash
git clone https://github.com/ArenaX-Labs/booster_soccer_showdown.git
cd booster_soccer_showdown
```

2. **Create & activate a Python 3.10+ environment**

```bash
# any env manager is fine; here are a few options
# --- venv ---
python3 -m venv .venv
source .venv/bin/activate  # Windows: .venv\Scripts\activate

# --- conda ---
# conda create -n booster-ssl python=3.11 -y && conda activate booster-ssl
```

3. **Install dependencies**

```bash
pip install -r requirements.txt
```

---

### Teleoperation

Booster Soccer Showdown supports keyboard teleop out of the box.

```bash
python booster_control/teleoperate.py \
  --env LowerT1GoaliePenaltyKick-v0 
```

**Default bindings (example):**

* `W/S`: move forward/backward
* `A/D`: move left/right
* `Q/E`: rotate left/right
* `L`: reset commands
* `P`: reset environment

---

⚠️ **Note for macOS and Windows users**
Because different renderers are used on macOS and Windows, you may need to adjust the **position** and **rotation** sensitivity for smooth teleoperation.
Run the following command with the sensitivity flags set explicitly:

```bash
python booster_control/teleoperate.py \
  --env LowerT1GoaliePenaltyKick-v0 \
  --pos_sensitivity 1.5 \
  --rot_sensitivity 1.5
```

(Tune `--pos_sensitivity` and `--rot_sensitivity` as needed for your setup.)

---

### Training

We provide a minimal reinforcement learning pipeline for training agents with **Deep Deterministic Policy Gradient (DDPG)** in the Booster Soccer Showdown environments in the `training_scripts/` folder. The training stack consists of three scripts:

#### 1) `ddpg.py`

Defines the **DDPG_FF model**, including:

* Actor and Critic neural networks with configurable hidden layers and activation functions.
* Target networks and soft-update mechanism for stability.
* Training step implementation (critic loss with MSE, actor loss with policy gradient).
* Utility functions for forward passes, action selection, and backpropagation.

---

#### 2) `training.py`

Provides the **training loop** and supporting components:

* **ReplayBuffer** for experience storage and sampling.
* **Exploration noise** injection to encourage policy exploration.
* Iterative training loop that:

  * Interacts with the environment.
  * Stores experiences.
  * Periodically samples minibatches to update actor/critic networks.
* Tracks and logs progress (episode rewards, critic/actor loss) with `tqdm`.

---

#### 3) `main.py`

Serves as the **entry point** to run training:

* Initializes the Booster Soccer Showdown environment via the **SAI client**.
* Defines a **Preprocessor** to normalize and concatenate robot state, ball state, and environment info into a training-ready observation vector.
* Instantiates a **DDPG_FF model** with custom architecture.
* Defines an **action function** that rescales raw policy outputs to environment-specific action bounds.
* Calls the training loop, and after training, supports:

  * `sai.watch(...)` for visualizing learned behavior.
  * `sai.benchmark(...)` for local benchmarking.

---

#### Example: Run Training

```bash
python training_scripts/main.py
```

This will:

1. Build the environment.
2. Initialize the model.
3. Run the training loop with replay buffer and DDPG updates.
4. Launch visualization and benchmarking after training.


#### Example: Test pretrained model

```bash
python training_scripts/test.py --env LowerT1KickToTarget-v0
```