MuJoCo Playground: massively-parallel PPO on a quadruped

Glossary primer (12 min)

• MuJoCo Playground — Google DeepMind's 2025 GPU-accelerated MuJoCo wrapper. Runs 4096 parallel envs on a single H100 with full physics. Released v0.1.0 in 2024-Q4.
• MJX — MuJoCo translated to JAX, runs on GPU/TPU. The simulationSimulation & Sim-to-RealSimulationA virtual environment where robots can be trained or tested. engine under MuJoCo Playground.
• Brax — DeepMind's earlier JAX-based RLImitation & Reinforcement LearningReinforcement Learning (RL)Teaching a robot through trial and error using rewards. library. Many Playground envs are inherited.
• Massively-parallel RLImitation & Reinforcement LearningReinforcement Learning (RL)Teaching a robot through trial and error using rewards. — 1000s of envs run synchronously on GPU. Unlocks throughput-based RLImitation & Reinforcement LearningReinforcement Learning (RL)Teaching a robot through trial and error using rewards. (16M env steps/sec).
• Go1 / Go2 — Unitree's quadruped robots; standard sim-to-realSimulation & Sim-to-RealSim-to-real (sim2real)Transferring a policy trained in simulation to a real robot. benchmarks.
• EpisodeRobot LearningEpisodeOne full attempt at a task from start to finish. returnImitation & Reinforcement LearningReturnThe total accumulated reward over time. — Sum of rewards in one episodeRobot LearningEpisodeOne full attempt at a task from start to finish.. The headline metricEvaluation & ResearchMetricA numerical measure of performance..
• Domain rand. config — Parameters (mass, frictionMovement, Mechanics & Robot BodyFrictionResistance between contacting surfaces that affects sliding and grasping., motor strength) randomized at episodeRobot LearningEpisodeOne full attempt at a task from start to finish. start.
• RewardImitation & Reinforcement LearningRewardA score that tells the robot how well it is doing. shaping — Adding auxiliary terms (alive bonus, actionCore ConceptsActionA command the robot sends to its motors, controller, or low-level system. smoothness, contactMovement, Mechanics & Robot BodyContactPhysical interaction between the robot and an object or surface. penalty) to dense rewardImitation & Reinforcement LearningDense rewardA reward signal given frequently throughout the task.. Critical for locomotionNavigation & LocomotionLocomotionMovement of the robot body through space, like walking, rolling, or running..

Real-world analogy

Hour 1 — Reading

• MuJoCo Playground v0.1.0 release notes (~10 min): https://github.com/google-deepmind/mujoco_playground
• Brax paper, abstract + Section 3 (~25 min): https://arxiv.org/abs/2106.13281
• Watch MuJoCo Playground tour (DeepMind release video, if available; otherwise the README's GIFs)

Hour 2 — Install + first env

ssh -i ~/.ssh/nebius_key ubuntu@<your-h100-ip>
cd ~ && mkdir -p robo47-rl && cd robo47-rl
uv venv --python 3.12 .venv && source .venv/bin/activate
uv pip install -U "jax[cuda12]" "mujoco-mjx>=3.7" "mujoco-playground"
uv pip install brax flax orbax wandb tensorboard

# Verify GPU JAX
python -c "
import jax; print('devices:', jax.devices())
"

# Smoke test: Spot quadruped env loads
python -c "
from mujoco_playground import registry, locomotion
env = registry.load('SpotJoystickFlatTerrain')
print(f'obs dim: {env.observation_size}, act dim: {env.action_size}')
"

obs dim: 48, act dim: 12