Contextual Control Suite

In this project, we build the Contextual Control Suite, a modular suite of continuous control environments built on the DeepMind Control Suite, designed for studying zero-shot and transfer generalization in reinforcement learning. In these environments, both reward functions (e.g., desired speed) and dynamics (e.g., body size, weight, inertia) are parametrically varied, enabling systematic evaluation across families of related tasks. This setup provides a controlled suite for contextual MDPs, where agents must generalize behaviours to unseen combinations of reward and dynamics parameters. The Contextual Control Suite was introduced and used to evaluate HyperZero, demonstrating significant improvements over multitask and meta-RL baselines in zero-shot transfer.