Adapting Diffusion Policies to Novel Environments via Policy-Steered Optimization

We propose a novel method of adapting generative robot control policies to succeed in unfamiliar environments with novel runtime constraints. We use a model-based planner to optimize for trajectories that obey both the novel environmental constraints and the implicit task constraints learned by the policy model. We achieve this by evaluating a policy-alignment objective that measures the policy model’s success at reconstructing noised trajectories. We demonstrate this approach’s ability to generalize to two novel simulation environments with obstacles not seen during training.

Recommended citation: Skye Thompson, Sergio Orozco, Eric Rosen, Karl Schmeckpeper, George Konidaris, ICLR ReALM Gens Workshop, 2026
Download Paper | Download Bibtex