Code to Control: Synthesizing Fast, Parameterized Python Controllers
A new approach, Code to Control, synthesizes Python controllers that allow real-time policy execution without LLM inference or planning at runtime, improving action selection speed and adaptability for control tasks.
The Code to Control method introduces a way to synthesize Python controllers that execute directly as control policies. Unlike previous LLM-based methods that require a language model or explicit planning at every decision, this approach separates the program's structural components (generated by an LLM) from its parameters (optimized via derivative-free search using environmental feedback).
How Code to Control Works
The method first uses an LLM to generate the overall structure of the control policy in Python code. Then, it tunes the parameters of this code using a search algorithm that does not require gradients. The resulting controller can run as a standard Python function and takes actions directly, without consulting an LLM or running a planner during inference.
Developer Impact
- Controllers are pure Python code—no runtime dependence on LLMs or planners.
- Enables real-time control and gameplay with lower latency than PPO policies in comparable settings.
- Can be applied across different environments and tasks, including Atari games and MuJoCo locomotion.