ICRA 2024: Generative Modeling of Residuals for Real-Time Risk-Sensitive Safety with Discrete-Time Control Barrier Functions

Ryan K. Cosner*, Igor Sadalski, Jana K. Woo, Preston Culbertson, and Aaron D. Ames.

paper headshot orio

Submitted to the International Conference on Robotics and Automation (ICRA) 2024, Yokohama



A key source of brittleness for robotic systems is the presence of model uncertainty and external disturbances. Most existing approaches to robust control either seek to bound the worst-case disturbance (which results in conservative behavior), or to learn a deterministic dynamics model (which is unable to capture uncertain dynamics or disturbances).

This work proposes a different approach: training a state- conditioned generative model to represent the distribution of error residuals between the nominal dynamics and the actual system. In particular we introduce the Online Risk-Informed Optimization controller (ORIO), which uses Discrete-Time Control Barrier Functions, combined with a learned, generative disturbance model, to ensure the safety of the system up to some level of risk.

We demonstrate our approach in both simulations and hardware, and show our method can learn a disturbance model that is accurate enough to enable risk-sensitive control of a quadrotor flying aggressively with an unmodelled slung load. We use a conditional variational autoencoder (CVAE) to learn a state-conditioned dynamics residual distribution, and find that the resulting probabilistic safety controller, which can be run at 100Hz on an embedded computer, exhibits less conservative behavior while retaining theoretical safety properties.

Extended Paper Version

paper headshot

Project Video


Special thanks to Albert Li for his help with understanding and coding the CVAEs and to Amy Li for her insightful feedback on our manuscript. This work was also supported by BP.


If you have any questions, please feel free to contact Ryan Cosner.