Nonlinear learning-based model predictive control supporting state and input dependent model uncertainty estimates
Corresponding Author
Kim P. Wabersich
Institute for Dynamic Systems and Control, ETH Zurich, Zurich, Switzerland
Correspondence Kim P. Wabersich, Institute for Dynamic Systems and Control, ETH Zurich, Zurich, Switzerland.
Email: [email protected]
Search for more papers by this authorMelanie N. Zeilinger
Institute for Dynamic Systems and Control, ETH Zurich, Zurich, Switzerland
Search for more papers by this authorCorresponding Author
Kim P. Wabersich
Institute for Dynamic Systems and Control, ETH Zurich, Zurich, Switzerland
Correspondence Kim P. Wabersich, Institute for Dynamic Systems and Control, ETH Zurich, Zurich, Switzerland.
Email: [email protected]
Search for more papers by this authorMelanie N. Zeilinger
Institute for Dynamic Systems and Control, ETH Zurich, Zurich, Switzerland
Search for more papers by this authorFunding information: Swiss National Science Foundation, PP00P2 157601/1
Abstract
While model predictive control (MPC) methods have proven their efficacy when applied to systems with safety specifications and physical limitations, their performance heavily relies on an accurate prediction model. As a consequence, a significant effort in the design of MPC controllers is dedicated to the modeling part and often requires advanced physical expertise. In order to facilitate the controller design, we present an MPC scheme supporting nonlinear learning-based prediction models, that is, data-driven models with probabilistic parameter uncertainties. A tube-based MPC formulation in combination with an additional implicit state and input constraint forces the closed-loop system to be operated in domains of sufficient model confidence, thereby ensuring asymptotic stability and constraint satisfaction at a prespecified level of probability. Furthermore, by relying on tube-based MPC concepts, the proposed learning-based MPC formulation offers a general framework for addressing different problem classes, such as economic MPC, while providing a general interface to probabilistic prediction models based, for example, on Bayesian regression or Gaussian processes. A design procedure is proposed for approximately linear systems and the efficiency of the method is illustrated using numerical examples.
CONFLICT OF INTEREST
The authors declare no potential conflict of interest.
Open Research
DATA AVAILABILITY STATEMENT
Data sharing is not applicable to this article as no new real-world data were created or analyzed in this article.
REFERENCES
- 1Morari M, Lee JH. Model predictive control: past, present and future. Comput Chem Eng. 1999; 23(4): 667-682.
- 2Qin SJ, Badgwell TA. An overview of nonlinear model predictive control applications. Nonlinear Model Predictive Control. Basel: Birkhäuser; 2000: 369-392.
- 3 Hewing L, Kabzan J, Zeilinger MN. Cautious model predictive control using Gaussian process regression. IEEE Trans Control Syst Technol. 2019; 28: 1-12.
- 4Carron A, Arcari E, Wermelinger M, Hewing L, Hutter M, Zeilinger MN. Data-driven model predictive control for trajectory tracking with a robotic arm. IEEE Robot Automat Lett. 2019; 4(4): 3758-3765.
- 5Kamthe S, Deisenroth MP. Data-efficient reinforcement learning with probabilistic model predictive control. Paper presented at: Proceedings of the International Conference on Artificial Intelligence and Statistics, AISTATS 2018, Lanzarote, Canary Islands; Vol. 84, 2018:1701-1710.
- 6Koller T, Berkenkamp F, Turchetta M, Krause A. Learning-based model predictive control for safe exploration. Paper presented at: Proceedings of the 2018 IEEE Conference on Decision and Control (CDC), Miami Beach, Florida, USA; 2018:6059-6066.
- 7Soloperto R, Müller MA, Trimpe S, Allgöwer F. Learning-based robust model predictive control with state-dependent uncertainty. Paper presented at: Proceedings of the 6th IFAC Conference on Nonlinear Model Predictive Control; 2018:442-447
- 8Hewing L, Liniger A, Zeilinger MN. Cautious NMPC with Gaussian process dynamics for autonomous miniature race cars. Paper presented at: Proceedings of the 2018 European Control Conference (ECC), Limassol, Cyprus; 2018:1341-1348.
- 9Ostafew CJ, Schoellig AP, Barfoot TD. Robust constrained learning-based NMPC enabling reliable mobile robot path tracking. Int J Robot Res. 2016; 35(13): 1547-1563.
- 10Calliess JP. Conservative Decision-Making and Inference in Uncertain Dynamical Systems [PhD thesis]. Oxford, England: University of Oxford; 2014.
- 11Terzi E, Fagiano L, Farina M, Scattolini R. Learning multi-step prediction models for receding horizon control. Paper presented at: Proceedings of the 2018 European Control Conference, ECC 2018, Limassol, Cyprus; 2018:1335-1340
- 12Aswani A, Gonzalez H, Sastry SS, Tomlin CJ. Provably safe and robust learning-based model predictive control. Automatica. 2013; 49(5): 1216-1226.
- 13Bouffard P, Aswani A, Tomlin CJ. Learning-based model predictive control on a quadrotor: onboard implementation and experimental results. Paper presented at: Proceedings of the IEEE International Conference on Robotics and Automation, Saint Paul, Minnesota, USA; 2012:279-284.
- 14Aswani A, Bouffard P, Tomlin C. Extensions of learning-based model predictive control for real-time application to a quadrotor helicopter. Paper presented at: Proceedings of the American Control Conference, Montreal, Canada; 2012:4661-4666.
- 15Aswani A, Master N, Taneja J, Culler D, Tomlin C. Reducing transient and steady state electricity consumption in HVAC using learning-based model-predictive control. Proc IEEE. 2012; 100(1): 240-253.
- 16Aswani A, Master N, Taneja J, Krioukov A, Culler D, Tomlin CJ. Energy-efficient building HVAC control using hybrid system LBMPC. Paper presented at: Proceedings of the 4th IFAC Conference on Nonlinear Model Predictive Control, Noordwijkerhout, Netherlands; 2012:496-501.
- 17Beliakov G. Interpolation of Lipschitz functions. J Comput Appl Math. 2006; 196(1): 20-44.
- 18Calliess JP, Roberts S, Rasmussen C, Maciejowski J. Nonlinear set membership regression with adaptive hyper-parameter estimation for online learning and control. Paper presented at: Proceedings of the 2018 European Control Conference, ECC 2018, Limassol, Cyprus; 2018:3167-3172.
- 19Limon D, Calliess J, Maciejowski JM. Learning-based nonlinear model predictive control. Paper presented at: Proceedings of the 20th IFAC World Congress, Berlin, Germany; 2017:7769-7776.
- 20Fukushima H, Kim TH, Sugie T. Adaptive model predictive control for a class of constrained linear systems based on the comparison model. Automatica. 2007; 43(2): 301-308.
- 21Tanaskovic M, Fagiano L, Smith R, Morari M. Adaptive receding horizon control for constrained MIMO systems. Automatica. 2014; 50(12): 3019-3029.
- 22Adetola V, DeHaan D, Guay M. Adaptive model predictive control for constrained nonlinear systems. Syst Control Lett. 2009; 58(5): 320-326.
- 23Adetola V, Guay M. Robust adaptive MPC for constrained uncertain nonlinear systems. Int J Adapt Control Signal Process. 2011; 25(2): 155-167.
- 24Gonçalves GAA, Guay M. Robust discrete-time set-based adaptive predictive control for nonlinear systems. J Process Control. 2016; 39: 111-122.
- 25Di Cairano S. Indirect adaptive model predictive control for linear systems with polytopic uncertainty. Paper presented at: Proceedings of the American Control Conference, Boston, Massachusetts, USA; 2016:3570-3575.
- 26Lorenzen M, Cannon M, Allgöwer F. Robust MPC with recursive model update. Automatica. 2019; 103: 461-471.
- 27Mesbah A. Stochastic model predictive control: an overview and perspectives for future research. IEEE Control Syst Mag. 2016; 36(6): 30-44.
- 28Hewing L, Zeilinger MN. Scenario-based probabilistic reachable sets for recursively feasible stochastic model predictive control. IEEE Control Syst Lett. 2019; 4(2): 450-455.
10.1109/LCSYS.2019.2949194 Google Scholar
- 29Mammarella M, Alamo T, Lucia S, Dabbene F. A probabilistic validation approach for penalty function design in stochastic model predictive control; 2020. arXiv preprint arXiv:2003.07241.
- 30Wabersich KP, Zeilinger MN. Linear model predictive safety certification for learning-based control. Paper presented at: Proceedings of the 57th IEEE Conference on Decision and Control, Miami Beach, Florida, USA; 2018:7130-7135.
- 31Wabersich KP, Hewing L, Carron A, Zeilinger MN. Probabilistic model predictive safety certification for learning-based control; 2019. arXiv:1906.10417.
- 32Wabersich KP, Zeilinger MN. Safe exploration of nonlinear dynamical systems: a predictive safety filter for reinforcement learning; 2018. arXiv:1812.05506.
- 33Prajna S, Jadbabaie A, Pappas GJ. Stochastic safety verification using barrier certificates. Paper presented at: Proceedings of the 43rd IEEE Conference on Decision and Control (CDC), Paradise Island, Bahamas; 2004:929-934.
- 34Huang C, Chen X, Lin W, Yang Z, Li X. Probabilistic safety verification of stochastic hybrid systems using barrier certificates. ACM Trans Embedded Comput Syst. 2017; 16(5s): 1-19.
- 35Gelman A, Carlin JB, Stern HS, Dunson DB, Vehtari A, Rubin DB. Bayesian Data Analysis. Boca Raton, FL: Chapman & Hall/CRC Press; 2013.
10.1201/b16018 Google Scholar
- 36 Rasmussen CE, CKI W. Gaussian Processes for Machine Learning. Cambridge, Massachusetts: The MIT Press; 2006.
- 37Srinivas N, Krause A, Kakade SM, Seeger MW. Information-theoretic regret bounds for Gaussian process optimization in the bandit setting. IEEE Trans Inf Theory. 2012; 58(5): 3250-3265.
- 38Chowdhury SR, Gopalan A. On kernelized multi-armed bandits. Paper presented at: Proceedings of the International Conference on Machine Learning (ICML), Sidney, Australia; 2017:844-853.
- 39Bayer FA, Müller MA, Allgöwer F. Tube-based robust economic model predictive control. J Process Control. 2014; 24(8): 1237-1246.
- 40 Rawlings JB, Mayne DQ, Diehl MM. Model Predictive Control: Theory, Computation, and Design. 2nd ed. Madison, Wisconsin Nob Hill Publishing; 2017.
- 41Mayne DQ, Seron MM, Raković S. Robust model predictive control of constrained linear systems with bounded disturbances. Automatica. 2005; 41(2): 219-224.
- 42Yu S, Maier C, Chen H, Allgöwer F. Tube MPC scheme based on robust control invariant set with application to Lipschitz nonlinear systems. Syst Control Lett. 2013; 62(2): 194-200.
- 43Borrelli F. Constrained Optimal Control of Linear and Hybrid Systems. New York, NY: Springer; 2003: 290.
- 44Limon D, Alvarado I, Alamo T, Camacho E. Robust tube-based MPC for tracking of constrained linear systems with additive disturbances. J Process Control. 2010; 20(3): 248-260.
- 45Villanueva ME, Quirynen R, Diehl M, Chachuat B, Houska B. Robust MPC via min–max differential inequalities. Automatica. 2017; 77: 311-321.
- 46Singh S, Majumdar A, Slotine JJ, Pavone M. Robust online motion planning via contraction theory and convex optimization. Paper presented at: Proceedings of the 2017 IEEE International Conference on Robotics and Automation (ICRA), Singapore; 2017:5883-5890.
- 47 Köhler J, Soloperto R, Müller MA, Allgöwer F. A computationally efficient robust model predictive control framework for uncertain nonlinear systems. IEEE Trans Autom Control. 2021; 66(2): 794-801.
- 48Wabersich KP, Bayer FA, Müller MA, Allgöwer F. Economic model predictive control for robust periodic operation with guaranteed closed-loop performance. Paper presented at: Proceedings of the 2018 European Control Conference (ECC), Limassol, Cyprus; 2018:507-513.
- 49Bishop CM. Pattern Recognition and Machine Learning (Information Science and Statistics). Berlin/Heidelberg, Germany: Springer-Verlag; 2006.
10.1007/978-0-387-45528-0 Google Scholar
- 50Hastie T, Tibshirani R, Friedman J. The Elements of Statistical Learning. New York, NY: Springer New York Inc; 2001.
10.1007/978-0-387-21606-5 Google Scholar
- 51Slotani M. Tolerance regions for a multivariate normal population. Ann Inst Stat Math. 1964; 16(1): 135-153.
- 52Limon D, Alvarado I, Alamo T, Camacho E. On the design of robust tube-based MPC for tracking. IFAC Proc Vol. 2008; 41(2): 15333-15338.
10.3182/20080706-5-KR-1001.02593 Google Scholar
- 53Boyd S, Vandenberghe L. Convex Optimization. New York, NY: Cambridge University Press; 2004.
10.1017/CBO9780511804441 Google Scholar
- 54Derinkuyu K, Pınar MÇ. On the S-procedure and some variants. Math Meth Oper Res. 2006; 64(1): 55-77.
- 55Boyd S, El Ghaoui L, Feron E, Balakrishnan V. Linear Matrix Inequalities in System and Control Theory. SIAM Studies in Applied Mathematics. Philadelphia, PA: SIAM; 1994: 15.
- 56Rakovic SV, Kerrigan EC, Kouramas KI, Mayne DQ. Invariant approximations of the minimal robust positively invariant set. IEEE Trans Autom Control. 2005; 50: 406-410.
- 57Darup MS, Teichrib D. Efficient computation of RPI sets for tube-based robust MPC. Paper presented at: Proceedings of the 18th European Control Conference (ECC), Naples, Italy; 2019:325-330.
- 58Amrit R, Rawlings JB, Angeli D. Economic optimization using model predictive control with a terminal cost. Annu Rev Control. 2011; 35(2): 178-186.
- 59Wabersich KP, Zeilinger MN. Scalable synthesis of safety certificates from data with application to learning-based control. Paper presented at: Proceedings of the 2018 European Control Conference, ECC 2018; 2018:1691-1697
- 60Hu H, Feng X, Quirynen R, Villanueva ME, Houska B. Real-time tube MPC applied to a 10-state quadrotor model. Paper presented at: Proceedings of the 2018 Annual American Control Conference (ACC), Milwaukee, Wisconsin, USA; 2018:3135-3140.
- 61Bayer F, Bürger M, Allgöwer F. Discrete-time incremental ISS: a framework for robust NMPC. Paper presented at: Proceedings of the 2013 European Control Conference (ECC), Zurich, Switzerland; 2013:2068-2073.