RESEARCH ARTICLE
Resilient control for wireless networked control systems under DoS attack via a hierarchical game
Huanhuan Yuan,
Yuanqing Xia,
Hongjiu Yang,
Yuan Yuan,
Huanhuan Yuan
School of Automation, Beijing Institute of Technology, Beijing, China
Search for more papers by this authorCorresponding Author
Yuanqing Xia
School of Automation, Beijing Institute of Technology, Beijing, China
Correspondence
Yuanqing Xia, School of Automation, Beijing Institute of Technology, Beijing 100081, China.
Email: [email protected]
Search for more papers by this authorHongjiu Yang
Institute of Electrical Engineering, Yanshan University, Qinhuangdao, China
Search for more papers by this authorYuan Yuan
Department of Computer Science, Brunel University London, Uxbridge, Middlesex, UK
Search for more papers by this authorHuanhuan Yuan,
Yuanqing Xia,
Hongjiu Yang,
Yuan Yuan,
Huanhuan Yuan
School of Automation, Beijing Institute of Technology, Beijing, China
Search for more papers by this authorCorresponding Author
Yuanqing Xia
School of Automation, Beijing Institute of Technology, Beijing, China
Correspondence
Yuanqing Xia, School of Automation, Beijing Institute of Technology, Beijing 100081, China.
Email: [email protected]
Search for more papers by this authorHongjiu Yang
Institute of Electrical Engineering, Yanshan University, Qinhuangdao, China
Search for more papers by this authorYuan Yuan
Department of Computer Science, Brunel University London, Uxbridge, Middlesex, UK
Search for more papers by this authorSummary
In this paper, the resilient control problem is investigated for a wireless networked control system (WNCS) under denial-of-service (DoS) attack via a hierarchical game approach. In the presence of a wireless network, a DoS attacker leads to extra packet dropout in the cyber layer of WNCS by launching interference power. A zero-sum Markov game is exploited to model the interaction between the transmitter and the DoS attacker under dynamic network environment. Additionally, with the attack-induced packet loss, an H∞ minimax controller is designed in the physical layer by using a delta operator approach. Both value iteration and Q-learning methods are used to solve the hierarchical game problem for the WNCS. The proposed method is applied to a load frequency control system to illustrate the effectiveness.
REFERENCES
- 1Chorppath AK, Alpcan T, Boche H. Bayesian mechanisms and detection methods for wireless network with malicious users. IEEE Trans Mob Comput. 2016; 15(10): 2452-2465.
- 2Jia L, Yao F, Sun Y, Niu Y, Zhu Y. Bayesian Stackelberg game for antijamming transmission with incomplete information. IEEE Commun Lett. 2016; 20(10): 1991-1994.
- 3Yang D, Xue G, Zhang J, Richa A, Fang X. Coping with a smart jammer in wireless networks: a Stackelberg game approach. IEEE Trans Wirel Commun. 2013; 12(8): 4038-4047.
- 4Li Y, Shi L, Cheng P, Chen J, Quevedo DE. Jamming attacks on remote state estimation in cyber-physical systems: a game-theoretic approach. IEEE Trans Autom Control. 2015; 60(10): 2831-2836.
- 5Zhang H, Cheng P, Shi L, Chen J. Optimal DoS attack scheduling in wireless networked control system. IEEE Trans Control Syst Technol. 2016; 24(3): 843-852.
- 6Yang C, Ren X, Yang W, Shi H, Shi L. Jamming attack in centralized state estimation. Paper presented at: 2015 34th Chinese Control Conference (CCC); 2015; Hangzhou, China.
- 7Li Y, Quevedo DE, Dey S, Shi L. SINR-based DoS attack on remote state estimation: a game-theoretic approach. IEEE Trans Control Netw Syst. 2017; 4(3): 632-642.
- 8Mo Y, Chabukswar R, Sinopoli B. Detecting integrity attacks on SCADA systems. IEEE Trans Control Syst Technol. 2014; 22(4): 1396-1407.
- 9Pang Z-H, Liu G-P. Design and implementation of secure networked predictive control systems under deception attacks. IEEE Trans Control Syst Technol. 2012; 20(5): 1334-1342.
- 10Zhu M, Martinez S. On the performance analysis of resilient control networked control systems under relay attacks. IEEE Trans Autom Control. 2014; 59(3): 804-808.
- 11Pang Z-H, Liu G-P, Zhou D, Hou F, Sun D. Two-channel false data injection attacks against output tracking control of networked control systems. IEEE Trans Ind Electron. 2016; 63(5): 3242-3251.
- 12Zhu Q, Basar T. Game-theoretic methods for robustness, security, and resilience of cyberphysical control systems: games-in-games principle for optimal cross-layer resilient control systems. IEEE Control Syst. 2015; 35(1): 46-65.
- 13Teixeira A, Pérez D, Sandberg H, Johansson KH. Attack models and scenarios for networked control systems. In: Proceedings of the 1st International Conference on High Confidence Networked Systems (HiCoNS); 2012; Beijing, China.
- 14De Persis C, Tesi P. Input-to-state stabilizing control under denial-of-service. IEEE Trans Autom Control. 2015; 60(11): 2930-2944.
- 15Feng S, Tesi P. Resilient control under denial-of-service: robust design. Automatica. 2017; 79: 42-51.
- 16Yuan Y, Yuan H, Guo L, Yang H, Sun S. Resilient control of networked control system under DoS attacks: a unified game approach. IEEE Trans Ind Inform. 2016; 12(5): 1786-1794.
- 17Ginoya D, Shendge PD, Phadke SB. Disturbance observer based sliding mode control of nonlinear mismatched uncertain systems. Commun Nonlinear Sci Numer Simul. 2015; 26(1-3): 98-107.
- 18Yao X, Guo L. Composite anti-disturbance control for Markovian jump nonlinear systems via disturbance observer. Automatica. 2013; 49(8): 2538-2545.
- 19Guo L, Cao S. Anti-disturbance control theory for systems with multiple disturbances: a survey. ISA Trans. 2014; 53(4): 846-849.
- 20Moon J, Başar T. Minimax control over unreliable communication channels. Automatica. 2015; 59: 182-193.
- 21Yuan Y, Sun F, Liu H. Resilient control of cyber-physical systems against intelligent attacker: a hierarchal Stackelberg game approach. Int J Syst Sci. 2016; 47(9): 2067-2077.
- 22Suchomski P. A J-lossless coprime factorisation approach to H∞ control in delta domain. Automatica. 2002; 38(10): 1807-1814.
- 23Yuan Y, Zhang P, Guo L, Yang H. Towards quantifying the impact of randomly occurred attacks on a class of networked control systems. J Franklin Inst. 2017; 354(12): 4966-4988.
- 24Yuan Y, Zhang P, Wang Z, Guo L, Yang H. Active disturbance rejection control for the ranger neutral buoyancy vehicle: a delta operator approach. IEEE Trans Ind Electron. 2017; 64(12): 9410-9420. https://doi.org/10.1109/TIE.2017.2711538
- 25Yang H, Xia Y, Shi P, Zhao L. Analysis and Synthesis of Delta Operator Systems. Berlin, Germany:Springer-Verlag Berlin Heidelberg; 2012.
10.1007/978-3-642-28774-9 Google Scholar
- 26Yang H, Xia Y, Shi P. Stabilization of networked control systems with nonuniform random sampling periods. Int J Robust Nonlinear Control. 2011; 21(5): 501-526.
- 27Yuan Y, Yuan H, Guo L, Yang H. Multi-tasking optimal control of networked control systems: a delta operator approach. Int J Robust Nonlinear Control. 2016; 27(16): 2842-2860. https://doi.org/10.1002/rnc.3714
- 28Yang H, Xia Y, Yuan H, Yan J. Quantized stabilization of networked control systems with actuator saturation. Int J Robust Nonlinear Control. 2016; 26(16): 3595-3610.
- 29Altman E, Avrachenkov K, Garnaev A. Jamming in wireless networks under uncertainty. Mob Netw Appl. 2011; 16(2): 246-254.
- 30Proakis J, Salehi M. Digital Communications, 5th ed. New York, NY: McGraw-Hill Education; 2007.
- 31Hasslinger G, Hohlfeld O. The Gilbert-Elliott model for packet loss in real time services on the internet. Paper presented at: 14th GI/ITG Conference - Measurement, Modelling and Evaluation of Computer and Communication Systems; 2008; Dortmund, Germany.
- 32Mo Y, Garone E, Sinopoli B. LQG control with Markovian packet loss. Paper presented at: 2013 European Control Conference (ECC); 2013; Zurich, Switzerland.
- 33Gilbert EN. Capacity of a burst-noise channel. Bell Syst Tech J. 1960; 39(5): 1253-1265.
- 34Yang H, Xu Y, Zhang J. Event-driven control for networked control systems with quantization and Markov packet losses. IEEE Trans Cybern. 2017; 47(8): 2235-2243.
- 35Başar T, Bernhard P. H∞ - Optimal Control and Related Minimax Design Problems: A Dynamic Game Approach, 2nd ed. New York, NY: Springer Science+Media LLC; 1995.
- 36Littman ML. Markov games as a framework for multi-agent reinforcement learning. In: Proceedings of the Eleventh International Conference; 1994; New Brunswick, NJ.
- 37Shapley LS. Stochastic games. Proc Natl Acad Sci USA. 1953; 39(10): 1095-1100.
- 38Hu J, Wellman MP. Nash Q-learning for general-sum stochastic games. J Mach Learn Res. 2003; 4: 1039-1069.
- 39Watkins CJCH, Dayan P. Q-learning. Mach Learn. 1992; 8(3-4): 279-292.
- 40Raghavan TES, Filar JA. Algorithms for stochastic games - a survey. Math Methods Oper Res. 2003; 35(6): 437-472.
10.1007/BF01415989 Google Scholar
- 41Al-Tamimi A, Lewis FL, Abu-Khalaf M. Mode-free Q-learning designs for linear discrete-time zero-sum games with application to H∞ control. Automatica. 2007; 43(3): 473-481.
- 42Wang B, Wu Y, Liu KJR, Clancy TC. An anti-jamming stochastic game for cognitive radio networks. IEEE J Sel Area Commun. 2011; 29(4): 877-889.
- 43Han Q, Yang B, Wang X, et al. Hierarchical-game-based uplink power control in femtocell networks. IEEE Trans Veh Technol. 2014; 63(6): 2819-2835.
- 44Mi Y, Fu Y, Wang C, Wang P. Decentralized sliding mode load frequency control for multi-area power systems. IEEE Trans Power Syst. 2013; 28(4): 4301-4309.
