The 4th International Workshop on Reliability and Testing for Artificial Intelligence Systems (RTAIS 2023)


Artificial intelligence systems have played important roles in many fields such as image recognition, speech recognition, machine translation, and intrusion detection. At present, artificial intelligence systems have an increased role in safety-critical systems, which makes its correctness and reliability crucial. The intelligent characteristic and learning capability of artificial intelligence systems bring a new failure type that traditional software does not have. Therefore, it is necessary to put forward suitable and effective testing techniques and reliability assurance methods for artificial intelligence systems.

This workshop seeks to bring together researchers and practitioners working toward the improvement of reliability for intelligent systems by discussing recent developments and current challenges and exchanging their research findings and experiences. The workshop welcomes papers and presentations in the field of artificial intelligence systems, dealing with intelligent software reliability and testing.


The list of topics includes, but is not limited to:

  • Artificial intelligence system failure mechanism
  • Machine learning attack and threat models
  • Defect detection, defect removal, and fault tolerance in artificial intelligence system
  • Test method, test workflow, and test adequacy for artificial intelligence system
  • Fuzz testing and mutation testing for machine learning and deep learning
  • Testing technology of autopilot software, internet of things (IoT), and natural language interface (NLI)
  • Artificial intelligence system reliability evaluation methodologies and metrics
  • Correctness and predictability of artificial intelligence system behavior
  • Reliability, robustness, and vulnerability of machine learning and deep learning
  • Robustness and adversarial perturbation in neural network
  • Dependability, reliability, safety and robustness of autopilot software, internet of things (IoT), and natural language interface (NLI)
  • Architecture, design, and implementation of reliable artificial intelligence system
  • Data quality of artificial intelligence system and bug detection in data
  • Practical experiences, empirical studies, and testbeds for artificial intelligence system
  • Industrial experiences and best practices for artificial intelligence system testing
  • Reliability and testing studies of any other artificial intelligence related systems


Authors are invited to submit original unpublished research papers as well as industrial practice papers. Simultaneous submissions to other conferences are not permitted.

Papers should be written in English and submitted in PDF format. The length of a camera ready paper will be limited to ten pages, including the title of the paper, the name and affiliation of each author, a 150-word abstract, and up to 6 keywords. Shorter version papers (up to four pages) are also allowed.

Authors must follow the DSA conference proceedings format (PDF | Word DOCX | Latex) to prepare their papers. Each submission will be reviewed by at least three program committee members. Paper selection is based on originality, technical contribution, presentation quality, and relevance to the workshop.

At least one of the authors of each accepted paper is required to pay a full registration fee and present the paper at the workshop. Arrangements are being made to publish selected accepted papers in reputable journals.


Program Chairs

Jun Ai's avatar
Jun Ai China

Beihang University, China

Xing-Ya Wang's avatar
Xing-Ya Wang China

Nanjing Tech University, China

Program Committee

Jun AiBeihang UniversityChina
Minyan LuBeihang UniversityChina
Lingzhong MengInstitute of Software, Chinese Academy of SciencesChina

Previous RTAIS

  • RTAIS 2022 - Wulumuqi (in conjunction with DSA 2022)
  • RTAIS 2021 - Yingchuan (in conjunction with DSA 2021)
  • RTAIS 2020 - Xi'an (in conjunction with DSA 2020)