CAN/CSA-ISO/IEC TR 29119-11-2021 Software and systems engineering - Software testing - Part 11: Guidelines on the testing of AI-based systems (ISO/IEC TR 29119-11:2020, IDT)
Full Description CSA Preface Standards development within the Information Technology sector is harmonized with international standards development. Through the CSA Technical Committee on Information Technology (TCIT), Canadians serve as the SCC Mirror Committee (SMC) on ISO/IEC Joint Technical Committee 1 on Information Technology (ISO/IEC JTC1) for the Standards Council of Canada (SCC), the ISO member body for Canada and sponsor of the Canadian National Committee of the IEC. Also, as a member of the International Telecommunication Union (ITU), Canada participates in the International Telegraph and Telephone Consultative Committee (ITU-T). For brevity, this Standard will be referred to as \"CSA ISO/IEC TR 29119-11\" throughout. At the time of publication, ISO/IEC TR 29119-11:2020 is available from ISO and IEC in English only. CSA Group will publish the French version when it becomes available from ISO and IEC. This Standard has been formally approved, without modification, by the Technical Committee and has been developed in compliance with Standards Council of Canada requirements for National Standards of Canada. It has been published as a National Standard of Canada by CSA Group. Scope This document provides an introduction to AI-based systems. These systems are typically complex (e.g. deep neural nets), are sometimes based on big data, can be poorly specified and can be non-deterministic, which creates new challenges and opportunities for testing them. This document explains those characteristics which are specific to AI-based systems and explains the corresponding difficulties of specifying the acceptance criteria for such systems. This document presents the challenges of testing AI-based systems, the main challenge being the test oracle problem, whereby testers find it difficult to determine expected results for testing and therefore whether tests have passed or failed. It covers testing of these systems across the life cycle and gives guidelines on how AI-based systems in general can be tested using black-box approaches and introduces white-box testing specifically for neural networks. It describes options for the test environments and test scenarios used for testing AI-based systems. In this document an AI-based system is a system that includes at least one AI component.
CAN/CSA-ISO/IEC TR 29119-11-2021 history
2021CAN/CSA-ISO/IEC TR 29119-11-2021 Software and systems engineering - Software testing - Part 11: Guidelines on the testing of AI-based systems (ISO/IEC TR 29119-11:2020, IDT)