For safety in automotive, for data privacy protection, or for the Internet of Things (IoT), to be legally compliant requires testing the impact of any action before allowing it to occur. However, system boundaries change at runtime, more and more software-intense products contain artificial intelligence (AI) and use deep learning instead of coded algorithms. Support vector machines (SVM) are readily used to exhibit behavior not testable before delivery. Tests, if any, must happen anytime, anywhere, becoming part of continuous delivery.
This tutorial explains the theory and implementation of the framework for autonomous real-time testing (ART) of a software-intense system while in operation for autonomous cars. The key to continuous testing complex systems is understanding the needs (or values) of the customer, or user, in general. The hype for autonomous cars is over but Advanced Driver Assistance Systems (ADAS) are state of the art and will appear in vehicles everywhere, shortly. They contain deep learning devices such as visual recognition systems that share learning among a wide range of cars.
ART addresses the need to automatically generate test cases based on an initial set of test stories. Automatic test case generation requires avoiding a combinatorial explosion. This is where quality function deployment (QFD) and the needs of drives come in. Using Six Sigma transfer functions, we can effectively limit the growth of test cases and keep focus on the train operator’s needs.
Neither QFD nor Six Sigma is widely known among automotive, testing, or DevOps engineers, and this tutorial will change that. Participants learn not only the theoretical background, but as well how to set up and manage ART for an ADAS, to avoid detecting software failures during operations.
The session schedule is as follows:
- Functional Size
- The ISO/IEC 19761 COSMIC approach
- Automated model construction
- Automated Measurement
Among some of the topics we will go over include:
- Test metrics
- Test size
- Test density
- Test coverage
- The convergence gap
- Defect density prediction
- Measuring test coverage
- Selecting relevant test cases
- Autonomous real-time testing
- Continuous delivery with CloudBees CodeShip
- Continuous safety metrics with CloudBees DevOptics
- The DevOps challenge for testing
- Discussion how such metrics can become a standard and who should set standards
Throughout this sessionl we will assume DevOps development methods, CI/CD and use its terminology.
