Certified Tester AI Testing (CT-AI)
Overview
The ISTQB® AI Testing (CT-AI) certification extends understanding of artificial intelligence and/or deep (machine) learning, most specifically testing AI-based systems and using AI in testing.
Audience
The Certified Tester AI Testing certification is aimed at anyone involved in testing AI-based systems and/or AI for testing. This includes people in roles such as testers, test analysts, data analysts, test engineers, test consultants, test managers, user acceptance testers, and software developers. This certification is also appropriate for anyone who wants a basic understanding of testing AI-based systems and/or AI for testing, such as project managers, quality managers, software development managers, business analysts, operations team members, IT directors, and management consultants.
To gain this certification, candidates must hold the Certified Tester Foundation Level certificate.
Content
ISTQB® Certified Tester – AI Testing (CT-AI)
Introduction to AI
Definition of IA and AI Effect
Narrow, General and Super AI
AI-Based and Conventional Systems
AI Technologies
AI Development Frameworks
Hardware for AI-Based Systems
AI as a Service (AIaaS)
Pre-Trained Models
standards, Regulations and AI
Quality Characteristic for AI Based Systems
Flexibility and Adaptability
Autonomy
Evolution
Bias
Ethics
Side Effects and Reward Hacking
Transparency, Interpretability and Explainability
Safety and AI
Machine Learning (ML) – Overview
forms of ML
ML Workflow
Selecting a Form of ML
factors Involved in ML Algorithm Selection
Overfitting and Underfitting
ML Data
Data Preparation as Part of the ML Workflow
Training, Validation and Test Datasets in the ML Workflow
Dataset Quality Issues
Data Quality and its Effect on the ML Model
Data Labelling for Supervised Learning
ML Functional performance Metrics
Confution Matrix
Add ML Functional performance Metrics for Classification, Regression and Clustering
Limitations of ML Functional Performance Metrics
Selecting ML Functional Performance Metrics
Benchmark Suites for ML Performance
ML Neural
Networks and Testing
Neural Networks
Coverage Measures for Neural Networks
Testing AI-Based Systems – Overview
specification of AI-Based Systems
Test Levels of AI-Based systems
Test Data for Testing
AI-Based Systems
Testing for Automation Bias in AI-Based Systems
Documenting an AI-Based Component
Testing for Concept Drift
Selecting a Test Approach for an ML System
Testing AI-Specific Quality Characteristics
Challenges Testing Self-Learning Systems
Testing Autonomous Self-Learning Systems
Testing for Algorithmic, Sample and Inappropriate Bias
Challenges Testing Probabilistic and Non-Deterministic AI-Based Systems
Challenges Testing Complex AI-Based Systems
Testing Transparency Interpretability and Explainability of AI-Based Systems
Test Oracles for AI-Based Systems
Test objectives and
Acceptance Criteria
Methods and Techniques for the Testing of AI-Based Systems
Adversarial Attacks and Data Poisoning
Pairwise Testing
A/B Testing
Back-to-Back Testing
Metamorphic Testing (MT)
Experience-Based Testing of AI-Based Systems
Selecting Test Techniques for AI-Based Systems
Testing Environments for AI-Based Systems
Test Environments for AI-Based Systems
Virtual Test Environments for Testing AI-Based Systems
Using AI for Testing
AI Technologies for Testing
Using AI to Analyze Defect Reports
Using AI to Test Case Generation
Using AI for the Optimization of Regression Test Suites
Using AI for Defect Prediction
Using AI for Tseting User Interfaces
Exam Structure
- No. of Questions: 40
- Passing Score: 31
- Total Points: 47
- Exam Length (mins): 60 (+25% Non-Native Language)
Business Outcomes
Individuals who hold the ISTQB® Certified Tester- AI Testing certification should be able to accomplish the following business outcomes:
- Understand the current state and expected trends of AI
- Experience the implementation and testing of a ML model and recognize where testers can best influence its quality
- Understand the challenges associated with testing AI-Based systems, such as their self-learning capabilities, bias, ethics, complexity, non-determinism, transparency and explainability
- Contribute to the test strategy for an AI-Based system
- Design and execute test cases for AI-based systems
- Recognize the special requirements for the test infrastructure to support the testing of AI-based systems
- Understand how AI can be used to support software testing
More Information
- The other AI relates syllabi from A4Q, AiU and CSTQB/KSTQB will be valid to October 12th 2022
- Accredited courses require accreditation of training materials, as described in the ISTQB® Accreditation Process
- Holders of the A4Q, AiU and KSTQB & CSTQB AIT previous versions continue to hold a valid certification
Training is available from Accredited Training Providers (classroom, virtual, and e-learning). We highly recommend attending accredited training as it ensures that an ISTQB® Member Board has assessed the materials for relevance and consistency against the syllabus.
Self-study, using the syllabus and recommended reading material, is also an option when preparing for the exam.
Holders of this certification may choose to proceed to other Core, Agile, or Specialist stream certifications.
