A Standard for Trustworthiness Testing of AI Systems
This page provides a structured overview of the OWASP AI Testing Guide v1.0, a unified framework for testing the security and trustworthiness of AI systems across application, model, infrastructure, and data layers.
This is an explanatory companion to the OWASP AI Testing Guide v1.0, not a replacement. Use it to:
- Understand the high‑level structure and objectives of the Guide.
- Map testing activities to your AI architecture layers.
- Prioritize test families for your threat model.
Stay Updated on Security Guides
Get updates when new OWASP AI security guidance and testing frameworks are published.
Overview & Purpose
The OWASP AI Testing Guide establishes a practical standard for trustworthiness testing of AI systems. It goes beyond traditional security testing by addressing how AI systems learn, adapt, and fail in non‑deterministic ways, and provides a repeatable method to verify that AI behaves safely as intended.
The Guide organizes testing across four primary layers: AI Application, AI Model, AI Infrastructure, and AI Data. Each layer is further broken down into concrete test cases (AITG-APP, AITG-MOD, AITG-INF, AITG-DAT) that can be integrated into existing security testing programs.
Principles & Objectives
- Establish a standardized, technology‑agnostic methodology for AI testing.
- Cover the full AI lifecycle – from data collection and model training to deployment, monitoring, and runtime behavior.
- Align with Responsible / Trustworthy AI principles: security, safety, fairness, privacy, and explainability.
- Provide reusable test cases that can be embedded into existing AppSec and QA pipelines.
- Define a standardized methodology for AI & LLM security and trustworthiness testing.
- Provide repeatable test cases across application, model, infrastructure, and data layers.
- Align with emerging standards such as NIST AI RMF, SAIF, ISO/IEC 42001, and OWASP Top 10 for LLMs.
- Support risk, compliance, and engineering teams with a common testing vocabulary.
Why AI Testing is Unique
Traditional software testing assumes deterministic behavior and static code. AI systems— particularly ML and LLM‑based systems—are probabilistic, data‑driven, and adaptive. They can be manipulated through inputs, poisoned through data, or degraded over time.
The Guide emphasizes that security alone is insufficient: what matters is overall trustworthiness, including robustness, safety, fairness, privacy, and transparency.
- Prompt injection, jailbreaks, and model evasion.
- Sensitive information leakage and data exfiltration.
- Hallucinations, misinformation, and unsafe outputs.
- Data/model poisoning across the supply chain.
- Excessive or unsafe agency of agentic systems.
- Bias, unfairness, and misalignment with policies.
Threat Modeling AI Systems
Chapter 2 of the Guide focuses on threat modeling AI systems, with an emphasis on mapping threats to AI architectural components and identifying Responsible AI / Trustworthy AI risks.
- Identify AI System Threats – enumerate how adversaries can influence inputs, models, tools, and outputs.
- Map OWASP AI Threats to AI Architectural Components (2.1.1) – connect threats to specific components (data pipelines, models, agents, plugins, infra).
- Identify Responsible/Trustworthy AI Threats (2.1.2) – e.g., bias, explainability gaps, safety failures, over‑reliance on AI.
The Guide recommends integrating AI‑specific threat modeling into existing processes such as STRIDE, LINDDUN, NIST AI RMF, and OWASP threat modeling practices.
- AI data‑flow diagrams with model, data, and tool boundaries.
- Threat‑to‑component mapping (application, model, infra, data).
- Attack trees for high‑impact scenarios (e.g., prompt injection → unsafe actions).
- Testing plan that selects relevant AITG tests for each component.
OWASP AI Testing Framework
Chapter 3 defines the OWASP AI Testing Framework, organized into four main testing domains. Each domain contains numbered test cases (AITG‑APP, AITG‑MOD, AITG‑INF, AITG‑DAT) that can be turned into concrete test procedures, playbooks, or automation.
Tests the behavior of AI applications, prompts, tools, and user interactions.
Prefix: AITG‑APP‑xx
Evaluates model robustness, poisoning, privacy, and alignment characteristics.
Prefix: AITG‑MOD‑xx
Focuses on runtime environment, plugins, supply chain, and resource abuse.
Prefix: AITG‑INF‑xx
Assesses training, inference, and dataset properties, including privacy and quality.
Prefix: AITG‑DAT‑xx
This matrix is an independent visualization built from the OWASP AI Testing Guide v1.0 to help map risks to AITG test families. For authoritative definitions and procedures, see the official OWASP project repository at OWASP/www-project-ai-testing-guide.
AI Application Testing (AITG‑APP)
Application‑level tests focus on how AI applications interact with users, tools, prompts, and external systems. They are particularly relevant for LLM apps, chatbots, agentic systems, and RAG pipelines.
- AITG‑APP‑01 – Testing for Prompt Injection
- AITG‑APP‑02 – Testing for Indirect Prompt Injection
- AITG‑APP‑07 – Testing for Prompt Disclosure
- AITG‑APP‑08 – Testing for Embedding Manipulation
- AITG‑APP‑03 – Testing for Sensitive Data Leak
- AITG‑APP‑04 – Testing for Input Leakage
- AITG‑APP‑05 – Testing for Unsafe Outputs
- AITG‑APP‑12 – Testing for Toxic Output
- AITG‑APP‑11 – Testing for Hallucinations
- AITG‑APP‑06 – Testing for Agentic Behavior Limits
- AITG‑APP‑10 – Testing for Content Bias
- AITG‑APP‑13 – Testing for Over‑Reliance on AI
- AITG‑APP‑14 – Testing for Explainability & Interpretability
AI Model Testing (AITG‑MOD)
Model‑level tests examine how models behave under adversarial conditions, how they protect training data, and how robust they are to drift and new data.
- AITG‑MOD‑01 – Testing for Evasion Attacks
- AITG‑MOD‑06 – Testing for Robustness to New Data
- AITG‑MOD‑07 – Testing for Goal Alignment
- AITG‑MOD‑02 – Testing for Runtime Model Poisoning
- AITG‑MOD‑03 – Testing for Poisoned Training Sets
- AITG‑MOD‑04 – Testing for Membership Inference
- AITG‑MOD‑05 – Testing for Inversion Attacks
- Related: model extraction tests mapped from OWASP & NIST AML taxonomy.
AI Infrastructure Testing (AITG‑INF)
Infrastructure‑level tests focus on the runtime environment, tool and plugin boundaries, resource controls, and the AI/ML supply chain.
- AITG‑INF‑01 – Testing for Supply Chain Tampering
- AITG‑INF‑06 – Testing for Dev‑Time Model Theft
- AITG‑INF‑05 – Testing for Fine‑tuning Poisoning
- AITG‑INF‑02 – Testing for Resource Exhaustion
- AITG‑INF‑03 – Testing for Plugin Boundary Violations
- AITG‑INF‑04 – Testing for Capability Misuse
AI Data Testing (AITG‑DAT)
Data‑level tests verify that training and inference data handling is privacy‑preserving, well‑governed, and fit for purpose.
- AITG‑DAT‑01 – Testing for Training Data Exposure
- AITG‑DAT‑02 – Testing for Runtime Exfiltration
- AITG‑DAT‑03 – Testing for Dataset Diversity & Coverage
- AITG‑DAT‑04 – Testing for Harmful Content in Data
- AITG‑DAT‑05 – Testing for Data Minimization & Consent
Appendices, Mappings & References
- Appendix A – Rationale for using SAIF (Secure AI Framework).
- Appendix B – DIE (Distributed, Immutable, Ephemeral) threat identification.
- Appendix C – Risk lifecycle for secure AI systems.
- Appendix D – Threat enumeration mapped to AI architecture components.
- Appendix E – Mapping AI threats to CVEs, CWEs, and real‑world vulnerabilities.
- NIST AI Risk Management Framework (AI RMF 1.0).
- ISO/IEC 42001 – AI management system requirements.
- OWASP Top 10 for LLM Applications.
- OWASP Web Security Testing Guide & AI Red Teaming Framework.
- MITRE ATLAS and other AI‑specific knowledge bases.
For full details, consult the official OWASP AI Testing Guide v1.0 PDF and the referenced standards.
Stay Updated on Security Guides
Stay updated on new AI security testing methodologies, OWASP guidance, and red teaming playbooks.
Nessus Vulnerability Scanner
Partner SolutionThe industry's most widely deployed vulnerability scanner. Identify security vulnerabilities, misconfigurations, and compliance issues across your infrastructure, cloud, and container environments. Essential for AI security assessments and penetration testing.
BlackBox AI Code Generation Platform
Partner ToolAI-powered code generation platform for developers. Generate, test, and secure AI code with advanced security features. Perfect for building secure AI applications and testing code vulnerabilities.