OWASP Artificial Intelligence Vulnerability Scoring System

This initiative centers on developing and delivering a comprehensive AIVSS (Artificial Intelligence Vulnerability Scoring System) Framework Package for evaluating security risks across all types of AI systems – not just Large Language Models (LLMs) or Generative AI. The project aims to provide a structured and quantifiable methodology to identify, assess, and mitigate vulnerabilities specific to AI, leading to a complete AIVSS Framework Package that serves as a baseline for understanding AI security risks. The core deliverable is a standardized and scalable AIVSS framework validated across a diverse range of AI applications and models. This project is an independent effort focused on building a robust, generalizable, and community-driven AI security assessment tool.

This project is proposed to be a Top-Level OWASP project because:

Fills a Critical Gap in AI Security (Broad Scope): It moves beyond securing specific AI implementations and focuses on quantifying the offensive potential and vulnerabilities inherent in the underlying AI technologies themselves. This is a fundamental shift that is currently underserved within existing OWASP projects. This universal focus positions AIVSS as a vital resource for understanding and mitigating risks across the entire AI landscape.

Addresses Emerging Threats (Future-Proofing): As AI technology evolves and diversifies, AIVSS provides a framework that can adapt to new types of models, algorithms, and applications. This future-proof approach ensures that OWASP remains at the forefront of AI security, addressing emerging threats as they arise.

Provides a Standardized Scoring System (Universal Applicability): The AIVSS framework offers a quantifiable and standardized approach to assessing AI security vulnerabilities, regardless of the specific AI technology being used. This allows for consistent comparisons across different AI systems, facilitates informed decision-making, and enables benchmarking across the industry.

Promotes Collaboration and Knowledge Sharing (Open and Inclusive): By developing the AIVSS framework as an open-source project under the OWASP umbrella, this initiative fosters collaboration and knowledge sharing among security professionals, AI researchers, and industry stakeholders across all AI domains. This collaborative approach is essential for accelerating innovation in AI security and ensuring that the AIVSS framework remains relevant and effective.

Global Impact (Technology Agnostic): The security of AI systems is a global concern, irrespective of the specific AI technology in use. By becoming a Top-Level OWASP project, AIVSS will have a greater reach and impact, helping to improve the security of AI systems worldwide. Its technology-agnostic design facilitates its adoption across different regions and sectors.

Potential for Wide Adoption (Versatility): The framework and tools are designed to be broadly applicable to all types of AI systems, from traditional machine learning models to cutting-edge deep learning algorithms. This potential for wide adoption makes it a strong candidate for a Top-Level project, impacting a vast range of applications.

Alignment with OWASP’s Mission (Core Focus): AIVSS directly addresses the core mission of OWASP by providing a framework for securing a rapidly evolving and increasingly critical technology – AI. Its focus on identifying, assessing, and mitigating vulnerabilities in AI systems aligns perfectly with OWASP’s commitment to securing the web and its underlying technologies.

Additional justification for this new project:

Strategic Importance (Across the AI Spectrum): Given the increasing reliance on AI across all sectors and the potential for widespread impact from AI-related vulnerabilities, addressing AI security is a strategic imperative for OWASP.

Specialized Expertise (Beyond LLMs): While knowledge of LLMs is valuable, the AIVSS requires expertise in a broader range of AI technologies, including traditional machine learning, computer vision, and robotics.

Applicable Across AI Architectures: Tangible deliverables (Framework Guide, Scoring Calculator, Assessment Reports) that are designed to be adapted across diverse AI architectures are in high need.

Adaptable for Future Technologies: Designed for extensibility, the AIVSS framework can incorporate any new AI technologies or models, increasing the project relevance.

Road Map

The following is the roadmap:

1: AIVSS Framework Core Definition (Months 1-3):

Define the core AIVSS metrics, ensuring they are universally applicable to AI systems. Focus on clarity and precision in definitions.

Develop initial scoring rubrics for these core metrics, providing general guidance for assigning scores.

2: AIVSS Framework Specialization (Months 4-6):

Develop specialized scoring rubrics for specific AI system types. This involves identifying factors unique to each AI type that influence vulnerability assessment.

Create templates for AIVSS assessment reports, ensuring they can be adapted to different AI system types.

3: AIVSS Scoring Calculator Development (Months 7-9):

Develop the core functionality of the AIVSS scoring calculator, ensuring it supports the core AIVSS metrics and scoring rubrics.

Implement the ability to add new AIVSS metrics and scoring rubrics to the calculator.

4: AIVSS Tool Testing and Refinement (Months 10-12):

Run the AIVSS scoring calculator against a diverse set of AI systems, generating AIVSS assessment reports for each.

Refine the AIVSS metrics, scoring rubrics, and calculator based on feedback from users and the results of the assessments.

5: Documentation and Release (Month 12):

Finalize the AIVSS Framework Guide, ensuring it is comprehensive, clear, and easy to understand.

Release the AIVSS Scoring Calculator as an open-source project under a permissive license.

Publish the AIVSS assessment reports on the OWASP project website.

Create a detailed documentation of the project, including a guide for integrating the AIVSS framework into existing software development lifecycles.

Community Engagement (Ongoing Throughout Project):

Month 1: Create a project Slack channel.

Month 2: Start bi-weekly community meetings (30-60 minutes).

Months 3, 6, 9, 12: Publish blog posts on the OWASP website or Medium/Substack summarizing project progress, highlighting community contributions, and soliciting feedback.

Actively solicit contributions from the community, such as new AIVSS metrics, scoring rubrics, assessment reports, or code improvements.

Participate in industry conferences and workshops to promote the AIVSS framework and engage with the broader AI security community. Present project findings and solicit feedback.

Multi-Year Project Suggestions:

Year 2: Apply AIVSS to the Financial and Healthcare Industries: Develop industry-specific guidelines for applying the AIVSS framework to AI systems used in the financial and healthcare sectors. This may involve developing new AIVSS metrics or scoring rubrics that are tailored to the unique risks and challenges of these industries.

Create case studies and assessment reports showcasing the application of AIVSS to real-world AI systems in the financial and healthcare industries.

Collaborate with industry experts to validate the AIVSS framework and ensure that it is aligned with industry best practices.

Year 2/3: Expand the scope of the AIVSS framework to address emerging AI security threats, such as new threats in Agentic AI, Develop new AIVSS metrics and scoring rubrics to address these threats.

Year 3+: Explore the possibility of creating a certification program for security professionals who are proficient in using the AIVSS framework. This could help to drive adoption of the AIVSS framework and raise the overall level of AI security expertise in the industry.