Policy development & AI governance
Create and revise policies, map them to controls, and organize approved AI tools and their reviews.
Why AI Forces a New Trust Model
AI does not fit neatly into existing trust frameworks. It requires a new operating model, one built around explainability, monitoring, and accountability.
Published March 1, 2025 by Megan Fulton.
Category: AI & Security
What you’ll take away
- What Changes with AI
- The EU AI Act: A Framework in Practice
- A New Trust Model for AI
Our point of view
AI governance and technical security evaluation are complementary, not interchangeable. A policy cannot establish technical behavior, and a test cannot assign business accountability.
Key points
- Governance defines approved uses, data boundaries and human oversight.
- Testing examines agreed behavior and configurations under authorized conditions.
- Findings need owners, decisions and review when systems change.
Artificial intelligence does not fit cleanly into existing trust frameworks. The controls that govern how organizations protect data, manage access, and ensure system reliability were designed for systems with predictable, deterministic behavior. AI systems are not deterministic. Their outputs are probabilistic. Their decision-making is often opaque. Their behavior can change as underlying models are updated.
This creates a trust gap that existing compliance programs do not address. An organization can be fully SOC 2 compliant, with excellent access controls and comprehensive monitoring, while operating AI systems that produce discriminatory outputs, make high-stakes decisions without human oversight, or expose sensitive training data through model inference.
What Changes with AI
- Transparency: AI systems often cannot explain their decisions in the way that traditional systems can. Trust frameworks require that organizations understand and can account for how their systems behave.
- Accountability: when an AI system makes a consequential error, identifying where the failure occurred — in training data, model design, deployment configuration, or use case definition — requires different analytical approaches than debugging traditional software.
- Data governance: AI systems are trained on data, and the characteristics of that data — its quality, its biases, its provenance — directly affect system outputs. Traditional data governance frameworks do not address these AI-specific dimensions.
- Third-party AI: organizations are increasingly using AI capabilities embedded in vendor products, creating a new category of third-party risk that existing vendor assessment frameworks are not equipped to evaluate.
- Regulatory requirements: AI-specific regulations are emerging globally, including the EU AI Act, NIST AI RMF, and sector-specific guidance in healthcare and financial services. Compliance programs must evolve to incorporate these requirements.
The EU AI Act: A Framework in Practice
The European Union AI Act establishes a risk-based regulatory framework for AI systems, with requirements that scale based on the risk level of the AI application. High-risk AI systems — those used in consequential decisions about employment, credit, education, or public services — face mandatory conformity assessment requirements, technical documentation obligations, and ongoing post-market monitoring.
For organizations operating in or serving customers in the EU, the AI Act creates compliance obligations that must be addressed now. The high-risk provisions are being phased in over 2024 and 2025. Organizations that have not begun assessing their AI systems against the Act's requirements are behind.
A New Trust Model for AI
AI governance requires extending the existing trust model in three dimensions. First, AI-specific controls must be added to cover the unique risks that AI systems create — controls around model validation, data governance for training data, human oversight requirements, and monitoring for model drift and performance degradation.
Second, vendor AI risk assessment must become a distinct component of third-party risk management. A vendor that uses AI to deliver its service creates risks that traditional vendor security assessments do not capture.
Third, AI governance documentation — the technical documentation, conformity assessments, and impact assessments required by emerging regulations — must be integrated into the organization's broader compliance documentation infrastructure.
How VioletX Approaches AI Governance
VioletX is developing AI governance capabilities that build on its existing trust infrastructure model. Organizations working with VioletX on AI governance build programs aligned with current regulatory requirements, operated with the same continuous assurance discipline that defines VioletX's compliance programs, and designed to adapt as the regulatory landscape continues to evolve.
AI does not change what trust requires. It raises the bar for proving it.
How VioletX helps
Relevant work is scoped to your systems, responsibilities, and decision boundaries.
AI / ML security testing
Assess agreed AI-powered applications for prompt injection, sensitive-data leakage, and unsafe tool access.
More information
Understand the distinct roles of policies, tool approvals, human oversight and technical testing.
AI, with Accountability: AI governance & scoped security evaluationRelated reading
- AI governance and AI security testing solve different problems — Separate policy and accountability from authorized technical evaluation.
- What a penetration test does—and does not—tell your board — Interpret a scoped test without turning a point-in-time finding into an assurance guarantee.