Kintsugi AI

AI agents are becoming capable of using tools, accessing data, communicating with other agents, and taking real-world actions—but unlike traditional software, there is no widely adopted way to systematically *crash-test* an agent before deploying it.

A small prompt injection, unsafe tool call, excessive privilege, or unexpected agent-to-agent interaction can turn an otherwise useful agent into a security and reliability risk. Traditional unit tests are not enough because agent behavior is dynamic, context-dependent, and often non-deterministic.

**Agent CrashTest** is a security and reliability testing platform for AI agents. It creates controlled scenarios, executes agents inside isolated environments, routes tool and agent-to-agent interactions through policy controls, tracks the complete execution trajectory, and evaluates the resulting behavior against security and safety invariants.

Teams can use it to:

* **Red-team AI agents** against prompt injection, tool misuse, privilege escalation, unsafe actions, and data-leakage scenarios.
* **Test agent-to-agent (A2A) interactions** and delegation boundaries.
* **Generate adaptive adversarial scenarios** instead of relying only on a fixed test suite.
* **Reproduce failures** using captured state, trajectories, and deterministic scenario definitions.
* **Evaluate and certify agents** based on observable behavior and policy violations rather than subjective model output alone.
* **Retest after remediation** to verify that a previously discovered failure has actually been fixed.

The goal is simple: **give AI agents the equivalent of a crash-test laboratory before they are trusted with real tools, data, and actions.**

Repository: `https://github.com/pankajg-git/Kintsugi_Demo`

### 1. Testing Agent Behavior Without Making the Test Environment the Problem

The biggest challenge was creating realistic agent interactions while keeping tests isolated and reproducible. An agent may call tools, change state, delegate work to another agent, or fail halfway through an execution.

We addressed this by separating the execution pipeline into dedicated components for **scenario generation, environment state, agent execution, interaction routing, tool access, policy enforcement, trajectory tracking, and evaluation**. This lets us test individual security boundaries while still exercising the complete agent workflow.

### 2. Persistence and State Across Failures

Agent testing is highly stateful. A test can modify the environment before failing, and restarting the process must not silently lose the state required to reproduce the failure.

We introduced explicit environment-state tracking, database-backed snapshots/savepoints, state fingerprinting, and trajectory persistence so executions can be inspected and replayed rather than treated as one-off LLM conversations.

### 3. Making Security Decisions Deterministic

LLM-based evaluation alone is unreliable for proving that a security violation occurred. We therefore separated **execution from evaluation** and designed policy/invariant checks around observable events such as tool calls, authorization decisions, state transitions, and data movement.

This gives us a deterministic layer that can identify concrete violations while still allowing AI-assisted analysis to explain what happened.

### 4. Adaptive Scenario Generation

A static list of attack prompts quickly becomes predictable. We built scenario-generation and adaptive-planning components that can create and execute scenarios based on the target agent and previous execution results, allowing the testing process to explore different failure paths instead of repeatedly running the same cases.

### 5. Integration Across the Full Pipeline

The hardest part was not building individual components—it was making the entire chain work together:

**Scenario → Environment → Controller → Agent → Tool Gateway → Policy Engine → State → Trajectory → Evaluation → Finding → Replay → Remediation → Retest**

We continuously tested API boundaries, failure injection, concurrent operations, A2A communication, persistence, and frontend/backend integration to catch issues that unit tests alone would miss.