v1.1.0 release - Contributors, Sponsors and Enquiries are most welcome 😌

Red Teaming

AI safety & red-teaming toolkit for proactive security testing of AI agents — adversarial attacks, vulnerability scanning, jailbreak detection, and compliance checking.

The red-teaming toolkit lets you proactively probe AI agents and LLM applications for vulnerabilities — generating adversarial attacks, scanning for prompt injection and data leakage, detecting jailbreaks, and verifying regulatory compliance before they reach production.

Installation

bash
pnpm add @lov3kaizen/agentsea-redteam

Quick Start

Create a red team instance, point it at your target, and run the full suite of security tests:

typescript
import {
  createRedTeam,
  createAttackLibrary,
  createVulnerabilityScanner,
} from '@lov3kaizen/agentsea-redteam';

// Create a red team instance
const redTeam = createRedTeam({
  config: {
    target: {
      type: 'agent',
      name: 'my-agent',
      endpoint: 'https://api.example.com/chat',
    },
  },
});

// Run security tests
const results = await redTeam.run();
console.log('Risk Score:', results.summary.riskScore);
console.log('Vulnerabilities:', results.summary.vulnerabilities);

Attack Library

Generate and manage adversarial attacks:

typescript
import {
  createAttackLibrary,
  createAttackRegistry,
  createMutationGenerator,
  createCombinationGenerator,
  createAdversarialGenerator,
} from '@lov3kaizen/agentsea-redteam';

// Use the default attack library
const library = createAttackLibrary();

// Create mutation-based attacks
const mutator = createMutationGenerator({
  strategies: ['character-swap', 'encoding', 'obfuscation'],
});

// Combine attack strategies
const combiner = createCombinationGenerator({
  strategies: ['sequential', 'nested', 'layered'],
});

// Generate adversarial inputs
const adversarial = createAdversarialGenerator({
  strategies: ['roleplay', 'hypothetical', 'translation'],
});

Vulnerability Scanning

Scan your agents for common vulnerabilities:

typescript
import {
  createVulnerabilityScanner,
  createPromptAnalyzer,
  createSystemPromptAudit,
} from '@lov3kaizen/agentsea-redteam';

// Comprehensive vulnerability scan
const scanner = createVulnerabilityScanner({
  target: myAgent,
  categories: ['injection', 'jailbreak', 'data-leakage', 'bias'],
});

const scanResults = await scanner.scan();

// Analyze prompt safety
const analyzer = createPromptAnalyzer();
const analysis = await analyzer.analyze('Your system prompt here');

// Audit system prompt
const audit = createSystemPromptAudit();
const auditResults = await audit.audit(systemPrompt);

Jailbreak Detection

Detect jailbreak attempts in real-time:

typescript
import { createJailbreakDetector } from '@lov3kaizen/agentsea-redteam';

const detector = createJailbreakDetector({
  sensitivity: 'high',
});

const result = await detector.detect('Ignore all previous instructions...');
console.log('Is jailbreak:', result.isJailbreak);
console.log('Confidence:', result.confidence);
console.log('Category:', result.category);

Compliance Checking

Verify compliance with AI regulations:

typescript
import { ComplianceChecker } from '@lov3kaizen/agentsea-redteam';

const checker = new ComplianceChecker({
  frameworks: ['eu-ai-act', 'nist-ai-rmf'],
});

const compliance = await checker.check(myAgent);
console.log('Compliant:', compliance.isCompliant);
console.log('Findings:', compliance.findings);

Audit & Evidence

The AuditLogger keeps an append-only, hash-chained trail: every entry stores the hash of the previous one, so any later mutation is detectable via verifyIntegrity(). Collect evidence and maintain audit trails:

typescript
import { AuditLogger, EvidenceCollector } from '@lov3kaizen/agentsea-redteam';

const auditLogger = new AuditLogger();
const evidenceCollector = new EvidenceCollector();

// Log a security event (id, timestamp, and hashes are filled in for you)
auditLogger.log({
  eventType: 'finding_created',
  action: 'vulnerability-found',
  actor: { id: 'scanner', type: 'system' },
  resource: { id: 'my-agent', type: 'agent' },
  outcome: 'success',
  severity: 'high',
  details: { type: 'prompt-injection', category: 'direct' },
});

// Collect evidence and bundle it into a hashed package
evidenceCollector.collect({
  type: 'transcript',
  content: JSON.stringify({ input: attackInput, output: agentResponse }),
});
const pkg = evidenceCollector.createPackage('case-001');

// Verify the chain has not been tampered with
const integrity = auditLogger.verifyIntegrity();
console.log('Audit status:', integrity.status); // 'valid' | 'tampered'

Persistent Audit Logging

By default the hash chain lives in memory. Attach an AuditStore to make it survive process restarts. The built-in FileAuditStore is an append-only JSONL backend — each entry is one JSON line, so writes are O(1) appends and the whole tamper-evident chain can be replayed (and re-verified) on load with loadFromStore().

typescript
import {
  AuditLogger,
  createFileAuditStore,
} from '@lov3kaizen/agentsea-redteam';

// Append-only JSONL store, durable on every log() call
const store = createFileAuditStore('./audit/redteam.jsonl');
const auditLogger = new AuditLogger({ store });

// On startup, replay the persisted chain to resume the existing trail
const loaded = await auditLogger.loadFromStore();
console.log(`Resumed ${loaded} audit entries`);

// New entries link onto the restored chain and are persisted as they happen
auditLogger.log({
  eventType: 'scan_completed',
  action: 'scheduled-scan',
  actor: { id: 'ci', type: 'system' },
  resource: { id: 'my-agent', type: 'agent' },
  outcome: 'success',
});

// The replayed-plus-new chain still verifies end to end
console.log(auditLogger.verifyIntegrity().status); // 'valid'
Writes use synchronous file appends, so an entry is durable the moment log() returns — losing the most recent entries on a crash would otherwise break the chain. Implement the AuditStore interface (append, loadAll, clear) to back the trail with your own datastore.

Continuous Testing

ContinuousTesting runs a test function on demand or on a schedule, records each run, and feeds the resulting metrics into an AlertManager. Provide a runner that returns a summary plus the metrics you want to alert on:

typescript
import {
  ContinuousTesting,
  AlertManager,
} from '@lov3kaizen/agentsea-redteam';

const alerts = new AlertManager({
  channels: [
    {
      id: 'sec-slack',
      type: 'slack',
      name: 'Security Slack',
      enabled: true,
      config: { webhookUrl: process.env.SLACK_WEBHOOK_URL },
      severities: ['warning', 'critical'],
    },
  ],
  rules: [
    {
      id: 'risk-high',
      name: 'Risk score too high',
      enabled: true,
      severity: 'critical',
      condition: { type: 'threshold', metric: 'riskScore', operator: 'greater', value: 70 },
      channelIds: ['sec-slack'],
    },
  ],
});

const continuous = new ContinuousTesting({
  runner: async () => {
    const results = await redTeam.run();
    return {
      summary: { /* RunSummary */ },
      metrics: { riskScore: results.summary.riskScore },
    };
  },
  alerts,
});

// Run once on demand (also evaluates rules and delivers any alerts)
const run = await continuous.runOnce('manual');
console.log('Triggered alerts:', run.alerts.length);

Cron-Driven Schedules

Schedule recurring runs with scheduleRuns(). The built-in frequencies (hourly, daily, weekly, monthly) cover the common cases, and frequency: 'custom' lets you supply a standard cronExpression (parsed with cron-parser) plus an optional timezone:

typescript
// Run every 6 hours via a cron expression
continuous.scheduleRuns({
  frequency: 'custom',
  cronExpression: '0 */6 * * *',
  timezone: 'America/New_York',
});

// Start the wall-clock loop (ticks every minute by default)
continuous.scheduler.start();

// ...or drive it deterministically in tests
await continuous.scheduler.tick(Date.now());

You can also compute the next fire time directly with the nextCronAt helper:

typescript
import { nextCronAt } from '@lov3kaizen/agentsea-redteam';

// Next fire time (epoch ms) strictly after `from`; null for an invalid expression
const next = nextCronAt('0 9 * * 1', Date.now(), 'UTC'); // Mondays at 09:00 UTC

Alert Delivery

AlertManager evaluates threshold/change rules against a metrics snapshot and delivers a notification to each configured channel. deliver() actually contacts the channels — webhook, Slack, Teams, and Discord post via fetch, PagerDuty uses the Events API v2 (the channel's config.apiKey is the routing key), and email goes through the optional nodemailer dependency. Per-channel severities route alerts, and each attempt — including failures — is recorded on the alert's notificationHistory so one bad channel never blocks the others.

typescript
import { AlertManager } from '@lov3kaizen/agentsea-redteam';

const manager = new AlertManager({
  channels: [
    {
      id: 'oncall',
      type: 'pagerduty',
      name: 'On-call',
      enabled: true,
      config: { apiKey: process.env.PAGERDUTY_ROUTING_KEY }, // Events API v2 routing key
      severities: ['critical'],
    },
    {
      id: 'team-email',
      type: 'email',
      name: 'Security Team',
      enabled: true,
      config: {
        emails: ['security@example.com'],
        settings: { smtp: { host: 'smtp.example.com', port: 587 } },
      },
      severities: ['warning', 'critical'],
    },
  ],
  rules: [
    {
      id: 'vulns-found',
      name: 'Vulnerabilities found',
      enabled: true,
      severity: 'critical',
      condition: { type: 'threshold', metric: 'vulnerabilities', operator: 'greater', value: 0 },
      channelIds: ['oncall', 'team-email'],
    },
  ],
});

// Evaluate rules against metrics and deliver to matching channels
const triggered = await manager.evaluateAndDeliver({ vulnerabilities: 3 });
for (const alert of triggered) {
  console.log(alert.title, alert.notificationHistory); // per-channel sent/failed records
}
Email alerts require the optional nodemailer package — install it (or use a different channel type) and provide config.settings.smtp.host. All other channel types work out of the box using fetch.

CI/CD Integration

Run security tests in your CI pipeline:

typescript
import { createCIIntegration } from '@lov3kaizen/agentsea-redteam';

const ci = createCIIntegration({
  failOnHighSeverity: true,
  reportFormat: 'junit',
  outputPath: './security-report.xml',
});

const results = await ci.run();
process.exit(results.passed ? 0 : 1);

AgentSea Integration

Run security tests directly against an AgentSea agent:

typescript
import { createAgentSeaIntegration } from '@lov3kaizen/agentsea-redteam';
import {
  Agent,
  AnthropicProvider,
  ToolRegistry,
} from '@lov3kaizen/agentsea-core';

const agent = new Agent(
  { name: 'my-agent', model: 'claude-sonnet-4-6', provider: 'anthropic' },
  new AnthropicProvider(process.env.ANTHROPIC_API_KEY),
  new ToolRegistry(),
);

const integration = createAgentSeaIntegration({
  agent,
  testCategories: ['injection', 'jailbreak', 'data-leakage'],
});

const results = await integration.run();

Test Suites

Build custom test suites:

typescript
import {
  createTestSuite,
  TestSuiteBuilder,
} from '@lov3kaizen/agentsea-redteam';

const suite = new TestSuiteBuilder()
  .addAttack('prompt-injection')
  .addAttack('jailbreak')
  .addScan('system-prompt')
  .addBenchmark('safety')
  .build();

const results = await suite.run({ target: myAgent });

Report Generation

Generate detailed security reports:

typescript
import { createReportGenerator } from '@lov3kaizen/agentsea-redteam';

const reporter = createReportGenerator({
  format: 'html',
  branding: { logo: './logo.png', company: 'Acme Corp' },
  sections: ['executive-summary', 'methodology', 'findings', 'recommendations'],
});

const report = await reporter.generate(results);

Sub-Package Imports

Import specific modules directly for smaller bundles:

typescript
// Import specific modules
import { RedTeam } from '@lov3kaizen/agentsea-redteam/core';
import { AttackLibrary } from '@lov3kaizen/agentsea-redteam/attacks';
import { VulnerabilityScanner } from '@lov3kaizen/agentsea-redteam/scanning';
import { SafetyBenchmark } from '@lov3kaizen/agentsea-redteam/benchmarks';
import { JailbreakDetector } from '@lov3kaizen/agentsea-redteam/detection';
import { ComplianceChecker } from '@lov3kaizen/agentsea-redteam/compliance';
import { AuditLogger } from '@lov3kaizen/agentsea-redteam/audit';
import { ContinuousTesting } from '@lov3kaizen/agentsea-redteam/continuous';

Next Steps