Product guide

AI Agent Release Verification

Your CrewAI, LangGraph, or custom agent can finish a task and still be wrong. TestNeo is the verification and release layer — generate golden tests, execute them after agent runs, validate PRs (PASS / WARN / BLOCK), and gate releases. Wire what you already have. No new TestNeo backend required.

One-liner: Use any agent to do the work. Use TestNeo to prove outcomes and decide when it is safe to ship — with evidence in the UI.

agent runs → TestNeo golden tests → execution evidence → PR validation → release bundle + gate

Reference architecture

YOUR AI AGENT (CrewAI / LangGraph / custom)
        ↓  non-deterministic
TESTNEO VERIFICATION
  • Generate NLP web + API tests (MCP / VS Code / UI)
  • Execute → execution_id + steps in dashboard
  • Optional: Playwright AI SDK in CI
        ↓
TESTNEO RELEASE INTELLIGENCE
  • PR validation → PASS / WARN / BLOCK
  • Engineering Memory matches
  • Release bundle → gate verdict

Prerequisites & MCP setup

Account at app.testneo.ai + API key (tn_…)
Node.js 20+ for @testneo/mcp-server
MCP in Cursor or VS Code

Add to your MCP config:

{
  "mcpServers": {
    "testneo": {
      "command": "npx",
      "args": ["-y", "@testneo/mcp-server@0.1.7"],
      "env": {
        "TESTNEO_BASE_URL": "https://app.testneo.ai",
        "TESTNEO_WEB_APP_URL": "https://app.testneo.ai",
        "TESTNEO_API_KEY": "tn_YOUR_KEY",
        "TESTNEO_MCP_ALLOW_WRITE": "false",
        "TESTNEO_MCP_DEFAULT_EXECUTION_MODE": "cloud"
      }
    }
  }
}

Set TESTNEO_MCP_ALLOW_WRITE=true only when generating or executing tests. Restart MCP after env changes.

Step 1 — Generate verification tests

All paths persist test cases to your TestNeo project (visible in the UI).

Source	Tool / surface
Swagger / OpenAPI	`testneo_swagger_upload_and_generate`
Figma / requirements	`testneo_generate_tests_from_context`
Code in repo	VS Code extension
Playwright hybrid	Playwright AI SDK

MCP prompt example:

Call testneo_bootstrap_web_mcp_project with confirm true:
  name "Agent Staging", website_url "https://your-app.example.com",
  project_environment_name "staging", add_base_url_variable true.

Call testneo_generate_tests_from_context with project_id <ID>,
  include_ui_tests true, include_api_tests true, max_tests 10.

Step 2 — Post-agent verification

After your agent completes, run golden tests. Do not trust the agent summary.

Call testneo_run_generated_test_pipeline with test_case_id <ID>,
  confirm true, environment_name "staging",
  idempotency_key "agent-run-001", include_steps true.

Pass → log execution_id and allow promote. Fail → block and open the run in Test Runner.

CrewAI teams: wire a post-run script that calls the same MCP tool or TestNeo REST execute API. See MCP customer E2E for REST patterns.

Step 3 — PR validation

Validate my open PR for project <PROJECT_ID>.
Use git for changed files.
Call testneo_pr_validation_workflow with confirm false.
Show risk_score, PASS/WARN/BLOCK, workflow_id, PR comment draft.

Band	Score	Meaning
PASS	0–34	Lower verification risk
WARN	35–69	Review; run impacted tests
BLOCK	70–100	Address before merge

Full guide: Release Intelligence

Step 4 — Release bundle & gate

Call testneo_create_release_bundle with project_id <ID>,
  release_name "v1.0.0", workflow_ids ["<wf1>", "<wf2>"].

Call testneo_evaluate_release_gate with bundle_id "<from above>".

End-to-end checklist

Step	Done when
Account + MCP	`testneo_validate_connection` succeeds
Golden tests	Tests visible in project UI
Agent hook	Post-run execute returns pass/fail + execution_id
PR hook	Workflow returns band + workflow_id
Release	Bundle + gate verdict in Release Readiness

AI Agent Release Verification

Reference architecture

Prerequisites & MCP setup

Step 1 — Generate verification tests

Step 2 — Post-agent verification

Step 3 — PR validation

Step 4 — Release bundle & gate

End-to-end checklist

Related docs