Catch AI-visibility regressions in your CI/CD pipeline

Why this belongs in CI, not a quarterly audit

The most common way sites lose AI visibility isn't neglect — it's a routine change with an invisible blast radius. Migrating to a new framework that defaults to client-side rendering, tightening bot rules at the CDN, adding a consent wall, or a one-line robots.txt edit can each take you from an A to an F for one or more agents, and you won't see it in analytics for weeks because AI referral traffic is diffuse. A quarterly audit catches it in month three. A CI check catches it in the pull request.

What to assert on every build

• Crawl access per agent — GPTBot, ClaudeBot, PerplexityBot and friends all get a 200, not a 403 or a challenge.
• Content in raw HTML — the pre-JavaScript response actually contains your copy, not an empty <div id="root">.
• Score threshold — the overall AI Readiness Score stays at or above a floor you set (say, 80).
• Structured data intact — the JSON-LD you depend on still parses.

The building blocks: API, CLI, Action

Three surfaces, one metering model (audit credits), so you can wire this into whatever you already run:

1. REST API — POST /v1/audits to run a scan, plus a lightweight quick-check for fast per-agent access assertions, and webhooks if you'd rather be notified than poll. Stable error codes and idempotency keys so retries are safe.
2. CLI — the aibotchecker npm package wraps the API for local runs and scripts, exiting non-zero when a threshold is missed so it drops straight into any pipeline.
3. GitHub Action — a ready-made step that scores a URL and annotates the PR, no glue code required.

Example: fail a PR when the score drops

Point the Action at your preview deployment and set a minimum score. The shape looks like this (see /developers for the exact inputs and outputs):

# .github/workflows/ai-visibility.yml
name: AI visibility
on: [pull_request]

jobs:
  ai-readiness:
    runs-on: ubuntu-latest
    steps:
      - uses: aibotchecker/scan-action@v1
        with:
          url: ${{ env.PREVIEW_URL }}
          min-score: 80          # fail the build below this
          agents: gptbot,claudebot,perplexitybot
        env:
          AIBOTCHECKER_API_KEY: ${{ secrets.AIBOTCHECKER_API_KEY }}

Prefer a script? The CLI does the same thing anywhere:

npx aibotchecker scan https://preview.example.com \
  --min-score 80 \
  --agents gptbot,claudebot,perplexitybot
# exits non-zero if the score is below the floor

Where in the pipeline to run it

Run it against the preview/staging deploy, not localhost — you want to catch the CDN and edge config, which are exactly the layers that break crawlability and don't exist locally. Gate the merge on it like any other required check. For belt-and-braces, add a scheduled nightly run against production so anything that changes outside your deploys (a DNS or WAF tweak, an expired cert) still trips an alert. Keep the credit spend sane by scoring a representative URL or two per run, not the whole site.

FAQ

Wire it up: read the developer docs, grab an API key, and run a scan to see the output your pipeline will assert on. Related: when Cloudflare blocks AI crawlers.