# Serge — Full Documentation

> Serge tells e-commerce sites whether users behind AI agents can actually find their products and buy them. For a quick overview, see [llms.txt](https://serge.ai/llms.txt).

## Overview

Serge scans e-commerce sites and measures how easy it is for an AI agent (Claude, ChatGPT, Operator) — acting on behalf of a human shopper — to find a product on the site and place it in a cart. The result is a single score from 0 to 100. Deterministic. No LLM in the loop at scan time.

When a customer asks Claude, ChatGPT, or Operator to buy something on a site, the agent fires up a browser (via a Model Context Protocol server, a built-in computer-use tool, or a headless automation layer), navigates the site, tries to find the product, tries to add it to the cart, and either succeeds or gives up. Most enterprise e-commerce sites today fail silently at this test. Serge measures the conditions that predict failure — and tells e-commerce teams exactly what to fix.

## The scope — user → agent → eshop

Serge occupies the middle layer of a three-layer problem:

- **Upstream (visibility):** Does ChatGPT mention you in its answers? GEO territory. Covered by Athena, Profound, Scrunch, Peec, Otterly, Semrush AI Visibility, Adobe LLM Optimizer.
- **Middle (arrival + traversal):** Can the agent actually find and buy your products once it lands on your site? **This is Serge's layer.** No other tool covers it.
- **Downstream (attribution):** What share of revenue came from LLM-mediated sessions? Attribution tooling — Dreamdata, HockeyStack, Bizible (all partially broken for agent-mediated traffic).

## What the scanner measures

One question: **can a user behind an AI agent find a product on this site and place it in a cart?**

### Scoring methodology status

**The scoring methodology is being rebuilt from the ground up as of April 2026.** The previous version measured "agent readiness" across five invented dimensions (Access, Identity, Interaction, Content, Integration) with 49 checks. That framework has been retired because it forced buyers to learn a proprietary ontology before understanding the value. The replacement is simpler and grounded in the specific question above.

The research direction for the rebuild covers the structural conditions an agent needs to complete the user → agent → eshop journey:

- **Can the agent reach the site at all?** Bot protection posture, robots.txt permissions, WAF behavior, CDN rules, cookie walls.
- **Can the agent find products?** Sitemap quality, navigation semantics, product URL discoverability, internal link structure, search functionality.
- **Can the agent parse product data?** Schema.org Product markup, Offer data, Availability, clear pricing, variant metadata, inventory visibility.
- **Can the agent interact with the page?** Real `<button>` elements vs. `<div onClick>`, accessible names, ARIA roles, keyboard semantics, form labeling, landmark structure.
- **Can the agent add the product to the cart?** Cart state exposure, variant selection mechanics, stock signaling, error messaging.

The final check list, weights, and scoring rubric are the output of a research + validation workstream and are not locked. The full methodology is published at https://serge.ai/methodology — that page is rewritten as the research concludes.

### Live agent runs vs. deterministic scanning

The public scanner is **deterministic**. Running a real agent against every domain that hits the site would make the scanner unaffordable. The scanner crawls the site and inspects structural properties that predict whether an agent would succeed.

**Live agent runs happen in the paid product.** Investigate Mode (the enterprise deliverable) runs a real AI agent — Claude Desktop via MCP first, Operator and GPT Agent later — against a specific URL with a standard purchase task. Serge captures the full session (reasoning, DOM events, screenshots, failure points) and renders a shareable replay with root-cause analysis. This is the artifact that closes enterprise sales. It's gated behind the paid tier for unit-economics reasons.

## Scoring mechanics

Each individual check produces a status: pass (full credit), warn (half credit), or fail (zero credit). Blocked checks (where bot protection prevented evaluation) are excluded from the score denominator.

Raw scores are passed through a calibration curve so that early improvements are rewarding, the middle range has clear separation, and reaching the top requires substantial engineering investment. This follows the same principle as Google Lighthouse and SecurityScorecard.

Score interpretation:
- **85-100:** Fully traversable — the site supports agents finding products and adding them to the cart across the typical journey
- **65-84:** Well-positioned — most of the journey works for agents with minor gaps
- **45-64:** Making progress — agents can reach products but often fail at variant selection or cart mechanics
- **25-44:** Room to grow — agents may reach the site but frequently fail before completing the purchase task
- **0-24:** Just getting started — the structural conditions for agent traversal are largely missing

Band labels and meanings are being updated as part of the methodology rebuild to describe outcomes in product-findability terms.

## API reference

### POST /api/scan

Initiate a domain scan. Returns a Server-Sent Events stream with real-time scan progress.

Request:
```json
{
  "domain": "stripe.com"
}
```

The domain must be a valid hostname (e.g., "stripe.com", "api.example.com"). No protocol prefix.

Response: Server-Sent Events stream with the following event types:

- `status` — Scan status update. Data includes current phase description.
- `crawl` — Crawl progress update. Data includes URL being crawled and status.
- `layer` — A layer scan completed. Data includes layer number, name, score, and individual check results.
- `complete` — Scan finished. Data includes scan ID, domain, overall score, and per-layer scores.
- `error` — Scan failed. Data includes error message.

Authentication (required):
- Browser: an authenticated Auth0 session — sign in with Google in one click at https://serge.ai/scanner
- Programmatic: a secret key via `Authorization: Bearer sk_serge_...` header
- Anonymous requests are rejected with `401 { "code": "auth_required" }`
- Rate limits: 60 scans per hour per workspace, 5 scans per hour per domain, 10 scans per hour per target IP (anti-amplification)
- Create secret keys at https://serge.ai/member/api-keys (free account required)
- 429 responses include Retry-After header

### GET /api/scan/{id}

Retrieve results for a completed scan.

Path parameters:
- id (required): UUID of the scan

Response:
```json
{
  "id": "550e8400-e29b-41d4-a716-446655440000",
  "domain": "stripe.com",
  "overallScore": 72,
  "layerScores": {
    "access": 83,
    "identity": 90,
    "interaction": 58,
    "content": 65,
    "integration": 60
  },
  "checkResults": [
    {
      "layer": 1,
      "key": "llms_txt",
      "status": "pass",
      "message": "llms.txt found with 15 lines and API documentation references",
      "fix": null,
      "evidence": {
        "lineCount": 15,
        "hasApiMentions": true
      }
    }
  ],
  "createdAt": "2026-03-24T10:00:00Z",
  "lastSeenAt": "2026-03-24T10:00:00Z",
  "seenCount": 1
}
```

Cache: Responses are cached for 1 hour (Cache-Control: public, max-age=3600).

Error responses:
- 400: Invalid scan ID format
- 404: Scan not found

## Example usage

### Scan a domain with curl (API key required)

A scan needs an authenticated caller. Programmatically, pass a Bearer
API key (create one at https://serge.ai/member/api-keys). Without it the
request returns `401 { "code": "auth_required" }`.

```bash
curl -X POST https://serge.ai/api/scan \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer sk_serge_your_key_here" \
  -d '{"domain": "stripe.com"}'
```

### Retrieve scan results

Reading a cached scan by id is public — no auth needed.

```bash
curl https://serge.ai/api/scan/550e8400-e29b-41d4-a716-446655440000
```

### JavaScript example

```javascript
// Initiate a scan via SSE — a scan requires auth; pass a Bearer API key
// for programmatic calls (browser flows use the Auth0 session cookie)
const response = await fetch('https://serge.ai/api/scan', {
  method: 'POST',
  headers: {
    'Content-Type': 'application/json',
    'Authorization': 'Bearer sk_serge_your_key_here',
  },
  body: JSON.stringify({ domain: 'stripe.com' }),
})

const reader = response.body.getReader()
const decoder = new TextDecoder()

while (true) {
  const { done, value } = await reader.read()
  if (done) break
  const text = decoder.decode(value)
  // Parse SSE events
  console.log(text)
}
```

## Webhook events

Serge supports webhook notifications for monitored domains:

- `score.changed` — Fires when a monitored domain's Serge score changes. Includes old score, new score, and domain.

Webhook payloads are JSON with the following structure:
```json
{
  "event": "score.changed",
  "domain": "example.com",
  "oldScore": 31,
  "newScore": 45,
  "timestamp": "2026-03-24T06:00:00Z"
}
```

## SergeBot (crawler)

SergeBot is the crawler that powers the Serge scanner. It runs only when a user initiates a scan — it is not an autonomous crawler.

User-Agent: `SergeBot/1.0 (+https://serge.ai/bot; product-findability-scanner)`

What it does:
- Checks for agent-friendly metadata: llms.txt, openapi.json, agent.json, sitemap.xml, structured product data
- Parses HTML for structured data (JSON-LD, meta descriptions)
- Queries public registries (MCP Registry, PulseMCP, npm) — no crawling involved

What it does NOT do:
- Scrape, index, or store page content
- Follow links or crawl entire sites
- Train AI models on your content
- Access authenticated or private pages

Rate limits: ~20 requests per domain per scan, 6 concurrent, 8-second timeout per request. Respects robots.txt.

How to allow:
```
User-agent: SergeBot
Allow: /
```

How to block:
```
User-agent: SergeBot
Disallow: /
```

Full documentation: https://serge.ai/bot
Contact: bot@serge.ai

## Agent Journey Tests (paid product)

The paid product runs a real Claude agent (Operator and GPT Agent on the roadmap) against a customer's live storefront with a defined buying task ("find a 13-inch laptop under $1200 and add it to cart"). The agent's full session is captured — reasoning, DOM events, screenshots, the exact failure point — and rendered as a shareable journey-test report with a paste-ready fix snippet per failure.

Pricing: free deterministic scan, free journey-test credits to start, then Pro at CHF 159/mo for daily journey tests on the live store with 12 months of test history. Agency tier for distributors and multi-workspace use — contact-us pricing.

A public preview of a journey-test report is at https://serge.ai/preview/agent-journey-tests/live-demo — the meta variant where an agent shops serge.ai itself and picks the right plan. Five steps, all green.

## Comparison pages

Side-by-side coverage tables + comparative-intent FAQ for the tools buyers most often evaluate against Serge:

- **vs SimGym** (https://serge.ai/compare/simgym) — Shopify's SimGym simulates synthetic human shoppers to A/B test themes before launch and explicitly excludes checkout, pricing, and discounts. Serge runs a real Claude agent — the class of software customers delegate purchases to (Claude, ChatGPT, Operator) — through a real buying task on the live store (find product, choose variant, add to cart) and reports where it fails. Imagined shoppers vs real agents.
- **vs Profound** (https://serge.ai/compare/profound) — visibility tracking vs the agent journey itself. Two different layers of the same problem.
- **vs Hotjar** (https://serge.ai/compare/hotjar) — human session replay vs agent journey tests. Different traffic types; both useful.
- **vs Contentsquare** (https://serge.ai/compare/contentsquare) — enterprise human DX analytics + session replay vs agent journey tests.
- **vs FullStory** (https://serge.ai/compare/fullstory) — broad human session capture vs agent journey tests.
- **vs Quantum Metric** (https://serge.ai/compare/quantum-metric) — Felix AI detection (flags agent-like traffic) vs agent task completion (tests whether agents can buy).

Honest framing across the cluster: Serge does not replace these tools for what they do well. It fills the agent layer they structurally don't cover.

## Company

Serge is built by Superstellar LLC, based in Zug, Switzerland.

- Website: https://serge.ai
- API documentation: https://serge.ai/docs
- Methodology: https://serge.ai/methodology
- OpenAPI specification: https://serge.ai/openapi.json
- Bot documentation: https://serge.ai/bot
- Contact: hello@serge.ai