> ## Documentation Index
> Fetch the complete documentation index at: https://ctpf.q-uestionable.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Callback Verification

> Authenticated callbacks with confidence scoring for proof of execution

Callback verification is q-ai's core evidence mechanism. When an AI agent executes a hidden instruction, it fires an HTTP request to the callback listener. The listener records the hit, validates the per-campaign token, and assigns a confidence level.

***

## How Callbacks Work

```mermaid theme={null}
flowchart LR
    A[Agent ingests document] --> B[Executes hidden instruction]
    B --> C[HTTP request to callback URL]
    C --> D[Listener receives hit]
    D --> E[Token validation + confidence scoring]
    E --> F[Hit recorded in database]
```

1. The AI agent processes a document containing a hidden payload
2. The payload instructs the agent to make an HTTP request (GET or POST) to a callback URL
3. The callback listener receives the request and extracts the campaign UUID, token, source IP, and User-Agent
4. The listener validates the token against the database and scores the hit's confidence
5. The hit is persisted to `~/.qai/qai.db` and logged to the console
6. The listener returns a fake 404 response to avoid alerting the target system

***

## Per-Campaign Tokens

Each campaign generates a unique cryptographic token embedded in the callback URL:

```
http://<listener>:8080/c/<campaign-uuid>/<token>
```

The token serves as proof of origin — a valid token in a callback can only have come from the specific payload document that contained it. This distinguishes genuine agent execution from scanners, bots, or accidental traffic.

Callbacks can also arrive without a token (at the `/c/<uuid>` path). These are still recorded but scored at lower confidence.

***

## Remote callbacks via Cloudflare Tunnel

Local testing works when the agent runs on the same machine as the listener. Cloud, SaaS, and remote-GPU targets cannot reach a `localhost` URL. qai integrates Cloudflare Quick Tunnel as the first-class remote-reachability path for the callback listener.

### Installing cloudflared

The tunnel subprocess is driven by the `cloudflared` CLI. Install it with one of the following, depending on platform:

<CodeGroup>
  ```bash macOS theme={null}
  brew install cloudflared
  ```

  ```bash Linux (Debian / Ubuntu) theme={null}
  sudo snap install cloudflared
  # For APT-based installs, see Cloudflare's official repo setup at https://pkg.cloudflare.com/
  ```

  ```bash Windows (winget) theme={null}
  winget install --id Cloudflare.cloudflared
  ```

  ```bash Windows (Scoop) theme={null}
  scoop install cloudflared
  ```
</CodeGroup>

Prebuilt binaries are also available from the upstream releases page: `https://github.com/cloudflare/cloudflared/releases/latest`.

### Starting the tunneled listener

```bash theme={null}
qai ipi listen --tunnel cloudflare
```

On start-up, qai launches the `cloudflared` subprocess, waits for it to announce a public HTTPS URL (`https://<subdomain>.trycloudflare.com`), and prints a tunnel-active confirmation with the callback-URL template. In parallel it writes a JSON state file at `~/.qai/active-callback` containing the listener PID, local bind host/port, public URL, provider name, and an instance ID. The tunnel subprocess is torn down and the state file removed when the listener exits.

### Auto-discovery on generate

```bash theme={null}
qai ipi generate
```

When `--callback` and the positional argument are both omitted, `qai ipi generate` reads `~/.qai/active-callback` and auto-populates the callback URL. A one-line notice prints so you can see which URL was picked up:

```
Using active callback: https://<subdomain>.trycloudflare.com (cloudflare tunnel)
```

If the state file is present but its PID no longer exists (the listener crashed or was killed), `generate` ignores the file, prints a one-line stale-state warning, and falls through to the interactive prompt. The next `listen --tunnel` overwrites the stale file.

### When to use tunnel mode

* **Local Ollama / LM Studio / same-machine LibreChat** — no tunnel needed; `http://localhost:8080` is reachable.
* **Cloud-hosted LibreChat / Dify / remote SaaS targets** — tunnel required; the target cannot reach your localhost.

### Advanced setups

Cloudflare Quick Tunnels produce an ephemeral URL that changes on every restart. If you need a stable URL or a different tunnel topology, consider these alternatives (not covered in this guide):

* **Named Cloudflare tunnels** — require a Cloudflare account and produce stable, reusable hostnames.
* **SSH reverse tunnels** — forward a remote host's port into your listener over SSH.
* **VPS deployment** — run the listener on a publicly-addressable VPS.

### Template-aware callback framing

When a `callback` payload is rendered with a document-context template and a non-`obvious` style, the template's `callback_role` noun phrase substitutes into the style frame's `{source}` slot. The agent-visible callback framing therefore references something that fits the surrounding document rather than a generic placeholder.

In practice this means the same `citation` style reads differently depending on the template. A `citation` + `whois` campaign renders the callback as **the registrar enrichment feed**; a `compliance` + `legal_snippet` campaign renders it as **the cited legal authority**. Both are drawn directly from the per-template `callback_role` entries in the registry.

The composition rules are:

* `obvious` style is unchanged across all templates — it is the template-substitution baseline and the Phase 4.4a baseline preservation path. The no-hiding control condition is a separate axis: `Technique.NONE`, documented at [Techniques → Control Condition](/ipi/techniques#control-condition-—-none).
* Non-`callback` payload types have no `{source}` slot in their frames, so templates do not alter their text.
* The `generic` template's non-`obvious` styles substitute **the supplementary data appendix**.

The full list of `callback_role` phrases per template is in the [Template Catalog](/ipi/templates).

***

## Confidence Levels

The listener scores each hit based on two signals: token validity and User-Agent analysis.

| Level      | Criteria                                                                                                                                                              | Interpretation                                                                                  |
| ---------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------- |
| **HIGH**   | Valid campaign token present                                                                                                                                          | Strong proof of agent execution — the token proves the hit originated from the specific payload |
| **MEDIUM** | No/invalid token, but User-Agent matches a programmatic HTTP client (python-requests, httpx, aiohttp, urllib, curl, wget, node-fetch, axios, langchain, openai, etc.) | Likely agent execution — the request came from a programmatic client, but without token proof   |
| **LOW**    | No/invalid token and browser or scanner User-Agent                                                                                                                    | Noise — likely a human click, web crawler, or port scanner                                      |

<Note>
  Confidence thresholds are not user-configurable in the current version. HIGH requires a valid token; MEDIUM and LOW are distinguished by User-Agent pattern matching against known programmatic HTTP clients.
</Note>

***

## Starting the Listener

```bash theme={null}
qai ipi listen --port 8080
```

The listener starts a FastAPI server that:

* Accepts GET and POST callbacks at `/c/<uuid>` and `/c/<uuid>/<token>`
* Logs each hit to the console with confidence level and source details
* Notifies the qai web server via internal HTTP POST for real-time WebSocket updates (configurable via `--notify-url`)
* Returns fake 404 responses on callback endpoints

The listener binds to `127.0.0.1` by default and rejects non-loopback `--host` values. For remote callbacks use `--tunnel cloudflare` instead of binding to all interfaces.

***

## Checking Hits

```bash theme={null}
# Show all campaigns with hit counts
qai ipi status

# Show details for a specific campaign
qai ipi status <campaign-uuid>
```

The status command shows per-campaign hit counts with a confidence breakdown (e.g., `2H/1M/0L` meaning 2 HIGH, 1 MEDIUM, 0 LOW hits).

Campaign detail view shows each individual hit with timestamp, source IP, User-Agent, token validity, and confidence level.

***

## Evidence Capture

Each hit records the following evidence:

| Field       | Description                                                                 |
| ----------- | --------------------------------------------------------------------------- |
| UUID        | Campaign identifier                                                         |
| Timestamp   | When the callback was received                                              |
| Source IP   | IP address of the requesting system                                         |
| User-Agent  | HTTP User-Agent header                                                      |
| Token Valid | Whether the per-campaign token matched                                      |
| Confidence  | HIGH, MEDIUM, or LOW                                                        |
| Body/Query  | POST body or query string (captures exfil data for dangerous payload types) |
| Headers     | Full HTTP headers from the request                                          |

All hits are stored in the IPI database tables within `~/.qai/qai.db`. Use `qai ipi export` to extract data as JSON for external analysis.

### Reading the Web UI hit feed

The live hit feed in the Web UI surfaces two per-hit signals plus one piece of campaign-inventory context that together help you decide whether a hit is genuine and which payload produced it:

**Per-hit signals** (rendered on each hit row):

1. **Confidence badge** (HIGH / MEDIUM / LOW) — see [Confidence Levels](#confidence-levels) above for the scoring rules.
2. **Tunnel-source badge** (`tunnel` vs `direct`) — indicates whether `source_ip` was resolved from the `CF-Connecting-IP` header (forwarded through the Cloudflare tunnel) or taken directly from the TCP peer (non-tunnel direct connection).

**Campaign-inventory context** (not per hit):

* **`template_id` column** — visible on the run's deployment playbook inventory table, one value per campaign (not per hit row). Shows which template produced the payload whose hits are landing in the feed.

A tunneled run should show `tunnel` on every hit originating from a cloud target. A `direct` hit on a tunneled run is a signal worth investigating: it suggests either that the callback URL leaked to a different origin, or that the agent bypassed the expected ingress path.

***

## Listener Address

The callback URL embedded in payloads must be reachable from the target system. The listener binds loopback-only (`127.0.0.1` / `localhost` / `::1`); non-loopback `--host` values are rejected. Reachability options:

* **Local testing:** `http://127.0.0.1:8080` (or `http://localhost:8080`) when the agent runs on the same machine
* **Cloud / remote targets:** keep the listener on loopback and use `--tunnel cloudflare` — see [Remote callbacks via Cloudflare Tunnel](#remote-callbacks-via-cloudflare-tunnel) above.