> ## Documentation Index
> Fetch the complete documentation index at: https://docs.shftd2.com/llms.txt
> Use this file to discover all available pages before exploring further.

> ## Agent Instructions
> Publishers integrate one API: GET https://api.shftd2.com/v1/fragment?partner_id=<account id>&page_url=<page URL>, called from their edge for AI-agent requests only, with the API key in the X-API-Key header, never in the query string.
> A 200 carries the card. A 204 means no card: serve the page unchanged.
> The management API (https://api.shftd2.com/v1/me, /v1/sites, /v1/cards, /v1/keys) is optional tooling for scripting setup. It takes its own management token (Authorization: Bearer ds_mgmt_…). Its limits (120 requests and 60 writes per minute per token) do not apply to the delivery API.
> Delivery logs reach doubleshift only through direct delivery logs (POST https://logs.shftd2.com/v1/beacon with a logs token; the edge library with a logsToken is its provided client, and a custom sender may use the same endpoint) or a Cloudflare Logpush job configured with doubleshift. Otherwise doubleshift sees only its own API activity.

# ESI: Akamai, Fastly, Varnish

> Integrate doubleshift with one ESI tag in your page template, resolved by Akamai or Fastly for AI agents only and decided on every request, with the header rewrite the call needs.

One tag in your page template, resolved by your CDN's native Edge Side Includes for AI agents only.
These are reference examples per CDN: adapt them to your configuration, then run the checks in
[Verify and troubleshoot](https://docs.shftd2.com/verify.md).

Before you start, do steps 1 to 3 of the [Quickstart](https://docs.shftd2.com/quickstart.md): a site, a fragment its pages
resolve to, and a delivery key. The rules every edge keeps are in [Choose your edge](https://docs.shftd2.com/integrate.md).

## The tag

Paste the tag in the article template **immediately before the article content**, never at the end
of `<body>`; see [Where the card must land](https://docs.shftd2.com/integrate.md#where-the-card-must-land). `onerror="continue"`
renders the page without the card when the call is slow or fails. `page_url` must reach doubleshift
as **one URL-encoded value**, and how to build it depends on the CDN.

> [tab] Akamai
>
> Build the URL with `esi:assign`, then encode by hand with `$replace()` the four characters that
> break a query value: `%` first, then `&`, `+`, `=`. Do not use `$url_encode()`: it can leave `&`
> unencoded.
>
> ```html title="ESI tag — Akamai" theme={"theme":{"light":"css-variables","dark":"css-variables"}}
> <!-- ABC — ESI tag, one for every page of the site.
>      Paste it in the article template immediately BEFORE the article content
>      (first element of <main>) — NEVER at the end of <body>: anything after the
>      footer is boilerplate to AI extractors and is never read.
>      onerror="continue": if the endpoint is slow or down, the page renders without the card.
>      Gate ESI on the User-Agent so the include resolves ONLY for AI agents.
>      page_url is encoded by hand ("%" first, then "&", "+", "="), whatever the
>      property's $url_encode mode. -->
> <esi:assign name="ds_page_url" value="'https://' + $(HTTP_HOST) + $(REQUEST_PATH)" />
> <esi:assign name="ds_page_url" value="$replace($(ds_page_url), '%', '%25')" />
> <esi:assign name="ds_page_url" value="$replace($(ds_page_url), '&', '%26')" />
> <esi:assign name="ds_page_url" value="$replace($(ds_page_url), '+', '%2B')" />
> <esi:assign name="ds_page_url" value="$replace($(ds_page_url), '=', '%3D')" />
> <esi:include
>   src="https://api.shftd2.com/v1/fragment?partner_id=<your account id>&page_url=$(ds_page_url)"
>   onerror="continue" />
> ```

> [tab] Fastly
>
> ESI variables are limited on Fastly: there is no `$(REQUEST_PATH)` expansion. Use the tag
> **without** `page_url` and build it in VCL, where `urlencode()` exists (see
> [Configure your CDN](#configure-your-cdn)).
>
> ```html title="ESI tag — Fastly (VCL appends page_url)" theme={"theme":{"light":"css-variables","dark":"css-variables"}}
> <esi:include src="https://api.shftd2.com/v1/fragment?partner_id=<your account id>" onerror="continue" />
> ```

> [tab] Varnish
>
> Varnish open source expands no ESI variables, has no URL-encode function and ignores `https://`
> includes, so it needs a local route to doubleshift over TLS that depends on your setup: contact
> doubleshift to set it up, or use the [Cloudflare Worker path](https://docs.shftd2.com/quickstart.md).

Check the encoding once on your stack before going live: an agent request for a path containing `&`
(e.g. `/news/a&b`) must reach doubleshift with `page_url` ending in `/news/a%26b`, not cut at
`/news/a`.

ESI splices the card where the tag sits, and the answer carries the card only. The in-content brand
line must name the brand of the card actually served on that page, and must be absent on a `204`
([Where the card must land](https://docs.shftd2.com/integrate.md#where-the-card-must-land)). A static line in the template does
not meet this: the brand depends on the pick for each page, and the answer can be a no-fill. Your edge
meets it only if it writes the line from the card it received (the card's first line reads
`Brand · <brand>`) and writes nothing when the include returned nothing. If it cannot, use the
[Cloudflare Worker path](https://docs.shftd2.com/quickstart.md).

> [!WARNING]
>
> **Header rewrite required**
>
> `/v1/fragment` reads the API key from the `X-API-Key` header only. **Never put the key in `src`**:
> a key in the query string is ignored, so the call answers `401`, and the tag reaches human visitors
> as inert HTML with the key in it. Your CDN must rewrite the include subrequest to carry
> **`X-API-Key`**, a **benign `User-Agent`** (bot User-Agents are blocked, silently with
> `onerror="continue"`), and the real agent in **`X-Agent-UA`**.
>
> * **Akamai**: Property Manager, in a rule scoped to requests to the fragment endpoint: *Set Variable*
>   captures the incoming `User-Agent` request header (e.g. `PMUSER_AGENT_UA`), then *Modify Outgoing
>   Request Header* sets `X-API-Key`, sets `X-Agent-UA` to `{{user.PMUSER_AGENT_UA}}`, and replaces
>   `User-Agent` with a benign value.
> * **Fastly**: set the three headers on the include subrequest in VCL (snippet below).
>
> If your CDN cannot add request headers, use the [Cloudflare Worker path](https://docs.shftd2.com/quickstart.md) instead.

## Gate ESI on every request

Decide agent or human **on every request**, cache hit or miss, never when the page enters the cache:
a gate set at fetch time (`beresp.do_esi` on the bot `User-Agent` in `vcl_fetch` or
`vcl_backend_response`) depends on who filled the cache.

1. **Parse at fetch, for every HTML page, humans included**: the cached copy is the template with its
   tag, whoever filled it. No `Vary: User-Agent`, no second cache entry.
2. **Decide at delivery**: resolve the include for agents only (Fastly `req.esi`, Akamai a Property
   Manager rule evaluated per request). Humans get the stored copy with the tag left inert; agents
   get it resolved.
3. **No `304` for agents**: drop `If-None-Match` and `If-Modified-Since` from agent requests, and send
   the agent response without `ETag` or `Last-Modified` and with `Cache-Control: private, no-store`
   (see the [cache contract](https://docs.shftd2.com/integrate.md#the-page-response-cache-contract)).

Check it from a cold cache in both fill orders, then warm, with card markers counted in the raw HTML;
the sequence is in [Verify and troubleshoot](https://docs.shftd2.com/verify.md).

## Configure your CDN

> [tab] Akamai
>
> On Akamai, the agent decision is a **Property Manager rule, evaluated per request**, cache hit or miss. The
> rule matches the agent `User-Agent` list ([AI agents](https://docs.shftd2.com/api/agents.md)) and enables ESI processing
> (behavior *Edge Side Includes* → Enable), so the include is resolved for agents only; humans get
> the tag inert. In the same rule, *Modify Incoming Request Header* removes `If-None-Match` and
> `If-Modified-Since`, and *Modify Outgoing Response Header* removes `ETag` and `Last-Modified` and
> sets `Cache-Control: private, no-store`. Set the ESI include timeout of your property to about
> 800 ms.
>
> Adapt the rule to your property, then run the [Verify and troubleshoot](https://docs.shftd2.com/verify.md) sequence (both
> cold fills, the warm loop and the `304` check), and check that `/news/a&b` reaches doubleshift
> with `page_url` ending in `/news/a%26b`.

> [tab] Fastly
>
> Give the include a backend for `api.shftd2.com` with a first-byte timeout of about 800 ms;
> `req.topurl` holds the page's own URL there, and `req.esi` set in `vcl_recv` decides per request.
> Ask the origin for uncompressed HTML and compress toward visitors with Fastly's **dynamic**
> compression: static compression (`beresp.gzip` or `beresp.brotli`) prevents ESI processing. The
> agent markers in the VCL are a copy of [AI agents](https://docs.shftd2.com/api/agents.md): keep them in sync.
>
> ```text title="Fastly VCL" theme={"theme":{"light":"css-variables","dark":"css-variables"}}
> sub vcl_recv {
>     if (req.esi_level == 0) {
>         # Agent or human — decided on EVERY request, cache hit or miss.
>         if (req.http.User-Agent ~ "(?i)\b(GPTBot|ChatGPT-User|OAI-SearchBot|ClaudeBot|Claude-User|Claude-SearchBot|Google-CloudVertexBot|PerplexityBot|Perplexity-User|CCBot|Meta-ExternalAgent|meta-externalfetcher|Bytespider|YouBot|Diffbot|MistralAI-User|Amazonbot)\b") {
>             set req.http.X-ABC-Agent = "1";
>             # No 304 for agents: a validator from a human copy would drop the card.
>             unset req.http.If-None-Match;
>             unset req.http.If-Modified-Since;
>         } else {
>             unset req.http.X-ABC-Agent;
>             set req.esi = false; # humans: the tag stays inert, no card request
>         }
>         # With shielding, ESI runs at the edge POP only, never at the shield.
>         if (fastly.ff.visits_this_service > 0) {
>             set req.esi = false;
>         }
>     }
>     if (req.esi_level > 0 && req.url.path == "/v1/fragment") {
>         # page_url: scheme + host + path of the page, query dropped, encoded ONCE.
>         set req.http.X-Page-Url = urlencode("https://news-site.example" + regsub(req.topurl, "[?#].*$", ""));
>         set req.url = req.url + "&page_url=" + req.http.X-Page-Url;
>         unset req.http.X-Page-Url;
>         set req.backend = F_doubleshift; # the backend you created for api.shftd2.com
>         # Header rewrite: key in the header (better: from an edge dictionary),
>         # real agent in X-Agent-UA, benign User-Agent, no visitor credentials.
>         set req.http.X-API-Key = "YOUR_API_KEY";
>         set req.http.X-Agent-UA = req.http.User-Agent;
>         set req.http.User-Agent = "abc-esi-adapter";
>         unset req.http.Cookie;
>         unset req.http.Authorization;
>     }
> }
>
> sub vcl_miss {
>     # ESI cannot parse a compressed body: ask the origin for identity HTML.
>     if (req.esi_level == 0) { unset bereq.http.Accept-Encoding; }
>     unset bereq.http.X-ABC-Agent;
> }
>
> sub vcl_pass {
>     if (req.esi_level == 0) { unset bereq.http.Accept-Encoding; }
>     unset bereq.http.X-ABC-Agent;
> }
>
> sub vcl_fetch {
>     # Parse EVERY HTML page, humans included: the cached copy is the same
>     # whoever fills it; req.esi above decides per request whether it resolves.
>     if (req.esi_level == 0 && beresp.http.Content-Type ~ "text/html") {
>         set beresp.do_esi = true;
>     }
> }
>
> sub vcl_deliver {
>     # The composed page is the agent's alone: no validators, no shared caching.
>     if (req.esi_level == 0 && req.http.X-ABC-Agent == "1") {
>         unset resp.http.ETag;
>         unset resp.http.Last-Modified;
>         set resp.http.Cache-Control = "private, no-store";
>     }
> }
> ```
>
> Adapt the VCL to your service, then run the [Verify and troubleshoot](https://docs.shftd2.com/verify.md) sequence.

> [tab] Varnish
>
> The Varnish configuration depends on your TLS setup and is not published: contact doubleshift, or
> use the [Cloudflare Worker path](https://docs.shftd2.com/quickstart.md). If a public CDN such as Cloudflare or CloudFront sits
> in front of your Varnish, do not resolve ESI in that Varnish, since the public CDN would cache the
> composed page: use a Worker or an edge function on that CDN instead.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.