Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion apps/docs/content/docs/meta.json
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@
"index",
"---Usage---",
"pageviews",
"events"
"events",
"server-side-collection"
]
}
131 changes: 131 additions & 0 deletions apps/docs/content/docs/server-side-collection.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,131 @@
---
title: Server-side collection (beta)
description: Count AI agents and robots that request your pages without running the script
---

AI agents and robots request your pages without running the script, so the script cannot count them.
With server-side collection, `trackRequest` reports each page request your Next.js server receives to Simple Analytics, which decides whether an AI agent, a robot or a person made it.
Requests from people are discarded, because the script already counts them.

<Callout title="Beta">
AI agents appear in your reports and count toward your usage. Robots stay hidden by default and do not count toward your usage.
The beta covers how we classify and show this data.
</Callout>

## Set it up with an AI agent

Using Claude Code, Codex, Cursor or another AI agent?
Ask it, with your website's hostname in Simple Analytics instead of `example.com`:

```txt
Set up Simple Analytics server-side collection for example.com, follow https://docs.simpleanalytics.com/server-side-collection.md
```

The guide has the agent use `trackRequest` from this package, ask you which query parameters to report, and test the setup without storing anything.

## Set it up yourself

<Steps>
<Step>

### Set the environment variables

Server-side collection is off until you turn it on.
Set these environment variables for your production deployment, and remove the second one to turn it off again:

```txt title=".env"
SIMPLE_ANALYTICS_HOSTNAME=example.com
SIMPLE_ANALYTICS_SERVER_SIDE_COLLECTION=1
```

</Step>

<Step>

### Call `trackRequest` in your proxy

Add `trackRequest` to `proxy.ts` (`middleware.ts` before Next.js 16), next to your project's `app` or `pages` folder.
If the file already exists, add the `event.waitUntil` line to the top of its function.

<Tabs items={['Next.js 16 or later', 'Next.js 15 and earlier']}>
<Tab>

```ts title="proxy.ts"
import { type NextFetchEvent, type NextRequest, NextResponse } from "next/server";
import { trackRequest } from "@simpleanalytics/next/server";

export function proxy(request: NextRequest, event: NextFetchEvent) {
event.waitUntil(trackRequest({ request }));

return NextResponse.next();
}
```

</Tab>

<Tab>

```ts title="middleware.ts"
import { type NextFetchEvent, type NextRequest, NextResponse } from "next/server";
import { trackRequest } from "@simpleanalytics/next/server";

export function middleware(request: NextRequest, event: NextFetchEvent) {
event.waitUntil(trackRequest({ request }));

return NextResponse.next();
}
```

</Tab>
</Tabs>

`event.waitUntil` lets the page respond without waiting for the report.
If your file has a `matcher`, make sure it matches every page you want counted.

</Step>
</Steps>

## What it reports

- Only `GET` requests. It skips stylesheets, JavaScript, images, fonts, audio, video and JSON by their path, and the paths the plugin uses for the script.
- The hostname you set, the path, and only these query parameters: `utm_source`, `utm_medium`, `utm_campaign`, `utm_content`, `utm_term`, `source`, `medium`, `campaign`, `content`, `term` and `ref`.
- The request headers `User-Agent`, `Referer`, `Accept`, `Accept-Language`, `Sec-Fetch-Mode`, `Sec-Fetch-Dest`, `Sec-Fetch-Site`, `Sec-Fetch-User`, `Sec-CH-UA`, `Signature-Agent` and `From`, and the address headers `X-Forwarded-For`, `CF-Connecting-IP` and `True-Client-IP`. Simple Analytics uses addresses in memory only and never stores them.
- Never cookies or authorization headers.

It runs before Next.js answers, so it does not know the response: pages that do not exist are reported too.
It sends from production deployments only, gives up after 2 seconds, never retries, and never throws.

<Callout title="Pages that are not reported" type="warn">
Pages that a CDN or cache in front of your Next.js server answers never reach your proxy, and a static export (`output: "export"`) has no server to run it.
</Callout>

## Options

| Option | Default | Description |
| ---------------------- | ---------------------------------------------------------- | ----------------------------------------------------------------------------- |
| `request` | Required | The request your `proxy` or `middleware` function receives |
| `hostname` | `SIMPLE_ANALYTICS_HOSTNAME` | The hostname of your website in Simple Analytics |
| `serverSideCollection` | `SIMPLE_ANALYTICS_SERVER_SIDE_COLLECTION` is `1` or `true` | Turns server-side collection on or off |
| `allowParams` | `[]` | Query parameters to report besides the ones listed above, such as `["gclid"]` |
| `testMode` | `SIMPLE_ANALYTICS_TEST_MODE` is `1` or `true` | Asks Simple Analytics to store nothing and prints its reply |

## Test mode

Test mode checks your setup without storing anything, also on your own computer.
Start your app with test mode on:

```bash
SIMPLE_ANALYTICS_SERVER_SIDE_COLLECTION=1 SIMPLE_ANALYTICS_TEST_MODE=1 npm run dev
```

Request a page the way an AI agent does:

```bash
curl -A "Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; ChatGPT-User/1.0; +https://openai.com/bot" http://localhost:3000/
```

Your server console then shows what was sent, including the addresses, and the reply from Simple Analytics, which explains what it would store.

<Callout type="warn">
Turn test mode off when you are done: while it is on, nothing is stored and visitors' addresses appear in your server console.
</Callout>
1 change: 1 addition & 0 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@
"dev": "turbo run dev",
"build:package": "turbo run build --filter=@simpleanalytics/next",
"lint": "turbo run lint",
"test": "pnpm --filter @simpleanalytics/next test",
"format": "prettier --write \"**/*.{ts,tsx,md}\""
},
"devDependencies": {
Expand Down
47 changes: 47 additions & 0 deletions packages/analytics/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -115,3 +115,50 @@ export async function exampleAction() {
return { success: true };
}
```

### Server-side collection (beta)

AI agents and robots request your pages without running the script, so the script cannot count them. With server-side collection, `trackRequest` reports each page request your Next.js server receives to Simple Analytics, which decides whether an AI agent, a robot or a person made it. Requests from people are discarded, because the script already counts them.

AI agents appear in your reports and count toward your usage. Robots stay hidden by default and do not count toward your usage. The beta covers how we classify and show this data.

#### Set it up with an AI agent

Using Claude Code, Codex, Cursor or another AI agent? Ask it, with your website's hostname in Simple Analytics instead of `example.com`:

```txt
Set up Simple Analytics server-side collection for example.com, follow https://docs.simpleanalytics.com/server-side-collection.md
```

The guide has the agent use `trackRequest` from this package and test the setup without storing anything.

#### Set it up yourself

Server-side collection is off until you turn it on. Set these environment variables for your production deployment, and remove the second one to turn it off again:

```txt
SIMPLE_ANALYTICS_HOSTNAME=example.com
SIMPLE_ANALYTICS_SERVER_SIDE_COLLECTION=1
```

Then call `trackRequest` in `proxy.ts` (`middleware.ts` before Next.js 16), next to your project's `app` or `pages` folder. If the file already exists, add the `event.waitUntil` line to the top of its function.

```typescript
import {
type NextFetchEvent,
type NextRequest,
NextResponse,
} from "next/server";
import { trackRequest } from "@simpleanalytics/next/server";

// Before Next.js 16: name the file middleware.ts and the function middleware.
export function proxy(request: NextRequest, event: NextFetchEvent) {
event.waitUntil(trackRequest({ request }));

return NextResponse.next();
}
```

`event.waitUntil` lets the page respond without waiting for the report. If your file has a `matcher`, make sure it matches every page you want counted.

See [server-side collection](https://simpleanalytics-next-docs.vercel.app/docs/server-side-collection) in the documentation for what it reports, all options and how to check your setup with test mode.
3 changes: 2 additions & 1 deletion packages/analytics/package.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "@simpleanalytics/next",
"version": "1.1.1",
"version": "1.2.0",
"description": "Simple Analytics for Next.js",
"keywords": [
"analytics",
Expand All @@ -23,6 +23,7 @@
},
"scripts": {
"build": "tsup",
"test": "node --experimental-strip-types --disable-warning=ExperimentalWarning --disable-warning=MODULE_TYPELESS_PACKAGE_JSON --test test/*.test.mjs",
"format": "prettier --write ."
},
"dependencies": {
Expand Down
1 change: 1 addition & 0 deletions packages/analytics/src/server/index.ts
Original file line number Diff line number Diff line change
@@ -1 +1,2 @@
export { trackEvent, trackPageview } from "./simple-analytics";
export { trackRequest } from "./track-request";
132 changes: 132 additions & 0 deletions packages/analytics/src/server/request-payload.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,132 @@
export const DEFAULT_ALLOWED_PARAMS = [
"utm_source",
"utm_medium",
"utm_campaign",
"utm_content",
"utm_term",
"source",
"medium",
"campaign",
"content",
"term",
"ref",
];

// Paths the plugin rewrites to the Simple Analytics script and queue.
export const PROXY_PATHS = /^\/(proxy\.js|auto-events\.js|simple\/.*)$/;

// Stylesheets, JavaScript, images, fonts, audio, video and JSON (with source
// maps and web app manifests).
const SKIPPED_EXTENSIONS =
/\.(css|js|mjs|cjs|apng|avif|bmp|gif|ico|jpe?g|png|svg|tiff?|webp|eot|otf|ttf|woff2?|aac|flac|m4a|mp3|oga|ogg|opus|wav|weba|m4v|mov|mp4|ogv|webm|json|map|webmanifest)$/i;

function isSkipped(request: Request, path: string) {
return (
request.method !== "GET" ||
SKIPPED_EXTENSIONS.test(path) ||
// Next.js build assets and the image optimizer.
path.includes("/_next/") ||
PROXY_PATHS.test(path) ||
// Pages Router JSON data, which Next.js shows middleware under the page
// path. App Router payloads cannot be told apart: Next.js removes their
// markers before middleware runs.
request.headers.has("x-nextjs-data")
);
}

// Request headers copied into the body, named in lower case with underscores.
// They cannot travel under their own names: fetch sets its own Accept,
// Accept-Language, User-Agent and Sec-Fetch-Mode on the request to the queue.
const COPIED_HEADERS = {
user_agent: "User-Agent",
referer: "Referer",
accept: "Accept",
accept_language: "Accept-Language",
sec_fetch_mode: "Sec-Fetch-Mode",
sec_fetch_dest: "Sec-Fetch-Dest",
sec_fetch_site: "Sec-Fetch-Site",
sec_fetch_user: "Sec-Fetch-User",
sec_ch_ua: "Sec-CH-UA",
signature_agent: "Signature-Agent",
from: "From",
} as const;

type CopiedHeaderField = keyof typeof COPIED_HEADERS;

// Address headers travel to the queue under their own names, unchanged.
// Next.js gives proxy code no connection address, but it sets X-Forwarded-For
// to that address when a request arrives without one, and Vercel sets it to
// the visitor's address.
const ADDRESS_HEADERS = [
"X-Forwarded-For",
"CF-Connecting-IP",
"True-Client-IP",
] as const;

export type RequestPayload = {
version: 1;
platform: "nextjs";
hostname: string;
path: string;
query?: string;
test?: true;
} & { [Field in CopiedHeaderField]?: string };

function filterQuery(searchParams: URLSearchParams, allowParams: string[]) {
const allowed = new Set([...DEFAULT_ALLOWED_PARAMS, ...allowParams]);
const query = new URLSearchParams();

for (const [name, value] of searchParams) {
if (allowed.has(name)) {
query.append(name, value);
}
}

return query.toString();
}

export function buildRequestReport(
request: Request,
options: {
hostname: string;
allowParams?: string[] | undefined;
test?: boolean | undefined;
},
) {
const url = new URL(request.url);

if (isSkipped(request, url.pathname)) {
return undefined;
}

const query = filterQuery(url.searchParams, options.allowParams ?? []);

const body: RequestPayload = {
version: 1,
platform: "nextjs",
hostname: options.hostname,
path: url.pathname,
...(query && { query }),
...(options.test && { test: true }),
};

for (const field of Object.keys(COPIED_HEADERS) as CopiedHeaderField[]) {
const value = request.headers.get(COPIED_HEADERS[field]);

if (value !== null) {
body[field] = value;
}
}

const headers: Record<string, string> = {};

for (const name of ADDRESS_HEADERS) {
const value = request.headers.get(name);

if (value) {
headers[name] = value;
}
}

return { body, headers };
}
3 changes: 1 addition & 2 deletions packages/analytics/src/server/simple-analytics.ts
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@ import {
} from "./utils";
import { parseHeaders } from "./headers";
import { parseUtmParameters } from "./utm";
import { PROXY_PATHS } from "./request-payload";

type TrackEventOptions = TrackingOptions & (ServerContext | HeaderOnlyContext);

Expand Down Expand Up @@ -78,8 +79,6 @@ export async function trackEvent(
}
}

const PROXY_PATHS = /^\/(proxy\.js|auto-events\.js|simple\/.*)$/;

type TrackPageviewOptions = TrackingOptions & ServerContext;

export async function trackPageview(options: TrackPageviewOptions) {
Expand Down
Loading
Loading