feat: add node batches 2 and 3 with their templates

CONTEXT.md

@@ -63,9 +63,14 @@ One execution of a pipeline over a set of input items.
 _Avoid_: job, batch, execution
 
 **Item**:
-One unit that flows through a pipeline: an image, data (JSON or text) or a document (PDF).
+One unit that flows through a pipeline: an image, data (JSON or text) or a document (PDF or ICO).
 _Avoid_: file, asset
 
+**Document**:
+An item that is a finished file Hexlode makes but does not edit further, such as a PDF or an ICO
+icon file.
+_Avoid_: binary, attachment
+
 **Accepts**:
 The kinds and formats of items a node can process.
 _Avoid_: supports, input types
@@ -83,7 +88,8 @@ An item a node accepted but could not process.
 _Avoid_: errored, broken
 
 **Combining node**:
-A node that waits for every upstream item and then runs once over all of them, such as Contact sheet.
+A node that waits for every upstream item and then runs once over all of them, such as Contact
+sheet, Images to PDF or Deduplicate.
 _Avoid_: gather node, aggregate node, merge node
 
 **Step cache**:

docs/adr/0011-combining-nodes-pass-inputs-by-reference.md

@@ -0,0 +1,19 @@
+# Combining nodes pass their inputs on by reference
+
+Deduplicate must see every item before it can say which are duplicates, so it is a combining node,
+yet it sends most items on unchanged. Instead of returning the items it read, a combining node
+may return `{ port, input }`: the index of an item it received. The engine reloads that item from
+where it waited, so the node never holds the batch in memory, and the item keeps its step cache
+key and the source it came from, so later nodes reuse their cache and results still list the file.
+
+## Considered options
+
+- Returning the items themselves holds every kept item in memory at once, which grows with batch
+  size.
+- Deduplicating per item would need a fingerprint store shared between workers, and items in
+  flight in other workers could still race past each other.
+
+## Consequences
+
+- Step cache entries record the input index of passed items, and a cache hit reloads them from the
+  waiting items of the current run, which every run writes before a combining node runs.

docs/adr/0012-ico-files-are-documents.md

@@ -0,0 +1,7 @@
+# ICO files are documents
+
+Favicon / App icons makes `favicon.ico`. We treat ICO as a document format, beside PDF, rather
+than an image format. Image formats are the ones Hexlode decodes and encodes, and every node that
+takes images accepts all of them; an ICO image would reach Resize or Convert and fail. As a
+document, an ICO file is a finished file that only Output and Rename take, and connections from
+Favicon / App icons to image nodes narrow to the PNG icons.

idea.md

@@ -1,6 +1,6 @@
 # Hexlode product
 
-> Updated: 2026-09-28 (directed Studio scenes on the home page, footer credits and wordmark glow)
+> Updated: 2026-11-04 (node batches 2 and 3)
 > Delivery plan: [implementation.md](./implementation.md). Vocabulary: [CONTEXT.md](./CONTEXT.md).
 > Decisions and their reasons: [docs/adr/](./docs/adr/).
 
@@ -189,7 +189,7 @@ Items are images, data (JSON or text) or documents (PDF). Batch numbers match th
 | Files | Takes dropped files or a folder. Starts every pipeline. | 1 |
 | Filter | Routes items by rules on format, file size, dimensions, orientation or transparency. Each rule has its own output, plus an output for everything else. An item leaves by the first rule it matches. | 1 |
 | Inspect | Shows format, dimensions, size and metadata per item. Passes items through unchanged. | 1 |
-| Deduplicate | Drops exact duplicates, or near duplicates by image fingerprint. | 2 |
+| Deduplicate | Drops exact duplicates (the same pixels), or near duplicates by image fingerprint. Waits for every image; duplicates leave by their own output. | 2 |
 
 ### Size and shape
 
@@ -225,7 +225,7 @@ Items are images, data (JSON or text) or documents (PDF). Batch numbers match th
 | Node | What it does | Batch |
 |---|---|---|
 | Strip metadata | Removes all metadata, only location data, or everything except copyright, without re-encoding. Keeps the colour profile unless told otherwise, and keeps a camera orientation tag so photos stay upright. | 1 |
-| Set copyright / author | Writes author and copyright fields. | 3 |
+| Set copyright / author | Writes author and copyright fields without re-encoding. Empty fields stay as they are. | 3 |
 
 ### Output and encoding
 
@@ -238,12 +238,12 @@ Items are images, data (JSON or text) or documents (PDF). Batch numbers match th
 | Output | Saves items, passes them on, and delivers a ZIP or folder. | 1 |
 | Compare | Shows a before and after slider and the size difference. Passes items through. | 1 |
 | Best format | Encodes several formats and keeps the smallest. | 3 |
-| Responsive set | Makes several widths and a ready `srcset` HTML snippet. | 3 |
-| Favicon / App icons | Makes every favicon and app icon size from one image. | 3 |
-| Placeholder | Generates a tiny blurred preview code for websites. | 3 |
+| Responsive set | Makes several widths and a ready `srcset` HTML snippet. Widths larger than an image become its own width. | 3 |
+| Favicon / App icons | Makes every favicon and app icon size from one image, an ICO file, a web app manifest and the HTML tags. | 3 |
+| Placeholder | Generates a tiny blurred preview code for websites: a data URI or a BlurHash, with the size and average colour, as JSON. | 3 |
 | Palette | Extracts the main colours as JSON or CSS variables. | 3 |
-| Contact sheet | Combines many images into one grid image. | 3 |
-| Images to PDF | Combines images into one PDF. | 3 |
+| Contact sheet | Combines many images into one grid image, with file names if wanted. Starts a new sheet when one is full. | 3 |
+| Images to PDF | Combines images into one PDF, one page each. JPEG files go in as they are. | 3 |
 
 ## Saving and sharing pipelines
 

implementation.md

@@ -1,6 +1,6 @@
 # Hexlode implementation plan
 
-> Updated: 2026-09-27
+> Updated: 2026-11-04
 > Product: [idea.md](./idea.md). Vocabulary: [CONTEXT.md](./CONTEXT.md). Decisions: [docs/adr/](./docs/adr/).
 
 ## Goal
@@ -10,7 +10,7 @@ Studio with all 33 nodes from the catalogue in [idea.md](./idea.md). It ships in
 builds the whole application and node batch 1. Phases 2 and 3 only add node batches 2 and 3. Each
 phase is deployable and passes its exit gate before the next phase starts.
 
-Active phase: **Phase 1**.
+Active phase: **Phase 3**. Phases 2 and 3 are built and pass their exit gates.
 
 ## Phase 1: the application and node batch 1
 
@@ -87,6 +87,12 @@ Watermark and compress and Instagram carousel templates.
 Exit gate: each node has output tests on real fixtures, the node pair matrix passes with the new
 nodes, and the new templates run end to end.
 
+Deduplicate is a combining node with two outputs, Unique and Duplicates, and passes items on by
+reference ([ADR 0011](./docs/adr/0011-combining-nodes-pass-inputs-by-reference.md)). Image
+watermark logos are kept in the pipeline itself, at most 512 KB; an SVG logo is drawn to a PNG
+when it is chosen, because workers cannot draw SVG. Text is drawn with an OffscreenCanvas and
+blended with the engine's own compositing.
+
 ## Phase 3: node batch 3
 
 Best format, Responsive set, Favicon / App icons, Placeholder, Palette, Contact sheet, Images to
@@ -95,17 +101,24 @@ PDF, Set copyright / author. Adds the Responsive image set template.
 Exit gate: same as phase 2, plus tests for nodes that change the item count (one image to many,
 many images to one) and nodes that produce data or documents.
 
+Output and Rename take every kind of item; the other nodes take images. Favicon / App icons makes
+an ICO file, which is a document ([ADR 0012](./docs/adr/0012-ico-files-are-documents.md)), and puts
+each image's icons in a folder named after it. Live previews show the text of data items, such as
+a palette or a snippet, in the inspector.
+
 ## Engine design
 
 - A pipeline is a directed graph without cycles. Items flow through it one by one, so an item can
   reach the last node while later items are still at the first node.
-- Nodes that combine items (Contact sheet, Images to PDF, near-duplicate Deduplicate) wait until
-  every upstream item has arrived.
+- Nodes that combine items (Contact sheet, Images to PDF, Deduplicate) wait until every upstream
+  item has arrived. A combining node that passes items on unchanged returns them by reference, so
+  it never holds the batch ([ADR 0011](./docs/adr/0011-combining-nodes-pass-inputs-by-reference.md)).
 - Decoding happens at most once per item, the first time a node needs pixels; nodes that only read
   headers or metadata (Filter by format, Inspect, Rename, Strip metadata) never decode. Nodes pass
-  pixels between them. Encoding happens at Convert, Compress to size and Optimize PNG. An image
-  that reaches Output without an encoding node keeps its source format, and its original bytes
-  when nothing changed its pixels.
+  pixels between them. Encoding happens at Convert, Compress to size and Optimize PNG, and at
+  the nodes that make finished files: Best format, Responsive set, Favicon / App icons and Contact
+  sheet. An image that reaches Output without an encoding node keeps its source format, and its
+  original bytes when nothing changed its pixels.
 - Metadata is rewritten in the container without re-encoding for JPEG, PNG, WebP and JPEG XL.
   AVIF metadata is read and can be removed in place but not added; QOI holds none. The run reports
   a warning when a format cannot keep metadata.
@@ -139,7 +152,9 @@ removed.
 - Fixtures are small files in the repository, generated or with a licence that allows it, covering
   every input format, transparency, EXIF orientation, location metadata and a malformed file.
 - The node pair matrix connects every pair of node types and checks that the Studio's accept or
-  refuse decision matches what the engine does when it runs that pair.
+  refuse decision matches what the engine does when it runs that pair. It feeds one fixture per
+  format plus one different picture, so nodes that route by content send items down every output,
+  and runs as four shards that browsers run in parallel.
 - Worker, OPFS, codec and ZIP tests run in real browsers through Vitest browser mode. Locally they
   run in Chromium with `PLAYWRIGHT_CHROMIUM_EXECUTABLE_PATH`; `VITEST_BROWSERS=firefox` or
   `webkit` picks another engine. CI runs Chromium, Firefox and WebKit. Playwright's WebKit has no

src/features/engine/__tests__/estimate.test.ts

@@ -45,6 +45,18 @@ describe('estimateRun', () => {
     expect(estimate).toMatchObject({ decodes: 4, encodes: 4 })
   })
 
+  it('follows items through Deduplicate, counting every item as unique', () => {
+    const estimate = estimateRun({
+      pipeline: chain(['deduplicate'], ['convert', { format: 'webp' }]),
+      registry: productRegistry,
+      sources: sources(5),
+      workers: 1,
+    })
+    // Deduplicate decodes each item for its fingerprint. The item waits without its pixels, so
+    // Convert decodes it again.
+    expect(estimate).toMatchObject({ decodes: 10, encodes: 5 })
+  })
+
   it('counts items that will skip a branch', () => {
     const estimate = estimateRun({
       pipeline: chain(['optimize-png']),

src/features/engine/__tests__/pool-host.browser.test.ts

@@ -89,6 +89,91 @@ describe('worker pool', () => {
     for (const name of await listNames(root)) await root.removeEntry(name, { recursive: true })
   })
 
+  it('passes the items Deduplicate keeps on from storage, and reuses them on the next run', async () => {
+    const root = await appDirectory()
+    const index = await createStepCacheIndex(root, 1024 ** 3)
+    const host = createWorkerPoolHost({ size: 2, createWorker, index })
+    const pipeline = chain(['deduplicate', { mode: 'exact' }])
+    // photo.png and photo.qoi have the same pixels; photo.jpg is lossy, so it differs.
+    const names = ['photo.png', 'photo.jpg', 'photo.qoi']
+    // A new run deletes the last run's files, so each delivery is read before the next run.
+    const delivered = async (result: Awaited<ReturnType<typeof run>>['result']) => {
+      const entries = await unzip(result.deliveries[0].archive as Blob)
+      expect(entries.map(({ name }) => name)).toEqual(['photo.png', 'photo.jpg'])
+      expect(entries[0].bytes).toEqual(await fixtureBytes('photo.png'))
+      expect(entries[1].bytes).toEqual(await fixtureBytes('photo.jpg'))
+    }
+    const first = await run(pipeline, host, names)
+    expect(first.result.status).toBe('complete')
+    expect(first.statuses('deduplicate-1')).toEqual(['processed'])
+    await delivered(first.result)
+    const second = await run(pipeline, host, names)
+    expect(second.statuses('deduplicate-1')).toEqual(['cached'])
+    await delivered(second.result)
+    host.dispose()
+  })
+
+  it('delivers documents and data that nodes make in workers', async () => {
+    const host = createWorkerPoolHost({ size: 2, createWorker, index: null })
+    const node = (id: string, type: string, settings = {}) => ({
+      id,
+      type,
+      settings,
+      position: { x: 0, y: 0 },
+    })
+    const pipeline: Pipeline = {
+      nodes: [
+        node('files', 'files'),
+        node('pdf', 'images-to-pdf', { name: 'album' }),
+        node('palette', 'palette', { count: 2 }),
+        node('icons', 'favicon'),
+        node('out-pdf', 'output'),
+        node('out-palette', 'output'),
+        node('out-icons', 'output'),
+      ],
+      connections: [
+        ['files', 'pdf'],
+        ['files', 'palette'],
+        ['files', 'icons'],
+        ['pdf', 'out-pdf'],
+        ['palette', 'out-palette'],
+        ['icons', 'out-icons'],
+      ].map(([source, target]) => ({
+        id: `${source}-${target}`,
+        source,
+        sourcePort: 'out',
+        target,
+      })),
+    }
+    const { result } = await run(pipeline, host, ['photo.jpg', 'photo.png'])
+    host.dispose()
+    expect(result.status).toBe('complete')
+    const delivered = async (nodeId: string) => {
+      const delivery = result.deliveries.find((entry) => entry.nodeId === nodeId)
+      return unzip(delivery?.archive as Blob)
+    }
+    const [pdf] = await delivered('out-pdf')
+    expect(pdf.name).toBe('album.pdf')
+    const pdfText = new TextDecoder('latin1').decode(pdf.bytes)
+    expect(pdfText.startsWith('%PDF-1.4')).toBe(true)
+    expect(pdfText.match(/\/Type \/Page(?!s)/g)).toHaveLength(2)
+    const palettes = await delivered('out-palette')
+    // Workers finish in any order, so Output numbers whichever arrives second.
+    expect(palettes.map(({ name }) => name).sort()).toEqual([
+      'photo.palette-2.json',
+      'photo.palette.json',
+    ])
+    const values = palettes.map(({ bytes }) => JSON.parse(new TextDecoder().decode(bytes)))
+    const png = values.find((value) => value.image === 'photo.png')
+    expect(png.colours.map(({ hex }: { hex: string }) => hex)).toEqual(['#0000ff', '#ff0000'])
+    const icons = (await delivered('out-icons')).map(({ name }) => name)
+    expect(icons).toContain('photo/favicon.ico')
+    expect(icons).toContain('photo/site.webmanifest')
+    // Both images are named photo, so Output numbers the second set's files.
+    expect(icons.filter((name) => name.startsWith('photo/icon-512'))).toHaveLength(2)
+    expect(icons).toHaveLength(18)
+  })
+
   it('runs items in workers and delivers a ZIP backed by OPFS', async () => {
     const host = createWorkerPoolHost({ size: 2, createWorker, index: null })
     const { result } = await run(chain(['convert', { format: 'png' }]), host, ALL_FORMAT_FIXTURES)

src/features/engine/__tests__/runner.test.ts

@@ -241,6 +241,107 @@ describe('runPipeline', () => {
     expect(order.slice(3)).toEqual(['join:1.png+2.png+3.png', '%PDF 3 pages'])
   })
 
+  describe('combining nodes that pass their inputs on', () => {
+    /** Waits for every item, then sends items with an odd first byte to `out`, others to `rest`. */
+    const pick = defineTestNode({
+      type: 'pick',
+      mode: 'all',
+      ports: ['out', 'rest'],
+      run: async (input) => {
+        if (input.mode !== 'all') throw new Error('all only')
+        const results = []
+        let index = 0
+        for await (const item of input.items) {
+          const odd = (item.payload as TestPayload).bytes[0] % 2 === 1
+          results.push({ port: odd ? 'out' : 'rest', input: index })
+          index += 1
+        }
+        return results
+      },
+    })
+
+    function setup() {
+      const seen: { name: string; bytes: number[] }[] = []
+      const after = defineTestNode({
+        type: 'after',
+        run: async (input) => {
+          if (input.mode !== 'each') throw new Error('each only')
+          const { bytes } = input.item.payload as TestPayload
+          seen.push({ name: input.item.meta.name, bytes: [...bytes] })
+          return [{ port: 'out', item: input.item }]
+        },
+      })
+      const registry = createTestRegistry([sourceNode, pick, after])
+      const pipeline = pipelineOf(
+        [
+          ['files', 'source'],
+          ['pick', 'pick'],
+          ['after', 'after'],
+        ],
+        [
+          ['files', 'pick'],
+          ['pick', 'after'],
+        ],
+      )
+      return { seen, registry, pipeline }
+    }
+
+    it('sends the items on unchanged, in order, still knowing their source file', async () => {
+      const { seen, registry, pipeline } = setup()
+      const { events } = await collect({
+        pipeline,
+        registry,
+        sources: sources(['a.png'], ['b.png'], ['c.png']),
+        host: createInlineHost({ registry }),
+      })
+      // The test items hold [index + 1, 7], so a.png and c.png have odd first bytes.
+      expect(seen).toEqual([
+        { name: 'a.png', bytes: [1, 7] },
+        { name: 'c.png', bytes: [3, 7] },
+      ])
+      const after = events.flatMap((event) =>
+        event.type === 'node-item' && event.nodeId === 'after' ? [event.source] : [],
+      )
+      expect(after).toEqual([0, 2])
+    })
+
+    it('lets later nodes reuse their step cache for the items passed on', async () => {
+      const { seen, registry, pipeline } = setup()
+      const host = createInlineHost({ registry, cache: createMemoryStepCache() })
+      const input = sources(['a.png'], ['b.png'], ['c.png'])
+      await collect({ pipeline, registry, sources: input, host })
+      seen.length = 0
+      const { events } = await collect({ pipeline, registry, sources: input, host })
+      expect(statuses(events, 'pick')).toEqual(['cached'])
+      expect(statuses(events, 'after')).toEqual(['cached', 'cached'])
+      expect(seen).toEqual([])
+    })
+
+    it('fails a per-item node that tries to pass its input on', async () => {
+      const cheat = defineTestNode({ type: 'cheat', run: async () => [{ port: 'out', input: 0 }] })
+      const registry = createTestRegistry([sourceNode, cheat])
+      const { events } = await collect({
+        pipeline: pipelineOf(
+          [
+            ['files', 'source'],
+            ['cheat', 'cheat'],
+          ],
+          [['files', 'cheat']],
+        ),
+        registry,
+        sources: sources(['a.png']),
+        host: createInlineHost({ registry }),
+      })
+      expect(events).toContainEqual(
+        expect.objectContaining({
+          nodeId: 'cheat',
+          status: 'failed',
+          error: 'Only combining nodes can pass their inputs on.',
+        }),
+      )
+    })
+  })
+
   it('runs only the changed node and the nodes after it when run again', async () => {
     const { calls, passthrough } = counter()
     const registry = createTestRegistry([

src/features/engine/constants.ts

@@ -20,6 +20,8 @@ export const PREVIEW_SAMPLE_EDGE = 1024
 export const PREVIEW_THUMBNAIL_EDGE = 192
 /** Compare nodes show a larger before and after pair. */
 export const PREVIEW_COMPARE_EDGE = 640
+/** Data a node makes, such as a palette or a snippet, shows up to this many characters. */
+export const PREVIEW_TEXT_CHARACTERS = 4000
 
 /** Shown when an engine worker fails without a message, which is how a script load failure looks. */
 export const ENGINE_LOAD_FAILED_MESSAGE =