Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
38 changes: 34 additions & 4 deletions apps/docs/content/docs/integrations/file.mdx
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
title: File
description: Read, get content, fetch, write, append, compress, decompress, and manage sharing for files
description: Read, search, get content, fetch, write, append, compress, decompress, and manage sharing for files
---

import { BlockInfoCard } from "@/components/ui/block-info-card"
Expand All @@ -11,23 +11,24 @@ import { BlockInfoCard } from "@/components/ui/block-info-card"
/>

{/* MANUAL-CONTENT-START:intro */}
The File block is a built-in Sim block for working with files stored in the workspace, or fetched from external URLs. It handles reading, writing, appending, compressing, decompressing, and sharing files as part of a workflow.
The File block is a built-in Sim block for working with files stored in the workspace, or fetched from external URLs. It handles reading, searching, writing, appending, compressing, decompressing, and sharing files as part of a workflow.

With the File block, you can:

- **Read and extract content**: Load workspace file objects and extract their text content
- **Search workspace content**: Find literal text across indexed active workspace files with bounded line-level results
- **Fetch from URLs**: Retrieve and parse files from external URLs with custom headers
- **Write and append**: Create new workspace files or append content to existing ones
- **Compress and decompress**: Bundle files into a .zip archive or extract an archive into the workspace
- **Manage sharing**: Enable or disable a public share link for a file, with public, password, email, or SSO access modes

In Sim, the File block allows your agents to read and extract text from workspace files, fetch and parse files from URLs, write or append content to files, bundle files into or out of .zip archives, and control public sharing access for a file—all programmatically as steps in a workflow. This makes it possible to move file content into and out of a workflow, package outputs for download or transfer, and expose files to external users through a managed share link.
In Sim, the File block allows your agents to search, read, and extract text from workspace files, fetch and parse files from URLs, write or append content to files, bundle files into or out of .zip archives, and control public sharing access for a file—all programmatically as steps in a workflow. This makes it possible to explore workspace content, move file content into and out of a workflow, package outputs for download or transfer, and expose files to external users through a managed share link.
{/* MANUAL-CONTENT-END */}


## Usage Instructions

Read workspace file objects, extract the text content of files, fetch and parse files from URLs with optional headers, write new workspace files, append content to existing files, compress files into a .zip archive, extract a .zip archive into the workspace, or manage the public share link for a file.
Read workspace file objects, search indexed text across all active workspace files, extract the text content of files, fetch and parse files from URLs with optional headers, write new workspace files, append content to existing files, compress files into a .zip archive, extract a .zip archive into the workspace, or manage the public share link for a file.



Expand Down Expand Up @@ -67,6 +68,35 @@ Extract the text content of one or more workspace files from selected file objec
| --------- | ---- | ----------- |
| `contents` | array | Array of file text contents, one entry per file in input order |

### File Search

Search indexed text across active workspace files using literal smart-case substring matching.

#### Input

| Parameter | Type | Required | Description |
| --------- | ---- | -------- | ----------- |
| `query` | string | Yes | Literal text to find \(3-512 characters\). Uppercase Unicode letters make matching case-sensitive. |
| `maxResults` | number | No | Hard result cap configured by the workflow builder \(1-200, default 50\). |

#### Output

| Parameter | Type | Description |
| --------- | ---- | ----------- |
| `results` | array | Matching logical lines with their workspace file ID and 1-based line number. |
| ↳ `fileId` | string | Canonical workspace file ID. |
| ↳ `lineNumber` | number | 1-based logical line number. |
| ↳ `text` | string | Matching line or bounded match-centered preview. |
| `count` | number | Number of returned matching lines. |
| `truncated` | boolean | Whether more matching lines exist beyond the configured hard cap. |
| `complete` | boolean | Whether indexing has no pending or failed current revisions; skipped and partial coverage is reported separately. |
| `indexStatus` | object | Current workspace search-index coverage by file status. |
| ↳ `readyFiles` | number | Files whose current revision is searchable. |
| ↳ `pendingFiles` | number | Files still waiting to be indexed. |
| ↳ `failedFiles` | number | Files whose current indexing attempt failed. |
| ↳ `skippedFiles` | number | Files intentionally excluded because they are unsupported or oversized. |
| ↳ `partialFiles` | number | Searchable files whose extracted text was truncated by the parser or cap. |

### File Fetch

Fetch and parse a file from a URL with optional custom headers.
Expand Down
68 changes: 68 additions & 0 deletions apps/sim/app/api/cron/workspace-file-search-dispatch/route.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
/**
* @vitest-environment node
*/
import { createMockRequest } from '@sim/testing'
import { beforeEach, describe, expect, it, vi } from 'vitest'

const mocks = vi.hoisted(() => ({
enqueueDispatch: vi.fn(),
verifyCronAuth: vi.fn(),
}))

vi.mock('@/lib/auth/internal', () => ({ verifyCronAuth: mocks.verifyCronAuth }))
vi.mock('@/lib/workspace-files/search/enqueue-dispatch', () => ({
enqueueWorkspaceFileSearchDispatch: mocks.enqueueDispatch,
}))

import { GET } from '@/app/api/cron/workspace-file-search-dispatch/route'

function request() {
return createMockRequest(
'GET',
undefined,
{},
'http://localhost:3000/api/cron/workspace-file-search-dispatch'
)
}

describe('workspace file search dispatch route', () => {
beforeEach(() => {
vi.clearAllMocks()
mocks.verifyCronAuth.mockReturnValue(null)
})

it('returns as soon as Trigger.dev accepts the dispatcher run', async () => {
mocks.enqueueDispatch.mockResolvedValue({ backend: 'trigger-dev', jobId: 'run-1' })

const response = await GET(request())

expect(response.status).toBe(202)
await expect(response.json()).resolves.toEqual({
success: true,
triggered: true,
backend: 'trigger-dev',
jobId: 'run-1',
})
})

it('returns the cron auth refusal without dispatching', async () => {
mocks.verifyCronAuth.mockReturnValue(new Response(null, { status: 401 }))

const response = await GET(request())

expect(response.status).toBe(401)
expect(mocks.enqueueDispatch).not.toHaveBeenCalled()
})

it('fails closed when Trigger.dev does not accept the dispatcher run', async () => {
mocks.enqueueDispatch.mockRejectedValue(new Error('trigger unavailable'))

const response = await GET(request())

expect(response.status).toBe(500)
await expect(response.json()).resolves.toEqual({
success: false,
error: 'Dispatcher enqueue failed',
})
})
})
30 changes: 30 additions & 0 deletions apps/sim/app/api/cron/workspace-file-search-dispatch/route.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
import { createLogger } from '@sim/logger'
import { toError } from '@sim/utils/errors'
import { type NextRequest, NextResponse } from 'next/server'
import { verifyCronAuth } from '@/lib/auth/internal'
import { withRouteHandler } from '@/lib/core/utils/with-route-handler'
import { enqueueWorkspaceFileSearchDispatch } from '@/lib/workspace-files/search/enqueue-dispatch'

const logger = createLogger('WorkspaceFileSearchDispatchRoute')

export const dynamic = 'force-dynamic'
export const maxDuration = 60

export const GET = withRouteHandler(async (request: NextRequest) => {
Comment thread
icecrasher321 marked this conversation as resolved.
const authError = verifyCronAuth(request, 'Workspace file search dispatcher')
if (authError) return authError

try {
const result = await enqueueWorkspaceFileSearchDispatch()
logger.info('Workspace file search dispatcher accepted', result)
return NextResponse.json({ success: true, triggered: true, ...result }, { status: 202 })
} catch (error) {
logger.error('Workspace file search dispatcher enqueue failed', {
error: toError(error).message,
})
return NextResponse.json(
{ success: false, error: 'Dispatcher enqueue failed' },
{ status: 500 }
)
}
})
45 changes: 45 additions & 0 deletions apps/sim/background/workspace-file-search-dispatch.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
/**
* @vitest-environment node
*/
import { beforeEach, describe, expect, it, vi } from 'vitest'

const mocks = vi.hoisted(() => ({
dispatch: vi.fn(),
task: vi.fn((config: unknown) => config),
}))

vi.mock('@trigger.dev/sdk', () => ({ task: mocks.task }))
vi.mock('@/lib/workspace-files/search/dispatcher', () => ({
dispatchWorkspaceFileSearchIndexJobs: mocks.dispatch,
}))

import { FILE_SEARCH_DISPATCH_MAX_DURATION_SECONDS } from '@/lib/workspace-files/search/constants'
import { workspaceFileSearchDispatchTask } from '@/background/workspace-file-search-dispatch'

describe('workspace file search dispatch task', () => {
beforeEach(() => {
vi.clearAllMocks()
})

it('serializes bounded dispatcher runs outside the cron request', async () => {
expect(workspaceFileSearchDispatchTask).toMatchObject({
id: 'workspace-file-search-dispatch',
machine: 'small-1x',
maxDuration: FILE_SEARCH_DISPATCH_MAX_DURATION_SECONDS,
retry: { maxAttempts: 3 },
queue: {
name: 'workspace-file-search-dispatch',
concurrencyLimit: 1,
},
})

mocks.dispatch.mockResolvedValue({
dispatchedFiles: 2,
backfilledFiles: 1000,
reapedClaims: 0,
lockAcquired: true,
})
await workspaceFileSearchDispatchTask.run()
expect(mocks.dispatch).toHaveBeenCalledOnce()
})
})
19 changes: 19 additions & 0 deletions apps/sim/background/workspace-file-search-dispatch.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
import { task } from '@trigger.dev/sdk'
import { FILE_SEARCH_DISPATCH_MAX_DURATION_SECONDS } from '@/lib/workspace-files/search/constants'
import { dispatchWorkspaceFileSearchIndexJobs } from '@/lib/workspace-files/search/dispatcher'

/**
* Runs the bounded search-index control plane outside the cron request. Per-file parsing remains
* isolated in `workspace-file-search-index`; this task only backfills, claims, and enqueues work.
*/
export const workspaceFileSearchDispatchTask = task({
id: 'workspace-file-search-dispatch',
machine: 'small-1x',
maxDuration: FILE_SEARCH_DISPATCH_MAX_DURATION_SECONDS,
retry: { maxAttempts: 3 },
queue: {
name: 'workspace-file-search-dispatch',
concurrencyLimit: 1,
},
run: () => dispatchWorkspaceFileSearchIndexJobs(),
})
68 changes: 68 additions & 0 deletions apps/sim/background/workspace-file-search-index.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
/**
* @vitest-environment node
*/
import { beforeEach, describe, expect, it, vi } from 'vitest'

const mocks = vi.hoisted(() => ({
indexWorkspaceFile: vi.fn(),
markFailed: vi.fn(),
task: vi.fn((config: unknown) => config),
}))

vi.mock('@trigger.dev/sdk', () => ({ task: mocks.task }))
vi.mock('@/lib/workspace-files/search/indexing', () => ({
indexWorkspaceFileForSearch: mocks.indexWorkspaceFile,
markWorkspaceFileSearchIndexFailed: mocks.markFailed,
}))

import {
FILE_SEARCH_INDEX_GLOBAL_CONCURRENCY,
FILE_SEARCH_INDEX_MAX_DURATION_SECONDS,
} from '@/lib/workspace-files/search/constants'
import { workspaceFileSearchIndexTask } from '@/background/workspace-file-search-index'

const payload = {
workspaceId: 'workspace-1',
fileId: 'file-1',
sourceContentUpdatedAt: '2026-08-29T12:00:00.000Z',
}

describe('workspace file search index task', () => {
beforeEach(() => {
vi.clearAllMocks()
})

it('uses isolated medium workers with a hard global concurrency and duration cap', () => {
expect(workspaceFileSearchIndexTask).toMatchObject({
id: 'workspace-file-search-index',
machine: 'medium-1x',
maxDuration: FILE_SEARCH_INDEX_MAX_DURATION_SECONDS,
retry: { maxAttempts: 3 },
queue: {
name: 'workspace-file-search-index',
concurrencyLimit: FILE_SEARCH_INDEX_GLOBAL_CONCURRENCY,
},
})
})

it('passes the Trigger.dev abort signal to the single-revision indexer', async () => {
const signal = new AbortController().signal
mocks.indexWorkspaceFile.mockResolvedValue(undefined)

await workspaceFileSearchIndexTask.run(payload, { signal })

expect(mocks.indexWorkspaceFile).toHaveBeenCalledWith(payload, signal)
})

it('marks the revision failed only from the terminal onFailure hook', async () => {
mocks.indexWorkspaceFile.mockRejectedValue(new Error('retryable parser failure'))

await expect(
workspaceFileSearchIndexTask.run(payload, { signal: new AbortController().signal })
).rejects.toThrow('retryable parser failure')
expect(mocks.markFailed).not.toHaveBeenCalled()

await workspaceFileSearchIndexTask.onFailure({ payload })
expect(mocks.markFailed).toHaveBeenCalledWith(payload)
})
})
30 changes: 30 additions & 0 deletions apps/sim/background/workspace-file-search-index.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
import { task } from '@trigger.dev/sdk'
import {
FILE_SEARCH_INDEX_GLOBAL_CONCURRENCY,
FILE_SEARCH_INDEX_MAX_DURATION_SECONDS,
} from '@/lib/workspace-files/search/constants'
import {
indexWorkspaceFileForSearch,
markWorkspaceFileSearchIndexFailed,
type WorkspaceFileSearchIndexPayload,
} from '@/lib/workspace-files/search/indexing'

/**
* Builds one immutable workspace-file search revision. PostgreSQL owns the durable state; this
* task only supplies isolated compute, retries, and a hard global execution cap.
*/
export const workspaceFileSearchIndexTask = task({
id: 'workspace-file-search-index',
machine: 'medium-1x',
maxDuration: FILE_SEARCH_INDEX_MAX_DURATION_SECONDS,
retry: { maxAttempts: 3 },
queue: {
name: 'workspace-file-search-index',
concurrencyLimit: FILE_SEARCH_INDEX_GLOBAL_CONCURRENCY,
},
run: (payload: WorkspaceFileSearchIndexPayload, { signal }) =>
indexWorkspaceFileForSearch(payload, signal),
onFailure: async ({ payload }) => {
await markWorkspaceFileSearchIndexFailed(payload)
},
})
1 change: 1 addition & 0 deletions apps/sim/blocks/blocks.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -170,6 +170,7 @@ describe.concurrent('Blocks Module', () => {
expect(block?.subBlocks[0].options?.map((option) => option.id)).toEqual([
'file_read',
'file_get_content',
'file_search',
'file_fetch',
'file_write',
'file_append',
Expand Down
Loading
Loading