You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit 9bfec03
Browse filesBrowse the repository at this point in the historyBrowse files
fix(mrscraper): route html-only scrapes through the fetch endpoint
- send HTML-only and screenshot-only scrapes as GET with the target URL in
the query; the fetch endpoint replays the request method, so a POST fails
on sites that reject it
- read raw text/html responses and take token usage from response headers
- send screenshot=full and detect PNG vs JPEG from the returned image
- keep the API key out of the query string
- report the requested search page; the SERP API returns current_page 1
for every page
- show retry and token-cap fields only for Markdown requests
- use defaultValue for switch defaults
- regenerate the integration catalog, tool metadata, block names, and docs
manifest
Use [MrScraper](https://mrscraper.com) to retrieve a page as HTML, Markdown, or a screenshot, search Google, and check your account's remaining tokens. Use a MrScraper block for a configured workflow step, or connect the [MrScraper MCP server](#connect-the-mcp-server) to give its tools to an Agent.
@@ -25,7 +24,6 @@ You need a [MrScraper account](https://app.mrscraper.com) and an API token for t
25
24
The API token authenticates requests. Plan tokens measure MrScraper usage; they are a separate balance. Calls use your MrScraper account's quota and limits.
26
25
{/* MANUAL-CONTENT-END */}
27
26
28
-
29
27
## Use the block [#usage-instructions]
30
28
31
29
{/* MANUAL-CONTENT-START:usage */}
@@ -59,10 +57,12 @@ Retrieve web pages as HTML, Markdown, or screenshots with browser rendering and
59
57
60
58
## Actions
61
59
62
-
### mrscraper_scrape
60
+
### MrScraper Scrape
63
61
64
62
Retrieve a web page as HTML, Markdown, or a screenshot with automatic browser fallback, optional real-device routing, geolocation, and retry controls.
65
63
64
+
<ReferenceDetails>
65
+
66
66
#### Input
67
67
68
68
| Parameter | Type | Required | Description |
@@ -71,14 +71,14 @@ Retrieve a web page as HTML, Markdown, or a screenshot with automatic browser fa
71
71
|`url`| string | Yes | Absolute HTTP or HTTPS URL of the page to retrieve. |
72
72
|`html`| boolean | No | Include HTML content. Defaults to true. |
73
73
|`markdown`| boolean | No | Include Markdown content. |
74
-
|`screenshot`| boolean | No | Capture a JPEG screenshot as a stored file. Always enables browser rendering. |
74
+
|`screenshot`| boolean | No | Capture a full-page screenshot as a stored file. Always enables browser rendering. |
75
75
|`browserRendering`| string | No | auto uses automatic browser fallback; true forces a browser; false disables it. |
76
76
|`super`| boolean | No | Use real-device routing independently of browser rendering. |
77
77
|`geoCode`| string | No | Proxy country code, such as US, GB, ID, or SG. Defaults to US. |
78
78
|`proxyCountry`| string | No | Compatibility country option sent as proxyCountry. |
79
79
|`timeout`| number | No | Page timeout in seconds, from 1 to 300. Defaults to 120. |
80
-
|`maxRetries`| number | No | Maximum retry count as a nonnegative integer. Zero disables retries. Defaults to 3. |
81
-
|`tokenCap`| number | No |Nonnegative retry-token budget. The initial request always runs. |
80
+
|`maxRetries`| number | No | Maximum retry count for Markdown requests. Zero disables retries. Defaults to 3. |
81
+
|`tokenCap`| number | No |Token budget, at least 1, for a Markdown request and its retries. The initial request always runs. |
82
82
|`waitForSelector`| string | No | CSS selector to wait for when browser rendering is enabled. |
83
83
|`blockResources`| boolean | No | Block images, CSS, and fonts to reduce bandwidth. |
84
84
|`homePage`| boolean | No | Visit the site root before loading the target URL. |
@@ -94,17 +94,21 @@ Retrieve a web page as HTML, Markdown, or a screenshot with automatic browser fa
94
94
| --------- | ---- | ----------- |
95
95
|`html`| string | Page HTML, when requested. |
96
96
|`markdown`| string | Page Markdown, when requested. |
97
-
|`screenshot`| file |JPEG screenshot stored in execution files. |
97
+
|`screenshot`| file |Page screenshot\(PNG or JPEG\) stored in execution files. |
98
98
|`tokenUsage`| number | Plan tokens consumed by the page request. |
99
99
|`runtime`| number | Page request runtime in seconds. |
100
100
|`retryCount`| number | Number of retries performed. |
101
101
|`targetStatusCode`| number | Target page HTTP status from x-status-code. |
102
102
|`error`| string | Page retrieval error, when reported by MrScraper. |
103
103
104
-
### mrscraper_search
104
+
</ReferenceDetails>
105
+
106
+
### MrScraper Search
105
107
106
108
Search Google from query text or a Google search URL and return organic results, search metadata, pagination, or the raw results-page HTML.
107
109
110
+
<ReferenceDetails>
111
+
108
112
#### Input
109
113
110
114
| Parameter | Type | Required | Description |
@@ -149,7 +153,9 @@ Search Google from query text or a Google search URL and return organic results,
149
153
| ↳ `link`| string | Google result-page URL. |
150
154
|`tokenUsage`| number | Plan tokens consumed by the search request. |
@@ -183,17 +189,17 @@ Check account identity, subscription status, token usage, remaining quota, and r
183
189
184
190
The action tables above describe Sim's outputs. For scraping, `html` and `markdown` contain the requested page content. When requested, `screenshot` is a stored file that you can pass to a later file-compatible block. Request it only when you need an image; screenshots can be large.
185
191
186
-
Check `targetStatusCode`, `error`, and the returned content before processing a scrape. The target page can return an error or a challenge page even when MrScraper accepted the request. `tokenUsage`, `runtime`, and `retryCount` help explain how the request completed. Format and site behavior determine which content fields are populated.
192
+
Check `targetStatusCode`, `error`, and the returned content before processing a scrape. The target page can return an error or a challenge page even when MrScraper accepted the request. `tokenUsage`, `runtime`, and `retryCount` help explain how the request completed. An HTML-only request uses MrScraper's fetch endpoint, which reports `tokenUsage` but leaves `runtime`, `retryCount`, and `targetStatusCode` empty; enable Markdown to get all of them.
187
193
188
194
For JSON search results, use `searchResults` for organic results, `general` for query metadata, and `pagination` for available result pages. An empty results array is a valid outcome. For HTML search, use the `html` output. These are search results, not a guarantee that each linked page is accessible.
189
195
190
196
The account action returns a selected summary. API tokens and billing identifiers from the provider's account record are excluded.
191
197
192
198
## Control time and token usage
193
199
194
-
Scrape `timeout` is a page-loading timeout in seconds, from `1` to `300`, with a default of `120`. It is separate from workflow execution limits. `maxRetries` defaults to `3`; set it to `0` to disable retries.
200
+
Scrape `timeout` is a page-loading timeout in seconds, from `1` to `300`, with a default of `120`. It is separate from workflow execution limits. `maxRetries`and `tokenCap` apply to Markdown requests; the HTML-only fetch endpoint retries on its own. `maxRetries`defaults to `3`; set it to `0` to disable retries.
195
201
196
-
`tokenCap` stops further retries when the running token total reaches the cap. The first attempt still runs even if it exceeds the cap, so this setting does not guarantee a maximum charge. Web fetching uses runtime and bandwidth: one plan token per 30 seconds and one per 0.25 MB, rounded up for each component. See MrScraper's [Token Plan](https://docs.mrscraper.com/docs/getting-started/api-token) for the maintained calculation.
202
+
`tokenCap`must be at least `1` and stops further retries when the running token total reaches the cap. The first attempt still runs even if it exceeds the cap, so this setting does not guarantee a maximum charge. Web fetching uses runtime and bandwidth: one plan token per 30 seconds and one per 0.25 MB, rounded up for each component. See MrScraper's [Token Plan](https://docs.mrscraper.com/docs/getting-started/api-token) for the maintained calculation.
197
203
198
204
The combined API response must fit Sim's 10 MiB tool response limit, including the screenshot's base64 representation before it is stored as a file. If the response is too large, request fewer formats or omit the screenshot.
0 commit comments