Scrape

​

Description

This endpoint allows users to scrape data from a specified URL, format the scraped data, and optionally clean it before returning the result.

​

Endpoint

POST /api/v1/scrape

​

Headers

• Content-Type: application/json
• Authorization: Bearer <API_KEY> (required)

​

Request Body

{
  "url": "string", // Required. The URL to scrape.
  "format": "string", // Optional. The format of the output. Valid values: "markdown", "html", "screenshot".
  "cleaned": "boolean", // Optional. Whether the output should be cleaned.
  "renderJs": "boolean" // Optional. Whether to render JavaScript before scraping. Default is true.
}

​

Responses

​

Success (200)

Returns the scraped data in the specified format.

{
  "title": "string",
  "metadata": "object",
  "url": "string",
  "format": "string", // "markdown", "html", "screenshot"
  "cleaned": "boolean",
  "content": "string"
}

• Content-Type: application/json
• X-RateLimit-Limit: The rate limit for the user.
• X-RateLimit-Remaining: The remaining number of requests for the user.

​

Example Request

curl -X POST https://app.dumplingai.com/api/v1/scrape \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -d '{
    "url": "https://example.com",
    "format": "markdown",
    "cleaned": true,
    "renderJs": true
  }'

​

Rate Limiting

Rate limit headers (X-RateLimit-Limit and X-RateLimit-Remaining) are included in the response to indicate the user’s current rate limit status.

​

Notes

• This endpoint uses 1 credits per request.
• Disable JavaScript rendering by setting renderJs to false for faster results if you don’t need it.