Skip to content

API Reference

Crawlee Cloud provides a REST API that is fully compatible with the Apify API v2.

Base URL

https://your-server.com/v2

Authentication

All requests require a Bearer token in the Authorization header:

bash
curl -H "Authorization: Bearer YOUR_TOKEN" \
  https://your-server.com/v2/datasets

Datasets

Store and retrieve scraped data.

MethodEndpointDescription
GET/v2/datasetsList all datasets
POST/v2/datasetsCreate a new dataset
GET/v2/datasets/{id}Get dataset details
DELETE/v2/datasets/{id}Delete a dataset
POST/v2/datasets/{id}/itemsPush items to dataset
GET/v2/datasets/{id}/itemsRetrieve items

Push Items

bash
curl -X POST \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '[{"title": "Item 1"}, {"title": "Item 2"}]' \
  https://your-server.com/v2/datasets/{id}/items

Key-Value Stores

Store arbitrary data by key.

MethodEndpointDescription
GET/v2/key-value-storesList all stores
POST/v2/key-value-storesCreate a new store
GET/v2/key-value-stores/{id}Get store details
PUT/v2/key-value-stores/{id}/records/{key}Set a record
GET/v2/key-value-stores/{id}/records/{key}Get a record
DELETE/v2/key-value-stores/{id}/records/{key}Delete a record

Common Keys

  • INPUT — Actor input configuration
  • OUTPUT — Actor output/results

Request Queues

Manage URLs to crawl with automatic deduplication.

MethodEndpointDescription
GET/v2/request-queuesList all queues
POST/v2/request-queuesCreate a new queue
POST/v2/request-queues/{id}/requestsAdd requests to queue
POST/v2/request-queues/{id}/head/lockLock and fetch requests
DELETE/v2/request-queues/{id}/requests/{requestId}/lockRelease a lock
PUT/v2/request-queues/{id}/requests/{requestId}Update request status

Deduplication

Requests are deduplicated by uniqueKey. Adding a request with an existing uniqueKey is a no-op.


Actors

Manage Actor definitions.

MethodEndpointDescription
GET/v2/actsList all Actors
POST/v2/actsCreate an Actor
GET/v2/acts/{id}Get Actor details
PUT/v2/acts/{id}Update an Actor
DELETE/v2/acts/{id}Delete an Actor
POST/v2/acts/{id}/runsStart a new run

Runs

Monitor Actor executions.

MethodEndpointDescription
GET/v2/actor-runsList all runs
GET/v2/actor-runs/{id}Get run status
POST/v2/actor-runs/{id}/abortAbort a running Actor
GET/v2/actor-runs/{id}/logGet run logs

Run Status Values

StatusDescription
READYQueued, waiting to start
RUNNINGCurrently executing
SUCCEEDEDCompleted successfully
FAILEDExecution failed
ABORTEDManually stopped
TIMED-OUTExceeded time limit

Response Format

All successful responses wrap data in a data field:

json
{
  "data": {
    "id": "abc123",
    "name": "my-dataset",
    "itemCount": 42
  }
}

Error Responses

json
{
  "error": {
    "type": "RECORD_NOT_FOUND",
    "message": "Dataset with ID 'xyz' was not found"
  }
}
HTTP CodeDescription
400Bad request / validation error
401Authentication required
403Permission denied
404Resource not found
500Internal server error

Released under the MIT License.