Firecrawl

Overview

This doc outlines how to use the Firecrawl connector to scrape data from any URL.

Setup

Create connector
From the settings panel or from the node catalog, search for the Firecrawl integration and navigate to the 'create connector' view.

Retrieve your API key
Navigate to your Firecrawl account, find the API key section on the left panel, and copy your API key.

Setup step 1

Configure the connector
Create a name and a slug for your connector. Then, paste your Firecrawl API key into the dedicated field.

Actions

Scrape - Scrape data from a URL
Enter the URL of the website you want to scrape. Firecrawl will return the MDX content found on the page.

Crawl - Scrape data from a URL
Enter the URL of the website you want to crawl. The user can identify the maximum depth to crawl relative to an entered URL as well as inclusion and exclusion terms.

FAQ

Which rate limits should I set for this connector?
Refer to the Firecrawl API documentation for rate limits: Firecrawl API rate limits