Firecrawl

What is the Firecrawl Connection?

The Firecrawl connection connects Firecrawl's web crawling and scraping capabilities to Duvo, enabling your assignments to extract structured data from websites at scale. This is a user-configured connection -- you will need to connect your Firecrawl account before your assignments can use it.

What Can It Do?

The Firecrawl connection provides actions that allow your assignments to:

• Crawl websites: Navigate through multiple pages of a website to gather content systematically

• Scrape web pages: Extract text, data, and structured content from individual pages

• Extract structured data: Pull specific data points from web pages in an organized format

Why This Matters

Many business workflows depend on data that lives on external websites -- competitor pricing, product listings, public records, or content from partner portals. The Firecrawl connection allows your assignments to gather this information automatically and at scale, replacing manual copy-paste work or custom scraping scripts with a reliable, integrated solution.

When to Use It

Use the Firecrawl connection when your assignment needs to:

• Monitor competitor pricing: Crawl competitor websites to track price changes and product availability

• Gather market intelligence: Scrape industry publications, directories, or listing sites for business insights

• Extract product data: Pull product details, specifications, or reviews from e-commerce sites

• Collect public records: Gather publicly available data from government sites, directories, or databases

• Aggregate content: Compile information from multiple web pages into a single structured dataset

How It Works

After connecting your Firecrawl account, your assignments can crawl and scrape websites using Firecrawl's infrastructure. When you include Firecrawl actions in your assignment's SOP, it will visit web pages, extract the content you specify, and return structured data that your assignment can then process, analyze, or forward to other systems.

Key Benefits

• Scale web data collection: Gather data from many pages without manual browsing or copy-pasting

• Structured output: Get clean, organized data instead of raw HTML

• Reliable extraction: Purpose-built infrastructure handles dynamic pages and complex site structures

• Integrated into workflows: Feed scraped data directly into downstream steps like analysis, reporting, or notifications

Works Well With

• Google Sheets or Microsoft Excel: Scrape web data and write it directly into spreadsheets for tracking or analysis

• Gmail or Microsoft Outlook: Crawl websites for updates and send summary emails when changes are detected

• Slack: Scrape competitor or market data and post automated alerts to relevant team channels