Sitemap Crawler

Map every page of a website, inspect the results as JSON, then download the full site as a Markdown archive that mirrors the URL structure — great for offline or retrieval-augmented workflows. Available at chat-lab.ai/sitemapper.

How it works

Mapping: your URL is sent to Firecrawl's /map endpoint (v2), which discovers the site's pages (via sitemap and/or crawling) and returns a JSON list of links. The raw JSON is displayed for copying.
Fetching (optional): click Process & Download to fetch each mapped page as Markdown via the Jina Reader API, rate-limit paced, with per-page progress.
Packaging: pages are assembled in memory into a ZIP whose folder structure mirrors the site (example.com/docs/intro.md, example.com/blog/index.md, …).

User options

Firecrawl API key (optional): the app's free Firecrawl key has usage limits; supply your own key (the student program has a generous free tier) to avoid them. Session-only, never stored.
Sitemap mode: include (sitemap + discovery, default), skip (ignore sitemap), or only (sitemap URLs only).
Include subdomains: whether to map beyond the root domain.
Max links: cap the number of pages mapped (default 500).
Search filter: e.g. blog to return only URLs relevant to that term.

Data handling

The app crawls only the site you request. Mapping runs through Firecrawl; page fetches run through Jina Reader. Results are built in memory and exist only in your browser session; nothing is written to server disk. See Privacy & Data Retention.

Crawl responsibly

Only map and download sites you have the right to archive, and respect the target site's terms of service.