Back to skills
SkillHub ClubWrite Technical DocsFull StackData / AITech Writer

crawl

Crawl any website and save pages as local markdown files. Use when you need to download documentation, knowledge bases, or web content for offline access or analysis. No code required - just provide a URL.

Packaged view

This page reorganizes the original catalog entry around fit, installability, and workflow context first. The original raw source lives below.

Stars
105
Hot score
94
Updated
March 20, 2026
Overall rating
C4.0
Composite score
4.0
Best-practice grade
A88.4

Install command

npx @skill-hub/cli install tavily-ai-skills-crawl
web-crawlingdata-extractiondocumentationautomationoffline-access

Repository

tavily-ai/skills

Skill path: skills/tavily/crawl

Crawl any website and save pages as local markdown files. Use when you need to download documentation, knowledge bases, or web content for offline access or analysis. No code required - just provide a URL.

Open repository

Best for

Primary workflow: Write Technical Docs.

Technical facets: Full Stack, Data / AI, Tech Writer.

Target audience: Development teams looking for install-ready agent workflows..

License: Unknown.

Original source

Catalog source: SkillHub Club.

Repository owner: tavily-ai.

This is still a mirrored public skill entry. Review the repository before installing into production workflows.

What it helps with

  • Install crawl into Claude Code, Codex CLI, Gemini CLI, or OpenCode workflows
  • Review https://github.com/tavily-ai/skills before adding crawl to shared team environments
  • Use crawl for development workflows

Works across

Claude CodeCodex CLIGemini CLIOpenCode

Favorites: 0.

Sub-skills: 0.

Aggregator: No.

crawl | SkillHub