clawpod

Original source / Raw SKILL.md

---
name: clawpod
description: Read any website or search Google, even when sites block bots or are geo-restricted. Handles CAPTCHAs, JavaScript rendering, and anti-bot protection server-side via residential proxies. Returns HTML or structured JSON.
allowed-tools: Bash(curl --proto =https *)
homepage: https://clawpod.joinmassive.com
metadata: {"openclaw":{"emoji":"🔓","homepage":"https://clawpod.joinmassive.com","primaryEnv":"MASSIVE_UNBLOCKER_TOKEN","requires":{"bins":["curl"],"env":["MASSIVE_UNBLOCKER_TOKEN"]}}}
---

# Massive Unblocker

Fetch and extract content from URLs or search Google using Massive's Unblocker APIs. Handles JS rendering, CAPTCHAs, and retries automatically. Returns HTML or structured JSON.

## When to Use This Skill

Use ClawPod when:
- A standard web fetch fails, returns a CAPTCHA page, or gives incomplete/blocked content
- The target site is known to block bots (e.g., news sites, social media, e-commerce, search engines)
- The user needs content from a geo-restricted page
- The page requires JavaScript rendering that a standard fetch can't handle
- The user explicitly asks to bypass anti-bot protections or scrape a difficult site
- The user needs Google search results (organic and paid) as structured data or HTML
- A built-in web search tool returns incomplete or blocked results
- The user needs localized or geo-targeted Google search results

If another fetch or search tool fails or returns blocked content, suggest retrying with ClawPod.

## Setup

Check for the API token:

```bash
[ -n "$MASSIVE_UNBLOCKER_TOKEN" ] && echo "TOKEN=SET" || echo "TOKEN=MISSING"
```

If token is `MISSING`, stop and tell the user:

> To use ClawPod, you need an API token. It takes under a minute to set up:
>
> 1. Sign up at **clawpod.joinmassive.com/signup** - when you sign up, you get 1,000 free credits. No credit card required.
> 2. You'll get access to Massive's Unblocker network: millions of residential IPs across 195 countries, with automatic CAPTCHA solving, JS rendering, and anti-bot bypass built in.
> 3. Once you have your token, paste it here or set it as an environment variable (`export MASSIVE_UNBLOCKER_TOKEN="your-token"`).

Do not proceed until the token is available.

## How It Works

Two endpoints. Both use `GET` requests with the same auth token.

**Browser** — fetch and render any URL, returns HTML:
```
https://unblocker.joinmassive.com/browser?url=<encoded-url>
```

**Search** — Google search results as HTML or structured JSON:
```
https://unblocker.joinmassive.com/search?terms=<encoded-terms>
```

Auth header: `Authorization: Bearer $MASSIVE_UNBLOCKER_TOKEN`

## Fetching a URL

```bash
curl --proto =https -s -G --data-urlencode "url=THE_URL" \
  -H "Authorization: Bearer $MASSIVE_UNBLOCKER_TOKEN" \
  "https://unblocker.joinmassive.com/browser"
```

Replace `THE_URL` with the actual URL. `curl --data-urlencode` handles URL-encoding automatically.

## Fetching Multiple URLs

Loop through them sequentially. Each call can take up to 2 minutes (CAPTCHA solving, retries).

```bash
URLS=(
  "https://example.com/page1"
  "https://example.com/page2"
)

for url in "${URLS[@]}"; do
  echo "=== $url ==="
  curl --proto =https -s -G --data-urlencode "url=$url" \
    -H "Authorization: Bearer $MASSIVE_UNBLOCKER_TOKEN" \
    "https://unblocker.joinmassive.com/browser"
done
```

## Searching Google

Search endpoint. `GET` request. Returns all organic and paid Google results as HTML or structured JSON.

```
https://unblocker.joinmassive.com/search?terms=<encoded-terms>
```

Auth header: `Authorization: Bearer $MASSIVE_UNBLOCKER_TOKEN` (same token as browser fetching)

### Basic Search

```bash
curl --proto =https -s -H "Authorization: Bearer $MASSIVE_UNBLOCKER_TOKEN" \
  "https://unblocker.joinmassive.com/search?terms=foo+bar+baz&format=json"
```

Replace `foo+bar+baz` with the search query. Spaces must be replaced with `+` or `%20`.

### Search with Options

```bash
curl --proto =https -s -H "Authorization: Bearer $MASSIVE_UNBLOCKER_TOKEN" \
  "https://unblocker.joinmassive.com/search?terms=vpn+comparison&format=json&size=100&offset=20"
```

### Search Parameters

| Parameter | Required | Values | Default | Use when |
|-----------|----------|--------|---------|----------|
| `terms` | yes | search query (`+` for spaces) | — | Always required |
| `format` | no | `html`, `json` | `html` | Use `json` for structured results |
| `serps` | no | `1` to `10` | `1` | Need multiple pages of results |
| `size` | no | `0` to `100` | unset | Control results per page |
| `offset` | no | `0` to `100` | `0` | Skip initial results |
| `language` | no | name, ISO code, or Google code | unset | Localize search language |
| `uule` | no | encoded location string | unset | Geo-target the search location |
| `expiration` | no | `0` to N (days) | `1` | Set `0` to bypass cache |
| `subaccount` | no | up to 255 chars | unset | Separate billing |

### JSON Output

When `format=json`, results are returned as structured nested objects with organic results, paid results, and metadata parsed out — no HTML parsing needed.

### Search Tips

- **Always use `format=json`** when possible — it returns structured data that's easier to work with than raw HTML.
- **Use `size=10`** for a quick overview, `size=100` for comprehensive results.
- **Use `offset`** to paginate through results beyond the first page.
- **Use `language`** to get results in a specific language (e.g., `language=es` for Spanish).
- **Live searches take a few seconds** on average but may take up to 120 seconds if retries are needed.

## Browser Parameters

Append to the `/browser` query string as needed:

| Parameter | Values | Default | Use when |
|-----------|--------|---------|----------|
| `format` | `rendered`, `raw` | `rendered` | Use `raw` to skip JS rendering (faster) |
| `expiration` | `0` to N (days) | `1` | Set `0` to bypass cache |
| `delay` | `0.1` to `10` (seconds) | none | Page needs extra time to load dynamic content |
| `device` | device name string | desktop | Need mobile-specific content |
| `ip` | `residential`, `isp` | `residential` | ISP IPs for less detection |

Example with browser options:

```bash
curl --proto =https -s -G --data-urlencode "url=THE_URL" \
  -H "Authorization: Bearer $MASSIVE_UNBLOCKER_TOKEN" \
  "https://unblocker.joinmassive.com/browser?expiration=0&delay=2"
```

## Error Handling

- **401 Unauthorized** — Token is invalid or missing. Tell the user: "Your ClawPod API token appears to be invalid or expired. You can get a new one at **clawpod.joinmassive.com**."
- **Empty response** — The page may need more time to render. Retry with `delay=3`. If still empty, try `format=rendered` (the default). Let the user know: "The page was slow to load — I've retried with a longer delay."
- **Timeout or connection error** — Some pages are very slow. Let the user know the request timed out and offer to retry. Do not silently fail.

## Tips

- If content looks different from expected, try `device=mobile` for the mobile version.
- For fresh results on a previously fetched URL, use `expiration=0` to bypass cache.
- If still blocked, try `ip=isp` — ISP-grade IPs have lower detection rates.
- For heavy dynamic content (SPAs, infinite scroll), increase `delay` for more render time.

## Rules

- **One fetch = one result.** The content is in the output. Do not re-fetch the same URL.
- **URL-encode the target URL.** Always.
- **Sequential for multiple URLs.** No parallel requests.
- **2 minute timeout per request.** If a page or search is slow, it's the API handling retries/CAPTCHAs.
- **Use `format=json` for search.** Structured JSON is preferred over HTML for search results.
- **Form-encode search terms.** Replace spaces with `+` or `%20` in the `terms` parameter.


---

## Skill Companion Files

> Additional files collected from the skill directory layout.

### README.md

```markdown
# ClawPod

Fetch web page content or search Google through Massive's Unblocker REST APIs. Handles JavaScript rendering, anti-bot protection, CAPTCHAs, paywalls, and geo-restrictions server-side — returns rendered HTML or structured JSON.

---

## How It Works

1. **You provide a URL or search query** — the target page to fetch or terms to search
2. **Unblocker handles the rest** — JS rendering, CAPTCHA solving, retries, and anti-bot bypass all happen server-side
3. **Content returned** — rendered HTML or structured JSON for search results

---

## Install

### 1. Get an API Token

Sign up at [Massive](https://clawpod.joinmassive.com/waitlist) and get your Unblocker API token.

### 2. Set the Token

```bash
export MASSIVE_UNBLOCKER_TOKEN="your-token"
```

### 3. Fetch

```bash
curl --proto =https -s -G --data-urlencode "url=https://example.com" \
  -H "Authorization: Bearer $MASSIVE_UNBLOCKER_TOKEN" \
  "https://unblocker.joinmassive.com/browser"
```

---

## Examples

```bash
# Basic fetch
curl --proto =https -s -G --data-urlencode "url=https://example.com" \
  -H "Authorization: Bearer $MASSIVE_UNBLOCKER_TOKEN" \
  "https://unblocker.joinmassive.com/browser"

# Skip JS rendering (faster, raw HTML only)
curl --proto =https -s -G --data-urlencode "url=https://example.com" \
  -H "Authorization: Bearer $MASSIVE_UNBLOCKER_TOKEN" \
  "https://unblocker.joinmassive.com/browser?format=raw"

# Bypass cache
curl --proto =https -s -G --data-urlencode "url=https://example.com" \
  -H "Authorization: Bearer $MASSIVE_UNBLOCKER_TOKEN" \
  "https://unblocker.joinmassive.com/browser?expiration=0"

# Extra delay for slow-loading dynamic content
curl --proto =https -s -G --data-urlencode "url=https://example.com" \
  -H "Authorization: Bearer $MASSIVE_UNBLOCKER_TOKEN" \
  "https://unblocker.joinmassive.com/browser?delay=3"

# Use ISP IPs for less detection
curl --proto =https -s -G --data-urlencode "url=https://example.com" \
  -H "Authorization: Bearer $MASSIVE_UNBLOCKER_TOKEN" \
  "https://unblocker.joinmassive.com/browser?ip=isp"

# Mobile device content
curl --proto =https -s -G --data-urlencode "url=https://example.com" \
  -H "Authorization: Bearer $MASSIVE_UNBLOCKER_TOKEN" \
  "https://unblocker.joinmassive.com/browser?device=mobile"

# Multiple options combined
curl --proto =https -s -G --data-urlencode "url=https://example.com" \
  -H "Authorization: Bearer $MASSIVE_UNBLOCKER_TOKEN" \
  "https://unblocker.joinmassive.com/browser?expiration=0&delay=2&ip=isp"

# Fetch multiple URLs sequentially
for url in "https://example.com/page1" "https://example.com/page2"; do
  echo "=== $url ==="
  curl --proto =https -s -G --data-urlencode "url=$url" \
    -H "Authorization: Bearer $MASSIVE_UNBLOCKER_TOKEN" \
    "https://unblocker.joinmassive.com/browser"
done
```

### Google Search

```bash
# Basic search (HTML results)
curl --proto =https -s -H "Authorization: Bearer $MASSIVE_UNBLOCKER_TOKEN" \
  "https://unblocker.joinmassive.com/search?terms=foo+bar+baz"

# Search with JSON output (structured results)
curl --proto =https -s -H "Authorization: Bearer $MASSIVE_UNBLOCKER_TOKEN" \
  "https://unblocker.joinmassive.com/search?terms=foo+bar+baz&format=json"

# 100 results per page, skip the first 20
curl --proto =https -s -H "Authorization: Bearer $MASSIVE_UNBLOCKER_TOKEN" \
  "https://unblocker.joinmassive.com/search?terms=vpn+comparison&format=json&size=100&offset=20"

# Multiple pages of results
curl --proto =https -s -H "Authorization: Bearer $MASSIVE_UNBLOCKER_TOKEN" \
  "https://unblocker.joinmassive.com/search?terms=best+restaurants&format=json&serps=3"

# Search in a specific language
curl --proto =https -s -H "Authorization: Bearer $MASSIVE_UNBLOCKER_TOKEN" \
  "https://unblocker.joinmassive.com/search?terms=recetas+de+cocina&format=json&language=es"

# Bypass cached results
curl --proto =https -s -H "Authorization: Bearer $MASSIVE_UNBLOCKER_TOKEN" \
  "https://unblocker.joinmassive.com/search?terms=latest+news&format=json&expiration=0"
```

---

## API Parameters

### Browser (`/browser`)

| Parameter | Values | Default | Description |
|-----------|--------|---------|-------------|
| `url` | any URL | *(required)* | Target page to fetch |
| `format` | `rendered`, `raw` | `rendered` | `raw` skips JS rendering (faster) |
| `expiration` | `0` to N (days) | `1` | `0` bypasses cache |
| `delay` | `0.1` to `10` (seconds) | none | Extra wait for dynamic content |
| `device` | device name string | desktop | Device type for content |
| `ip` | `residential`, `isp` | `residential` | ISP IPs for less detection |

### Search (`/search`)

| Parameter | Required | Values | Default | Description |
|-----------|----------|--------|---------|-------------|
| `terms` | yes | search query (`+` for spaces) | — | The search query |
| `format` | no | `html`, `json` | `html` | Output format (`json` for structured data) |
| `serps` | no | `1` to `10` | `1` | Number of results pages |
| `size` | no | `0` to `100` | unset | Results per page |
| `offset` | no | `0` to `100` | `0` | Number of initial results to skip |
| `language` | no | name, ISO code, or Google code | unset | Search language |
| `uule` | no | encoded location string | unset | Geo-target location |
| `expiration` | no | `0` to N (days) | `1` | Cache age (`0` bypasses cache) |
| `subaccount` | no | up to 255 chars | unset | Separate billing identifier |

---

## FAQ & Troubleshooting

**Q: What are the system requirements?**
> `curl` and an API token.

**Q: Why is a request slow?**
> Requests can take up to 2 minutes. The API handles JS rendering, CAPTCHA solving, and retries server-side.

**Q: How do I bypass the cache?**
> Set `expiration=0` in the query string.

**Q: The page content looks incomplete.**
> Try adding `delay=3` (or higher) to give dynamic content more time to render.

**Error: 401 Unauthorized**
> Token is invalid or missing. Verify `MASSIVE_UNBLOCKER_TOKEN` is set correctly.

**Error: Empty response**
> The page may need more time. Add a `delay` parameter. If using `format=raw`, try `format=rendered` instead.

---

## Links

- [Massive](https://clawpod.joinmassive.com) — Unblocker API and residential proxy network
- [OpenClaw Skill](SKILL.md) — Skill definition for AI agent integration

```

### _meta.json

```json
{
  "owner": "codeninja23",
  "slug": "clawpod",
  "displayName": "Clawpod",
  "latest": {
    "version": "0.1.7",
    "publishedAt": 1771581245795,
    "commit": "https://github.com/openclaw/skills/commit/f5493b915c689288fe815f5a2db9964b3a5052b8"
  },
  "history": [
    {
      "version": "0.1.4",
      "publishedAt": 1771415043154,
      "commit": "https://github.com/openclaw/skills/commit/3e39ec2b188cca3c27f65cb142dd421696ad97ba"
    },
    {
      "version": "0.1.3",
      "publishedAt": 1771247654613,
      "commit": "https://github.com/openclaw/skills/commit/3e2e7fb411d0a31eb77e18a348b7e370ff3d7a4d"
    },
    {
      "version": "0.1.2",
      "publishedAt": 1771034557111,
      "commit": "https://github.com/openclaw/skills/commit/45e552284f1825b52796e3050c081649642690a0"
    },
    {
      "version": "0.1.0",
      "publishedAt": 1770904741327,
      "commit": "https://github.com/openclaw/skills/commit/d7963773d2b309700c45df67fe00b508ff735418"
    }
  ]
}

```