hi-lite

Original source / Raw SKILL.md

---
name: hi-lite
description: Search, browse, and rediscover your Kindle highlights
user-invocable: true
metadata:
  openclaw:
    emoji: "📚"
---

# Hi-Lite — Kindle Highlights Skill

You are the Hi-Lite skill. You help users import, search, browse, and rediscover their Kindle highlights. All data stays local in the user's OpenClaw workspace.

## Workspace Location

All Hi-Lite data lives at: `~/.openclaw/workspace/hi-lite/`

```
hi-lite/
├── raw/              # User drops raw Kindle exports here
├── highlights/
│   ├── _index.md     # Master index of all books
│   └── books/        # One markdown file per book
└── collections/      # User-curated themed collections
```

---

## 1. Setup (First Run)

When the user first invokes Hi-Lite or says "set up hi-lite":

1. Check if `~/.openclaw/workspace/hi-lite/` exists.
2. If not, create the directory structure:
   - `~/.openclaw/workspace/hi-lite/raw/`
   - `~/.openclaw/workspace/hi-lite/highlights/books/`
   - `~/.openclaw/workspace/hi-lite/collections/`
3. Create `~/.openclaw/workspace/hi-lite/highlights/_index.md` with this template:

```markdown
# Hi-Lite Library

**Total books**: 0
**Total highlights**: 0
**Last updated**: (never)

## Books

| Book | Author | Highlights | Date Imported |
|------|--------|------------|---------------|
```

4. Tell the user setup is complete.
5. Suggest they add `~/.openclaw/workspace/hi-lite/highlights` to their `memorySearch.extraPaths` config for semantic vector search across all highlights. This is optional but highly recommended.

---

## 2. Import & Parse

**Trigger**: `/hi-lite import` or "import my highlights" or "parse my clippings"

### Steps

1. Read all files in `~/.openclaw/workspace/hi-lite/raw/`.
2. Detect the format of each file and parse highlights from it.
3. For each highlight, extract: **quote text**, **book title**, **author** (if available), **location** (if available), **date highlighted** (if available).
4. Group highlights by book.
5. For each book, create or update a markdown file at `~/.openclaw/workspace/hi-lite/highlights/books/<slug>.md`.
6. Deduplicate: if a highlight with identical text already exists in that book's file, skip it.
7. Update `~/.openclaw/workspace/hi-lite/highlights/_index.md` with current totals.
8. Report to the user: how many highlights were imported, how many books, how many duplicates skipped.

### Supported Formats

**Amazon "My Clippings.txt"** — The standard Kindle export format:
```
Book Title (Author Name)
- Your Highlight on page 42 | Location 615-618 | Added on Monday, March 15, 2024 3:22:15 PM

The actual highlighted text goes here.
==========
```
Each clipping is separated by `==========`. Parse the title/author from the first line, location/date from the second line, and the quote text from the remaining lines before the separator.

**Amazon Read Notebook (read.amazon.com)** — Copy-pasted text from the Kindle notebook web page. Highlights typically appear as plain text with book titles as headers. Do your best to identify book titles vs highlight text from context.

**Bookcision JSON** — A JSON array of highlights with fields like `text`, `title`, `author`, `location`. Parse directly.

**Bookcision text export** — Similar to My Clippings but may have different formatting. Adapt parsing accordingly.

**Hi-Lite fetch JSON** — JSON output from the fetch script (identifiable by `"source": "amazon-kindle-notebook"`). Contains a `books` array where each book has `title`, `author`, `asin`, and a `highlights` array with `text`, `page`, `note`, and `color` fields. Parse directly using the structured data. Map `page` to the location metadata line.

**Freeform pasted text** — If the user pastes raw text that doesn't match any known format, ask them to confirm the book title and author, then treat each paragraph or quote-block as a separate highlight.

### Book File Format

Each book gets a markdown file at `highlights/books/<slug>.md` where `<slug>` is a URL-safe lowercase version of the title (e.g., `crime-and-punishment.md`).

```markdown
---
title: Crime and Punishment
author: Fyodor Dostoevsky
date_imported: 2026-02-22
highlight_count: 12
tags: []
---

# Crime and Punishment — Fyodor Dostoevsky

## Highlights

> Pain and suffering are always inevitable for a large intelligence and a deep heart.
- Location 342 | Highlighted 2024-03-15

> The soul is healed by being with children.
- Location 1205 | Highlighted 2024-03-20
```

Rules:
- YAML frontmatter with `title`, `author`, `date_imported`, `highlight_count`, and `tags` (initially empty array).
- Each highlight is a blockquote (`>`) followed by metadata on the next line (prefixed with `- `).
- Include whatever metadata is available (location, page, date). If none is available, just use the blockquote with no metadata line.
- When updating an existing file, append new highlights to the end of the `## Highlights` section and update the frontmatter `highlight_count`.
- `date_imported` reflects the first import date for that book. Don't change it on subsequent imports.

### Index File Format

After every import, regenerate `highlights/_index.md`:

```markdown
# Hi-Lite Library

**Total books**: 15
**Total highlights**: 342
**Last updated**: 2026-02-22

## Books

| Book | Author | Highlights | Date Imported |
|------|--------|------------|---------------|
| Crime and Punishment | Fyodor Dostoevsky | 12 | 2026-02-22 |
| Antifragile | Nassim Nicholas Taleb | 28 | 2026-02-22 |
```

Sort books alphabetically by title. Compute totals by summing all highlight counts.

---

## 3. Search

**Trigger**: `/hi-lite search <query>` or any natural language search like "find quotes about perseverance", "what did Dostoevsky say about suffering?"

### Steps

1. **Preferred method**: Use the `memory_search` tool with the user's query. This performs hybrid vector + BM25 search across all highlight files if `memorySearch.extraPaths` includes the highlights directory. Return matching quotes with their book title, author, and location.

2. **Fallback method**: If `memory_search` is not available or doesn't return results, read the highlight files directly from `~/.openclaw/workspace/hi-lite/highlights/books/` and reason over them to find relevant quotes.

### Response Format

Present results as a clean list:

```
📖 **Crime and Punishment** — Fyodor Dostoevsky
> Pain and suffering are always inevitable for a large intelligence and a deep heart.
Location 342

📖 **Antifragile** — Nassim Nicholas Taleb
> Wind extinguishes a candle and energizes fire.
Location 89
```

If no results are found, say so and suggest alternative search terms.

---

## 4. Browse

**Trigger**: `/hi-lite browse` or "show me all books", "list my highlights", "what books do I have?"

### Capabilities

- **"Show me all books"** — Read `_index.md` and display the books table.
- **"Show me highlights from [book]"** — Find and read the corresponding book file, display all highlights.
- **"Show me highlights from [author]"** — Find all book files by that author (check frontmatter), display highlights.
- **"Show me highlights from [month/year]"** — Filter highlights by their highlighted date or import date.
- **"Show me my most highlighted books"** — Read `_index.md`, sort by highlight count descending, display top results.
- **"How many highlights do I have?"** — Read `_index.md` and report totals.

### Response Format

Keep responses clean and scannable. Use the books table for listings. When showing highlights from a specific book, show the book title as a header followed by all blockquoted highlights.

---

## 5. Random Quotes

**Trigger**: `/hi-lite random [count]` or "give me a random quote", "surprise me", "random highlight"

### Steps

1. List all book files in `highlights/books/`.
2. Read one or more book files (chosen randomly).
3. Pick random highlights from the loaded files.
4. Default count is **1** if not specified. The user can request any number (e.g., "give me 5 random quotes").
5. Try to pick from different books for variety when count > 1.

### Response Format

```
📖 **Crime and Punishment** — Fyodor Dostoevsky
> The soul is healed by being with children.
```

For multiple quotes, separate each with a blank line.

---

## 6. Collections

**Trigger**: `/hi-lite collection <name>` or "make a collection about courage", "create a [theme] collection"

### Steps

1. Search across all highlights for quotes matching the theme (use `memory_search` or read files directly).
2. Curate a selection of the most relevant quotes.
3. Save as `~/.openclaw/workspace/hi-lite/collections/<slug>.md`.
4. Present the collection to the user.

### Collection File Format

```markdown
---
name: Quotes About Courage
created: 2026-02-22
highlight_count: 8
---

# Quotes About Courage

> Pain and suffering are always inevitable for a large intelligence and a deep heart.
— Fyodor Dostoevsky, *Crime and Punishment*

> Wind extinguishes a candle and energizes fire.
— Nassim Nicholas Taleb, *Antifragile*
```

Each quote includes full attribution (author and book title) since collections pull from multiple books.

### Managing Collections

- **"Show my collections"** — List all files in `collections/`.
- **"Show collection [name]"** — Read and display the specified collection.
- **"Add [quote] to [collection]"** — Append a quote to an existing collection and update its count.
- **"Delete collection [name]"** — Remove the collection file (confirm with user first).

---

## 7. Fetch from Amazon

**Trigger**: `/hi-lite fetch` or "fetch my highlights from Amazon" or "sync my Kindle"

### First-Time Setup

Check if Playwright is available by running `python3 -c "from playwright.sync_api import sync_playwright"`. If it fails, guide the user:

```bash
pip install "playwright>=1.40.0"
playwright install chromium
```

### Execution

When the user triggers a fetch:

1. Write the following Python script to `~/.openclaw/workspace/hi-lite/raw/fetch_highlights.py`.
2. Run it via bash: `python3 ~/.openclaw/workspace/hi-lite/raw/fetch_highlights.py` (append `--amazon-domain amazon.co.uk` etc. if the user specifies a non-US domain).
3. The script opens a visible Chromium window. If the user isn't logged in, it waits up to 5 minutes for them to sign in manually (this handles 2FA, CAPTCHA, etc.). Session cookies are saved at `~/.openclaw/workspace/hi-lite/.browser-data/` so future fetches skip login.
4. The script iterates through all annotated books in the sidebar, extracts highlights, and saves a JSON file to `~/.openclaw/workspace/hi-lite/raw/kindle-fetch-{timestamp}.json`.
5. After the script finishes, delete the script file (`fetch_highlights.py`) from `raw/` so it doesn't get parsed as an import.
6. Then automatically run the standard import flow (Section 2) on the fetched JSON file.

**The script to write:**

```python
#!/usr/bin/env python3
"""Fetch Kindle highlights from Amazon's read.amazon.com/notebook page."""

import argparse
import json
import os
import sys
import time
from datetime import datetime, timezone
from pathlib import Path

try:
    from playwright.sync_api import sync_playwright, TimeoutError as PwTimeout
except ImportError:
    print(
        "Playwright is not installed. Run:\n"
        "  pip install 'playwright>=1.40.0'\n"
        "  playwright install chromium"
    )
    sys.exit(1)

DEFAULT_BROWSER_DATA = os.path.expanduser(
    "~/.openclaw/workspace/hi-lite/.browser-data"
)
DEFAULT_OUTPUT_DIR = os.path.expanduser("~/.openclaw/workspace/hi-lite/raw")
DEFAULT_DOMAIN = "amazon.com"
LOGIN_TIMEOUT_SEC = 300


def parse_args():
    parser = argparse.ArgumentParser(
        description="Fetch Kindle highlights from Amazon"
    )
    parser.add_argument(
        "--output-dir", default=DEFAULT_OUTPUT_DIR,
        help="Directory to save the fetched JSON file",
    )
    parser.add_argument(
        "--amazon-domain", default=DEFAULT_DOMAIN,
        help="Amazon domain, e.g. amazon.co.uk",
    )
    parser.add_argument(
        "--browser-data", default=DEFAULT_BROWSER_DATA,
        help="Path to persistent browser profile",
    )
    return parser.parse_args()


def wait_for_login(page, timeout_sec=LOGIN_TIMEOUT_SEC):
    print("Login required — please sign in to Amazon in the browser window.")
    print(f"Waiting up to {timeout_sec // 60} minutes for login...")
    deadline = time.time() + timeout_sec
    while time.time() < deadline:
        url = page.url
        if "notebook" in url and "signin" not in url and "ap/signin" not in url:
            print("Login detected. Continuing...")
            return True
        time.sleep(2)
    print("Login timed out.")
    return False


def scroll_to_load_all(page, container_selector, item_selector):
    previous_count = 0
    stale_rounds = 0
    while stale_rounds < 3:
        items = page.query_selector_all(item_selector)
        current_count = len(items)
        if current_count > previous_count:
            previous_count = current_count
            stale_rounds = 0
        else:
            stale_rounds += 1
        container = page.query_selector(container_selector)
        if container:
            container.evaluate("el => el.scrollTop = el.scrollHeight")
        else:
            page.evaluate("window.scrollTo(0, document.body.scrollHeight)")
        time.sleep(1)
    return previous_count


def extract_highlights_from_pane(page):
    highlights = []
    annotations = page.query_selector_all(".a-row.a-spacing-base")
    for annotation in annotations:
        header = annotation.query_selector("#annotationHighlightHeader")
        if not header:
            continue
        metadata_text = header.inner_text().strip()
        color = ""
        page_num = ""
        if "|" in metadata_text:
            parts = [p.strip() for p in metadata_text.split("|")]
            if parts:
                color = parts[0].replace("highlight", "").strip()
            if len(parts) > 1 and ":" in parts[1]:
                page_num = parts[1].split(":", 1)[1].strip()
        text_el = annotation.query_selector("#highlight")
        text = text_el.inner_text().strip() if text_el else ""
        note_el = annotation.query_selector("#note")
        note = note_el.inner_text().strip() if note_el else ""
        if text:
            highlights.append({
                "text": text, "page": page_num,
                "note": note, "color": color,
            })
    return highlights


def fetch_highlights(args):
    domain = args.amazon_domain
    notebook_url = f"https://read.{domain}/notebook"
    output_dir = Path(args.output_dir)
    output_dir.mkdir(parents=True, exist_ok=True)
    browser_data = Path(args.browser_data)
    browser_data.mkdir(parents=True, exist_ok=True)

    with sync_playwright() as pw:
        context = pw.chromium.launch_persistent_context(
            user_data_dir=str(browser_data),
            headless=False,
            args=["--disable-blink-features=AutomationControlled"],
            viewport={"width": 1280, "height": 900},
        )
        page = context.pages[0] if context.pages else context.new_page()

        print(f"Navigating to {notebook_url} ...")
        page.goto(notebook_url, wait_until="domcontentloaded", timeout=60000)
        time.sleep(3)

        if "signin" in page.url or "ap/signin" in page.url:
            if not wait_for_login(page):
                context.close()
                sys.exit(1)
            page.goto(notebook_url, wait_until="domcontentloaded", timeout=60000)
            time.sleep(3)

        print("Waiting for notebook to load...")
        try:
            page.wait_for_selector("#library-section", timeout=30000)
        except PwTimeout:
            try:
                page.wait_for_selector(
                    ".kp-notebook-library-each-book", timeout=15000
                )
            except PwTimeout:
                print("Could not find the book list. The page may have changed.")
                context.close()
                sys.exit(1)

        time.sleep(2)

        print("Discovering books in your library...")
        scroll_to_load_all(
            page, "#library-section", ".kp-notebook-library-each-book"
        )

        book_elements = page.query_selector_all(
            ".kp-notebook-library-each-book"
        )
        total_books = len(book_elements)
        print(f"Found {total_books} annotated books.")

        if total_books == 0:
            print("No annotated books found.")
            context.close()
            return

        books_data = []
        for i in range(total_books):
            book_elements = page.query_selector_all(
                ".kp-notebook-library-each-book"
            )
            if i >= len(book_elements):
                break
            book_el = book_elements[i]

            title_el = book_el.query_selector("h2, .kp-notebook-searchable")
            sidebar_title = (
                title_el.inner_text().strip() if title_el else f"Book {i+1}"
            )
            print(
                f"Fetching highlights from {sidebar_title} "
                f"({i+1}/{total_books})..."
            )

            book_el.click()
            time.sleep(2)

            try:
                page.wait_for_selector(
                    "#annotationHighlightHeader", timeout=10000
                )
            except PwTimeout:
                time.sleep(2)

            title = ""
            author = ""
            asin = ""

            title_header = page.query_selector(
                ".kp-notebook-metadata h3, "
                ".kp-notebook-metadata .a-size-base-plus"
            )
            if title_header:
                title = title_header.inner_text().strip()
            if not title:
                title = sidebar_title

            author_el = page.query_selector(
                ".kp-notebook-metadata .a-color-secondary, "
                ".kp-notebook-metadata p"
            )
            if author_el:
                author = (
                    author_el.inner_text().strip()
                    .replace("By: ", "").replace("by: ", "").strip()
                )

            asin_attr = book_el.get_attribute("id") or ""
            if asin_attr.startswith("B"):
                asin = asin_attr

            scroll_to_load_all(
                page,
                "#annotations-container, .a-row.a-spacing-base",
                "#annotationHighlightHeader",
            )

            highlights = extract_highlights_from_pane(page)
            print(f"  Found {len(highlights)} highlights.")

            books_data.append({
                "title": title, "author": author,
                "asin": asin, "highlights": highlights,
            })

        context.close()

    timestamp = datetime.now(timezone.utc).strftime("%Y-%m-%dT%H:%M:%S")
    output = {
        "source": "amazon-kindle-notebook",
        "fetched_at": timestamp,
        "amazon_domain": domain,
        "books": books_data,
    }

    filename = (
        f"kindle-fetch-"
        f"{datetime.now(timezone.utc).strftime('%Y%m%d-%H%M%S')}.json"
    )
    output_path = output_dir / filename
    with open(output_path, "w", encoding="utf-8") as f:
        json.dump(output, f, indent=2, ensure_ascii=False)

    total_hl = sum(len(b["highlights"]) for b in books_data)
    print(f"\nDone! Fetched {total_hl} highlights from {len(books_data)} books.")
    print(f"Saved to: {output_path}")


if __name__ == "__main__":
    args = parse_args()
    fetch_highlights(args)
```

### Re-Fetch

Re-fetching is safe. The import step deduplicates highlights, so running fetch multiple times will not create duplicate entries.

### Non-US Amazon Domains

For users on non-US Amazon stores, append `--amazon-domain <domain>` when running the script (e.g., `--amazon-domain amazon.co.uk`). Ask the user which Amazon store they use if unclear.

---

## General Behavior

- Always be conversational and helpful. The user is interacting through a chat interface.
- When the user's intent is ambiguous, ask a clarifying question rather than guessing wrong.
- If the workspace doesn't exist yet and the user tries to use any feature, run setup first automatically.
- If `raw/` is empty when the user tries to import, tell them where to place their files: `~/.openclaw/workspace/hi-lite/raw/`
- Keep responses concise. Don't dump 50 highlights at once — show 5-10 at a time and offer to show more.
- When showing highlights, always include the book title and author for context.


---

## Skill Companion Files

> Additional files collected from the skill directory layout.

### README.md

```markdown
# Hi-Lite

**Search, browse, and rediscover your Kindle highlights — locally, for free.**

Hi-Lite is an [OpenClaw](https://openclaw.org) skill that turns your raw Kindle highlights into a searchable, browsable personal library. No accounts, no subscriptions, no cloud — everything stays on your machine.

## Prerequisites

- [OpenClaw](https://openclaw.org) installed and running

## Installation

**Option A: ClawHub**

```
clawhub install hi-lite
```

**Option B: Git clone**

```bash
git clone https://github.com/gofordylan/hi-lite.git ~/.openclaw/workspace/skills/hi-lite/
```

**Option C: Paste the repo URL** to your OpenClaw assistant and it will install it for you.

## Setup

Tell your OpenClaw assistant:

> Set up hi-lite

This creates the workspace directory at `~/.openclaw/workspace/hi-lite/` with the necessary subdirectories.

### Optional: Enable Semantic Search

For the best search experience, add the highlights directory to your OpenClaw memory search config:

```json
{
  "memorySearch": {
    "extraPaths": ["~/.openclaw/workspace/hi-lite/highlights"]
  }
}
```

This enables hybrid vector + BM25 search across all your highlights. Without it, Hi-Lite still works — it just reads files directly instead of using semantic search.

## Importing Highlights

1. Place your raw Kindle export files into `~/.openclaw/workspace/hi-lite/raw/`
2. Tell your assistant:

> Import my highlights

### Supported Sources

- **My Clippings.txt** — The file from your Kindle device (connect via USB, find it in the `documents` folder)
- **Amazon Read Notebook** — Copy-paste from [read.amazon.com/notebook](https://read.amazon.com/notebook)
- **Bookcision** — JSON or text exports from the [Bookcision](https://readwise.io/bookcision) browser extension
- **Any raw text** — Paste or drop any text with highlights and Hi-Lite will do its best to parse it

## Auto-Fetch from Amazon

Fetch all your Kindle highlights directly from Amazon with a single command — no manual exporting needed.

### Prerequisites

- Python 3.8+
- pip

### First-Time Setup

```bash
pip install "playwright>=1.40.0"
playwright install chromium
```

### Usage

Tell your OpenClaw assistant:

> Fetch my highlights from Amazon

Or:

> /hi-lite fetch

A browser window will open and navigate to Amazon's Kindle notebook. If you're not already logged in, you'll be prompted to sign in manually (the script handles 2FA and CAPTCHA by letting you do it yourself in the real browser). Your session is saved, so future fetches won't require logging in again.

After fetching, Hi-Lite automatically imports the highlights into your library.

### Non-US Amazon Domains

If your Amazon account is on a non-US store (e.g., amazon.co.uk, amazon.de), just mention it to your assistant.

---

## Usage

All interaction happens through natural conversation with your OpenClaw assistant.

### Search

> Find quotes about perseverance

> What did Dostoevsky say about suffering?

> /hi-lite search meaning of life

### Browse

> Show me all my books

> Show me highlights from Antifragile

> What are my most highlighted books?

### Random Quotes

> Give me a random quote

> Surprise me with 5 highlights

> /hi-lite random 3

### Collections

> Make a collection of quotes about courage

> Show my collections

> Add that last quote to my "favorites" collection

## How It Works

Hi-Lite stores everything as plain markdown files:

```
~/.openclaw/workspace/hi-lite/
├── raw/              # Your raw Kindle exports
├── highlights/
│   ├── _index.md     # Master index of all books
│   └── books/        # One markdown file per book
└── collections/      # Themed collections you create
```

Each book gets its own markdown file with YAML frontmatter and blockquoted highlights. This format is human-readable, git-friendly, and optimized for OpenClaw's vector search.

## Privacy

All your data stays on your machine. Hi-Lite is just a set of instructions that teach your local OpenClaw agent how to work with your files. Nothing is sent to any server.

## License

MIT

```

### _meta.json

```json
{
  "owner": "gofordylan",
  "slug": "hi-lite",
  "displayName": "Hi Lite",
  "latest": {
    "version": "1.0.0",
    "publishedAt": 1771824927938,
    "commit": "https://github.com/openclaw/skills/commit/ffd3891c137c10e688877b09429a3ed6fb0cc0ee"
  },
  "history": []
}

```