markdown-token-optimizer

Original source / Raw SKILL.md

---
name: markdown-token-optimizer
description: "Analyzes markdown files for token efficiency. TRIGGERS: optimize markdown, reduce tokens, token count, token bloat, too many tokens, make concise, shrink file, file too large, optimize for AI, token efficiency, verbose markdown, reduce file size"
license: MIT
metadata:
  author: Microsoft
  version: "1.0.0"
---

# Markdown Token Optimizer

This skill analyzes markdown files and suggests optimizations to reduce token consumption while maintaining clarity.

## When to Use

- Optimize markdown files for token efficiency
- Reduce SKILL.md file size or check for bloat
- Make documentation more concise for AI consumption

## Workflow

1. **Count** - Calculate tokens (~4 chars = 1 token), report totals
2. **Scan** - Find patterns: emojis, verbosity, duplication, large blocks
3. **Suggest** - Table with location, issue, fix, savings estimate
4. **Summary** - Current/potential/savings with top recommendations

See [ANTI-PATTERNS.md](references/ANTI-PATTERNS.md) for detection patterns and [OPTIMIZATION-PATTERNS.md](references/OPTIMIZATION-PATTERNS.md) for techniques.

## Rules

- Suggest only (no auto-modification)  
- Preserve clarity in all optimizations
- SKILL.md target: <500 tokens, references: <1000 tokens

## References

- [OPTIMIZATION-PATTERNS.md](references/OPTIMIZATION-PATTERNS.md) - Optimization techniques
- [ANTI-PATTERNS.md](references/ANTI-PATTERNS.md) - Token-wasting patterns


---

## Referenced Files

> The following files are referenced in this skill and included for context.

### references/ANTI-PATTERNS.md

```markdown
# Anti-Patterns: Common Token Wasters

Patterns that inflate token counts without adding value.

## Decorative Elements

### Excessive Emojis

**Bad:**
```markdown
# 🚀 Getting Started 🎉

Welcome! 👋 Let's get you set up! 💪

## 📋 Prerequisites ✨

You'll need: 🔧
- Node.js 🟢
- Azure CLI ☁️
```

**Good:**
```markdown
# Getting Started

## Prerequisites

- Node.js 18+
- Azure CLI
```

**Token cost:** ~2 tokens per emoji. 10 decorative emojis = 20 wasted tokens.

**When emojis ARE appropriate:**
- Status indicators in tables
- Actual meaning (not decoration)

### Horizontal Rules Overuse

**Bad:**
```markdown
## Section One
Content here.
---
## Section Two
More content.
---
```

**Good:**
```markdown
## Section One
Content here.

## Section Two
More content.
```

**Token cost:** ~3 tokens per `---`

## Redundant Content

### Repeated Introductions

**Bad:**
```markdown
## Prerequisites
Before you begin, ensure these are installed:
- Node.js (with redundant explanations)
- Azure CLI (with redundant explanations)
```

**Good:**
```markdown
## Prerequisites

- Node.js 18+
- Azure CLI 2.50+
```

### Echo Headers

**Bad:**
```markdown
## Troubleshooting

This section covers troubleshooting steps.

### Common Issues

Here are some common issues you might encounter.
```

**Good:**
```markdown
## Troubleshooting

| Issue | Solution |
|-------|----------|
| Auth fails | Run `az login` |
```

## Verbose Constructions

### Filler Words

**Bad:**
```markdown
To successfully deploy your application to Azure, 
you will first need to make sure that you have properly 
configured all of the necessary prerequisites.
```

**Good:**
```markdown
Before deploying to Azure, configure these prerequisites:
```

**Savings:** ~20 tokens

### Unnecessary Qualifiers

**Bad:**
```markdown
- It's very important to always remember to...
- You should definitely make sure to...
- It's absolutely essential that you...
```

**Good:**
```markdown
- Remember to...
- Ensure...
- Required:...
```

## Structural Issues

### Inline Documentation That Should Be Referenced

**Bad (in SKILL.md):**
```markdown
## Complete API Reference
### GET /users
Returns all users. Accepts params...
[continues for 200 more lines]
```

**Good:**
```markdown
## API Reference
See [references/API.md](references/API.md) for endpoints.
Key endpoints: `GET /users`, `POST /users`, `GET /users/:id`
```

### Duplicate Examples

**Bad:** Three separate examples with 95% identical base command

**Good:**
```markdown
## Examples
Basic: `az storage account create -n NAME -g RG -l LOCATION`
Options: `--tags env=dev`, `--sku Standard_GRS`

More: [references/EXAMPLES.md](references/EXAMPLES.md)
```

## Detection Checklist

When reviewing a file, check for:

- [ ] More than 3 decorative emojis
- [ ] Sections that start with "This section..."
- [ ] Verbose constructions (see Filler Words section)
- [ ] Same command/text appearing 3+ times
- [ ] Code blocks longer than 10 lines
- [ ] Tables with 10+ rows in main SKILL.md
- [ ] Headers followed immediately by similar subheaders

```

### references/OPTIMIZATION-PATTERNS.md

```markdown
# Optimization Patterns

Techniques for reducing token count while maintaining clarity.

## Content Restructuring

### Move Details to References

**Before (in SKILL.md):**
```markdown
## API Reference

| Endpoint | Method | Parameters | Response |
|----------|--------|------------|----------|
| /users | GET | limit, offset | User[] |
| /users/:id | GET | id | User |
| /users | POST | name, email | User |
... (50 more rows)
```

**After:**
```markdown
## API Reference

See [references/API.md](references/API.md) for the complete endpoint list.
```

**Savings:** 200+ tokens for large tables

### Use Reference Links for Repeated Content

**Before:**
```markdown
Run `az login --tenant YOUR_TENANT_ID` to authenticate.
...
Make sure you've run `az login --tenant YOUR_TENANT_ID` first.
...
If not authenticated, use `az login --tenant YOUR_TENANT_ID`.
```

**After:**
```markdown
Run `az login --tenant YOUR_TENANT_ID` to authenticate.
...
Make sure you've authenticated (see above).
...
If not authenticated, see the authentication step above.
```

**Savings:** ~20 tokens per removed duplicate

## Text Optimization

### Replace Verbose Phrases

| Verbose | Concise | Savings |
|---------|---------|---------|
| "In order to" | "To" | 2 tokens |
| "It is important to note that" | "Note:" | 5 tokens |
| "At this point in time" | "Now" | 4 tokens |
| "Due to the fact that" | "Because" | 4 tokens |
| "In the event that" | "If" | 3 tokens |
| "For the purpose of" | "To" / "For" | 3 tokens |
| "A large number of" | "Many" | 3 tokens |
| "In close proximity to" | "Near" | 3 tokens |

### Consolidate Similar Sections

**Before:** Three identical prerequisite sections (Windows/macOS/Linux)

**After:**
```markdown
## Prerequisites (All Platforms)
- Node.js 18+
- Azure CLI

Platform-specific: [references/INSTALL.md](references/INSTALL.md)
```

**Savings:** 50+ tokens

## Formatting Optimization

### Tables vs Lists

**Before (verbose list):**
```markdown
- **Storage Account**: Minimum 3 characters, maximum 24 characters, 
  only lowercase letters and numbers allowed, must be globally unique.
- **Key Vault**: Minimum 3 characters, maximum 24 characters,
  alphanumerics and hyphens allowed, must be globally unique.
```

**After (compact table):**
```markdown
| Resource | Min | Max | Allowed | Global |
|----------|-----|-----|---------|--------|
| Storage | 3 | 24 | a-z, 0-9 | Yes |
| Key Vault | 3 | 24 | a-z, 0-9, - | Yes |
```

**Savings:** ~30 tokens per 5 items

### Inline Code vs Code Blocks

**Before:**
```markdown
To list resources, run:

\`\`\`bash
az resource list --resource-group mygroup
\`\`\`
```

**After:**
```markdown
List resources: `az resource list -g mygroup`
```

**Savings:** ~10 tokens per instance

## Structural Patterns

### Progressive Disclosure Structure

```
SKILL.md (< 500 tokens)
├── Overview (what + when)
├── Quick workflow (numbered steps)
├── Key commands/tools table
└── Links to references

references/
├── DETAILED-GUIDE.md (< 1000 tokens each)
├── TROUBLESHOOTING.md
└── EXAMPLES.md
```

### Header Consolidation

**Before:**
```markdown
## Overview
## Introduction  
## About This Skill
## What This Does
```

**After:**
```markdown
## Overview
```

**Savings:** ~15 tokens per removed header + content

```