arc-shield

Original source / Raw SKILL.md

---
name: arc-shield
version: 1.0.0
category: security
tags: [security, sanitization, secrets, output-filter, privacy]
requires: [bash, python3]
author: OpenClaw
description: Output sanitization for agent responses - prevents accidental secret leaks
---

# arc-shield

**Output sanitization for agent responses.** Scans ALL outbound messages for leaked secrets, tokens, keys, passwords, and PII before they leave the agent.

⚠️ **This is NOT an input scanner** — `clawdefender` already handles that. This is an **OUTPUT filter** for catching things your agent accidentally includes in its own responses.

## Why You Need This

Agents have access to sensitive data: 1Password vaults, environment variables, config files, wallet keys. Sometimes they accidentally include these in responses when:
- Debugging and showing full command output
- Copying file contents that contain secrets
- Generating code examples with real credentials
- Summarizing logs that include tokens

Arc-shield catches these leaks before they reach Discord, Signal, X, or any external channel.

## What It Detects

### 🔴 CRITICAL (blocks in `--strict` mode)
- **API Keys & Tokens**: 1Password (`ops_*`), GitHub (`ghp_*`), OpenAI (`sk-*`), Stripe, AWS, Bearer tokens
- **Passwords**: Assignments like `password=...` or `passwd: ...`
- **Private Keys**: Ethereum (0x + 64 hex), SSH keys, PGP blocks
- **Wallet Mnemonics**: 12/24 word recovery phrases
- **PII**: Social Security Numbers, credit card numbers
- **Platform Tokens**: Slack, Telegram, Discord

### 🟠 HIGH (warns loudly)
- **High-entropy strings**: Shannon entropy > 4.5 for strings > 16 chars (catches novel secret patterns)
- **Credit cards**: 16-digit card numbers
- **Base64 credentials**: Long base64 strings that look like tokens

### 🟡 WARN (informational)
- **Secret file paths**: `~/.secrets/*`, paths containing "password", "token", "key"
- **Environment variables**: `ENV_VAR=secret_value` exports
- **Database URLs**: Connection strings with credentials

## Installation

```bash
cd ~/.openclaw/workspace/skills
git clone <arc-shield-repo> arc-shield
chmod +x arc-shield/scripts/*.sh arc-shield/scripts/*.py
```

Or download as a skill bundle.

## Usage

### Command-line

```bash
# Scan agent output before sending
agent-response.txt | arc-shield.sh

# Block if critical secrets found (use before external messaging)
echo "Message text" | arc-shield.sh --strict || echo "BLOCKED"

# Redact secrets and return sanitized text
cat response.txt | arc-shield.sh --redact

# Full report
arc-shield.sh --report < conversation.log

# Python version with entropy detection
cat message.txt | output-guard.py --strict
```

### Integration with OpenClaw Agents

#### Pre-send hook (recommended)

Add to your messaging skill or wrapper:

```bash
#!/bin/bash
# send-message.sh wrapper

MESSAGE="$1"
CHANNEL="$2"

# Sanitize output
SANITIZED=$(echo "$MESSAGE" | arc-shield.sh --strict --redact)
EXIT_CODE=$?

if [[ $EXIT_CODE -eq 1 ]]; then
    echo "ERROR: Message contains critical secrets and was blocked." >&2
    exit 1
fi

# Send sanitized message
openclaw message send --channel "$CHANNEL" "$SANITIZED"
```

#### Manual pipe

Before any external message:

```bash
# Generate response
RESPONSE=$(agent-generate-response)

# Sanitize
CLEAN=$(echo "$RESPONSE" | arc-shield.sh --redact)

# Send
signal send "$CLEAN"
```

### Testing

```bash
cd skills/arc-shield/tests
./run-tests.sh
```

Includes test cases for:
- Real leaked patterns (1Password tokens, Instagram passwords, wallet mnemonics)
- False positive prevention (normal URLs, email addresses, file paths)
- Redaction accuracy
- Strict mode blocking

## Configuration

Patterns are defined in `config/patterns.conf`:

```conf
CRITICAL|GitHub PAT|ghp_[a-zA-Z0-9]{36,}
CRITICAL|OpenAI Key|sk-[a-zA-Z0-9]{20,}
WARN|Secret Path|~\/\.secrets\/[^\s]*
```

Edit to add custom patterns or adjust severity levels.

## Modes

| Mode | Behavior | Exit Code | Use Case |
|------|----------|-----------|----------|
| Default | Pass through + warnings to stderr | 0 | Development, logging |
| `--strict` | Block on CRITICAL findings | 1 if critical | Production outbound messages |
| `--redact` | Replace secrets with `[REDACTED:TYPE]` | 0 | Safe logging, auditing |
| `--report` | Analysis only, no pass-through | 0 | Auditing conversations |

## Entropy Detection

The Python version (`output-guard.py`) includes Shannon entropy analysis to catch secrets that don't match regex patterns:

```python
# Detects high-entropy strings like:
kJ8nM2pQ5rT9vWxY3zA6bC4dE7fG1hI0  # Novel API key format
Zm9vOmJhcg==                      # Base64 credentials
```

Threshold: **4.5 bits** (configurable with `--entropy-threshold`)

## Performance

- **Bash version**: ~10ms for typical message (< 1KB)
- **Python version**: ~50ms with entropy analysis
- **Zero external dependencies**: bash + Python stdlib only

Fast enough to run on every outbound message without noticeable delay.

## Real-World Catches

From our own agent sessions:

```bash
# 1Password token
"ops_eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9..."

# Instagram password in debug output
"instagram login: [email protected] / MyInsT@Gr4mP4ss!"

# Wallet mnemonic in file listing
"cat ~/.secrets/wallet-recovery-phrase.txt
abandon ability able about above absent absorb abstract..."

# GitHub PAT in git config
"[remote "origin"]
url = https://ghp_abc123:@github.com/user/repo"
```

All blocked by arc-shield before reaching external channels.

## Best Practices

1. **Always use `--strict` for external messages** (Discord, Signal, X, email)
2. **Use `--redact` for logs** you want to review later
3. **Run tests after adding custom patterns** to check for false positives
4. **Pipe through both** bash and Python versions for maximum coverage:
   ```bash
   message | arc-shield.sh --strict | output-guard.py --strict
   ```
5. **Don't rely on this alone** — educate your agent to avoid including secrets in the first place (see AGENTS.md output sanitization directive)

## Limitations

- **Context-free**: Can't distinguish between "here's my password: X" (bad) and "set your password to X" (instruction)
- **No semantic understanding**: Won't catch "my token is in the previous message"
- **Pattern-based**: New secret formats require pattern updates

Use in combination with agent instructions and careful prompt engineering.

## Integration Example

Full OpenClaw agent integration:

```bash
# In your agent's message wrapper
send_external_message() {
    local message="$1"
    local channel="$2"
    
    # Pre-flight sanitization
    if ! echo "$message" | arc-shield.sh --strict > /dev/null 2>&1; then
        echo "ERROR: Message blocked by arc-shield (contains secrets)" >&2
        return 1
    fi
    
    # Double-check with entropy detection
    if ! echo "$message" | output-guard.py --strict > /dev/null 2>&1; then
        echo "ERROR: High-entropy secret detected" >&2
        return 1
    fi
    
    # Safe to send
    openclaw message send --channel "$channel" "$message"
}
```

## Troubleshooting

**False positives on normal text:**
- Adjust entropy threshold: `output-guard.py --entropy-threshold 5.0`
- Edit `config/patterns.conf` to refine regex patterns
- Add exceptions to the pattern file

**Secrets not detected:**
- Check pattern file for coverage
- Run with `--report` to see what's being scanned
- Test with `tests/run-tests.sh` using your sample
- Consider lowering entropy threshold (but watch for false positives)

**Performance issues:**
- Use bash version only (skip entropy detection)
- Limit input size with `head -c 10000`
- Run in background: `arc-shield.sh --report &`

## Contributing

Add new patterns to `config/patterns.conf` following the format:

```
SEVERITY|Category Name|regex_pattern
```

Test with `tests/run-tests.sh` before deploying.

## License

MIT — use freely, protect your secrets.

---

**Remember**: Arc-shield is your safety net, not your strategy. Train your agent to never include secrets in responses. This tool catches mistakes, not malice.


---

## Referenced Files

> The following files are referenced in this skill and included for context.

### tests/run-tests.sh

```bash
#!/usr/bin/env bash
# Test suite for arc-shield

set -euo pipefail

SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
ARC_SHIELD="${SCRIPT_DIR}/../scripts/arc-shield.sh"
OUTPUT_GUARD="${SCRIPT_DIR}/../scripts/output-guard.py"
TEST_SAMPLES="${SCRIPT_DIR}/test-samples.txt"

RED='\033[0;31m'
GREEN='\033[0;32m'
YELLOW='\033[1;33m'
NC='\033[0m'

PASSED=0
FAILED=0

echo "=== ARC-SHIELD TEST SUITE ==="
echo

test_detection() {
    local name=$1
    local input=$2
    local should_detect=$3
    
    # Test with arc-shield.sh
    set +e
    result=$(echo "$input" | "$ARC_SHIELD" --report 2>&1)
    exit_code=$?
    set -e
    
    detected=0
    if echo "$result" | grep -q "CRITICAL\|HIGH\|WARN"; then
        detected=1
    fi
    
    if [[ $should_detect -eq 1 && $detected -eq 1 ]]; then
        echo -e "${GREEN}✓${NC} PASS: $name (detected as expected)"
        ((PASSED++))
    elif [[ $should_detect -eq 0 && $detected -eq 0 ]]; then
        echo -e "${GREEN}✓${NC} PASS: $name (no false positive)"
        ((PASSED++))
    else
        echo -e "${RED}✗${NC} FAIL: $name"
        echo "  Expected detect=$should_detect, got detect=$detected"
        echo "  Input: $input"
        ((FAILED++))
    fi
}

# Parse test samples
echo "Testing arc-shield.sh..."
echo

while IFS= read -r line; do
    # Skip comments and empty lines
    if [[ "$line" =~ ^#.*$ ]] || [[ -z "$line" ]]; then
        continue
    fi
    
    # Parse format: [DETECT:CATEGORY] sample text
    if [[ "$line" =~ ^\[DETECT:([A-Z_]+)\]\ (.+)$ ]]; then
        category="${BASH_REMATCH[1]}"
        text="${BASH_REMATCH[2]}"
        test_detection "$category" "$text" 1
    elif [[ "$line" =~ ^\[IGNORE\]\ (.+)$ ]]; then
        text="${BASH_REMATCH[1]}"
        test_detection "IGNORE (${text:0:30}...)" "$text" 0
    fi
done < "$TEST_SAMPLES"

echo
echo "Testing output-guard.py..."
echo

# Test Python version with entropy detection
test_python() {
    local name=$1
    local input=$2
    local should_detect=$3
    
    set +e
    result=$(echo "$input" | python3 "$OUTPUT_GUARD" --report 2>&1)
    detected=0
    if echo "$result" | grep -q "CRITICAL\|HIGH\|WARN"; then
        detected=1
    fi
    set -e
    
    if [[ $should_detect -eq 1 && $detected -eq 1 ]]; then
        echo -e "${GREEN}✓${NC} PASS: $name (Python)"
        ((PASSED++))
    elif [[ $should_detect -eq 0 && $detected -eq 0 ]]; then
        echo -e "${GREEN}✓${NC} PASS: $name (Python)"
        ((PASSED++))
    else
        echo -e "${RED}✗${NC} FAIL: $name (Python)"
        ((FAILED++))
    fi
}

# Test high-entropy detection
test_python "High Entropy Detection" "Token: kJ8nM2pQ5rT9vWxY3zA6bC4dE7fG1hI0jK2lM4nO6p" 1
test_python "Normal Text" "This is just a regular sentence without any secrets" 0

# Test redaction
echo
echo "Testing redaction..."
echo

redacted=$(echo "My key is ghp_1234567890abcdefghijklmnopqrstuvwx" | "$ARC_SHIELD" --redact)
if echo "$redacted" | grep -q "\[REDACTED:GITHUB_PAT\]"; then
    echo -e "${GREEN}✓${NC} PASS: Redaction works"
    ((PASSED++))
else
    echo -e "${RED}✗${NC} FAIL: Redaction failed"
    echo "  Output: $redacted"
    ((FAILED++))
fi

# Test strict mode
echo
echo "Testing strict mode..."
echo

set +e
echo "Safe message" | "$ARC_SHIELD" --strict > /dev/null 2>&1
exit_code=$?
set -e

if [[ $exit_code -eq 0 ]]; then
    echo -e "${GREEN}✓${NC} PASS: Strict mode allows safe messages"
    ((PASSED++))
else
    echo -e "${RED}✗${NC} FAIL: Strict mode blocked safe message"
    ((FAILED++))
fi

set +e
echo "Leaked token: ghp_abc123def456ghi789jkl012mno345pqr" | "$ARC_SHIELD" --strict > /dev/null 2>&1
exit_code=$?
set -e

if [[ $exit_code -eq 1 ]]; then
    echo -e "${GREEN}✓${NC} PASS: Strict mode blocks critical secrets"
    ((PASSED++))
else
    echo -e "${RED}✗${NC} FAIL: Strict mode allowed critical secret"
    ((FAILED++))
fi

# Summary
echo
echo "=== TEST SUMMARY ==="
echo -e "Passed: ${GREEN}${PASSED}${NC}"
echo -e "Failed: ${RED}${FAILED}${NC}"
echo

if [[ $FAILED -eq 0 ]]; then
    echo -e "${GREEN}All tests passed!${NC}"
    exit 0
else
    echo -e "${RED}Some tests failed.${NC}"
    exit 1
fi

```



---

## Skill Companion Files

> Additional files collected from the skill directory layout.

### README.md

```markdown
# 🛡️ arc-shield

**Output sanitization for AI agents** — Catches leaked secrets before they escape.

This is **NOT** an input scanner (clawdefender does that). This is an **OUTPUT filter** that scans every outbound message for accidentally leaked secrets, tokens, keys, passwords, and PII.

## Quick Start

```bash
# Install
cd ~/.openclaw/workspace/skills
git clone <this-repo> arc-shield
chmod +x arc-shield/scripts/*.sh arc-shield/scripts/*.py

# Test
cd arc-shield/tests
./quick-test.sh

# Use
echo "My secret: ghp_abc123..." | arc-shield/scripts/arc-shield.sh --strict
```

## The Problem

Your AI agent has access to:
- 1Password vaults
- Environment variables
- Config files with API keys
- Wallet private keys
- Database credentials

Sometimes it accidentally includes these in responses when:
- Debugging with full command output
- Showing file contents
- Generating code examples
- Summarizing logs

**Arc-shield catches these leaks before they reach Discord, Signal, X, or anywhere else.**

## What Gets Detected

### 🔴 CRITICAL (blocks in `--strict` mode)
- 1Password tokens (`ops_*`)
- GitHub PATs (`ghp_*`)
- OpenAI keys (`sk-*`)
- Stripe keys, AWS keys
- Bearer tokens
- Password assignments
- Ethereum private keys
- SSH/PGP private keys
- Wallet mnemonics (12/24 words)
- SSNs, credit cards

### 🟠 HIGH (warns loudly)
- High-entropy strings (Shannon entropy > 4.5)
- Base64 credentials

### 🟡 WARN (informational)
- Secret file paths (`~/.secrets/*`)
- Environment variable exports
- Database URLs with credentials

See [SKILL.md](SKILL.md) for full details.

## Usage

### Basic Scanning

```bash
# Scan and pass through with warnings
echo "Message text" | arc-shield.sh

# Block if critical secrets found
echo "Token: ghp_abc..." | arc-shield.sh --strict
# Exit code 1 + error message

# Redact secrets
echo "Token: ghp_abc..." | arc-shield.sh --redact
# Output: Token: [REDACTED:GITHUB_PAT]

# Full report
arc-shield.sh --report < conversation.log
```

### With OpenClaw Agent

**Before sending to external channels:**

```bash
#!/bin/bash
# In your message wrapper

RESPONSE=$(generate_agent_response)

# Sanitize
if ! echo "$RESPONSE" | arc-shield.sh --strict > /dev/null 2>&1; then
    echo "ERROR: Response contains secrets, blocked" >&2
    exit 1
fi

# Safe to send
openclaw message send --channel discord "$RESPONSE"
```

### Python Version (with entropy detection)

```bash
# Better at catching novel secret formats
cat message.txt | output-guard.py --strict

# JSON report for automation
output-guard.py --json < log.txt
```

## Testing

```bash
cd tests

# Quick smoke test (10 checks, ~1 second)
./quick-test.sh

# Full test suite (all patterns)
./run-tests.sh
```

## Real-World Catches

From our own agent sessions:

```
✗ 1Password service account token
  ops_eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9...

✗ Instagram password in debug output
  instagram login: MyInsT@Gr4mP4ss!

✗ Wallet mnemonic in file listing
  abandon ability able about above absent absorb...

✗ GitHub PAT in git config
  https://ghp_abc123:@github.com/user/repo

✗ File path leak
  Check ~/.secrets/wallet-recovery-phrase.txt
```

All blocked before reaching external channels.

## Configuration

Edit `config/patterns.conf` to add custom patterns:

```conf
CRITICAL|Custom Token|mytoken_[a-zA-Z0-9]{32,}
HIGH|Internal Secret|SECRET_[A-Z0-9]{16,}
WARN|Dev Path|/internal/secrets/[^\s]*
```

## Integration Examples

### Pre-send Hook

```bash
# ~/.openclaw/workspace/skills/messaging/send-external.sh
send_message() {
    local message="$1"
    local channel="$2"
    
    # Sanitize with arc-shield
    if ! echo "$message" | arc-shield.sh --strict 2>/dev/null; then
        echo "⚠️ Message blocked: contains secrets" >&2
        return 1
    fi
    
    # Send
    openclaw message send --channel "$channel" "$message"
}
```

### Log Sanitization

```bash
# Clean logs before committing
cat agent-session.log | arc-shield.sh --redact > clean.log
git add clean.log
```

### Audit Conversations

```bash
# Check what was leaked in past conversations
arc-shield.sh --report < old-conversation.txt

# JSON for automation
output-guard.py --json < *.log | jq '.summary.critical'
```

## Performance

- **Bash version**: ~10ms per message (<1KB)
- **Python version**: ~50ms with entropy analysis
- **Zero dependencies**: bash + Python stdlib only

Fast enough to run on every outbound message.

## Limitations

1. **Context-free**: Can't tell "here's my password: X" (bad) from "set your password to X" (instruction)
2. **No semantic understanding**: Won't catch "my token is in the previous message"
3. **Pattern-based**: New secret formats need pattern updates

**Solution**: Use arc-shield + agent training (see AGENTS.md output sanitization directive).

## Best Practices

1. ✅ **Always use `--strict` for external messages**
2. ✅ **Use `--redact` for logs you review**
3. ✅ **Run tests after adding patterns**
4. ✅ **Combine bash + Python for max coverage**
5. ✅ **Train your agent to avoid secrets in responses**

## Files

```
arc-shield/
├── scripts/
│   ├── arc-shield.sh       # Fast regex-based scanner
│   └── output-guard.py     # Entropy detection version
├── config/
│   └── patterns.conf       # Configurable patterns
├── tests/
│   ├── quick-test.sh       # Smoke test (10 checks)
│   ├── run-tests.sh        # Full test suite
│   └── test-samples.txt    # Test cases
├── SKILL.md                # Full documentation
└── README.md               # This file
```

## Contributing

Add patterns to `config/patterns.conf`, test with `./tests/quick-test.sh`, submit PR.

## License

MIT — protect your secrets freely.

---

**Remember**: Arc-shield is your safety net, not your strategy. Train your agent to never include secrets. This catches mistakes, not malice.

```

### _meta.json

```json
{
  "owner": "arc-claw-bot",
  "slug": "arc-shield",
  "displayName": "Arc Shield",
  "latest": {
    "version": "1.0.0",
    "publishedAt": 1770642853017,
    "commit": "https://github.com/openclaw/skills/commit/5728310dd5b83d89be6d4caad1201edf3eb35416"
  },
  "history": []
}

```

### scripts/arc-shield.sh

```bash
#!/usr/bin/env bash
# arc-shield.sh - Output sanitization for agent responses
# Scans outbound messages for leaked secrets, tokens, keys, passwords, and PII

set -euo pipefail

VERSION="1.0.0"
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
CONFIG_FILE="${ARC_SHIELD_CONFIG:-${SCRIPT_DIR}/../config/patterns.conf}"

# Mode flags
MODE="scan"  # scan, redact, report, strict
SEVERITY_THRESHOLD="WARN"  # WARN, HIGH, CRITICAL
FOUND_CRITICAL=0
FOUND_HIGH=0
FOUND_WARN=0

# Colors for output
RED='\033[0;31m'
YELLOW='\033[1;33m'
ORANGE='\033[0;33m'
NC='\033[0m' # No Color

usage() {
    cat << EOF
arc-shield ${VERSION} - Output sanitization for agent responses

USAGE:
    arc-shield.sh [OPTIONS] < input.txt
    echo "message" | arc-shield.sh [OPTIONS]

OPTIONS:
    --strict        Exit with code 1 if CRITICAL findings detected
    --redact        Replace detected secrets with [REDACTED]
    --report        Show findings report only (no output pass-through)
    --quiet         Suppress warnings, only show critical
    -h, --help      Show this help

MODES:
    Default: Scan and pass through with warnings to stderr
    --redact: Replace secrets and output sanitized text
    --report: Analyze and report findings only

EXAMPLES:
    # Scan agent output before sending
    agent-response.txt | arc-shield.sh --strict
    
    # Sanitize and replace secrets
    echo "My token is ghp_abc123" | arc-shield.sh --redact
    
    # Audit mode
    arc-shield.sh --report < conversation.log

EOF
    exit 0
}

# Parse arguments
while [[ $# -gt 0 ]]; do
    case $1 in
        --strict) MODE="strict"; shift ;;
        --redact) MODE="redact"; shift ;;
        --report) MODE="report"; shift ;;
        --quiet) SEVERITY_THRESHOLD="CRITICAL"; shift ;;
        -h|--help) usage ;;
        *) echo "Unknown option: $1" >&2; exit 1 ;;
    esac
done

# Read input
INPUT=$(cat)

# Detection functions
detect_1password_tokens() {
    echo "$INPUT" | grep -oE 'ops_[a-zA-Z0-9_-]{40,}' || true
}

detect_github_pats() {
    echo "$INPUT" | grep -oE 'ghp_[a-zA-Z0-9]{32,}' || true
}

detect_openai_keys() {
    echo "$INPUT" | grep -oE 'sk-[a-zA-Z0-9]{20,}' || true
}

detect_stripe_keys() {
    echo "$INPUT" | grep -oE 'sk_(test|live)_[a-zA-Z0-9]{24,}' || true
}

detect_aws_keys() {
    echo "$INPUT" | grep -oE 'AKIA[0-9A-Z]{16}' || true
}

detect_bearer_tokens() {
    echo "$INPUT" | grep -oE 'Bearer [a-zA-Z0-9_\-\.]{20,}' || true
}

detect_passwords() {
    # Match password assignments and similar patterns
    # Use [[:space:]] for whitespace to be portable
    echo "$INPUT" | grep -iE '(password|passwd|pwd)[[:space:]"]*[:=][[:space:]"]*[^[:space:]"]{6,}' || true
}

detect_eth_private_keys() {
    # 0x followed by 64 hex chars
    echo "$INPUT" | grep -oE '0x[0-9a-fA-F]{64}' || true
}

detect_ssh_keys() {
    echo "$INPUT" | grep -E -- '-----BEGIN (RSA|OPENSSH|DSA|EC) PRIVATE KEY-----' || true
}

detect_pgp_blocks() {
    echo "$INPUT" | grep -E -- '-----BEGIN PGP PRIVATE KEY BLOCK-----' || true
}

detect_mnemonics() {
    # 12 or 24 word phrases (simplified detection)
    echo "$INPUT" | grep -iE '\b([a-z]{3,}[\s]+){11,23}[a-z]{3,}\b' | \
        grep -iwE '(abandon|ability|able|about|above|absent|absorb|abstract|absurd|abuse|access|accident|account|accuse|achieve|acid|acoustic|acquire|across|act|action|actor|actress|actual|adapt|add|addict|address|adjust|admit|adult|advance|advice|aerobic|affair|afford|afraid|again|age|agent|agree|ahead|aim|air|airport|aisle|alarm|album|alcohol|alert|alien|all|alley|allow|almost|alone|alpha|already|also|alter|always|amateur|amazing|among|amount|amused|analyst|anchor|ancient|anger|angle|angry|animal|ankle|announce|annual|another|answer|antenna|antique|anxiety|any|apart|apology|appear|apple|approve|april|arch|arctic|area|arena|argue|arm|armed|armor|army|around|arrange|arrest|arrive|arrow|art|artefact|artist|artwork|ask|aspect|assault|asset|assist|assume|asthma|athlete|atom|attack|attend|attitude|attract|auction|audit|august|aunt|author|auto|autumn|average|avocado|avoid|awake|aware|away|awesome|awful|awkward|axis|baby|bachelor|bacon|badge|bag|balance|balcony|ball|bamboo|banana|banner|bar|barely|bargain|barrel|base|basic|basket|battle|beach|bean|beauty|because|become|beef|before|begin|behave|behind|believe|below|belt|bench|benefit|best|betray|better|between|beyond|bicycle|bid|bike|bind|biology|bird|birth|bitter|black|blade|blame|blanket|blast|bleak|bless|blind|blood|blossom|blouse|blue|blur|blush|board|boat|body|boil|bomb|bone|bonus|book|boost|border|boring|borrow|boss|bottom|bounce|box|boy|bracket|brain|brand|brass|brave|bread|breeze|brick|bridge|brief|bright|bring|brisk|broccoli|broken|bronze|broom|brother|brown|brush|bubble|buddy|budget|buffalo|build|bulb|bulk|bullet|bundle|bunker|burden|burger|burst|bus|business|busy|butter|buyer|buzz)' || true
}

detect_secret_paths() {
    echo "$INPUT" | grep -E '(~\/\.secrets\/|\/\.?secrets?\/|[^\s]*\/(password|token|key|secret)[^\s]*\.[a-z]{2,4})' || true
}

detect_env_vars() {
    echo "$INPUT" | grep -E '^[A-Z_][A-Z0-9_]*=[^\s]{10,}' || true
}

detect_emails_sensitive() {
    # Only flag emails in sensitive contexts (near password, token, etc)
    echo "$INPUT" | grep -iB2 -A2 'password\|token\|secret\|key' | grep -oE '[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}' || true
}

detect_ssn() {
    echo "$INPUT" | grep -oE '\b[0-9]{3}-[0-9]{2}-[0-9]{4}\b' || true
}

detect_credit_cards() {
    echo "$INPUT" | grep -oE '\b[0-9]{4}[\s-]?[0-9]{4}[\s-]?[0-9]{4}[\s-]?[0-9]{4}\b' || true
}

detect_phone_numbers() {
    # Only in sensitive contexts
    echo "$INPUT" | grep -iB2 -A2 'password\|token\|secret\|2fa\|verification' | \
        grep -oE '(\+?1[\s.-]?)?\(?[0-9]{3}\)?[\s.-]?[0-9]{3}[\s.-]?[0-9]{4}' || true
}

# Report finding
report_finding() {
    local severity=$1
    local category=$2
    local matches=$3
    
    if [[ -z "$matches" ]]; then
        return
    fi
    
    case $severity in
        CRITICAL) ((FOUND_CRITICAL++)) ;;
        HIGH) ((FOUND_HIGH++)) ;;
        WARN) ((FOUND_WARN++)) ;;
    esac
    
    if [[ "$SEVERITY_THRESHOLD" == "CRITICAL" && "$severity" != "CRITICAL" ]]; then
        return
    fi
    
    local color=$NC
    case $severity in
        CRITICAL) color=$RED ;;
        HIGH) color=$ORANGE ;;
        WARN) color=$YELLOW ;;
    esac
    
    echo -e "${color}[${severity}]${NC} ${category}" >&2
    echo "$matches" | while IFS= read -r match; do
        # Truncate long matches for display
        if [[ ${#match} -gt 60 ]]; then
            match="${match:0:57}..."
        fi
        echo "  → ${match}" >&2
    done
}

# Redaction function
redact_patterns() {
    local text="$1"
    
    # Redact in order of specificity
    text=$(echo "$text" | sed -E 's/ops_[a-zA-Z0-9_-]{40,}/[REDACTED:1PASSWORD_TOKEN]/g')
    text=$(echo "$text" | sed -E 's/ghp_[a-zA-Z0-9]{32,}/[REDACTED:GITHUB_PAT]/g')
    text=$(echo "$text" | sed -E 's/sk-[a-zA-Z0-9]{20,}/[REDACTED:OPENAI_KEY]/g')
    text=$(echo "$text" | sed -E 's/sk_(test|live)_[a-zA-Z0-9]{24,}/[REDACTED:STRIPE_KEY]/g')
    text=$(echo "$text" | sed -E 's/AKIA[0-9A-Z]{16}/[REDACTED:AWS_KEY]/g')
    text=$(echo "$text" | sed -E 's/Bearer [a-zA-Z0-9_\-\.]{20,}/Bearer [REDACTED:TOKEN]/g')
    text=$(echo "$text" | sed -E 's/(password|passwd|pwd)([[:space:]"]*[:=][[:space:]"]*)[^[:space:]"]{6,}/[REDACTED:PASSWORD]/gI')
    text=$(echo "$text" | sed -E 's/0x[0-9a-fA-F]{64}/[REDACTED:PRIVATE_KEY]/g')
    text=$(echo "$text" | sed -E 's/-----BEGIN (RSA|OPENSSH|DSA|EC) PRIVATE KEY-----/-----BEGIN PRIVATE KEY [REDACTED]-----/g')
    text=$(echo "$text" | sed -E 's/~\/\.secrets\/[^\s]*/[REDACTED:SECRET_PATH]/g')
    text=$(echo "$text" | sed -E 's/[0-9]{3}-[0-9]{2}-[0-9]{4}/[REDACTED:SSN]/g')
    text=$(echo "$text" | sed -E 's/[0-9]{4}[\s-]?[0-9]{4}[\s-]?[0-9]{4}[\s-]?[0-9]{4}/[REDACTED:CC]/g')
    
    echo "$text"
}

# Main detection
main() {
    # Run all detectors
    local findings_1pass=$(detect_1password_tokens)
    local findings_github=$(detect_github_pats)
    local findings_openai=$(detect_openai_keys)
    local findings_stripe=$(detect_stripe_keys)
    local findings_aws=$(detect_aws_keys)
    local findings_bearer=$(detect_bearer_tokens)
    local findings_passwords=$(detect_passwords)
    local findings_eth=$(detect_eth_private_keys)
    local findings_ssh=$(detect_ssh_keys)
    local findings_pgp=$(detect_pgp_blocks)
    local findings_paths=$(detect_secret_paths)
    local findings_env=$(detect_env_vars)
    local findings_ssn=$(detect_ssn)
    local findings_cc=$(detect_credit_cards)
    
    # Report findings
    report_finding "CRITICAL" "1Password Service Account Token" "$findings_1pass"
    report_finding "CRITICAL" "GitHub Personal Access Token" "$findings_github"
    report_finding "CRITICAL" "OpenAI API Key" "$findings_openai"
    report_finding "CRITICAL" "Stripe API Key" "$findings_stripe"
    report_finding "CRITICAL" "AWS Access Key" "$findings_aws"
    report_finding "CRITICAL" "Bearer Token" "$findings_bearer"
    report_finding "CRITICAL" "Password Assignment" "$findings_passwords"
    report_finding "CRITICAL" "Ethereum Private Key" "$findings_eth"
    report_finding "CRITICAL" "SSH Private Key" "$findings_ssh"
    report_finding "CRITICAL" "PGP Private Key" "$findings_pgp"
    report_finding "CRITICAL" "Social Security Number" "$findings_ssn"
    report_finding "HIGH" "Credit Card Number" "$findings_cc"
    report_finding "WARN" "Secret File Path" "$findings_paths"
    report_finding "WARN" "Environment Variable Assignment" "$findings_env"
    
    # Mode-specific output
    if [[ "$MODE" == "report" ]]; then
        echo -e "\n=== ARC-SHIELD SCAN REPORT ===" >&2
        echo -e "CRITICAL: ${FOUND_CRITICAL}" >&2
        echo -e "HIGH: ${FOUND_HIGH}" >&2
        echo -e "WARN: ${FOUND_WARN}" >&2
    elif [[ "$MODE" == "redact" ]]; then
        redact_patterns "$INPUT"
    elif [[ "$MODE" == "strict" ]]; then
        echo "$INPUT"
        if [[ $FOUND_CRITICAL -gt 0 ]]; then
            echo -e "\n${RED}[BLOCKED]${NC} Critical secrets detected. Message blocked." >&2
            exit 1
        fi
    else
        # Default: pass through
        echo "$INPUT"
    fi
    
    # Exit code based on findings
    if [[ $FOUND_CRITICAL -gt 0 && "$MODE" == "strict" ]]; then
        exit 1
    fi
}

main

```

### scripts/output-guard.py

```python
#!/usr/bin/env python3
"""
output-guard.py - Advanced output sanitization with entropy detection
Catches secrets that regex patterns miss using Shannon entropy analysis
"""

import sys
import re
import math
import json
from typing import List, Dict, Tuple
from collections import Counter

VERSION = "1.0.0"

# Severity levels
CRITICAL = "CRITICAL"
HIGH = "HIGH"
WARN = "WARN"

class Finding:
    def __init__(self, severity: str, category: str, value: str, position: int):
        self.severity = severity
        self.category = category
        self.value = value
        self.position = position
    
    def __repr__(self):
        truncated = self.value[:60] + "..." if len(self.value) > 60 else self.value
        return f"[{self.severity}] {self.category}: {truncated}"

class OutputGuard:
    def __init__(self):
        self.findings: List[Finding] = []
        self.mode = "scan"  # scan, redact, report, strict
        
        # Regex patterns (same as bash version)
        self.patterns = {
            CRITICAL: {
                "1Password Token": re.compile(r'ops_[a-zA-Z0-9_-]{40,}'),
                "GitHub PAT": re.compile(r'ghp_[a-zA-Z0-9]{36,}'),
                "OpenAI Key": re.compile(r'sk-[a-zA-Z0-9]{20,}'),
                "Stripe Key": re.compile(r'sk_(test|live)_[a-zA-Z0-9]{24,}'),
                "AWS Key": re.compile(r'AKIA[0-9A-Z]{16}'),
                "Bearer Token": re.compile(r'Bearer [a-zA-Z0-9_\-\.]{20,}'),
                "Password Assignment": re.compile(r'(password|passwd|pwd)["\s]*[:=]["\s]*[^ "\n]{6,}', re.IGNORECASE),
                "Ethereum Private Key": re.compile(r'0x[0-9a-fA-F]{64}'),
                "SSH Private Key": re.compile(r'-----BEGIN (RSA|OPENSSH|DSA|EC) PRIVATE KEY-----'),
                "PGP Private Key": re.compile(r'-----BEGIN PGP PRIVATE KEY BLOCK-----'),
                "SSN": re.compile(r'\b[0-9]{3}-[0-9]{2}-[0-9]{4}\b'),
            },
            HIGH: {
                "Credit Card": re.compile(r'\b[0-9]{4}[\s-]?[0-9]{4}[\s-]?[0-9]{4}[\s-]?[0-9]{4}\b'),
                "High Entropy String": None,  # Handled separately
            },
            WARN: {
                "Secret Path": re.compile(r'(~\/\.secrets\/|\/\.?secrets?\/|[^\s]*\/(password|token|key|secret)[^\s]*\.[a-z]{2,4})'),
                "Environment Variable": re.compile(r'^[A-Z_][A-Z0-9_]*=[^\s]{10,}', re.MULTILINE),
            }
        }
    
    def shannon_entropy(self, data: str) -> float:
        """Calculate Shannon entropy of a string"""
        if not data:
            return 0.0
        
        entropy = 0.0
        counter = Counter(data)
        length = len(data)
        
        for count in counter.values():
            probability = count / length
            if probability > 0:
                entropy -= probability * math.log2(probability)
        
        return entropy
    
    def detect_high_entropy_strings(self, text: str, threshold: float = 4.5, min_length: int = 16) -> List[Tuple[str, int]]:
        """Detect strings with high entropy that might be secrets"""
        findings = []
        
        # Find alphanumeric sequences
        pattern = re.compile(r'[a-zA-Z0-9_\-\.]{16,}')
        
        for match in pattern.finditer(text):
            value = match.group()
            
            # Skip if it looks like normal text (high ratio of common words)
            if self.looks_like_normal_text(value):
                continue
            
            # Skip if it's a URL or file path
            if '/' in value or '.' in value and value.count('.') > 2:
                continue
            
            entropy = self.shannon_entropy(value)
            
            if entropy >= threshold and len(value) >= min_length:
                # Additional heuristics to reduce false positives
                if self.looks_like_secret(value):
                    findings.append((value, match.start()))
        
        return findings
    
    def looks_like_normal_text(self, text: str) -> bool:
        """Check if text looks like normal English rather than a secret"""
        # If it's mostly lowercase with spaces, probably normal text
        if text.islower() and ' ' in text:
            return True
        
        # If it has repeating patterns, might be a UUID or similar
        if len(set(text)) < len(text) * 0.3:  # Less than 30% unique chars
            return False
        
        return False
    
    def looks_like_secret(self, text: str) -> bool:
        """Heuristics to determine if high-entropy string is likely a secret"""
        # Mix of upper, lower, and numbers is common in secrets
        has_upper = any(c.isupper() for c in text)
        has_lower = any(c.islower() for c in text)
        has_digit = any(c.isdigit() for c in text)
        
        # Secrets often have at least 2 of these
        char_types = sum([has_upper, has_lower, has_digit])
        
        if char_types < 2:
            return False
        
        # Common secret prefixes/patterns
        secret_indicators = ['key', 'token', 'secret', 'api', 'auth', 'pass']
        text_lower = text.lower()
        
        # Check if near a secret keyword (would need context, simplified here)
        return char_types >= 2
    
    def scan(self, text: str):
        """Scan text for all secret patterns"""
        # Regex-based detection
        for severity, patterns in self.patterns.items():
            for category, pattern in patterns.items():
                if pattern is None:
                    continue
                
                for match in pattern.finditer(text):
                    finding = Finding(severity, category, match.group(), match.start())
                    self.findings.append(finding)
        
        # Entropy-based detection
        high_entropy = self.detect_high_entropy_strings(text)
        for value, position in high_entropy:
            finding = Finding(HIGH, "High Entropy String", value, position)
            self.findings.append(finding)
    
    def redact(self, text: str) -> str:
        """Redact secrets from text"""
        result = text
        
        # Redact in reverse order to maintain positions
        replacements = []
        
        for severity, patterns in self.patterns.items():
            for category, pattern in patterns.items():
                if pattern is None:
                    continue
                
                for match in pattern.finditer(text):
                    redacted = f"[REDACTED:{category.upper().replace(' ', '_')}]"
                    replacements.append((match.start(), match.end(), redacted))
        
        # Sort by position (reverse) and apply
        replacements.sort(key=lambda x: x[0], reverse=True)
        
        for start, end, redacted in replacements:
            result = result[:start] + redacted + result[end:]
        
        return result
    
    def report(self) -> Dict:
        """Generate findings report"""
        critical = [f for f in self.findings if f.severity == CRITICAL]
        high = [f for f in self.findings if f.severity == HIGH]
        warn = [f for f in self.findings if f.severity == WARN]
        
        return {
            "summary": {
                "critical": len(critical),
                "high": len(high),
                "warn": len(warn),
                "total": len(self.findings)
            },
            "findings": {
                "critical": [str(f) for f in critical],
                "high": [str(f) for f in high],
                "warn": [str(f) for f in warn]
            }
        }
    
    def print_report(self):
        """Print human-readable report to stderr"""
        report = self.report()
        
        print("\n=== OUTPUT-GUARD SCAN REPORT ===", file=sys.stderr)
        print(f"CRITICAL: {report['summary']['critical']}", file=sys.stderr)
        print(f"HIGH: {report['summary']['high']}", file=sys.stderr)
        print(f"WARN: {report['summary']['warn']}", file=sys.stderr)
        
        if report['summary']['critical'] > 0:
            print("\nCRITICAL FINDINGS:", file=sys.stderr)
            for finding in report['findings']['critical']:
                print(f"  {finding}", file=sys.stderr)
        
        if report['summary']['high'] > 0:
            print("\nHIGH FINDINGS:", file=sys.stderr)
            for finding in report['findings']['high']:
                print(f"  {finding}", file=sys.stderr)

def main():
    import argparse
    
    parser = argparse.ArgumentParser(
        description="Output sanitization with entropy detection",
        formatter_class=argparse.RawDescriptionHelpFormatter,
        epilog="""
EXAMPLES:
    # Scan with entropy detection
    echo "secret: sk-abc123xyz456" | output-guard.py
    
    # Redact secrets
    cat response.txt | output-guard.py --redact
    
    # Strict mode (block on critical)
    output-guard.py --strict < message.txt
    
    # JSON report
    output-guard.py --json < conversation.log
        """
    )
    
    parser.add_argument('--redact', action='store_true', help='Redact secrets and output sanitized text')
    parser.add_argument('--strict', action='store_true', help='Exit with code 1 if critical findings detected')
    parser.add_argument('--report', action='store_true', help='Show findings report only')
    parser.add_argument('--json', action='store_true', help='Output report as JSON')
    parser.add_argument('--entropy-threshold', type=float, default=4.5, help='Shannon entropy threshold (default: 4.5)')
    parser.add_argument('--min-length', type=int, default=16, help='Minimum string length for entropy check (default: 16)')
    parser.add_argument('--version', action='version', version=f'%(prog)s {VERSION}')
    
    args = parser.parse_args()
    
    # Read input
    text = sys.stdin.read()
    
    # Initialize guard
    guard = OutputGuard()
    guard.scan(text)
    
    # Mode handling
    if args.report or args.json:
        if args.json:
            print(json.dumps(guard.report(), indent=2))
        else:
            guard.print_report()
    elif args.redact:
        print(guard.redact(text))
    elif args.strict:
        print(text)
        critical_count = len([f for f in guard.findings if f.severity == CRITICAL])
        if critical_count > 0:
            guard.print_report()
            print("\n[BLOCKED] Critical secrets detected. Message blocked.", file=sys.stderr)
            sys.exit(1)
    else:
        # Default: pass through with warnings
        print(text)
        if guard.findings:
            guard.print_report()

if __name__ == "__main__":
    main()

```