example-data-processor

Original source / Raw SKILL.md

---
name: example-data-processor
description: Process CSV data files by cleaning, transforming, and analyzing them. Use this when users need to work with CSV files, clean data, or perform basic data analysis tasks.
---

# Example Data Processor

This skill demonstrates a complete skill structure with scripts, references, and proper documentation.

## What This Skill Does

Processes CSV data files with these capabilities:
- Clean and validate data
- Transform columns
- Generate summary statistics
- Export results

## Usage

### Process a CSV file

To process a CSV file:
```
Process the data in myfile.csv
```

The skill will:
1. Read the CSV file
2. Clean the data (remove nulls, fix formats)
3. Generate statistics
4. Output a summary report

### Custom Processing

For custom processing options:
```
Process sales.csv and group by region
```

## Scripts

**scripts/process_csv.py** - Main data processing script
- Reads CSV files
- Applies transformations
- Generates output

**scripts/fetch_data.py** - API data fetcher (demonstrates uv dependencies)
- Fetches data from APIs using requests
- Beautiful output formatting with rich
- **Auto-installs dependencies** via uv inline metadata (PEP 723)
- No manual pip install needed!

**scripts/validate.py** - Data validation script
- Checks data quality
- Reports issues

## Configuration

The scripts use these environment variables:
- `OUTPUT_DIR` - Where to save processed files (optional)
- `MAX_ROWS` - Maximum rows to process (optional)

Set them using:
```
Set OUTPUT_DIR to /path/to/output
```

## Reference Documentation

For detailed information:
- [Data Formats](references/formats.md) - Supported data formats and schemas
- [Examples](references/examples.md) - Common usage examples

## Troubleshooting

**"File not found" error:**
- Ensure the CSV file exists
- Provide the full path to the file

**"Invalid data" error:**
- Check the CSV format matches expected schema
- See [Data Formats](references/formats.md) for requirements


---

## Referenced Files

> The following files are referenced in this skill and included for context.

### references/formats.md

```markdown
# Data Formats Reference

This document describes the expected data formats for the data processor skill.

## CSV Format

### Basic Requirements

CSV files must:
- Have a header row with column names
- Use comma (,) as the delimiter
- Use double quotes (") for fields containing commas or newlines
- Be UTF-8 encoded

### Example Format

```csv
name,email,age,city
John Doe,[email protected],30,New York
Jane Smith,[email protected],25,San Francisco
Bob Johnson,[email protected],35,Chicago
```

## Supported Column Types

### Text Columns
- Any string data
- Can contain spaces and special characters
- Quotes are automatically handled

### Numeric Columns
- Integers: 1, 42, -5
- Decimals: 3.14, -0.5, 1.234

### Date Columns
Supported formats:
- ISO 8601: 2024-01-15
- US Format: 01/15/2024
- Full: 2024-01-15 14:30:00

## Special Values

### Missing Data
Represented as:
- Empty field: ,,
- NULL text: ,NULL,
- NA text: ,NA,

### Boolean Values
- True: true, TRUE, 1, yes, YES
- False: false, FALSE, 0, no, NO

## File Size Limits

- Maximum file size: 100 MB
- Maximum rows: 1,000,000 (configurable via MAX_ROWS)
- Maximum columns: 1,000

## Common Issues

### Issue: "Invalid CSV format"
**Cause:** File is not properly formatted CSV
**Solution:** Ensure file has comma delimiters and proper headers

### Issue: "Encoding error"
**Cause:** File is not UTF-8 encoded
**Solution:** Convert file to UTF-8 encoding

### Issue: "Too many rows"
**Cause:** File exceeds MAX_ROWS limit
**Solution:** Set MAX_ROWS environment variable higher or split the file

```