Back to skills
SkillHub ClubShip Full StackFull StackTesting
skill-tester
Skill Tester
Packaged view
This page reorganizes the original catalog entry around fit, installability, and workflow context first. The original raw source lives below.
Stars
3,084
Hot score
99
Updated
March 20, 2026
Overall rating
C4.6
Composite score
4.6
Best-practice grade
C64.8
Install command
npx @skill-hub/cli install openclaw-skills-skill-tester
Repository
Best for
Primary workflow: Ship Full Stack.
Technical facets: Full Stack, Testing.
Target audience: everyone.
License: Unknown.
Original source
Catalog source: SkillHub Club.
Repository owner: openclaw.
This is still a mirrored public skill entry. Review the repository before installing into production workflows.
What it helps with
- Install skill-tester into Claude Code, Codex CLI, Gemini CLI, or OpenCode workflows
- Review https://github.com/openclaw/skills before adding skill-tester to shared team environments
- Use skill-tester for development workflows
Works across
Claude CodeCodex CLIGemini CLIOpenCode
Favorites: 0.
Sub-skills: 0.
Aggregator: No.
Original source / Raw SKILL.md
---
name: "skill-tester"
description: "Skill Tester"
---
# Skill Tester
---
**Name**: skill-tester
**Tier**: POWERFUL
**Category**: Engineering Quality Assurance
**Dependencies**: None (Python Standard Library Only)
**Author**: Claude Skills Engineering Team
**Version**: 1.0.0
**Last Updated**: 2026-02-16
---
## Description
The Skill Tester is a comprehensive meta-skill designed to validate, test, and score the quality of skills within the claude-skills ecosystem. This powerful quality assurance tool ensures that all skills meet the rigorous standards required for BASIC, STANDARD, and POWERFUL tier classifications through automated validation, testing, and scoring mechanisms.
As the gatekeeping system for skill quality, this meta-skill provides three core capabilities:
1. **Structure Validation** - Ensures skills conform to required directory structures, file formats, and documentation standards
2. **Script Testing** - Validates Python scripts for syntax, imports, functionality, and output format compliance
3. **Quality Scoring** - Provides comprehensive quality assessment across multiple dimensions with letter grades and improvement recommendations
This skill is essential for maintaining ecosystem consistency, enabling automated CI/CD integration, and supporting both manual and automated quality assurance workflows. It serves as the foundation for pre-commit hooks, pull request validation, and continuous integration processes that maintain the high-quality standards of the claude-skills repository.
## Core Features
### Comprehensive Skill Validation
- **Structure Compliance**: Validates directory structure, required files (SKILL.md, README.md, scripts/, references/, assets/, expected_outputs/)
- **Documentation Standards**: Checks SKILL.md frontmatter, section completeness, minimum line counts per tier
- **File Format Validation**: Ensures proper Markdown formatting, YAML frontmatter syntax, and file naming conventions
### Advanced Script Testing
- **Syntax Validation**: Compiles Python scripts to detect syntax errors before execution
- **Import Analysis**: Enforces standard library only policy, identifies external dependencies
- **Runtime Testing**: Executes scripts with sample data, validates argparse implementation, tests --help functionality
- **Output Format Compliance**: Verifies dual output support (JSON + human-readable), proper error handling
### Multi-Dimensional Quality Scoring
- **Documentation Quality (25%)**: SKILL.md depth and completeness, README clarity, reference documentation quality
- **Code Quality (25%)**: Script complexity, error handling robustness, output format consistency, maintainability
- **Completeness (25%)**: Required directory presence, sample data adequacy, expected output verification
- **Usability (25%)**: Example clarity, argparse help text quality, installation simplicity, user experience
### Tier Classification System
Automatically classifies skills based on complexity and functionality:
#### BASIC Tier Requirements
- Minimum 100 lines in SKILL.md
- At least 1 Python script (100-300 LOC)
- Basic argparse implementation
- Simple input/output handling
- Essential documentation coverage
#### STANDARD Tier Requirements
- Minimum 200 lines in SKILL.md
- 1-2 Python scripts (300-500 LOC each)
- Advanced argparse with subcommands
- JSON + text output formats
- Comprehensive examples and references
- Error handling and edge case management
#### POWERFUL Tier Requirements
- Minimum 300 lines in SKILL.md
- 2-3 Python scripts (500-800 LOC each)
- Complex argparse with multiple modes
- Sophisticated output formatting and validation
- Extensive documentation and reference materials
- Advanced error handling and recovery mechanisms
- CI/CD integration capabilities
## Architecture & Design
### Modular Design Philosophy
The skill-tester follows a modular architecture where each component serves a specific validation purpose:
- **skill_validator.py**: Core structural and documentation validation engine
- **script_tester.py**: Runtime testing and execution validation framework
- **quality_scorer.py**: Multi-dimensional quality assessment and scoring system
### Standards Enforcement
All validation is performed against well-defined standards documented in the references/ directory:
- **Skill Structure Specification**: Defines mandatory and optional components
- **Tier Requirements Matrix**: Detailed requirements for each skill tier
- **Quality Scoring Rubric**: Comprehensive scoring methodology and weightings
### Integration Capabilities
Designed for seamless integration into existing development workflows:
- **Pre-commit Hooks**: Prevents substandard skills from being committed
- **CI/CD Pipelines**: Automated quality gates in pull request workflows
- **Manual Validation**: Interactive command-line tools for development-time validation
- **Batch Processing**: Bulk validation and scoring of existing skill repositories
## Implementation Details
### skill_validator.py Core Functions
```python
# Primary validation workflow
validate_skill_structure() -> ValidationReport
check_skill_md_compliance() -> DocumentationReport
validate_python_scripts() -> ScriptReport
generate_compliance_score() -> float
```
Key validation checks include:
- SKILL.md frontmatter parsing and validation
- Required section presence (Description, Features, Usage, etc.)
- Minimum line count enforcement per tier
- Python script argparse implementation verification
- Standard library import enforcement
- Directory structure compliance
- README.md quality assessment
### script_tester.py Testing Framework
```python
# Core testing functions
syntax_validation() -> SyntaxReport
import_validation() -> ImportReport
runtime_testing() -> RuntimeReport
output_format_validation() -> OutputReport
```
Testing capabilities encompass:
- Python AST-based syntax validation
- Import statement analysis and external dependency detection
- Controlled script execution with timeout protection
- Argparse --help functionality verification
- Sample data processing and output validation
- Expected output comparison and difference reporting
### quality_scorer.py Scoring System
```python
# Multi-dimensional scoring
score_documentation() -> float # 25% weight
score_code_quality() -> float # 25% weight
score_completeness() -> float # 25% weight
score_usability() -> float # 25% weight
calculate_overall_grade() -> str # A-F grade
```
Scoring dimensions include:
- **Documentation**: Completeness, clarity, examples, reference quality
- **Code Quality**: Complexity, maintainability, error handling, output consistency
- **Completeness**: Required files, sample data, expected outputs, test coverage
- **Usability**: Help text quality, example clarity, installation simplicity
## Usage Scenarios
### Development Workflow Integration
```bash
# Pre-commit hook validation
skill_validator.py path/to/skill --tier POWERFUL --json
# Comprehensive skill testing
script_tester.py path/to/skill --timeout 30 --sample-data
# Quality assessment and scoring
quality_scorer.py path/to/skill --detailed --recommendations
```
### CI/CD Pipeline Integration
```yaml
# GitHub Actions workflow example
- name: "validate-skill-quality"
run: |
python skill_validator.py engineering/${{ matrix.skill }} --json | tee validation.json
python script_tester.py engineering/${{ matrix.skill }} | tee testing.json
python quality_scorer.py engineering/${{ matrix.skill }} --json | tee scoring.json
```
### Batch Repository Analysis
```bash
# Validate all skills in repository
find engineering/ -type d -maxdepth 1 | xargs -I {} skill_validator.py {}
# Generate repository quality report
quality_scorer.py engineering/ --batch --output-format json > repo_quality.json
```
## Output Formats & Reporting
### Dual Output Support
All tools provide both human-readable and machine-parseable output:
#### Human-Readable Format
```
=== SKILL VALIDATION REPORT ===
Skill: engineering/example-skill
Tier: STANDARD
Overall Score: 85/100 (B)
Structure Validation: ✓ PASS
├─ SKILL.md: ✓ EXISTS (247 lines)
├─ README.md: ✓ EXISTS
├─ scripts/: ✓ EXISTS (2 files)
└─ references/: ⚠ MISSING (recommended)
Documentation Quality: 22/25 (88%)
Code Quality: 20/25 (80%)
Completeness: 18/25 (72%)
Usability: 21/25 (84%)
Recommendations:
• Add references/ directory with documentation
• Improve error handling in main.py
• Include more comprehensive examples
```
#### JSON Format
```json
{
"skill_path": "engineering/example-skill",
"timestamp": "2026-02-16T16:41:00Z",
"validation_results": {
"structure_compliance": {
"score": 0.95,
"checks": {
"skill_md_exists": true,
"readme_exists": true,
"scripts_directory": true,
"references_directory": false
}
},
"overall_score": 85,
"letter_grade": "B",
"tier_recommendation": "STANDARD",
"improvement_suggestions": [
"Add references/ directory",
"Improve error handling",
"Include comprehensive examples"
]
}
}
```
## Quality Assurance Standards
### Code Quality Requirements
- **Standard Library Only**: No external dependencies (pip packages)
- **Error Handling**: Comprehensive exception handling with meaningful error messages
- **Output Consistency**: Standardized JSON schema and human-readable formatting
- **Performance**: Efficient validation algorithms with reasonable execution time
- **Maintainability**: Clear code structure, comprehensive docstrings, type hints where appropriate
### Testing Standards
- **Self-Testing**: The skill-tester validates itself (meta-validation)
- **Sample Data Coverage**: Comprehensive test cases covering edge cases and error conditions
- **Expected Output Verification**: All sample runs produce verifiable, reproducible outputs
- **Timeout Protection**: Safe execution of potentially problematic scripts with timeout limits
### Documentation Standards
- **Comprehensive Coverage**: All functions, classes, and modules documented
- **Usage Examples**: Clear, practical examples for all use cases
- **Integration Guides**: Step-by-step CI/CD and workflow integration instructions
- **Reference Materials**: Complete specification documents for standards and requirements
## Integration Examples
### Pre-Commit Hook Setup
```bash
#!/bin/bash
# .git/hooks/pre-commit
echo "Running skill validation..."
python engineering/skill-tester/scripts/skill_validator.py engineering/new-skill --tier STANDARD
if [ $? -ne 0 ]; then
echo "Skill validation failed. Commit blocked."
exit 1
fi
echo "Validation passed. Proceeding with commit."
```
### GitHub Actions Workflow
```yaml
name: "skill-quality-gate"
on:
pull_request:
paths: ['engineering/**']
jobs:
validate-skills:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- name: "setup-python"
uses: actions/setup-python@v4
with:
python-version: '3.11'
- name: "validate-changed-skills"
run: |
changed_skills=$(git diff --name-only ${{ github.event.before }} | grep -E '^engineering/[^/]+/' | cut -d'/' -f1-2 | sort -u)
for skill in $changed_skills; do
echo "Validating $skill..."
python engineering/skill-tester/scripts/skill_validator.py $skill --json
python engineering/skill-tester/scripts/script_tester.py $skill
python engineering/skill-tester/scripts/quality_scorer.py $skill --minimum-score 75
done
```
### Continuous Quality Monitoring
```bash
#!/bin/bash
# Daily quality report generation
echo "Generating daily skill quality report..."
timestamp=$(date +"%Y-%m-%d")
python engineering/skill-tester/scripts/quality_scorer.py engineering/ \
--batch --json > "reports/quality_report_${timestamp}.json"
echo "Quality trends analysis..."
python engineering/skill-tester/scripts/trend_analyzer.py reports/ \
--days 30 > "reports/quality_trends_${timestamp}.md"
```
## Performance & Scalability
### Execution Performance
- **Fast Validation**: Structure validation completes in <1 second per skill
- **Efficient Testing**: Script testing with timeout protection (configurable, default 30s)
- **Batch Processing**: Optimized for repository-wide analysis with parallel processing support
- **Memory Efficiency**: Minimal memory footprint for large-scale repository analysis
### Scalability Considerations
- **Repository Size**: Designed to handle repositories with 100+ skills
- **Concurrent Execution**: Thread-safe implementation supports parallel validation
- **Resource Management**: Automatic cleanup of temporary files and subprocess resources
- **Configuration Flexibility**: Configurable timeouts, memory limits, and validation strictness
## Security & Safety
### Safe Execution Environment
- **Sandboxed Testing**: Scripts execute in controlled environment with timeout protection
- **Resource Limits**: Memory and CPU usage monitoring to prevent resource exhaustion
- **Input Validation**: All inputs sanitized and validated before processing
- **No Network Access**: Offline operation ensures no external dependencies or network calls
### Security Best Practices
- **No Code Injection**: Static analysis only, no dynamic code generation
- **Path Traversal Protection**: Secure file system access with path validation
- **Minimal Privileges**: Operates with minimal required file system permissions
- **Audit Logging**: Comprehensive logging for security monitoring and troubleshooting
## Troubleshooting & Support
### Common Issues & Solutions
#### Validation Failures
- **Missing Files**: Check directory structure against tier requirements
- **Import Errors**: Ensure only standard library imports are used
- **Documentation Issues**: Verify SKILL.md frontmatter and section completeness
#### Script Testing Problems
- **Timeout Errors**: Increase timeout limit or optimize script performance
- **Execution Failures**: Check script syntax and import statement validity
- **Output Format Issues**: Ensure proper JSON formatting and dual output support
#### Quality Scoring Discrepancies
- **Low Scores**: Review scoring rubric and improvement recommendations
- **Tier Misclassification**: Verify skill complexity against tier requirements
- **Inconsistent Results**: Check for recent changes in quality standards or scoring weights
### Debugging Support
- **Verbose Mode**: Detailed logging and execution tracing available
- **Dry Run Mode**: Validation without execution for debugging purposes
- **Debug Output**: Comprehensive error reporting with file locations and suggestions
## Future Enhancements
### Planned Features
- **Machine Learning Quality Prediction**: AI-powered quality assessment using historical data
- **Performance Benchmarking**: Execution time and resource usage tracking across skills
- **Dependency Analysis**: Automated detection and validation of skill interdependencies
- **Quality Trend Analysis**: Historical quality tracking and regression detection
### Integration Roadmap
- **IDE Plugins**: Real-time validation in popular development environments
- **Web Dashboard**: Centralized quality monitoring and reporting interface
- **API Endpoints**: RESTful API for external integration and automation
- **Notification Systems**: Automated alerts for quality degradation or validation failures
## Conclusion
The Skill Tester represents a critical infrastructure component for maintaining the high-quality standards of the claude-skills ecosystem. By providing comprehensive validation, testing, and scoring capabilities, it ensures that all skills meet or exceed the rigorous requirements for their respective tiers.
This meta-skill not only serves as a quality gate but also as a development tool that guides skill authors toward best practices and helps maintain consistency across the entire repository. Through its integration capabilities and comprehensive reporting, it enables both manual and automated quality assurance workflows that scale with the growing claude-skills ecosystem.
The combination of structural validation, runtime testing, and multi-dimensional quality scoring provides unparalleled visibility into skill quality while maintaining the flexibility needed for diverse skill types and complexity levels. As the claude-skills repository continues to grow, the Skill Tester will remain the cornerstone of quality assurance and ecosystem integrity.
---
## Skill Companion Files
> Additional files collected from the skill directory layout.
### README.md
```markdown
# Skill Tester - Quality Assurance Meta-Skill
A POWERFUL-tier skill that provides comprehensive validation, testing, and quality scoring for skills in the claude-skills ecosystem.
## Overview
The Skill Tester is a meta-skill that ensures quality and consistency across all skills in the repository through:
- **Structure Validation** - Verifies directory structure, file presence, and documentation standards
- **Script Testing** - Tests Python scripts for syntax, functionality, and compliance
- **Quality Scoring** - Provides comprehensive quality assessment across multiple dimensions
## Quick Start
### Validate a Skill
```bash
# Basic validation
python scripts/skill_validator.py engineering/my-skill
# Validate against specific tier
python scripts/skill_validator.py engineering/my-skill --tier POWERFUL --json
```
### Test Scripts
```bash
# Test all scripts in a skill
python scripts/script_tester.py engineering/my-skill
# Test with custom timeout
python scripts/script_tester.py engineering/my-skill --timeout 60 --json
```
### Score Quality
```bash
# Get quality assessment
python scripts/quality_scorer.py engineering/my-skill
# Detailed scoring with improvement suggestions
python scripts/quality_scorer.py engineering/my-skill --detailed --json
```
## Components
### Scripts
- **skill_validator.py** (700+ LOC) - Validates skill structure and compliance
- **script_tester.py** (800+ LOC) - Tests script functionality and quality
- **quality_scorer.py** (1100+ LOC) - Multi-dimensional quality assessment
### Reference Documentation
- **skill-structure-specification.md** - Complete structural requirements
- **tier-requirements-matrix.md** - Tier-specific quality standards
- **quality-scoring-rubric.md** - Detailed scoring methodology
### Sample Assets
- **sample-skill/** - Complete sample skill for testing the tester itself
## Features
### Validation Capabilities
- SKILL.md format and content validation
- Directory structure compliance checking
- Python script syntax and import validation
- Argparse implementation verification
- Tier-specific requirement enforcement
### Testing Framework
- Syntax validation using AST parsing
- Import analysis for external dependencies
- Runtime execution testing with timeout protection
- Help functionality verification
- Sample data processing validation
- Output format compliance checking
### Quality Assessment
- Documentation quality scoring (25%)
- Code quality evaluation (25%)
- Completeness assessment (25%)
- Usability analysis (25%)
- Letter grade assignment (A+ to F)
- Tier recommendation generation
- Improvement roadmap creation
## CI/CD Integration
### GitHub Actions Example
```yaml
name: Skill Quality Gate
on:
pull_request:
paths: ['engineering/**']
jobs:
validate-skills:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- name: Setup Python
uses: actions/setup-python@v4
with:
python-version: '3.11'
- name: Validate Skills
run: |
for skill in $(git diff --name-only ${{ github.event.before }} | grep -E '^engineering/[^/]+/' | cut -d'/' -f1-2 | sort -u); do
python engineering/skill-tester/scripts/skill_validator.py $skill --json
python engineering/skill-tester/scripts/script_tester.py $skill
python engineering/skill-tester/scripts/quality_scorer.py $skill --minimum-score 75
done
```
### Pre-commit Hook
```bash
#!/bin/bash
# .git/hooks/pre-commit
python engineering/skill-tester/scripts/skill_validator.py engineering/my-skill --tier STANDARD
if [ $? -ne 0 ]; then
echo "Skill validation failed. Commit blocked."
exit 1
fi
```
## Quality Standards
### All Scripts
- **Zero External Dependencies** - Python standard library only
- **Comprehensive Error Handling** - Meaningful error messages and recovery
- **Dual Output Support** - Both JSON and human-readable formats
- **Proper Documentation** - Comprehensive docstrings and comments
- **CLI Best Practices** - Full argparse implementation with help text
### Validation Accuracy
- **Structure Checks** - 100% accurate directory and file validation
- **Content Analysis** - Deep parsing of SKILL.md and documentation
- **Code Analysis** - AST-based Python code validation
- **Compliance Scoring** - Objective, repeatable quality assessment
## Self-Testing
The skill-tester can validate itself:
```bash
# Validate the skill-tester structure
python scripts/skill_validator.py . --tier POWERFUL
# Test the skill-tester scripts
python scripts/script_tester.py .
# Score the skill-tester quality
python scripts/quality_scorer.py . --detailed
```
## Advanced Usage
### Batch Validation
```bash
# Validate all skills in repository
find engineering/ -maxdepth 1 -type d | while read skill; do
echo "Validating $skill..."
python engineering/skill-tester/scripts/skill_validator.py "$skill"
done
```
### Quality Monitoring
```bash
# Generate quality report for all skills
python engineering/skill-tester/scripts/quality_scorer.py engineering/ \
--batch --json > quality_report.json
```
### Custom Scoring Thresholds
```bash
# Enforce minimum quality scores
python scripts/quality_scorer.py engineering/my-skill --minimum-score 80
# Exit code 0 = passed, 1 = failed, 2 = needs improvement
```
## Error Handling
All scripts provide comprehensive error handling:
- **File System Errors** - Missing files, permission issues, invalid paths
- **Content Errors** - Malformed YAML, invalid JSON, encoding issues
- **Execution Errors** - Script timeouts, runtime failures, import errors
- **Validation Errors** - Standards violations, compliance failures
## Output Formats
### Human-Readable
```
=== SKILL VALIDATION REPORT ===
Skill: engineering/my-skill
Overall Score: 85.2/100 (B+)
Tier Recommendation: STANDARD
STRUCTURE VALIDATION:
✓ PASS: SKILL.md found
✓ PASS: README.md found
✓ PASS: scripts/ directory found
SUGGESTIONS:
• Add references/ directory
• Improve error handling in main.py
```
### JSON Format
```json
{
"skill_path": "engineering/my-skill",
"overall_score": 85.2,
"letter_grade": "B+",
"tier_recommendation": "STANDARD",
"dimensions": {
"Documentation": {"score": 88.5, "weight": 0.25},
"Code Quality": {"score": 82.0, "weight": 0.25},
"Completeness": {"score": 85.5, "weight": 0.25},
"Usability": {"score": 84.8, "weight": 0.25}
}
}
```
## Requirements
- **Python 3.7+** - No external dependencies required
- **File System Access** - Read access to skill directories
- **Execution Permissions** - Ability to run Python scripts for testing
## Contributing
See [SKILL.md](SKILL.md) for comprehensive documentation and contribution guidelines.
The skill-tester itself serves as a reference implementation of POWERFUL-tier quality standards.
```
### _meta.json
```json
{
"owner": "alirezarezvani",
"slug": "skill-tester",
"displayName": "Skill Tester",
"latest": {
"version": "2.1.1",
"publishedAt": 1773162038873,
"commit": "https://github.com/openclaw/skills/commit/6947df6da83078c2a05bbd4149d6b98475ae8a53"
},
"history": [
{
"version": "1.0.1",
"publishedAt": 1772590274724,
"commit": "https://github.com/openclaw/skills/commit/e63c3a6fdd58a2c57916d638f4b4d36dee2ab131"
},
{
"version": "1.0.0",
"publishedAt": 1771270596426,
"commit": "https://github.com/openclaw/skills/commit/ea7fbb0eaf6f2e842fdfab7031b10b26ff73bac9"
}
]
}
```
### assets/sample-skill/README.md
```markdown
# Sample Text Processor
A basic text processing skill that demonstrates BASIC tier requirements for the claude-skills ecosystem.
## Quick Start
```bash
# Analyze a text file
python scripts/text_processor.py analyze sample.txt
# Get JSON output
python scripts/text_processor.py analyze sample.txt --format json
# Transform text to uppercase
python scripts/text_processor.py transform sample.txt --mode upper
# Process multiple files
python scripts/text_processor.py batch text_files/ --verbose
```
## Features
- Word count and text statistics
- Text transformations (upper, lower, title, reverse)
- Batch file processing
- JSON and human-readable output formats
- Comprehensive error handling
## Requirements
- Python 3.7 or later
- No external dependencies (standard library only)
## Usage
See [SKILL.md](SKILL.md) for comprehensive documentation and examples.
## Testing
Sample data files are provided in the `assets/` directory for testing the functionality.
```
### assets/sample-skill/SKILL.md
```markdown
# Sample Text Processor
---
**Name**: sample-text-processor
**Tier**: BASIC
**Category**: Text Processing
**Dependencies**: None (Python Standard Library Only)
**Author**: Claude Skills Engineering Team
**Version**: 1.0.0
**Last Updated**: 2026-02-16
---
## Description
The Sample Text Processor is a simple skill designed to demonstrate the basic structure and functionality expected in the claude-skills ecosystem. This skill provides fundamental text processing capabilities including word counting, character analysis, and basic text transformations.
This skill serves as a reference implementation for BASIC tier requirements and can be used as a template for creating new skills. It demonstrates proper file structure, documentation standards, and implementation patterns that align with ecosystem best practices.
The skill processes text files and provides statistics and transformations in both human-readable and JSON formats, showcasing the dual output requirement for skills in the claude-skills repository.
## Features
### Core Functionality
- **Word Count Analysis**: Count total words, unique words, and word frequency
- **Character Statistics**: Analyze character count, line count, and special characters
- **Text Transformations**: Convert text to uppercase, lowercase, or title case
- **File Processing**: Process single text files or batch process directories
- **Dual Output Formats**: Generate results in both JSON and human-readable formats
### Technical Features
- Command-line interface with comprehensive argument parsing
- Error handling for common file and processing issues
- Progress reporting for batch operations
- Configurable output formatting and verbosity levels
- Cross-platform compatibility with standard library only dependencies
## Usage
### Basic Text Analysis
```bash
python text_processor.py analyze document.txt
python text_processor.py analyze document.txt --output results.json
```
### Text Transformation
```bash
python text_processor.py transform document.txt --mode uppercase
python text_processor.py transform document.txt --mode title --output transformed.txt
```
### Batch Processing
```bash
python text_processor.py batch text_files/ --output results/
python text_processor.py batch text_files/ --format json --output batch_results.json
```
## Examples
### Example 1: Basic Word Count
```bash
$ python text_processor.py analyze sample.txt
=== TEXT ANALYSIS RESULTS ===
File: sample.txt
Total words: 150
Unique words: 85
Total characters: 750
Lines: 12
Most frequent word: "the" (8 occurrences)
```
### Example 2: JSON Output
```bash
$ python text_processor.py analyze sample.txt --format json
{
"file": "sample.txt",
"statistics": {
"total_words": 150,
"unique_words": 85,
"total_characters": 750,
"lines": 12,
"most_frequent": {
"word": "the",
"count": 8
}
}
}
```
### Example 3: Text Transformation
```bash
$ python text_processor.py transform sample.txt --mode title
Original: "hello world from the text processor"
Transformed: "Hello World From The Text Processor"
```
## Installation
This skill requires only Python 3.7 or later with the standard library. No external dependencies are required.
1. Clone or download the skill directory
2. Navigate to the scripts directory
3. Run the text processor directly with Python
```bash
cd scripts/
python text_processor.py --help
```
## Configuration
The text processor supports various configuration options through command-line arguments:
- `--format`: Output format (json, text)
- `--verbose`: Enable verbose output and progress reporting
- `--output`: Specify output file or directory
- `--encoding`: Specify text file encoding (default: utf-8)
## Architecture
The skill follows a simple modular architecture:
- **TextProcessor Class**: Core processing logic and statistics calculation
- **OutputFormatter Class**: Handles dual output format generation
- **FileManager Class**: Manages file I/O operations and batch processing
- **CLI Interface**: Command-line argument parsing and user interaction
## Error Handling
The skill includes comprehensive error handling for:
- File not found or permission errors
- Invalid encoding or corrupted text files
- Memory limitations for very large files
- Output directory creation and write permissions
- Invalid command-line arguments and parameters
## Performance Considerations
- Efficient memory usage for large text files through streaming
- Optimized word counting using dictionary lookups
- Batch processing with progress reporting for large datasets
- Configurable encoding detection for international text
## Contributing
This skill serves as a reference implementation and contributions are welcome to demonstrate best practices:
1. Follow PEP 8 coding standards
2. Include comprehensive docstrings
3. Add test cases with sample data
4. Update documentation for any new features
5. Ensure backward compatibility
## Limitations
As a BASIC tier skill, some advanced features are intentionally omitted:
- Complex text analysis (sentiment, language detection)
- Advanced file format support (PDF, Word documents)
- Database integration or external API calls
- Parallel processing for very large datasets
This skill demonstrates the essential structure and quality standards required for BASIC tier skills in the claude-skills ecosystem while remaining simple and focused on core functionality.
```
### assets/sample-skill/assets/sample_text.txt
```text
This is a sample text file for testing the text processor skill.
It contains multiple lines of text with various words and punctuation.
The quick brown fox jumps over the lazy dog.
This sentence contains all 26 letters of the English alphabet.
Some additional content:
- Numbers: 123, 456, 789
- Special characters: !@#$%^&*()
- Mixed case: CamelCase, snake_case, PascalCase
Lorem ipsum dolor sit amet, consectetur adipiscing elit.
Sed do eiusmod tempor incididunt ut labore et dolore magna aliqua.
Ut enim ad minim veniam, quis nostrud exercitation ullamco.
This file serves as a basic test case for:
1. Word counting functionality
2. Character analysis
3. Line counting
4. Text transformations
5. Statistical analysis
The text processor should handle this content correctly and produce
meaningful statistics and transformations for testing purposes.
```
### assets/sample-skill/expected_outputs/sample_text_analysis.json
```json
{
"file": "assets/sample_text.txt",
"file_size": 855,
"total_words": 116,
"unique_words": 87,
"total_characters": 855,
"lines": 19,
"average_word_length": 4.7,
"most_frequent": {
"word": "the",
"count": 5
}
}
```
### assets/sample-skill/references/api-reference.md
```markdown
# Text Processor API Reference
## Classes
### TextProcessor
Main class for text processing operations.
#### `__init__(self, encoding: str = 'utf-8')`
Initialize the text processor with specified encoding.
**Parameters:**
- `encoding` (str): Character encoding for file operations. Default: 'utf-8'
#### `analyze_text(self, text: str) -> Dict[str, Any]`
Analyze text and return comprehensive statistics.
**Parameters:**
- `text` (str): Text content to analyze
**Returns:**
- `dict`: Statistics including word count, character count, lines, most frequent word
**Example:**
```python
processor = TextProcessor()
stats = processor.analyze_text("Hello world")
# Returns: {'total_words': 2, 'unique_words': 2, ...}
```
#### `transform_text(self, text: str, mode: str) -> str`
Transform text according to specified mode.
**Parameters:**
- `text` (str): Text to transform
- `mode` (str): Transformation mode ('upper', 'lower', 'title', 'reverse')
**Returns:**
- `str`: Transformed text
**Raises:**
- `ValueError`: If mode is not supported
### OutputFormatter
Static methods for output formatting.
#### `format_json(data: Dict[str, Any]) -> str`
Format data as JSON string.
#### `format_human_readable(data: Dict[str, Any]) -> str`
Format data as human-readable text.
### FileManager
Handles file operations and batch processing.
#### `find_text_files(self, directory: str) -> List[str]`
Find all text files in a directory recursively.
**Supported Extensions:**
- .txt
- .md
- .rst
- .csv
- .log
## Command Line Interface
### Commands
#### `analyze`
Analyze text file statistics.
```bash
python text_processor.py analyze <file> [options]
```
#### `transform`
Transform text file content.
```bash
python text_processor.py transform <file> --mode <mode> [options]
```
#### `batch`
Process multiple files in a directory.
```bash
python text_processor.py batch <directory> [options]
```
### Global Options
- `--format {json,text}`: Output format (default: text)
- `--output FILE`: Output file path (default: stdout)
- `--encoding ENCODING`: Text file encoding (default: utf-8)
- `--verbose`: Enable verbose output
## Error Handling
The text processor handles several error conditions:
- **FileNotFoundError**: When input file doesn't exist
- **UnicodeDecodeError**: When file encoding doesn't match specified encoding
- **PermissionError**: When file access is denied
- **ValueError**: When invalid transformation mode is specified
All errors are reported to stderr with descriptive messages.
```
### assets/sample-skill/scripts/text_processor.py
```python
#!/usr/bin/env python3
"""
Sample Text Processor - Basic text analysis and transformation tool
This script demonstrates the basic structure and functionality expected in
BASIC tier skills. It provides text processing capabilities with proper
argument parsing, error handling, and dual output formats.
Usage:
python text_processor.py analyze <file> [options]
python text_processor.py transform <file> --mode <mode> [options]
python text_processor.py batch <directory> [options]
Author: Claude Skills Engineering Team
Version: 1.0.0
Dependencies: Python Standard Library Only
"""
import argparse
import json
import os
import sys
from collections import Counter
from pathlib import Path
from typing import Dict, List, Any, Optional
class TextProcessor:
"""Core text processing functionality"""
def __init__(self, encoding: str = 'utf-8'):
self.encoding = encoding
def analyze_text(self, text: str) -> Dict[str, Any]:
"""Analyze text and return statistics"""
lines = text.split('\n')
words = text.lower().split()
# Calculate basic statistics
stats = {
'total_words': len(words),
'unique_words': len(set(words)),
'total_characters': len(text),
'lines': len(lines),
'average_word_length': sum(len(word) for word in words) / len(words) if words else 0
}
# Find most frequent word
if words:
word_counts = Counter(words)
most_common = word_counts.most_common(1)[0]
stats['most_frequent'] = {
'word': most_common[0],
'count': most_common[1]
}
else:
stats['most_frequent'] = {'word': '', 'count': 0}
return stats
def transform_text(self, text: str, mode: str) -> str:
"""Transform text according to specified mode"""
if mode == 'upper':
return text.upper()
elif mode == 'lower':
return text.lower()
elif mode == 'title':
return text.title()
elif mode == 'reverse':
return text[::-1]
else:
raise ValueError(f"Unknown transformation mode: {mode}")
def process_file(self, file_path: str) -> Dict[str, Any]:
"""Process a single text file"""
try:
with open(file_path, 'r', encoding=self.encoding) as file:
content = file.read()
stats = self.analyze_text(content)
stats['file'] = file_path
stats['file_size'] = os.path.getsize(file_path)
return stats
except FileNotFoundError:
raise FileNotFoundError(f"File not found: {file_path}")
except UnicodeDecodeError:
raise UnicodeDecodeError(f"Cannot decode file with {self.encoding} encoding: {file_path}")
except PermissionError:
raise PermissionError(f"Permission denied accessing file: {file_path}")
class OutputFormatter:
"""Handles dual output format generation"""
@staticmethod
def format_json(data: Dict[str, Any]) -> str:
"""Format data as JSON"""
return json.dumps(data, indent=2, ensure_ascii=False)
@staticmethod
def format_human_readable(data: Dict[str, Any]) -> str:
"""Format data as human-readable text"""
lines = []
lines.append("=== TEXT ANALYSIS RESULTS ===")
lines.append(f"File: {data.get('file', 'Unknown')}")
lines.append(f"File size: {data.get('file_size', 0)} bytes")
lines.append(f"Total words: {data.get('total_words', 0)}")
lines.append(f"Unique words: {data.get('unique_words', 0)}")
lines.append(f"Total characters: {data.get('total_characters', 0)}")
lines.append(f"Lines: {data.get('lines', 0)}")
lines.append(f"Average word length: {data.get('average_word_length', 0):.1f}")
most_frequent = data.get('most_frequent', {})
lines.append(f"Most frequent word: \"{most_frequent.get('word', '')}\" ({most_frequent.get('count', 0)} occurrences)")
return "\n".join(lines)
class FileManager:
"""Manages file I/O operations and batch processing"""
def __init__(self, verbose: bool = False):
self.verbose = verbose
def log_verbose(self, message: str):
"""Log verbose message if verbose mode enabled"""
if self.verbose:
print(f"[INFO] {message}", file=sys.stderr)
def find_text_files(self, directory: str) -> List[str]:
"""Find all text files in directory"""
text_extensions = {'.txt', '.md', '.rst', '.csv', '.log'}
text_files = []
try:
for file_path in Path(directory).rglob('*'):
if file_path.is_file() and file_path.suffix.lower() in text_extensions:
text_files.append(str(file_path))
except PermissionError:
raise PermissionError(f"Permission denied accessing directory: {directory}")
return text_files
def write_output(self, content: str, output_path: Optional[str] = None):
"""Write content to file or stdout"""
if output_path:
try:
# Create directory if needed
output_dir = os.path.dirname(output_path)
if output_dir and not os.path.exists(output_dir):
os.makedirs(output_dir)
with open(output_path, 'w', encoding='utf-8') as file:
file.write(content)
self.log_verbose(f"Output written to: {output_path}")
except PermissionError:
raise PermissionError(f"Permission denied writing to: {output_path}")
else:
print(content)
def analyze_command(args: argparse.Namespace) -> int:
"""Handle analyze command"""
try:
processor = TextProcessor(args.encoding)
file_manager = FileManager(args.verbose)
file_manager.log_verbose(f"Analyzing file: {args.file}")
# Process the file
results = processor.process_file(args.file)
# Format output
if args.format == 'json':
output = OutputFormatter.format_json(results)
else:
output = OutputFormatter.format_human_readable(results)
# Write output
file_manager.write_output(output, args.output)
return 0
except FileNotFoundError as e:
print(f"Error: {e}", file=sys.stderr)
return 1
except UnicodeDecodeError as e:
print(f"Error: {e}", file=sys.stderr)
print(f"Try using --encoding option with different encoding", file=sys.stderr)
return 1
except Exception as e:
print(f"Error: {e}", file=sys.stderr)
return 1
def transform_command(args: argparse.Namespace) -> int:
"""Handle transform command"""
try:
processor = TextProcessor(args.encoding)
file_manager = FileManager(args.verbose)
file_manager.log_verbose(f"Transforming file: {args.file}")
# Read and transform the file
with open(args.file, 'r', encoding=args.encoding) as file:
content = file.read()
transformed = processor.transform_text(content, args.mode)
# Write transformed content
file_manager.write_output(transformed, args.output)
return 0
except FileNotFoundError as e:
print(f"Error: {e}", file=sys.stderr)
return 1
except ValueError as e:
print(f"Error: {e}", file=sys.stderr)
return 1
except Exception as e:
print(f"Error: {e}", file=sys.stderr)
return 1
def batch_command(args: argparse.Namespace) -> int:
"""Handle batch command"""
try:
processor = TextProcessor(args.encoding)
file_manager = FileManager(args.verbose)
file_manager.log_verbose(f"Finding text files in: {args.directory}")
# Find all text files
text_files = file_manager.find_text_files(args.directory)
if not text_files:
print(f"No text files found in directory: {args.directory}", file=sys.stderr)
return 1
file_manager.log_verbose(f"Found {len(text_files)} text files")
# Process all files
all_results = []
for i, file_path in enumerate(text_files, 1):
try:
file_manager.log_verbose(f"Processing {i}/{len(text_files)}: {file_path}")
results = processor.process_file(file_path)
all_results.append(results)
except Exception as e:
print(f"Warning: Failed to process {file_path}: {e}", file=sys.stderr)
continue
if not all_results:
print("Error: No files could be processed successfully", file=sys.stderr)
return 1
# Format batch results
batch_summary = {
'total_files': len(all_results),
'total_words': sum(r.get('total_words', 0) for r in all_results),
'total_characters': sum(r.get('total_characters', 0) for r in all_results),
'files': all_results
}
if args.format == 'json':
output = OutputFormatter.format_json(batch_summary)
else:
lines = []
lines.append("=== BATCH PROCESSING RESULTS ===")
lines.append(f"Total files processed: {batch_summary['total_files']}")
lines.append(f"Total words across all files: {batch_summary['total_words']}")
lines.append(f"Total characters across all files: {batch_summary['total_characters']}")
lines.append("")
lines.append("Individual file results:")
for result in all_results:
lines.append(f" {result['file']}: {result['total_words']} words")
output = "\n".join(lines)
# Write output
file_manager.write_output(output, args.output)
return 0
except PermissionError as e:
print(f"Error: {e}", file=sys.stderr)
return 1
except Exception as e:
print(f"Error: {e}", file=sys.stderr)
return 1
def main():
"""Main entry point with argument parsing"""
parser = argparse.ArgumentParser(
description="Sample Text Processor - Basic text analysis and transformation",
formatter_class=argparse.RawDescriptionHelpFormatter,
epilog="""
Examples:
Analysis:
python text_processor.py analyze document.txt
python text_processor.py analyze document.txt --format json --output results.json
Transformation:
python text_processor.py transform document.txt --mode upper
python text_processor.py transform document.txt --mode title --output transformed.txt
Batch processing:
python text_processor.py batch text_files/ --verbose
python text_processor.py batch text_files/ --format json --output batch_results.json
Transformation modes:
upper - Convert to uppercase
lower - Convert to lowercase
title - Convert to title case
reverse - Reverse the text
"""
)
parser.add_argument('--format',
choices=['json', 'text'],
default='text',
help='Output format (default: text)')
parser.add_argument('--output',
help='Output file path (default: stdout)')
parser.add_argument('--encoding',
default='utf-8',
help='Text file encoding (default: utf-8)')
parser.add_argument('--verbose',
action='store_true',
help='Enable verbose output')
subparsers = parser.add_subparsers(dest='command', help='Available commands')
# Analyze subcommand
analyze_parser = subparsers.add_parser('analyze', help='Analyze text file statistics')
analyze_parser.add_argument('file', help='Text file to analyze')
# Transform subcommand
transform_parser = subparsers.add_parser('transform', help='Transform text file')
transform_parser.add_argument('file', help='Text file to transform')
transform_parser.add_argument('--mode',
required=True,
choices=['upper', 'lower', 'title', 'reverse'],
help='Transformation mode')
# Batch subcommand
batch_parser = subparsers.add_parser('batch', help='Process multiple files')
batch_parser.add_argument('directory', help='Directory containing text files')
args = parser.parse_args()
if not args.command:
parser.print_help()
return 1
try:
if args.command == 'analyze':
return analyze_command(args)
elif args.command == 'transform':
return transform_command(args)
elif args.command == 'batch':
return batch_command(args)
else:
print(f"Unknown command: {args.command}", file=sys.stderr)
return 1
except KeyboardInterrupt:
print("\nOperation interrupted by user", file=sys.stderr)
return 130
except Exception as e:
print(f"Unexpected error: {e}", file=sys.stderr)
return 1
if __name__ == "__main__":
sys.exit(main())
```
### references/quality-scoring-rubric.md
```markdown
# Quality Scoring Rubric
**Version**: 1.0.0
**Last Updated**: 2026-02-16
**Authority**: Claude Skills Engineering Team
## Overview
This document defines the comprehensive quality scoring methodology used to assess skills within the claude-skills ecosystem. The scoring system evaluates four key dimensions, each weighted equally at 25%, to provide an objective and consistent measure of skill quality.
## Scoring Framework
### Overall Scoring Scale
- **A+ (95-100)**: Exceptional quality, exceeds all standards
- **A (90-94)**: Excellent quality, meets highest standards consistently
- **A- (85-89)**: Very good quality, minor areas for improvement
- **B+ (80-84)**: Good quality, meets most standards well
- **B (75-79)**: Satisfactory quality, meets standards adequately
- **B- (70-74)**: Below average, several areas need improvement
- **C+ (65-69)**: Poor quality, significant improvements needed
- **C (60-64)**: Minimal acceptable quality, major improvements required
- **C- (55-59)**: Unacceptable quality, extensive rework needed
- **D (50-54)**: Very poor quality, fundamental issues present
- **F (0-49)**: Failing quality, does not meet basic standards
### Dimension Weights
Each dimension contributes equally to the overall score:
- **Documentation Quality**: 25%
- **Code Quality**: 25%
- **Completeness**: 25%
- **Usability**: 25%
## Documentation Quality (25% Weight)
### Scoring Components
#### SKILL.md Quality (40% of Documentation Score)
**Component Breakdown:**
- **Length and Depth (25%)**: Line count and content substance
- **Frontmatter Quality (25%)**: Completeness and accuracy of YAML metadata
- **Section Coverage (25%)**: Required and recommended section presence
- **Content Depth (25%)**: Technical detail and comprehensiveness
**Scoring Criteria:**
| Score Range | Length | Frontmatter | Sections | Depth |
|-------------|--------|-------------|----------|-------|
| 90-100 | 400+ lines | All fields complete + extras | All required + 4+ recommended | Rich technical detail, examples |
| 80-89 | 300-399 lines | All required fields complete | All required + 2-3 recommended | Good technical coverage |
| 70-79 | 200-299 lines | Most required fields | All required + 1 recommended | Adequate technical content |
| 60-69 | 150-199 lines | Some required fields | Most required sections | Basic technical information |
| 50-59 | 100-149 lines | Minimal frontmatter | Some required sections | Limited technical detail |
| Below 50 | <100 lines | Missing/invalid frontmatter | Few/no required sections | Insufficient content |
#### README.md Quality (25% of Documentation Score)
**Scoring Criteria:**
- **Excellent (90-100)**: 1000+ chars, comprehensive usage guide, examples, troubleshooting
- **Good (75-89)**: 500-999 chars, clear usage instructions, basic examples
- **Satisfactory (60-74)**: 200-499 chars, minimal usage information
- **Poor (40-59)**: <200 chars or confusing content
- **Failing (0-39)**: Missing or completely inadequate
#### Reference Documentation (20% of Documentation Score)
**Scoring Criteria:**
- **Excellent (90-100)**: Multiple comprehensive reference docs (2000+ chars total)
- **Good (75-89)**: 2-3 reference files with substantial content
- **Satisfactory (60-74)**: 1-2 reference files with adequate content
- **Poor (40-59)**: Minimal reference content or poor quality
- **Failing (0-39)**: No reference documentation
#### Examples and Usage Clarity (15% of Documentation Score)
**Scoring Criteria:**
- **Excellent (90-100)**: 5+ diverse examples, clear usage patterns
- **Good (75-89)**: 3-4 examples covering different scenarios
- **Satisfactory (60-74)**: 2-3 basic examples
- **Poor (40-59)**: 1-2 minimal examples
- **Failing (0-39)**: No examples or unclear usage
## Code Quality (25% Weight)
### Scoring Components
#### Script Complexity and Architecture (25% of Code Score)
**Evaluation Criteria:**
- Lines of code per script relative to tier requirements
- Function and class organization
- Code modularity and reusability
- Algorithm sophistication
**Scoring Matrix:**
| Tier | Excellent (90-100) | Good (75-89) | Satisfactory (60-74) | Poor (Below 60) |
|------|-------------------|--------------|---------------------|-----------------|
| BASIC | 200-300 LOC, well-structured | 150-199 LOC, organized | 100-149 LOC, basic | <100 LOC, minimal |
| STANDARD | 400-500 LOC, modular | 350-399 LOC, structured | 300-349 LOC, adequate | <300 LOC, basic |
| POWERFUL | 600-800 LOC, sophisticated | 550-599 LOC, advanced | 500-549 LOC, solid | <500 LOC, simple |
#### Error Handling Quality (25% of Code Score)
**Scoring Criteria:**
- **Excellent (90-100)**: Comprehensive exception handling, specific error types, recovery mechanisms
- **Good (75-89)**: Good exception handling, meaningful error messages, logging
- **Satisfactory (60-74)**: Basic try/except blocks, simple error messages
- **Poor (40-59)**: Minimal error handling, generic exceptions
- **Failing (0-39)**: No error handling or inappropriate handling
**Error Handling Checklist:**
- [ ] Try/except blocks for risky operations
- [ ] Specific exception types (not just Exception)
- [ ] Meaningful error messages for users
- [ ] Proper error logging or reporting
- [ ] Graceful degradation where possible
- [ ] Input validation and sanitization
#### Code Structure and Organization (25% of Code Score)
**Evaluation Elements:**
- Function decomposition and single responsibility
- Class design and inheritance patterns
- Import organization and dependency management
- Documentation and comments quality
- Consistent naming conventions
- PEP 8 compliance
**Scoring Guidelines:**
- **Excellent (90-100)**: Exemplary structure, comprehensive docstrings, perfect style
- **Good (75-89)**: Well-organized, good documentation, minor style issues
- **Satisfactory (60-74)**: Adequate structure, basic documentation, some style issues
- **Poor (40-59)**: Poor organization, minimal documentation, style problems
- **Failing (0-39)**: No clear structure, no documentation, major style violations
#### Output Format Support (25% of Code Score)
**Required Capabilities:**
- JSON output format support
- Human-readable output format
- Proper data serialization
- Consistent output structure
- Error output handling
**Scoring Criteria:**
- **Excellent (90-100)**: Dual format + custom formats, perfect serialization
- **Good (75-89)**: Dual format support, good serialization
- **Satisfactory (60-74)**: Single format well-implemented
- **Poor (40-59)**: Basic output, formatting issues
- **Failing (0-39)**: Poor or no structured output
## Completeness (25% Weight)
### Scoring Components
#### Directory Structure Compliance (25% of Completeness Score)
**Required Directories by Tier:**
- **BASIC**: scripts/ (required), assets/ + references/ (recommended)
- **STANDARD**: scripts/ + assets/ + references/ (required), expected_outputs/ (recommended)
- **POWERFUL**: scripts/ + assets/ + references/ + expected_outputs/ (all required)
**Scoring Calculation:**
```
Structure Score = (Required Present / Required Total) * 0.6 +
(Recommended Present / Recommended Total) * 0.4
```
#### Asset Availability and Quality (25% of Completeness Score)
**Scoring Criteria:**
- **Excellent (90-100)**: 5+ diverse assets, multiple file types, realistic data
- **Good (75-89)**: 3-4 assets, some diversity, good quality
- **Satisfactory (60-74)**: 2-3 assets, basic variety
- **Poor (40-59)**: 1-2 minimal assets
- **Failing (0-39)**: No assets or unusable assets
**Asset Quality Factors:**
- File diversity (JSON, CSV, YAML, etc.)
- Data realism and complexity
- Coverage of use cases
- File size appropriateness
- Documentation of asset purpose
#### Expected Output Coverage (25% of Completeness Score)
**Evaluation Criteria:**
- Correspondence with asset files
- Coverage of success and error scenarios
- Output format variety
- Reproducibility and accuracy
**Scoring Matrix:**
- **Excellent (90-100)**: Complete output coverage, all scenarios, verified accuracy
- **Good (75-89)**: Good coverage, most scenarios, mostly accurate
- **Satisfactory (60-74)**: Basic coverage, main scenarios
- **Poor (40-59)**: Minimal coverage, some inaccuracies
- **Failing (0-39)**: No expected outputs or completely inaccurate
#### Test Coverage and Validation (25% of Completeness Score)
**Assessment Areas:**
- Sample data processing capability
- Output verification mechanisms
- Edge case handling
- Error condition testing
- Integration test scenarios
**Scoring Guidelines:**
- **Excellent (90-100)**: Comprehensive test coverage, automated validation
- **Good (75-89)**: Good test coverage, manual validation possible
- **Satisfactory (60-74)**: Basic testing capability
- **Poor (40-59)**: Minimal testing support
- **Failing (0-39)**: No testing or validation capability
## Usability (25% Weight)
### Scoring Components
#### Installation and Setup Simplicity (25% of Usability Score)
**Evaluation Factors:**
- Dependency requirements (Python stdlib preferred)
- Setup complexity
- Environment requirements
- Installation documentation clarity
**Scoring Criteria:**
- **Excellent (90-100)**: Zero external dependencies, single-file execution
- **Good (75-89)**: Minimal dependencies, simple setup
- **Satisfactory (60-74)**: Some dependencies, documented setup
- **Poor (40-59)**: Complex dependencies, unclear setup
- **Failing (0-39)**: Unable to install or excessive complexity
#### Usage Clarity and Help Quality (25% of Usability Score)
**Assessment Elements:**
- Command-line help comprehensiveness
- Usage example clarity
- Parameter documentation quality
- Error message helpfulness
**Help Quality Checklist:**
- [ ] Comprehensive --help output
- [ ] Clear parameter descriptions
- [ ] Usage examples included
- [ ] Error messages are actionable
- [ ] Progress indicators where appropriate
**Scoring Matrix:**
- **Excellent (90-100)**: Exemplary help, multiple examples, perfect error messages
- **Good (75-89)**: Good help quality, clear examples, helpful errors
- **Satisfactory (60-74)**: Adequate help, basic examples
- **Poor (40-59)**: Minimal help, confusing interface
- **Failing (0-39)**: No help or completely unclear interface
#### Documentation Accessibility (25% of Usability Score)
**Evaluation Criteria:**
- README quick start effectiveness
- SKILL.md navigation and structure
- Reference material organization
- Learning curve considerations
**Accessibility Factors:**
- Information hierarchy clarity
- Cross-reference quality
- Beginner-friendly explanations
- Advanced user shortcuts
- Troubleshooting guidance
#### Practical Example Quality (25% of Usability Score)
**Assessment Areas:**
- Example realism and relevance
- Complexity progression (simple to advanced)
- Output demonstration
- Common use case coverage
- Integration scenarios
**Scoring Guidelines:**
- **Excellent (90-100)**: 5+ examples, perfect progression, real-world scenarios
- **Good (75-89)**: 3-4 examples, good variety, practical scenarios
- **Satisfactory (60-74)**: 2-3 examples, adequate coverage
- **Poor (40-59)**: 1-2 examples, limited practical value
- **Failing (0-39)**: No examples or completely impractical
## Scoring Calculations
### Dimension Score Calculation
Each dimension score is calculated as a weighted average of its components:
```python
def calculate_dimension_score(components):
total_weighted_score = 0
total_weight = 0
for component_name, component_data in components.items():
score = component_data['score']
weight = component_data['weight']
total_weighted_score += score * weight
total_weight += weight
return total_weighted_score / total_weight if total_weight > 0 else 0
```
### Overall Score Calculation
The overall score combines all dimensions with equal weighting:
```python
def calculate_overall_score(dimensions):
return sum(dimension.score * 0.25 for dimension in dimensions.values())
```
### Letter Grade Assignment
```python
def assign_letter_grade(overall_score):
if overall_score >= 95: return "A+"
elif overall_score >= 90: return "A"
elif overall_score >= 85: return "A-"
elif overall_score >= 80: return "B+"
elif overall_score >= 75: return "B"
elif overall_score >= 70: return "B-"
elif overall_score >= 65: return "C+"
elif overall_score >= 60: return "C"
elif overall_score >= 55: return "C-"
elif overall_score >= 50: return "D"
else: return "F"
```
## Quality Improvement Recommendations
### Score-Based Recommendations
#### For Scores Below 60 (C- or Lower)
**Priority Actions:**
1. Address fundamental structural issues
2. Implement basic error handling
3. Add essential documentation sections
4. Create minimal viable examples
5. Fix critical functionality issues
#### For Scores 60-74 (C+ to B-)
**Improvement Areas:**
1. Expand documentation comprehensiveness
2. Enhance error handling sophistication
3. Add more diverse examples and use cases
4. Improve code organization and structure
5. Increase test coverage and validation
#### For Scores 75-84 (B to B+)
**Enhancement Opportunities:**
1. Refine documentation for expert-level quality
2. Implement advanced error recovery mechanisms
3. Add comprehensive reference materials
4. Optimize code architecture and performance
5. Develop extensive example library
#### For Scores 85+ (A- or Higher)
**Excellence Maintenance:**
1. Regular quality audits and updates
2. Community feedback integration
3. Best practice evolution tracking
4. Mentoring lower-quality skills
5. Innovation and cutting-edge feature adoption
### Dimension-Specific Improvement Strategies
#### Low Documentation Scores
- Expand SKILL.md with technical details
- Add comprehensive API reference
- Include architecture diagrams and explanations
- Develop troubleshooting guides
- Create contributor documentation
#### Low Code Quality Scores
- Refactor for better modularity
- Implement comprehensive error handling
- Add extensive code documentation
- Apply advanced design patterns
- Optimize performance and efficiency
#### Low Completeness Scores
- Add missing directories and files
- Develop comprehensive sample datasets
- Create expected output libraries
- Implement automated testing
- Add integration examples
#### Low Usability Scores
- Simplify installation process
- Improve command-line interface design
- Enhance help text and documentation
- Create beginner-friendly tutorials
- Add interactive examples
## Quality Assurance Process
### Automated Scoring
The quality scorer runs automated assessments based on this rubric:
1. File system analysis for structure compliance
2. Content analysis for documentation quality
3. Code analysis for quality metrics
4. Asset inventory and quality assessment
### Manual Review Process
Human reviewers validate automated scores and provide qualitative insights:
1. Content quality assessment beyond automated metrics
2. Usability testing with real-world scenarios
3. Technical accuracy verification
4. Community value assessment
### Continuous Improvement
The scoring rubric evolves based on:
- Community feedback and usage patterns
- Industry best practice changes
- Tool capability enhancements
- Quality trend analysis
This quality scoring rubric ensures consistent, objective, and comprehensive assessment of all skills within the claude-skills ecosystem while providing clear guidance for quality improvement.
```
### references/skill-structure-specification.md
```markdown
# Skill Structure Specification
**Version**: 1.0.0
**Last Updated**: 2026-02-16
**Authority**: Claude Skills Engineering Team
## Overview
This document defines the mandatory and optional components that constitute a well-formed skill within the claude-skills ecosystem. All skills must adhere to these structural requirements to ensure consistency, maintainability, and quality across the repository.
## Directory Structure
### Mandatory Components
```
skill-name/
├── SKILL.md # Primary skill documentation (REQUIRED)
├── README.md # Usage instructions and quick start (REQUIRED)
└── scripts/ # Python implementation scripts (REQUIRED)
└── *.py # At least one Python script
```
### Recommended Components
```
skill-name/
├── SKILL.md
├── README.md
├── scripts/
│ └── *.py
├── assets/ # Sample data and input files (RECOMMENDED)
│ ├── samples/
│ ├── examples/
│ └── data/
├── references/ # Reference documentation (RECOMMENDED)
│ ├── api-reference.md
│ ├── specifications.md
│ └── external-links.md
└── expected_outputs/ # Expected results for testing (RECOMMENDED)
├── sample_output.json
├── example_results.txt
└── test_cases/
```
### Optional Components
```
skill-name/
├── [mandatory and recommended components]
├── tests/ # Unit tests and validation scripts
├── examples/ # Extended examples and tutorials
├── docs/ # Additional documentation
├── config/ # Configuration files
└── templates/ # Template files for code generation
```
## File Requirements
### SKILL.md Requirements
The `SKILL.md` file serves as the primary documentation for the skill and must contain:
#### Mandatory YAML Frontmatter
```yaml
---
Name: skill-name
Tier: [BASIC|STANDARD|POWERFUL]
Category: [Category Name]
Dependencies: [None|List of dependencies]
Author: [Author Name]
Version: [Semantic Version]
Last Updated: [YYYY-MM-DD]
---
```
#### Required Sections
- **Description**: Comprehensive overview of the skill's purpose and capabilities
- **Features**: Detailed list of key features and functionality
- **Usage**: Instructions for using the skill and its components
- **Examples**: Practical usage examples with expected outcomes
#### Recommended Sections
- **Architecture**: Technical architecture and design decisions
- **Installation**: Setup and installation instructions
- **Configuration**: Configuration options and parameters
- **Troubleshooting**: Common issues and solutions
- **Contributing**: Guidelines for contributors
- **Changelog**: Version history and changes
#### Content Requirements by Tier
- **BASIC**: Minimum 100 lines of substantial content
- **STANDARD**: Minimum 200 lines of substantial content
- **POWERFUL**: Minimum 300 lines of substantial content
### README.md Requirements
The `README.md` file provides quick start instructions and must include:
#### Mandatory Content
- Brief description of the skill
- Quick start instructions
- Basic usage examples
- Link to full SKILL.md documentation
#### Recommended Content
- Installation instructions
- Prerequisites and dependencies
- Command-line usage examples
- Troubleshooting section
- Contributing guidelines
#### Length Requirements
- Minimum 200 characters of substantial content
- Recommended 500+ characters for comprehensive coverage
### Scripts Directory Requirements
The `scripts/` directory contains all Python implementation files:
#### Mandatory Requirements
- At least one Python (.py) file
- All scripts must be executable Python 3.7+
- No external dependencies outside Python standard library
- Proper file naming conventions (lowercase, hyphens for separation)
#### Script Content Requirements
- **Shebang line**: `#!/usr/bin/env python3`
- **Module docstring**: Comprehensive description of script purpose
- **Argparse implementation**: Command-line argument parsing
- **Main guard**: `if __name__ == "__main__":` protection
- **Error handling**: Appropriate exception handling and user feedback
- **Dual output**: Support for both JSON and human-readable output formats
#### Script Size Requirements by Tier
- **BASIC**: 100-300 lines of code per script
- **STANDARD**: 300-500 lines of code per script
- **POWERFUL**: 500-800 lines of code per script
### Assets Directory Structure
The `assets/` directory contains sample data and supporting files:
```
assets/
├── samples/ # Sample input data
│ ├── simple_example.json
│ ├── complex_dataset.csv
│ └── test_configuration.yaml
├── examples/ # Example files demonstrating usage
│ ├── basic_workflow.py
│ ├── advanced_usage.sh
│ └── integration_example.md
└── data/ # Static data files
├── reference_data.json
├── lookup_tables.csv
└── configuration_templates/
```
#### Content Requirements
- At least 2 sample files demonstrating different use cases
- Files should represent realistic usage scenarios
- Include both simple and complex examples where applicable
- Provide diverse file formats (JSON, CSV, YAML, etc.)
### References Directory Structure
The `references/` directory contains detailed reference documentation:
```
references/
├── api-reference.md # Complete API documentation
├── specifications.md # Technical specifications and requirements
├── external-links.md # Links to related resources
├── algorithms.md # Algorithm descriptions and implementations
└── best-practices.md # Usage best practices and patterns
```
#### Content Requirements
- Each file should contain substantial technical content (500+ words)
- Include code examples and technical specifications
- Provide external references and links where appropriate
- Maintain consistent documentation format and style
### Expected Outputs Directory Structure
The `expected_outputs/` directory contains reference outputs for testing:
```
expected_outputs/
├── basic_example_output.json
├── complex_scenario_result.txt
├── error_cases/
│ ├── invalid_input_error.json
│ └── timeout_error.txt
└── test_cases/
├── unit_test_outputs/
└── integration_test_results/
```
#### Content Requirements
- Outputs correspond to sample inputs in assets/ directory
- Include both successful and error case examples
- Provide outputs in multiple formats (JSON, text, CSV)
- Ensure outputs are reproducible and verifiable
## Naming Conventions
### Directory Names
- Use lowercase letters only
- Use hyphens (-) to separate words
- Keep names concise but descriptive
- Avoid special characters and spaces
Examples: `data-processor`, `api-client`, `ml-trainer`
### File Names
- Use lowercase letters for Python scripts
- Use hyphens (-) to separate words in script names
- Use underscores (_) only when required by Python conventions
- Use descriptive names that indicate purpose
Examples: `data-processor.py`, `api-client.py`, `quality_scorer.py`
### Script Internal Naming
- Use PascalCase for class names
- Use snake_case for function and variable names
- Use UPPER_CASE for constants
- Use descriptive names that indicate purpose
## Quality Standards
### Documentation Standards
- All documentation must be written in clear, professional English
- Use proper Markdown formatting and structure
- Include code examples with syntax highlighting
- Provide comprehensive coverage of all features
- Maintain consistent terminology throughout
### Code Standards
- Follow PEP 8 Python style guidelines
- Include comprehensive docstrings for all functions and classes
- Implement proper error handling with meaningful error messages
- Use type hints where appropriate
- Maintain reasonable code complexity and readability
### Testing Standards
- Provide sample data that exercises all major functionality
- Include expected outputs for verification
- Cover both successful and error scenarios
- Ensure reproducible results across different environments
## Validation Criteria
Skills are validated against the following criteria:
### Structural Validation
- All mandatory files and directories present
- Proper file naming conventions followed
- Directory structure matches specification
- File permissions and accessibility correct
### Content Validation
- SKILL.md meets minimum length and section requirements
- README.md provides adequate quick start information
- Scripts contain required components (argparse, main guard, etc.)
- Sample data and expected outputs are complete and realistic
### Quality Validation
- Documentation is comprehensive and accurate
- Code follows established style and quality guidelines
- Examples are practical and demonstrate real usage
- Error handling is appropriate and user-friendly
## Compliance Levels
### Full Compliance
- All mandatory components present and complete
- All recommended components present with substantial content
- Exceeds minimum quality thresholds for tier
- Demonstrates best practices throughout
### Partial Compliance
- All mandatory components present
- Most recommended components present
- Meets minimum quality thresholds for tier
- Generally follows established patterns
### Non-Compliance
- Missing mandatory components
- Inadequate content quality or length
- Does not meet minimum tier requirements
- Significant deviations from established standards
## Migration and Updates
### Existing Skills
Skills created before this specification should be updated to comply within:
- **POWERFUL tier**: 30 days
- **STANDARD tier**: 60 days
- **BASIC tier**: 90 days
### Specification Updates
- Changes to this specification require team consensus
- Breaking changes must provide 90-day migration period
- All changes must be documented with rationale and examples
- Automated validation tools must be updated accordingly
## Tools and Automation
### Validation Tools
- `skill_validator.py` - Validates structure and content compliance
- `script_tester.py` - Tests script functionality and quality
- `quality_scorer.py` - Provides comprehensive quality assessment
### Integration Points
- Pre-commit hooks for basic validation
- CI/CD pipeline integration for pull request validation
- Automated quality reporting and tracking
- Integration with code review processes
## Examples and Templates
### Minimal BASIC Tier Example
```
basic-skill/
├── SKILL.md # 100+ lines
├── README.md # Basic usage instructions
└── scripts/
└── main.py # 100-300 lines with argparse
```
### Complete POWERFUL Tier Example
```
powerful-skill/
├── SKILL.md # 300+ lines with comprehensive sections
├── README.md # Detailed usage and setup
├── scripts/ # Multiple sophisticated scripts
│ ├── main_processor.py # 500-800 lines
│ ├── data_analyzer.py # 500-800 lines
│ └── report_generator.py # 500-800 lines
├── assets/ # Diverse sample data
│ ├── samples/
│ ├── examples/
│ └── data/
├── references/ # Comprehensive documentation
│ ├── api-reference.md
│ ├── specifications.md
│ └── best-practices.md
└── expected_outputs/ # Complete test outputs
├── json_outputs/
├── text_reports/
└── error_cases/
```
This specification serves as the authoritative guide for skill structure within the claude-skills ecosystem. Adherence to these standards ensures consistency, quality, and maintainability across all skills in the repository.
```
### references/tier-requirements-matrix.md
```markdown
# Tier Requirements Matrix
**Version**: 1.0.0
**Last Updated**: 2026-02-16
**Authority**: Claude Skills Engineering Team
## Overview
This document provides a comprehensive matrix of requirements for each skill tier within the claude-skills ecosystem. Skills are classified into three tiers based on complexity, functionality, and comprehensiveness: BASIC, STANDARD, and POWERFUL.
## Tier Classification Philosophy
### BASIC Tier
Entry-level skills that provide fundamental functionality with minimal complexity. Suitable for simple automation tasks, basic data processing, or straightforward utilities.
### STANDARD Tier
Intermediate skills that offer enhanced functionality with moderate complexity. Suitable for business processes, advanced data manipulation, or multi-step workflows.
### POWERFUL Tier
Advanced skills that provide comprehensive functionality with sophisticated implementation. Suitable for complex systems, enterprise-grade tools, or mission-critical applications.
## Requirements Matrix
| Component | BASIC | STANDARD | POWERFUL |
|-----------|-------|----------|----------|
| **SKILL.md Lines** | ≥100 | ≥200 | ≥300 |
| **Scripts Count** | ≥1 | ≥1 | ≥2 |
| **Script Size (LOC)** | 100-300 | 300-500 | 500-800 |
| **Required Directories** | scripts | scripts, assets, references | scripts, assets, references, expected_outputs |
| **Argparse Implementation** | Basic | Advanced | Complex with subcommands |
| **Output Formats** | Human-readable | JSON + Human-readable | JSON + Human-readable + Custom |
| **Error Handling** | Basic | Comprehensive | Advanced with recovery |
| **Documentation Depth** | Functional | Comprehensive | Expert-level |
| **Examples Provided** | ≥1 | ≥3 | ≥5 |
| **Test Coverage** | Basic validation | Sample data testing | Comprehensive test suite |
## Detailed Requirements by Tier
### BASIC Tier Requirements
#### Documentation Requirements
- **SKILL.md**: Minimum 100 lines of substantial content
- **Required Sections**: Name, Description, Features, Usage, Examples
- **README.md**: Basic usage instructions (200+ characters)
- **Content Quality**: Clear and functional documentation
- **Examples**: At least 1 practical usage example
#### Code Requirements
- **Scripts**: Minimum 1 Python script (100-300 LOC)
- **Argparse**: Basic command-line argument parsing
- **Main Guard**: `if __name__ == "__main__":` protection
- **Dependencies**: Python standard library only
- **Output**: Human-readable format with clear messaging
- **Error Handling**: Basic exception handling with user-friendly messages
#### Structure Requirements
- **Mandatory Directories**: `scripts/`
- **Recommended Directories**: `assets/`, `references/`
- **File Organization**: Logical file naming and structure
- **Assets**: Optional sample data files
#### Quality Standards
- **Code Style**: Follows basic Python conventions
- **Documentation**: Adequate coverage of functionality
- **Usability**: Clear usage instructions and examples
- **Completeness**: All essential components present
### STANDARD Tier Requirements
#### Documentation Requirements
- **SKILL.md**: Minimum 200 lines with comprehensive coverage
- **Required Sections**: All BASIC sections plus Architecture, Installation
- **README.md**: Detailed usage instructions (500+ characters)
- **References**: Technical documentation in `references/` directory
- **Content Quality**: Professional-grade documentation with technical depth
- **Examples**: At least 3 diverse usage examples
#### Code Requirements
- **Scripts**: 1-2 Python scripts (300-500 LOC each)
- **Argparse**: Advanced argument parsing with subcommands and validation
- **Output Formats**: Both JSON and human-readable output support
- **Error Handling**: Comprehensive exception handling with specific error types
- **Code Structure**: Well-organized classes and functions
- **Documentation**: Comprehensive docstrings for all functions
#### Structure Requirements
- **Mandatory Directories**: `scripts/`, `assets/`, `references/`
- **Recommended Directories**: `expected_outputs/`
- **Assets**: Multiple sample files demonstrating different use cases
- **References**: Technical specifications and API documentation
- **Expected Outputs**: Sample results for validation
#### Quality Standards
- **Code Quality**: Advanced Python patterns and best practices
- **Documentation**: Expert-level technical documentation
- **Testing**: Sample data processing with validation
- **Integration**: Consideration for CI/CD and automation use
### POWERFUL Tier Requirements
#### Documentation Requirements
- **SKILL.md**: Minimum 300 lines with expert-level comprehensiveness
- **Required Sections**: All STANDARD sections plus Troubleshooting, Contributing, Advanced Usage
- **README.md**: Comprehensive guide with installation and setup (1000+ characters)
- **References**: Multiple technical documents with specifications
- **Content Quality**: Publication-ready documentation with architectural details
- **Examples**: At least 5 examples covering simple to complex scenarios
#### Code Requirements
- **Scripts**: 2-3 Python scripts (500-800 LOC each)
- **Argparse**: Complex argument parsing with multiple modes and configurations
- **Output Formats**: JSON, human-readable, and custom format support
- **Error Handling**: Advanced error handling with recovery mechanisms
- **Code Architecture**: Sophisticated design patterns and modular structure
- **Performance**: Optimized for efficiency and scalability
#### Structure Requirements
- **Mandatory Directories**: `scripts/`, `assets/`, `references/`, `expected_outputs/`
- **Optional Directories**: `tests/`, `examples/`, `docs/`
- **Assets**: Comprehensive sample data covering edge cases
- **References**: Complete technical specification suite
- **Expected Outputs**: Full test result coverage including error cases
- **Testing**: Comprehensive validation and test coverage
#### Quality Standards
- **Enterprise Grade**: Production-ready code with enterprise patterns
- **Documentation**: Comprehensive technical documentation suitable for technical teams
- **Integration**: Full CI/CD integration capabilities
- **Maintainability**: Designed for long-term maintenance and extension
## Tier Assessment Criteria
### Automatic Tier Classification
Skills are automatically classified based on quantitative metrics:
```python
def classify_tier(skill_metrics):
if (skill_metrics['skill_md_lines'] >= 300 and
skill_metrics['script_count'] >= 2 and
skill_metrics['min_script_size'] >= 500 and
all_required_dirs_present(['scripts', 'assets', 'references', 'expected_outputs'])):
return 'POWERFUL'
elif (skill_metrics['skill_md_lines'] >= 200 and
skill_metrics['script_count'] >= 1 and
skill_metrics['min_script_size'] >= 300 and
all_required_dirs_present(['scripts', 'assets', 'references'])):
return 'STANDARD'
else:
return 'BASIC'
```
### Manual Tier Override
Manual tier assignment may be considered when:
- Skill provides exceptional value despite not meeting all quantitative requirements
- Skill addresses critical infrastructure or security needs
- Skill demonstrates innovative approaches or cutting-edge techniques
- Skill provides essential integration or compatibility functions
### Tier Promotion Criteria
Skills may be promoted to higher tiers when:
- All quantitative requirements for higher tier are met
- Quality assessment scores exceed tier thresholds
- Community usage and feedback indicate higher value
- Continuous integration and maintenance demonstrate reliability
### Tier Demotion Criteria
Skills may be demoted to lower tiers when:
- Quality degradation below tier standards
- Lack of maintenance or updates
- Compatibility issues or security vulnerabilities
- Community feedback indicates reduced value
## Implementation Guidelines by Tier
### BASIC Tier Implementation
```python
# Example argparse implementation for BASIC tier
parser = argparse.ArgumentParser(description="Basic skill functionality")
parser.add_argument("input", help="Input file or parameter")
parser.add_argument("--output", help="Output destination")
parser.add_argument("--verbose", action="store_true", help="Verbose output")
# Basic error handling
try:
result = process_input(args.input)
print(f"Processing completed: {result}")
except FileNotFoundError:
print("Error: Input file not found")
sys.exit(1)
except Exception as e:
print(f"Error: {str(e)}")
sys.exit(1)
```
### STANDARD Tier Implementation
```python
# Example argparse implementation for STANDARD tier
parser = argparse.ArgumentParser(
description="Standard skill with advanced functionality",
formatter_class=argparse.RawDescriptionHelpFormatter,
epilog="Examples:\n python script.py input.json --format json\n python script.py data/ --batch --output results/"
)
parser.add_argument("input", help="Input file or directory")
parser.add_argument("--format", choices=["json", "text"], default="json", help="Output format")
parser.add_argument("--batch", action="store_true", help="Process multiple files")
parser.add_argument("--output", help="Output destination")
# Advanced error handling with specific exception types
try:
if args.batch:
results = batch_process(args.input)
else:
results = single_process(args.input)
if args.format == "json":
print(json.dumps(results, indent=2))
else:
print_human_readable(results)
except FileNotFoundError as e:
logging.error(f"File not found: {e}")
sys.exit(1)
except ValueError as e:
logging.error(f"Invalid input: {e}")
sys.exit(2)
except Exception as e:
logging.error(f"Unexpected error: {e}")
sys.exit(1)
```
### POWERFUL Tier Implementation
```python
# Example argparse implementation for POWERFUL tier
parser = argparse.ArgumentParser(
description="Powerful skill with comprehensive functionality",
formatter_class=argparse.RawDescriptionHelpFormatter,
epilog="""
Examples:
Basic usage:
python script.py process input.json --output results/
Advanced batch processing:
python script.py batch data/ --format json --parallel 4 --filter "*.csv"
Custom configuration:
python script.py process input.json --config custom.yaml --dry-run
"""
)
subparsers = parser.add_subparsers(dest="command", help="Available commands")
# Process subcommand
process_parser = subparsers.add_parser("process", help="Process single file")
process_parser.add_argument("input", help="Input file path")
process_parser.add_argument("--config", help="Configuration file")
process_parser.add_argument("--dry-run", action="store_true", help="Show what would be done")
# Batch subcommand
batch_parser = subparsers.add_parser("batch", help="Process multiple files")
batch_parser.add_argument("directory", help="Input directory")
batch_parser.add_argument("--parallel", type=int, default=1, help="Number of parallel processes")
batch_parser.add_argument("--filter", help="File filter pattern")
# Comprehensive error handling with recovery
try:
if args.command == "process":
result = process_with_recovery(args.input, args.config, args.dry_run)
elif args.command == "batch":
result = batch_process_with_monitoring(args.directory, args.parallel, args.filter)
else:
parser.print_help()
sys.exit(1)
# Multiple output format support
output_formatter = OutputFormatter(args.format)
output_formatter.write(result, args.output)
except KeyboardInterrupt:
logging.info("Processing interrupted by user")
sys.exit(130)
except ProcessingError as e:
logging.error(f"Processing failed: {e}")
if e.recoverable:
logging.info("Attempting recovery...")
# Recovery logic here
sys.exit(1)
except ValidationError as e:
logging.error(f"Validation failed: {e}")
logging.info("Check input format and try again")
sys.exit(2)
except Exception as e:
logging.critical(f"Critical error: {e}")
logging.info("Please report this issue")
sys.exit(1)
```
## Quality Scoring by Tier
### Scoring Thresholds
- **POWERFUL Tier**: Overall score ≥80, all dimensions ≥75
- **STANDARD Tier**: Overall score ≥70, 3+ dimensions ≥65
- **BASIC Tier**: Overall score ≥60, meets minimum requirements
### Dimension Weights (All Tiers)
- **Documentation**: 25%
- **Code Quality**: 25%
- **Completeness**: 25%
- **Usability**: 25%
### Tier-Specific Quality Expectations
#### BASIC Tier Quality Profile
- Documentation: Functional and clear (60+ points expected)
- Code Quality: Clean and maintainable (60+ points expected)
- Completeness: Essential components present (60+ points expected)
- Usability: Easy to understand and use (60+ points expected)
#### STANDARD Tier Quality Profile
- Documentation: Professional and comprehensive (70+ points expected)
- Code Quality: Advanced patterns and best practices (70+ points expected)
- Completeness: All recommended components (70+ points expected)
- Usability: Well-designed user experience (70+ points expected)
#### POWERFUL Tier Quality Profile
- Documentation: Expert-level and publication-ready (80+ points expected)
- Code Quality: Enterprise-grade implementation (80+ points expected)
- Completeness: Comprehensive test and validation coverage (80+ points expected)
- Usability: Exceptional user experience with extensive help (80+ points expected)
## Tier Migration Process
### Promotion Process
1. **Assessment**: Quality scorer evaluates skill against higher tier requirements
2. **Review**: Engineering team reviews assessment and implementation
3. **Testing**: Comprehensive testing against higher tier standards
4. **Approval**: Team consensus on tier promotion
5. **Update**: Skill metadata and documentation updated to reflect new tier
### Demotion Process
1. **Issue Identification**: Quality degradation or standards violation identified
2. **Assessment**: Current quality evaluated against tier requirements
3. **Notice**: Skill maintainer notified of potential demotion
4. **Grace Period**: 30-day period for remediation
5. **Final Review**: Re-assessment after grace period
6. **Action**: Tier adjustment or removal if standards not met
### Tier Change Communication
- All tier changes logged in skill CHANGELOG.md
- Repository-level tier change notifications
- Integration with CI/CD systems for automated handling
- Community notifications for significant changes
## Compliance Monitoring
### Automated Monitoring
- Daily quality assessment scans
- Tier compliance validation in CI/CD
- Automated reporting of tier violations
- Integration with code review processes
### Manual Review Process
- Quarterly tier review cycles
- Community feedback integration
- Expert panel reviews for complex cases
- Appeals process for tier disputes
### Enforcement Actions
- **Warning**: First violation or minor issues
- **Probation**: Repeated violations or moderate issues
- **Demotion**: Serious violations or quality degradation
- **Removal**: Critical violations or abandonment
This tier requirements matrix serves as the definitive guide for skill classification and quality standards within the claude-skills ecosystem. Regular updates ensure alignment with evolving best practices and community needs.
```
### scripts/quality_scorer.py
```python
#!/usr/bin/env python3
"""
Quality Scorer - Scores skills across multiple quality dimensions
This script provides comprehensive quality assessment for skills in the claude-skills
ecosystem by evaluating documentation, code quality, completeness, and usability.
Generates letter grades, tier recommendations, and improvement roadmaps.
Usage:
python quality_scorer.py <skill_path> [--detailed] [--minimum-score SCORE] [--json]
Author: Claude Skills Engineering Team
Version: 1.0.0
Dependencies: Python Standard Library Only
"""
import argparse
import ast
import json
import os
import re
import sys
from datetime import datetime
from pathlib import Path
from typing import Dict, List, Any, Optional, Tuple
try:
import yaml
except ImportError:
# Minimal YAML subset: parse simple key: value frontmatter without pyyaml
class _YamlStub:
class YAMLError(Exception):
pass
@staticmethod
def safe_load(text):
result = {}
for line in text.strip().splitlines():
if ':' in line:
key, _, value = line.partition(':')
result[key.strip()] = value.strip()
return result if result else None
yaml = _YamlStub()
class QualityDimension:
"""Represents a quality scoring dimension"""
def __init__(self, name: str, weight: float, description: str):
self.name = name
self.weight = weight
self.description = description
self.score = 0.0
self.max_score = 100.0
self.details = {}
self.suggestions = []
def add_score(self, component: str, score: float, max_score: float, details: str = ""):
"""Add a component score"""
self.details[component] = {
"score": score,
"max_score": max_score,
"percentage": (score / max_score * 100) if max_score > 0 else 0,
"details": details
}
def calculate_final_score(self):
"""Calculate the final weighted score for this dimension"""
if not self.details:
self.score = 0.0
return
total_score = sum(detail["score"] for detail in self.details.values())
total_max = sum(detail["max_score"] for detail in self.details.values())
self.score = (total_score / total_max * 100) if total_max > 0 else 0.0
def add_suggestion(self, suggestion: str):
"""Add an improvement suggestion"""
self.suggestions.append(suggestion)
class QualityReport:
"""Container for quality assessment results"""
def __init__(self, skill_path: str):
self.skill_path = skill_path
self.timestamp = datetime.utcnow().isoformat() + "Z"
self.dimensions = {}
self.overall_score = 0.0
self.letter_grade = "F"
self.tier_recommendation = "BASIC"
self.improvement_roadmap = []
self.summary_stats = {}
def add_dimension(self, dimension: QualityDimension):
"""Add a quality dimension"""
self.dimensions[dimension.name] = dimension
def calculate_overall_score(self):
"""Calculate overall weighted score"""
if not self.dimensions:
return
total_weighted_score = 0.0
total_weight = 0.0
for dimension in self.dimensions.values():
total_weighted_score += dimension.score * dimension.weight
total_weight += dimension.weight
self.overall_score = total_weighted_score / total_weight if total_weight > 0 else 0.0
# Calculate letter grade
if self.overall_score >= 95:
self.letter_grade = "A+"
elif self.overall_score >= 90:
self.letter_grade = "A"
elif self.overall_score >= 85:
self.letter_grade = "A-"
elif self.overall_score >= 80:
self.letter_grade = "B+"
elif self.overall_score >= 75:
self.letter_grade = "B"
elif self.overall_score >= 70:
self.letter_grade = "B-"
elif self.overall_score >= 65:
self.letter_grade = "C+"
elif self.overall_score >= 60:
self.letter_grade = "C"
elif self.overall_score >= 55:
self.letter_grade = "C-"
elif self.overall_score >= 50:
self.letter_grade = "D"
else:
self.letter_grade = "F"
# Recommend tier based on overall score and specific criteria
self._calculate_tier_recommendation()
# Generate improvement roadmap
self._generate_improvement_roadmap()
# Calculate summary statistics
self._calculate_summary_stats()
def _calculate_tier_recommendation(self):
"""Calculate recommended tier based on quality scores"""
doc_score = self.dimensions.get("Documentation", QualityDimension("", 0, "")).score
code_score = self.dimensions.get("Code Quality", QualityDimension("", 0, "")).score
completeness_score = self.dimensions.get("Completeness", QualityDimension("", 0, "")).score
usability_score = self.dimensions.get("Usability", QualityDimension("", 0, "")).score
# POWERFUL tier requirements (all dimensions must be strong)
if (self.overall_score >= 80 and
all(score >= 75 for score in [doc_score, code_score, completeness_score, usability_score])):
self.tier_recommendation = "POWERFUL"
# STANDARD tier requirements (most dimensions good)
elif (self.overall_score >= 70 and
sum(1 for score in [doc_score, code_score, completeness_score, usability_score] if score >= 65) >= 3):
self.tier_recommendation = "STANDARD"
# BASIC tier (minimum viable quality)
else:
self.tier_recommendation = "BASIC"
def _generate_improvement_roadmap(self):
"""Generate prioritized improvement suggestions"""
all_suggestions = []
# Collect suggestions from all dimensions with scores
for dim_name, dimension in self.dimensions.items():
for suggestion in dimension.suggestions:
priority = "HIGH" if dimension.score < 60 else "MEDIUM" if dimension.score < 75 else "LOW"
all_suggestions.append({
"priority": priority,
"dimension": dim_name,
"suggestion": suggestion,
"current_score": dimension.score
})
# Sort by priority and score
priority_order = {"HIGH": 0, "MEDIUM": 1, "LOW": 2}
all_suggestions.sort(key=lambda x: (priority_order[x["priority"]], x["current_score"]))
self.improvement_roadmap = all_suggestions[:10] # Top 10 suggestions
def _calculate_summary_stats(self):
"""Calculate summary statistics"""
scores = [dim.score for dim in self.dimensions.values()]
self.summary_stats = {
"highest_dimension": max(self.dimensions.items(), key=lambda x: x[1].score)[0] if scores else "None",
"lowest_dimension": min(self.dimensions.items(), key=lambda x: x[1].score)[0] if scores else "None",
"score_variance": sum((score - self.overall_score) ** 2 for score in scores) / len(scores) if scores else 0,
"dimensions_above_70": sum(1 for score in scores if score >= 70),
"dimensions_below_50": sum(1 for score in scores if score < 50)
}
class QualityScorer:
"""Main quality scoring engine"""
def __init__(self, skill_path: str, detailed: bool = False, verbose: bool = False):
self.skill_path = Path(skill_path).resolve()
self.detailed = detailed
self.verbose = verbose
self.report = QualityReport(str(self.skill_path))
def log_verbose(self, message: str):
"""Log verbose message if verbose mode enabled"""
if self.verbose:
print(f"[VERBOSE] {message}", file=sys.stderr)
def assess_quality(self) -> QualityReport:
"""Main quality assessment entry point"""
try:
self.log_verbose(f"Starting quality assessment for {self.skill_path}")
# Check if skill path exists
if not self.skill_path.exists():
raise ValueError(f"Skill path does not exist: {self.skill_path}")
# Score each dimension
self._score_documentation()
self._score_code_quality()
self._score_completeness()
self._score_usability()
# Calculate overall metrics
self.report.calculate_overall_score()
self.log_verbose(f"Quality assessment completed. Overall score: {self.report.overall_score:.1f}")
except Exception as e:
print(f"Quality assessment failed: {str(e)}", file=sys.stderr)
raise
return self.report
def _score_documentation(self):
"""Score documentation quality (25% weight)"""
self.log_verbose("Scoring documentation quality...")
dimension = QualityDimension("Documentation", 0.25, "Quality of documentation and written materials")
# Score SKILL.md
self._score_skill_md(dimension)
# Score README.md
self._score_readme(dimension)
# Score reference documentation
self._score_references(dimension)
# Score examples and usage clarity
self._score_examples(dimension)
dimension.calculate_final_score()
self.report.add_dimension(dimension)
def _score_skill_md(self, dimension: QualityDimension):
"""Score SKILL.md quality"""
skill_md_path = self.skill_path / "SKILL.md"
if not skill_md_path.exists():
dimension.add_score("skill_md_existence", 0, 25, "SKILL.md does not exist")
dimension.add_suggestion("Create comprehensive SKILL.md file")
return
try:
content = skill_md_path.read_text(encoding='utf-8')
lines = [line for line in content.split('\n') if line.strip()]
# Score based on length and depth
line_count = len(lines)
if line_count >= 400:
length_score = 25
elif line_count >= 300:
length_score = 20
elif line_count >= 200:
length_score = 15
elif line_count >= 100:
length_score = 10
else:
length_score = 5
dimension.add_score("skill_md_length", length_score, 25,
f"SKILL.md has {line_count} lines")
if line_count < 300:
dimension.add_suggestion("Expand SKILL.md with more detailed sections")
# Score frontmatter quality
frontmatter_score = self._score_frontmatter(content)
dimension.add_score("skill_md_frontmatter", frontmatter_score, 25,
"Frontmatter completeness and accuracy")
# Score section completeness
section_score = self._score_sections(content)
dimension.add_score("skill_md_sections", section_score, 25,
"Required and recommended section coverage")
# Score content depth
depth_score = self._score_content_depth(content)
dimension.add_score("skill_md_depth", depth_score, 25,
"Content depth and technical detail")
except Exception as e:
dimension.add_score("skill_md_readable", 0, 25, f"Error reading SKILL.md: {str(e)}")
dimension.add_suggestion("Fix SKILL.md file encoding or format issues")
def _score_frontmatter(self, content: str) -> float:
"""Score SKILL.md frontmatter quality"""
required_fields = ["Name", "Tier", "Category", "Dependencies", "Author", "Version"]
recommended_fields = ["Last Updated", "Description"]
try:
if not content.startswith('---'):
return 5 # Partial credit for having some structure
end_marker = content.find('---', 3)
if end_marker == -1:
return 5
frontmatter_text = content[3:end_marker].strip()
frontmatter = yaml.safe_load(frontmatter_text)
if not isinstance(frontmatter, dict):
return 5
score = 0
# Required fields (15 points)
present_required = sum(1 for field in required_fields if field in frontmatter)
score += (present_required / len(required_fields)) * 15
# Recommended fields (5 points)
present_recommended = sum(1 for field in recommended_fields if field in frontmatter)
score += (present_recommended / len(recommended_fields)) * 5
# Quality of field values (5 points)
quality_bonus = 0
for field, value in frontmatter.items():
if isinstance(value, str) and len(value.strip()) > 3:
quality_bonus += 0.5
score += min(quality_bonus, 5)
return min(score, 25)
except yaml.YAMLError:
return 5 # Some credit for attempting frontmatter
def _score_sections(self, content: str) -> float:
"""Score section completeness"""
required_sections = ["Description", "Features", "Usage", "Examples"]
recommended_sections = ["Architecture", "Installation", "Troubleshooting", "Contributing"]
score = 0
# Required sections (15 points)
present_required = 0
for section in required_sections:
if re.search(rf'^#+\s*{re.escape(section)}\s*$', content, re.MULTILINE | re.IGNORECASE):
present_required += 1
score += (present_required / len(required_sections)) * 15
# Recommended sections (10 points)
present_recommended = 0
for section in recommended_sections:
if re.search(rf'^#+\s*{re.escape(section)}\s*$', content, re.MULTILINE | re.IGNORECASE):
present_recommended += 1
score += (present_recommended / len(recommended_sections)) * 10
return score
def _score_content_depth(self, content: str) -> float:
"""Score content depth and technical detail"""
score = 0
# Code examples (8 points)
code_blocks = len(re.findall(r'```[\w]*\n.*?\n```', content, re.DOTALL))
score += min(code_blocks * 2, 8)
# Technical depth indicators (8 points)
depth_indicators = ['API', 'algorithm', 'architecture', 'implementation', 'performance',
'scalability', 'security', 'integration', 'configuration', 'parameters']
depth_score = sum(1 for indicator in depth_indicators if indicator.lower() in content.lower())
score += min(depth_score * 0.8, 8)
# Usage examples (9 points)
example_patterns = [r'Example:', r'Usage:', r'```bash', r'```python', r'```yaml']
example_count = sum(len(re.findall(pattern, content, re.IGNORECASE)) for pattern in example_patterns)
score += min(example_count * 1.5, 9)
return score
def _score_readme(self, dimension: QualityDimension):
"""Score README.md quality"""
readme_path = self.skill_path / "README.md"
if not readme_path.exists():
dimension.add_score("readme_existence", 10, 25, "README.md exists (partial credit)")
dimension.add_suggestion("Create README.md with usage instructions")
return
try:
content = readme_path.read_text(encoding='utf-8')
# Length and substance
if len(content.strip()) >= 1000:
length_score = 25
elif len(content.strip()) >= 500:
length_score = 20
elif len(content.strip()) >= 200:
length_score = 15
else:
length_score = 10
dimension.add_score("readme_quality", length_score, 25,
f"README.md content quality ({len(content)} characters)")
if len(content.strip()) < 500:
dimension.add_suggestion("Expand README.md with more detailed usage examples")
except Exception:
dimension.add_score("readme_readable", 5, 25, "README.md exists but has issues")
def _score_references(self, dimension: QualityDimension):
"""Score reference documentation quality"""
references_dir = self.skill_path / "references"
if not references_dir.exists():
dimension.add_score("references_existence", 0, 25, "No references directory")
dimension.add_suggestion("Add references directory with documentation")
return
ref_files = list(references_dir.glob("*.md")) + list(references_dir.glob("*.txt"))
if not ref_files:
dimension.add_score("references_content", 5, 25, "References directory empty")
dimension.add_suggestion("Add reference documentation files")
return
# Score based on number and quality of reference files
score = min(len(ref_files) * 5, 20) # Up to 20 points for multiple files
# Bonus for substantial content
total_content = 0
for ref_file in ref_files:
try:
content = ref_file.read_text(encoding='utf-8')
total_content += len(content.strip())
except:
continue
if total_content >= 2000:
score += 5 # Bonus for substantial reference content
dimension.add_score("references_quality", score, 25,
f"References: {len(ref_files)} files, {total_content} chars")
def _score_examples(self, dimension: QualityDimension):
"""Score examples and usage clarity"""
score = 0
# Look for example files in various locations
example_locations = ["examples", "assets", "scripts"]
example_files = []
for location in example_locations:
location_path = self.skill_path / location
if location_path.exists():
example_files.extend(location_path.glob("*example*"))
example_files.extend(location_path.glob("*sample*"))
example_files.extend(location_path.glob("*demo*"))
# Score based on example availability
if len(example_files) >= 3:
score = 25
elif len(example_files) >= 2:
score = 20
elif len(example_files) >= 1:
score = 15
else:
score = 10
dimension.add_suggestion("Add more usage examples and sample files")
dimension.add_score("examples_availability", score, 25,
f"Found {len(example_files)} example/sample files")
def _score_code_quality(self):
"""Score code quality (25% weight)"""
self.log_verbose("Scoring code quality...")
dimension = QualityDimension("Code Quality", 0.25, "Quality of Python scripts and implementation")
scripts_dir = self.skill_path / "scripts"
if not scripts_dir.exists():
dimension.add_score("scripts_existence", 0, 100, "No scripts directory")
dimension.add_suggestion("Create scripts directory with Python files")
dimension.calculate_final_score()
self.report.add_dimension(dimension)
return
python_files = list(scripts_dir.glob("*.py"))
if not python_files:
dimension.add_score("python_scripts", 0, 100, "No Python scripts found")
dimension.add_suggestion("Add Python scripts to scripts directory")
dimension.calculate_final_score()
self.report.add_dimension(dimension)
return
# Score script complexity and quality
self._score_script_complexity(python_files, dimension)
# Score error handling
self._score_error_handling(python_files, dimension)
# Score code structure
self._score_code_structure(python_files, dimension)
# Score output format support
self._score_output_support(python_files, dimension)
dimension.calculate_final_score()
self.report.add_dimension(dimension)
def _score_script_complexity(self, python_files: List[Path], dimension: QualityDimension):
"""Score script complexity and sophistication"""
total_complexity = 0
script_count = len(python_files)
for script_path in python_files:
try:
content = script_path.read_text(encoding='utf-8')
# Count lines of code (excluding empty lines and comments)
lines = content.split('\n')
loc = len([line for line in lines if line.strip() and not line.strip().startswith('#')])
# Score based on LOC
if loc >= 800:
complexity_score = 25
elif loc >= 500:
complexity_score = 20
elif loc >= 300:
complexity_score = 15
elif loc >= 100:
complexity_score = 10
else:
complexity_score = 5
total_complexity += complexity_score
except Exception:
continue
avg_complexity = total_complexity / script_count if script_count > 0 else 0
dimension.add_score("script_complexity", avg_complexity, 25,
f"Average script complexity across {script_count} scripts")
if avg_complexity < 15:
dimension.add_suggestion("Consider expanding scripts with more functionality")
def _score_error_handling(self, python_files: List[Path], dimension: QualityDimension):
"""Score error handling quality"""
total_error_score = 0
script_count = len(python_files)
for script_path in python_files:
try:
content = script_path.read_text(encoding='utf-8')
error_score = 0
# Check for try/except blocks
try_count = content.count('try:')
error_score += min(try_count * 5, 15) # Up to 15 points for try/except
# Check for specific exception handling
exception_types = ['Exception', 'ValueError', 'FileNotFoundError', 'KeyError', 'TypeError']
for exc_type in exception_types:
if exc_type in content:
error_score += 2 # 2 points per specific exception type
# Check for logging or error reporting
if any(indicator in content for indicator in ['print(', 'logging.', 'sys.stderr']):
error_score += 5 # 5 points for error reporting
total_error_score += min(error_score, 25) # Cap at 25 per script
except Exception:
continue
avg_error_score = total_error_score / script_count if script_count > 0 else 0
dimension.add_score("error_handling", avg_error_score, 25,
f"Error handling quality across {script_count} scripts")
if avg_error_score < 15:
dimension.add_suggestion("Improve error handling with try/except blocks and meaningful error messages")
def _score_code_structure(self, python_files: List[Path], dimension: QualityDimension):
"""Score code structure and organization"""
total_structure_score = 0
script_count = len(python_files)
for script_path in python_files:
try:
content = script_path.read_text(encoding='utf-8')
structure_score = 0
# Check for functions and classes
function_count = content.count('def ')
class_count = content.count('class ')
structure_score += min(function_count * 2, 10) # Up to 10 points for functions
structure_score += min(class_count * 3, 9) # Up to 9 points for classes
# Check for docstrings
docstring_patterns = ['"""', "'''", 'def.*:\n.*"""', 'class.*:\n.*"""']
for pattern in docstring_patterns:
if re.search(pattern, content):
structure_score += 1 # 1 point per docstring indicator
# Check for if __name__ == "__main__"
if 'if __name__ == "__main__"' in content:
structure_score += 3
# Check for imports organization
if content.lstrip().startswith(('import ', 'from ')):
structure_score += 2 # Imports at top
total_structure_score += min(structure_score, 25)
except Exception:
continue
avg_structure_score = total_structure_score / script_count if script_count > 0 else 0
dimension.add_score("code_structure", avg_structure_score, 25,
f"Code structure quality across {script_count} scripts")
if avg_structure_score < 15:
dimension.add_suggestion("Improve code structure with more functions, classes, and documentation")
def _score_output_support(self, python_files: List[Path], dimension: QualityDimension):
"""Score output format support"""
total_output_score = 0
script_count = len(python_files)
for script_path in python_files:
try:
content = script_path.read_text(encoding='utf-8')
output_score = 0
# Check for JSON support
if any(indicator in content for indicator in ['json.dump', 'json.load', '--json']):
output_score += 12 # JSON support
# Check for formatted output
if any(indicator in content for indicator in ['print(f"', 'print("', '.format(', 'f"']):
output_score += 8 # Human-readable output
# Check for argparse help
if '--help' in content or 'add_help=' in content:
output_score += 5 # Help functionality
total_output_score += min(output_score, 25)
except Exception:
continue
avg_output_score = total_output_score / script_count if script_count > 0 else 0
dimension.add_score("output_support", avg_output_score, 25,
f"Output format support across {script_count} scripts")
if avg_output_score < 15:
dimension.add_suggestion("Add support for both JSON and human-readable output formats")
def _score_completeness(self):
"""Score completeness (25% weight)"""
self.log_verbose("Scoring completeness...")
dimension = QualityDimension("Completeness", 0.25, "Completeness of required components and assets")
# Score directory structure
self._score_directory_structure(dimension)
# Score asset availability
self._score_assets(dimension)
# Score expected outputs
self._score_expected_outputs(dimension)
# Score test coverage
self._score_test_coverage(dimension)
dimension.calculate_final_score()
self.report.add_dimension(dimension)
def _score_directory_structure(self, dimension: QualityDimension):
"""Score directory structure completeness"""
required_dirs = ["scripts"]
recommended_dirs = ["assets", "references", "expected_outputs"]
score = 0
# Required directories (15 points)
for dir_name in required_dirs:
if (self.skill_path / dir_name).exists():
score += 15 / len(required_dirs)
# Recommended directories (10 points)
present_recommended = 0
for dir_name in recommended_dirs:
if (self.skill_path / dir_name).exists():
present_recommended += 1
score += (present_recommended / len(recommended_dirs)) * 10
dimension.add_score("directory_structure", score, 25,
f"Directory structure completeness")
missing_recommended = [d for d in recommended_dirs if not (self.skill_path / d).exists()]
if missing_recommended:
dimension.add_suggestion(f"Add recommended directories: {', '.join(missing_recommended)}")
def _score_assets(self, dimension: QualityDimension):
"""Score asset availability and quality"""
assets_dir = self.skill_path / "assets"
if not assets_dir.exists():
dimension.add_score("assets_existence", 5, 25, "Assets directory missing")
dimension.add_suggestion("Create assets directory with sample data")
return
asset_files = [f for f in assets_dir.rglob("*") if f.is_file()]
if not asset_files:
dimension.add_score("assets_content", 10, 25, "Assets directory empty")
dimension.add_suggestion("Add sample data files to assets directory")
return
# Score based on number and diversity of assets
score = min(len(asset_files) * 3, 20) # Up to 20 points for multiple assets
# Bonus for diverse file types
extensions = set(f.suffix.lower() for f in asset_files if f.suffix)
if len(extensions) >= 3:
score += 5 # Bonus for file type diversity
dimension.add_score("assets_quality", score, 25,
f"Assets: {len(asset_files)} files, {len(extensions)} types")
def _score_expected_outputs(self, dimension: QualityDimension):
"""Score expected outputs availability"""
expected_dir = self.skill_path / "expected_outputs"
if not expected_dir.exists():
dimension.add_score("expected_outputs", 10, 25, "Expected outputs directory missing")
dimension.add_suggestion("Add expected_outputs directory with sample results")
return
output_files = [f for f in expected_dir.rglob("*") if f.is_file()]
if len(output_files) >= 3:
score = 25
elif len(output_files) >= 2:
score = 20
elif len(output_files) >= 1:
score = 15
else:
score = 10
dimension.add_suggestion("Add expected output files for testing")
dimension.add_score("expected_outputs", score, 25,
f"Expected outputs: {len(output_files)} files")
def _score_test_coverage(self, dimension: QualityDimension):
"""Score test coverage and validation"""
# This is a simplified scoring - in a more sophisticated system,
# this would integrate with actual test runners
score = 15 # Base score for having a structure
# Check for test-related files
test_indicators = ["test", "spec", "check"]
test_files = []
for indicator in test_indicators:
test_files.extend(self.skill_path.rglob(f"*{indicator}*"))
if test_files:
score += 10 # Bonus for test files
dimension.add_score("test_coverage", score, 25,
f"Test coverage indicators: {len(test_files)} files")
if not test_files:
dimension.add_suggestion("Add test files or validation scripts")
def _score_usability(self):
"""Score usability (25% weight)"""
self.log_verbose("Scoring usability...")
dimension = QualityDimension("Usability", 0.25, "Ease of use and user experience")
# Score installation simplicity
self._score_installation(dimension)
# Score usage clarity
self._score_usage_clarity(dimension)
# Score help and documentation accessibility
self._score_help_accessibility(dimension)
# Score practical examples
self._score_practical_examples(dimension)
dimension.calculate_final_score()
self.report.add_dimension(dimension)
def _score_installation(self, dimension: QualityDimension):
"""Score installation simplicity"""
# Check for installation complexity indicators
score = 25 # Start with full points for standard library only approach
# Check for requirements.txt or setup.py (would reduce score)
if (self.skill_path / "requirements.txt").exists():
score -= 5 # Minor penalty for external dependencies
dimension.add_suggestion("Consider removing external dependencies for easier installation")
if (self.skill_path / "setup.py").exists():
score -= 3 # Minor penalty for complex setup
dimension.add_score("installation_simplicity", max(score, 15), 25,
"Installation complexity assessment")
def _score_usage_clarity(self, dimension: QualityDimension):
"""Score usage clarity"""
score = 0
# Check README for usage instructions
readme_path = self.skill_path / "README.md"
if readme_path.exists():
try:
content = readme_path.read_text(encoding='utf-8').lower()
if 'usage' in content or 'how to' in content:
score += 10
if 'example' in content:
score += 5
except:
pass
# Check scripts for help text quality
scripts_dir = self.skill_path / "scripts"
if scripts_dir.exists():
python_files = list(scripts_dir.glob("*.py"))
help_quality = 0
for script_path in python_files:
try:
content = script_path.read_text(encoding='utf-8')
if 'argparse' in content and 'help=' in content:
help_quality += 2
except:
continue
score += min(help_quality, 10) # Up to 10 points for help text
dimension.add_score("usage_clarity", score, 25, "Usage instructions and help quality")
if score < 15:
dimension.add_suggestion("Improve usage documentation and help text")
def _score_help_accessibility(self, dimension: QualityDimension):
"""Score help and documentation accessibility"""
score = 0
# Check for comprehensive help in scripts
scripts_dir = self.skill_path / "scripts"
if scripts_dir.exists():
python_files = list(scripts_dir.glob("*.py"))
for script_path in python_files:
try:
content = script_path.read_text(encoding='utf-8')
# Check for detailed help text
if 'epilog=' in content or 'description=' in content:
score += 5 # Detailed help
# Check for examples in help
if 'examples:' in content.lower() or 'example:' in content.lower():
score += 3 # Examples in help
except:
continue
# Check for documentation files
doc_files = list(self.skill_path.glob("*.md"))
if len(doc_files) >= 2:
score += 5 # Multiple documentation files
dimension.add_score("help_accessibility", min(score, 25), 25,
"Help and documentation accessibility")
if score < 15:
dimension.add_suggestion("Add more comprehensive help text and documentation")
def _score_practical_examples(self, dimension: QualityDimension):
"""Score practical examples quality"""
score = 0
# Look for example files
example_patterns = ["*example*", "*sample*", "*demo*", "*tutorial*"]
example_files = []
for pattern in example_patterns:
example_files.extend(self.skill_path.rglob(pattern))
# Score based on example availability and quality
if len(example_files) >= 5:
score = 25
elif len(example_files) >= 3:
score = 20
elif len(example_files) >= 2:
score = 15
elif len(example_files) >= 1:
score = 10
else:
score = 5
dimension.add_suggestion("Add more practical examples and sample files")
dimension.add_score("practical_examples", score, 25,
f"Practical examples: {len(example_files)} files")
class QualityReportFormatter:
"""Formats quality reports for output"""
@staticmethod
def format_json(report: QualityReport) -> str:
"""Format report as JSON"""
return json.dumps({
"skill_path": report.skill_path,
"timestamp": report.timestamp,
"overall_score": round(report.overall_score, 1),
"letter_grade": report.letter_grade,
"tier_recommendation": report.tier_recommendation,
"summary_stats": report.summary_stats,
"dimensions": {
name: {
"name": dim.name,
"weight": dim.weight,
"score": round(dim.score, 1),
"description": dim.description,
"details": dim.details,
"suggestions": dim.suggestions
}
for name, dim in report.dimensions.items()
},
"improvement_roadmap": report.improvement_roadmap
}, indent=2)
@staticmethod
def format_human_readable(report: QualityReport, detailed: bool = False) -> str:
"""Format report as human-readable text"""
lines = []
lines.append("=" * 70)
lines.append("SKILL QUALITY ASSESSMENT REPORT")
lines.append("=" * 70)
lines.append(f"Skill: {report.skill_path}")
lines.append(f"Timestamp: {report.timestamp}")
lines.append(f"Overall Score: {report.overall_score:.1f}/100 ({report.letter_grade})")
lines.append(f"Recommended Tier: {report.tier_recommendation}")
lines.append("")
# Dimension scores
lines.append("QUALITY DIMENSIONS:")
for name, dimension in report.dimensions.items():
lines.append(f" {name}: {dimension.score:.1f}/100 ({dimension.weight * 100:.0f}% weight)")
if detailed and dimension.details:
for component, details in dimension.details.items():
lines.append(f" • {component}: {details['score']:.1f}/{details['max_score']} - {details['details']}")
lines.append("")
# Summary statistics
if report.summary_stats:
lines.append("SUMMARY STATISTICS:")
lines.append(f" Highest Dimension: {report.summary_stats['highest_dimension']}")
lines.append(f" Lowest Dimension: {report.summary_stats['lowest_dimension']}")
lines.append(f" Dimensions Above 70%: {report.summary_stats['dimensions_above_70']}")
lines.append(f" Dimensions Below 50%: {report.summary_stats['dimensions_below_50']}")
lines.append("")
# Improvement roadmap
if report.improvement_roadmap:
lines.append("IMPROVEMENT ROADMAP:")
for i, item in enumerate(report.improvement_roadmap[:5], 1):
priority_symbol = "🔴" if item["priority"] == "HIGH" else "🟡" if item["priority"] == "MEDIUM" else "🟢"
lines.append(f" {i}. {priority_symbol} [{item['dimension']}] {item['suggestion']}")
lines.append("")
return "\n".join(lines)
def main():
"""Main entry point"""
parser = argparse.ArgumentParser(
description="Score skill quality across multiple dimensions",
formatter_class=argparse.RawDescriptionHelpFormatter,
epilog="""
Examples:
python quality_scorer.py engineering/my-skill
python quality_scorer.py engineering/my-skill --detailed --json
python quality_scorer.py engineering/my-skill --minimum-score 75
Quality Dimensions (each 25%):
Documentation - SKILL.md quality, README, references, examples
Code Quality - Script complexity, error handling, structure, output
Completeness - Directory structure, assets, expected outputs, tests
Usability - Installation simplicity, usage clarity, help accessibility
Letter Grades: A+ (95+), A (90+), A- (85+), B+ (80+), B (75+), B- (70+), C+ (65+), C (60+), C- (55+), D (50+), F (<50)
"""
)
parser.add_argument("skill_path",
help="Path to the skill directory to assess")
parser.add_argument("--detailed",
action="store_true",
help="Show detailed component scores")
parser.add_argument("--minimum-score",
type=float,
default=0,
help="Minimum acceptable score (exit with error if below)")
parser.add_argument("--json",
action="store_true",
help="Output results in JSON format")
parser.add_argument("--verbose",
action="store_true",
help="Enable verbose logging")
args = parser.parse_args()
try:
# Create scorer and assess quality
scorer = QualityScorer(args.skill_path, args.detailed, args.verbose)
report = scorer.assess_quality()
# Format and output report
if args.json:
print(QualityReportFormatter.format_json(report))
else:
print(QualityReportFormatter.format_human_readable(report, args.detailed))
# Check minimum score requirement
if report.overall_score < args.minimum_score:
print(f"\nERROR: Quality score {report.overall_score:.1f} is below minimum {args.minimum_score}", file=sys.stderr)
sys.exit(1)
# Exit with different codes based on grade
if report.letter_grade in ["A+", "A", "A-"]:
sys.exit(0) # Excellent
elif report.letter_grade in ["B+", "B", "B-"]:
sys.exit(0) # Good
elif report.letter_grade in ["C+", "C", "C-"]:
sys.exit(0) # Acceptable
elif report.letter_grade == "D":
sys.exit(2) # Needs improvement
else: # F
sys.exit(1) # Poor quality
except KeyboardInterrupt:
print("\nQuality assessment interrupted by user", file=sys.stderr)
sys.exit(130)
except Exception as e:
print(f"Quality assessment failed: {str(e)}", file=sys.stderr)
if args.verbose:
import traceback
traceback.print_exc()
sys.exit(1)
if __name__ == "__main__":
main()
```
### scripts/script_tester.py
```python
#!/usr/bin/env python3
"""
Script Tester - Tests Python scripts in a skill directory
This script validates and tests Python scripts within a skill directory by checking
syntax, imports, runtime execution, argparse functionality, and output formats.
It ensures scripts meet quality standards and function correctly.
Usage:
python script_tester.py <skill_path> [--timeout SECONDS] [--json] [--verbose]
Author: Claude Skills Engineering Team
Version: 1.0.0
Dependencies: Python Standard Library Only
"""
import argparse
import ast
import json
import os
import subprocess
import sys
import tempfile
import time
from datetime import datetime
from pathlib import Path
from typing import Dict, List, Any, Optional, Tuple, Union
import threading
class TestError(Exception):
"""Custom exception for testing errors"""
pass
class ScriptTestResult:
"""Container for individual script test results"""
def __init__(self, script_path: str):
self.script_path = script_path
self.script_name = Path(script_path).name
self.timestamp = datetime.utcnow().isoformat() + "Z"
self.tests = {}
self.overall_status = "PENDING"
self.execution_time = 0.0
self.errors = []
self.warnings = []
def add_test(self, test_name: str, passed: bool, message: str = "", details: Dict = None):
"""Add a test result"""
self.tests[test_name] = {
"passed": passed,
"message": message,
"details": details or {}
}
def add_error(self, error: str):
"""Add an error message"""
self.errors.append(error)
def add_warning(self, warning: str):
"""Add a warning message"""
self.warnings.append(warning)
def calculate_status(self):
"""Calculate overall test status"""
if not self.tests:
self.overall_status = "NO_TESTS"
return
failed_tests = [name for name, result in self.tests.items() if not result["passed"]]
if not failed_tests:
self.overall_status = "PASS"
elif len(failed_tests) <= len(self.tests) // 2:
self.overall_status = "PARTIAL"
else:
self.overall_status = "FAIL"
class TestSuite:
"""Container for all test results"""
def __init__(self, skill_path: str):
self.skill_path = skill_path
self.timestamp = datetime.utcnow().isoformat() + "Z"
self.script_results = {}
self.summary = {}
self.global_errors = []
def add_script_result(self, result: ScriptTestResult):
"""Add a script test result"""
self.script_results[result.script_name] = result
def add_global_error(self, error: str):
"""Add a global error message"""
self.global_errors.append(error)
def calculate_summary(self):
"""Calculate summary statistics"""
if not self.script_results:
self.summary = {
"total_scripts": 0,
"passed": 0,
"partial": 0,
"failed": 0,
"overall_status": "NO_SCRIPTS"
}
return
statuses = [result.overall_status for result in self.script_results.values()]
self.summary = {
"total_scripts": len(self.script_results),
"passed": statuses.count("PASS"),
"partial": statuses.count("PARTIAL"),
"failed": statuses.count("FAIL"),
"no_tests": statuses.count("NO_TESTS")
}
# Determine overall status
if self.summary["failed"] == 0 and self.summary["no_tests"] == 0:
self.summary["overall_status"] = "PASS"
elif self.summary["passed"] > 0:
self.summary["overall_status"] = "PARTIAL"
else:
self.summary["overall_status"] = "FAIL"
class ScriptTester:
"""Main script testing engine"""
def __init__(self, skill_path: str, timeout: int = 30, verbose: bool = False):
self.skill_path = Path(skill_path).resolve()
self.timeout = timeout
self.verbose = verbose
self.test_suite = TestSuite(str(self.skill_path))
def log_verbose(self, message: str):
"""Log verbose message if verbose mode enabled"""
if self.verbose:
print(f"[VERBOSE] {message}", file=sys.stderr)
def test_all_scripts(self) -> TestSuite:
"""Main entry point - test all scripts in the skill"""
try:
self.log_verbose(f"Starting script testing for {self.skill_path}")
# Check if skill path exists
if not self.skill_path.exists():
self.test_suite.add_global_error(f"Skill path does not exist: {self.skill_path}")
return self.test_suite
scripts_dir = self.skill_path / "scripts"
if not scripts_dir.exists():
self.test_suite.add_global_error("No scripts directory found")
return self.test_suite
# Find all Python scripts
python_files = list(scripts_dir.glob("*.py"))
if not python_files:
self.test_suite.add_global_error("No Python scripts found in scripts directory")
return self.test_suite
self.log_verbose(f"Found {len(python_files)} Python scripts to test")
# Test each script
for script_path in python_files:
try:
result = self.test_single_script(script_path)
self.test_suite.add_script_result(result)
except Exception as e:
# Create a failed result for the script
result = ScriptTestResult(str(script_path))
result.add_error(f"Failed to test script: {str(e)}")
result.overall_status = "FAIL"
self.test_suite.add_script_result(result)
# Calculate summary
self.test_suite.calculate_summary()
except Exception as e:
self.test_suite.add_global_error(f"Testing failed with exception: {str(e)}")
return self.test_suite
def test_single_script(self, script_path: Path) -> ScriptTestResult:
"""Test a single Python script comprehensively"""
result = ScriptTestResult(str(script_path))
start_time = time.time()
try:
self.log_verbose(f"Testing script: {script_path.name}")
# Read script content
try:
content = script_path.read_text(encoding='utf-8')
except Exception as e:
result.add_test("file_readable", False, f"Cannot read file: {str(e)}")
result.add_error(f"Cannot read script file: {str(e)}")
result.overall_status = "FAIL"
return result
result.add_test("file_readable", True, "Script file is readable")
# Test 1: Syntax validation
self._test_syntax(content, result)
# Test 2: Import validation
self._test_imports(content, result)
# Test 3: Argparse validation
self._test_argparse_implementation(content, result)
# Test 4: Main guard validation
self._test_main_guard(content, result)
# Test 5: Runtime execution tests
if result.tests.get("syntax_valid", {}).get("passed", False):
self._test_script_execution(script_path, result)
# Test 6: Help functionality
if result.tests.get("syntax_valid", {}).get("passed", False):
self._test_help_functionality(script_path, result)
# Test 7: Sample data processing (if available)
self._test_sample_data_processing(script_path, result)
# Test 8: Output format validation
self._test_output_formats(script_path, result)
except Exception as e:
result.add_error(f"Unexpected error during testing: {str(e)}")
finally:
result.execution_time = time.time() - start_time
result.calculate_status()
return result
def _test_syntax(self, content: str, result: ScriptTestResult):
"""Test Python syntax validity"""
self.log_verbose("Testing syntax...")
try:
ast.parse(content)
result.add_test("syntax_valid", True, "Python syntax is valid")
except SyntaxError as e:
result.add_test("syntax_valid", False, f"Syntax error: {str(e)}",
{"error": str(e), "line": getattr(e, 'lineno', 'unknown')})
result.add_error(f"Syntax error: {str(e)}")
def _test_imports(self, content: str, result: ScriptTestResult):
"""Test import statements for external dependencies"""
self.log_verbose("Testing imports...")
try:
tree = ast.parse(content)
external_imports = self._find_external_imports(tree)
if not external_imports:
result.add_test("imports_valid", True, "Uses only standard library imports")
else:
result.add_test("imports_valid", False,
f"Uses external imports: {', '.join(external_imports)}",
{"external_imports": external_imports})
result.add_error(f"External imports detected: {', '.join(external_imports)}")
except Exception as e:
result.add_test("imports_valid", False, f"Error analyzing imports: {str(e)}")
def _find_external_imports(self, tree: ast.AST) -> List[str]:
"""Find external (non-stdlib) imports"""
# Comprehensive standard library module list
stdlib_modules = {
# Built-in modules
'argparse', 'ast', 'json', 'os', 'sys', 'pathlib', 'datetime', 'typing',
'collections', 're', 'math', 'random', 'itertools', 'functools', 'operator',
'csv', 'sqlite3', 'urllib', 'http', 'html', 'xml', 'email', 'base64',
'hashlib', 'hmac', 'secrets', 'tempfile', 'shutil', 'glob', 'fnmatch',
'subprocess', 'threading', 'multiprocessing', 'queue', 'time', 'calendar',
'locale', 'gettext', 'logging', 'warnings', 'unittest', 'doctest',
'pickle', 'copy', 'pprint', 'reprlib', 'enum', 'dataclasses',
'contextlib', 'abc', 'atexit', 'traceback', 'gc', 'weakref', 'types',
'decimal', 'fractions', 'statistics', 'cmath', 'platform', 'errno',
'io', 'codecs', 'unicodedata', 'stringprep', 'textwrap', 'string',
'struct', 'difflib', 'heapq', 'bisect', 'array', 'uuid', 'mmap',
'ctypes', 'winreg', 'msvcrt', 'winsound', 'posix', 'pwd', 'grp',
'crypt', 'termios', 'tty', 'pty', 'fcntl', 'resource', 'nis',
'syslog', 'signal', 'socket', 'ssl', 'select', 'selectors',
'asyncio', 'asynchat', 'asyncore', 'netrc', 'xdrlib', 'plistlib',
'mailbox', 'mimetypes', 'encodings', 'pkgutil', 'modulefinder',
'runpy', 'importlib', 'imp', 'zipimport', 'zipfile', 'tarfile',
'gzip', 'bz2', 'lzma', 'zlib', 'binascii', 'quopri', 'uu',
'configparser', 'netrc', 'xdrlib', 'plistlib', 'token', 'tokenize',
'keyword', 'heapq', 'bisect', 'array', 'weakref', 'types',
'copyreg', 'shelve', 'marshal', 'dbm', 'sqlite3', 'zoneinfo'
}
external_imports = []
for node in ast.walk(tree):
if isinstance(node, ast.Import):
for alias in node.names:
module_name = alias.name.split('.')[0]
if module_name not in stdlib_modules and not module_name.startswith('_'):
external_imports.append(alias.name)
elif isinstance(node, ast.ImportFrom) and node.module:
module_name = node.module.split('.')[0]
if module_name not in stdlib_modules and not module_name.startswith('_'):
external_imports.append(node.module)
return list(set(external_imports))
def _test_argparse_implementation(self, content: str, result: ScriptTestResult):
"""Test argparse implementation"""
self.log_verbose("Testing argparse implementation...")
try:
tree = ast.parse(content)
# Check for argparse import
has_argparse_import = False
has_parser_creation = False
has_parse_args = False
for node in ast.walk(tree):
if isinstance(node, (ast.Import, ast.ImportFrom)):
if (isinstance(node, ast.Import) and
any(alias.name == 'argparse' for alias in node.names)):
has_argparse_import = True
elif (isinstance(node, ast.ImportFrom) and
node.module == 'argparse'):
has_argparse_import = True
elif isinstance(node, ast.Call):
# Check for ArgumentParser creation
if (isinstance(node.func, ast.Attribute) and
isinstance(node.func.value, ast.Name) and
node.func.value.id == 'argparse' and
node.func.attr == 'ArgumentParser'):
has_parser_creation = True
# Check for parse_args call
if (isinstance(node.func, ast.Attribute) and
node.func.attr == 'parse_args'):
has_parse_args = True
argparse_score = sum([has_argparse_import, has_parser_creation, has_parse_args])
if argparse_score == 3:
result.add_test("argparse_implementation", True, "Complete argparse implementation found")
elif argparse_score > 0:
result.add_test("argparse_implementation", False,
"Partial argparse implementation",
{"missing_components": [
comp for comp, present in [
("import", has_argparse_import),
("parser_creation", has_parser_creation),
("parse_args", has_parse_args)
] if not present
]})
result.add_warning("Incomplete argparse implementation")
else:
result.add_test("argparse_implementation", False, "No argparse implementation found")
result.add_error("Script should use argparse for command-line arguments")
except Exception as e:
result.add_test("argparse_implementation", False, f"Error analyzing argparse: {str(e)}")
def _test_main_guard(self, content: str, result: ScriptTestResult):
"""Test for if __name__ == '__main__' guard"""
self.log_verbose("Testing main guard...")
has_main_guard = 'if __name__ == "__main__"' in content or "if __name__ == '__main__'" in content
if has_main_guard:
result.add_test("main_guard", True, "Has proper main guard")
else:
result.add_test("main_guard", False, "Missing main guard")
result.add_error("Script should have 'if __name__ == \"__main__\"' guard")
def _test_script_execution(self, script_path: Path, result: ScriptTestResult):
"""Test basic script execution"""
self.log_verbose("Testing script execution...")
try:
# Try to run the script with no arguments (should not crash immediately)
process = subprocess.run(
[sys.executable, str(script_path)],
capture_output=True,
text=True,
timeout=self.timeout,
cwd=script_path.parent
)
# Script might exit with error code if no args provided, but shouldn't crash
if process.returncode in (0, 1, 2): # 0=success, 1=general error, 2=misuse
result.add_test("basic_execution", True,
f"Script runs without crashing (exit code: {process.returncode})")
else:
result.add_test("basic_execution", False,
f"Script crashed with exit code {process.returncode}",
{"stdout": process.stdout, "stderr": process.stderr})
except subprocess.TimeoutExpired:
result.add_test("basic_execution", False,
f"Script execution timed out after {self.timeout} seconds")
result.add_error(f"Script execution timeout ({self.timeout}s)")
except Exception as e:
result.add_test("basic_execution", False, f"Execution error: {str(e)}")
result.add_error(f"Script execution failed: {str(e)}")
def _test_help_functionality(self, script_path: Path, result: ScriptTestResult):
"""Test --help functionality"""
self.log_verbose("Testing help functionality...")
try:
# Test --help flag
process = subprocess.run(
[sys.executable, str(script_path), '--help'],
capture_output=True,
text=True,
timeout=self.timeout,
cwd=script_path.parent
)
if process.returncode == 0:
help_output = process.stdout
# Check for reasonable help content
help_indicators = ['usage:', 'positional arguments:', 'optional arguments:',
'options:', 'description:', 'help']
has_help_content = any(indicator in help_output.lower() for indicator in help_indicators)
if has_help_content and len(help_output.strip()) > 50:
result.add_test("help_functionality", True, "Provides comprehensive help text")
else:
result.add_test("help_functionality", False,
"Help text is too brief or missing key sections",
{"help_output": help_output})
result.add_warning("Help text could be more comprehensive")
else:
result.add_test("help_functionality", False,
f"Help command failed with exit code {process.returncode}",
{"stderr": process.stderr})
result.add_error("--help flag does not work properly")
except subprocess.TimeoutExpired:
result.add_test("help_functionality", False, "Help command timed out")
except Exception as e:
result.add_test("help_functionality", False, f"Help test error: {str(e)}")
def _test_sample_data_processing(self, script_path: Path, result: ScriptTestResult):
"""Test script against sample data if available"""
self.log_verbose("Testing sample data processing...")
assets_dir = self.skill_path / "assets"
if not assets_dir.exists():
result.add_test("sample_data_processing", True, "No sample data to test (assets dir missing)")
return
# Look for sample input files
sample_files = list(assets_dir.rglob("*sample*")) + list(assets_dir.rglob("*test*"))
sample_files = [f for f in sample_files if f.is_file() and not f.name.startswith('.')]
if not sample_files:
result.add_test("sample_data_processing", True, "No sample data files found to test")
return
tested_files = 0
successful_tests = 0
for sample_file in sample_files[:3]: # Test up to 3 sample files
try:
self.log_verbose(f"Testing with sample file: {sample_file.name}")
# Try to run script with the sample file as input
process = subprocess.run(
[sys.executable, str(script_path), str(sample_file)],
capture_output=True,
text=True,
timeout=self.timeout,
cwd=script_path.parent
)
tested_files += 1
if process.returncode == 0:
successful_tests += 1
else:
self.log_verbose(f"Sample test failed for {sample_file.name}: {process.stderr}")
except subprocess.TimeoutExpired:
tested_files += 1
result.add_warning(f"Sample data test timed out for {sample_file.name}")
except Exception as e:
tested_files += 1
self.log_verbose(f"Sample test error for {sample_file.name}: {str(e)}")
if tested_files == 0:
result.add_test("sample_data_processing", True, "No testable sample data found")
elif successful_tests == tested_files:
result.add_test("sample_data_processing", True,
f"Successfully processed all {tested_files} sample files")
elif successful_tests > 0:
result.add_test("sample_data_processing", False,
f"Processed {successful_tests}/{tested_files} sample files",
{"success_rate": successful_tests / tested_files})
result.add_warning("Some sample data processing failed")
else:
result.add_test("sample_data_processing", False,
"Failed to process any sample data files")
result.add_error("Script cannot process sample data")
def _test_output_formats(self, script_path: Path, result: ScriptTestResult):
"""Test output format compliance"""
self.log_verbose("Testing output formats...")
# Test if script supports JSON output
json_support = False
human_readable_support = False
try:
# Read script content to check for output format indicators
content = script_path.read_text(encoding='utf-8')
# Look for JSON-related code
if any(indicator in content.lower() for indicator in ['json.dump', 'json.load', '"json"', '--json']):
json_support = True
# Look for human-readable output indicators
if any(indicator in content for indicator in ['print(', 'format(', 'f"', "f'"]):
human_readable_support = True
# Try running with --json flag if it looks like it supports it
if '--json' in content:
try:
process = subprocess.run(
[sys.executable, str(script_path), '--json', '--help'],
capture_output=True,
text=True,
timeout=10,
cwd=script_path.parent
)
if process.returncode == 0:
json_support = True
except:
pass
# Evaluate dual output support
if json_support and human_readable_support:
result.add_test("output_formats", True, "Supports both JSON and human-readable output")
elif json_support or human_readable_support:
format_type = "JSON" if json_support else "human-readable"
result.add_test("output_formats", False,
f"Supports only {format_type} output",
{"json_support": json_support, "human_readable_support": human_readable_support})
result.add_warning("Consider adding dual output format support")
else:
result.add_test("output_formats", False, "No clear output format support detected")
result.add_warning("Output format support is unclear")
except Exception as e:
result.add_test("output_formats", False, f"Error testing output formats: {str(e)}")
class TestReportFormatter:
"""Formats test reports for output"""
@staticmethod
def format_json(test_suite: TestSuite) -> str:
"""Format test suite as JSON"""
return json.dumps({
"skill_path": test_suite.skill_path,
"timestamp": test_suite.timestamp,
"summary": test_suite.summary,
"global_errors": test_suite.global_errors,
"script_results": {
name: {
"script_path": result.script_path,
"timestamp": result.timestamp,
"overall_status": result.overall_status,
"execution_time": round(result.execution_time, 2),
"tests": result.tests,
"errors": result.errors,
"warnings": result.warnings
}
for name, result in test_suite.script_results.items()
}
}, indent=2)
@staticmethod
def format_human_readable(test_suite: TestSuite) -> str:
"""Format test suite as human-readable text"""
lines = []
lines.append("=" * 60)
lines.append("SCRIPT TESTING REPORT")
lines.append("=" * 60)
lines.append(f"Skill: {test_suite.skill_path}")
lines.append(f"Timestamp: {test_suite.timestamp}")
lines.append("")
# Summary
if test_suite.summary:
lines.append("SUMMARY:")
lines.append(f" Total Scripts: {test_suite.summary['total_scripts']}")
lines.append(f" Passed: {test_suite.summary['passed']}")
lines.append(f" Partial: {test_suite.summary['partial']}")
lines.append(f" Failed: {test_suite.summary['failed']}")
lines.append(f" Overall Status: {test_suite.summary['overall_status']}")
lines.append("")
# Global errors
if test_suite.global_errors:
lines.append("GLOBAL ERRORS:")
for error in test_suite.global_errors:
lines.append(f" • {error}")
lines.append("")
# Individual script results
for script_name, result in test_suite.script_results.items():
lines.append(f"SCRIPT: {script_name}")
lines.append(f" Status: {result.overall_status}")
lines.append(f" Execution Time: {result.execution_time:.2f}s")
lines.append("")
# Tests
if result.tests:
lines.append(" TESTS:")
for test_name, test_result in result.tests.items():
status = "✓ PASS" if test_result["passed"] else "✗ FAIL"
lines.append(f" {status}: {test_result['message']}")
lines.append("")
# Errors
if result.errors:
lines.append(" ERRORS:")
for error in result.errors:
lines.append(f" • {error}")
lines.append("")
# Warnings
if result.warnings:
lines.append(" WARNINGS:")
for warning in result.warnings:
lines.append(f" • {warning}")
lines.append("")
lines.append("-" * 40)
lines.append("")
return "\n".join(lines)
def main():
"""Main entry point"""
parser = argparse.ArgumentParser(
description="Test Python scripts in a skill directory",
formatter_class=argparse.RawDescriptionHelpFormatter,
epilog="""
Examples:
python script_tester.py engineering/my-skill
python script_tester.py engineering/my-skill --timeout 60 --json
python script_tester.py engineering/my-skill --verbose
Test Categories:
- Syntax validation (AST parsing)
- Import validation (stdlib only)
- Argparse implementation
- Main guard presence
- Basic execution testing
- Help functionality
- Sample data processing
- Output format compliance
"""
)
parser.add_argument("skill_path",
help="Path to the skill directory containing scripts to test")
parser.add_argument("--timeout",
type=int,
default=30,
help="Timeout for script execution tests in seconds (default: 30)")
parser.add_argument("--json",
action="store_true",
help="Output results in JSON format")
parser.add_argument("--verbose",
action="store_true",
help="Enable verbose logging")
args = parser.parse_args()
try:
# Create tester and run tests
tester = ScriptTester(args.skill_path, args.timeout, args.verbose)
test_suite = tester.test_all_scripts()
# Format and output results
if args.json:
print(TestReportFormatter.format_json(test_suite))
else:
print(TestReportFormatter.format_human_readable(test_suite))
# Exit with appropriate code
if test_suite.global_errors:
sys.exit(1)
elif test_suite.summary.get("overall_status") == "FAIL":
sys.exit(1)
elif test_suite.summary.get("overall_status") == "PARTIAL":
sys.exit(2) # Partial success
else:
sys.exit(0) # Success
except KeyboardInterrupt:
print("\nTesting interrupted by user", file=sys.stderr)
sys.exit(130)
except Exception as e:
print(f"Testing failed: {str(e)}", file=sys.stderr)
if args.verbose:
import traceback
traceback.print_exc()
sys.exit(1)
if __name__ == "__main__":
main()
```
### scripts/skill_validator.py
```python
#!/usr/bin/env python3
"""
Skill Validator - Validates skill directories against quality standards
This script validates a skill directory structure, documentation, and Python scripts
against the claude-skills ecosystem standards. It checks for required files, proper
formatting, and compliance with tier-specific requirements.
Usage:
python skill_validator.py <skill_path> [--tier TIER] [--json] [--verbose]
Author: Claude Skills Engineering Team
Version: 1.0.0
Dependencies: Python Standard Library Only
"""
import argparse
import ast
import json
import re
import sys
try:
import yaml
except ImportError:
# Minimal YAML subset: parse simple key: value frontmatter without pyyaml
class _YamlStub:
class YAMLError(Exception):
pass
@staticmethod
def safe_load(text):
result = {}
for line in text.strip().splitlines():
if ':' in line:
key, _, value = line.partition(':')
result[key.strip()] = value.strip()
return result if result else None
yaml = _YamlStub()
import datetime as dt
from pathlib import Path
from typing import Dict, List, Any, Optional, Tuple
class ValidationError(Exception):
"""Custom exception for validation errors"""
pass
class ValidationReport:
"""Container for validation results"""
def __init__(self, skill_path: str):
self.skill_path = skill_path
self.timestamp = dt.datetime.now(dt.timezone.utc).isoformat().replace("+00:00", "Z")
self.checks = {}
self.warnings = []
self.errors = []
self.suggestions = []
self.overall_score = 0.0
self.compliance_level = "FAIL"
def add_check(self, check_name: str, passed: bool, message: str = "", score: float = 0.0):
"""Add a validation check result"""
self.checks[check_name] = {
"passed": passed,
"message": message,
"score": score
}
def add_warning(self, message: str):
"""Add a warning message"""
self.warnings.append(message)
def add_error(self, message: str):
"""Add an error message"""
self.errors.append(message)
def add_suggestion(self, message: str):
"""Add an improvement suggestion"""
self.suggestions.append(message)
def calculate_overall_score(self):
"""Calculate overall compliance score"""
if not self.checks:
self.overall_score = 0.0
return
total_score = sum(check["score"] for check in self.checks.values())
max_score = len(self.checks) * 1.0
self.overall_score = (total_score / max_score) * 100 if max_score > 0 else 0.0
# Determine compliance level
if self.overall_score >= 90:
self.compliance_level = "EXCELLENT"
elif self.overall_score >= 75:
self.compliance_level = "GOOD"
elif self.overall_score >= 60:
self.compliance_level = "ACCEPTABLE"
elif self.overall_score >= 40:
self.compliance_level = "NEEDS_IMPROVEMENT"
else:
self.compliance_level = "POOR"
class SkillValidator:
"""Main skill validation engine"""
# Tier requirements
TIER_REQUIREMENTS = {
"BASIC": {
"min_skill_md_lines": 100,
"min_scripts": 1,
"script_size_range": (100, 300),
"required_dirs": ["scripts"],
"optional_dirs": ["assets", "references", "expected_outputs"],
"features_required": ["argparse", "main_guard"]
},
"STANDARD": {
"min_skill_md_lines": 200,
"min_scripts": 1,
"script_size_range": (300, 500),
"required_dirs": ["scripts", "assets", "references"],
"optional_dirs": ["expected_outputs"],
"features_required": ["argparse", "main_guard", "json_output", "help_text"]
},
"POWERFUL": {
"min_skill_md_lines": 300,
"min_scripts": 2,
"script_size_range": (500, 800),
"required_dirs": ["scripts", "assets", "references", "expected_outputs"],
"optional_dirs": [],
"features_required": ["argparse", "main_guard", "json_output", "help_text", "error_handling"]
}
}
REQUIRED_SKILL_MD_SECTIONS = [
"Name", "Description", "Features", "Usage", "Examples"
]
FRONTMATTER_REQUIRED_FIELDS = [
"Name", "Tier", "Category", "Dependencies", "Author", "Version"
]
def __init__(self, skill_path: str, target_tier: Optional[str] = None, verbose: bool = False):
self.skill_path = Path(skill_path).resolve()
self.target_tier = target_tier
self.verbose = verbose
self.report = ValidationReport(str(self.skill_path))
def log_verbose(self, message: str):
"""Log verbose message if verbose mode enabled"""
if self.verbose:
print(f"[VERBOSE] {message}", file=sys.stderr)
def validate_skill_structure(self) -> ValidationReport:
"""Main validation entry point"""
try:
self.log_verbose(f"Starting validation of {self.skill_path}")
# Check if path exists
if not self.skill_path.exists():
self.report.add_error(f"Skill path does not exist: {self.skill_path}")
return self.report
if not self.skill_path.is_dir():
self.report.add_error(f"Skill path is not a directory: {self.skill_path}")
return self.report
# Run all validation checks
self._validate_required_files()
self._validate_skill_md()
self._validate_readme()
self._validate_directory_structure()
self._validate_python_scripts()
self._validate_tier_compliance()
# Calculate overall score
self.report.calculate_overall_score()
self.log_verbose(f"Validation completed. Score: {self.report.overall_score:.1f}")
except Exception as e:
self.report.add_error(f"Validation failed with exception: {str(e)}")
return self.report
def _validate_required_files(self):
"""Validate presence of required files"""
self.log_verbose("Checking required files...")
# Check SKILL.md
skill_md_path = self.skill_path / "SKILL.md"
if skill_md_path.exists():
self.report.add_check("skill_md_exists", True, "SKILL.md found", 1.0)
else:
self.report.add_check("skill_md_exists", False, "SKILL.md missing", 0.0)
self.report.add_error("SKILL.md is required but missing")
# Check README.md
readme_path = self.skill_path / "README.md"
if readme_path.exists():
self.report.add_check("readme_exists", True, "README.md found", 1.0)
else:
self.report.add_check("readme_exists", False, "README.md missing", 0.0)
self.report.add_warning("README.md is recommended but missing")
self.report.add_suggestion("Add README.md with usage instructions and examples")
def _validate_skill_md(self):
"""Validate SKILL.md content and format"""
self.log_verbose("Validating SKILL.md...")
skill_md_path = self.skill_path / "SKILL.md"
if not skill_md_path.exists():
return
try:
content = skill_md_path.read_text(encoding='utf-8')
lines = content.split('\n')
line_count = len([line for line in lines if line.strip()])
# Check line count
min_lines = self._get_tier_requirement("min_skill_md_lines", 100)
if line_count >= min_lines:
self.report.add_check("skill_md_length", True,
f"SKILL.md has {line_count} lines (≥{min_lines})", 1.0)
else:
self.report.add_check("skill_md_length", False,
f"SKILL.md has {line_count} lines (<{min_lines})", 0.0)
self.report.add_error(f"SKILL.md too short: {line_count} lines, minimum {min_lines}")
# Validate frontmatter
self._validate_frontmatter(content)
# Check required sections
self._validate_required_sections(content)
except Exception as e:
self.report.add_check("skill_md_readable", False, f"Error reading SKILL.md: {str(e)}", 0.0)
self.report.add_error(f"Cannot read SKILL.md: {str(e)}")
def _validate_frontmatter(self, content: str):
"""Validate SKILL.md frontmatter"""
self.log_verbose("Validating frontmatter...")
# Extract frontmatter
if content.startswith('---'):
try:
end_marker = content.find('---', 3)
if end_marker == -1:
self.report.add_check("frontmatter_format", False,
"Frontmatter closing marker not found", 0.0)
return
frontmatter_text = content[3:end_marker].strip()
frontmatter = yaml.safe_load(frontmatter_text)
if not isinstance(frontmatter, dict):
self.report.add_check("frontmatter_format", False,
"Frontmatter is not a valid dictionary", 0.0)
return
# Check required fields
missing_fields = []
for field in self.FRONTMATTER_REQUIRED_FIELDS:
if field not in frontmatter:
missing_fields.append(field)
if not missing_fields:
self.report.add_check("frontmatter_complete", True,
"All required frontmatter fields present", 1.0)
else:
self.report.add_check("frontmatter_complete", False,
f"Missing fields: {', '.join(missing_fields)}", 0.0)
self.report.add_error(f"Missing frontmatter fields: {', '.join(missing_fields)}")
except yaml.YAMLError as e:
self.report.add_check("frontmatter_format", False,
f"Invalid YAML frontmatter: {str(e)}", 0.0)
self.report.add_error(f"Invalid YAML frontmatter: {str(e)}")
else:
self.report.add_check("frontmatter_exists", False,
"No frontmatter found", 0.0)
self.report.add_error("SKILL.md must start with YAML frontmatter")
def _validate_required_sections(self, content: str):
"""Validate required sections in SKILL.md"""
self.log_verbose("Checking required sections...")
missing_sections = []
for section in self.REQUIRED_SKILL_MD_SECTIONS:
pattern = rf'^#+\s*{re.escape(section)}\s*$'
if not re.search(pattern, content, re.MULTILINE | re.IGNORECASE):
missing_sections.append(section)
if not missing_sections:
self.report.add_check("required_sections", True,
"All required sections present", 1.0)
else:
self.report.add_check("required_sections", False,
f"Missing sections: {', '.join(missing_sections)}", 0.0)
self.report.add_error(f"Missing required sections: {', '.join(missing_sections)}")
def _validate_readme(self):
"""Validate README.md content"""
self.log_verbose("Validating README.md...")
readme_path = self.skill_path / "README.md"
if not readme_path.exists():
return
try:
content = readme_path.read_text(encoding='utf-8')
# Check minimum content length
if len(content.strip()) >= 200:
self.report.add_check("readme_substantial", True,
"README.md has substantial content", 1.0)
else:
self.report.add_check("readme_substantial", False,
"README.md content is too brief", 0.5)
self.report.add_suggestion("Expand README.md with more detailed usage instructions")
except Exception as e:
self.report.add_check("readme_readable", False,
f"Error reading README.md: {str(e)}", 0.0)
def _validate_directory_structure(self):
"""Validate directory structure against tier requirements"""
self.log_verbose("Validating directory structure...")
required_dirs = self._get_tier_requirement("required_dirs", ["scripts"])
optional_dirs = self._get_tier_requirement("optional_dirs", [])
# Check required directories
missing_required = []
for dir_name in required_dirs:
dir_path = self.skill_path / dir_name
if dir_path.exists() and dir_path.is_dir():
self.report.add_check(f"dir_{dir_name}_exists", True,
f"{dir_name}/ directory found", 1.0)
else:
missing_required.append(dir_name)
self.report.add_check(f"dir_{dir_name}_exists", False,
f"{dir_name}/ directory missing", 0.0)
if missing_required:
self.report.add_error(f"Missing required directories: {', '.join(missing_required)}")
# Check optional directories and provide suggestions
missing_optional = []
for dir_name in optional_dirs:
dir_path = self.skill_path / dir_name
if not (dir_path.exists() and dir_path.is_dir()):
missing_optional.append(dir_name)
if missing_optional:
self.report.add_suggestion(f"Consider adding optional directories: {', '.join(missing_optional)}")
def _validate_python_scripts(self):
"""Validate Python scripts in the scripts directory"""
self.log_verbose("Validating Python scripts...")
scripts_dir = self.skill_path / "scripts"
if not scripts_dir.exists():
return
python_files = list(scripts_dir.glob("*.py"))
min_scripts = self._get_tier_requirement("min_scripts", 1)
# Check minimum number of scripts
if len(python_files) >= min_scripts:
self.report.add_check("min_scripts_count", True,
f"Found {len(python_files)} Python scripts (≥{min_scripts})", 1.0)
else:
self.report.add_check("min_scripts_count", False,
f"Found {len(python_files)} Python scripts (<{min_scripts})", 0.0)
self.report.add_error(f"Insufficient scripts: {len(python_files)}, minimum {min_scripts}")
# Validate each script
for script_path in python_files:
self._validate_single_script(script_path)
def _validate_single_script(self, script_path: Path):
"""Validate a single Python script"""
script_name = script_path.name
self.log_verbose(f"Validating script: {script_name}")
try:
content = script_path.read_text(encoding='utf-8')
# Count lines of code (excluding empty lines and comments)
lines = content.split('\n')
loc = len([line for line in lines if line.strip() and not line.strip().startswith('#')])
# Check script size against tier requirements
size_range = self._get_tier_requirement("script_size_range", (100, 1000))
min_size, max_size = size_range
if min_size <= loc <= max_size:
self.report.add_check(f"script_size_{script_name}", True,
f"{script_name} has {loc} LOC (within {min_size}-{max_size})", 1.0)
else:
self.report.add_check(f"script_size_{script_name}", False,
f"{script_name} has {loc} LOC (outside {min_size}-{max_size})", 0.5)
if loc < min_size:
self.report.add_suggestion(f"Consider expanding {script_name} (currently {loc} LOC)")
else:
self.report.add_suggestion(f"Consider refactoring {script_name} (currently {loc} LOC)")
# Parse and validate Python syntax
try:
tree = ast.parse(content)
self.report.add_check(f"script_syntax_{script_name}", True,
f"{script_name} has valid Python syntax", 1.0)
# Check for required features
self._validate_script_features(tree, script_name, content)
except SyntaxError as e:
self.report.add_check(f"script_syntax_{script_name}", False,
f"{script_name} has syntax error: {str(e)}", 0.0)
self.report.add_error(f"Syntax error in {script_name}: {str(e)}")
except Exception as e:
self.report.add_check(f"script_readable_{script_name}", False,
f"Cannot read {script_name}: {str(e)}", 0.0)
self.report.add_error(f"Cannot read {script_name}: {str(e)}")
def _validate_script_features(self, tree: ast.AST, script_name: str, content: str):
"""Validate required script features"""
required_features = self._get_tier_requirement("features_required", ["argparse", "main_guard"])
# Check for argparse usage
if "argparse" in required_features:
has_argparse = self._check_argparse_usage(tree)
self.report.add_check(f"script_argparse_{script_name}", has_argparse,
f"{'Uses' if has_argparse else 'Missing'} argparse in {script_name}", 1.0 if has_argparse else 0.0)
if not has_argparse:
self.report.add_error(f"{script_name} must use argparse for command-line arguments")
# Check for main guard
if "main_guard" in required_features:
has_main_guard = 'if __name__ == "__main__"' in content
self.report.add_check(f"script_main_guard_{script_name}", has_main_guard,
f"{'Has' if has_main_guard else 'Missing'} main guard in {script_name}", 1.0 if has_main_guard else 0.0)
if not has_main_guard:
self.report.add_error(f"{script_name} must have 'if __name__ == \"__main__\"' guard")
# Check for external imports (should only use stdlib)
external_imports = self._check_external_imports(tree)
if not external_imports:
self.report.add_check(f"script_imports_{script_name}", True,
f"{script_name} uses only standard library", 1.0)
else:
self.report.add_check(f"script_imports_{script_name}", False,
f"{script_name} uses external imports: {', '.join(external_imports)}", 0.0)
self.report.add_error(f"{script_name} uses external imports: {', '.join(external_imports)}")
def _check_argparse_usage(self, tree: ast.AST) -> bool:
"""Check if the script uses argparse"""
for node in ast.walk(tree):
if isinstance(node, ast.Import):
for alias in node.names:
if alias.name == 'argparse':
return True
elif isinstance(node, ast.ImportFrom):
if node.module == 'argparse':
return True
return False
def _check_external_imports(self, tree: ast.AST) -> List[str]:
"""Check for external (non-stdlib) imports"""
# Simplified check - a more comprehensive solution would use a stdlib module list
stdlib_modules = {
'argparse', 'ast', 'json', 'os', 'sys', 'pathlib', 'datetime', 'typing',
'collections', 're', 'math', 'random', 'itertools', 'functools', 'operator',
'csv', 'sqlite3', 'urllib', 'http', 'html', 'xml', 'email', 'base64',
'hashlib', 'hmac', 'secrets', 'tempfile', 'shutil', 'glob', 'fnmatch',
'subprocess', 'threading', 'multiprocessing', 'queue', 'time', 'calendar',
'zoneinfo', 'locale', 'gettext', 'logging', 'warnings', 'unittest',
'doctest', 'pickle', 'copy', 'pprint', 'reprlib', 'enum', 'dataclasses',
'contextlib', 'abc', 'atexit', 'traceback', 'gc', 'weakref', 'types',
'copy', 'pprint', 'reprlib', 'enum', 'decimal', 'fractions', 'statistics',
'cmath', 'platform', 'errno', 'io', 'codecs', 'unicodedata', 'stringprep',
'textwrap', 'string', 'struct', 'difflib', 'heapq', 'bisect', 'array',
'weakref', 'types', 'copyreg', 'uuid', 'mmap', 'ctypes'
}
external_imports = []
for node in ast.walk(tree):
if isinstance(node, ast.Import):
for alias in node.names:
module_name = alias.name.split('.')[0]
if module_name not in stdlib_modules:
external_imports.append(alias.name)
elif isinstance(node, ast.ImportFrom) and node.module:
module_name = node.module.split('.')[0]
if module_name not in stdlib_modules:
external_imports.append(node.module)
return list(set(external_imports))
def _validate_tier_compliance(self):
"""Validate overall tier compliance"""
if not self.target_tier:
return
self.log_verbose(f"Validating {self.target_tier} tier compliance...")
# This is a summary check - individual checks are done in other methods
critical_checks = ["skill_md_exists", "min_scripts_count", "skill_md_length"]
failed_critical = [check for check in critical_checks
if check in self.report.checks and not self.report.checks[check]["passed"]]
if not failed_critical:
self.report.add_check("tier_compliance", True,
f"Meets {self.target_tier} tier requirements", 1.0)
else:
self.report.add_check("tier_compliance", False,
f"Does not meet {self.target_tier} tier requirements", 0.0)
self.report.add_error(f"Failed critical checks for {self.target_tier} tier: {', '.join(failed_critical)}")
def _get_tier_requirement(self, requirement: str, default: Any) -> Any:
"""Get tier-specific requirement value"""
if self.target_tier and self.target_tier in self.TIER_REQUIREMENTS:
return self.TIER_REQUIREMENTS[self.target_tier].get(requirement, default)
return default
class ReportFormatter:
"""Formats validation reports for output"""
@staticmethod
def format_json(report: ValidationReport) -> str:
"""Format report as JSON"""
return json.dumps({
"skill_path": report.skill_path,
"timestamp": report.timestamp,
"overall_score": round(report.overall_score, 1),
"compliance_level": report.compliance_level,
"checks": report.checks,
"warnings": report.warnings,
"errors": report.errors,
"suggestions": report.suggestions
}, indent=2)
@staticmethod
def format_human_readable(report: ValidationReport) -> str:
"""Format report as human-readable text"""
lines = []
lines.append("=" * 60)
lines.append("SKILL VALIDATION REPORT")
lines.append("=" * 60)
lines.append(f"Skill: {report.skill_path}")
lines.append(f"Timestamp: {report.timestamp}")
lines.append(f"Overall Score: {report.overall_score:.1f}/100 ({report.compliance_level})")
lines.append("")
# Group checks by category
structure_checks = {k: v for k, v in report.checks.items() if k.startswith(('skill_md', 'readme', 'dir_'))}
script_checks = {k: v for k, v in report.checks.items() if k.startswith('script_')}
other_checks = {k: v for k, v in report.checks.items() if k not in structure_checks and k not in script_checks}
if structure_checks:
lines.append("STRUCTURE VALIDATION:")
for check_name, result in structure_checks.items():
status = "✓ PASS" if result["passed"] else "✗ FAIL"
lines.append(f" {status}: {result['message']}")
lines.append("")
if script_checks:
lines.append("SCRIPT VALIDATION:")
for check_name, result in script_checks.items():
status = "✓ PASS" if result["passed"] else "✗ FAIL"
lines.append(f" {status}: {result['message']}")
lines.append("")
if other_checks:
lines.append("OTHER CHECKS:")
for check_name, result in other_checks.items():
status = "✓ PASS" if result["passed"] else "✗ FAIL"
lines.append(f" {status}: {result['message']}")
lines.append("")
if report.errors:
lines.append("ERRORS:")
for error in report.errors:
lines.append(f" • {error}")
lines.append("")
if report.warnings:
lines.append("WARNINGS:")
for warning in report.warnings:
lines.append(f" • {warning}")
lines.append("")
if report.suggestions:
lines.append("SUGGESTIONS:")
for suggestion in report.suggestions:
lines.append(f" • {suggestion}")
lines.append("")
return "\n".join(lines)
def main():
"""Main entry point"""
parser = argparse.ArgumentParser(
description="Validate skill directories against quality standards",
formatter_class=argparse.RawDescriptionHelpFormatter,
epilog="""
Examples:
python skill_validator.py engineering/my-skill
python skill_validator.py engineering/my-skill --tier POWERFUL --json
python skill_validator.py engineering/my-skill --verbose
Tier Options:
BASIC - Basic skill requirements (100+ lines SKILL.md, 1+ script)
STANDARD - Standard skill requirements (200+ lines, advanced features)
POWERFUL - Powerful skill requirements (300+ lines, comprehensive features)
"""
)
parser.add_argument("skill_path",
help="Path to the skill directory to validate")
parser.add_argument("--tier",
choices=["BASIC", "STANDARD", "POWERFUL"],
help="Target tier for validation (optional)")
parser.add_argument("--json",
action="store_true",
help="Output results in JSON format")
parser.add_argument("--verbose",
action="store_true",
help="Enable verbose logging")
args = parser.parse_args()
try:
# Create validator and run validation
validator = SkillValidator(args.skill_path, args.tier, args.verbose)
report = validator.validate_skill_structure()
# Format and output report
if args.json:
print(ReportFormatter.format_json(report))
else:
print(ReportFormatter.format_human_readable(report))
# Exit with error code if validation failed
if report.errors or report.overall_score < 60:
sys.exit(1)
else:
sys.exit(0)
except KeyboardInterrupt:
print("\nValidation interrupted by user", file=sys.stderr)
sys.exit(130)
except Exception as e:
print(f"Validation failed: {str(e)}", file=sys.stderr)
if args.verbose:
import traceback
traceback.print_exc()
sys.exit(1)
if __name__ == "__main__":
main()
```