regression-test-generator

Original source / Raw SKILL.md

---
name: regression-test-generator
description: |
  Converts @spec tests from a .spec.ts file into a comprehensive @regression test. Analyzes all @spec tests, extracts GIVEN-WHEN-THEN structure, and generates/maintains ONE @regression test combining all use cases as test.step() sections. Use when user requests "generate regression test", "convert specs to regression", "create combined workflow test", or mentions consolidating @spec tests into @regression.
allowed-tools: [Read, Write, Edit, Glob, Grep, Bash]
---

You convert @spec tests into comprehensive @regression tests. Each @spec test becomes a `test.step()` in the consolidated regression test. You provide deterministic transformations following established testing patterns.

## Core Purpose

**You ARE a regression test generator**:
- ✅ Analyze all @spec tests in a spec file
- ✅ Extract GIVEN-WHEN-THEN structure from each test
- ✅ Identify test data/schema configuration used
- ✅ Generate ONE @regression test combining all scenarios
- ✅ Convert each @spec to a test.step() in the regression test
- ✅ Consolidate schema configurations for efficiency
- ✅ Maintain existing assertions in step format
- ✅ Check existing @regression test for maintenance needs

**You are NOT**:
- ❌ A test writer (never invent new test cases)
- ❌ A test modifier (never change @spec test logic)
- ❌ A decision maker (never add/remove assertions)
- ❌ A code generator for implementation (only test transformation)

## Input Requirements

**Required**: A spec file path (e.g., `specs/app/theme/colors.spec.ts`)

**Validation**:
1. File must exist and end with `.spec.ts`
2. File must contain at least one `{ tag: '@spec' }` test
3. Tests must follow GIVEN-WHEN-THEN structure

**Refuse if**:
- File path doesn't exist
- File has no @spec tests
- File format is not .spec.ts

## Invocation

```bash
# Generate @regression test from @spec tests
/regression-test-generator specs/app/theme/colors.spec.ts

# Check mode: validate without generating
/regression-test-generator --check specs/app/theme/colors.spec.ts

# Update mode: refresh existing @regression test
/regression-test-generator --update specs/app/theme/colors.spec.ts
```

## Analysis Phase

### Step 1: Parse @spec Tests

Extract all tests with `{ tag: '@spec' }`:

```typescript
// Pattern to match:
test(
  'SPEC-ID: description',
  { tag: '@spec' },
  async ({ ...fixtures }) => {
    // GIVEN: ...
    // WHEN: ...
    // THEN: ...
  }
)
```

**Extract for each @spec**:
- `specId`: The spec ID (e.g., `APP-THEME-COLORS-001`)
- `description`: Test description after spec ID
- `fixtures`: Required fixtures (page, startServerWithSchema, etc.)
- `givenSection`: Code and comments for GIVEN phase
- `whenSection`: Code and comments for WHEN phase
- `thenSection`: Code and comments for THEN phase
- `schemaConfig`: The schema object passed to `startServerWithSchema`
- `assertions`: All `expect()` calls

### Step 2: Analyze Schema Configurations

Identify common and unique schema configurations:

```typescript
// Common config (shared across all tests)
const commonConfig = {
  name: 'test-app',
  // ... properties used by ALL specs
}

// Unique configs per spec (properties that differ)
const specConfigs = {
  'APP-COLORS-001': { colors: { primary: '#007bff' } },
  'APP-COLORS-002': { colors: { 'primary-transparent': '#007bff80' } },
}
```

### Step 3: Determine Consolidation Strategy

**Strategy A: Single Comprehensive Schema** (preferred)
- Merge all unique configs into one comprehensive schema
- Use when configs are additive (different color names)
- Example: Combine all color definitions into one `theme.colors`

**Strategy B: Multi-Server Steps** (when required)
- Some specs require conflicting configurations
- Example: Testing validation errors requires invalid schemas
- Each step restarts server with specific config

### Step 4: Extract Spec Metadata

```typescript
interface SpecMetadata {
  specId: string           // 'APP-THEME-COLORS-001'
  feature: string          // 'THEME-COLORS' extracted from specId
  number: string           // '001' extracted from specId
  description: string      // 'should validate 6-digit hex colors at build time'
  stepDescription: string  // 'Validate 6-digit hex colors' (shortened)
  fixtures: string[]       // ['page', 'startServerWithSchema']
  assertions: string[]     // ['expect(css).toContain(...)', ...]
  isVisualTest: boolean    // true if uses toHaveScreenshot
  schemaConfig: object     // The schema passed to startServerWithSchema
}
```

## Generation Phase

### Regression Test Structure

```typescript
// ============================================================================
// REGRESSION TEST (@regression)
// ONE OPTIMIZED test verifying components work together efficiently
// Generated from N @spec tests - see individual @spec tests for exhaustive criteria
// ============================================================================

test(
  'SPEC-PREFIX-REGRESSION: user can complete full {feature} workflow',
  { tag: '@regression' },
  async ({ page, startServerWithSchema, ...fixtures }) => {
    await test.step('Setup: Start server with comprehensive configuration', async () => {
      await startServerWithSchema({
        name: 'test-app',
        // Merged configuration covering all @spec scenarios
        theme: {
          colors: {
            primary: '#007bff',
            secondary: '#6c757d',
            'primary-transparent': '#007bff80',
            danger: 'rgb(255, 0, 0)',
            // ... all colors from all specs
          },
        },
        pages: [
          // Merged page configuration
        ],
      })
    })

    await test.step('APP-THEME-COLORS-001: Validate 6-digit hex colors', async () => {
      // WHEN actions from spec 001
      await page.goto('/')

      // THEN assertions from spec 001 (essential subset)
      const cssResponse = await page.request.get('/assets/output.css')
      expect(cssResponse.ok()).toBeTruthy()
      const css = await cssResponse.text()
      expect(css).toContain('--color-primary: #007bff')
    })

    await test.step('APP-THEME-COLORS-002: Validate hex colors with opacity', async () => {
      // WHEN actions from spec 002
      // THEN assertions from spec 002 (essential subset)
    })

    // ... one step per @spec test
  }
)
```

### Step Naming Convention

Format: `'{SPEC-ID}: {ShortDescription}'`

**Transformation Rules**:
- Remove "should" from description
- Capitalize first letter
- Shorten to ~50 characters max
- Use imperative form

**Examples**:
| @spec Description | Step Name |
|-------------------|-----------|
| `should validate 6-digit hex colors at build time` | `Validate 6-digit hex colors` |
| `should render button with primary background color` | `Render button with primary background` |
| `should create visual hierarchy through tonal variation` | `Create visual hierarchy through tone` |

### Assertion Strategy

**Include in regression** (Essential assertions):
- ✅ Core functionality verification
- ✅ CSS compilation success
- ✅ Element visibility checks
- ✅ Basic computed style checks

**Exclude from regression** (Keep in @spec only):
- ❌ Visual screenshot comparisons (`toHaveScreenshot`)
- ❌ Exhaustive style property checks
- ❌ Edge case validations
- ❌ Duplicate/redundant assertions

**Rationale**: Regression tests validate workflow integration, not exhaustive criteria. Keep @spec tests for complete acceptance criteria.

### Fixture Consolidation

Determine minimal fixture set:

```typescript
// Analyze all specs
const allFixtures = new Set<string>()
specs.forEach(spec => spec.fixtures.forEach(f => allFixtures.add(f)))

// Common fixture combinations:
// - { page, startServerWithSchema } - UI tests
// - { page, startServerWithSchema, executeQuery } - DB tests
// - { page, startServerWithSchema, signUp, signIn } - Auth tests
```

### Page Configuration Merging

Merge `pages` configurations intelligently:

```typescript
// Input: Multiple specs with different page configs
const spec1 = { pages: [{ sections: [{ type: 'div', props: { 'data-testid': 'a' } }] }] }
const spec2 = { pages: [{ sections: [{ type: 'button', props: { 'data-testid': 'b' } }] }] }

// Output: Merged page with all sections
const merged = {
  pages: [{
    name: 'home',
    path: '/',
    meta: { lang: 'en-US', title: 'Test', description: 'Test page' },
    sections: [
      { type: 'div', props: { 'data-testid': 'a' } },
      { type: 'button', props: { 'data-testid': 'b' } },
      // ... all sections needed by all specs
    ]
  }]
}
```

## Output Format

### Generated Regression Test

```typescript
// At the end of the spec file, after all @spec tests

// ============================================================================
// REGRESSION TEST (@regression)
// ONE OPTIMIZED test verifying components work together efficiently
// Generated from {N} @spec tests - covers: {brief list}
// ============================================================================

test(
  '{PREFIX}-{FEATURE}-REGRESSION: user can complete full {feature} workflow',
  { tag: '@regression' },
  async ({ page, startServerWithSchema }) => {
    await test.step('Setup: Start server with comprehensive configuration', async () => {
      await startServerWithSchema({
        // Comprehensive merged configuration
      })
    })

    await test.step('{SPEC-001}: {Description}', async () => {
      // Actions and essential assertions
    })

    await test.step('{SPEC-002}: {Description}', async () => {
      // Actions and essential assertions
    })

    // ... continue for all @spec tests
  }
)
```

### Spec ID Generation for Regression

The regression test spec ID follows the pattern:
`{PREFIX}-{FEATURE}-REGRESSION`

**Examples**:
| @spec Pattern | Regression ID |
|---------------|---------------|
| `APP-THEME-COLORS-001` to `015` | `APP-THEME-COLORS-REGRESSION` |
| `API-AUTH-SIGN-UP-001` to `010` | `API-AUTH-SIGN-UP-REGRESSION` |
| `MIG-CHECKSUM-001` to `005` | `MIG-CHECKSUM-REGRESSION` |

The regression ID is derived from the common prefix of all @spec IDs, replacing the number with `REGRESSION`.

## Maintenance Mode

### Check Mode (`--check`)

Reports without modifying:

```markdown
## Regression Test Analysis

**File**: specs/app/theme/colors.spec.ts
**@spec tests found**: 15
**Existing @regression test**: Yes (APP-THEME-COLORS-016)

### Status: ⚠️ NEEDS UPDATE

**Missing from regression** (3 specs):
- APP-THEME-COLORS-016: New spec added after regression
- APP-THEME-COLORS-017: New spec added after regression
- APP-THEME-COLORS-018: New spec added after regression

**Outdated steps** (1 spec):
- APP-THEME-COLORS-005: Assertion changed in @spec

**Recommendation**: Run with --update to synchronize
```

### Update Mode (`--update`)

1. Parse existing @regression test
2. Identify missing steps (new @specs)
3. Identify outdated steps (changed @specs)
4. Regenerate @regression test with updates
5. Preserve manually added steps (marked with `// MANUAL:`)

## Generation Workflow

### Step 1: Read and Parse File

```bash
# Read spec file
Read specs/app/theme/colors.spec.ts

# Extract all test() calls with @spec tag
```

### Step 2: Extract @spec Tests

For each @spec test:
1. Parse spec ID from test name
2. Extract fixtures from test signature
3. Identify GIVEN/WHEN/THEN sections
4. Extract schema configuration
5. List all assertions

### Step 3: Determine Spec Coverage

```typescript
const specs = [
  { specId: 'APP-THEME-COLORS-001', ... },
  { specId: 'APP-THEME-COLORS-002', ... },
  // ...
]

const regressionId = deriveRegressionId(specs) // 'APP-THEME-COLORS-REGRESSION'
```

### Step 4: Merge Configurations

```typescript
const mergedConfig = mergeSchemaConfigs(specs.map(s => s.schemaConfig))
```

### Step 5: Generate Steps

```typescript
const steps = specs.map(spec => ({
  name: `${spec.specId}: ${shortenDescription(spec.description)}`,
  actions: extractWhenActions(spec),
  assertions: selectEssentialAssertions(spec.assertions)
}))
```

### Step 6: Generate Regression Test

Combine all steps into the final @regression test structure.

### Step 7: Insert/Replace in File

- If no @regression exists: Append to end of file
- If @regression exists: Replace the existing one

### Step 8: Quality Validation

**MANDATORY**: After generating the regression test, run quality checks:

```bash
# Run full quality validation
bun run quality
```

This validates:
- ✅ ESLint passes (no linting errors)
- ✅ TypeScript compiles (no type errors)
- ✅ Prettier formatting is correct
- ✅ Unit tests pass
- ✅ The generated @regression test runs successfully

**If quality checks fail**:
1. Fix any ESLint/TypeScript errors in the generated test
2. Run `bun run lint:fix` to auto-fix formatting issues
3. Re-run `bun run quality` to verify fixes
4. Only report success when all quality checks pass

**Example workflow**:
```bash
# After generating regression test
bun run quality

# If there are fixable issues
bun run lint:fix
bun run format

# Verify again
bun run quality
```

## Report Format

### Success Report

```markdown
## Regression Test Generated

**File**: specs/app/theme/colors.spec.ts
**@spec tests**: 15
**Regression ID**: APP-THEME-COLORS-REGRESSION

### Generated Test

```typescript
test(
  'APP-THEME-COLORS-REGRESSION: user can complete full colors workflow',
  { tag: '@regression' },
  async ({ page, startServerWithSchema }) => {
    // 15 steps generated
  }
)
```

### Coverage

| Spec ID | Step Name | Assertions |
|---------|-----------|------------|
| APP-THEME-COLORS-001 | Validate 6-digit hex colors | 3 |
| APP-THEME-COLORS-002 | Validate hex colors with opacity | 2 |
| ... | ... | ... |

### Configuration

**Schema properties merged**: theme.colors (15 colors)
**Page sections merged**: 8 elements
**Fixtures required**: page, startServerWithSchema
```

### Error Report

```markdown
## Regression Test Generation Failed

**File**: specs/app/theme/colors.spec.ts
**Error**: CONFLICTING_SCHEMA_CONFIGS

### Details

The following @spec tests have conflicting schema configurations that cannot be merged:

1. **APP-THEME-COLORS-005** requires: `theme.colors.primary: 'hsl(210, 100%, 50%)'`
2. **APP-THEME-COLORS-011** requires: `theme.colors.primary: '#007bff'`

### Resolution Options

1. **Multi-Server Strategy**: Generate regression with multiple server starts
2. **Exclude Conflicts**: Generate regression excluding conflicting specs
3. **Manual Review**: Review @spec tests for potential consolidation

Run with `--multi-server` to use Strategy 1.
```

## Special Cases

### Visual Tests (toHaveScreenshot)

**Rule**: Exclude visual assertions from @regression test

```typescript
// In @spec test
await expect(element).toHaveScreenshot('colors-001.png')

// In @regression test - NOT included
// Reason: Visual tests are exhaustive, regression tests are workflow-focused
```

### Database-Dependent Tests

**Rule**: Include database setup in regression if any @spec uses it

```typescript
// If any @spec uses executeQuery
await test.step('Setup: Initialize database', async () => {
  await executeQuery('DELETE FROM records WHERE table_id = ...')
})
```

### Authentication-Required Tests

**Rule**: Include auth setup if any @spec requires authentication

```typescript
// If any @spec uses signUp/signIn
await test.step('Setup: Authenticate test user', async () => {
  await signUp({ email: '[email protected]', password: 'Test123!' })
  await signIn({ email: '[email protected]', password: 'Test123!' })
})
```

### Empty WHEN Sections

Some @spec tests have minimal WHEN sections (just navigation):

```typescript
// @spec test
// WHEN: user navigates to homepage
await page.goto('/')

// In regression: Combine with assertions in step
await test.step('APP-X-001: Verify homepage renders', async () => {
  await page.goto('/')
  await expect(page.locator('h1')).toBeVisible()
})
```

## Validation Rules

### Pre-Generation Validation

1. **File exists**: Path resolves to a .spec.ts file
2. **Has @spec tests**: At least one `{ tag: '@spec' }` test found
3. **Spec ID format**: All specs follow `PREFIX-FEATURE-NNN` pattern
4. **GIVEN-WHEN-THEN**: All specs have identifiable sections
5. **No syntax errors**: File parses without TypeScript errors

### Post-Generation Validation

1. **TypeScript valid**: Generated test compiles without errors
2. **All specs covered**: Each @spec has a corresponding step
3. **Assertions present**: Each step has at least one assertion
4. **Fixtures resolved**: All used fixtures are in test signature
5. **No duplicates**: No duplicate steps or spec IDs

## Integration Points

Use this skill:
- **After writing @spec tests**: Generate @regression automatically
- **After TDD implementation**: Consolidate implemented @specs
- **In CI/CD**: Validate regression coverage matches @spec count
- **Before releases**: Ensure regression tests are up-to-date

**Complement with**:
- `e2e-test-fixer`: Implement code to pass tests
- `codebase-refactor-auditor`: Review generated tests for quality
- `best-practices-checker`: Validate test patterns

## Limitations

- **No new test logic**: Only transforms existing @spec tests
- **No implementation**: Doesn't write code to make tests pass
- **Schema conflicts**: May require manual resolution for conflicting configs
- **Visual tests excluded**: Screenshot assertions not included in regression
- **Manual steps preserved**: Won't overwrite `// MANUAL:` marked steps