testing

Original source / Raw SKILL.md

---
name: testing
description: "Applies TDD process, test quality criteria, and mock guidelines. Use when: writing unit tests, using mocks, or reviewing test quality."
---

# Testing Rules

## Language-Specific References

For language-specific rules, also read:
- **TypeScript/Vitest**: [references/typescript.md](references/typescript.md)

## TDD Process [MANDATORY for all code changes]

**Execute this process for every code change:**

### RED Phase
1. Write test that defines expected behavior
2. Run test
3. Confirm test FAILS (if it passes, the test is wrong)

### GREEN Phase
1. Write MINIMAL code to make test pass
2. Run test
3. Confirm test PASSES

### REFACTOR Phase
1. Improve code quality
2. Run test
3. Confirm test STILL PASSES

### VERIFY Phase [MANDATORY - 0 ERRORS REQUIRED]
1. Execute ALL quality check commands for your language/project
2. Fix any errors until ALL commands pass with 0 errors
3. Confirm no regressions
4. ENFORCEMENT: Cannot proceed with ANY errors or warnings

**Exceptions (no TDD required):**
- Pure configuration files
- Documentation only
- Emergency fixes (but add tests immediately after)
- Exploratory spikes (discard or rewrite with tests before merging)
- Build/deployment scripts (unless they contain business logic)

## Basic Testing Policy

### Quality Requirements
- **Coverage**: Unit test coverage must be 80% or higher
- **Independence**: Each test can run independently
- **Reproducibility**: Tests are environment-independent

### Coverage Requirements
**Mandatory**: Unit test coverage must be 80% or higher
**Metrics**: Statements, Branches, Functions, Lines

### Test Types and Scope
1. **Unit Tests**
   - Verify behavior of individual units (functions, modules, or components)
   - Mock all external dependencies
   - Fast execution (milliseconds)

2. **Integration Tests**
   - Verify coordination between multiple components
   - Use actual dependencies when appropriate
   - Test real system interactions

3. **E2E Tests (End-to-End Tests)**
   - Verify complete user workflows across entire system
   - Test real-world scenarios with all components integrated
   - Validate system behavior from user perspective
   - Ensure business requirements are met end-to-end

4. **Cross-functional Verification in E2E Tests** [MANDATORY for feature modifications]

   **Purpose**: Prevent regression and ensure existing features remain stable when introducing new features or modifications.

   **When Required**:
   - Adding new features that interact with existing components
   - Modifying core business logic or workflows
   - Changing shared resources or data structures
   - Updating APIs or integration points

   **Integration Point Analysis**:
   - **High Impact**: Changes to core process flows, breaking changes, or workflow modifications
     - Mandatory comprehensive E2E test coverage
     - Full regression test suite required
     - Performance benchmarking before/after

   - **Medium Impact**: Data usage modifications, shared state changes, or new dependencies
     - Integration tests minimum requirement
     - Targeted E2E tests for affected workflows
     - Edge case coverage mandatory

   - **Low Impact**: Read-only operations, logging additions, or monitoring hooks
     - Unit test coverage sufficient
     - Smoke tests for integration points

   **Success Criteria**:
   - Zero breaking changes in existing workflows
   - Performance degradation within project-defined acceptable limits
   - No new errors in previously stable features
   - All integration points maintain expected contracts
   - Backward compatibility preserved where required

## Test Design Principles

### Test Case Structure
- Tests consist of three stages: **Setup, Execute, Verify** (also known as Arrange-Act-Assert or Given-When-Then)
- Clear naming that shows purpose of each test
- One test case verifies only one behavior
- Test names should describe expected behavior, not implementation

### Test Data Management
- Manage test data in dedicated directories
- Define test-specific configuration values
- Always mock sensitive information (passwords, tokens, API keys)
- Keep test data minimal, using only data directly related to test case verification

### Test Independence
- Each test should be able to run in isolation
- Tests should not depend on execution order
- Clean up test state after each test
- Avoid shared mutable state between tests

## Mock and Stub Usage Policy

✅ **Recommended: Mock external dependencies in unit tests**
- Merit: Ensures test independence and reproducibility
- Practice: Mock databases, APIs, file systems, and other external dependencies
- Use framework-appropriate mocking tools

❌ **Avoid: Actual external connections in unit tests**
- Reason: Slows test speed and causes environment-dependent problems
- Exception: Integration tests that specifically test external integration

### Mock Decision Criteria
| Mock Characteristics | Response Policy |
|---------------------|-----------------|
| **Simple and stable** | Consolidate in common helpers |
| **Complex or frequently changing** | Individual implementation |
| **Duplicated in 3+ places** | Consider consolidation |
| **Test-specific logic** | Individual implementation |

## Test Granularity Principles

### Core Principle: Observable Behavior Only
**MUST Test**:
- Public APIs and contracts
- Return values and outputs
- Exceptions and error conditions
- External calls and side effects
- Persisted state changes

**MUST NOT Test**:
- Internal implementation details not exposed publicly
- Internal state that's not observable from outside
- Algorithm implementation details
- Framework/library internals

### Test Failure Response Decision Criteria

**Fix tests when:**
- Expected values are wrong
- Tests reference non-existent features
- Tests depend on implementation details
- Tests were written only for coverage

**Fix implementation when:**
- Tests represent valid specifications
- Business logic requirements have changed
- Important edge cases are failing

**When in doubt**: Confirm with stakeholders or domain experts

## Test Implementation Best Practices

### Naming Conventions
- Test files: Follow your language/framework conventions
- Test suites: Names describing target features or situations
- Test cases: Names describing expected behavior (not implementation)

### Test Code Quality Rules

✅ **Recommended: Keep all tests always active**
- Merit: Guarantees test suite completeness
- Practice: Fix problematic tests and activate them

❌ **Avoid: Skipping or commenting out tests**
- Reason: Creates test gaps and incomplete quality checks
- Solution: Either fix the test or completely delete if truly unnecessary

## Test Quality Criteria [MANDATORY]

1. **Boundary coverage**: Include empty/zero/max/error cases with happy paths
2. **Literal expectations**: Use literal values in assertions, not computed expressions
   - Expected value ≠ mock return value (implementation processes data)
3. **Result verification**: Assert return values and state, not call order
4. **Meaningful assertions**: Every test must have at least one assertion
5. **Mock external I/O only**: Mock DB/API/filesystem, use real internal utilities

### Test Helper Guidelines

**Basic Principles**
Test helpers should reduce duplication and improve maintainability.

**Usage Examples**
- Builder patterns for test data creation
- Custom assertions for domain-specific validation
- Shared setup/teardown utilities
- Common mock configurations

## Quality Check Commands [MANDATORY for VERIFY phase]

**Execute quality checks appropriate for your language and project setup:**

### Required Quality Checks
Your project MUST have mechanisms to verify:

1. **All Tests Pass**
   - Unit tests execute successfully
   - Integration tests (if applicable) pass
   - No test failures or errors

2. **Code Builds Successfully**
   - Compilation succeeds (for compiled languages)
   - No build errors or warnings

3. **Code Style Compliance**
   - Linting rules are satisfied
   - Formatting standards are met
   - Style guide adherence verified

4. **Type Safety** (for typed languages)
   - Type checking passes
   - No type errors or warnings

### ENFORCEMENT
- Cannot proceed with task completion if ANY quality check fails
- Must fix all errors and warnings before marking task complete
- If your project lacks certain quality tools, establish them or document the gap


---

## Referenced Files

> The following files are referenced in this skill and included for context.

### references/typescript.md

```markdown
# TypeScript/Vitest Testing Rules

## Test Framework
- **Vitest**: This project uses Vitest
- Test imports: `import { describe, it, expect, beforeEach, vi } from 'vitest'`
- Mock creation: Use `vi.mock()`

## Directory Structure

**File structure:**
- src/application/services/service.ts: Main service file
- src/application/services/__tests__/service.test.ts: Unit tests
- src/application/services/__tests__/service.int.test.ts: Integration tests

**Naming Conventions:**
- Test files: `{target-file-name}.test.ts`
- Integration test files: `{target-file-name}.int.test.ts`

## Cross-functional E2E Test Patterns

```typescript
describe('Cross-functional E2E Tests', () => {
  // Pattern 1: Baseline → Change → Verify
  it('should maintain existing behavior after new feature', async () => {
    // 1. Capture baseline
    const baseline = await testExistingFeature()

    // 2. Enable new feature
    await enableNewFeature()

    // 3. Verify continuity
    const result = await testExistingFeature()
    expect(result).toEqual(baseline)
    expect(result.responseTime).toBeLessThan(
      baseline.responseTime * 1.2 // Project-specific threshold
    )
  })

  // Pattern 2: Data integrity across features
  it('should preserve data integrity', async () => {
    const data = await createTestData()
    await newFeatureOperation(data.id)

    const retrieved = await existingFeatureGet(data.id)
    expect(retrieved).toEqual(data) // No unexpected mutations
  })
})
```

**Note**: LLM outputs naturally vary - test behavior, not exact matches

## Test Helper Usage Examples

```typescript
// ✅ Recommended: Utilize builder pattern
const testData = new TestDataBuilder()
  .withDefaults()
  .withName('Test User')
  .build()

// ✅ Recommended: Custom assertions
function assertValidUser(user: unknown): asserts user is User {
  // Validation logic
}

// ❌ Avoid: Individual implementation of duplicate complex mocks
```

## Test Granularity Examples

```typescript
// ✅ Test observable behavior
expect(calculatePrice(100, 0.1)).toBe(110)

// ❌ Test implementation details (as any access)
expect((calculator as any).taxRate).toBe(0.1)
expect((service as any).validate(input)).toBe(true)
```

## Mock Type Safety Enforcement

### Minimal Type Definition Requirements
```typescript
// ✅ Only required parts
type TestRepo = Pick<Repository, 'find' | 'save'>
const mock: TestRepo = { find: vi.fn(), save: vi.fn() }

// Only when absolutely necessary, with clear justification
const sdkMock = {
  call: vi.fn()
} as unknown as ExternalSDK // Complex external SDK type structure
```

## Basic Vitest Example

```typescript
import { describe, it, expect, beforeEach, vi } from 'vitest'

// Mock setup example
vi.mock('./userService', () => ({
  getUserById: vi.fn(),
  updateUser: vi.fn()
}))

describe('ComponentName', () => {
  it('should follow AAA pattern', () => {
    // Arrange
    const input = 'test'

    // Act
    const result = someFunction(input)

    // Assert
    expect(result).toBe('expected')
  })
})
```

## Quality Check Commands [MANDATORY for VERIFY phase]

**ALL TypeScript/JavaScript commands MUST pass with 0 errors before task completion:**

```bash
npm test              # MUST pass all tests
npm run build        # MUST build successfully
npm run lint         # MUST have 0 lint errors
npm run type-check   # MUST have 0 type errors
```

**ENFORCEMENT:**
- Run ALL applicable commands listed above
- Fix ANY errors or warnings before marking task complete
- If command doesn't exist in package.json, skip that specific command
- Document which commands were run in task completion

```