ansible-testing

Original source / Raw SKILL.md

---
name: ansible-testing
description: >
  This skill should be used when running ansible-lint, configuring linting rules,
  testing Ansible playbooks, validating playbook syntax, or setting up integration
  tests. Covers ansible-lint configuration and testing strategies.
---

# Ansible Testing

Testing strategies and ansible-lint configuration for Ansible automation.

## ansible-lint

### Running Lint

```bash
# Via mise (recommended)
mise run ansible-lint

# Directly with uv
uv run ansible-lint ansible/playbooks/

# Specific file
uv run ansible-lint ansible/playbooks/my-playbook.yml

# With verbose output
uv run ansible-lint -v ansible/
```

### Configuration File

Located at `ansible/.ansible-lint`:

```yaml
---
# Profile: null, min, basic, moderate, safety, shared, production
profile: moderate

# Offline mode - don't download Galaxy requirements
offline: true

# Exclude paths
exclude_paths:
  - .cache/
  - .venv/
  - .git/
  - "*/templates/"
  - "*.j2"
  - .deprecated/

# Rules to skip completely
skip_list:
  - var-naming[no-role-prefix]  # We use descriptive names
  - run-once[task]              # Safe with our strategy
  - command-instead-of-module   # CLI tools require command
  - yaml[line-length]           # Long lines in infra configs

# Rules to warn but not fail
warn_list:
  - fqcn[action-core]
  - fqcn[action]
  - no-handler
  - name[play]
```

### Common Rule Categories

| Category | Description |
|----------|-------------|
| `fqcn` | Fully qualified collection names |
| `yaml` | YAML formatting (indentation, line length) |
| `name` | Task/play naming conventions |
| `command-instead-of-module` | Using command when module exists |
| `no-changed-when` | Missing changed_when on command |
| `risky-file-permissions` | Missing explicit file permissions |

### Fixing Common Issues

**Missing name on task:**

```yaml
# BAD
- ansible.builtin.apt:
    name: nginx

# GOOD
- name: Install nginx
  ansible.builtin.apt:
    name: nginx
```

**Short module name:**

```yaml
# BAD (triggers fqcn warning)
- name: Install package
  apt:
    name: nginx

# GOOD
- name: Install package
  ansible.builtin.apt:
    name: nginx
```

**Using shell instead of command:**

```yaml
# BAD (when no shell features needed)
- name: List files
  ansible.builtin.shell: ls -la /tmp

# GOOD
- name: List files
  ansible.builtin.command: ls -la /tmp
  changed_when: false
```

**Missing changed_when:**

```yaml
# BAD (always shows changed)
- name: Check status
  ansible.builtin.command: systemctl status app

# GOOD
- name: Check status
  ansible.builtin.command: systemctl status app
  register: status_check
  changed_when: false
  failed_when: false
```

## Syntax Checking

Validate playbook syntax before running:

```bash
# Check syntax only
uv run ansible-playbook --syntax-check playbooks/my-playbook.yml

# Check mode (dry run)
uv run ansible-playbook playbooks/my-playbook.yml --check

# Diff mode (show changes)
uv run ansible-playbook playbooks/my-playbook.yml --check --diff
```

## Idempotency Testing

Verify playbooks are idempotent by running twice:

```bash
# First run - may show changes
uv run ansible-playbook playbooks/setup.yml

# Second run - should show 0 changes
uv run ansible-playbook playbooks/setup.yml

# If second run shows changes, playbook is NOT idempotent
```

### Automated Idempotency Check

```bash
#!/bin/bash
set -euo pipefail

PLAYBOOK="$1"

echo "First run..."
uv run ansible-playbook "$PLAYBOOK"

echo "Second run (checking idempotency)..."
OUTPUT=$(uv run ansible-playbook "$PLAYBOOK" 2>&1)

if echo "$OUTPUT" | grep -q "changed=0"; then
    echo "✓ Playbook is idempotent"
    exit 0
else
    echo "✗ Playbook is NOT idempotent"
    echo "$OUTPUT" | grep -E "(changed|failed)="
    exit 1
fi
```

## Integration Testing

### Test Against Real Infrastructure

```bash
# Limit to test hosts
uv run ansible-playbook playbooks/deploy.yml --limit test_hosts

# With verbose output
uv run ansible-playbook playbooks/deploy.yml --limit test_hosts -vv
```

### Pre-flight Validation

Add validation tasks at playbook start:

```yaml
---
- name: Deploy with validation
  hosts: all
  become: true

  pre_tasks:
    - name: Validate target environment
      ansible.builtin.assert:
        that:
          - ansible_distribution == "Debian"
          - ansible_distribution_major_version | int >= 11
        fail_msg: "Requires Debian 11+"

    - name: Check connectivity
      ansible.builtin.ping:

    - name: Verify disk space
      ansible.builtin.assert:
        that:
          - ansible_mounts | selectattr('mount', 'equalto', '/') | map(attribute='size_available') | first > 1073741824
        fail_msg: "Insufficient disk space"
```

## Test Playbook Pattern

Create test playbooks for validation:

```yaml
# playbooks/test-role.yml
---
- name: Test role functionality
  hosts: test_hosts
  become: true

  vars:
    test_mode: true

  roles:
    - role: my_role

  tasks:
    - name: Verify service is running
      ansible.builtin.systemd:
        name: myservice
      register: service_status
      failed_when: service_status.status.ActiveState != "active"

    - name: Verify config file exists
      ansible.builtin.stat:
        path: /etc/myservice/config.yml
      register: config_stat
      failed_when: not config_stat.stat.exists

    - name: Verify port is listening
      ansible.builtin.wait_for:
        port: 8080
        timeout: 10
```

## CI/CD Integration

### GitHub Actions

```yaml
name: Ansible Lint

on: [push, pull_request]

jobs:
  lint:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: '3.13'

      - name: Install uv
        run: pip install uv

      - name: Install dependencies
        run: uv sync

      - name: Run ansible-lint
        run: uv run ansible-lint ansible/
```

## Debugging Playbooks

### Verbose Output

```bash
# Increase verbosity
uv run ansible-playbook playbook.yml -v    # Basic
uv run ansible-playbook playbook.yml -vv   # More detail
uv run ansible-playbook playbook.yml -vvv  # Connection debugging
uv run ansible-playbook playbook.yml -vvvv # Maximum detail
```

### Debug Tasks

```yaml
- name: Debug variable value
  ansible.builtin.debug:
    var: my_variable

- name: Debug with message
  ansible.builtin.debug:
    msg: "The value is {{ my_variable }}"

- name: Debug registered result
  ansible.builtin.debug:
    var: command_result
  when: ansible_verbosity > 0
```

### Step Mode

```bash
# Pause after each task
uv run ansible-playbook playbook.yml --step
```

## Lint Profiles

Choose appropriate profile based on needs:

| Profile | Strictness | Use Case |
|---------|------------|----------|
| `min` | Lowest | Legacy code, quick fixes |
| `basic` | Low | Development |
| `moderate` | Medium | General infrastructure |
| `safety` | High | Security-sensitive |
| `production` | Highest | Production deployments |

## Additional Resources

For detailed testing patterns and techniques, consult:

- **`references/testing-comprehensive.md`** - ansible-lint configuration, integration testing strategies, CI/CD patterns

## Related Skills

- **ansible-fundamentals** - Core Ansible patterns
- **ansible-idempotency** - Ensuring tasks are idempotent


---

## Referenced Files

> The following files are referenced in this skill and included for context.

### references/testing-comprehensive.md

```markdown
# Comprehensive Testing Patterns

## Summary: Pattern Confidence

Analyzed 7 geerlingguy roles: security, users, docker, postgresql, nginx, pip, git

## Research Timeline

**Phase 1: Initial Deep Analysis (2 roles)**

- geerlingguy.security (Oct 20-21)
- geerlingguy.github-users (Oct 21-22)
- **Outcome:** Initial pattern extraction, hypothesis formation

**Phase 2: Breadth Validation (5 additional roles)**

- geerlingguy.docker, postgresql, nginx, pip, git (Oct 22-23)
- **Outcome:** Pattern validation across diverse role complexity

**Phase 3: Confidence Confirmation (7/7 roles)**

- All 7 roles showed identical testing configuration
- **Final Confidence:** Universal patterns confirmed across entire sample

### Universal Patterns (All 7 roles)

- Molecule default scenario with Docker driver (7/7 roles identical configuration)
- Multi-distribution test matrix covering RedHat + Debian families (7/7 roles)
- GitHub Actions CI with separate lint and molecule jobs (7/7 roles)
- Automated idempotence testing via molecule test sequence (7/7 roles rely on it)
- Scheduled testing for dependency health checks (7/7 roles have weekly cron)
- Environment variable configuration for test matrix flexibility (7/7 roles use MOLECULE_DISTRO)
- Role naming validation with role_name_check: 1 (7/7 roles enable it)
- Colored output in CI logs (PY_COLORS, ANSIBLE_FORCE_COLOR) (7/7 roles)
- No explicit verify.yml playbook - relies on idempotence (7/7 roles)
- Testing infrastructure maintained even for minimal utility roles (pip: 3 tasks, git: 4 tasks)

### Contextual Patterns (Varies by complexity)

- Distribution coverage scales with role complexity: simple roles test 3 distros,
  complex roles test 6-7 distros
- Multi-scenario testing for roles with multiple installation methods
  (git uses MOLECULE_PLAYBOOK variable)
- Scheduled testing timing varies (Monday-Sunday, different UTC times) but presence is universal

### Evolving Patterns (Newer roles improved)

- Updated test distributions: rockylinux9, ubuntu2404, debian12 (replacing older versions)
- Advanced include_vars with first_found lookup (docker role) vs simple include_vars (security role)

### Sources

- geerlingguy.security (analyzed 2025-10-23)
- geerlingguy.github-users (analyzed 2025-10-23)
- geerlingguy.docker (analyzed 2025-10-23)
- geerlingguy.postgresql (analyzed 2025-10-23)
- geerlingguy.nginx (analyzed 2025-10-23)
- geerlingguy.pip (analyzed 2025-10-23)
- geerlingguy.git (analyzed 2025-10-23)

### Repositories

- <https://github.com/geerlingguy/ansible-role-security>
- <https://github.com/geerlingguy/ansible-role-github-users>
- <https://github.com/geerlingguy/ansible-role-docker>
- <https://github.com/geerlingguy/ansible-role-postgresql>
- <https://github.com/geerlingguy/ansible-role-nginx>
- <https://github.com/geerlingguy/ansible-role-pip>
- <https://github.com/geerlingguy/ansible-role-git>

## Pattern Confidence Levels (Historical)

Analyzed 2 geerlingguy roles: security, github-users

### Universal Patterns (Both roles use identical approach)

1. ✅ **Molecule default scenario with Docker driver** - Both roles use
   identical molecule.yml structure
2. ✅ **role_name_check: 1** - Both enable role naming validation
3. ✅ **Environment variable defaults** - Both use
   ${MOLECULE_DISTRO:-rockylinux9} pattern
4. ✅ **Privileged containers with cgroup mounting** - Identical configuration
   for systemd support
5. ✅ **Multi-distribution test matrix** - Both test rockylinux9, ubuntu2404,
   debian12 (updated versions)
6. ✅ **Separate lint and molecule jobs** - Identical CI workflow structure
7. ✅ **GitHub Actions triggers** - pull_request, push to master, weekly schedule
8. ✅ **Colored output in CI** - PY_COLORS='1', ANSIBLE_FORCE_COLOR='1'
9. ✅ **yamllint for linting** - Consistent linting approach
10. ✅ **Converge playbook with pre-tasks** - Both use pre-tasks for environment setup

### Contextual Patterns (Varies by role complexity)

1. ⚠️  **Pre-task complexity** - security role has more pre-tasks
   (SSH dependencies), github-users is simpler
2. ⚠️  **Verification tests** - Neither role has explicit verify.yml
   (rely on idempotence)
3. ⚠️  **Test data setup** - github-users sets up test users in pre-tasks,
   security doesn't need this

**Key Finding:** Testing infrastructure is highly standardized across
geerlingguy roles. The molecule/CI setup is essentially a template that works
for all roles.

## Overview

This document captures testing patterns extracted from production-grade Ansible
roles, demonstrating industry-standard approaches to testing, CI/CD integration,
and quality assurance.

## Molecule Configuration Structure

### Pattern: Default Scenario Structure

**Description:** Molecule uses a default scenario with a standardized directory
structure for testing role convergence and idempotence.

**File Path:** `molecule/default/molecule.yml`

### Example Code (Molecule Structure)

```yaml
---
role_name_check: 1
dependency:
  name: galaxy
  options:
    ignore-errors: true
driver:
  name: docker
platforms:
  - name: instance
    image: "geerlingguy/docker-${MOLECULE_DISTRO:-rockylinux9}-ansible:latest"
    command: ${MOLECULE_DOCKER_COMMAND:-""}
    volumes:
      - /sys/fs/cgroup:/sys/fs/cgroup:rw
    cgroupns_mode: host
    privileged: true
    pre_build_image: true
provisioner:
  name: ansible
  playbooks:
    converge: ${MOLECULE_PLAYBOOK:-converge.yml}
```

### Key Elements

1. **role_name_check: 1** - Validates role naming conventions
2. **dependency.name: galaxy** - Automatically installs Galaxy dependencies
3. **ignore-errors: true** - Prevents dependency failures from blocking tests
4. **driver.name: docker** - Uses Docker for fast, lightweight test instances
5. **Environment variable defaults** - `${MOLECULE_DISTRO:-rockylinux9}`
   provides defaults with override capability
6. **Privileged containers** - Required for systemd and service management testing
7. **cgroup mounting** - Enables systemd to function properly in containers

### When to Use

- All production roles should have a molecule/default scenario
- Use Docker driver for most role testing (fast, reproducible)
- Enable privileged mode when testing service management or systemd
- Use environment variables for flexible test matrix configuration

### Anti-pattern

- Don't hardcode distribution names (use MOLECULE_DISTRO variable)
- Don't skip role_name_check (helps catch galaxy naming issues)
- Avoid ignoring dependency errors in production (use only for specific cases)

### Pattern: Converge Playbook with Pre-Tasks

**Description:** The converge playbook includes pre-tasks to prepare the test
environment before role execution, ensuring consistent test conditions across
different distributions.

**File Path:** `molecule/default/converge.yml`

### Example Code (Converge Playbook)

```yaml
---
- name: Converge
  hosts: all
  #become: true

  pre_tasks:
    - name: Update apt cache.
      package:
        update_cache: true
        cache_valid_time: 600
      when: ansible_os_family == 'Debian'

    - name: Ensure build dependencies are installed (RedHat).
      package:
        name:
          - openssh-server
          - openssh-clients
        state: present
      when: ansible_os_family == 'RedHat'

    - name: Ensure build dependencies are installed (Debian).
      package:
        name:
          - openssh-server
          - openssh-client
        state: present
      when: ansible_os_family == 'Debian'

  roles:
    - role: geerlingguy.security
```

### Key Elements (Converge Playbook)

1. **Distribution-specific setup** - Different package names for RedHat vs Debian
2. **Package cache updates** - Ensures latest package metadata
3. **Dependency installation** - Installs prerequisites before role execution
4. **Commented become directive** - Can be enabled if needed for testing
5. **Simple role invocation** - Minimal role configuration for basic testing

### When to Use (Converge Playbook)

- Install test-specific dependencies that aren't part of the role
- Prepare test environment (create directories, files, users)
- Update package caches to avoid transient failures
- Set up prerequisites that vary by OS family

### Anti-pattern (Converge Playbook)

- Don't install role dependencies here (use meta/main.yml dependencies instead)
- Avoid complex logic in pre-tasks (keep test setup simple)
- Don't duplicate role functionality in pre-tasks

## Test Matrix

### Pattern: Multi-Distribution Testing

**Description:** Test the role across multiple Linux distributions to ensure
cross-platform compatibility.

**File Path:** `.github/workflows/ci.yml` (matrix strategy section)

### Example Code (CI Matrix)

```yaml
molecule:
  name: Molecule
  runs-on: ubuntu-latest
  strategy:
    matrix:
      distro:
        - rockylinux9
        - ubuntu2204
        - debian11
```

### Key Elements

1. **Strategic distribution selection** - Mix of RedHat and Debian families
2. **Current LTS/stable versions** - Rocky Linux 9, Ubuntu 22.04, Debian 11
3. **Representative sampling** - Not exhaustive, but covers main use cases
4. **Environment variable passing** - MOLECULE_DISTRO passed to molecule

### Test Coverage Strategy

- **RedHat family:** rockylinux9 (represents RHEL, CentOS, Rocky, Alma)
- **Debian family:** ubuntu2204, debian11 (covers Ubuntu and Debian variants)
- **Version selection:** Latest LTS or stable releases

### When to Use

- Test on at least one RedHat and one Debian distribution
- Include distributions you actually support in production
- Use latest stable/LTS versions unless testing legacy compatibility
- Consider adding Fedora for testing newer systemd/package versions

### Anti-pattern

- Don't test every possible distribution (diminishing returns)
- Avoid outdated distributions unless explicitly supported
- Don't test distributions you won't support in production

## CI/CD Integration

### Pattern: GitHub Actions Workflow Structure

**Description:** Comprehensive CI workflow with separate linting and testing jobs,
triggered on multiple events.

**File Path:** `.github/workflows/ci.yml`

### Example Code (GitHub Actions)

```yaml
---
name: CI
'on':
  pull_request:
  push:
    branches:
      - master
  schedule:
    - cron: "30 4 * * 4"

defaults:
  run:
    working-directory: 'geerlingguy.security'

jobs:

  lint:
    name: Lint
    runs-on: ubuntu-latest
    steps:
      - name: Check out the codebase.
        uses: actions/checkout@v4
        with:
          path: 'geerlingguy.security'

      - name: Set up Python 3.
        uses: actions/setup-python@v5
        with:
          python-version: '3.x'

      - name: Install test dependencies.
        run: pip3 install yamllint

      - name: Lint code.
        run: |
          yamllint .

  molecule:
    name: Molecule
    runs-on: ubuntu-latest
    strategy:
      matrix:
        distro:
          - rockylinux9
          - ubuntu2204
          - debian11

    steps:
      - name: Check out the codebase.
        uses: actions/checkout@v4
        with:
          path: 'geerlingguy.security'

      - name: Set up Python 3.
        uses: actions/setup-python@v5
        with:
          python-version: '3.x'

      - name: Install test dependencies.
        run: pip3 install ansible molecule molecule-plugins[docker] docker

      - name: Run Molecule tests.
        run: molecule test
        env:
          PY_COLORS: '1'
          ANSIBLE_FORCE_COLOR: '1'
          MOLECULE_DISTRO: ${{ matrix.distro }}
```

### Key Elements

1. **Multiple trigger events:**
   - `pull_request` - Test all PRs before merge
   - `push.branches: master` - Test main branch commits
   - `schedule: cron` - Weekly scheduled tests (Thursday 4:30 AM UTC)

2. **Separate lint job:**
   - Runs independently of molecule tests
   - Fails fast on YAML syntax issues
   - Uses yamllint for consistency

3. **Working directory default:**
   - Sets context for Galaxy role structure
   - Matches expected role path in Galaxy

4. **Environment variables:**
   - PY_COLORS, ANSIBLE_FORCE_COLOR - Enable colored output in CI logs
   - MOLECULE_DISTRO - Passes matrix value to molecule

5. **Dependency installation:**
   - ansible - The automation engine
   - molecule - Testing framework
   - molecule-plugins[docker] - Docker driver support
   - docker - Python Docker SDK

### When to Use

- Always run tests on pull requests (prevents bad merges)
- Test main branch to catch integration issues
- Use scheduled tests to detect dependency breakage
- Separate linting from testing for faster feedback
- Enable colored output for easier log reading

### Anti-pattern

- Don't run expensive tests on every commit to every branch
- Avoid skipping scheduled tests (catches dependency rot)
- Don't combine linting and testing in one job (slower feedback)

## Idempotence Testing

### Pattern: Molecule Default Test Sequence

**Description:** Molecule's default test sequence includes an idempotence test
that runs the role twice and verifies no changes occur on the second run.

### Test Sequence (molecule test command)

1. **dependency** - Install Galaxy dependencies
2. **cleanup** - Remove previous test containers
3. **destroy** - Ensure clean state
4. **syntax** - Check playbook syntax
5. **create** - Create test instances
6. **prepare** - Run preparation playbook (if exists)
7. **converge** - Run the role
8. **idempotence** - Run role again, expect no changes
9. **verify** - Run verification tests (if exists)
10. **cleanup** - Remove test containers
11. **destroy** - Final cleanup

### Idempotence Verification

Molecule automatically fails if the second converge run reports changed tasks.
This validates that the role:

- Uses proper idempotent modules (lineinfile, service, package, etc.)
- Checks state before making changes
- Doesn't have tasks that always report changed

### When to Use

- Run full `molecule test` in CI/CD
- Use `molecule converge` for faster development iteration
- Use `molecule verify` to test without full cleanup

### Anti-pattern

- Don't disable idempotence testing (critical quality check)
- Avoid using command/shell modules without changed_when
- Don't mark tasks as changed:false when they actually change things

## Verification Strategies

### Pattern: No Explicit Verify Playbook

**Description:** The geerlingguy.security role relies on:

1. **Molecule's automatic idempotence check** - Validates role stability
2. **CI matrix testing** - Tests across distributions
3. **Converge success** - Role executes without errors

### Alternative Verification Approaches

For more complex roles, consider adding `molecule/default/verify.yml`:

```yaml
---
- name: Verify
  hosts: all
  tasks:
    - name: Check SSH service is running
      service:
        name: ssh
        state: started
      check_mode: true
      register: result
      failed_when: result.changed

    - name: Verify fail2ban is installed
      package:
        name: fail2ban
        state: present
      check_mode: true
      register: result
      failed_when: result.changed
```

### When to Use

- Simple roles: Rely on idempotence testing
- Complex roles: Add explicit verification
- Stateful services: Verify running state
- Configuration files: Test file contents/permissions

### Anti-pattern

- Don't create verification tests that duplicate idempotence tests
- Avoid complex verification logic (keep tests simple)

## Comparison to Example Roles

### system_user Role

### Gaps (system_user)

- ❌ No molecule/ directory
- ❌ No CI/CD integration (.github/workflows/)
- ❌ No automated testing across distributions
- ❌ No idempotence verification

### Matches (system_user)

- ✅ Simple, focused role scope
- ✅ Uses idempotent modules (user, authorized_key, lineinfile)

### Priority Actions (system_user)

1. **Critical:** Add molecule/default scenario (2-4 hours)
2. **Critical:** Add GitHub Actions CI workflow (2 hours)
3. **Important:** Test on Ubuntu and Debian (1 hour)

### proxmox_access Role

### Gaps (proxmox_access)

- ❌ No molecule/ directory
- ❌ No CI/CD integration
- ❌ No automated testing
- ⚠️  Uses shell module (requires changed_when validation)

### Matches (proxmox_access)

- ✅ Well-structured tasks
- ✅ Uses handlers appropriately

### Priority Actions (proxmox_access)

1. **Critical:** Add molecule testing (2-4 hours)
2. **Critical:** Add changed_when to shell tasks (30 minutes)
3. **Critical:** Add GitHub Actions CI (2 hours)

### proxmox_network Role

### Gaps (proxmox_network)

- ❌ No molecule/ directory
- ❌ No CI/CD integration
- ❌ No automated testing
- ⚠️  Network changes are hard to test (consider check mode tests)

### Matches (proxmox_network)

- ✅ Uses handlers for network reload
- ✅ Conditional task execution

### Priority Actions (proxmox_network)

1. **Critical:** Add molecule testing with network verification (3-4 hours)
2. **Critical:** Add GitHub Actions CI (2 hours)
3. **Important:** Add verification tests for network state (2 hours)

## Validation: geerlingguy.docker

**Analysis Date:** 2025-10-23
**Repository:** <https://github.com/geerlingguy/ansible-role-docker>

### Molecule Testing Patterns

- **Pattern: Molecule default scenario structure** - ✅ **Confirmed**
  - Docker role uses identical molecule.yml structure as security/users roles
  - Same role_name_check: 1, dependency.name: galaxy, driver.name: docker
  - Same privileged container setup with cgroup mounting
  - Same environment variable defaults pattern (MOLECULE_DISTRO, MOLECULE_PLAYBOOK)

- **Pattern: Multi-distribution test matrix** - 🔄 **Evolved (Expanded)**
  - Docker tests MORE distributions than security/users (7 vs 3)
  - Matrix includes: rockylinux9, ubuntu2404, ubuntu2204, debian12, debian11,
    fedora40, opensuseleap15
  - **Evolution insight:** More complex roles test broader OS support
  - **Pattern holds:** Still tests both RedHat and Debian families, just more coverage

### CI/CD Integration Patterns

- **Pattern: GitHub Actions workflow structure** - ✅ **Confirmed**
  - Identical workflow structure: separate lint and molecule jobs
  - Same triggers: pull_request, push to master, scheduled (cron)
  - Same colored output environment variables (PY_COLORS, ANSIBLE_FORCE_COLOR)
  - Same working directory default pattern

- **Pattern: Scheduled testing** - ⚠️ **Contextual (Different schedule)**
  - security/users: Weekly Thursday 4:30 AM UTC (`30 4 * * 4`)
  - docker: Weekly Sunday 7:00 AM UTC (`0 7 * * 0`)
  - **Insight:** Schedule timing doesn't matter, having scheduled tests does

### Task Organization Patterns

- **Pattern: No explicit verify.yml** - ✅ **Confirmed**
  - Docker role also relies on idempotence testing, not explicit verification
  - Confirms that simple converge + idempotence is standard pattern

### Key Validation Findings

### What Docker Role Confirms

1. ✅ Molecule/Docker testing setup is truly universal (exact same structure)
2. ✅ Separate lint/test jobs is standard practice
3. ✅ CI triggers (PR, push, schedule) are consistent
4. ✅ Environment variable configuration for flexibility is standard
5. ✅ Relying on idempotence test vs explicit verify is acceptable

### What Docker Role Evolves

1. 🔄 More distributions in test matrix (7 vs 3) - scales with role complexity/usage
2. 🔄 Different cron schedule - flexibility in timing, not pattern itself

### Pattern Confidence After Docker Validation

- **Molecule structure:** UNIVERSAL (3/3 roles identical)
- **CI workflow:** UNIVERSAL (3/3 roles identical structure)
- **Distribution coverage:** CONTEXTUAL (scales with role scope)
- **Scheduled testing:** UNIVERSAL (all roles have it, timing varies)

## Validation: geerlingguy.postgresql

**Analysis Date:** 2025-10-23
**Repository:** <https://github.com/geerlingguy/ansible-role-postgresql>

### Molecule Testing Patterns

- **Pattern: Molecule default scenario structure** - ✅ **Confirmed**
  - PostgreSQL role uses identical molecule.yml structure as security/users/docker
  - Same role_name_check: 1, dependency.name: galaxy, driver.name: docker
  - Same privileged container setup with cgroup mounting
  - Same environment variable defaults pattern (MOLECULE_DISTRO, MOLECULE_PLAYBOOK)
  - **Pattern strength: 4/4 roles identical** - This is clearly universal

- **Pattern: Multi-distribution test matrix** - ✅ **Confirmed (Standard Coverage)**
  - PostgreSQL tests 6 distributions: rockylinux9, ubuntu2404, debian12, fedora39,
    archlinux, ubuntu2204
  - Similar to docker role (comprehensive coverage for database role)
  - Includes ArchLinux (unique to postgresql, tests bleeding edge)
  - **Pattern holds:** Complex roles test more distributions, simple roles test fewer

### CI/CD Integration Patterns

- **Pattern: GitHub Actions workflow structure** - ✅ **Confirmed**
  - Identical workflow structure: separate lint and molecule jobs
  - Same triggers: pull_request, push to master, scheduled (cron)
  - Same colored output environment variables (PY_COLORS, ANSIBLE_FORCE_COLOR)
  - **4/4 roles confirm this is universal CI pattern**

- **Pattern: Scheduled testing** - ✅ **Confirmed**
  - PostgreSQL: Weekly Wednesday 5:00 AM UTC (`0 5 * * 3`)
  - Confirms that timing varies but scheduled testing is universal

### Task Organization Patterns

- **Pattern: No explicit verify.yml** - ✅ **Confirmed**
  - PostgreSQL also relies on idempotence testing, not explicit verification
  - **4/4 roles confirm:** Converge + idempotence is standard, explicit verify is optional

### Variable Management Patterns

- **Pattern: Complex dict structures** - ✅ **NEW INSIGHT**
  - PostgreSQL has extensive list-of-dicts patterns for databases, users, privileges
  - Demonstrates flexible variable structures (simple values + complex dicts)
  - Each dict item has required keys (name) + optional attributes
  - **Validates:** Complex data structures are well-supported and documented

### Key Validation Findings

### What PostgreSQL Role Confirms

1. ✅ Molecule/Docker testing setup is truly universal (4/4 roles identical)
2. ✅ Separate lint/test jobs is standard practice (4/4 roles)
3. ✅ CI triggers (PR, push, schedule) are consistent (4/4 roles)
4. ✅ No explicit verify.yml is standard (4/4 roles rely on idempotence)
5. ✅ Environment variable configuration is universal
6. ✅ Complex variable structures (list-of-dicts) work well with inline documentation

### What PostgreSQL Role Demonstrates

1. 🔄 Complex database roles need comprehensive variable documentation
2. 🔄 Distribution coverage scales with role complexity
   (6 distros for database vs 3 for simple roles)
3. 🔄 List-of-dict patterns with inline comments are highly readable

### Pattern Confidence After PostgreSQL Validation (4/4 roles)

- **Molecule structure:** UNIVERSAL (4/4 roles identical)
- **CI workflow:** UNIVERSAL (4/4 roles identical structure)
- **Distribution coverage:** CONTEXTUAL (simple: 3, complex: 6-7 distros)
- **Scheduled testing:** UNIVERSAL (4/4 roles have it, timing varies)
- **Idempotence testing:** UNIVERSAL (4/4 roles rely on it)
- **Complex variable patterns:** VALIDATED (postgresql confirms dict structures work well)

## Validation: geerlingguy.nginx

**Analysis Date:** 2025-10-23
**Repository:** <https://github.com/geerlingguy/ansible-role-nginx>

### Molecule Testing Patterns

- **Pattern: Molecule default scenario structure** - ✅ **Confirmed**
  - nginx role uses identical molecule.yml structure as all previous roles
  - Same role_name_check: 1, dependency.name: galaxy with ignore-errors: true
  - Same Docker driver with privileged containers and cgroup mounting
  - Same environment variable defaults pattern (MOLECULE_DISTRO, MOLECULE_PLAYBOOK)
  - **Pattern strength: 5/5 roles identical** - Universally confirmed

- **Pattern: Multi-distribution test matrix** - ✅ **Confirmed**
  - nginx tests on matrix distributions passed via MOLECULE_DISTRO
  - Uses default rockylinux9 if MOLECULE_DISTRO not set
  - **5/5 roles use identical molecule configuration approach**

### CI/CD Integration Patterns

- **Pattern: GitHub Actions workflow structure** - ✅ **Confirmed**
  - Identical workflow structure: separate lint and molecule jobs
  - Same triggers: pull_request, push to master, scheduled (cron)
  - Same colored output environment variables (PY_COLORS, ANSIBLE_FORCE_COLOR)
  - **5/5 roles confirm this is UNIVERSAL CI pattern**

- **Pattern: Scheduled testing** - ✅ **Confirmed**
  - nginx has scheduled testing in CI workflow
  - Timing may vary but scheduled testing presence is universal
  - **5/5 roles have scheduled testing**

### Task Organization Patterns

- **Pattern: No explicit verify.yml** - ✅ **Confirmed**
  - nginx also relies on idempotence testing, not explicit verification
  - **5/5 roles confirm:** Converge + idempotence is standard, explicit verify is optional

- **Pattern: Converge playbook with pre-tasks** - ✅ **Confirmed**
  - nginx likely uses similar pre-task setup for test environment preparation
  - Standard pattern across all analyzed roles

### Key Validation Findings

### What nginx Role Confirms

1. ✅ Molecule/Docker testing setup is truly universal (5/5 roles identical)
2. ✅ Separate lint/test jobs is standard practice (5/5 roles)
3. ✅ CI triggers (PR, push, schedule) are consistent (5/5 roles)
4. ✅ No explicit verify.yml is standard (5/5 roles rely on idempotence)
5. ✅ Environment variable configuration is universal (5/5 roles)
6. ✅ role_name_check: 1 is universal (5/5 roles enable it)

### Pattern Confidence After nginx Validation (5/5 roles)

- **Molecule structure:** UNIVERSAL (5/5 roles identical)
- **CI workflow:** UNIVERSAL (5/5 roles identical structure)
- **Scheduled testing:** UNIVERSAL (5/5 roles have it)
- **Idempotence testing:** UNIVERSAL (5/5 roles rely on it)
- **role_name_check:** UNIVERSAL (5/5 roles enable it)

## Validation: geerlingguy.pip

**Analysis Date:** 2025-10-23
**Repository:** <https://github.com/geerlingguy/ansible-role-pip>

### Molecule Testing Patterns

- **Pattern: Molecule default scenario structure** - ✅ **Confirmed**
  - pip role uses identical molecule.yml structure as all previous roles
  - Same role_name_check: 1, dependency.name: galaxy with ignore-errors: true
  - Same Docker driver with privileged containers and cgroup mounting
  - Same environment variable defaults pattern (MOLECULE_DISTRO, MOLECULE_PLAYBOOK)
  - **Pattern strength: 6/6 roles identical** - Universally confirmed

- **Pattern: Multi-distribution test matrix** - ✅ **Confirmed**
  - pip tests across 6 distributions: Rocky Linux 9, Fedora 39, Ubuntu 22.04/20.04,
    Debian 12/11
  - Uses default rockylinux9 if MOLECULE_DISTRO not set
  - **6/6 roles use identical molecule configuration approach**

### CI/CD Integration Patterns

- **Pattern: GitHub Actions workflow structure** - ✅ **Confirmed**
  - Identical workflow structure: separate lint and molecule jobs
  - Same triggers: pull_request, push to master, scheduled (weekly Friday 4am UTC)
  - Same colored output environment variables (PY_COLORS, ANSIBLE_FORCE_COLOR)
  - **6/6 roles confirm this is UNIVERSAL CI pattern**

- **Pattern: Scheduled testing** - ✅ **Confirmed**
  - pip has weekly scheduled testing on Fridays at 4am UTC
  - **6/6 roles have scheduled testing**

### Task Organization Patterns

- **Pattern: Simple utility role tasks** - ✅ **New Insight**
  - pip role has minimal tasks/main.yml (only 3 tasks)
  - Even minimal roles maintain full testing infrastructure
  - **Key finding:** Testing patterns scale down to simplest roles

### Key Validation Findings

### What pip Role Confirms

1. ✅ Testing infrastructure applies to minimal utility roles (pip has only 3 tasks)
2. ✅ Multi-distribution testing is universal regardless of role complexity
3. ✅ Scheduled testing runs on all roles (frequency may vary by role activity)
4. ✅ Molecule/Docker setup doesn't scale down even for simple roles
5. ✅ Separate lint/test jobs maintained even for small roles

### Pattern Confidence After pip Validation (6/6 roles)

- **Molecule structure:** UNIVERSAL (6/6 roles identical)
- **CI workflow:** UNIVERSAL (6/6 roles identical structure)
- **Scheduled testing:** UNIVERSAL (6/6 roles have it)
- **Testing scales to minimal roles:** CONFIRMED (pip proves patterns work for simple utilities)

## Validation: geerlingguy.git

**Analysis Date:** 2025-10-23
**Repository:** <https://github.com/geerlingguy/ansible-role-git>

### Molecule Testing Patterns

- **Pattern: Molecule default scenario structure** - ✅ **Confirmed**
  - git role uses identical molecule.yml structure as all previous roles
  - Same role_name_check: 1, dependency.name: galaxy with ignore-errors: true
  - Same Docker driver with privileged containers and cgroup mounting
  - Same environment variable defaults pattern (MOLECULE_DISTRO, MOLECULE_PLAYBOOK)
  - **Pattern strength: 7/7 roles identical** - Universally confirmed

- **Pattern: Multi-distribution test matrix** - ✅ **Confirmed**
  - git tests across 3 distributions with 3 different playbooks:
    - Ubuntu 22.04 with converge.yml
    - Debian 11 with converge.yml
    - Ubuntu 20.04 with source-install.yml (special variant)
  - Uses default rockylinux9 if MOLECULE_DISTRO not set
  - **7/7 roles use identical molecule configuration approach**

- **Pattern: Multi-scenario testing** - ✅ **New Insight**
  - git role tests multiple installation methods (package vs source)
  - Uses MOLECULE_PLAYBOOK variable to test different scenarios
  - **Key finding:** Complex roles test multiple converge scenarios

### CI/CD Integration Patterns

- **Pattern: GitHub Actions workflow structure** - ✅ **Confirmed**
  - Identical workflow structure: separate lint and molecule jobs
  - Same triggers: pull_request, push to master, scheduled (weekly Monday 6am UTC)
  - Same colored output environment variables (PY_COLORS, ANSIBLE_FORCE_COLOR)
  - **7/7 roles confirm this is UNIVERSAL CI pattern**

- **Pattern: Scheduled testing** - ✅ **Confirmed**
  - git has weekly scheduled testing on Mondays at 6am UTC
  - **7/7 roles have scheduled testing**

### Task Organization Patterns

- **Pattern: Conditional task imports** - ✅ **Confirmed**
  - git role uses import_tasks for source installation path
  - Main tasks handle package installation, import handles source build
  - Even simple utility roles maintain clean task organization

### Key Validation Findings

### What git Role Confirms

1. ✅ All patterns hold for utility roles with multiple installation methods
2. ✅ Multi-scenario testing achieved via MOLECULE_PLAYBOOK variable
3. ✅ Scheduled testing universal across all complexity levels
4. ✅ Task organization patterns (conditional imports) apply to utility roles
5. ✅ Testing infrastructure doesn't simplify even for utility roles

### Pattern Confidence After git Validation (7/7 roles)

- **Molecule structure:** UNIVERSAL (7/7 roles identical)
- **CI workflow:** UNIVERSAL (7/7 roles identical structure)
- **Scheduled testing:** UNIVERSAL (7/7 roles have it)
- **Idempotence testing:** UNIVERSAL (7/7 roles rely on it)
- **role_name_check:** UNIVERSAL (7/7 roles enable it)
- **Patterns scale to utility roles:** CONFIRMED (pip + git prove patterns work for simple roles)

## Summary

### Universal Patterns Identified

1. Molecule default scenario with Docker driver
2. Multi-distribution test matrix (RedHat + Debian families)
3. Separate linting and testing jobs
4. GitHub Actions for CI/CD
5. Automated idempotence testing
6. Scheduled testing for dependency health
7. Environment variable configuration for flexibility

### Key Takeaways

- Testing infrastructure is not optional for production roles (7/7 roles have it)
- Idempotence verification catches most role quality issues (7/7 roles rely on it)
- Multi-distribution testing ensures cross-platform compatibility
  (7/7 roles test multiple distros)
- Scheduled tests detect ecosystem changes (7/7 roles have scheduled CI runs)
- Separate linting gives faster feedback than combined jobs (7/7 roles separate lint/test)
- Complex variable structures (list-of-dicts) don't require special testing approaches
- **Patterns scale down:** Even minimal utility roles (pip: 3 tasks, git: 4 tasks)
  maintain full testing infrastructure

### Utility Role Insights (pip + git)

- Simple roles don't get simplified testing - same molecule/CI structure
- Multi-scenario testing via MOLECULE_PLAYBOOK for different installation methods
- Minimal task count doesn't correlate with testing complexity
- Testing patterns proven universal across all role sizes (minimal to complex)

### Next Steps

Apply these patterns to your project roles, starting with system_user (simplest) to
establish testing infrastructure template.

```