typo3-testing
Agent Skill: TYPO3 extension testing (unit, functional, E2E, architecture, mutation). Use when setting up test infrastructure, writing tests, configuring PHPUnit, or CI/CD. By Netresearch.
Packaged view
This page reorganizes the original catalog entry around fit, installability, and workflow context first. The original raw source lives below.
Install command
npx @skill-hub/cli install netresearch-claude-code-marketplace-typo3-testing
Repository
Skill path: skills/typo3-testing/skills/typo3-testing
Agent Skill: TYPO3 extension testing (unit, functional, E2E, architecture, mutation). Use when setting up test infrastructure, writing tests, configuring PHPUnit, or CI/CD. By Netresearch.
Open repositoryBest for
Primary workflow: Write Technical Docs.
Technical facets: Full Stack, DevOps, Tech Writer, Testing.
Target audience: everyone.
License: Unknown.
Original source
Catalog source: SkillHub Club.
Repository owner: netresearch.
This is still a mirrored public skill entry. Review the repository before installing into production workflows.
What it helps with
- Install typo3-testing into Claude Code, Codex CLI, Gemini CLI, or OpenCode workflows
- Review https://github.com/netresearch/claude-code-marketplace before adding typo3-testing to shared team environments
- Use typo3-testing for development workflows
Works across
Favorites: 0.
Sub-skills: 0.
Aggregator: No.
Original source / Raw SKILL.md
---
name: typo3-testing
description: "Agent Skill: TYPO3 extension testing (unit, functional, E2E, architecture, mutation). Use when setting up test infrastructure, writing tests, configuring PHPUnit, or CI/CD. By Netresearch."
---
# TYPO3 Testing Skill
Templates, scripts, and references for comprehensive TYPO3 extension testing.
## Test Type Selection
| Type | Use When | Speed |
|------|----------|-------|
| **Unit** | Pure logic, no DB, validators, utilities | Fast (ms) |
| **Functional** | DB interactions, repositories, controllers | Medium (s) |
| **Architecture** | Layer constraints, dependency rules (phpat) | Fast (ms) |
| **E2E (Playwright)** | User workflows, browser, accessibility | Slow (s-min) |
| **Fuzz** | Security, parsers, malformed input | Manual |
| **Crypto** | Encryption, secrets, key management | Fast (ms) |
| **Mutation** | Test quality verification, 70%+ coverage | CI/Release |
## Commands
```bash
# Setup infrastructure
scripts/setup-testing.sh [--with-e2e]
# Run tests via runTests.sh
Build/Scripts/runTests.sh -s unit
Build/Scripts/runTests.sh -s functional
Build/Scripts/runTests.sh -s architecture
Build/Scripts/runTests.sh -s e2e
# Quality tools
Build/Scripts/runTests.sh -s lint
Build/Scripts/runTests.sh -s phpstan
Build/Scripts/runTests.sh -s mutation
```
## Scoring
| Criterion | Requirement |
|-----------|-------------|
| Unit tests | Required, 70%+ coverage |
| Functional tests | Required for DB operations |
| Architecture tests | **phpat required** for full points |
| PHPStan | Level 10 (max) |
> **Note:** Full conformance requires phpat architecture tests enforcing layer boundaries.
## References
- `references/unit-testing.md` - UnitTestCase, mocking, assertions
- `references/functional-testing.md` - FunctionalTestCase, fixtures, database
- `references/functional-test-patterns.md` - Container reset, PHPUnit 10+ migration
- `references/typo3-v14-final-classes.md` - Testing final/readonly classes, interface extraction
- `references/architecture-testing.md` - phpat rules, layer constraints
- `references/e2e-testing.md` - Playwright, Page Object Model
- `references/accessibility-testing.md` - axe-core, WCAG compliance
- `references/fuzz-testing.md` - nikic/php-fuzzer, security
- `references/crypto-testing.md` - Encryption, secrets, sodium
- `references/mutation-testing.md` - Infection, test quality
- `references/ci-cd.md` - GitHub Actions, GitLab CI
## Templates
- `templates/UnitTests.xml`, `templates/FunctionalTests.xml` - PHPUnit configs
- `templates/phpat.php` - Architecture test rules
- `templates/Build/playwright/` - Playwright setup
- `templates/runTests.sh` - Test orchestration
- `templates/github-actions-tests.yml` - CI workflow
## External Resources
- [TYPO3 Testing Docs](https://docs.typo3.org/m/typo3/reference-coreapi/main/en-us/Testing/)
- [Tea Extension](https://github.com/TYPO3BestPractices/tea) - Reference implementation
- [phpat](https://github.com/carlosas/phpat) - PHP Architecture Tester
---
> **Contributing:** https://github.com/netresearch/typo3-testing-skill
---
## Referenced Files
> The following files are referenced in this skill and included for context.
### references/unit-testing.md
```markdown
# Unit Testing in TYPO3
Unit tests are fast, isolated tests that verify individual components without external dependencies like databases or file systems.
## When to Use Unit Tests
✅ **Ideal for:**
- Testing pure business logic
- Validators, calculators, transformers
- Value objects and DTOs
- Utilities and helper functions
- Domain models without persistence
- **Controllers with dependency injection** (new in TYPO3 13)
- **Services with injected dependencies**
❌ **Not suitable for:**
- Database operations (use functional tests)
- File system operations
- Methods using `BackendUtility` or global state
- Complex TYPO3 framework integration
- Parent class behavior from framework classes
## Base Class
All unit tests extend `TYPO3\TestingFramework\Core\Unit\UnitTestCase`:
```php
<?php
declare(strict_types=1);
namespace Vendor\Extension\Tests\Unit\Domain\Validator;
use PHPUnit\Framework\Attributes\Test;
use TYPO3\TestingFramework\Core\Unit\UnitTestCase;
use Vendor\Extension\Domain\Validator\EmailValidator;
/**
* Unit tests for EmailValidator.
*
* @covers \Vendor\Extension\Domain\Validator\EmailValidator
*/
final class EmailValidatorTest extends UnitTestCase
{
private EmailValidator $subject;
protected function setUp(): void
{
parent::setUp();
$this->subject = new EmailValidator();
}
#[Test]
public function validEmailPassesValidation(): void
{
$result = $this->subject->validate('[email protected]');
self::assertFalse($result->hasErrors());
}
#[Test]
public function invalidEmailFailsValidation(): void
{
$result = $this->subject->validate('invalid-email');
self::assertTrue($result->hasErrors());
}
}
```
> **Note:** TYPO3 13+ with PHPUnit 11/12 uses PHP attributes (`#[Test]`) instead of `@test` annotations.
> Use `private` instead of `protected` for properties when possible (better encapsulation).
## Key Principles
### 1. No External Dependencies
Unit tests should NOT:
- Access the database
- Read/write files
- Make HTTP requests
- Use TYPO3 framework services
### 2. Fast Execution
Unit tests should run in milliseconds:
- No I/O operations
- Minimal object instantiation
- Use mocks for dependencies
### 3. Test Independence
Each test should:
- Be runnable standalone
- Not depend on execution order
- Clean up in tearDown()
## Test Structure
### Arrange-Act-Assert Pattern
```php
/**
* @test
*/
public function calculatesTotalPrice(): void
{
// Arrange: Set up test data
$cart = new ShoppingCart();
$cart->addItem(new Item('product1', 10.00, 2));
$cart->addItem(new Item('product2', 5.50, 1));
// Act: Execute the code under test
$total = $cart->calculateTotal();
// Assert: Verify the result
self::assertSame(25.50, $total);
}
```
### setUp() and tearDown()
```php
protected function setUp(): void
{
parent::setUp();
// Initialize test subject and dependencies
$this->subject = new Calculator();
}
protected function tearDown(): void
{
// Clean up resources
unset($this->subject);
parent::tearDown();
}
```
## Testing with Dependency Injection (TYPO3 13+)
Modern TYPO3 13 controllers and services use constructor injection. Here's how to test them:
### Basic Constructor Injection Test
```php
<?php
declare(strict_types=1);
namespace Vendor\Extension\Tests\Unit\Controller;
use PHPUnit\Framework\Attributes\Test;
use PHPUnit\Framework\MockObject\MockObject;
use TYPO3\CMS\Core\Resource\ResourceFactory;
use TYPO3\TestingFramework\Core\Unit\UnitTestCase;
use Vendor\Extension\Controller\ImageController;
final class ImageControllerTest extends UnitTestCase
{
private ImageController $subject;
/** @var ResourceFactory&MockObject */
private ResourceFactory $resourceFactoryMock;
protected function setUp(): void
{
parent::setUp();
/** @var ResourceFactory&MockObject $resourceFactoryMock */
$resourceFactoryMock = $this->createMock(ResourceFactory::class);
$this->resourceFactoryMock = $resourceFactoryMock;
$this->subject = new ImageController($this->resourceFactoryMock);
}
#[Test]
public function getFileRetrievesFileFromFactory(): void
{
$fileId = 123;
$fileMock = $this->createMock(\TYPO3\CMS\Core\Resource\File::class);
$this->resourceFactoryMock
->expects(self::once())
->method('getFileObject')
->with($fileId)
->willReturn($fileMock);
$result = $this->subject->getFile($fileId);
self::assertSame($fileMock, $result);
}
}
```
### Multiple Dependencies with Intersection Types
PHPUnit mocks require proper type hints using intersection types for PHPStan compliance:
```php
<?php
declare(strict_types=1);
namespace Vendor\Extension\Tests\Unit\Controller;
use PHPUnit\Framework\Attributes\Test;
use PHPUnit\Framework\MockObject\MockObject;
use TYPO3\CMS\Core\Log\LogManager;
use TYPO3\CMS\Core\Resource\ResourceFactory;
use TYPO3\TestingFramework\Core\Unit\UnitTestCase;
use Vendor\Extension\Controller\ImageController;
use Vendor\Extension\Utils\ImageProcessor;
final class ImageControllerTest extends UnitTestCase
{
private ImageController $subject;
/** @var ResourceFactory&MockObject */
private ResourceFactory $resourceFactoryMock;
/** @var ImageProcessor&MockObject */
private ImageProcessor $imageProcessorMock;
/** @var LogManager&MockObject */
private LogManager $logManagerMock;
protected function setUp(): void
{
parent::setUp();
/** @var ResourceFactory&MockObject $resourceFactoryMock */
$resourceFactoryMock = $this->createMock(ResourceFactory::class);
/** @var ImageProcessor&MockObject $imageProcessorMock */
$imageProcessorMock = $this->createMock(ImageProcessor::class);
/** @var LogManager&MockObject $logManagerMock */
$logManagerMock = $this->createMock(LogManager::class);
$this->resourceFactoryMock = $resourceFactoryMock;
$this->imageProcessorMock = $imageProcessorMock;
$this->logManagerMock = $logManagerMock;
$this->subject = new ImageController(
$this->resourceFactoryMock,
$this->imageProcessorMock,
$this->logManagerMock,
);
}
#[Test]
public function processImageUsesInjectedProcessor(): void
{
$fileMock = $this->createMock(\TYPO3\CMS\Core\Resource\File::class);
$processedFileMock = $this->createMock(\TYPO3\CMS\Core\Resource\ProcessedFile::class);
$this->imageProcessorMock
->expects(self::once())
->method('process')
->with($fileMock, ['width' => 800])
->willReturn($processedFileMock);
$result = $this->subject->processImage($fileMock, ['width' => 800]);
self::assertSame($processedFileMock, $result);
}
}
```
**Key Points:**
- Use intersection types: `ResourceFactory&MockObject` for proper PHPStan type checking
- Assign mocks to properly typed variables before passing to constructor
- This pattern works with PHPUnit 11/12 and PHPStan Level 10
### Handling $GLOBALS and Singleton State
Some TYPO3 components still use global state. Handle this properly:
```php
final class BackendControllerTest extends UnitTestCase
{
protected bool $resetSingletonInstances = true;
#[Test]
public function checksBackendUserPermissions(): void
{
// Mock backend user
$backendUserMock = $this->createMock(BackendUserAuthentication::class);
$backendUserMock->method('isAdmin')->willReturn(true);
$GLOBALS['BE_USER'] = $backendUserMock;
$result = $this->subject->hasAccess();
self::assertTrue($result);
}
#[Test]
public function returnsFalseWhenNoBackendUser(): void
{
$GLOBALS['BE_USER'] = null;
$result = $this->subject->hasAccess();
self::assertFalse($result);
}
}
```
**Important:** Set `protected bool $resetSingletonInstances = true;` when tests interact with TYPO3 singletons to prevent test pollution.
## Mocking Dependencies
Use PHPUnit's built-in mocking (PHPUnit 11/12):
```php
<?php
declare(strict_types=1);
namespace Vendor\Extension\Tests\Unit\Service;
use PHPUnit\Framework\Attributes\Test;
use PHPUnit\Framework\MockObject\MockObject;
use TYPO3\TestingFramework\Core\Unit\UnitTestCase;
use Vendor\Extension\Domain\Model\User;
use Vendor\Extension\Domain\Repository\UserRepository;
use Vendor\Extension\Service\UserService;
final class UserServiceTest extends UnitTestCase
{
private UserService $subject;
/** @var UserRepository&MockObject */
private UserRepository $repositoryMock;
protected function setUp(): void
{
parent::setUp();
/** @var UserRepository&MockObject $repositoryMock */
$repositoryMock = $this->createMock(UserRepository::class);
$this->repositoryMock = $repositoryMock;
$this->subject = new UserService($this->repositoryMock);
}
#[Test]
public function findsUserByEmail(): void
{
$email = '[email protected]';
$user = new User('John');
$this->repositoryMock
->expects(self::once())
->method('findByEmail')
->with($email)
->willReturn($user);
$result = $this->subject->getUserByEmail($email);
self::assertSame('John', $result->getName());
}
#[Test]
public function throwsExceptionWhenUserNotFound(): void
{
$email = '[email protected]';
$this->repositoryMock
->method('findByEmail')
->with($email)
->willReturn(null);
$this->expectException(\RuntimeException::class);
$this->expectExceptionMessage('User not found');
$this->subject->getUserByEmail($email);
}
}
```
> **Note:** TYPO3 13+ with PHPUnit 11/12 uses `createMock()` instead of Prophecy.
> Prophecy is deprecated and should not be used in new tests.
## Assertions
### Common Assertions
```php
// Equality
self::assertEquals($expected, $actual);
self::assertSame($expected, $actual); // Strict comparison
// Boolean
self::assertTrue($condition);
self::assertFalse($condition);
// Null checks
self::assertNull($value);
self::assertNotNull($value);
// Type checks
self::assertIsString($value);
self::assertIsInt($value);
self::assertIsArray($value);
self::assertInstanceOf(User::class, $object);
// Collections
self::assertCount(3, $array);
self::assertEmpty($array);
self::assertContains('item', $array);
// Exceptions
$this->expectException(\InvalidArgumentException::class);
$this->expectExceptionMessage('Invalid input');
$subject->methodThatThrows();
```
### Specific Over Generic
```php
// ❌ Too generic
self::assertTrue($result > 0);
self::assertEquals(true, $isValid);
// ✅ Specific and clear
self::assertGreaterThan(0, $result);
self::assertTrue($isValid);
```
## Data Providers
Test multiple scenarios with data providers:
```php
/**
* @test
* @dataProvider validEmailProvider
*/
public function validatesEmails(string $email, bool $expected): void
{
$result = $this->subject->isValid($email);
self::assertSame($expected, $result);
}
public static function validEmailProvider(): array
{
return [
'valid email' => ['[email protected]', true],
'email with subdomain' => ['[email protected]', true],
'missing @' => ['userexample.com', false],
'missing domain' => ['user@', false],
'empty string' => ['', false],
];
}
```
## Testing Private/Protected Methods
**Preferred Approach**: Test through public API whenever possible:
```php
// ✅ Best approach - test through public interface
$result = $subject->publicMethodThatUsesPrivateMethod();
self::assertSame($expected, $result);
```
**When Reflection is Acceptable**: Sometimes protected methods contain complex logic that deserves dedicated testing (e.g., URL validation, attribute resolution). In these cases, use a helper method:
```php
<?php
declare(strict_types=1);
namespace Vendor\Extension\Tests\Unit\Controller;
use PHPUnit\Framework\Attributes\Test;
use ReflectionMethod;
use TYPO3\TestingFramework\Core\Unit\UnitTestCase;
use Vendor\Extension\Controller\ImageController;
final class ImageControllerTest extends UnitTestCase
{
private ImageController $subject;
/**
* Helper method to access protected methods.
*
* @param array<int, mixed> $args
*/
private function callProtectedMethod(string $methodName, array $args): mixed
{
$reflection = new ReflectionMethod($this->subject, $methodName);
$reflection->setAccessible(true);
return $reflection->invokeArgs($this->subject, $args);
}
#[Test]
public function isExternalImageReturnsTrueForHttpsUrls(): void
{
$result = $this->callProtectedMethod('isExternalImage', ['https://example.com/image.jpg']);
self::assertTrue($result);
}
#[Test]
public function isExternalImageReturnsFalseForLocalPaths(): void
{
$result = $this->callProtectedMethod('isExternalImage', ['/fileadmin/images/test.jpg']);
self::assertFalse($result);
}
}
```
**Important Considerations**:
- Only use reflection when testing protected methods with complex logic worth testing independently
- Never test private methods - refactor to protected if testing is needed
- Prefer testing through public API when the logic is simple
- Document why reflection testing is used for a specific method
## Configuration
### PHPUnit XML (Build/phpunit/UnitTests.xml)
```xml
<phpunit
bootstrap="../../vendor/autoload.php"
cacheResult="false"
beStrictAboutTestsThatDoNotTestAnything="true"
beStrictAboutOutputDuringTests="true"
failOnDeprecation="true"
failOnNotice="true"
failOnWarning="true"
failOnRisky="true">
<testsuites>
<testsuite name="Unit tests">
<directory>../../Tests/Unit/</directory>
</testsuite>
</testsuites>
</phpunit>
```
## Best Practices
1. **One Assert Per Test**: Focus tests on single behavior
2. **Clear Test Names**: Describe what is tested and expected result
3. **Arrange-Act-Assert**: Follow consistent structure
4. **No Logic in Tests**: Tests should be simple and readable
5. **Test Edge Cases**: Empty strings, null, zero, negative numbers
6. **Use Data Providers**: Test multiple scenarios efficiently
7. **Mock External Dependencies**: Keep tests isolated and fast
## Testing PHP Syntax Variants
When testing code that parses or analyzes PHP (like Extension Scanner matchers), test all syntax variants that PHP allows. Different syntaxes may be parsed differently.
### Dynamic Method Calls
PHP supports multiple forms of dynamic method calls:
```php
// DataProvider for testing dynamic call handling
public static function dynamicCallSyntaxDataProvider(): array
{
return [
// Standard dynamic method call - variable holds method name
'dynamic method call with variable' => [
'<?php
$methodName = "someMethod";
$object->$methodName();',
[], // no match expected, must not crash
],
// Expression-based dynamic call - expression evaluated for method name
'dynamic method call with expression' => [
'<?php
$object->{$this->getMethodName()}();',
[], // no match expected, must not crash
],
// Curly brace syntax with variable
'dynamic method call with curly brace variable' => [
'<?php
$object->{$methodName}();',
[], // no match expected, must not crash
],
];
}
```
**Why This Matters**: PhpParser represents these differently:
- `$obj->$var()` → `$node->name` is `PhpParser\Node\Expr\Variable`
- `$obj->{$expr}()` → `$node->name` is `PhpParser\Node\Expr\MethodCall` or other expression
- `$obj->method()` → `$node->name` is `PhpParser\Node\Identifier`
Code assuming `$node->name` is always an `Identifier` will crash on dynamic calls.
### Dynamic Function Calls
```php
'dynamic function call' => [
'<?php
$func = "myFunction";
$func();',
[],
],
'variable function with call_user_func' => [
'<?php
call_user_func($callback, $arg);',
[],
],
```
### Static Method Variants
```php
'dynamic static method call' => [
'<?php
$method = "staticMethod";
SomeClass::$method();',
[],
],
'variable class static call' => [
'<?php
$class = "SomeClass";
$class::staticMethod();',
[],
],
```
### Testing Pattern
Always include regression tests with clear comments:
```php
// Regression test for issue #108413: $object->$var() syntax must not crash
'no match for dynamic method call with variable' => [
[
'Foo->aMethod' => [
'numberOfMandatoryArguments' => 0,
'maximumNumberOfArguments' => 2,
'restFiles' => ['Foo-1.rst'],
],
],
'<?php
$methodName = "someMethod";
$someVar->$methodName();',
[], // no match, must not crash
],
```
## Common Pitfalls
❌ **Testing Framework Code**
```php
// Don't test TYPO3 core functionality
$this->assertTrue(is_array([])); // Useless test
```
❌ **Slow Tests**
```php
// Don't access file system in unit tests
file_put_contents('/tmp/test.txt', 'data');
```
❌ **Test Interdependence**
```php
// Don't depend on test execution order
/** @depends testCreate */
public function testUpdate(): void { }
```
✅ **Focused, Fast, Isolated Tests**
```php
/**
* @test
*/
public function calculatesPriceWithDiscount(): void
{
$calculator = new PriceCalculator();
$price = $calculator->calculate(100.0, 0.2);
self::assertSame(80.0, $price);
}
```
## Running Unit Tests
```bash
# Via runTests.sh
Build/Scripts/runTests.sh -s unit
# Via PHPUnit directly
vendor/bin/phpunit -c Build/phpunit/UnitTests.xml
# Via Composer
composer ci:test:php:unit
# Single test file
vendor/bin/phpunit Tests/Unit/Domain/Validator/EmailValidatorTest.php
# Single test method
vendor/bin/phpunit --filter testValidEmail
```
## Troubleshooting Common Issues
### PHPStan Errors with Mocks
**Problem**: PHPStan complains about mock type mismatches.
```
Method expects ResourceFactory but got ResourceFactory&MockObject
```
**Solution**: Use intersection type annotations:
```php
/** @var ResourceFactory&MockObject */
private ResourceFactory $resourceFactoryMock;
protected function setUp(): void
{
parent::setUp();
/** @var ResourceFactory&MockObject $resourceFactoryMock */
$resourceFactoryMock = $this->createMock(ResourceFactory::class);
$this->resourceFactoryMock = $resourceFactoryMock;
$this->subject = new MyController($this->resourceFactoryMock);
}
```
### Undefined Array Key Warnings
**Problem**: Tests throw warnings about missing array keys.
```
Undefined array key "fileId"
```
**Solution**: Always provide all required keys in mock arrays:
```php
// ❌ Incomplete mock data
$requestMock->method('getQueryParams')->willReturn([
'fileId' => 123,
]);
// ✅ Complete mock data
$requestMock->method('getQueryParams')->willReturn([
'fileId' => 123,
'table' => 'tt_content',
'P' => [],
]);
```
### Tests Requiring Functional Setup
**Problem**: Unit tests fail with cache or framework errors.
```
NoSuchCacheException: A cache with identifier "runtime" does not exist.
```
**Solution**: Identify methods that require TYPO3 framework infrastructure and move them to functional tests:
- Methods using `BackendUtility::getPagesTSconfig()`
- Methods calling parent class framework behavior
- Methods requiring global state like `$GLOBALS['TYPO3_CONF_VARS']`
Add comments explaining the limitation:
```php
// Note: getMaxDimensions tests require functional test setup due to BackendUtility dependency
// These are better tested in functional tests
```
### Singleton State Pollution
**Problem**: Tests interfere with each other due to singleton state.
**Solution**: Enable singleton reset in your test class:
```php
final class MyControllerTest extends UnitTestCase
{
protected bool $resetSingletonInstances = true;
#[Test]
public function testWithGlobals(): void
{
$GLOBALS['BE_USER'] = $this->createMock(BackendUserAuthentication::class);
// Test will clean up automatically
}
}
```
### Exception Flow Issues
**Problem**: Catching and re-throwing exceptions masks the original error.
```php
// ❌ Inner exception caught by outer catch
try {
$file = $this->factory->getFile($id);
if ($file->isDeleted()) {
throw new RuntimeException('Deleted', 1234);
}
} catch (Exception $e) {
throw new RuntimeException('Not found', 5678);
}
```
**Solution**: Separate concerns - catch only what you need:
```php
// ✅ Proper exception flow
try {
$file = $this->factory->getFile($id);
} catch (Exception $e) {
throw new RuntimeException('Not found', 5678, $e);
}
if ($file->isDeleted()) {
throw new RuntimeException('Deleted', 1234);
}
```
## Testing DataHandler Hooks
DataHandler hooks (`processDatamap_*`, `processCmdmap_*`) require careful testing as they interact with TYPO3 globals.
### Example: Testing processDatamap_postProcessFieldArray
```php
<?php
declare(strict_types=1);
namespace Vendor\Extension\Tests\Unit\Database;
use PHPUnit\Framework\Attributes\Test;
use PHPUnit\Framework\MockObject\MockObject;
use TYPO3\CMS\Core\Configuration\ExtensionConfiguration;
use TYPO3\CMS\Core\Context\Context;
use TYPO3\CMS\Core\DataHandling\DataHandler;
use TYPO3\CMS\Core\Http\RequestFactory;
use TYPO3\CMS\Core\Log\LogManager;
use TYPO3\CMS\Core\Log\Logger;
use TYPO3\CMS\Core\Resource\DefaultUploadFolderResolver;
use TYPO3\CMS\Core\Resource\ResourceFactory;
use TYPO3\TestingFramework\Core\Unit\UnitTestCase;
use Vendor\Extension\Database\MyDataHandlerHook;
/**
* Unit tests for MyDataHandlerHook.
*
* @covers \Vendor\Extension\Database\MyDataHandlerHook
*/
final class MyDataHandlerHookTest extends UnitTestCase
{
protected bool $resetSingletonInstances = true;
private MyDataHandlerHook $subject;
/** @var ExtensionConfiguration&MockObject */
private ExtensionConfiguration $extensionConfigurationMock;
/** @var LogManager&MockObject */
private LogManager $logManagerMock;
/** @var ResourceFactory&MockObject */
private ResourceFactory $resourceFactoryMock;
/** @var Context&MockObject */
private Context $contextMock;
/** @var RequestFactory&MockObject */
private RequestFactory $requestFactoryMock;
/** @var DefaultUploadFolderResolver&MockObject */
private DefaultUploadFolderResolver $uploadFolderResolverMock;
/** @var Logger&MockObject */
private Logger $loggerMock;
protected function setUp(): void
{
parent::setUp();
// Create all required mocks with intersection types for PHPStan compliance
/** @var ExtensionConfiguration&MockObject $extensionConfigurationMock */
$extensionConfigurationMock = $this->createMock(ExtensionConfiguration::class);
/** @var LogManager&MockObject $logManagerMock */
$logManagerMock = $this->createMock(LogManager::class);
/** @var ResourceFactory&MockObject $resourceFactoryMock */
$resourceFactoryMock = $this->createMock(ResourceFactory::class);
/** @var Context&MockObject $contextMock */
$contextMock = $this->createMock(Context::class);
/** @var RequestFactory&MockObject $requestFactoryMock */
$requestFactoryMock = $this->createMock(RequestFactory::class);
/** @var DefaultUploadFolderResolver&MockObject $uploadFolderResolverMock */
$uploadFolderResolverMock = $this->createMock(DefaultUploadFolderResolver::class);
/** @var Logger&MockObject $loggerMock */
$loggerMock = $this->createMock(Logger::class);
// Configure extension configuration mock with willReturnCallback
$extensionConfigurationMock
->method('get')
->willReturnCallback(function ($extension, $key) {
if ($extension === 'my_extension') {
return match ($key) {
'enableFeature' => true,
'timeout' => 30,
default => null,
};
}
return null;
});
// Configure log manager to return logger mock
$logManagerMock
->method('getLogger')
->with(MyDataHandlerHook::class)
->willReturn($loggerMock);
// Assign mocks to properties
$this->extensionConfigurationMock = $extensionConfigurationMock;
$this->logManagerMock = $logManagerMock;
$this->resourceFactoryMock = $resourceFactoryMock;
$this->contextMock = $contextMock;
$this->requestFactoryMock = $requestFactoryMock;
$this->uploadFolderResolverMock = $uploadFolderResolverMock;
$this->loggerMock = $loggerMock;
// Create subject with all dependencies
$this->subject = new MyDataHandlerHook(
$this->extensionConfigurationMock,
$this->logManagerMock,
$this->resourceFactoryMock,
$this->contextMock,
$this->requestFactoryMock,
$this->uploadFolderResolverMock,
);
}
#[Test]
public function constructorInitializesWithDependencyInjection(): void
{
// Verify subject was created successfully with all dependencies
self::assertInstanceOf(MyDataHandlerHook::class, $this->subject);
}
#[Test]
public function processDatamapPostProcessFieldArrayHandlesFieldCorrectly(): void
{
$status = 'update';
$table = 'tt_content';
$id = '123';
$fieldArray = ['bodytext' => '<p>Content with processing</p>'];
/** @var DataHandler&MockObject $dataHandlerMock */
$dataHandlerMock = $this->createMock(DataHandler::class);
// Mock TCA configuration for RTE field
$GLOBALS['TCA']['tt_content']['columns']['bodytext']['config'] = [
'type' => 'text',
'enableRichtext' => true,
];
// Test the hook processes the field
$this->subject->processDatamap_postProcessFieldArray(
$status,
$table,
$id,
$fieldArray,
$dataHandlerMock,
);
// Assert field was processed (actual assertion depends on implementation)
self::assertNotEmpty($fieldArray['bodytext']);
}
#[Test]
public function constructorLoadsExtensionConfiguration(): void
{
/** @var ExtensionConfiguration&MockObject $configMock */
$configMock = $this->createMock(ExtensionConfiguration::class);
$configMock
->expects(self::exactly(2))
->method('get')
->willReturnCallback(function ($extension, $key) {
self::assertSame('my_extension', $extension);
return match ($key) {
'enableFeature' => true,
'timeout' => 30,
default => null,
};
});
new MyDataHandlerHook(
$configMock,
$this->logManagerMock,
$this->resourceFactoryMock,
$this->contextMock,
$this->requestFactoryMock,
$this->uploadFolderResolverMock,
);
}
}
```
**Key Testing Patterns for DataHandler Hooks:**
1. **Intersection Types for PHPStan**: Use `ResourceFactory&MockObject` for strict type compliance
2. **TCA Globals**: Set `$GLOBALS['TCA']` in tests to simulate TYPO3 table configuration
3. **Extension Configuration**: Use `willReturnCallback` with `match` expressions for flexible config mocking
4. **DataHandler Mock**: Create mock for `$dataHandler` parameter (required in hook signature)
5. **Reset Singletons**: Always set `protected bool $resetSingletonInstances = true;`
6. **Constructor DI**: Inject all dependencies via constructor (TYPO3 13+ best practice)
## Resources
- [TYPO3 Unit Testing Documentation](https://docs.typo3.org/m/typo3/reference-coreapi/main/en-us/Testing/UnitTests.html)
- [PHPUnit Documentation](https://phpunit.de/documentation.html)
- [PHPUnit 11 Migration Guide](https://phpunit.de/announcements/phpunit-11.html)
- [TYPO3 DataHandler Hooks](https://docs.typo3.org/m/typo3/reference-coreapi/main/en-us/ApiOverview/Hooks/DataHandler/Index.html)
```
### references/functional-testing.md
```markdown
# Functional Testing in TYPO3
Functional tests verify components that interact with external systems like databases, using a full TYPO3 instance.
## When to Use Functional Tests
- Testing database operations (repositories, queries)
- Controller and plugin functionality
- Hook and event implementations
- DataHandler operations
- File and folder operations
- Extension configuration behavior
## Base Class
All functional tests extend `TYPO3\TestingFramework\Core\Functional\FunctionalTestCase`:
```php
<?php
declare(strict_types=1);
namespace Vendor\Extension\Tests\Functional\Domain\Repository;
use TYPO3\TestingFramework\Core\Functional\FunctionalTestCase;
use Vendor\Extension\Domain\Model\Product;
use Vendor\Extension\Domain\Repository\ProductRepository;
final class ProductRepositoryTest extends FunctionalTestCase
{
protected ProductRepository $subject;
protected array $testExtensionsToLoad = [
'typo3conf/ext/my_extension',
];
protected function setUp(): void
{
parent::setUp();
$this->subject = $this->get(ProductRepository::class);
}
/**
* @test
*/
public function findsProductsByCategory(): void
{
$this->importCSVDataSet(__DIR__ . '/../Fixtures/Products.csv');
$products = $this->subject->findByCategory(1);
self::assertCount(3, $products);
}
}
```
## Test Database
Functional tests use an isolated test database:
- Created before test execution
- Populated with fixtures
- Destroyed after test completion
- Supports: MySQL, MariaDB, PostgreSQL, SQLite
### Database Configuration
Set via environment or `FunctionalTests.xml`:
```xml
<php>
<env name="typo3DatabaseDriver" value="mysqli"/>
<env name="typo3DatabaseHost" value="localhost"/>
<env name="typo3DatabasePort" value="3306"/>
<env name="typo3DatabaseUsername" value="root"/>
<env name="typo3DatabasePassword" value=""/>
<env name="typo3DatabaseName" value="typo3_test"/>
</php>
```
## Database Fixtures
### CSV Format
Create fixtures in `Tests/Functional/Fixtures/`:
```csv
# pages.csv
uid,pid,title,doktype
1,0,"Root",1
2,1,"Products",1
3,1,"Services",1
```
```csv
# tx_myext_domain_model_product.csv
uid,pid,title,price,category
1,2,"Product A",10.00,1
2,2,"Product B",20.00,1
3,2,"Product C",15.00,2
```
### Import Fixtures
```php
/**
* @test
*/
public function findsProducts(): void
{
// Import fixture
$this->importCSVDataSet(__DIR__ . '/../Fixtures/Products.csv');
// Test repository
$products = $this->subject->findAll();
self::assertCount(3, $products);
}
```
### Multiple Fixtures
```php
protected function setUp(): void
{
parent::setUp();
// Import common fixtures
$this->importCSVDataSet(__DIR__ . '/../Fixtures/pages.csv');
$this->importCSVDataSet(__DIR__ . '/../Fixtures/be_users.csv');
$this->subject = $this->get(ProductRepository::class);
}
```
## Dependency Injection
Use `$this->get()` to retrieve services:
```php
protected function setUp(): void
{
parent::setUp();
// Get service from container
$this->subject = $this->get(ProductRepository::class);
$this->dataMapper = $this->get(DataMapper::class);
}
```
## Testing Extensions
### Load Test Extensions
```php
protected array $testExtensionsToLoad = [
'typo3conf/ext/my_extension',
'typo3conf/ext/dependency_extension',
];
```
### Core Extensions
```php
protected array $coreExtensionsToLoad = [
'form',
'workspaces',
];
```
## Site Configuration
Create site configuration for frontend tests:
```php
protected function setUp(): void
{
parent::setUp();
$this->importCSVDataSet(__DIR__ . '/../Fixtures/pages.csv');
$this->writeSiteConfiguration(
'test',
[
'rootPageId' => 1,
'base' => 'http://localhost/',
]
);
}
```
## Frontend Requests
Test frontend rendering:
```php
use TYPO3\TestingFramework\Core\Functional\Framework\Frontend\InternalRequest;
/**
* @test
*/
public function rendersProductList(): void
{
$this->importCSVDataSet(__DIR__ . '/../Fixtures/pages.csv');
$this->importCSVDataSet(__DIR__ . '/../Fixtures/Products.csv');
$this->writeSiteConfiguration('test', ['rootPageId' => 1]);
$response = $this->executeFrontendSubRequest(
new InternalRequest('http://localhost/products')
);
self::assertStringContainsString('Product A', (string)$response->getBody());
}
```
## Testing DataHandler Hooks (SC_OPTIONS)
Test DataHandler SC_OPTIONS hook integration with real framework:
```php
<?php
declare(strict_types=1);
namespace Vendor\Extension\Tests\Functional\Database;
use TYPO3\CMS\Core\DataHandling\DataHandler;
use TYPO3\TestingFramework\Core\Functional\FunctionalTestCase;
use Vendor\Extension\Database\MyDataHandlerHook;
final class MyDataHandlerHookTest extends FunctionalTestCase
{
protected array $testExtensionsToLoad = [
'typo3conf/ext/my_extension',
];
protected array $coreExtensionsToLoad = [
'typo3/cms-rte-ckeditor', // If testing RTE-related hooks
];
protected function setUp(): void
{
parent::setUp();
$this->importCSVDataSet(__DIR__ . '/Fixtures/pages.csv');
$this->importCSVDataSet(__DIR__ . '/Fixtures/tt_content.csv');
}
private function createSubject(): MyDataHandlerHook
{
// Get services from container with proper DI
return new MyDataHandlerHook(
$this->get(ExtensionConfiguration::class),
$this->get(LogManager::class),
$this->get(ResourceFactory::class),
);
}
/**
* @test
*/
public function processDatamapPostProcessFieldArrayHandlesRteField(): void
{
$subject = $this->createSubject();
$status = 'update';
$table = 'tt_content';
$id = '1';
$fieldArray = [
'bodytext' => '<p>Test content with <img src="image.jpg" /></p>',
];
/** @var DataHandler $dataHandler */
$dataHandler = $this->get(DataHandler::class);
// Configure TCA for RTE field
/** @var array<string, mixed> $tcaConfig */
$tcaConfig = [
'type' => 'text',
'enableRichtext' => true,
];
// @phpstan-ignore-next-line offsetAccess.nonOffsetAccessible
$GLOBALS['TCA']['tt_content']['columns']['bodytext']['config'] = $tcaConfig;
$subject->processDatamap_postProcessFieldArray(
$status,
$table,
$id,
$fieldArray,
$dataHandler,
);
// Field should be processed by hook
self::assertArrayHasKey('bodytext', $fieldArray);
self::assertIsString($fieldArray['bodytext']);
self::assertNotEmpty($fieldArray['bodytext']);
self::assertStringContainsString('Test content', $fieldArray['bodytext']);
}
/**
* @test
*/
public function hookIsRegisteredInGlobals(): void
{
// Verify hook is properly registered in TYPO3_CONF_VARS
self::assertIsArray($GLOBALS['TYPO3_CONF_VARS']);
self::assertArrayHasKey('SC_OPTIONS', $GLOBALS['TYPO3_CONF_VARS']);
$scOptions = $GLOBALS['TYPO3_CONF_VARS']['SC_OPTIONS'];
self::assertIsArray($scOptions);
self::assertArrayHasKey('t3lib/class.t3lib_tcemain.php', $scOptions);
$tcemainOptions = $scOptions['t3lib/class.t3lib_tcemain.php'];
self::assertIsArray($tcemainOptions);
self::assertArrayHasKey('processDatamapClass', $tcemainOptions);
$registeredHooks = $tcemainOptions['processDatamapClass'];
self::assertIsArray($registeredHooks);
// Hook class should be registered
self::assertContains(MyDataHandlerHook::class, $registeredHooks);
}
}
```
### Key Patterns for DataHandler Hook Testing
1. **Use Factory Method Pattern**: Create `createSubject()` method to avoid uninitialized property PHPStan errors
2. **Test Real Framework Integration**: Don't mock DataHandler, test actual hook execution
3. **Configure TCA Dynamically**: Set up `$GLOBALS['TCA']` in tests for field configuration
4. **Verify Hook Registration**: Test that hooks are properly registered in `$GLOBALS['TYPO3_CONF_VARS']`
5. **Test Multiple Scenarios**: new vs update, single vs multiple fields, RTE vs non-RTE
## Testing File Abstraction Layer (FAL)
Test ResourceFactory and FAL storage integration:
```php
<?php
declare(strict_types=1);
namespace Vendor\Extension\Tests\Functional\Controller;
use TYPO3\CMS\Core\Resource\File;
use TYPO3\CMS\Core\Resource\Folder;
use TYPO3\CMS\Core\Resource\ResourceFactory;
use TYPO3\CMS\Core\Resource\ResourceStorage;
use TYPO3\TestingFramework\Core\Functional\FunctionalTestCase;
use Vendor\Extension\Controller\ImageRenderingController;
final class ImageRenderingControllerTest extends FunctionalTestCase
{
protected array $testExtensionsToLoad = [
'typo3conf/ext/my_extension',
];
protected function setUp(): void
{
parent::setUp();
$this->importCSVDataSet(__DIR__ . '/Fixtures/sys_file_storage.csv');
$this->importCSVDataSet(__DIR__ . '/Fixtures/sys_file.csv');
}
/**
* @test
*/
public function storageIsAccessible(): void
{
/** @var ResourceFactory $resourceFactory */
$resourceFactory = $this->get(ResourceFactory::class);
$storage = $resourceFactory->getStorageObject(1);
self::assertInstanceOf(ResourceStorage::class, $storage);
self::assertTrue($storage->isOnline());
}
/**
* @test
*/
public function canRetrieveFileFromStorage(): void
{
/** @var ResourceFactory $resourceFactory */
$resourceFactory = $this->get(ResourceFactory::class);
// Get file from test data
$file = $resourceFactory->getFileObject(1);
self::assertInstanceOf(File::class, $file);
self::assertSame('test-image.jpg', $file->getName());
}
/**
* @test
*/
public function canAccessStorageRootFolder(): void
{
/** @var ResourceFactory $resourceFactory */
$resourceFactory = $this->get(ResourceFactory::class);
$storage = $resourceFactory->getStorageObject(1);
$rootFolder = $storage->getRootLevelFolder();
self::assertInstanceOf(Folder::class, $rootFolder);
self::assertSame('/', $rootFolder->getIdentifier());
}
}
```
### FAL Test Fixtures
**sys_file_storage.csv:**
```csv
uid,pid,name,driver,configuration,is_default,is_browsable,is_public,is_writable,is_online
1,0,"fileadmin","Local","<?xml version=""1.0"" encoding=""utf-8"" standalone=""yes"" ?><T3FlexForms><data><sheet index=""sDEF""><language index=""lDEF""><field index=""basePath""><value index=""vDEF"">fileadmin/</value></field><field index=""pathType""><value index=""vDEF"">relative</value></field><field index=""caseSensitive""><value index=""vDEF"">1</value></field></language></sheet></data></T3FlexForms>",1,1,1,1,1
```
**sys_file.csv:**
```csv
uid,pid,storage,identifier,name,type,mime_type,size,sha1,extension
1,0,1,"/test-image.jpg","test-image.jpg",2,"image/jpeg",12345,"da39a3ee5e6b4b0d3255bfef95601890afd80709","jpg"
```
### Key Patterns for FAL Testing
1. **Test Storage Configuration**: Verify storage is properly configured and online
2. **Test File Retrieval**: Use `getFileObject()` to retrieve files from sys_file
3. **Test Folder Operations**: Verify folder access and structure
4. **Use CSV Fixtures**: Import sys_file_storage and sys_file test data
5. **Test Real Services**: Use container's ResourceFactory, don't mock
## PHPStan Type Safety in Functional Tests
### Handling $GLOBALS['TCA'] with PHPStan Level 9
PHPStan cannot infer types for runtime-configured `$GLOBALS` arrays. Use ignore annotations:
```php
// Configure TCA for RTE field
/** @var array<string, mixed> $tcaConfig */
$tcaConfig = [
'type' => 'text',
'enableRichtext' => true,
];
// @phpstan-ignore-next-line offsetAccess.nonOffsetAccessible
$GLOBALS['TCA']['tt_content']['columns']['bodytext']['config'] = $tcaConfig;
```
### Type Assertions for Dynamic Arrays
When testing field arrays that are modified by reference:
```php
// ❌ PHPStan cannot verify this is still an array
self::assertStringContainsString('Test', $fieldArray['bodytext']);
// ✅ Add type assertions
self::assertArrayHasKey('bodytext', $fieldArray);
self::assertIsString($fieldArray['bodytext']);
self::assertStringContainsString('Test', $fieldArray['bodytext']);
```
### Avoiding Uninitialized Property Errors
Use factory methods instead of properties initialized in setUp():
```php
// ❌ PHPStan warns about uninitialized property
private MyService $subject;
protected function setUp(): void
{
$this->subject = $this->get(MyService::class);
}
// ✅ Use factory method
private function createSubject(): MyService
{
return $this->get(MyService::class);
}
public function testSomething(): void
{
$subject = $this->createSubject();
// Use $subject
}
```
### PHPStan Annotations for Functional Tests
Common patterns:
```php
// Ignore $GLOBALS access
// @phpstan-ignore-next-line offsetAccess.nonOffsetAccessible
$GLOBALS['TCA']['table']['columns']['field']['config'] = $config;
// Type hint service retrieval
/** @var DataHandler $dataHandler */
$dataHandler = $this->get(DataHandler::class);
// Type hint config arrays
/** @var array<string, mixed> $tcaConfig */
$tcaConfig = ['type' => 'text'];
```
## Backend User Context
Test with backend user:
```php
use TYPO3\TestingFramework\Core\Functional\Framework\Frontend\InternalRequest;
/**
* @test
*/
public function editorCanEditRecord(): void
{
$this->importCSVDataSet(__DIR__ . '/../Fixtures/be_users.csv');
$this->importCSVDataSet(__DIR__ . '/../Fixtures/Products.csv');
$this->setUpBackendUser(1); // uid from be_users.csv
$dataHandler = $this->get(DataHandler::class);
$dataHandler->start(
[
'tx_myext_domain_model_product' => [
1 => ['title' => 'Updated Product']
]
],
[]
);
$dataHandler->process_datamap();
self::assertEmpty($dataHandler->errorLog);
}
```
## File Operations
Test file handling:
```php
/**
* @test
*/
public function uploadsFile(): void
{
$fileStorage = $this->get(StorageRepository::class)->getDefaultStorage();
$file = $fileStorage->addFile(
__DIR__ . '/../Fixtures/Files/test.jpg',
$fileStorage->getDefaultFolder(),
'test.jpg'
);
self::assertFileExists($file->getForLocalProcessing(false));
}
```
## Configuration
### PHPUnit XML (Build/phpunit/FunctionalTests.xml)
```xml
<phpunit
bootstrap="FunctionalTestsBootstrap.php"
cacheResult="false"
beStrictAboutTestsThatDoNotTestAnything="true"
failOnDeprecation="true"
failOnNotice="true"
failOnWarning="true">
<testsuites>
<testsuite name="Functional tests">
<directory>../../Tests/Functional/</directory>
</testsuite>
</testsuites>
<php>
<const name="TYPO3_TESTING_FUNCTIONAL_REMOVE_ERROR_HANDLER" value="true" />
<env name="TYPO3_CONTEXT" value="Testing"/>
<env name="typo3DatabaseDriver" value="mysqli"/>
</php>
</phpunit>
```
### Bootstrap (Build/phpunit/FunctionalTestsBootstrap.php)
```php
<?php
declare(strict_types=1);
call_user_func(static function () {
$testbase = new \TYPO3\TestingFramework\Core\Testbase();
$testbase->defineOriginalRootPath();
$testbase->createDirectory(ORIGINAL_ROOT . 'typo3temp/var/tests');
$testbase->createDirectory(ORIGINAL_ROOT . 'typo3temp/var/transient');
});
```
## Fixture Strategy
### Minimal Fixtures
Keep fixtures focused on test requirements:
```php
// ❌ Too much data
$this->importCSVDataSet(__DIR__ . '/../Fixtures/AllProducts.csv'); // 500 records
// ✅ Minimal test data
$this->importCSVDataSet(__DIR__ . '/../Fixtures/ProductsByCategory.csv'); // 3 records
```
### Reusable Fixtures
Create shared fixtures for common scenarios:
```
Tests/Functional/Fixtures/
├── pages.csv # Basic page tree
├── be_users.csv # Test backend users
├── Products/
│ ├── BasicProducts.csv # 3 simple products
│ ├── ProductsWithCategories.csv
│ └── ProductsWithImages.csv
```
### Fixture Documentation
Document fixture purpose in test or AGENTS.md:
```php
/**
* @test
*/
public function findsProductsByCategory(): void
{
// Fixture contains: 3 products in category 1, 2 products in category 2
$this->importCSVDataSet(__DIR__ . '/../Fixtures/ProductsByCategory.csv');
$products = $this->subject->findByCategory(1);
self::assertCount(3, $products);
}
```
## Best Practices
1. **Use setUp() for Common Setup**: Import shared fixtures in setUp()
2. **One Test Database**: Each test gets clean database instance
3. **Test Isolation**: Don't depend on other test execution
4. **Minimal Fixtures**: Only data required for specific test
5. **Clear Assertions**: Test specific behavior, not implementation
6. **Cleanup**: Testing framework handles cleanup automatically
## Common Pitfalls
❌ **Large Fixtures**
```php
// Don't import unnecessary data
$this->importCSVDataSet('AllData.csv'); // 10,000 records
```
❌ **No Fixtures**
```php
// Don't expect data to exist
$products = $this->subject->findAll();
self::assertCount(0, $products); // Always true without fixtures
```
❌ **Missing Extensions**
```php
// Don't forget to load extension under test
// Missing: protected array $testExtensionsToLoad = ['typo3conf/ext/my_extension'];
```
✅ **Focused, Well-Documented Tests**
```php
/**
* @test
*/
public function findsByCategory(): void
{
// Fixture: 3 products in category 1
$this->importCSVDataSet(__DIR__ . '/../Fixtures/CategoryProducts.csv');
$products = $this->subject->findByCategory(1);
self::assertCount(3, $products);
self::assertSame('Product A', $products[0]->getTitle());
}
```
## Running Functional Tests with DDEV
Functional tests require the mysqli extension which is typically not available on the host system. Run tests inside the DDEV container:
```bash
# ❌ Wrong - mysqli not available on host PHP
./vendor/bin/phpunit -c Build/phpunit/FunctionalTests.xml
# ✅ Correct - Run inside DDEV with database credentials
ddev exec typo3DatabaseHost=db typo3DatabaseUsername=db typo3DatabasePassword=db typo3DatabaseName=db \
./vendor/bin/phpunit -c Build/phpunit/FunctionalTests.xml
```
### DDEV Database Configuration
When using DDEV, the database credentials are:
- **Host**: `db`
- **Username**: `db`
- **Password**: `db`
- **Database**: `db`
## Handling cHash Validation Errors
Frontend tests with query parameters may fail with "cHash empty" errors. Exclude test parameters from cHash validation:
```php
final class MyFunctionalTest extends FunctionalTestCase
{
/**
* Exclude test parameters from cHash validation to avoid errors.
*/
protected array $configurationToUseInTestInstance = [
'FE' => [
'cacheHash' => [
'excludedParameters' => ['test', 'myTestParam'],
],
],
];
}
```
## InternalRequest Query Parameters
`InternalRequest` does not parse URL-embedded query strings. Always use `withQueryParameters()`:
```php
// ❌ Wrong - Query string not parsed by InternalRequest
$request = new InternalRequest('http://localhost/?id=1&test=1');
// ✅ Correct - Use withQueryParameters()
$request = (new InternalRequest('http://localhost/'))
->withQueryParameters(['id' => 1, 'test' => 1]);
$response = $this->executeFrontendSubRequest($request);
```
## Singleton Reset for Test Isolation
Singleton classes must provide a `reset()` method to ensure fresh state between tests:
```php
// Singleton class with reset capability
final class Container
{
private static ?self $instance = null;
public static function get(): self
{
return self::$instance ??= new self();
}
public static function reset(): void
{
self::$instance = null;
}
}
// In test setUp()
protected function setUp(): void
{
parent::setUp();
Container::reset(); // Ensure fresh state between tests
}
```
## Session State Isolation in Fixtures
When testing contexts or features that use session storage, disable sessions in test fixtures to prevent test pollution:
```csv
# tx_contexts_contexts.csv
# Column: use_session - Set to 0 to prevent session state from persisting between tests
"tx_contexts_contexts"
,"uid","pid","title","alias","type","type_conf","invert","use_session","disabled","hide_in_backend"
,1,1,"test get","testget","getparam","...",0,0,0,0
```
**Why this matters**: Session-based contexts can cause flaky tests when session state persists between test runs. Always set `use_session=0` in fixtures unless specifically testing session functionality.
## Safe tearDown Pattern
When `setUp()` might fail (e.g., database connection issues), `tearDown()` should handle incomplete initialization:
```php
protected function tearDown(): void
{
// Clean up test-specific globals
unset($_GET['test']);
// Handle cases where setUp() didn't complete
try {
parent::tearDown();
} catch (Error) {
// Setup didn't complete, nothing to tear down
}
}
```
## Site Configuration (TYPO3 v12+)
Use `SiteWriter` instead of the deprecated `writeSiteConfiguration()`:
```php
use TYPO3\CMS\Core\Configuration\SiteWriter;
protected function setUp(): void
{
parent::setUp();
$this->importCSVDataSet(__DIR__ . '/Fixtures/pages.csv');
// ❌ Deprecated in TYPO3 v12
// $this->writeSiteConfiguration('test', ['rootPageId' => 1, 'base' => '/']);
// ✅ TYPO3 v12+ with SiteWriter
$siteWriter = $this->get(SiteWriter::class);
$siteWriter->createNewBasicSite('website-local', 1, 'http://localhost/');
// Set up TypoScript for frontend rendering
$this->setUpFrontendRootPage(1, [
'EXT:my_extension/Tests/Functional/Fixtures/TypoScript/Basic.typoscript',
]);
}
```
## Running Functional Tests
```bash
# Via runTests.sh
Build/Scripts/runTests.sh -s functional
# Via PHPUnit directly (on host with mysqli)
vendor/bin/phpunit -c Build/phpunit/FunctionalTests.xml
# Via DDEV (recommended)
ddev exec typo3DatabaseHost=db typo3DatabaseUsername=db typo3DatabasePassword=db typo3DatabaseName=db \
vendor/bin/phpunit -c Build/phpunit/FunctionalTests.xml
# Via Composer
composer ci:test:php:functional
# With specific database driver
typo3DatabaseDriver=pdo_mysql vendor/bin/phpunit -c Build/phpunit/FunctionalTests.xml
# Single test
vendor/bin/phpunit Tests/Functional/Domain/Repository/ProductRepositoryTest.php
```
## Resources
- [TYPO3 Functional Testing Documentation](https://docs.typo3.org/m/typo3/reference-coreapi/main/en-us/Testing/FunctionalTests.html)
- [Testing Framework](https://github.com/typo3/testing-framework)
- [CSV Fixture Format](https://docs.typo3.org/m/typo3/reference-coreapi/main/en-us/Testing/FunctionalTests.html#importing-data)
```
### references/functional-test-patterns.md
```markdown
# Functional Test Patterns for TYPO3 12/13
> **Source**: Real-world patterns from netresearch/contexts extension testing (2024-12)
## Container Reset Between Tests
When testing classes that use dependency injection, reset the container between tests:
```php
protected function setUp(): void
{
parent::setUp();
// Reset container to ensure clean DI state
$this->resetContainer();
}
```
**Why**: Prevents test pollution from cached service instances.
## Site Configuration in Functional Tests
Create site configuration via YAML files, not PHP APIs:
```php
protected function setUp(): void
{
parent::setUp();
$this->importCSVDataSet(__DIR__ . '/Fixtures/pages.csv');
// Create site configuration directory
$siteConfigPath = $this->instancePath . '/config/sites/main';
GeneralUtility::mkdir_p($siteConfigPath);
// Write YAML configuration directly
file_put_contents(
$siteConfigPath . '/config.yaml',
Yaml::dump([
'rootPageId' => 1,
'base' => '/',
'languages' => [
[
'languageId' => 0,
'title' => 'English',
'locale' => 'en_US.UTF-8',
'base' => '/',
],
],
])
);
}
```
## Disabling Session for Context Fixtures
When testing contexts that don't need session:
```php
// Fixture: Tests/Functional/Fixtures/tx_contexts_contexts.csv
"uid","pid","title","type","type_conf","disabled","hide_in_backend"
1,0,"Test Context","ip","","0","0"
// Test class
protected function setUp(): void
{
parent::setUp();
$this->importCSVDataSet(__DIR__ . '/Fixtures/tx_contexts_contexts.csv');
// Disable session to avoid "session not available" errors
$GLOBALS['TYPO3_CONF_VARS']['FE']['sessionDataLifetime'] = 0;
}
```
## LinkVars Warning Fix
Avoid `linkVars not set` warnings in functional tests:
```php
// In test setup or fixture TypoScript
$GLOBALS['TSFE']->config['config']['linkVars'] = '';
```
Or in site TypoScript fixture:
```typoscript
config.linkVars =
```
## PHPUnit 10/11/12 Migration Patterns
### Removed: `$this->at()` Matcher
**PHPUnit 9** (deprecated):
```php
$mock->expects($this->at(0))->method('foo')->willReturn('first');
$mock->expects($this->at(1))->method('foo')->willReturn('second');
```
**PHPUnit 10+**:
```php
$mock->expects($this->exactly(2))
->method('foo')
->willReturnOnConsecutiveCalls('first', 'second');
```
### Callback Matcher for Complex Sequences
```php
$callCount = 0;
$mock->method('foo')
->willReturnCallback(function () use (&$callCount) {
return match (++$callCount) {
1 => 'first',
2 => 'second',
default => 'default',
};
});
```
## Database Credentials for DDEV
In `Build/phpunit/FunctionalTests.xml`:
```xml
<php>
<env name="typo3DatabaseDriver" value="mysqli"/>
<env name="typo3DatabaseHost" value="db"/>
<env name="typo3DatabasePort" value="3306"/>
<env name="typo3DatabaseUsername" value="db"/>
<env name="typo3DatabasePassword" value="db"/>
<env name="typo3DatabaseName" value="func_tests"/>
</php>
```
## Test Framework Compatibility Matrix
| PHPUnit | TYPO3 Testing Framework | TYPO3 Version |
|---------|------------------------|---------------|
| ^10.5 | ^8.0 | 12.4 LTS |
| ^11.0 | ^8.2 \|\| ^9.0 | 12.4, 13.4 |
| ^12.0 | ^9.0 | 13.4 LTS |
## Functional Test with Request Attribute (v13)
Testing code that uses PSR-7 request attributes:
```php
use TYPO3\CMS\Core\Http\ServerRequest;
use TYPO3\CMS\Frontend\Page\PageInformation;
public function testWithPageInformation(): void
{
$pageInfo = new PageInformation();
$pageInfo->setId(1);
$pageInfo->setRootLine([['uid' => 1]]);
$request = (new ServerRequest())
->withAttribute('frontend.page.information', $pageInfo);
$result = $this->subject->process($request);
self::assertSame(1, $result->getPageId());
}
```
```
### references/typo3-v14-final-classes.md
```markdown
# Testing TYPO3 v14 Final Classes
TYPO3 v14 introduces many `final` and `readonly` classes that cannot be mocked directly. This guide covers patterns to maintain testability.
## The Problem
TYPO3 v14 follows modern PHP best practices with `final readonly` classes:
```php
// TYPO3 Core - cannot be mocked
final readonly class SiteConfigurationLoadedEvent
{
public function __construct(
private string $siteIdentifier,
private array $configuration,
) {}
}
```
Attempting to mock these classes throws:
```
PHPUnit\Framework\MockObject\Generator\ClassIsFinalException:
Class "TYPO3\CMS\Core\Configuration\Event\SiteConfigurationLoadedEvent" is declared "final" and cannot be mocked.
```
## Pattern 1: Interface Extraction for Dependencies
When your class depends on a final class that you control, extract an interface.
### Before (Untestable)
```php
// Your final class
final class SiteConfigurationVaultProcessor
{
public function processConfiguration(array $configuration): array { }
}
// Consumer - cannot mock the dependency
final readonly class SiteConfigurationVaultListener
{
public function __construct(
private SiteConfigurationVaultProcessor $processor, // Cannot mock!
) {}
}
```
### After (Testable)
**Step 1: Create Interface**
```php
<?php
declare(strict_types=1);
namespace Vendor\Extension\Configuration;
interface SiteConfigurationVaultProcessorInterface
{
/**
* @param array<string, mixed> $configuration
* @return array<string, mixed>
*/
public function processConfiguration(array $configuration): array;
}
```
**Step 2: Implement Interface**
```php
final class SiteConfigurationVaultProcessor implements SiteConfigurationVaultProcessorInterface
{
public function processConfiguration(array $configuration): array
{
// Implementation
}
}
```
**Step 3: Register in Services.yaml**
```yaml
services:
Vendor\Extension\Configuration\SiteConfigurationVaultProcessorInterface:
alias: Vendor\Extension\Configuration\SiteConfigurationVaultProcessor
public: true
```
**Step 4: Inject Interface**
```php
final readonly class SiteConfigurationVaultListener
{
public function __construct(
private SiteConfigurationVaultProcessorInterface $processor, // Mockable!
) {}
}
```
**Step 5: Mock Interface in Tests**
```php
final class SiteConfigurationVaultListenerTest extends UnitTestCase
{
private SiteConfigurationVaultProcessorInterface&MockObject $processor;
private SiteConfigurationVaultListener $listener;
protected function setUp(): void
{
parent::setUp();
$this->processor = $this->createMock(SiteConfigurationVaultProcessorInterface::class);
$this->listener = new SiteConfigurationVaultListener($this->processor);
}
#[Test]
public function processesConfigurationWithVaultReferences(): void
{
$originalConfig = ['apiKey' => '%vault(my_key)%'];
$processedConfig = ['apiKey' => 'resolved_secret'];
$this->processor
->expects($this->once())
->method('processConfiguration')
->with($originalConfig)
->willReturn($processedConfig);
// Test your listener...
}
}
```
## Pattern 2: Real Event Instances for Final Events
TYPO3 PSR-14 events are often final. Create real instances instead of mocks.
### Wrong - Will Fail
```php
#[Test]
public function handlesEvent(): void
{
// ClassIsFinalException!
$event = $this->createMock(SiteConfigurationLoadedEvent::class);
}
```
### Correct - Real Instance
```php
#[Test]
public function handlesEvent(): void
{
// Create real event - it's a simple value object
$config = ['apiKey' => '%vault(my_key)%'];
$event = new SiteConfigurationLoadedEvent('test-site', $config);
// Mock the dependency, not the event
$this->processor
->method('processConfiguration')
->willReturn(['apiKey' => 'resolved']);
($this->listener)($event);
self::assertSame(['apiKey' => 'resolved'], $event->getConfiguration());
}
```
### Complete Test Example
```php
<?php
declare(strict_types=1);
namespace Vendor\Extension\Tests\Unit\EventListener;
use PHPUnit\Framework\Attributes\CoversClass;
use PHPUnit\Framework\Attributes\Test;
use PHPUnit\Framework\MockObject\MockObject;
use PHPUnit\Framework\TestCase;
use TYPO3\CMS\Core\Configuration\Event\SiteConfigurationLoadedEvent;
use Vendor\Extension\Configuration\SiteConfigurationVaultProcessorInterface;
use Vendor\Extension\EventListener\SiteConfigurationVaultListener;
#[CoversClass(SiteConfigurationVaultListener::class)]
final class SiteConfigurationVaultListenerTest extends TestCase
{
private SiteConfigurationVaultProcessorInterface&MockObject $processor;
private SiteConfigurationVaultListener $listener;
protected function setUp(): void
{
parent::setUp();
$this->processor = $this->createMock(SiteConfigurationVaultProcessorInterface::class);
$this->listener = new SiteConfigurationVaultListener($this->processor);
}
#[Test]
public function skipsProcessingWhenNoVaultReferences(): void
{
$config = [
'base' => 'https://example.com',
'languages' => [],
];
// Real event instance - not mocked
$event = new SiteConfigurationLoadedEvent('test-site', $config);
$this->processor->expects($this->never())->method('processConfiguration');
($this->listener)($event);
self::assertSame($config, $event->getConfiguration());
}
#[Test]
public function processesConfigurationWithVaultReferences(): void
{
$originalConfig = ['apiKey' => '%vault(my_key)%'];
$processedConfig = ['apiKey' => 'resolved_secret'];
// Real event instance
$event = new SiteConfigurationLoadedEvent('test-site', $originalConfig);
$this->processor
->expects($this->once())
->method('processConfiguration')
->with($originalConfig)
->willReturn($processedConfig);
($this->listener)($event);
self::assertSame($processedConfig, $event->getConfiguration());
}
#[Test]
public function handlesEmptyConfiguration(): void
{
$config = [];
$event = new SiteConfigurationLoadedEvent('test-site', $config);
$this->processor->expects($this->never())->method('processConfiguration');
($this->listener)($event);
self::assertSame($config, $event->getConfiguration());
}
}
```
## Pattern 3: Test Suite Organization
Separate tests by bootstrap requirements to avoid skipped tests.
### Directory Structure
```
Tests/
├── Build/
│ ├── phpunit.xml # Unit + Fuzz (no TYPO3 bootstrap)
│ └── FunctionalTests.xml # Functional (requires TYPO3)
├── Unit/ # Fast, isolated, mockable
├── Functional/ # Database, framework integration
└── Fuzz/ # Property-based testing
```
### phpunit.xml (Unit Tests Only)
```xml
<?xml version="1.0" encoding="UTF-8"?>
<phpunit
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:noNamespaceSchemaLocation="https://schema.phpunit.de/12.5/phpunit.xsd"
bootstrap="../bootstrap.php"
colors="true"
failOnRisky="true"
failOnWarning="true"
>
<testsuites>
<testsuite name="Unit">
<directory>../Unit</directory>
</testsuite>
<testsuite name="Fuzz">
<directory>../Fuzz</directory>
</testsuite>
<!-- Functional tests require TYPO3 bootstrap - run separately -->
</testsuites>
</phpunit>
```
### FunctionalTests.xml
```xml
<?xml version="1.0" encoding="UTF-8"?>
<phpunit
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:noNamespaceSchemaLocation="https://schema.phpunit.de/12.5/phpunit.xsd"
bootstrap="FunctionalTestsBootstrap.php"
colors="true"
>
<testsuites>
<testsuite name="Functional">
<directory>../Functional</directory>
</testsuite>
</testsuites>
</phpunit>
```
### composer.json Scripts
```json
{
"scripts": {
"test:unit": "phpunit -c Tests/Build/phpunit.xml",
"test:functional": "phpunit -c Tests/Build/FunctionalTests.xml",
"test:all": ["@test:unit", "@test:functional"]
}
}
```
## Decision Tree: What to Test Where
```
Is the class under test final?
├── Yes → Can you create it directly (simple constructor)?
│ ├── Yes → Create real instance (Pattern 2)
│ └── No → Does it need framework services?
│ ├── Yes → Move to Functional tests
│ └── No → Extract interface for dependency (Pattern 1)
└── No → Mock normally with createMock()
```
## Common TYPO3 v14 Final Classes
| Class | Testing Strategy |
|-------|------------------|
| `SiteConfigurationLoadedEvent` | Create real instance |
| `AfterStdWrapFunctionsExecutedEvent` | Create real instance |
| `ModifyButtonBarEvent` | Create real instance |
| `FlexFormValueContainer` | Move to functional test |
| `DataHandler` (partial) | Mock via interface or functional test |
## Anti-Patterns to Avoid
### Don't Skip Tests
```php
// BAD - leaves gaps in coverage
#[Test]
public function testSomething(): void
{
$this->markTestSkipped('Cannot mock final class');
}
```
### Don't Use Reflection to Bypass Final
```php
// BAD - fragile and defeats the purpose
$reflection = new ReflectionClass(FinalClass::class);
// ... hack to make it non-final
```
### Don't Copy TYPO3 Classes
```php
// BAD - maintenance nightmare
namespace Vendor\Extension\Tests\Fixtures;
class SiteConfigurationLoadedEvent { } // Copy of TYPO3's class
```
## Summary
1. **Interface Extraction**: For dependencies you control that are final
2. **Real Instances**: For simple value objects like events
3. **Test Suite Separation**: Unit vs Functional based on requirements
4. **Zero Skipped Tests**: Every test should run - reorganize if needed
```
### references/architecture-testing.md
```markdown
# Architecture Testing with phpat
PHP Architecture Tester (phpat) enforces architectural rules through automated tests.
## Installation
```bash
composer require --dev carlosas/phpat
```
## Configuration
Create `phpat.php` in project root:
```php
<?php
declare(strict_types=1);
use PhpAT\Rule\Rule;
use PhpAT\Selector\Selector;
use PhpAT\Test\ArchitectureTest;
final class ArchitectureTests extends ArchitectureTest
{
public function testServicesDoNotDependOnControllers(): Rule
{
return $this->newRule
->classesThat(Selector::haveClassName('*Service'))
->mustNotDependOn()
->classesThat(Selector::haveClassName('*Controller'))
->build();
}
public function testDomainDoesNotDependOnInfrastructure(): Rule
{
return $this->newRule
->classesThat(Selector::havePath('Domain/*'))
->mustNotDependOn()
->classesThat(Selector::havePath('Infrastructure/*'))
->build();
}
public function testEventsAreReadonly(): Rule
{
return $this->newRule
->classesThat(Selector::havePath('Event/*'))
->mustBeReadonly()
->build();
}
}
```
## TYPO3 Extension Rules
### Layer Constraints
```php
public function testCleanArchitecture(): Rule
{
return $this->newRule
->classesThat(Selector::havePath('Classes/Domain/*'))
->mustNotDependOn()
->classesThat(Selector::havePath('Classes/Controller/*'))
->andClassesThat(Selector::havePath('Classes/Command/*'))
->build();
}
```
### Service Layer Rules
```php
public function testServicesHaveInterface(): Rule
{
return $this->newRule
->classesThat(Selector::haveClassName('*Service'))
->excludingClassesThat(Selector::haveClassName('*Interface'))
->mustImplement()
->classesThat(Selector::haveClassName('*Interface'))
->build();
}
```
## Running Tests
```bash
# Via PHPUnit
vendor/bin/phpunit --testsuite Architecture
# Via runTests.sh
Build/Scripts/runTests.sh -s architecture
```
## PHPUnit Configuration
Add to `phpunit.xml`:
```xml
<testsuite name="Architecture">
<file>phpat.php</file>
</testsuite>
```
## Common Rules
| Rule | Purpose |
|------|---------|
| `mustNotDependOn` | Prevent unwanted dependencies |
| `mustImplement` | Enforce interface usage |
| `mustBeReadonly` | Enforce immutability (PHP 8.2+) |
| `mustBeFinal` | Prevent inheritance |
| `mustNotConstruct` | Enforce DI |
## Security-Critical Extensions
For security-critical code, enforce:
1. Events are readonly
2. Services don't construct other services (use DI)
3. Domain layer is isolated
4. No circular dependencies
```
### references/e2e-testing.md
```markdown
# E2E Testing with Playwright
TYPO3 Core uses **Playwright** exclusively for end-to-end and accessibility testing. This is the modern standard for browser-based testing in TYPO3 extensions.
**Reference:** [TYPO3 Core Build/tests/playwright](https://github.com/TYPO3/typo3/tree/main/Build/tests/playwright)
## When to Use E2E Tests
- Testing complete user journeys (login, browse, action)
- Frontend functionality validation
- Backend module interaction testing
- JavaScript-heavy interactions
- Visual regression testing
- Cross-browser compatibility
## Requirements
```json
// package.json
{
"engines": {
"node": ">=22.18.0 <23.0.0",
"npm": ">=11.5.2"
},
"devDependencies": {
"@playwright/test": "^1.56.1",
"@axe-core/playwright": "^4.9.0"
},
"scripts": {
"playwright:install": "playwright install",
"playwright:open": "playwright test --ui --ignore-https-errors",
"playwright:run": "playwright test",
"playwright:codegen": "playwright codegen",
"playwright:report": "playwright show-report"
}
}
```
## Directory Structure
```
Build/
├── playwright.config.ts # Main Playwright configuration
├── package.json # Node dependencies
├── .nvmrc # Node version (22.18)
└── tests/
└── playwright/
├── config.ts # TYPO3-specific config (baseUrl, credentials)
├── e2e/ # End-to-end tests
│ ├── backend/
│ │ └── module.spec.ts
│ └── frontend/
│ └── pages.spec.ts
├── accessibility/ # Accessibility tests (axe-core)
│ └── modules.spec.ts
├── fixtures/ # Page Object Models
│ ├── setup-fixtures.ts
│ └── backend-page.ts
└── helper/
└── login.setup.ts # Authentication setup
```
## Configuration
### Playwright Config
```typescript
// Build/playwright.config.ts
import { defineConfig } from '@playwright/test';
import config from './tests/playwright/config';
export default defineConfig({
testDir: './tests/playwright',
timeout: 30000,
expect: {
timeout: 10000,
},
fullyParallel: false,
forbidOnly: !!process.env.CI,
retries: process.env.CI ? 2 : 0,
workers: process.env.CI ? 1 : undefined,
reporter: [
['list'],
['html', { outputFolder: '../typo3temp/var/tests/playwright-reports' }],
],
outputDir: '../typo3temp/var/tests/playwright-results',
use: {
baseURL: config.baseUrl,
ignoreHTTPSErrors: true,
trace: 'on-first-retry',
},
projects: [
{
name: 'login setup',
testMatch: /helper\/login\.setup\.ts/,
},
{
name: 'accessibility',
testMatch: /accessibility\/.*\.spec\.ts/,
dependencies: ['login setup'],
use: {
storageState: './.auth/login.json',
},
},
{
name: 'e2e',
testMatch: /e2e\/.*\.spec\.ts/,
dependencies: ['login setup'],
use: {
storageState: './.auth/login.json',
},
},
],
});
```
### TYPO3-Specific Config
```typescript
// Build/tests/playwright/config.ts
export default {
baseUrl: process.env.PLAYWRIGHT_BASE_URL ?? 'http://web:80/typo3/',
admin: {
username: process.env.PLAYWRIGHT_ADMIN_USERNAME ?? 'admin',
password: process.env.PLAYWRIGHT_ADMIN_PASSWORD ?? 'password',
},
};
```
## Authentication Setup
Store authentication state to avoid repeated logins:
```typescript
// Build/tests/playwright/helper/login.setup.ts
import { test as setup, expect } from '@playwright/test';
import config from '../config';
setup('login', async ({ page }) => {
await page.goto('/');
await page.getByLabel('Username').fill(config.admin.username);
await page.getByLabel('Password').fill(config.admin.password);
await page.getByRole('button', { name: 'Login' }).click();
await page.waitForLoadState('networkidle');
// Verify login succeeded
await expect(page.locator('.t3js-topbar-button-modulemenu')).toBeVisible();
// Save authentication state
await page.context().storageState({ path: './.auth/login.json' });
});
```
## Page Object Model (Fixtures)
Create reusable page objects for TYPO3 backend:
```typescript
// Build/tests/playwright/fixtures/setup-fixtures.ts
import { test as base, type Locator, type Page, expect } from '@playwright/test';
export class BackendPage {
readonly page: Page;
readonly moduleMenu: Locator;
readonly contentFrame: ReturnType<Page['frameLocator']>;
constructor(page: Page) {
this.page = page;
this.moduleMenu = page.locator('#modulemenu');
this.contentFrame = page.frameLocator('#typo3-contentIframe');
}
async gotoModule(identifier: string): Promise<void> {
const moduleLink = this.moduleMenu.locator(
`[data-modulemenu-identifier="${identifier}"]`
);
await moduleLink.click();
await expect(moduleLink).toHaveClass(/modulemenu-action-active/);
}
async moduleLoaded(): Promise<void> {
await this.page.evaluate(() => {
return new Promise<void>((resolve) => {
document.addEventListener('typo3-module-loaded', () => resolve(), {
once: true,
});
});
});
}
async waitForModuleResponse(urlPattern: string | RegExp): Promise<void> {
await this.page.waitForResponse((response) => {
const url = response.url();
const matches =
typeof urlPattern === 'string'
? url.includes(urlPattern)
: urlPattern.test(url);
return matches && response.status() === 200;
});
}
}
export class Modal {
readonly page: Page;
readonly container: Locator;
readonly title: Locator;
readonly closeButton: Locator;
constructor(page: Page) {
this.page = page;
this.container = page.locator('.modal');
this.title = this.container.locator('.modal-title');
this.closeButton = this.container.locator('[data-bs-dismiss="modal"]');
}
async close(): Promise<void> {
await this.closeButton.click();
await expect(this.container).not.toBeVisible();
}
}
type BackendFixtures = {
backend: BackendPage;
modal: Modal;
};
export const test = base.extend<BackendFixtures>({
backend: async ({ page }, use) => {
await use(new BackendPage(page));
},
modal: async ({ page }, use) => {
await use(new Modal(page));
},
});
export { expect, Locator };
```
## Writing E2E Tests
### Basic Test Structure
```typescript
// Build/tests/playwright/e2e/backend/module.spec.ts
import { test, expect } from '../../fixtures/setup-fixtures';
test.describe('My Extension Backend Module', () => {
test('can access module', async ({ backend }) => {
await backend.gotoModule('web_myextension');
await backend.moduleLoaded();
const contentFrame = backend.contentFrame;
await expect(contentFrame.locator('h1')).toBeVisible();
});
test('can perform action in module', async ({ backend, modal }) => {
await backend.gotoModule('web_myextension');
await backend.contentFrame
.getByRole('button', { name: 'Create new record' })
.click();
await expect(modal.container).toBeVisible();
await expect(modal.title).toContainText('Create');
await modal.close();
});
test('can save form data', async ({ backend }) => {
await backend.gotoModule('web_myextension');
const contentFrame = backend.contentFrame;
await contentFrame.getByLabel('Title').fill('Test Title');
await contentFrame.getByLabel('Description').fill('Test Description');
await contentFrame.getByRole('button', { name: 'Save' }).click();
await backend.waitForModuleResponse(/module\/web\/myextension/);
await expect(contentFrame.locator('.alert-success')).toBeVisible();
});
});
```
### Common Actions
```typescript
// Navigation
await page.goto('/module/web/layout');
await page.goBack();
// Form interaction
await page.getByLabel('Title').fill('Value');
await page.getByRole('button', { name: 'Save' }).click();
await page.getByRole('combobox').selectOption('option-value');
await page.getByRole('checkbox').check();
// Assertions
await expect(page.locator('.success')).toBeVisible();
await expect(page.locator('h1')).toContainText('Title');
await expect(page).toHaveURL(/module\/web\/layout/);
// Waiting
await page.waitForLoadState('networkidle');
await page.waitForSelector('.loaded');
await page.waitForResponse(/api\/endpoint/);
```
## Running Tests
```bash
# Install Playwright browsers
npm run playwright:install
# Run all tests
npm run playwright:run
# Run with UI mode (interactive)
npm run playwright:open
# Run specific test file
npx playwright test e2e/backend/module.spec.ts
# Run tests matching pattern
npx playwright test --grep "can access"
# Generate test code (record & playback)
npm run playwright:codegen
# Run in headed mode (see browser)
npx playwright test --headed
# Debug mode
npx playwright test --debug
# Generate HTML report
npm run playwright:report
```
## DDEV Integration
```yaml
# .ddev/docker-compose.playwright.yaml
services:
playwright:
container_name: ddev-${DDEV_SITENAME}-playwright
image: mcr.microsoft.com/playwright:v1.56.1-noble
volumes:
- ../:/var/www/html
working_dir: /var/www/html/Build
environment:
- PLAYWRIGHT_BASE_URL=http://web:80/typo3/
depends_on:
- web
```
```bash
# Run Playwright in DDEV
ddev exec -s playwright npx playwright test
```
## CI/CD Integration
```yaml
# .github/workflows/playwright.yml
name: Playwright Tests
on: [push, pull_request]
jobs:
playwright:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: '22'
- name: Install dependencies
working-directory: ./Build
run: npm ci
- name: Install Playwright browsers
working-directory: ./Build
run: npx playwright install --with-deps chromium
- name: Start TYPO3
run: |
ddev start
ddev import-db --file=.ddev/db.sql.gz
- name: Run Playwright tests
working-directory: ./Build
run: npx playwright test
env:
PLAYWRIGHT_BASE_URL: https://myproject.ddev.site/typo3/
- uses: actions/upload-artifact@v4
if: always()
with:
name: playwright-report
path: typo3temp/var/tests/playwright-reports/
retention-days: 30
```
## Best Practices
**Do:**
- Use Page Object Model (fixtures) for reusability
- Store authentication state to avoid repeated logins
- Test user-visible behavior, not implementation details
- Use descriptive test names that explain the scenario
- Wait for specific elements, not arbitrary timeouts
- Use `data-testid` attributes for stable selectors
- Run tests in CI with proper environment setup
**Don't:**
- Use `page.waitForTimeout()` - use specific waits instead
- Depend on CSS classes that may change
- Test internal TYPO3 Core behavior
- Ignore flaky tests - fix the root cause
- Use hard-coded credentials in code (use env vars)
## Naming Conventions
- Pattern: `<feature>.spec.ts`
- Examples: `page-module.spec.ts`, `login.spec.ts`
- Location: `Build/tests/playwright/e2e/<category>/`
## Common Pitfalls
**No Waits for Dynamic Content**
```typescript
// Wrong
await page.click('Load More');
await expect(page.locator('.item')).toBeVisible(); // May fail
// Right
await page.click('Load More');
await page.waitForSelector('.item:nth-child(11)');
await expect(page.locator('.item')).toBeVisible();
```
**Brittle Selectors**
```typescript
// Wrong - fragile CSS path
await page.click('div.container > div:nth-child(3) > button');
// Right - stable selector
await page.click('[data-testid="add-to-cart"]');
await page.click('#product-add-button');
```
## E2E Testing for AJAX Endpoints
Backend modules often use AJAX routes for dynamic functionality. Test these endpoints thoroughly:
### Intercepting AJAX Requests
```typescript
// Build/tests/playwright/e2e/backend/ajax-module.spec.ts
import { test, expect } from '../../fixtures/setup-fixtures';
test.describe('AJAX Endpoint Testing', () => {
test('validates form via AJAX', async ({ page, backend }) => {
await backend.gotoModule('web_myextension_wizard');
// Intercept the AJAX validation request
const validationPromise = page.waitForResponse(
(response) =>
response.url().includes('/ajax/myext/wizard/validate') &&
response.status() === 200
);
// Fill form and trigger validation
await backend.contentFrame.getByLabel('Provider Name').fill('My Provider');
await backend.contentFrame.getByLabel('API Key').fill('sk-test-123');
await backend.contentFrame.getByRole('button', { name: 'Next' }).click();
// Verify AJAX response
const response = await validationPromise;
const json = await response.json();
expect(json.success).toBe(true);
expect(json.errors).toEqual({});
});
test('handles validation errors from AJAX', async ({ page, backend }) => {
await backend.gotoModule('web_myextension_wizard');
// Submit without required fields
await backend.contentFrame.getByRole('button', { name: 'Next' }).click();
// Wait for error response
const response = await page.waitForResponse(
(r) => r.url().includes('/ajax/myext/wizard/validate')
);
const json = await response.json();
expect(json.success).toBe(false);
expect(json.errors).toHaveProperty('name');
// Verify error is displayed in UI
await expect(
backend.contentFrame.locator('.invalid-feedback')
).toBeVisible();
});
});
```
### Testing Connection/Test Buttons
```typescript
test('tests API connection via AJAX', async ({ page, backend }) => {
await backend.gotoModule('web_myextension_wizard');
// Fill connection details
await backend.contentFrame.getByLabel('API Key').fill('sk-test-123');
// Mock successful connection response
await page.route('**/ajax/myext/wizard/test-connection', async (route) => {
await route.fulfill({
status: 200,
contentType: 'application/json',
body: JSON.stringify({
success: true,
message: 'Connection successful',
models: [
{ id: 'gpt-4o', name: 'GPT-4o' },
{ id: 'gpt-4o-mini', name: 'GPT-4o Mini' },
],
}),
});
});
// Click test button
const testButton = backend.contentFrame.getByRole('button', {
name: 'Test Connection',
});
await testButton.click();
// Verify success notification
await expect(page.locator('.alert-success')).toBeVisible();
// Verify models were populated
const modelSelect = backend.contentFrame.getByLabel('Model');
await expect(modelSelect.locator('option')).toHaveCount(3); // including empty option
});
test('handles connection failure gracefully', async ({ page, backend }) => {
await backend.gotoModule('web_myextension_wizard');
await backend.contentFrame.getByLabel('API Key').fill('invalid-key');
// Mock failed connection
await page.route('**/ajax/myext/wizard/test-connection', async (route) => {
await route.fulfill({
status: 400,
contentType: 'application/json',
body: JSON.stringify({
success: false,
message: 'Invalid API key',
}),
});
});
await backend.contentFrame
.getByRole('button', { name: 'Test Connection' })
.click();
// Verify error notification
await expect(page.locator('.alert-danger')).toBeVisible();
await expect(page.locator('.alert-danger')).toContainText('Invalid API key');
});
```
### Testing Multi-Step Wizards
```typescript
test('completes multi-step wizard', async ({ page, backend }) => {
await backend.gotoModule('web_myextension_wizard');
// Step 1: Provider
await backend.contentFrame.getByLabel('Provider Name').fill('OpenAI Prod');
await backend.contentFrame.getByLabel('API Key').fill('sk-test-key');
await backend.contentFrame
.getByRole('button', { name: 'Test Connection' })
.click();
// Wait for test to complete
await page.waitForResponse((r) =>
r.url().includes('/ajax/myext/wizard/test-connection')
);
await backend.contentFrame.getByRole('button', { name: 'Next' }).click();
// Step 2: Model (verify we advanced)
await expect(backend.contentFrame.locator('h2')).toContainText('Step 2');
await backend.contentFrame.getByLabel('Model').selectOption('gpt-4o');
await backend.contentFrame.getByRole('button', { name: 'Next' }).click();
// Step 3: Configuration
await expect(backend.contentFrame.locator('h2')).toContainText('Step 3');
await backend.contentFrame.getByLabel('Temperature').fill('0.7');
await backend.contentFrame.getByRole('button', { name: 'Finish' }).click();
// Verify completion
const saveResponse = await page.waitForResponse(
(r) =>
r.url().includes('/ajax/myext/wizard/save') && r.status() === 200
);
const result = await saveResponse.json();
expect(result.success).toBe(true);
// Verify redirect or success message
await expect(backend.contentFrame.locator('.wizard-complete')).toBeVisible();
});
```
### Testing Toggle Actions
```typescript
test('toggles record active state via AJAX', async ({ page, backend }) => {
await backend.gotoModule('web_myextension');
// Wait for list to load
await expect(backend.contentFrame.locator('table tbody tr')).toHaveCount(3);
// Click toggle button
const toggleButton = backend.contentFrame
.locator('tr')
.first()
.getByRole('button', { name: 'Toggle' });
await toggleButton.click();
// Verify AJAX call succeeded
const response = await page.waitForResponse(
(r) =>
r.url().includes('/ajax/myext/toggle') && r.status() === 200
);
const json = await response.json();
expect(json.success).toBe(true);
// Verify UI updated
await expect(toggleButton).toHaveAttribute('data-active', 'false');
});
```
### Network Request Assertions
```typescript
test('sends correct request payload', async ({ page, backend }) => {
await backend.gotoModule('web_myextension_wizard');
// Capture the request
const requestPromise = page.waitForRequest(
(r) => r.url().includes('/ajax/myext/wizard/validate')
);
await backend.contentFrame.getByLabel('Name').fill('Test Provider');
await backend.contentFrame.getByLabel('Type').selectOption('openai');
await backend.contentFrame.getByRole('button', { name: 'Validate' }).click();
const request = await requestPromise;
const postData = request.postDataJSON();
expect(postData).toEqual({
step: 'provider',
data: {
name: 'Test Provider',
type: 'openai',
},
});
});
```
### AJAX Timeout and Error Handling
```typescript
test('handles AJAX timeout gracefully', async ({ page, backend }) => {
await backend.gotoModule('web_myextension_wizard');
// Simulate slow/timeout response
await page.route('**/ajax/myext/wizard/test-connection', async (route) => {
await new Promise((resolve) => setTimeout(resolve, 35000)); // Exceed timeout
await route.abort('timedout');
});
await backend.contentFrame.getByLabel('API Key').fill('sk-test');
await backend.contentFrame
.getByRole('button', { name: 'Test Connection' })
.click();
// Verify timeout error displayed
await expect(page.locator('.alert-warning')).toBeVisible({ timeout: 40000 });
await expect(page.locator('.alert-warning')).toContainText('timed out');
});
```
## Resources
- [Playwright Documentation](https://playwright.dev/docs/intro)
- [TYPO3 Core Playwright Tests](https://github.com/TYPO3/typo3/tree/main/Build/tests/playwright)
- [Playwright Test API](https://playwright.dev/docs/api/class-test)
- [Page Object Model](https://playwright.dev/docs/pom)
- [Playwright Network Mocking](https://playwright.dev/docs/mock)
```
### references/accessibility-testing.md
```markdown
# Accessibility Testing with axe-core
TYPO3 extensions should test for WCAG 2.0/2.1 compliance at levels A and AA using **axe-core** integrated with Playwright.
**Reference:** [axe-core Documentation](https://www.deque.com/axe/)
## Requirements
```json
// package.json
{
"devDependencies": {
"@playwright/test": "^1.56.1",
"@axe-core/playwright": "^4.9.0"
}
}
```
## Directory Structure
```
Build/
└── tests/
└── playwright/
└── accessibility/
├── modules.spec.ts # Backend module accessibility
├── forms.spec.ts # Form accessibility
└── navigation.spec.ts # Navigation accessibility
```
## Basic Accessibility Test
```typescript
// Build/tests/playwright/accessibility/modules.spec.ts
import { test, expect } from '@playwright/test';
import AxeBuilder from '@axe-core/playwright';
const modules = [
{ name: 'My Extension Module', route: 'module/web/myextension' },
{ name: 'Settings', route: 'module/web/myextension/settings' },
];
for (const module of modules) {
test(`${module.name} has no accessibility violations`, async ({ page }) => {
await page.goto(module.route);
await page.waitForLoadState('networkidle');
const accessibilityScanResults = await new AxeBuilder({ page })
.include('#typo3-contentIframe')
.disableRules(['color-contrast']) // Reduce false positives
.analyze();
expect(accessibilityScanResults.violations).toEqual([]);
});
}
```
## Comprehensive Accessibility Tests
```typescript
// Build/tests/playwright/accessibility/comprehensive.spec.ts
import { test, expect } from '@playwright/test';
import AxeBuilder from '@axe-core/playwright';
test.describe('Accessibility - Comprehensive Checks', () => {
test('module menu has proper ARIA attributes', async ({ page }) => {
await page.goto('/');
await page.waitForLoadState('networkidle');
const moduleMenu = page.locator('#modulemenu');
await expect(moduleMenu).toHaveAttribute('role', 'navigation');
});
test('interactive elements are keyboard accessible', async ({ page }) => {
await page.goto('module/web/myextension');
await page.waitForLoadState('networkidle');
const contentFrame = page.frameLocator('#typo3-contentIframe');
// Tab through interactive elements
await page.keyboard.press('Tab');
// Verify focus is visible
const focusedElement = contentFrame.locator(':focus');
await expect(focusedElement).toBeVisible();
});
test('forms have proper labels', async ({ page }) => {
await page.goto('module/web/myextension/edit');
await page.waitForLoadState('networkidle');
const contentFrame = page.frameLocator('#typo3-contentIframe');
// All inputs should have associated labels
const inputs = contentFrame.locator('input:not([type="hidden"])');
const count = await inputs.count();
for (let i = 0; i < count; i++) {
const input = inputs.nth(i);
const id = await input.getAttribute('id');
if (id) {
const label = contentFrame.locator(`label[for="${id}"]`);
await expect(label).toBeVisible();
}
}
});
test('images have alt text', async ({ page }) => {
await page.goto('module/web/myextension');
await page.waitForLoadState('networkidle');
const contentFrame = page.frameLocator('#typo3-contentIframe');
const images = contentFrame.locator('img');
const count = await images.count();
for (let i = 0; i < count; i++) {
const img = images.nth(i);
const alt = await img.getAttribute('alt');
expect(alt).not.toBeNull();
}
});
test('color contrast is sufficient', async ({ page }) => {
await page.goto('module/web/myextension');
await page.waitForLoadState('networkidle');
const accessibilityScanResults = await new AxeBuilder({ page })
.include('#typo3-contentIframe')
.withRules(['color-contrast'])
.analyze();
// Log violations for debugging but don't fail
// (TYPO3 backend may have known contrast issues)
if (accessibilityScanResults.violations.length > 0) {
console.log('Color contrast issues:', accessibilityScanResults.violations);
}
});
});
```
## axe-core Configuration
### Include/Exclude Elements
```typescript
const results = await new AxeBuilder({ page })
.include('#main-content') // Only scan this element
.exclude('.third-party-widget') // Skip this element
.analyze();
```
### Specific Rules
```typescript
// Run only specific rules
const results = await new AxeBuilder({ page })
.withRules(['color-contrast', 'label'])
.analyze();
// Disable specific rules
const results = await new AxeBuilder({ page })
.disableRules(['color-contrast'])
.analyze();
```
### Tags (WCAG Levels)
```typescript
// Test WCAG 2.1 Level AA
const results = await new AxeBuilder({ page })
.withTags(['wcag2a', 'wcag2aa', 'wcag21aa'])
.analyze();
// Test only critical issues
const results = await new AxeBuilder({ page })
.withTags(['critical'])
.analyze();
```
## Handling Violations
```typescript
test('handles violations gracefully', async ({ page }) => {
await page.goto('module/web/myextension');
const results = await new AxeBuilder({ page })
.include('#typo3-contentIframe')
.analyze();
// Log violations with details
for (const violation of results.violations) {
console.log(`Rule: ${violation.id}`);
console.log(`Impact: ${violation.impact}`);
console.log(`Description: ${violation.description}`);
for (const node of violation.nodes) {
console.log(` Element: ${node.html}`);
console.log(` Fix: ${node.failureSummary}`);
}
}
// Assert no violations
expect(results.violations).toHaveLength(0);
});
```
## TYPO3 Backend Considerations
### Known TYPO3 Backend Issues
Some accessibility rules may produce false positives in TYPO3 backend:
```typescript
const results = await new AxeBuilder({ page })
.include('#typo3-contentIframe')
// Disable rules that conflict with TYPO3 backend design
.disableRules([
'color-contrast', // TYPO3 uses theme colors
'landmark-one-main', // Backend uses iframe structure
'region', // Content in iframes
])
.analyze();
```
### Testing Your Extension Only
Focus on elements your extension controls:
```typescript
const results = await new AxeBuilder({ page })
// Target your extension's content
.include('[data-extension="my_extension"]')
.analyze();
```
## Best Practices
**Do:**
- Test all backend modules your extension provides
- Test forms for proper labels and ARIA attributes
- Test keyboard navigation through interactive elements
- Test with screen reader users in mind
- Document known accessibility limitations
**Don't:**
- Disable all rules to make tests pass
- Skip accessibility testing entirely
- Assume TYPO3 backend handles all accessibility
- Ignore violations without documenting reason
## Checklist
- [ ] All modules tested with axe-core
- [ ] Forms have proper labels
- [ ] Interactive elements are keyboard accessible
- [ ] Images have alt text
- [ ] ARIA attributes are correct
- [ ] Focus states are visible
- [ ] Color is not the only means of conveying information
## Resources
- [axe-core Playwright Integration](https://github.com/dequelabs/axe-core-npm/tree/develop/packages/playwright)
- [WCAG 2.1 Guidelines](https://www.w3.org/WAI/WCAG21/quickref/)
- [axe-core Rules](https://dequeuniversity.com/rules/axe/)
- [TYPO3 Accessibility Guidelines](https://docs.typo3.org/m/typo3/reference-coreapi/main/en-us/Accessibility/)
```
### references/fuzz-testing.md
```markdown
# Fuzz Testing for TYPO3 Extensions
## Overview
Fuzz testing (fuzzing) automatically generates random/mutated inputs to find crashes, memory exhaustion, or unexpected exceptions. This is critical for code that parses untrusted input like HTML, XML, or user data.
> **Key Distinction**: Fuzz testing mutates **inputs** to find bugs. For testing that mutates **code** to verify test quality, see [Mutation Testing](mutation-testing.md).
## When to Use Fuzz Testing
- HTML/XML parsers (e.g., DOMDocument-based code)
- User input processors
- Data transformation services
- File format parsers
- Any code handling untrusted external data
## Tools
### nikic/php-fuzzer (Recommended)
Coverage-guided fuzzer for PHP library/parser testing.
**Installation:**
```bash
composer require --dev nikic/php-fuzzer:^0.0.11
```
**Key features:**
- Coverage-guided mutation (finds new code paths)
- Corpus management (saves interesting inputs)
- Crash detection and reproduction
- Memory limit enforcement
## Creating Fuzz Targets
### Basic Structure
Create fuzz targets in `Tests/Fuzz/` directory:
```php
<?php
declare(strict_types=1);
use MyVendor\MyExtension\Service\MyParser;
require_once dirname(__DIR__, 2) . '/.Build/vendor/autoload.php';
/** @var PhpFuzzer\Config $config */
$parser = new MyParser();
$config->setTarget(function (string $input) use ($parser): void {
// Call the method being fuzzed
$parser->parse($input);
});
// Limit input length to prevent memory exhaustion
$config->setMaxLen(65536);
```
### TYPO3 Extension Example
For a TYPO3 extension with HTML parsing (like an RTE image handler):
```php
<?php
declare(strict_types=1);
/**
* Fuzzing target for ImageAttributeParser.
*
* Tests parseImageAttributes() with random/mutated HTML inputs
* to find crashes, memory exhaustion, or unexpected exceptions.
*/
use MyVendor\MyExtension\Service\ImageAttributeParser;
require_once dirname(__DIR__, 2) . '/.Build/vendor/autoload.php';
/** @var PhpFuzzer\Config $config */
$parser = new ImageAttributeParser();
$config->setTarget(function (string $input) use ($parser): void {
// Test primary parsing method
$parser->parseImageAttributes($input);
// Test related methods with same input
$parser->parseLinkWithImages($input);
});
$config->setMaxLen(65536);
```
### Testing Classes with Dependencies
For classes requiring TYPO3 dependencies:
```php
<?php
declare(strict_types=1);
use MyVendor\MyExtension\DataHandling\SoftReference\MySoftReferenceParser;
use TYPO3\CMS\Core\Html\HtmlParser;
require_once dirname(__DIR__, 2) . '/.Build/vendor/autoload.php';
/** @var PhpFuzzer\Config $config */
// Create dependencies
$htmlParser = new HtmlParser();
$parser = new MySoftReferenceParser($htmlParser);
// Set required properties via reflection if needed
$reflection = new ReflectionClass($parser);
$parserKeyProp = $reflection->getProperty('parserKey');
$parserKeyProp->setValue($parser, 'my_parser_key');
$config->setTarget(function (string $input) use ($parser): void {
$parser->parse(
'tt_content',
'bodytext',
1,
$input,
);
});
$config->setMaxLen(65536);
```
## Seed Corpus
Create seed inputs that the fuzzer uses as starting points in `Tests/Fuzz/corpus/`:
```
Tests/Fuzz/
├── ImageAttributeParserTarget.php
├── SoftReferenceParserTarget.php
├── corpus/
│ ├── image-parser/
│ │ ├── basic-img.txt # <img src="test.jpg" alt="Test" />
│ │ ├── fal-reference.txt # <img data-htmlarea-file-uid="123" />
│ │ ├── nested-structure.txt # <a href="#"><img src="x.jpg" /></a>
│ │ └── malformed.txt # <img src="test
│ └── softref-parser/
│ ├── basic-content.txt
│ └── multiple-images.txt
└── README.md
```
### Good Seed Inputs
Include variety in seeds:
- Valid minimal inputs
- Valid complex inputs
- Edge cases (empty, very long)
- Malformed inputs
- Special characters and encoding edge cases
## Running Fuzz Tests
### Via Composer Scripts
```json
{
"scripts": {
"ci:fuzz:image-parser": [
".Build/bin/php-fuzzer fuzz Tests/Fuzz/ImageAttributeParserTarget.php Tests/Fuzz/corpus/image-parser --max-runs 10000"
],
"ci:fuzz:softref-parser": [
".Build/bin/php-fuzzer fuzz Tests/Fuzz/SoftReferenceParserTarget.php Tests/Fuzz/corpus/softref-parser --max-runs 10000"
],
"ci:fuzz": [
"@ci:fuzz:image-parser",
"@ci:fuzz:softref-parser"
]
}
}
```
### Via runTests.sh
```bash
# Add to Build/Scripts/runTests.sh
fuzz)
FUZZ_TARGET="${1:-Tests/Fuzz/ImageAttributeParserTarget.php}"
FUZZ_CORPUS="Tests/Fuzz/corpus/image-parser"
FUZZ_MAX_RUNS="${2:-10000}"
if [[ "${FUZZ_TARGET}" == *"SoftReference"* ]]; then
FUZZ_CORPUS="Tests/Fuzz/corpus/softref-parser"
fi
COMMAND=(.Build/bin/php-fuzzer fuzz "${FUZZ_TARGET}" "${FUZZ_CORPUS}" --max-runs "${FUZZ_MAX_RUNS}")
${CONTAINER_BIN} run ${CONTAINER_COMMON_PARAMS} --name fuzz-${SUFFIX} ${IMAGE_PHP} "${COMMAND[@]}"
SUITE_EXIT_CODE=$?
;;
```
Usage:
```bash
# Default target
Build/Scripts/runTests.sh -s fuzz
# Specific target
Build/Scripts/runTests.sh -s fuzz Tests/Fuzz/ImageAttributeParserTarget.php
# Custom max-runs
Build/Scripts/runTests.sh -s fuzz Tests/Fuzz/ImageAttributeParserTarget.php 50000
```
### Directly
```bash
# Run fuzzer with corpus directory (positional argument, not --corpus option!)
.Build/bin/php-fuzzer fuzz Tests/Fuzz/ImageAttributeParserTarget.php \
Tests/Fuzz/corpus/image-parser \
--max-runs 10000
```
## Interpreting Results
### Normal Output
```
Running fuzz test...
NEW: 0xabc123 - Found new coverage path
REDUCE: 0xdef456 - Simplified input while maintaining coverage
...
Fuzzing complete. 10000 runs, 0 crashes.
```
### Crash Found
```
CRASH: Tests/Fuzz/crashes/crash-abc123.txt
Error: Call to undefined method...
To reproduce:
php Tests/Fuzz/ImageAttributeParserTarget.php < Tests/Fuzz/crashes/crash-abc123.txt
```
### What to Look For
| Result | Meaning | Action |
|--------|---------|--------|
| NEW | Found input triggering new code path | Good - corpus expanding |
| REDUCE | Simplified input while keeping coverage | Good - efficient corpus |
| CRASH | Input caused exception/error | **Fix the bug** |
| TIMEOUT | Input caused infinite loop/hang | **Fix the performance issue** |
| OOM | Input caused memory exhaustion | **Fix memory handling** |
## CI Integration
Fuzz testing is typically **not run in CI** due to time requirements. Instead:
1. Run locally before releases
2. Run on schedule (weekly) for security-critical code
3. Run in dedicated security testing pipelines
### Optional CI Integration (Short Runs)
```yaml
# .github/workflows/fuzz.yml
name: Fuzz Testing
on:
schedule:
- cron: '0 0 * * 0' # Weekly on Sunday
workflow_dispatch: # Manual trigger
jobs:
fuzz:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: shivammathur/setup-php@v2
with:
php-version: '8.2'
- run: composer install
- name: Fuzz ImageAttributeParser
run: composer ci:fuzz:image-parser
continue-on-error: true
- name: Upload crash artifacts
if: failure()
uses: actions/upload-artifact@v4
with:
name: fuzz-crashes
path: Tests/Fuzz/crashes/
```
## Best Practices
1. **Target security-critical code** - Prioritize parsers handling untrusted input
2. **Use meaningful seeds** - Good starting corpus improves coverage
3. **Set memory limits** - Prevent runaway memory usage with `setMaxLen()`
4. **Fix crashes immediately** - Fuzzer-found bugs are often exploitable
5. **Don't ignore OOM** - Memory exhaustion can be a DoS vector
6. **Document findings** - Track what was fuzzed and any issues found
## Directory Structure
```
Tests/
├── Unit/
├── Functional/
├── E2E/
└── Fuzz/
├── README.md
├── ImageAttributeParserTarget.php
├── SoftReferenceParserTarget.php
├── corpus/
│ ├── image-parser/
│ │ ├── seed1.txt
│ │ └── seed2.txt
│ └── softref-parser/
│ └── seed1.txt
└── crashes/ # Auto-generated when crashes found
└── crash-xxx.txt
```
## Resources
- [nikic/php-fuzzer](https://github.com/nikic/PHP-Fuzzer) - PHP coverage-guided fuzzer
- [Google OSS-Fuzz](https://google.github.io/oss-fuzz/) - Continuous fuzzing infrastructure
- [OWASP Fuzzing](https://owasp.org/www-community/Fuzzing) - Security fuzzing concepts
```
### references/crypto-testing.md
```markdown
# Cryptographic Testing Patterns
Testing cryptographic code requires specific patterns to ensure security while maintaining testability.
## When to Apply
- Secrets management extensions
- Envelope encryption implementations
- Key derivation functions
- Token/credential storage
- Memory-safe secret handling
## Unit Testing Cryptographic Services
### Testing Encryption Services
```php
<?php
declare(strict_types=1);
namespace Vendor\Extension\Tests\Unit\Service;
use PHPUnit\Framework\Attributes\Test;
use TYPO3\TestingFramework\Core\Unit\UnitTestCase;
use Vendor\Extension\Service\EncryptionService;
final class EncryptionServiceTest extends UnitTestCase
{
private EncryptionService $subject;
private string $testKey;
protected function setUp(): void
{
parent::setUp();
// Use deterministic test key - NEVER use production keys
$this->testKey = sodium_crypto_secretbox_keygen();
$this->subject = new EncryptionService($this->testKey);
}
protected function tearDown(): void
{
// Clear sensitive test data from memory
sodium_memzero($this->testKey);
parent::tearDown();
}
#[Test]
public function encryptAndDecryptRoundTrip(): void
{
$plaintext = 'sensitive-api-key-12345';
$encrypted = $this->subject->encrypt($plaintext);
$decrypted = $this->subject->decrypt($encrypted);
self::assertSame($plaintext, $decrypted);
self::assertNotSame($plaintext, $encrypted);
}
#[Test]
public function encryptProducesDifferentCiphertextForSamePlaintext(): void
{
$plaintext = 'secret-value';
$encrypted1 = $this->subject->encrypt($plaintext);
$encrypted2 = $this->subject->encrypt($plaintext);
// Random nonce ensures different ciphertext each time
self::assertNotSame($encrypted1, $encrypted2);
}
#[Test]
public function decryptWithWrongKeyThrowsException(): void
{
$encrypted = $this->subject->encrypt('secret');
$wrongKey = sodium_crypto_secretbox_keygen();
$wrongService = new EncryptionService($wrongKey);
$this->expectException(\RuntimeException::class);
$this->expectExceptionMessage('Decryption failed');
$wrongService->decrypt($encrypted);
sodium_memzero($wrongKey);
}
}
```
### Testing Envelope Encryption (DEK + KEK Pattern)
Envelope encryption uses a Data Encryption Key (DEK) encrypted by a Key Encryption Key (KEK):
```php
<?php
declare(strict_types=1);
namespace Vendor\Extension\Tests\Unit\Service;
use PHPUnit\Framework\Attributes\Test;
use PHPUnit\Framework\MockObject\MockObject;
use TYPO3\TestingFramework\Core\Unit\UnitTestCase;
use Vendor\Extension\Service\EnvelopeEncryptionService;
use Vendor\Extension\Service\KeyManagementServiceInterface;
final class EnvelopeEncryptionServiceTest extends UnitTestCase
{
private EnvelopeEncryptionService $subject;
private KeyManagementServiceInterface&MockObject $keyManagementService;
private string $testKek;
protected function setUp(): void
{
parent::setUp();
$this->testKek = sodium_crypto_secretbox_keygen();
$this->keyManagementService = $this->createMock(KeyManagementServiceInterface::class);
$this->keyManagementService
->method('getKeyEncryptionKey')
->willReturn($this->testKek);
$this->subject = new EnvelopeEncryptionService($this->keyManagementService);
}
protected function tearDown(): void
{
sodium_memzero($this->testKek);
parent::tearDown();
}
#[Test]
public function storeGeneratesUniqueDekPerSecret(): void
{
$result1 = $this->subject->store('secret1');
$result2 = $this->subject->store('secret2');
// Each secret gets its own DEK
self::assertNotSame($result1['encrypted_dek'], $result2['encrypted_dek']);
}
#[Test]
public function retrieveDecryptsWithCorrectDek(): void
{
$original = 'my-api-secret';
$stored = $this->subject->store($original);
$retrieved = $this->subject->retrieve(
$stored['encrypted_value'],
$stored['encrypted_dek'],
$stored['nonce']
);
self::assertSame($original, $retrieved);
}
#[Test]
public function keyRotationReEncryptsWithNewKek(): void
{
$original = 'secret-to-rotate';
$stored = $this->subject->store($original);
$newKek = sodium_crypto_secretbox_keygen();
$rotated = $this->subject->rotateKey($stored, $this->testKek, $newKek);
// Encrypted DEK changes, but value remains accessible
self::assertNotSame($stored['encrypted_dek'], $rotated['encrypted_dek']);
sodium_memzero($newKek);
}
}
```
## Testing Memory-Safe Secret Handling
### Verifying sodium_memzero() Usage
For security-critical code, verify that secrets are cleared from memory:
```php
<?php
declare(strict_types=1);
namespace Vendor\Extension\Tests\Unit\Http;
use PHPUnit\Framework\Attributes\Test;
use PHPUnit\Framework\MockObject\MockObject;
use TYPO3\TestingFramework\Core\Unit\UnitTestCase;
use Vendor\Extension\Http\VaultHttpClient;
use Vendor\Extension\Service\VaultServiceInterface;
use Psr\Http\Client\ClientInterface;
final class VaultHttpClientTest extends UnitTestCase
{
private VaultHttpClient $subject;
private VaultServiceInterface&MockObject $vaultService;
private ClientInterface&MockObject $httpClient;
protected function setUp(): void
{
parent::setUp();
$this->vaultService = $this->createMock(VaultServiceInterface::class);
$this->httpClient = $this->createMock(ClientInterface::class);
$this->subject = new VaultHttpClient($this->vaultService, $this->httpClient);
}
#[Test]
public function secretIsRetrievedJustInTime(): void
{
// Verify secret is retrieved only when needed
$this->vaultService
->expects(self::once())
->method('retrieve')
->with('api-key-identifier')
->willReturn('secret-value');
$this->httpClient
->expects(self::once())
->method('sendRequest');
$this->subject->request('GET', 'https://api.example.com', [
'auth_secret' => 'api-key-identifier',
]);
}
#[Test]
public function secretNotRetrievedWhenNotNeeded(): void
{
// Verify no vault access for requests without auth
$this->vaultService
->expects(self::never())
->method('retrieve');
$this->subject->request('GET', 'https://api.example.com');
}
}
```
### Testing the Secret Clearing Pattern
While directly testing `sodium_memzero()` is difficult (the memory is zeroed), test the pattern:
```php
#[Test]
public function requestClearsSecretEvenOnException(): void
{
$this->vaultService
->method('retrieve')
->willReturn('secret-value');
$this->httpClient
->method('sendRequest')
->willThrowException(new \RuntimeException('Network error'));
// The implementation should use try/finally to ensure cleanup
try {
$this->subject->request('GET', 'https://api.example.com', [
'auth_secret' => 'test-key',
]);
} catch (\RuntimeException) {
// Expected - secret should still be cleared in finally block
}
// If we got here without memory issues, the pattern is correct
self::assertTrue(true);
}
```
## Test Data Patterns
### Deterministic Test Keys
```php
final class CryptoTestHelper
{
/**
* Generate a deterministic test key for reproducible tests.
* NEVER use in production - only for testing.
*/
public static function createTestKey(string $seed = 'test'): string
{
return sodium_crypto_generichash($seed, '', SODIUM_CRYPTO_SECRETBOX_KEYBYTES);
}
/**
* Create a test secret with cleanup callback.
* @return array{secret: string, cleanup: callable}
*/
public static function createTestSecret(string $value): array
{
$secret = $value;
return [
'secret' => $secret,
'cleanup' => static function () use (&$secret): void {
if ($secret !== '') {
sodium_memzero($secret);
}
},
];
}
}
```
### Using Test Helpers
```php
protected function setUp(): void
{
parent::setUp();
$this->testData = CryptoTestHelper::createTestSecret('api-key-123');
}
protected function tearDown(): void
{
($this->testData['cleanup'])();
parent::tearDown();
}
```
## Functional Testing Encrypted Storage
For database-backed secret storage:
```php
<?php
declare(strict_types=1);
namespace Vendor\Extension\Tests\Functional\Repository;
use PHPUnit\Framework\Attributes\Test;
use TYPO3\TestingFramework\Core\Functional\FunctionalTestCase;
use Vendor\Extension\Repository\SecretRepository;
final class SecretRepositoryTest extends FunctionalTestCase
{
protected array $testExtensionsToLoad = ['vendor/extension'];
private SecretRepository $subject;
private string $testKey;
protected function setUp(): void
{
parent::setUp();
$this->testKey = sodium_crypto_secretbox_keygen();
$this->subject = $this->get(SecretRepository::class);
}
protected function tearDown(): void
{
sodium_memzero($this->testKey);
parent::tearDown();
}
#[Test]
public function storedSecretIsEncryptedInDatabase(): void
{
$identifier = 'test-api-key';
$plaintext = 'super-secret-value';
$this->subject->store($identifier, $plaintext, $this->testKey);
// Direct database query to verify encryption
$row = $this->getConnectionPool()
->getConnectionForTable('tx_extension_secret')
->select(['*'], 'tx_extension_secret', ['identifier' => $identifier])
->fetchAssociative();
// Value in database should NOT match plaintext
self::assertNotSame($plaintext, $row['encrypted_value']);
self::assertNotEmpty($row['encrypted_dek']);
self::assertNotEmpty($row['nonce']);
}
#[Test]
public function retrieveReturnsDecryptedValue(): void
{
$identifier = 'test-secret';
$plaintext = 'my-secret-123';
$this->subject->store($identifier, $plaintext, $this->testKey);
$retrieved = $this->subject->retrieve($identifier, $this->testKey);
self::assertSame($plaintext, $retrieved);
}
}
```
## Algorithm-Specific Nonce Lengths
Different algorithms require different nonce lengths. Using wrong nonce length reduces security.
### The Problem
```php
// ❌ WRONG - Same nonce length for all algorithms reduces entropy
private const NONCE_LENGTH = 12; // AES-GCM length
public function encrypt(string $plaintext): string
{
$nonce = random_bytes(self::NONCE_LENGTH); // Always 12 bytes!
// For XChaCha20-Poly1305, this wastes 12 bytes of nonce entropy
// (should be 24 bytes)
if ($this->algorithm === 'xchacha20') {
$nonce = str_pad($nonce, 24, "\0"); // Padding with zeros = BAD!
}
}
```
### The Fix - Dynamic Nonce Length
```php
private function getNonceLength(): int
{
return match ($this->algorithm) {
'aes-256-gcm' => SODIUM_CRYPTO_AEAD_AES256GCM_NPUBBYTES, // 12 bytes
'xchacha20-poly1305' => SODIUM_CRYPTO_AEAD_XCHACHA20POLY1305_IETF_NPUBBYTES, // 24 bytes
default => throw new \InvalidArgumentException('Unknown algorithm'),
};
}
public function encrypt(string $plaintext): string
{
// ✅ CORRECT - Full entropy for each algorithm
$nonce = random_bytes($this->getNonceLength());
}
```
### Testing Nonce Length
```php
#[Test]
public function aesGcmUsesCorrectNonceLength(): void
{
$service = new EncryptionService('aes-256-gcm', $this->key);
$encrypted = $service->encrypt('test');
// Extract nonce from ciphertext
$nonce = substr($encrypted, 0, SODIUM_CRYPTO_AEAD_AES256GCM_NPUBBYTES);
self::assertSame(12, strlen($nonce));
}
#[Test]
public function xchachaUsesCorrectNonceLength(): void
{
$service = new EncryptionService('xchacha20-poly1305', $this->key);
$encrypted = $service->encrypt('test');
// Extract nonce from ciphertext
$nonce = substr($encrypted, 0, SODIUM_CRYPTO_AEAD_XCHACHA20POLY1305_IETF_NPUBBYTES);
self::assertSame(24, strlen($nonce));
}
#[Test]
public function noncesAreFullyRandom(): void
{
// Verify nonce doesn't contain padding zeros
$service = new EncryptionService('xchacha20-poly1305', $this->key);
$nonces = [];
for ($i = 0; $i < 10; $i++) {
$encrypted = $service->encrypt('test');
$nonce = substr($encrypted, 0, 24);
$nonces[] = $nonce;
// No nonce should end with 12 zero bytes (padding pattern)
$lastBytes = substr($nonce, 12);
self::assertNotSame(str_repeat("\0", 12), $lastBytes);
}
// All nonces should be unique
self::assertSame(10, count(array_unique($nonces)));
}
```
## Security Test Checklist
| Test Case | Purpose |
|-----------|---------|
| Round-trip encrypt/decrypt | Basic correctness |
| Different ciphertext for same input | Nonce randomness |
| Correct nonce length per algorithm | Algorithm compliance |
| Wrong key fails decryption | Key isolation |
| Tampered ciphertext fails | Integrity protection |
| Empty input handling | Edge case security |
| Key rotation preserves access | Migration safety |
| Secret cleared after use | Memory safety |
| No plaintext in logs | Audit safety |
## Anti-Patterns to Avoid
### Never Log Secrets
```php
// WRONG - logs actual secret
$this->logger->debug('Retrieved secret: ' . $secret);
// CORRECT - log only identifier
$this->logger->debug('Retrieved secret', ['identifier' => $identifier]);
```
### Never Use Weak Keys in Tests
```php
// WRONG - predictable key
$key = str_repeat('0', 32);
// CORRECT - proper key generation
$key = sodium_crypto_secretbox_keygen();
```
### Never Skip Cleanup in Tests
```php
// WRONG - secret remains in memory
protected function tearDown(): void
{
parent::tearDown();
}
// CORRECT - explicit cleanup
protected function tearDown(): void
{
if (isset($this->testKey)) {
sodium_memzero($this->testKey);
}
parent::tearDown();
}
```
## CI Integration
For security-critical extensions, run crypto tests in isolation:
```yaml
# .github/workflows/test.yml
jobs:
crypto-tests:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: shivammathur/setup-php@v2
with:
php-version: '8.2'
extensions: sodium
- run: composer install
- name: Run crypto-specific tests
run: |
vendor/bin/phpunit --testsuite Unit \
--filter 'Encryption|Crypto|Secret|Vault'
```
## Resources
- [libsodium Documentation](https://doc.libsodium.org/)
- [PHP Sodium Functions](https://www.php.net/manual/en/book.sodium.php)
- [OWASP Cryptographic Storage](https://cheatsheetseries.owasp.org/cheatsheets/Cryptographic_Storage_Cheat_Sheet.html)
```
### references/mutation-testing.md
```markdown
# Mutation Testing for TYPO3 Extensions
## Overview
Mutation testing verifies test suite quality by introducing small bugs (mutants) into the code and checking if tests catch them. If a test suite has good coverage but low mutation score, the tests may not be actually testing the important behaviors.
> **Key Distinction**: Mutation testing mutates **code** to verify test quality. For testing that mutates **inputs** to find crashes, see [Fuzz Testing](fuzz-testing.md).
| Aspect | Mutation Testing | Fuzz Testing |
|--------|-----------------|--------------|
| **Mutates** | Source code | Input data |
| **Purpose** | Verify test quality | Find crashes/vulnerabilities |
| **Example** | `if (x != y)` → `if (x == y)` | `<img src="` → `<img src="../../../../etc/passwd` |
| **Finds** | Weak/missing tests | Parsing bugs, security issues |
| **Tool (PHP)** | Infection | nikic/php-fuzzer |
## When to Use Mutation Testing
- After achieving high code coverage (70%+) to verify test quality
- Before releases to ensure critical paths are well-tested
- When refactoring to ensure tests catch regressions
- To identify "weak spots" in test coverage
## Tools
### Infection (Recommended)
PHP mutation testing framework with PHPUnit integration.
**Installation:**
```bash
composer require --dev infection/infection:^0.27
```
**Key features:**
- Mutates PHP code with various operators
- Integrates with PHPUnit and Pest
- Generates HTML and JSON reports
- Supports incremental analysis
## Configuration
Create `infection.json5` in project root:
```json5
{
"$schema": "https://raw.githubusercontent.com/infection/infection/master/resources/schema.json",
"source": {
"directories": [
"Classes"
],
"excludes": [
"Domain/Model" // Skip simple DTOs
]
},
"logs": {
"html": ".Build/logs/infection.html",
"text": ".Build/logs/infection.log",
"summary": ".Build/logs/infection-summary.log"
},
"mutators": {
"@default": true,
// Disable noisy mutators if needed
"TrueValue": false,
"FalseValue": false
},
"minMsi": 60, // Minimum Mutation Score Indicator
"minCoveredMsi": 80, // Minimum MSI for covered code only
"testFramework": "phpunit",
"testFrameworkOptions": "-c Build/phpunit/UnitTests.xml"
}
```
## Mutation Operators
Infection applies these types of mutations:
### Arithmetic Operators
```php
// Original
$result = $a + $b;
// Mutants
$result = $a - $b; // PlusToMinus
$result = $a * $b; // PlusToMultiplication
```
### Comparison Operators
```php
// Original
if ($value > 10) { ... }
// Mutants
if ($value >= 10) { ... } // GreaterThan to GreaterThanOrEqual
if ($value < 10) { ... } // GreaterThan to LessThan
if (true) { ... } // Always truthy
```
### Boolean Operators
```php
// Original
if ($a && $b) { ... }
// Mutants
if ($a || $b) { ... } // LogicalAnd to LogicalOr
if ($a) { ... } // Remove operand
```
### Return Values
```php
// Original
return $value;
// Mutants
return null; // Return null
return []; // Return empty array
return !$value; // Negate boolean
```
### Method Calls
```php
// Original
$this->save($entity);
// Mutant
// Line removed (method call deleted)
```
## Running Mutation Tests
### Via Composer Scripts
```json
{
"scripts": {
"ci:test:mutation": [
"@ci:test:php:unit",
".Build/bin/infection --threads=4"
],
"ci:test:mutation:quick": [
".Build/bin/infection --threads=4 --only-covered --min-msi=60"
]
}
}
```
### Via runTests.sh
```bash
# Add to Build/Scripts/runTests.sh
mutation)
# Run unit tests first to generate coverage
COMMAND=(.Build/bin/phpunit -c Build/phpunit/UnitTests.xml --coverage-xml=.Build/logs/coverage-xml --coverage-html=.Build/logs/coverage-html)
${CONTAINER_BIN} run ${CONTAINER_COMMON_PARAMS} --name unit-${SUFFIX} ${IMAGE_PHP} "${COMMAND[@]}"
# Run mutation testing
COMMAND=(.Build/bin/infection --threads=4 --coverage=.Build/logs/coverage-xml)
${CONTAINER_BIN} run ${CONTAINER_COMMON_PARAMS} --name mutation-${SUFFIX} ${IMAGE_PHP} "${COMMAND[@]}"
SUITE_EXIT_CODE=$?
;;
```
### Directly
```bash
# Full mutation test run
.Build/bin/infection --threads=4
# Quick run (only test covered code)
.Build/bin/infection --threads=4 --only-covered
# With existing coverage
.Build/bin/infection --threads=4 --coverage=.Build/logs/coverage-xml
# Filter to specific directory
.Build/bin/infection --threads=4 --filter=Classes/Service
```
## Interpreting Results
### Mutation Score Indicator (MSI)
```
Mutations: 150 total
Killed: 120 (80%) ← Tests caught the mutation
Escaped: 15 (10%) ← Tests MISSED the mutation (bad!)
Errors: 5 (3%) ← Mutation caused fatal error
Uncovered: 10 (7%) ← No tests for this code
MSI: 80% ← (Killed + Errors) / Total
Covered MSI: 86% ← MSI for covered code only
```
### Understanding Results
| Status | Meaning | Action |
|--------|---------|--------|
| **Killed** | Test failed when mutant introduced | Good - test is effective |
| **Escaped** | Test passed with mutant | **Bad - add/improve tests** |
| **Errors** | Mutant caused fatal error | Usually OK (type errors) |
| **Uncovered** | No test coverage | Add coverage first |
| **Timeout** | Test took too long with mutant | Usually OK |
| **Skipped** | Mutant not tested | Check config |
### Target Scores
| Level | MSI | Covered MSI | Use Case |
|-------|-----|-------------|----------|
| Basic | 50%+ | 60%+ | Initial implementation |
| Good | 70%+ | 80%+ | Production code |
| Excellent | 85%+ | 90%+ | Critical/security code |
## Improving Mutation Score
### 1. Fix Escaped Mutants
Review the HTML report to find escaped mutants:
```html
<!-- .Build/logs/infection.html -->
<!-- Shows: Original code, Mutated code, Test that should have caught it -->
```
### 2. Add Boundary Tests
```php
// If this escapes:
// if ($age >= 18) → if ($age > 18)
// Add boundary test:
public function testAgeExactly18IsAllowed(): void
{
self::assertTrue($this->validator->isAdult(18));
}
```
### 3. Add Negative Tests
```php
// If method removal escapes:
// $logger->error($message); // removed
// Add test verifying the call:
public function testErrorIsLogged(): void
{
$logger = $this->createMock(LoggerInterface::class);
$logger->expects(self::once())
->method('error')
->with('Expected message');
$service = new MyService($logger);
$service->doSomethingThatLogs();
}
```
### 4. Test Return Values
```php
// If return value mutation escapes:
// return $result; → return null;
// Verify return value explicitly:
public function testReturnsCalculatedValue(): void
{
$result = $calculator->compute(5, 3);
self::assertSame(8, $result); // Not just assertNotNull!
}
```
## CI Integration
### GitHub Actions
```yaml
# .github/workflows/tests.yml
mutation:
name: Mutation Testing
runs-on: ubuntu-latest
needs: [unit] # Run after unit tests pass
steps:
- uses: actions/checkout@v4
- uses: shivammathur/setup-php@v2
with:
php-version: '8.2'
coverage: pcov
- run: composer install
- name: Run unit tests with coverage
run: |
.Build/bin/phpunit -c Build/phpunit/UnitTests.xml \
--coverage-xml=.Build/logs/coverage-xml \
--log-junit=.Build/logs/phpunit.xml
- name: Run mutation testing
run: |
.Build/bin/infection \
--threads=4 \
--coverage=.Build/logs/coverage-xml \
--min-msi=60 \
--min-covered-msi=80 \
--skip-initial-tests
- name: Upload mutation report
if: always()
uses: actions/upload-artifact@v4
with:
name: mutation-report
path: .Build/logs/infection.html
```
### Quality Gate
```yaml
# Fail CI if mutation score drops
- name: Check mutation score
run: |
.Build/bin/infection \
--threads=4 \
--min-msi=60 \
--min-covered-msi=80 \
--only-covered
```
## Best Practices
1. **Run unit tests first** - Mutation testing needs passing tests
2. **Start with low thresholds** - Increase gradually (50% → 60% → 70%)
3. **Focus on covered code** - Use `--only-covered` for actionable results
4. **Prioritize escaped mutants** - These indicate weak tests
5. **Exclude trivial code** - Skip getters/setters/DTOs in config
6. **Run incrementally** - Use `--git-diff-filter=AM` for changed files only
7. **Document exclusions** - Explain why code is excluded from mutation testing
## Incremental Mutation Testing
For large codebases, run mutation testing only on changed files:
```bash
# Only test files changed in current branch
.Build/bin/infection \
--threads=4 \
--git-diff-filter=AM \
--git-diff-base=origin/main \
--only-covered
```
## Directory Structure
```
project/
├── infection.json5 # Infection configuration
├── .Build/
│ └── logs/
│ ├── infection.html # HTML report
│ ├── infection.log # Detailed log
│ └── infection-summary.log
└── Tests/
└── Unit/
└── Service/
└── MyServiceTest.php
```
## Resources
- [Infection PHP](https://infection.github.io/) - PHP mutation testing framework
- [Mutation Testing](https://en.wikipedia.org/wiki/Mutation_testing) - Concept overview
- [Pitest](https://pitest.org/) - Java mutation testing (for comparison)
- [Stryker](https://stryker-mutator.io/) - JavaScript/TypeScript mutation testing
```