ergebnis/phpunit-slow-test-detector

Provides facilities for detecting slow tests in phpunit/phpunit.

2.14.0 2024-04-08 06:35 UTC

Requires

• php: ~7.0.0 || ~7.1.0 || ~7.2.0 || ~7.3.0 || ~7.4.0 || ~8.0.0 || ~8.1.0 || ~8.2.0 || ~8.3.0
• phpunit/phpunit: ^6.5.0 || ^7.5.0 || ^8.5.19 || ^9.0.0 || ^10.0.0 || ^11.0.0

Requires (Dev)

• ergebnis/composer-normalize: ^2.42.0
• ergebnis/license: ^2.4.0
• ergebnis/php-cs-fixer-config: ^6.25.1
• fakerphp/faker: ~1.20.0
• psalm/plugin-phpunit: ~0.19.0
• psr/container: ~1.0.0
• rector/rector: ^1.0.4
• vimeo/psalm: ^5.23.1

MIT

• Andreas Möller

• phpunit
• test
• extension
• detector
• slow

README

This project provides a composer package and a Phar archive with an extension for detecting slow tests in phpunit/phpunit.

The extension is compatible with the following versions of phpunit/phpunit:

• phpunit/phpunit:^6.5.0
• phpunit/phpunit:^7.5.0
• phpunit/phpunit:^8.5.19
• phpunit/phpunit:^9.0.0
• phpunit/phpunit:^10.0.0
• phpunit/phpunit:^11.0.0

Installation

Installation with composer

Run

composer require --dev ergebnis/phpunit-slow-test-detector

to install ergebnis/phpunit-slow-test-detector as a composer package.

Installation as Phar

Download phpunit-slow-test-detector.phar from the latest release.

Usage

Bootstrapping the extension

Before the extension can detect slow tests in phpunit/phpunit, you need to bootstrap it. The bootstrapping mechanism depends on the version of phpunit/phpunit you are using.

Bootstrapping the extension as a composer package

To bootstrap the extension as a composer package when using

• phpunit/phpunit:^6.5.0

adjust your phpunit.xml configuration file and configure the

• listeners element on phpunit/phpunit:^6.5.0

 <phpunit
     xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
     xsi:noNamespaceSchemaLocation="vendor/phpunit/phpunit/phpunit.xsd"
     bootstrap="vendor/autoload.php"
 >
+    <listeners>
+        <listener class="Ergebnis\PHPUnit\SlowTestDetector\Extension"/>
+    </listeners>
     <testsuites>
         <testsuite name="unit">
             <directory>test/Unit/</directory>
         </testsuite>
     </testsuites>
 </phpunit>

To bootstrap the extension as a composer package when using

• phpunit/phpunit:^7.5.0
• phpunit/phpunit:^8.5.19
• phpunit/phpunit:^9.0.0

adjust your phpunit.xml configuration file and configure the

• extensions element on phpunit/phpunit:^7.5.0
• extensions element on phpunit/phpunit:^8.5.19
• extensions element on phpunit/phpunit:^9.0.0

 <phpunit
     xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
     xsi:noNamespaceSchemaLocation="vendor/phpunit/phpunit/phpunit.xsd"
     bootstrap="vendor/autoload.php"
 >
+    <extensions>
+        <extension class="Ergebnis\PHPUnit\SlowTestDetector\Extension"/>
+    </extensions>
     <testsuites>
         <testsuite name="unit">
             <directory>test/Unit/</directory>
         </testsuite>
     </testsuites>
 </phpunit>

To bootstrap the extension as a composer package when using

• phpunit/phpunit:^10.0.0
• phpunit/phpunit:^11.0.0

adjust your phpunit.xml configuration file and configure the

• extensions element on phpunit/phpunit:^10.0.0
• extensions element on phpunit/phpunit:^11.0.0

 <phpunit
     xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
     xsi:noNamespaceSchemaLocation="vendor/phpunit/phpunit/phpunit.xsd"
     bootstrap="vendor/autoload.php"
 >
+    <extensions>
+        <bootstrap class="Ergebnis\PHPUnit\SlowTestDetector\Extension"/>
+    </extensions>
     <testsuites>
         <testsuite name="unit">
             <directory>test/Unit/</directory>
         </testsuite>
     </testsuites>
 </phpunit>

Bootstrapping the extension as a PHAR

To bootstrap the extension as a PHAR when using

• phpunit/phpunit:^7.5.0
• phpunit/phpunit:^8.5.19
• phpunit/phpunit:^9.0.0

adjust your phpunit.xml configuration file and configure the

• extensionsDirectory attribute and the extensions element on phpunit/phpunit:^7.5.0
• extensionsDirectory attribute and the extensions element on phpunit/phpunit:^8.5.19
• extensionsDirectory attribute and the extensions element on phpunit/phpunit:^9.0.0

 <phpunit
     xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
     xsi:noNamespaceSchemaLocation="vendor/phpunit/phpunit/phpunit.xsd"
     bootstrap="vendor/autoload.php"
+    extensionsDirectory="directory/where/you/saved/the/extension/phars"
 >
+    <extensions>
+        <bootstrap class="Ergebnis\PHPUnit\SlowTestDetector\Extension"/>
+    </extensions>
     <testsuites>
         <testsuite name="unit">
             <directory>test/Unit/</directory>
         </testsuite>
     </testsuites>
 </phpunit>

To bootstrap the extension as a PHAR when using

• phpunit/phpunit:^10.0.0
• phpunit/phpunit:^11.0.0

adjust your phpunit.xml configuration file and configure the

• extensionsDirectory attribute and the extensions element on phpunit/phpunit:^10.0.0
• extensionsDirectory attribute and the extensions element on phpunit/phpunit:^11.0.0

 <phpunit
     xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
     xsi:noNamespaceSchemaLocation="vendor/phpunit/phpunit/phpunit.xsd"
     bootstrap="vendor/autoload.php"
+    extensionsDirectory="directory/where/you/saved/the/extension/phars"
 >
+    <extensions>
+        <extension class="Ergebnis\PHPUnit\SlowTestDetector\Extension"/>
+    </extensions>
     <testsuites>
         <testsuite name="unit">
             <directory>test/Unit/</directory>
         </testsuite>
     </testsuites>
 </phpunit>

Configuring the extension

You can configure the extension with the following options in your phpunit.xml configuration file:

• maximum-count, an int, the maximum count of slow test that should be reported, defaults to 10
• maximum-duration, an int, the maximum duration in milliseconds for a test before the extension considers it as a slow test, defaults to 500

The configuration mechanism depends on the version of phpunit/phpunit you are using.

Configuring the extension

To configure the extension when using

• phpunit/phpunit:^6.5.0

adjust your phpunit.xml configuration file and configure the

• arguments element on phpunit/phpunit:^6.5.0

The following example configures the maximum count of slow tests to three, and the maximum duration for all tests to 250 milliseconds:

 <phpunit
     xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
     xsi:noNamespaceSchemaLocation="vendor/phpunit/phpunit/phpunit.xsd"
     bootstrap="vendor/autoload.php"
 >
     <listeners>
-        <listener class="Ergebnis\PHPUnit\SlowTestDetector\Extension"/>
+        <listener class="Ergebnis\PHPUnit\SlowTestDetector\Extension">
+            <arguments>
+                <array>
+                    <element key="maximum-count">
+                        <integer>3</integer>
+                    </element>
+                    <element key="maximum-duration">
+                        <integer>250</integer>
+                    </element>
+                </array>
+            </arguments>
+        </listener>
     </listeners>
     <testsuites>
         <testsuite name="unit">
             <directory>test/Unit/</directory>
        </testsuite>
     </testsuites>
 </phpunit>

To configure the extension when using

• phpunit/phpunit:^7.5.0
• phpunit/phpunit:^8.5.19
• phpunit/phpunit:^9.0.0

adjust your phpunit.xml configuration file and configure the

• arguments element on phpunit/phpunit:^7.5.0
• arguments element on phpunit/phpunit:^8.5.19
• arguments element on phpunit/phpunit:^9.0.0

The following example configures the maximum count of slow tests to three, and the maximum duration for all tests to 250 milliseconds:

 <phpunit
     xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
     xsi:noNamespaceSchemaLocation="vendor/phpunit/phpunit/phpunit.xsd"
     bootstrap="vendor/autoload.php"
 >
     <extensions>
-        <extension class="Ergebnis\PHPUnit\SlowTestDetector\Extension"/>
+        <extension class="Ergebnis\PHPUnit\SlowTestDetector\Extension">
+            <arguments>
+                <array>
+                    <element key="maximum-count">
+                        <integer>3</integer>
+                    </element>
+                    <element key="maximum-duration">
+                        <integer>250</integer>
+                    </element>
+                </array>
+            </arguments>
+        </extension>
     </extensions>
     <testsuites>
         <testsuite name="unit">
             <directory>test/Unit/</directory>
        </testsuite>
     </testsuites>
 </phpunit>

To configure the extension when using

• phpunit/phpunit:^10.0.0
• phpunit/phpunit:^11.0.0

adjust your phpunit.xml configuration file and configure one or more

• parameter elements on phpunit/phpunit:^10.0.0
• parameter elements on phpunit/phpunit:^11.0.0

The following example configures the maximum count of slow tests to three, and the maximum duration for all tests to 250 milliseconds:

 <phpunit
     xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
     xsi:noNamespaceSchemaLocation="vendor/phpunit/phpunit/phpunit.xsd"
     bootstrap="vendor/autoload.php"
 >
     <extensions>
-        <bootstrap class="Ergebnis\PHPUnit\SlowTestDetector\Extension"/>
+        <bootstrap class="Ergebnis\PHPUnit\SlowTestDetector\Extension">
+            <parameter name="maximum-count" value="3"/>
+            <parameter name="maximum-duration" value="250"/>
+        </bootstrap>
     </extensions>
     <testsuites>
         <testsuite name="unit">
             <directory>test/Unit/</directory>
        </testsuite>
     </testsuites>
 </phpunit>

Configuring the maximum duration per test case

You can configure the maximum duration for a single test case with

• an Attribute\MaximumDuration attribute when using
  • phpunit/phpunit:^10.0.0
  • phpunit/phpunit:^11.0.0
• a @maximumDuration annotation in the DocBlock when using
  • phpunit/phpunit:^6.5.0
  • phpunit/phpunit:^7.5.0
  • phpunit/phpunit:^8.5.19
  • phpunit/phpunit:^9.0.0
• a @slowThreshold annotation in the DocBlock when using
  • phpunit/phpunit:^6.5.0
  • phpunit/phpunit:^7.5.0
  • phpunit/phpunit:^8.5.19
  • phpunit/phpunit:^9.0.0

The following example configures the maximum durations for single test cases to 5.000 ms, 4.000 ms, and 3.000 ms:

<?php

declare(strict_types=1);

use PHPUnit\Framework;
use Ergebnis\PHPUnit\SlowTestDetector;

final class ExtraSlowTest extends Framework\TestCase
{
    /**
     */
    #[SlowTestDetector\Attribute\MaximumDuration(5000)]
    public function testExtraExtraSlow(): void
    {
        // ...
    }

    /**
     * @maximumDuration 4000
     */
    public function testAlsoQuiteSlow(): void
    {
        // ...
    }

    /**
     * @slowThreshold 3000
     */
    public function testQuiteSlow(): void
    {
        // ...
    }
}

Note

Support for the @slowThreshold annotation exists only to help you move from johnkary/phpunit-speedtrap. It will be deprecated and removed in the near future.

Running tests

When you have bootstrapped the extension, you can run your tests as usually:

vendor/bin/phpunit

When the extension has detected slow tests, it will report them:

PHPUnit 10.0.0 by Sebastian Bergmann and contributors.

Runtime:       PHP 8.1.0
Configuration: test/EndToEnd/Default/phpunit.xml
Random Seed:   1676103726

.............                                                                                                                                                                                                                                                                                                   13 / 13 (100%)

Detected 11 tests where the duration exceeded the maximum duration.

 1. 1.604 (0.500) Ergebnis\PHPUnit\SlowTestDetector\Test\EndToEnd\Default\SleeperTest::testSleeperSleepsLongerThanDefaultMaximumDurationWithDataProvider#9
 2. 1.505 (0.500) Ergebnis\PHPUnit\SlowTestDetector\Test\EndToEnd\Default\SleeperTest::testSleeperSleepsLongerThanDefaultMaximumDurationWithDataProvider#8
 3. 1.403 (0.500) Ergebnis\PHPUnit\SlowTestDetector\Test\EndToEnd\Default\SleeperTest::testSleeperSleepsLongerThanDefaultMaximumDurationWithDataProvider#7
 4. 1.303 (0.500) Ergebnis\PHPUnit\SlowTestDetector\Test\EndToEnd\Default\SleeperTest::testSleeperSleepsLongerThanDefaultMaximumDurationWithDataProvider#6
 5. 1.205 (0.500) Ergebnis\PHPUnit\SlowTestDetector\Test\EndToEnd\Default\SleeperTest::testSleeperSleepsLongerThanDefaultMaximumDurationWithDataProvider#5
 6. 1.103 (0.500) Ergebnis\PHPUnit\SlowTestDetector\Test\EndToEnd\Default\SleeperTest::testSleeperSleepsLongerThanDefaultMaximumDurationWithDataProvider#4
 7. 1.005 (0.500) Ergebnis\PHPUnit\SlowTestDetector\Test\EndToEnd\Default\SleeperTest::testSleeperSleepsLongerThanDefaultMaximumDurationWithDataProvider#3
 8. 0.905 (0.500) Ergebnis\PHPUnit\SlowTestDetector\Test\EndToEnd\Default\SleeperTest::testSleeperSleepsLongerThanDefaultMaximumDurationWithDataProvider#2
 9. 0.805 (0.500) Ergebnis\PHPUnit\SlowTestDetector\Test\EndToEnd\Default\SleeperTest::testSleeperSleepsLongerThanDefaultMaximumDurationWithDataProvider#1
10. 0.705 (0.500) Ergebnis\PHPUnit\SlowTestDetector\Test\EndToEnd\Default\SleeperTest::testSleeperSleepsLongerThanDefaultMaximumDurationWithDataProvider#0

There is 1 additional slow test that is not listed here.

Time: 00:12.601, Memory: 8.00 MB

OK (13 tests, 13 assertions)

Understanding measured test durations

When using

• phpunit/phpunit:^6.5.0

• phpunit/phpunit:^7.5.0

• phpunit/phpunit:^8.5.19

• phpunit/phpunit:^9.0.0

• the extension uses the hooks event system of phpunit/phpunit.

The hooks event system supports eleven hook methods that phpunit/phpunit invokes during the execution of tests.

When the extension uses the hooks event system, it uses the PHPUnit\Runner\AfterTestHook, which receives the duration of invoking PHPUnit\Framework\TestCase::runBare() and more.

When phpunit/phpunit invokes PHPUnit\Framework\TestCase::runBare(), it will invoke the following methods before the first test method in the class:

• PHPUnit\Framework\TestCase::setUpBeforeClass() and methods annotated with @beforeClass

When phpunit/phpunit invokes PHPUnit\Framework\TestCase::runBare(), it will invoke the following methods before every test method in the class:

• PHPUnit\Framework\TestCase::setUp() and methods annotated with @before
• PHPUnit\Framework\TestCase::assertPreConditions()

When phpunit/phpunit invokes PHPUnit\Framework\TestCase::runBare(), it will invoke the following methods after every test method in the class:

• PHPUnit\Framework\TestCase::assertPostConditions()
• PHPUnit\Framework\TestCase::tearDown() and methods annotated with @after

When phpunit/phpunit invokes PHPUnit\Framework\TestCase::runBare(), it will invoke the following methods after the last test method in the class:

• PHPUnit\Framework\TestCase::tearDownAfterClass() and methods annotated with @afterClass

Note

Because of this behavior, the measured test durations can and will vary depending on the order in which phpunit/phpunit executes tests.

When using

• phpunit/phpunit:^10.0.0
• phpunit/phpunit:^11.0.0

the extension uses the new event system of phpunit/phpunit.

The new event system supports a wide range of events that phpunit/phpunit emits during the execution of tests.

When the extension uses the new event system, it uses and subscribes to the PHPUnit\Event\Test\PreparationStarted and PHPUnit\Event\Test\Finished events and measures the duration between the points in time when phpunit/phpunit emits the former and the latter.

When phpunit/phpunit invokes PHPUnit\Framework\TestCase::runBare(), it will invoke the following methods before the first test method in the class:

• PHPUnit\Framework\TestCase::setUpBeforeClass() and methods annotated with @beforeClass

When phpunit/phpunit invokes PHPUnit\Framework\TestCase::runBare(), it will invoke the following methods before every test method in the class:

• PHPUnit\Framework\TestCase::setUp() and methods annotated with @before
• PHPUnit\Framework\TestCase::assertPreConditions()

When phpunit/phpunit invokes PHPUnit\Framework\TestCase::runBare(), it will invoke the following methods after every test method in the class:

• PHPUnit\Framework\TestCase::assertPostConditions()
• PHPUnit\Framework\TestCase::tearDown() and methods annotated with @after

When phpunit/phpunit invokes PHPUnit\Framework\TestCase::runBare(), it will invoke the following methods after the last test method in the class:

• PHPUnit\Framework\TestCase::tearDownAfterClass() and methods annotated with @afterClass

Note

Because of this behavior, the measured test durations can and will vary depending on the order in which phpunit/phpunit executes tests.

Changelog

The maintainers of this project record notable changes to this project in a changelog.

Contributing

The maintainers of this project suggest following the contribution guide.

Code of Conduct

The maintainers of this project ask contributors to follow the code of conduct.

General Support Policy

The maintainers of this project provide limited support.

You can support the maintenance of this project by sponsoring @localheinz or requesting an invoice for services related to this project.

PHP Version Support Policy

This project supports PHP versions with active and security support.

The maintainers of this project add support for a PHP version following its initial release and drop support for a PHP version when it has reached the end of security support.

Security Policy

This project has a security policy.

License

This project uses the MIT license.

Credits

This package is inspired by johnkary/phpunit-speedtrap, originally licensed under MIT by John Kary.

Social

Follow @localheinz and @ergebnis on Twitter.