README

This package helps to run time/IO related unit tests (e.g., sleep function calls, database queries, API calls, etc) faster using Swoole.

How Does It Work
Installation
Use "counit" in Your Project
- Register the PHPUnit Extension
Examples
Compatibility with PHPUnit
Additional Notes
Local Development
Alternatives
License

How Does It Work

Package counit allows running multiple time/IO related tests concurrently within a single PHP process using Swoole. Counit is compatible with PHPUnit, which means:

Test cases can be written in the same way as those for PHPUnit.
Test cases can run directly under PHPUnit.

A typical test case of counit looks like this:

use Deminy\Counit\TestCase; // Here is the only change made for counit, comparing to test cases for PHPUnit.

class SleepTest extends TestCase
{
  public function testSleep(): void
  {
    $startTime = time();
    sleep(3);
    $endTime = time();

    self::assertEqualsWithDelta(3, ($endTime - $startTime), 1, 'The sleep() function call takes about 3 seconds to finish.');
  }
}

Comparing to PHPUnit, counit could make your test cases faster. Here is a comparison when running the same test suite using PHPUnit and counit for a real project. In the test suite, many tests make calls to method \Deminy\Counit\Counit::sleep() to wait something to happen (e.g., wait data to expire).

	# of Tests	# of Assertions	Time to Finish
counit (without Swoole), or PHPUnit	44	1148	9 minutes and 18 seconds
counit (with Swoole enabled)	44	1148	19 seconds

Installation

The package can be installed using Composer:

composer require deminy/counit --dev

Or, in your composer.json file, make sure to have package deminy/counit included:

{
  "require-dev": {
    "deminy/counit": "^1.1"
  }
}

Please pick the counit version matching the version of PHPUnit used in your project:

counit	PHPUnit	PHP
^1.1	~13.0	>= 8.4.1
^1.1	~12.5.24	>= 8.3
^0.3	~8.0, ~9.0	>= 7.2

Use "counit" in Your Project

Write unit tests in the same way as those for PHPUnit. However, to make those tests faster, please write those time/IO related tests using one of the following two approaches (details will be discussed in the next sections):
- The automatic approach (recommended): Use class Deminy\Counit\TestCase instead of PHPUnit\Framework\TestCase as the base class; every test method is then wrapped in a coroutine automatically.
- The manual approach: Wrap each test case inside the callback function for method Deminy\Counit\Counit::create(), and use method Deminy\Counit\Counit::sleep() instead of the PHP function sleep().
Use the binary executable ./vendor/bin/counit instead of ./vendor/bin/phpunit when running unit tests.
Have the Swoole extension installed. If not installed, counit will work exactly same as PHPUnit (in blocking mode).
Register PHPUnit extension Deminy\Counit\CounitExtension in your phpunit.xml / phpunit.xml.dist. See Register the PHPUnit extension below; this package's own phpunit.xml.dist registers it too.

Register the PHPUnit Extension

Without the extension registered, PHPUnit prints its summary while your tests' coroutines are still running, so the run's reported numbers describe an unfinished run. Every compatibility guarantee in Compatibility with PHPUnit — reported time, assertion totals, late failures, skips and risky verdicts — assumes it is registered. Register it unless you have a specific reason not to.

The syntax depends on your PHPUnit version. For PHPUnit 10 and above (counit ^1.1):

<extensions>
    <bootstrap class="Deminy\Counit\CounitExtension"/>
</extensions>

For PHPUnit 8 and 9 (counit ^0.3), where the class implements the older hook interfaces:

<extensions>
    <extension class="Deminy\Counit\CounitExtension"/>
</extensions>

A run that creates its first coroutine without the extension registered says so once on STDERR, so the failure mode below is not a silent one:

counit notice: PHPUnit extension Deminy\Counit\CounitExtension is not registered, so nothing waits for the tests'
coroutines: [...]. Set COUNIT_SILENCE_TEARDOWN_NOTICE=1 to silence this notice.

What goes wrong without it

The reported time is not the real time. A test's coroutine returns to PHPUnit at its first yield, so PHPUnit reports the test as finished and moves on while the body is still sleeping. With nothing waiting for those coroutines, the summary is printed almost immediately and the process then sits silent until they drain. A real example: a 358-test suite whose true wall-clock time was 9.6 seconds reported Time: 00:00.415, then took another 9 seconds to exit. Wall-clock time (e.g. time ./vendor/bin/counit) is the honest number in that situation; the extension makes PHPUnit's own number honest again.
Assertion totals are wrong, and not even self-consistent. Assertions performed after a yield land in whichever test's counting window happens to be open, and the up-front credit from Counit::create($callable, $count) is never reconciled. In the same suite, the 60 slowest tests reported 1652 assertions when run in isolation but implied 2142 when run as part of the full suite — the same tests, the same code. With the extension registered both figures agree at 1646, which is also what a fully blocking PHPUnit run reports. Treat any assertion total recorded without the extension as unreliable, including totals committed to a project's own documentation.
Late verdicts degrade to a STDERR block. A failure, error, skip or risky verdict that a test reaches after a yield is normally replayed through PHPUnit's own events at the end of the run, so it lands in the summary, the listings, the exit code and the JUnit report. That replay is the extension's work. Without it the runner falls back to printing such verdicts to STDERR (still forcing a non-zero exit code for failures), outside the summary and absent from --log-junit output, and the affected test's recorded status stays "passed".

The extension is a reporting fix, not a performance trade-off: it waits for coroutines that were going to run anyway, so it does not slow a suite down measurably (~0.01s on the 358-test suite above).

Examples

Folder ./tests/unit/automatic and ./tests/unit/manual contain some sample tests, where we have following time-related tests included:

Test slow HTTP requests.
Test long-running MySQL queries.
Test data expiration in Redis.
Test sleep() function calls in PHP.

Setup Test Environment

To run the sample tests, please start the Docker containers and install Composer packages first:

docker compose up -d
docker compose exec -ti swoole composer install -n

There are five containers started: a PHP container, a Swoole container, a Redis container, a MySQL container, and a web server. The PHP container doesn't have the Swoole extension installed, while the Swoole container has it installed and enabled.

As said previously, test cases can be written in the same way as those for PHPUnit. However, to run time/IO related tests faster with counit, we need to make some adjustments when writing those test cases; these adjustments can be made using two different approaches.

The Automatic Approach (recommended)

In this approach (previously called the "global" style), each test case runs in a separate coroutine automatically.

For test cases written using this approach, the only change to make on your existing test cases is to use class Deminy\Counit\TestCase instead of PHPUnit\Framework\TestCase as the base class.

A typical test case of the automatic approach looks like this:

use Deminy\Counit\TestCase; // Here is the only change made for counit, comparing to test cases for PHPUnit.

class SleepTest extends TestCase
{
  public function testSleep(): void
  {
    $startTime = time();
    sleep(3);
    $endTime = time();

    self::assertEqualsWithDelta(3, ($endTime - $startTime), 1, 'The sleep() function call takes about 3 seconds to finish.');
  }
}

When customized method setUpBeforeClass() and tearDownAfterClass() are defined in the test cases, please make sure to call their parent methods accordingly in these customized methods.

With the PHPUnit extension registered, the total # of assertions reported at the end of a run matches PHPUnit exactly, and the per-testcase counts in the JUnit XML report (--log-junit) are corrected as well — exact whenever counit can observe the test's yields (sleep()/usleep() calls in a namespaced test class, and Counit::sleep()). An assertion performed after a yield counit cannot observe (e.g. hooked network IO) goes missing from its own test's JUnit count — but is never added to another test's. See Compatibility with PHPUnit.

To find more tests written using this approach, please check tests under folder ./tests/unit/automatic (test suite "automatic").

The Manual Approach

In this approach (previously called the "case by case" style), you make changes directly on a test case to make it work asynchronously.

For test cases written using this approach, we need to use class Deminy\Counit\Counit accordingly in the test cases where we need to wait for PHP execution or to perform IO operations. Typically, following method calls will be used:

Use method Deminy\Counit\Counit::create() to wrap the test case.
Use method Deminy\Counit\Counit::sleep() instead of the PHP function sleep() to wait for PHP execution. You will need some knowledge on Swoole if you want to make other IO related tests run asynchronously.

A typical test case of the manual approach looks like this:

use Deminy\Counit\Counit;
use PHPUnit\Framework\TestCase;

class SleepTest extends TestCase
{
  public function testSleep(): void
  {
    Counit::create(function () { // To create a new coroutine manually to run the test case.
      $startTime = time();
      Counit::sleep(3); // Call this method instead of PHP function sleep().
      $endTime = time();

      self::assertEqualsWithDelta(3, ($endTime - $startTime), 1, 'The sleep() function call takes about 3 seconds to finish.');
    });
  }
}

In case you need to suppress warning message "This test did not perform any assertions" or to make the number of assertions match, you can include a 2nd parameter when creating the new coroutine:

use Deminy\Counit\Counit;
use PHPUnit\Framework\TestCase;

class SleepTest extends TestCase
{
  public function testSleep(): void
  {
    Counit::create( // To create a new coroutine manually to run the test case.
      function () {
        $startTime = time();
        Counit::sleep(3); // Call this method instead of PHP function sleep().
        $endTime = time();

        self::assertEqualsWithDelta(3, ($endTime - $startTime), 1, 'The sleep() function call takes about 3 seconds to finish.');
      },
      1 // Optional. To suppress warning message "This test did not perform any assertions", and to make the counters match.
    );
  }
}

The 2nd parameter is a request rather than a command: for a test that declares it performs no assertions (through attribute #[DoesNotPerformAssertions] or method expectNotToPerformAssertions()), Counit::create() declines the credit — crediting such a test would make PHPUnit report it as risky.

To find more tests written using this approach, please check tests under folder ./tests/unit/manual (test suite "manual").

Comparisons

Here we will run the tests under different environments, with or without Swoole.

#1 Run the test suites using PHPUnit:

# To run test suite "automatic":
docker compose exec -ti php    ./vendor/bin/phpunit --testsuite automatic
# or,
docker compose exec -ti swoole ./vendor/bin/phpunit --testsuite automatic

# To run test suite "manual":
docker compose exec -ti php    ./vendor/bin/phpunit --testsuite manual
# or,
docker compose exec -ti swoole ./vendor/bin/phpunit --testsuite manual

#2 Run the test suites using counit (without Swoole):

# To run test suite "automatic":
docker compose exec -ti php    ./counit --testsuite automatic

# To run test suite "manual":
docker compose exec -ti php    ./counit --testsuite manual

#3 Run the test suites using counit (with extension Swoole enabled):

# To run test suite "automatic":
docker compose exec -ti swoole ./counit --testsuite automatic

# To run test suite "manual":
docker compose exec -ti swoole ./counit --testsuite manual

The first two sets of commands take about same amount of time to finish. The last set of commands uses counit and runs in the Swoole container (where the Swoole extension is enabled); thus it's faster than the others:

	Approach	# of Tests	# of Assertions	Time to Finish
counit (without Swoole), or PHPUnit	automatic	16	24	48 seconds
counit (without Swoole), or PHPUnit	manual			48 seconds
counit (with Swoole enabled)	automatic			7 seconds
counit (with Swoole enabled)	manual			7 seconds

Compatibility with PHPUnit

Counit is designed as a drop-in companion to PHPUnit, not a replacement. In short:

When the Swoole extension is not enabled, unit tests written for PHPUnit and/or counit run in PHPUnit and/or counit in exactly the same way, without any changes: every counit API falls back to plain blocking behavior.
Unit tests written for counit run in PHPUnit without any issue, with or without the Swoole extension loaded: counit's coroutine behavior activates only inside the coroutine scheduler that the counit runner itself starts, which plain PHPUnit never does — a loaded-but-idle Swoole extension changes nothing.

That leaves one combination to document: running tests under the counit runner with Swoole enabled — the fast, concurrent mode this package exists for. docs/compatibility.md covers it in full: a matrix of compatible features, a matrix of incompatible ones (each with a Counit 0.x reference column for the PHPUnit ~8.0/~9.0 maintenance line), and the per-feature notes behind every ⚠️/❌ entry.

Additional Notes

Since this package allows running multiple tests simultaneously, we should not use same resources in different tests; otherwise, racing conditions could happen. For example, if multiple tests use the same Redis key, some of them could fail occasionally. In this case, we should use different Redis keys in different test cases. Method \Deminy\Counit\Helper::getNewKey() and \Deminy\Counit\Helper::getNewKeys() can be used to generate random and unique test keys.

The package works best for tests that have function call sleep() in use; It can also help to run some IO related tests faster, with limitations apply. Here is a list of limitations of this package:

The package makes tests running faster by performing time/IO operations simultaneously. For functions/extensions that work in blocking mode only, this package can't make their function calls faster. Here are some extensions that work in blocking mode only: MongoDB, Couchbase, and some ODBC drivers.
The package doesn't work exactly the same as when running under PHPUnit — see Compatibility with PHPUnit for the two feature matrices and the per-feature details. The one rule to remember: a test PHPUnit has already marked "passed" can still fail later under counit, so the exit code of the run — not the summary line alone — is the authoritative pass/fail signal.

Local Development

Docker images used to run the sample tests are built locally via docker-compose.yml, which builds the php and swoole services from the Dockerfiles under ./dockerfiles. See Setup Test Environment.

Alternatives

This package allows to use Swoole to run multiple time/IO related tests without multiprocessing, which means all tests can run within a single PHP process. To understand how exactly it works, I'd recommend checking this free online talk: CSP Programming in PHP (and here are the slides).

In the PHP ecosystem, there are other options to run unit tests in parallel, most end up using multiprocessing:

Process isolation in PHPUnit. This allows to run tests in separate PHP processes.
Package brianium/paratest
Package pestphp/pest

License

MIT license.

deminy / counit

Maintainers

Package info

Statistics

Security