keboola / phpunit-retry-annotations
Traits for retrying test methods and classes in PHPUnit
Requires
- php: ^7.4|^8.0
- ext-dom: *
Requires (Dev)
- phpcompatibility/php-compatibility: ^9.3
- phpstan/phpstan: ^1.6
- phpstan/phpstan-phpunit: ^1.1
- phpunit/phpunit: ^9.3
- squizlabs/php_codesniffer: ^3.6
This package is auto-updated.
Last update: 2024-09-08 10:43:02 UTC
README
Traits for retrying test methods and classes in PHPUnit.
Installation
composer require --dev keboola/phpunit-retry-annotations
Configuring retries
The default retries count is set to 3, when you want to change default retries for all tests copy&paste file phpunit-retry.xml.dist
from this library into the root directory
of your application and change value of baseRetryCount
Retry using a specified number of retries
/** * @retryAttempts 2 */ class MyTest extends PHPUnit\Framework\TestCase { use PHPUnitRetry\RetryTrait; public function testSomethingFlakeyTwice() { // Retry a flakey test up to two times } /** * @retryAttempts 3 */ public function testSomethingFlakeyThreeTimes() { // Retry a flakey test up to three times } }
NOTE: "Attempts" represents the number of times a test is retried. Providing "@retryAttempts" a value of 0 has no effect, and would not retry.
Retry until a specific duration has passed
/** * @retryForSeconds 90 */ class MyTest extends PHPUnit\Framework\TestCase { use PHPUnitRetry\RetryTrait; public function testSomethingFlakeyFor90Seconds() { // retries for 90 seconds } /** * @retryForSeconds 1800 */ public function testSomethingFlakeyFor30Minutes() { // retries for 30 minutes } }
Configuring retry conditions
Retry only for certain exceptions
By default, retrying happens when any exception other than
PHPUnit\Framework\IncompleteTestError
and PHPUnit\Framework\SkippedTestError
is thrown.
Because you may not always want to retry, you can configure your test to only retry under certain conditions. For example, you can only retry if your tests throw a certain exception.
/** * @retryAttempts 3 * @retryIfException MyApi\ResourceExhaustedException */
You can retry for multiple exceptions.
/** * @retryAttempts 3 * @retryIfException MyApi\RateLimitExceededException * @retryIfException ServiceUnavailableException */
Retry based on a custom method
For more complex logic surrounding whether you should retry, define a custom retry method:
/** * @retryAttempts 3 * @retryIfMethod isRateLimitExceededException */ public function testWithCustomRetryMethod() { // retries only if the method `isRateLimitExceededException` returns true. } /** * @param Exception $e */ private function isRateLimitExceededException(Exception $e) { // Check if HTTP Status code is 429 "Too many requests" return ($e instanceof HttpException && $e->getStatusCode() == 429); }
Define arbitrary arguments for your retry method by passing them into the annotation:
/** * @retryAttempts 3 * @retryIfMethod exceptionStatusCode 429 */ public function testWithCustomRetryMethod() { // retries only if the method `exceptionStatusCode` returns true. } /** * @param Exception $e */ private function exceptionStatusCode(Exception $e, $statusCode) { // Check if HTTP status code is $statusCode return ($e instanceof HttpException && $e->getStatusCode() == $statusCode); }
Configuring delay
Delay for a duration between each retry
/** * @retryAttempts 3 * @retryDelaySeconds 10 */
Delay for an amount increasing exponentially based on the retry attempt
/** * @retryAttempts 3 * @retryDelayMethod exponentialBackoff */
The behavior of the exponentialBackoff
method is to start at 1
second and increase to a maximum of 60 seconds. The maximum delay can be
customized by supplying a second argument to the annotation
/** * This test will delay with exponential backoff, with a maximum delay of 10 minutes. * * @retryAttempts 30 * @retryDelayMethod exponentialBackoff 600 */
Define a custom delay method
/** * @retryAttempts 3 * @retryDelayMethod myCustomDelay */ public function testWithCustomDelay() { // retries using the method `myCustomDelay`. } /** * @param int $attempt The current test attempt */ private function myCustomDelay($attempt) { // Doubles the sleep each attempt, but not longer than 10 seconds. sleep(min($attempt * 2, 10)); }
Define arbitrary arguments for your delay function by passing them into the annotation:
/** * @retryAttempts 3 * @retryDelayMethod myCustomDelay 10 60 */ public function testWithCustomDelay() { // retries using the method `myCustomDelay`. } /** * @param int $attempt The current test attempt. * @param int $multiplier Rate of exponential backoff delay. * @param int $maxDelay Maximum time to wait regardless of retry attempt. */ private function myCustomDelay($attempt, $multiplier, $maxDelay) { // Increases exponentially sleep(min($attempt * $multiplier, $max)); }