jamesblackwell/laravel-ab-testing

A/B Testing package for Laravel using Pennant

dev-main 2025-05-01 16:06 UTC

Requires

• php: ^8.1
• laravel/framework: ^10.0|^11.0|^12.0
• laravel/pennant: ^1.0
• markrogoyski/math-php: ^2.0

MIT

• James Blackwell

README

Laravel A/B Testing

A simple, robust A/B testing framework for Laravel using Laravel Pennant.

• Code driven - define experiments using Laravel Pennant. Track views and conversions in your code.
• Zero flicker - determine variants before rendering, no calls to external services.
• Privacy friendly - keep all data on server.
• Admin dashboard for real-time results and statistical significance
• Supports primary and secondary goals - track multiple goals per experiment.

Quickstart

1. Install composer require quizgecko/laravel-ab-testing and run the migrations.
2. Define a Pennant Feature with a 'test' and 'control' variant.
3. Use feature_flag('example-experiment', $userOrUniqueId) to get the variant and display the variation to the user e.g. a different view code block in your views.
4. Use experiment_view('example-experiment', $userOrUniqueId) to track views.
5. Use experiment_conversion('example-experiment', $userOrUniqueId) to track conversions. Optionally track secondary conversions with experiment_secondary_conversion('example-experiment', $userOrUniqueId).
6. View results in admin dashboard at yoursite.com/admin/ab.

Motivation

Laravel Pennant is a great package to handle feature flags, however it still requires you to setup a lot of tracking to run actual A/B tests. This package works on top of Pennant and adds various easy to use helpers to track views and conversions, then displays the results in a dashboard.

Installation

composer require quizgecko/laravel-ab-testing

Publish and run the migration:

php artisan vendor:publish --tag=ab-testing-migrations
php artisan migrate

Publish the configuration file (optional):

php artisan vendor:publish --tag=ab-testing-config

Setup

1. Define Experiments

Define your experiments in a service provider (e.g., AppServiceProvider). Use the Feature facade from Laravel Pennant. Always use a descriptive, kebab-case name including month/year.

Important: currently only two variants are supported and the Feature should return 'test' or 'control'. You can also return 'not-in-experiment' to exclude a user from the experiment. Only variants with 'test' or 'control' are considered in the admin dashboard.

use Illuminate\Support\Facades\Feature;
use Illuminate\Support\Arr;
use App\Models\User;

Feature::define('homepage-signup-copy-april-2025', function ($scope) {
    // Exclude paid users
    if ($scope instanceof User && $scope->is_paid) {
        return 'not-in-experiment';
    }
    // 50/50 split for guests and free users
    return Arr::random(['test', 'control']);
});

• Scope: Use $scope as either a User or a guest ID (like abid()).
• Return values: 'test', 'control', or 'not-in-experiment'.

1. Check Variant in Code

Use the feature_flag() helper to get the assigned variant:

$variant = feature_flag('homepage-signup-copy-april-2025', $userOrUniqueId);
if ($variant === 'test') {
    // Show test variation
} elseif ($variant === 'control') {
    // Show control
} else {
    // Not in experiment
}

1. Track Views and Conversions

Call these helpers at the appropriate places:

experiment_view('homepage-signup-copy-april-2025', $userOrUniqueId); // When user sees the experiment
experiment_conversion('homepage-signup-copy-april-2025', $userOrUniqueId); // When user completes the primary goal
experiment_secondary_conversion('homepage-signup-copy-april-2025', $userOrUniqueId); // For secondary goals

• Always pass the same scope (User or abid) as used in the feature definition.
• Views are only tracked once per user/guest per experiment.
• Conversions are only tracked if a view was previously tracked.

Admin Dashboard

Visit /admin/ab (requires auth and can:viewAdmin) to see:

• All running experiments
• Views, conversions, conversion rates for each variant
• Statistical significance (p-value, confidence)
• Primary and secondary goal tracking

Helper Functions

• feature_flag($experimentName, $scope = null)
• experiment_view($experimentName, $scope = null, $variant = null)
• experiment_conversion($experimentName, $scope = null)
• experiment_secondary_conversion($experimentName, $scope = null)
• abid()

abid

The abid() function returns a unique identifier for the current user or guest. It is used to track views and conversions for anonymous users.

$abid = abid();

For example, if you wanted to test a signup flow, use the abid as the scope:

$variant = feature_flag('homepage-signup-copy-april-2025', abid());

Database

The package creates an experiments table:

• experiment_name (string)
• variant (string)
• total_views (int)
• conversions (int)
• secondary_conversions (int)

Best Practices

• Use descriptive, kebab-case experiment names with month/year.
• Always pass the correct scope (User or abid) to helpers.
• Remove old experiments and helpers when finished.
• Place experiment_view() where the user meaningfully sees the experiment.
• Place experiment_conversion() where the primary goal is completed.
• Remove unused features from Pennant when done to prevent table bloat.
• Delete older experiments from the database to keep it clean.

License

MIT