A/B/n testing package for use within the Laravel 5 framework

v2.0.2 2018-08-27 06:56 UTC

README

A Laravel 5 a/b/n testing package.

Requirements

  • PHP >= 7.0
  • Laravel 5.4, 5.5

Installation

The install process is slightly different between Laravel v5.4 and v5.5+ due to the introduction of "package discovery". Refer to the appropriate install section below.

Installation for Laravel 5.5+

  1. Add "heaps-good-services/variant": "^2.0" to your composer.json file.
  2. Run "composer update".
  3. Run 'php artisan vendor:publish --provider="HeapsGoodServices\Variant\Providers\ServiceProvider"' to publish files.
  4. Register the newly published service provider in your config/app.php providers array: App\Providers\VariantServiceProvider

Installation for Laravel v5.4

  1. Add "heaps-good-services/variant": "^2.0" to your composer.json file.
  2. Run "composer update".
  3. Edit your config/app.php providers array to add the service provider "HeapsGoodServices\Variant\Providers\ServiceProvider"
  4. Run 'php artisan vendor:publish --provider="HeapsGoodServices\Variant\Providers\ServiceProvider"' to publish files.
  5. Now also add the newly published service provider in your providers array: "App\Providers\VariantServiceProvider"

Usage

Experiments are registered in the boot method of the VariantServiceProvider. The experiments are then available for use throughout the application.

Register Experiments:

Register an experiment to test the affect of a piece of text on the conversion rate of a sign up page.

$variant->registerExperiment('sign_up_message')
    ->addVariation('welcome')// A welcoming message.
    ->addVariation('free_ipad')// Free IPad on sign up.
    ->addVariation('not_welcome', 9);// Deter the user from signing up.

Weights

The higher the weight, the higher the likelyhood that specific variation will be selected from the random pool.

...
$variant->registerExperiment('sign_up_message')
    ->addVariation('welcome', 3)
    ->addVariation('free_ipad')
    ->addVariation('deter', 9);
...

The weights correspond to the number of tickets that variation has in the lottery. In the example given above the 'welcome' variation has a 3/13 probabiliy of being chosen.

Experiment Types

There are user and system experiments. User experiments require verification before data is propagated to the aggregate results. System experiments are stored directly in the aggregate results.

To register a system experiment set the 'isSystemExperiment' parameter to true.

...
$variant->registerExperiment('sms_gateway_reliability', true)
    ->addVariation('superstar_sms_company')
    ->addVariation('hindenburg_sms_systems');
...

Usage

The Variant facade is available everywhere using the helper function 'variant()'.

Adding the verification tags

The verification tags help to verify human requests and stop bots from skewing the results.

layouts/header.blade.php

<head>
...
{!! variant()->getVerificationTags() !!}
...
</head>

Interactions

An interaction represents the start of a specific experiment instance. The interactions are limited to one per user experiment regardless of how many times an interaction is triggered.

Composed

app/Http/ViewComposers/SignUpViewComposer.php

...
$variationName = interact('sign_up_message');

if($variationName === 'deter') {
    $signUpMessage = 'We don't want you on our site. Leave Now.';
} else if($variationName === 'free_ipad') {
    $signUpMessage = 'Recieve a free IPad with every sign up.';
} else {
    $signUpMessage = 'Welcome to my website. Please sign up below.';
}

$view->with(compact('signUpMessage));
...

In Blade

...
<h4>
@switch(interact('sign_up_message'))
    @case('deter')
        We don't want you on our site. Leave Now.
        @break
    @case('free_ipad')
        Free Ipad with every sign up.
        @break
    @default
        We welcome you with open arms.
@endswitch
</h4>
...

The switch directive is only available from Laravel 5.5. For earlier versions you can do the following:

...
<h4>
@if(interact('sign_up_message') === 'deter')
    We don't want you on our site. Leave Now.
@elseif(interact('sign_up_message') === 'free_ipad')
    Free Ipad with every sign up.
@else
    We welcome you with open arms.
@endif
</h4>
...

Conversions

app/Http/Controllers/SignUpController.php

function store(Request $request) {
    ...
    convert('sign_up_message');
    ...
}

The conversions are limited to one per user experiment regardless of how many times a conversion is triggered.

Example System Experiment

app/Factories/SmsGatewayFactory.php

function makeSmsGateway(string $smsGatewayName): SmsGatewayInterface {
    ...
    switch($smsGatewayName) {
        case 'superstar_sms_company':
            return new SuperStarSMSGateway(...);
        case 'hindenburg_sms_systems':
            return new HindenburgSMSGateway(...);
        default:
            ...
    }
    ...
}

app/Jobs/SendSmsMessage.php

function handle(SmsGatewayFactory $smsGatewayFactory) {
    $variation = variant()->getExperiment('sms_gateway_reliability')
        ->getVariation()
        ->addInteraction();
    if($smsGatewayFactory->makeSmsGateway($variation->getName())->send(...)) {
        $variation->addConversion();
    }
}

Console Commands

There are console commands to provide transparency into the experiments.

List Experiments

List the experiments and their variations.

Command:

php artisan variant:experiments

Output:

+-------------------------+----------------------------------------------+
| Name                    | Variations                                   |
+-------------------------+----------------------------------------------+
| sign_up_message         | welcome,free_ipad,deter                |
| sms_gateway_reliability | superstar_sms_company,hindenburg_sms_systems |
+-------------------------+----------------------------------------------+

Get Statistics

Get the statistics for one or all experiments.

Commands:

# All Experiment Stats
php artisan variant:get-stats

# Single Experiments Stats
php artisan variant:get-stats sms_gateway_reliability

Output for sms_gateway_reliability experiment:

Experiment: sms_gateway_reliability
+------------------------+--------+--------------+-------------+---------------------+
| Variation              | Weight | Interactions | Conversions | Conversion Rate (%) |
+------------------------+--------+--------------+-------------+---------------------+
| superstar_sms_company  | 1      | 1007         | 1005        | 99.8%               |
| hindenburg_sms_systems | 1      | 993          | 202         | 20.3%               |
| Totals                 | 2      | 2000         | 1207        | 60.4%               |
+------------------------+--------+--------------+-------------+---------------------+

Clear Statistics

Clear the statistics for a single experiment.

php artisan variant:clear sms_gateway_reliability

Notes

The functions 'interact' and 'convert' will suppress exceptions where as the functions 'interact_or_fail' and 'convert_or_fail' will not suppress exceptions.