A/B/n testing package for use within the Laravel 5 framework

v3.0.0 2019-03-22 06:32 UTC

This package is auto-updated.

Last update: 2024-03-18 16:52:28 UTC


README

A PHP 7 a/b/n testing package.

Requirements

  • PHP >= 7.0

Installation

The install process is slightly different between Laravel v5.4 and v5.5+ due to the introduction of "package discovery". Refer to the appropriate install section below.

Installation for Laravel 5.5+

  1. Add "heaps-good-services/variant": "^2.0" to your composer.json file.
  2. Run "composer update".

Configuration

Variant requires storage for aggregate results and a cache when executing user experiments but no implementations are included in the package. Implementations of the UserVariationRepository and the CacheInterface will need to be created.

There are both storage mechanisms for aggregate results and user results. The Aggregate results storage is used to store results from system experiments and verified user results. A user store is also required to cache the user results until they meet the verification requirements then the results will automatically be migrated to the aggregate results.

The user results will be most commonly stored in the session and the aggregate results can be stored the database or a persistent cache.

Caveat

If using a service container, you will need to bind an instance of the Experiments class as a singleton to have the experiments persist.

Usage

Experiments are registered in the boot method of the VariantServiceProvider. The experiments are then available for use throughout the application.

Register Experiments:

Register an experiment to test the affect of a piece of text on the conversion rate of a sign up page.

$variant->registerExperiment('sign_up_message')
    ->addVariation('welcome')// A welcoming message.
    ->addVariation('free_ipad')// Free IPad on sign up.
    ->addVariation('not_welcome', 9);// Deter the user from signing up.

Weights

The higher the weight, the higher the likelyhood that specific variation will be selected from the random pool.

...
$variant->registerExperiment('sign_up_message')
    ->addVariation('welcome', 3)
    ->addVariation('free_ipad')
    ->addVariation('deter', 9);
...

The weights correspond to the number of tickets that variation has in the lottery. In the example given above the 'welcome' variation has a 3/13 probabiliy of being chosen.

Experiment Types

There are user and system experiments. User experiments require verification before data is propagated to the aggregate results. System experiments are stored directly in the aggregate results.

To register a system experiment set the 'isSystemExperiment' parameter to true.

...
$variant->registerExperiment('sms_gateway_reliability', true)
    ->addVariation('superstar_sms_company')
    ->addVariation('hindenburg_sms_systems');
...

Usage

The Variant facade is available everywhere using the helper function 'variant()'.

Adding the verification tags

The verification tags help to verify human requests and stop bots from skewing the results.

Interactions

An interaction represents the start of a specific experiment instance. The interactions are limited to one per user experiment regardless of how many times an interaction is triggered.

Conversions

app/Http/Controllers/SignUpController.php

function store(Request $request) {
    ...
    convert('sign_up_message');
    ...
}

The conversions are limited to one per user experiment regardless of how many times a conversion is triggered.

Example System Experiment

app/Factories/SmsGatewayFactory.php

function makeSmsGateway(string $smsGatewayName): SmsGatewayInterface {
    ...
    switch($smsGatewayName) {
        case 'superstar_sms_company':
            return new SuperStarSMSGateway(...);
        case 'hindenburg_sms_systems':
            return new HindenburgSMSGateway(...);
        default:
            ...
    }
    ...
}

app/Jobs/SendSmsMessage.php

function handle(SmsGatewayFactory $smsGatewayFactory) {
    $variation = variant()->getExperiment('sms_gateway_reliability')
        ->getVariation()
        ->addInteraction();
    if($smsGatewayFactory->makeSmsGateway($variation->getName())->send(...)) {
        $variation->addConversion();
    }
}

Notes

The functions 'interact' and 'convert' will suppress exceptions where as the functions 'interact_or_fail' and 'convert_or_fail' will not suppress exceptions.