Winston

Winston is a AB/split testing library which utilizes Redis and basic machine learning. At it's core, Winston is a highly configurable, roll-your-own A/B testing tool. Winston comes with several flavors of AB testing out of the box based on user-defined configuration options. It supports machine learning so you can set and forget variations and let Winston determine the best variations to run. Winston also supports client side javascript events as well as events you can trigger in your PHP code.

Fork me on GitHub

Requirements

Winston has a base set of requirements for you to use it.

  • PHP 5.3+
  • Redis must be installed and accessible.
  • Composer is required for loading dependencies such as Predis, a popular PHP Redis client.
  • You must create server side API endpoints in your framework or custom rolled application for Winston to be able to interact with the server side Winston library. These endpoints will need to take in POST data, load the Winston library, and pass in the POST data to the Winston library. More documentation to come.

Winston and Machine Learning

You can optionally tell Winston whether you'd like to enable machine learning algorithm or not. If it's enabled, Winston performs the following:

  • It first checks if a test is reliably a favorite via confidence intervals.
  • If a test has no clear favorite, Winston decides on one of the following execution paths:
    • It falls back to picking a random test variation a certain percentage of the time (defaults to 10%)
    • It picks the current top performing variant the remaining percentage of the time (defaults to 90%).
  • If no test variations have any data collected, Winston will always pick at random.
  • The goal of Winston is to take the guess work out of displaying a top performing test variation. It's a set and forget operation on your part and Winston takes care of the rest. The default random percentages are entirely configurable.

Usage

Winston installation and setup is comprised of three core parts. Below is an overview of the three parts followed by details on configuring each of the parts.

1. Configuration

You need to setup your Winston configuration file settings to your liking and create tests and variations. You can store your configuration settings however you want (array, JSON, Yaml, database, key/val store) so long as you can convert the settings to a properly formatted array when initializing Winston.

Winston requires a fairly lengthy configuration array of settings, tests, and test variations. For a full picture of what a configuration array looks like, check out the basic example config file:

https://github.com/popdotco/winston/blob/master/examples/config.php

2. Client Side Code

You need to add code to your client facing frontend website to display variations and track performance.

The example below uses short tags, but you don't have to if they aren't enabled. In this example, we're checking to see if varying the page's headline/tagline has any affect on the frequency of clicking a button directly below it.

3. Server Side Code

You need to create a new server side file (a controller route and action if you use MVC) which you grant Winston access to POST data to. There are two specific API endpoints you'll need for Winston which ultimately load an instance of Winston and call the following:

$winston->recordEvent($_POST);
$winston->recordPageview($_POST);

The example below only contains minimal routing support to give you an idea for how to tie in the endpoints from the config file.

Supported Client Side Events

Winston supports triggering variation successes for all of the popular DOM events, however we suggest steering clear of mouse movement events given how frequently they trigger. The full list of supported events is click, submit, focus, blur, change, mouseover, mouseout, mousedown, mouseup, keypress, keydown, and keyup. Note that we do not handle preventing default event actions or stopping propagation. If you need that, add your own additional event bindings to the element(s).

To trigger an event in your client side code, simply call: $winston->event('name-of-your-test', EVENT_TYPE); where EVENT_TYPE is one of the events mentioned above. This method will then generate and return a DOM event string for you to output directly in your HTML, i.e.

Let's now bind a form submission event directly to a form as an example which will get attributed to the chosen variation. The order in which you call event() and variation() doesn't matter.

Adding Events Within Variations via Templating

With Winston, you can add event bindings directly within your variation text/html. In each variation, you can use the syntax {{EVENT_NAME}} where EVENT_NAME is one of the supported client events found in the section above. Winston will internally find and replace these matching template strings with DOM event handlers. If the JavaScript event is triggered, the currently selected variation will trigger successfully and an AJAX request will fire to your backend indicating the success.

Here's an example of a test you can setup in your configuration file which utilizes the basic template engine:

Redis Setup

Improve Default Redis Persistence

Redis is an in-memory key/value store. It's default configuration is to save snapshots of your data every 60 seconds or every 1000 keys changed. Because of this, you risk data loss if any of the following were to occur:

  • Redis fails/stops, i.e. if you run out of memory
  • A power outage occurs without a UPC
  • The machine crashes/restarts
  • If you run out of disk space

If you can't tolerate losses of this magnitude and are willing to sacrifice a bit write speed, you'll want to enable Append-only file data persistence in your redis configuration file. You can modify your config file, generally found in /etc/redis/redis.conf or /etc/redis.conf:

Before updating your redis.conf file, you'll want to first read the guide below to backup your existing Redis database via an RDB dump to ensure no data is lost during the transition.

You can read more about Redis persistence and configuration options here.

Securing Redis

You will likely want to increase the default security measures/precautions of your Redis install.

  1. Set up firewall rules (i.e. IPTables) to only allow certain machines to access the Redis port, i.e. 127.0.0.1 for the local machine or 192.168.1.X for a machine within your same subnet. Likewise, you'll want to edit your redis.conf file and add bind XXX.XXX.XXX.XXX with your allowed IP or IPs. If you need remote access, you can use bind 0.0.0.0, but only if you also create firewall rules to whitelist machines and grant them access to port 6379.
  2. Enabling password authentication is highly recommended as Redis defaults to no password with full access to all commands. Winston supports Redis authentication in the configuration file by adding 'auth' => 'yourredispassword'.

You can read more about Redis security and configuration options here.

Downloads

Downloads are available via github. The decision is all yours:

  • git clone git@github.com:popdotco/winston.git
  • git clone https://github.com/popdotco/winston.git
  • wget https://github.com/popdotco/winston/archive/master.zip
  • wget https://github.com/popdotco/winston/archive/master.tar.gz
Download as .zip Download as .tar.gz

Support

If you have any problems with GoogleAuthenticatorRedux, please file a ticket/issue/bug on Github and we will attempt to address it at my earliest convenience.

Winston Issues on Github

License

Winston is licensed under the MIT License.

The MIT License is simple and easy to understand and it places almost no restrictions on what you can do with Double Rainbow.

You are free to use Double Rainbow in commercial projects as long as any copyright headers and license file are left intact.

Changelog

  • Aug 14, 2014
    Removing debugging logs. Official beta release.
  • Jan 6, 2014
    Official alpha release.
  • Jan 3, 2014
    Initial commit of the incomplete library.

About the Author

is a full-stack web applications developer in Charlotte, NC with 9+ years professional experience. He holds a bachelors degree in Computer Science and has been working remotely since 2012. He specializes in LAMP/LEMP stack development with Laravel and WordPress. Corey is the owner and principal consultant at Craft Blue, a custom web applications development consultancy. He's also the co-organizer of the Queen City PHP meetup group in Charlotte. He is an entrepreneur, blogger, open source contributor, beer lover, startup advocate, chicken wrangler, hydroponics gardening dabbler, and homebrewer.

Corey works with agencies, startups, and businesses.

Contact Corey to see how Craft Blue can help you.