Tutorial

This tutorial covers getting started with split on a Rails 3.0 app.

Installation

First lets create our rails app:

$ gem install rails
$ rails new split-example
$ cd split-example

We'll also need redis (2.0 or greater) installed on for split to store its data.

If you are on a mac, you can install redis using homebrew:

$ brew install redis

And then follow the instructions.

If you are on ubuntu or debian you can install redis using apt-get:

$ sudo apt-get install redis-server

Otherwise you can compile it from source using the instructions on the redis.io website: http://redis.io/download

Note: redis is not officially supported on windows

Then we need to add split as a dependency, so we add the following to our Gemfile:

gem 'split'

If you are using Ruby 1.8.x then you will likely want to also use the SystemTimer gem if you want to make sure the Redis client will not hang.

Put the following in your gemfile as well:

gem 'SystemTimer'

Then update your bundle with:

$ bundle install

Config

Split is designed to work out of the box and requires zero configuration but there are a number of configuration options that can be overridden if you need something off the beaten track.

Robot detection

Split ignores visitors that appear to be robots or spiders to avoid skewing any results, by default compares the useragent of the visit with this regular expression:

/\b(Baidu|Gigabot|Googlebot|libwww-perl|lwp-trivial|msnbot|SiteUptime|Slurp|WordPress|ZIBB|ZyBorg)\b/i

You can override this by putting the following code in config/initializers/split.rb:

Split.configure do |config|
  config.robot_regex = /my_custom_robot_regex/
end

Ignore visits from certain ip addresses

Split also has the ability to ignore visits from a range of ip addresses, you may wish to ignore yourself or people from your office from skewing the results so you can add the following code to config/initializers/split.rb:

Split.configure do |config|
  config.ignore_ip_addresses << '81.19.48.130'
end

Note: config.ignore_ip_addresses expects an array of ip addresses

Redis configuration

By default split will connect to a local redis server on port 6379.

You may want to change the Redis host and port Split connects to, or set various other options at startup.

Split has a redis setter which can be given a string or a Redis object. This means if you're already using Redis in your app, Split can re-use the existing connection.

Some different examples of changing the redis config:

Split.redis = 'redis://localhost:6379'
Split.redis = $redis

For our rails app we have a config/initializers/split.rb file where we load config/split.yml by hand and set the Redis information appropriately.

Here's our config/split.yml:

development: localhost:6379
test: localhost:6379
staging: redis1.example.com:6379
fi: localhost:6379
production: redis1.example.com:6379

And our initializer (config/initializers/split):

rails_root = ENV['RAILS_ROOT'] || File.dirname(__FILE__) + '/../..'
rails_env = ENV['RAILS_ENV'] || 'development'

split_config = YAML.load_file(rails_root + '/config/split.yml')
Split.redis = 'redis://' + split_config[rails_env]

Namespaces

If you're running multiple, separate instances of Split you may want to namespace the keyspaces so they do not overlap. This is not unlike the approach taken by many memcached clients.

This feature is provided by the [redis-namespace][rs] library, which Split uses by default to separate the keys it manages from other keys in your Redis server.

Simply use the Split.redis.namespace accessor:

Split.redis.namespace = "split:blog"

Usage

Defining an A/B test

To define an experiment you use the ab_test method in your controller or views to provide two or more different interfaces that your visitors may see. For example, in your view:

<h2><%= ab_test('form_title', 'Sign up now', 'Join Today') %></h2>

This will set up a new experiment called form_title with two alternatives.

The first alternative ('Sign up now') is the control, it should be the original title that was already on the page and you test new alternatives against that. Do not add any new alternatives as you won't be able to tell if your test improved on the original or not.

The second alternative ('Join Today') will be tested against the first to see if it is an improvement. You can add as many alternatives as you like:

<%= link_to ab_test('link_text', 'Signup', 'Join', 'Become a member', 'Signin', 'Login'), signup_path %>

But remember that it will take longer to reach a good confidence level the more alternatives you provide.

Tracking conversions

To measure how the alternative has impacted the conversion rate of your experiment you need to mark a visitor as reaching the conversion point. You do this with the ab_finished method.

ab_finished('link_text')

Usually you would place this in your controller, for example after a user has been created during the signup process.

Once a user has finished the conversion their split session is reset.

Viewing the results

Dashboard

Split comes with a Sinatra-based front end to get an overview of how your experiments are doing.

You can mount this inside your app using Rack::URLMap in your config.ru

require 'split/dashboard'

run Rack::URLMap.new \
  "/"       => Your::App.new,
  "/split" => Split::Dashboard.new

You may want to password protect that page, you can do so with Rack::Auth::Basic

    Split::Dashboard.use Rack::Auth::Basic do |username, password|
      username == 'admin' && password == 'p4s5w0rd'
    end

You can also mount the dashboards within the rails router.

Change the Gemfile to the following:

gem 'split', require: 'split/dashboard' (For ruby 1.8.*, :require => 'split/dashboard')

Then in config/routes add:

Rails::Application.routes do
  mount Split::Dashboard, at: 'split' (For ruby 1.8.*, :at => 'split')
end