Managing Identity

For effective A/B tests, you want to:

  1. Randomly show different alternatives to different people
  2. Consistently show the same alternative to the same person
  3. Know which alternative you’re showing and tracking
  4. When running multiple tests at once, keep them independent

Vanity can assign each visitor a unique identifier and store it in a cookie that persists across sessions. That way, each visitor will get to see the same alternatives on repeating visits. (Assuming they use the same browser on all visits.)

If you have a more persistant way of tracking visitors, e.g. using sign in, you’ll want to use that instead. That way the same person gets treated to the same experiment, even if they switch between machines and browsers.

You can choose either option using the use_vanity method. In the first case, just call use_vanity from within the ApplicationController. In the second case, you’ll want to pass use_vanity either a block that returns an identity value, or the name of a method that returns an object which provides the identity when id is called on it. If you pass a method name and it returns nil, Vanity will fallback on the persistent cookie mechanism.

Sounds complicated? These two examples are equivalent:

class ApplicationController < ActionController::Base
  use_vanity :current_user

  def current_user
    User.find(session[:user_id])
  end
end

class ApplicationController < ActionController::Base
  use_vanity :current_user

  def current_user
    user = User.find(session[:user_id])

    user if user && user.id
  end
end

If current_user does not return nil for non-persisted accounts, e.g. it returns a NullUser, the id call should return nil.

If you pass a block, it must return a non-nil value. For example:

class ProjectController < ApplicationController
  use_vanity { |controller| controller.params[:project_id] ||  }
end

An identity can be anything. For example, if you’re running an experiment to test a new feature that will be available in some projects but not others, you’ll want to slice the audience by project identifier, not user ID.

You can also give each experiment a different identity using the identify callback. Here’s an example that tells one experiment to use a project identifier:

ab_test "New feature" do
  description "New feature only available to some projects"
  identify { |c| c.current_project.id }
end