Managing Identity

For effective A/B tests, you want to:

  1. Randomly show different alternatives to different people
  2. Consistently show the same alternative to the same person
  3. Know which alternative you’re showing and tracking
  4. When running multiple tests at once, keep them independent

Vanity will assign each visitor a unique identifier and store it in a cookie that persists across sessions. That way, each visitor will get to see the same alternatives on repeating visits. Assuming they use the same browser on all visits.

If you have a better way of tracking visitors, e.g. using sign in, you’ll want to use that instead. That way the same person gets treated to the same experiment, even if they switch between machines and browsers.

You can choose either option using the use_vanity method. In the first case, just call use_vanity from within the ApplicationController. In the second case, you’ll want to pass use_vanity either a block that returns an identity value, or the name of a method that returns an object which provides the identity.

Sounds complications? These two examples are equivalent:

class ApplicationController < ActionController::Base
  use_vanity :current_user
end

class ApplicationController < ActionController::Base
  use_vanity { |c| c.current_user && c.current_user.id }
end

If you use either block or method name and they return nil, Vanity will fallback on persistent cookie mechanism.

An identity can be anything. For example, if you’re running an experiment to test a new feature that will be available in some projects but not others, you’ll want to slice the audience by project identifier, not user ID.

You can also give each experiment a different identity using the identify callback. Here’s an example that tells one experiment to use a project identifier:

ab_test "New feature" do
  description "New feature only available to some projects"
  identify { |c| c.current_project.id }
end