Vanity is an Experiment Driven Development framework for Rails.

• All about Vanity: vanity.labnotes.org

• On github: github.com/assaf/vanity

A/B Testing With Rails (In 5 Easy Steps)

Step 1: Start using Vanity in your Rails application:

  == Rails 2.x configuration

  Rails::Initializer.run do |config|
    gem.config "vanity"

    config.after_initialize do
      require "vanity"
    end
  end
  
  == Rails 3 configuration

Add to your Gemfile:

    
  gem "vanity"

If using a relational database, run the generator and migrations to create the database schema:

  $ rails generate vanity
  $ rake db:migrate
  

Add to your application controller:

  class ApplicationController < ActionController::Base
    use_vanity :current_user
  end

Step 2: Define your first A/B test. This experiment goes in the file experiments/price_options.rb:

  ab_test "Price options" do
    description "Mirror, mirror on the wall, who's the better price of all?"
    alternatives 19, 25, 29
    metrics :signups
  end
  
  NOTE: If using a metric as above ("signups"), there needs to be a corresponding ruby file for that metric. Inside the "experiments" directory create a "metrics" directory with a file called "signup.rb". The contents of the file can describe the signup metric, refer to the "Metrics" Vanity documentation page for an example.

Step 3: Present the different options to your users:

  <h2>Get started for only $<%= ab_test :price_options %> a month!</h2>

Step 4: Measure conversion:

  class SignupController < ApplicationController
    def signup
      @account = Account.new(params[:account])
      if @account.save
        track! :signups
        redirect_to @acccount
      else
        render action: :offer
      end
    end
  end

Step 5: Check the report:

  vanity report --output vanity.html
  

Rails 3

  
  There is currently an issue with report generation. The vanity-talk Google Group has a couple posts that outline the issue for now. This is one of the posts: http://groups.google.com/group/vanity-talk/browse_thread/thread/343081a72a0cefb6
  
  If you are collecting data (in development you need to opt-in to this by setting Vanity.playground.collecting = true in environments/development.rb) you can view experiment results with the vanity dashboard instead of the report. 
  
  The vanity dashboard setup instructions with Vanity work for Rails 3.x except the route is different. A Rails 3.x-style route would look like this:
  
    `match '/vanity(/:action(/:id(.:format)))', :controller=>:vanity`

Registering participants with Javascript

If robots or spiders make up a significant portion of your sites traffic they can affect your conversion rate. Vanity can optionally add participants to the experiments using asynchronous javascript callbacks, which will keep almost all robots out. To set this up simply do the following:

• Add Vanity.playground.use_js!

• Set Vanity.playground.add_participant_path = ’/path/to/vanity/action’, this should point to the add_participant path that is added with Vanity::Rails::Dashboard, make sure that this action is available to all users

• Add <%= vanity_js %> to any page that needs uses an ab_test. vanity_js needs to be included after your call to ab_test so that it knows which version of the experiment the participant is a member of. The helper will render nothing if the there are no ab_tests running on the current page, so adding vanity_js to the bottom of your layouts is a good option. Keep in mind that if you call use_js! and don’t include vanity_js in your view no participants will be recorded.

Contributing

• Fork the project

• Please use a topic branch to make your changes, it’s easier to test them that way

• Fix, patch, enhance, document, improve, sprinkle pixie dust

• At minimum run rake test, if possible, please run rake test:all

• Tests. Please. Run rake test, of if you can, rake test:all

• Send a pull request on GitHub

Credits/License

Original code, copyright of Assaf Arkin, released under the MIT license.

Documentation available under the Creative Commons Attribution license.

For full list of credits and licenses: vanity.labnotes.org/credits.html.