Headless CI with RSpec + Capybara + CruiseControl.rb

I have recently setup a headless continuous integration test server that runs in a headless environment. It runs both my rspec tests and all of my capybara request tests. If I receive a failure in a request test, it will store a screenshot and the HTML code of the page.

There is a few key components to getting this setup correctly, I thought I would share insight on what I needed to do in order to get it working.

My setup consists of:

• A rails app with capybara and rspec tests.
• A headless server which has Xvfb, CruiseControl.rb and Firefox 3.6.x installed.

First off, you will need the following entries in your Gemfile:

``` ruby Gemfile gem “rspec-rails” gem “capybara” gem ‘headless’, :git => “https://github.com/masatomo/headless.git”, :branch => “fix_exit_code” gem “selenium-webdriver”

Now, you will need to configure your `spec_helper.rb` file.

At the top of the file add:

``` ruby spec_helper.rb
def example_filename(metadata)
  [metadata[:file_path].split(Rails.root.to_s)[1].gsub(/[\/\.]/, "_"), metadata[:line_number]]
end

def save_screenshot(example, headless)
  if example.exception && example.metadata[:type] == :request && example.metadata[:js]
    require 'capybara/util/save_and_open_page'
    save_base_path = "#{Rails.root}/tmp"
    metadata = example_filename(example.metadata)
    save_filename = "#{metadata[0]}_#{metadata[1]}"
    FileUtils.mkdir_p("#{save_base_path}/screenshots")
    Capybara.save_page(body, "#{save_filename}.html")
    screenshot_path = "#{save_base_path}/screenshots/#{save_filename}.png"
    if headless
      headless.take_screenshot(screenshot_path)
    elsif Capybara.current_driver == :selenium
      Capybara.page.driver.browser.save_screenshot(screenshot_path)
    else
      #For other drivers that support a plain .render call and only expect the path as a parameter
      #This includes e.g. capybara-webkit
      if capybara.page.driver.respond_to?(:render)
        capybara.page.driver.render(screenshot_path)
      end
    end
  end
rescue => e
  puts "Exception saving screenshot"
  puts e
  puts e.backtrace.join("\n")
end

if ENV['HEADLESS'] == 'true'
  require 'headless'
  headless = Headless.new
  headless.start
  at_exit do
    exit_status = $!.status if $!.is_a?(SystemExit)
    headless.destroy
    exit exit_status if exit_status
  end
end

Selenium::WebDriver::Firefox::Binary.path = ENV["FIREFOX_BIN"] if ENV["FIREFOX_BIN"]

The example_filename method is used to create a custom filename which we use to store screenshots and html dumps from failed request tests.

The save_screenshot method handles saving the screenshots and html dumps.

The next block allows us to run our tests in headless mode. Remember, you will need to have Xvfb installed to run the tests in headless mode.

The last line allows you to specify a custom firefox binary to use. Because we have found latest versions of firefox do not play nice with selenium (which capybara uses), we install firefox 3.6 into ~/firefox-3.6 and use FIREFOX_BIN=~/firefox-3.6/firefox.

Now, inside of your RSpec.configure do |config| block, put the following:

``` ruby spec_helper.rb config.before(:each) do if example.metadata[:type] == :request and example.metadata[:js] Capybara.current_driver = :selenium end end

config.after(:each) do save_screenshot(example, headless) Capybara.use_default_driver end

The first block sets the selenium driver for request tests.  The second
block resets the driver back to a non-selenium driver and also calls the
`save_screenshot` method that we defined earlier.

Now, we will need a custom script that runs our tests in
CruiseControl.rb

``` bash run-tests.sh
#!/bin/bash -l

rvm rvmrc load .rvmrc || exit 1
cp ../database.yml config/
mkdir -p log
RAILS_ENV=test ./bin/rake db:drop db:create db:schema:load && HEADLESS=true ./bin/rspec spec
EXIT_CODE=$?
mv tmp/capybara $CC_BUILD_ARTIFACTS/capybara
mv tmp/screenshots $CC_BUILD_ARTIFACTS/screenshots
exit $EXIT_CODE

We use rvm, so the 3rd line handles loading the correct rvm environment. Then we reset the DB and start our test run.

We also ensure that the test script exits with the right exit code.

Now, we setup out cruise_config.rb file as well.

``` ruby cruise_config.rb Project.configure do |project| project.email_notifier.emails = [‘brian@tecnobrat.com’] project.email_notifier.from = ‘noreply@tecnobrat.com’ project.build_command = ‘bash -l run-tests.sh’

project.scheduler.polling_interval = 1.minutes project.scheduler.always_build = false

project.triggered_by ScheduledBuildTrigger.new(project, :build_interval => 1.day, :start_time => 15.minutes.from_now) project.bundler_args = “–path=#{project.gem_install_path} –gemfile=#{project.gemfile} –no-color –binstubs” end ```

This is a pretty standard cruise_config.rb file only extra thing is that we run the custom bash script we created.

Now cruise control should be able to run your tests, and if any request tests fail, you will get a screenshot and a html dump added to your build artifacts in CC.rb

Some gotchas

• Ensure Xvfb is installed before running with HEADLESS=true
• Use an older version of firefox, it works best with selenium (use FIREFOX_BIN=path/to/bin)
• You need the fix_exit_code branch of headless, so ensure you have specified that in your Gemfile

So when you combine all of that together, you should be able to get a similar setup. If you have any questions please ask them in the comments!