Speeding Up Ruby Tests

Test-driven development (TDD) aims at creating stable projects that survive changes over time. By following best practices of TDD, it is possible to stay on the safe side and ensure proper operation of the system. One of the most important things in this process is getting test results as fast as possible (ideally right after the source code changes). However, there are some routines that slow down testing in Ruby.

In this post, we provide recommendations on how to save some precious seconds when you start Ruby-on-Rails tests. You will learn how to avoid re-starting tests each time a change to a file is made, as well as how to check the test coverage of your app.

Profiling

The easiest way to improve testing speed is to find the slowest tests and optimize them. To do this, you need to add the --profile option to you command line / .rspec file. After successful test execution, it will print 10 slowest tests.

Spork

Unfortunately, test optimizing doesn’t solve the problem with slow Rails startup. For example, for LVEE tests, it adds 15 seconds to testing time (all tests pass in 35 seconds).

To solve this problem, Spork was created. It preloads an application and runs tests in this state.

To integrate Spork with your Ruby (Rails) application, you need to add

gem 'spork' #gem "spork-rails"

to your Gemfile file, execute bundler install, and bootstrap config with the following.

spork rspec --bootstrap # if you use RSpec for your tests
spork cucumber --bootstrap  # if you use Cucumber for your tests

For RSpec, it will generate the config illustrated below (some comments ommited).

require 'rubygems'
require 'spork'
#uncomment the following line to use spork with the debugger
#require 'spork/ext/ruby-debug'

Spork.prefork do
  # Loading more in this block will cause your tests to run faster. However,
  # if you change any configuration or code from libraries loaded here, you'll
  # need to restart spork for it take effect.

end

Spork.each_run do
  # This code will be run each time you run your specs.

end

# Your spec_helper.rb content here

Here, the Spork.prefork part will be run only once (during Spork loading) and Spork.each_run will be executed during each test restart. So, it’s good idea to move as much as possible to the prefork block to speed up test running.

For Rails (in addition to your spec_helper content), you will need the following input (some lines depend on your gems).

Spork.prefork do
  # Preloading Rails
  require "rails/application"
  # This line required to proper routes reloading in Rails 3.1+
  Spork.trap_method(Rails::Application::RoutesReloader, :reload!)

  # Preloading your application
  require File.dirname(__FILE__) + "/../config/environment.rb"

  # Preloading RSpec
  require 'rspec/rails'

  require 'shoulda/matchers/integrations/rspec' # after require 'rspec/rails'

  RSpec.configure do |config|
    # Your RSpec config here
  end
end

Spork.each_run do
  # Reloading support files each run
  Dir[Rails.root.join("spec/support/**/*.rb")].each {|f| require f}

  # Reload FactoryGirl2 factories
  FactoryGirl.reload

  # Reload locales
  I18n.backend.reload!
end

Note: If you change your initializer scripts, you need to reload Spork.

To learn more about Spork, check out the project’s GitHub repo.

Guard

Now, we can start our tests really fast, but after each file change we need to rerun tests (from command line or using key binding), and it’s boring!

Guard automatically tracks changes in your project and runs tests or reloads Spork if the initializer changes. It can also run bundler if Gemfile changes.

To add it to your application, you need to modify your Gemfile.

group :development do
    gem "guard"
    gem "guard-rspec" # plugin to automate RSpec testing
    gem "guard-cucumber" # plugin to automate Cucumber testing
    gem 'guard-spork' # plugin to automate Spork reload
    gem 'guard-bundler' # plugin to automate Bundler tasks
  end

For a typical Ruby-on-Rails application, you can use the configuration shown below.

# Guardfile

guard 'bundler' do
  watch('Gemfile')
  # Uncomment next line if Gemfile contain `gemspec' command
  # watch(/^.+\.gemspec/)
end

guard 'spork', cucumber_env: { 'RAILS_ENV' => 'test' }, rspec_env: {'RAILS_ENV' => 'test' } do
  # Reloading Spork if any of following files changed
  watch('config/application.rb')
  watch('config/environment.rb')
  watch('config/environments/test.rb')
  watch(%r{^config/initializers/.+\.rb$})
  watch('Gemfile')
  watch('Gemfile.lock')
  # Reload and run specific test type
  watch('spec/spec_helper.rb') { :rspec }
  watch(%r{features/support/}) { :cucumber }
end

guard 'rspec', cli: '--drb --format Fuubar --color --profile' do
  # Rerun all tests
  watch(%r{^spec/.+_spec\.rb$})
  # Rerun tests with specific name
  watch(%r{^lib/(.+)\.rb$})     { |m| "spec/lib/#{m[1]}_spec.rb" }
  watch('spec/spec_helper.rb')  { "spec" }

  watch(%r{^app/(.+)\.rb$})                           { |m| "spec/#{m[1]}_spec.rb" }
  watch(%r{^app/controllers/(.+)_(controller)\.rb$})  { |m| ["spec/routing/#{m[1]}_routing_spec.rb", "spec/#{m[2]}s/#{m[1]}_#{m[2]}_spec.rb", "spec/acceptance/#{m[1]}_spec.rb"] }
  watch(%r{^spec/support/(.+)\.rb$})                  { "spec" }
  watch('config/routes.rb')                           { "spec/routing" }
  watch('app/controllers/application_controller.rb')  { "spec/controllers" }
end

guard 'cucumber', cli: "--drb" do
  watch(%r{^features/.+\.feature$})
  watch(%r{^features/support/.+$})          { 'features' }
  watch(%r{^features/step_definitions/(.+)_steps\.rb$}) { |m| Dir[File.join("**/#{m[1]}.feature")][0] || 'features' }
end

These Guard plugins are smart enough to rerun only changed and failed tests first. If tests are fixed, the plugins will run the whole test suite.

There is bunch of the Guard plugins to automate all the aspects of development. Here are some of them:

• guard-rails automatically reloads an application if required (e.g., initializers/configuration/library/gems were changed).
• guard-livereload automatically reloads a browser if you changing the view.
• guard-jekyll helps you to write posts to this blog 🙂
• guard-jasmine automatically runs Jasmine tests.

Bonus: Tests coverage

Tests coverage is very useful if you practice TDD: it helps to detect parts of the code that were not checked by tests.

For Ruby 1.9.3, the best gem to generate profiling information is simplecov. This gem generates pretty HTML reports.

To integrate it with your test, you need to add the following input.

require 'simplecov'
SimpleCov.start 'rails'

If you use Spork, you need to add simplecov in a bit different manner.

Spork.prefork do
  unless ENV['DRB']
    require 'simplecov'
    SimpleCov.start 'rails'
  end

  # other code ...
end

Spork.each_run do
  if ENV['DRB']
    require 'simplecov'
    SimpleCov.start 'rails'
  end

  # other code ...
end

This is required, because Spork works internally (using a fork), so the results will be inconsistent unless we start it in the each_run section.

Further reading

• Let’s Test It Well: Simply and Smartly
• How to Integrate Independent QA Testing to Shorten Development Cycles
• Running Capybara Tests in Remote Browsers