Buckblog: Integration Testing in Rails 1.1

Integration Testing in Rails 1.1

8 March 2006 — Introducing and demonstrating the new integration tests feature in Rails 1.1 — 5-minute read

Integration tests are a new feature of Rails 1.1 that take testing your applications to a new level. They are the next logical progression in the existing series of available tests:

Unit tests are very narrowly focused on testing a single model
Functional tests are very narrowly focused on testing a single constroller and the interactions between the models it employs
Integration tests are broad story-level tests that verify the interactions between the various actions supported by the application, across all controllers

This makes it easier to duplicate (in tests) bugs with session management and routing. Consider: what if you had a bug that was triggered by certain cruft accumulating in a user’s session? Hard to mimic that with functional tests.

For an example, consider a fictional financial application. We have a set of “stories” that describe how the application ought to function:

Bob wants to sign up for access. He goes to the login page, clicks the “signup” link, and fills out the form. After submitting the form, a new ledger is created for him, and he is automatically logged in and taken to the overview page.
Jim, an experienced user, has received a new credit card and wants to set up a new account for it. He logs in, selects the ledger he wants to add the account to, and adds the account. He is then forwarded to the register for that account.
Stacey is a disgruntled user. She has decided to cancel her account. Logging in, she goes to the “account preferences” page and cancels her account. Her data is all deleted and she is forwarded to a “sorry to see you go” page.

Starting with the first one, we might write something like the following. We’ll create this file (“stories_test.rb”) in the test/integration directory:

require "#{File.dirname(__FILE__)}/../test_helper"

class StoriesTest < ActionController::IntegrationTest
  fixtures :accounts, :ledgers, :registers, :people

  def test_signup_new_person
    get "/login"
    assert_response :success
    assert_template "login/index"

    get "/signup"
    assert_response :success
    assert_template "signup/index"

    post "/signup", :name => "Bob", :user_name => "bob",
      :password => "secret"
    assert_response :redirect
    follow_redirect!
    assert_response :success
    assert_template "ledger/index"
  end
end

We can run this by typing rake test:integration, or by invoking the file directly via ruby.

The code is pretty straightforward: first, we get the ”/login” url and assert that the response is what we expect. Then we get the ”/signup” url, then post the data to it, and then follow the redirect through to the ledger.

However, one of the best parts of the integration framework is the ability to extract a testing DSL out of your actions, making it really easy to tell stories like this. At the simplest, we can do that by adding some methods to the test:

require "#{File.dirname(__FILE__)}/../test_helper"

class StoriesTest < ActionController::IntegrationTest
  fixtures :accounts, :ledgers, :registers, :people

  def test_signup_new_person
    go_to_login
    go_to_signup
    signup :name => "Bob", :user_name => "bob", :password => "secret"
  end

  private

    def go_to_login
      get "/login"
      assert_response :success
      assert_template "login/index"
    end

    def go_to_signup
      get "/signup"
      assert_response :success
      assert_template "signup/index"
    end

    def signup(options)
      post "/signup", options
      assert_response :redirect
      follow_redirect!
      assert_response :success
      assert_template "ledger/index"
    end
end

Now, you can reuse those actions in other tests, making your tests very readable and easy to build. But it can be even neater! Taking advantage of the provided open_session method, you can create your own session instances and decorate them with custom methods. Consider this example:

require "#{File.dirname(__FILE__)}/../test_helper"

class StoriesTest < ActionController::IntegrationTest
  fixtures :accounts, :ledgers, :registers, :people

  def test_signup_new_person
    new_session do |bob|
      bob.goes_to_login
      bob.goes_to_signup
      bob.signs_up_with :name => "Bob", :user_name => "bob",
        :password => "secret"
    end
  end

  private

    module MyTestingDSL
      def goes_to_login
        get "/login"
        assert_response :success
        assert_template "login/index"
      end

      def goes_to_signup
        get "/signup"
        assert_response :success
        assert_template "signup/index"
      end

      def signs_up_with(options)
        post "/signup", options
        assert_response :redirect
        follow_redirect!
        assert_response :success
        assert_template "ledger/index"
      end
    end

    def new_session
      open_session do |sess|
        sess.extend(MyTestingDSL)
        yield sess if block_given?
      end
    end
end

The new_session method at the bottom simply uses open_session to create a new session and decorate it by mixing in our DSL module. By adding more methods to the MyTestingDSL module, you build up your DSL and make your tests richer and more expressive. You can even use named routes in your tests to ensure consistency between what your application is expecting and what your tests are asserting!

1234	def goes_to_login get login_url ...end

Note that the new_session method will actually return the new session as well. This means you could define a test that mimicked the behavior of two or more users interacting with your system at the same time:

def test_multiple_users
  jim = new_session_as(:jim)
  bob = new_session_as(:bob)
  stacey = new_session_as(:stacey)

  jim.adds_account(...)
  bob.goes_to_preferences
  stacey.cancels_account
end

private

  module MyTestingDSL
    ...
    
    attr_reader :person

    def logs_in_as(person)
      @person = people(person)
      post authenticate_url, :user_name => @person.user_name,
        :password => @person.password
      is_redirected_to "ledger/list"
    end
    
    ...
  end

  def new_session_as(person)
    new_session do |sess|
      sess.goes_to_login
      sess.logs_in_as(person)
      yield sess if block_given?
    end
  end

Just to further demonstrate how these DSL’s can be built, let’s implement the second of the three stories described at the beginning of this article: Jim adding a credit-card account.

require "#{File.dirname(__FILE__)}/../test_helper"

class StoriesTest < ActionController::IntegrationTest
  fixtures :accounts, :ledgers, :registers, :people

  ...

  def test_add_new_account
    new_session_as(:jim) do |jim|
      jim.selects_ledger(:jims)
      jim.adds_account(:name => "credit card")
    end
  end

  private

    module MyTestingDSL
      ...

      attr_accessor :ledger

      def is_redirected_to(template)
        assert_response :redirect
        follow_redirect!
        assert_response :success
        assert_template(template)
      end

      def selects_ledger(ledger)
        @ledger = ledgers(ledger)
        get ledger_url(:id => @ledger.id)
        assert_response :success
        assert_template "ledger/index"
      end

      def adds_account(options)
        post new_account_url(:id => @ledger.id), options
        is_redirected_to "register/index"
      end
    end

    ...
end

Note individual integration tests run slower than individual unit or functional tests, but that’s because they test so much more. Note that each of the tests shown above test multiple requests. Most functional tests only test one. Also, integration tests run through the entire stack, from the dispatcher, through the routes, into the controller and back. Functional tests skip straight to the controller.

Reader Comments

Jamis, shouldn't all of the multiple user sessions follow Jim's lead and be new_session_as rather than new_session?

  def test_multiple_users
    jim = new_session_as(:jim)
    bob = new_session(:bob)
    stacey = new_session(:stacey)

    jim.adds_account(...)
    bob.goes_to_preferences
    stacey.cancels_account
  end

Also, what do you see the role of regression/acceptance tests (like SeleniumOnRails) being versus the new integration tests? Great article, thanks.

vinbarnes
9 Mar 2006

vinbarnes, good catch! I've fixed it. Regarding automated in-browser testing, I do still think there is a place for that, too. Note that the integration testing framework does not test the Javascript on your page, or verify that an RJS template will really perform as expected. I'd *love* for a way to be found to hook integration tests up to a JS engine and DOM simulator, so it could all be done without requiring a browser, but for now that's a pipe dream. :)

Jamis
9 Mar 2006

You still need browser-based testing even if you have a JS/DOM simulator... the browsers have bugs too! BTW: You've created a beautiful thing! I can't wait to start my new project so I can whip up integration tests first!

Peter Fitzgibbons
9 Mar 2006

"fictional financial application" = BudgetWise? :)

Jacob Fugal
10 Mar 2006

Nice catch, Jacob. ;) BW is still on my radar, just pushed aside for a bit due to other projects.

Jamis
10 Mar 2006

assert_response doesn't work for me. Apparently the assert_response method has access to response but not to @response, so there is an error in the assertions.rb on line 60, when it tries to execute @response.send("#{type}?"). When I do a @response = response just before the assert_response-call everything works fine... (rev 3839)

Michael
11 Mar 2006

Michael, very strange! It seems to be working fine for me... Can you email me one of your tests? ([email protected])

Jamis
11 Mar 2006

Ah, great piece of software... I'm pretty glad to see that with ruby you can create astonishing software, which's code looks wonderful.

Piotr Usewicz
11 Mar 2006

Jamis, these "integration" tests are really nice and I think they hit a sweet spot in between exercising a large portion of your application without requiring too much testing infrastructure. These tests are certainly more functional tests than the so called functional tests are! (They are actually unit tests for a controller.) The Java community also started going down the path of full on browser simulation but have lately started abandoning it. The problem of course is that writing a complete browser is not that easy! So you end up getting bugs in the browser "simulation" that are different from the bugs in the browsers!

Jon Tirsen
12 Mar 2006

A built in automated in-browser testing will complete Ruby testing stack. It will be a great day when Ruby has the build-in equivalent of Watir/IEUnit for both IE and Firefox.

[email protected]
28 Mar 2006

I would like to simulate hitting my application with different host names. I can do this with functional tests by calling @request.host = 'whatever.com'. What's the best way to implement this in an Intregration Test?

[email protected]
28 Mar 2006

Keith, if you are using an "implicit" session, you can just do:

123	def test_something host! "foo.bar.com"end

If you are using explicit sessions, you can use the @host=@ accessor of the session:

1234	def test_something_else sess = open_session sess.host = "foo.bar.com"end

Jamis
29 Mar 2006

As of the 1.1 release, can you really use assert_template in integration tests? Here's my test: def test_signup get "/general/index" assert_response :success assert_template "/general/index" end When run, it produces: expecting <”/general/index”> but rendering with <”/usr/local/lib/ruby/gems/1.8/gems/actionpack-1.12.0/lib/action_controller/templates/rescues/unknown_action.rhtml”> The url ”/general/index” works fine in the browser, but the integration tests fail on the template assertion. Any ideas?

Tom Werner
31 Mar 2006

Hi Jamis - many thanks for this great introduction - I think I understand a lot of what you are describing, but I haven't come across the term 'DSL' before. A quick google hints at 'Domain Specific Language' - is this what you mean? If so, I'm guessing the term is appropriate because you are defining methods/helpers specific to the test class/suite. Newbie question I know, just wanting to get my foundations right before leaping any further. Thanks once again, Keeran

Keeran
4 Apr 2006

Tom, are you inheriting from ActionController::IntegrationTest? If not, you're probably getting the _other_ get method--the one defined for functional tests, and it won't like the "/general/index" URI, because it expects it to be the name of an action. Keeran, yup, DSL stands for Domain Specific Language.

Jamis
4 Apr 2006

The thing I'm struggling with is figuring out what the full response is to my tests. For example, I get a 200 instead of the expected re-direct. test.log does not have enough detail (ie, html of response or similar) for me to quickly track down what is going on. What other approaches are folks using. Is there a way to occasionaly dump the full response body to an integration test out somewhere when debugging? It would make the test enviroment a bit less of a black box if I could reach into it a bit more.

August
4 Apr 2006

August, to get the full response body, simply inspect response.body.

Jamis
4 Apr 2006

Jamis, As you say, the integration tests won't cover everything you need for your app, and there are some situations where you can't avoid a thorough scripted walk through the GUI. [Selenium](http://www.openqa.org/selenium/) is pretty much the best thing I've used in the past (and _will_ work with AJAX, you just have to modify some of the Selenium prototypes for it to handle the asynchronicity of it all). Although that's good, it does mean you have to fire up a browser etc., you may want to take a look at [Selenium on Rails](http://andthennothing.net/archives/2006/02/19/new-version-of-selenium-on-rails). Note, that although I've not tried it, it was thoroughly recommended by an [ex-colleague](http://blog.joshchisholm.com) (and even more of a testing lover than myself):

Paul Ingles
6 Apr 2006

Has there been any reported trouble with integration testing and render_components? My app has been working awesome for months, but my first integration test returns a "No action responded" in the response body on a call to a render_component.... Looks awesome though.

nate at inklingmarkets.com
7 Apr 2006

answered my own question. i see the patch: http://dev.rubyonrails.org/ticket/4632

nate at inklingmarkets.com
7 Apr 2006

Thanks for the great writeup. I am using ordered fixtures to ensure that rake loads my fixtures in order (otherwise has_many relationships will not link to correct record).

--- !omap
- zack:
   username: zackchandler
- franko:
   username: franko

The problem is that now I can't reference zack = new_session_as(:zack) as in your example. Any ideas?

Zack Chandler
26 Apr 2006

Hi, is there a way to specify the request environment variable HTTP_REFERER? I'm using redirect_to :back and can't find a way to set @request.env["HTTP_REFERER"] in my integration test.

Stefan
7 May 2006

So, integration tests does not replace functional test? From the examples above, it looks like it's limited to assert_response and assert_template checks. I tried inspecting the assigns() but it's always nil.

Ronaldinho
7 May 2006

Jamis, I just decided to dive into integration tests, but am having trouble building a login_as method for my users. I store the passwords hashed in the database, so there's no way to login_as :jchris without having to specify the password each time. I like the way acts_as_authenticated works for this, but I need a way to edit the session hash directly. It seems non-obvious. I'll probably have it figured out soon, but I just wanted to note that that might be a good example to have right up front. Chris

Chris Anderson
12 May 2006

Did Chris or anyone else manage to find a way to access passwords in an integration test when using acts_as_authenticated?

Rhyee
18 May 2006

I encountered a problem where I can't use this exact login method because my passwords are stored in hashed format. What would be the easiest way to cheat and modify the session variables manually?

deadsouls
31 Aug 2006

LOL! I just noticed all the other comments asking the exact same question. Does anyone know the answer to this?

deadsouls
31 Aug 2006

deadsouls, it's easy. Make the login method accept the hashed password as well and post to login with the username and the hashed_password. If you want extra security check in the login method of your controller that RAILS_ENV is set to 'test' to allow the hashed_password instead of the password.

s1lence
10 Sep 2006

After reading your article, I have to change my habits and do my "programming language talk" in Ruby. So, I can now ask my boss to express his needs in a DSL--- somehow an ad-hoc language :-) .

--- I'm still in the Java camp but I have to recongnise that what I saw it's really beautiful.

So thanks a lot, Jamis! Best regards from Romania!

Gerard Dragoi
13 Sep 2006

Hi Jamis,

I don’t know if you’re still checking these comments… trying to apply this pattern, I’m getting a weird error, pasted here:

http://pastie.caboo.se/27278

A global search and replace in vendor/rails/actionpack/lib/action_controller/integration.rb fixes the problem, e.g.: - attr_accessor :accept + attr_accessor :http_accept

The Accept header to send.

For some reason, it seems accept is not being interpreted as an attribute but as a function. I’d appreciate any pointers you can give :)

Daniel Haran
12 Dec 2006

The Buckblog

assorted ramblings by Jamis Buck

Integration Testing in Rails 1.1

Reader Comments