Today I Learned

A Hashrocket project

27 posts about #testing

RSpec Covers a Range

Today I used a feature of RSpec I've never used before, cover.

cover returns true if it's argument is covered by the given range:

expect(0..1).to cover(order.risk_score)

In my use case, I had an attribute risk_score that is calculated by a third-party API. It should always be between zero and one, but it changes on every test run because the API considers the test user more and more risky each time. cover allowed me to test this attribute, and the underlying logic, in a flexible way.

Append an RSpec Failure Message

Have you ever wanted to customize an RSpec failure message? It's possible, but we lose the original message. What if we want both?

Here's one technique:

begin
  expect(actual).to eq(expected)
rescue RSpec::Expectations::ExpectationNotMetError
  $!.message << <<-MESSAGE

You broke it.

Maybe you deleted the fixtures? Try running `rake db:create_fixtures`.
  MESSAGE
  raise
end

This rescues the failure ExpectationNotMetError, then shovels our HEREDOC string onto the message of $!, a built-in Ruby global variable representing the last error. Then, we raise.

The result is our RSpec error, with the provided message, followed by our custom text.

Cucumber Suite Hooks

Adding hooks that run before and after your RSpec test suite looks like this:

# spec/helper/spec_helper.rb

RSpec.configure do |config|
  config.before(:suite) do
    My::Service.build!
  end

  config.after(:suite) do
    My::Service.tear_down!
  end
end

But for Cucumber, the API isn't quite as obvious. I found some interesting discussion online about this subject, and eventually settled on the following:

# features/support/env.rb

My::Service.build!

at_exit do
  My::Service.tear_down!
end

Anything in features/support/env.rb gets run before the Cucumber test suite, so the first method call functions like a before suite block. at_exit is Ruby Kernel and runs when the program exits. This implementation works roughly the same as a before and after block.

ExtractRspecLet

From the Hashrocket Vault...

Today I got to see the :ExtractRspecLet command from the vim-weefactor plugin. It does what the names suggests, converting this:

# spec/model/foobar_spec.rb

foo = FactoryGirl.create :foobar

To this:

# spec/model/foobar_spec.rb

let(:foo) { FactoryGirl.create :foobar }

It also moved the new let from inside my it block to right underneath my context block. Awesome!

h/t Josh Davey and Dillon Hafer

Capybara Find with Wait

Integration tests covering JavaScript features are a prime location for timing-related flickers. Often our implementation and expectations are correct, but the test gives up and bails before the desired behavior can be observed.

Patience!

With Capybara, one way around this is to just wait a little longer. This code:

find('.mj-modal iframe', wait: 5)

will wait for the iframe up to five seconds, overriding Capybara's default max wait time of two seconds. On certain test runs this can make all the difference.

Partial RSpec argument matches

Sometimes when testing you want to verify that a set of arguments was passed to a method. However, you may not care about all of the arguments, maybe just a subset.

RSpec-Mocks has the idea of partial matching built in.

expect(double).to receive(:msg).with(hash_including(much: "wow"))

Capybara tests on a Rails Engine

If you have a Rails Engine you probably have a dummy app attached in test/dummy or spec/dummy to run and manual test the engine, right?

So you can use this dummy app to have some feature tests with capybara.

The trick is to change the RAILS_ROOT to point to the correct file.

For cucumber tests add the following into features/support/env.rb:

ENV["RAILS_ROOT"] ||= File.expand_path(File.dirname(__FILE__) + '/../../spec/dummy')

For rspec tests add the following into spec/rails_helper.rb:

ENV["RAILS_ROOT"] ||= File.expand_path(File.dirname(__FILE__) + '/dummy')

Rerun Failed Cucumber Tests

An underused trick in Cucumber is the rerun format.

Run your suite like so:

$ cucumber -f rerun -o rerun.txt

This runs your tests with the rerun formatter, sending the output to rerun.txt. After breaking a test, here's my generated file:

$ cat rerun.txt                                                                                                                                                 
features/visitor_views_posts.feature:9

Now we can run again against just the failing example!

$ cucumber @rerun.txt

Use this to drill down through a giant integration test suite.

Full Backtrace in RSpec

Sometimes an RSpec error message can seem overwhelmingly verbose, other times frustratingly terse. Today I learned that this can be customized in Rails, with the following configuration:

# spec/spec_helper.rb
config.full_backtrace = false # or true?

Turn this on, and every failure will include a stack trace. Turn it off, and it won't. You can override a false setting at the command line, like so:

$ rspec spec/some/thing_spec.rb --backtrace 

Great for fast debugging.

See rspec --help for more information.

Cucumber step inception

I learned that you can call a cucumber step inside another one. Super cool!

Image a simple step like this:

Then /^I should see "(.*)"$/ do |text|
  expect(page).to have_content(text)
end

Now you have modals on your web app and you need to restrict this assertion to inside the modal.

So you can start to duplicate the step above to match the within modal and all the ones that could be reusable with modals.

Or you can create a new generic step that scope into the modal and calls your original step, such as:

And /^(.*?) within the modal$/ do |step_definition|
  within ".modal", visible: true do
    step step_definition
  end
end

Now you can call both steps in your gherkin files:

Then I should see "Hello World!"

or

Then I should see "Hello World!" within the modal

Just be careful with generic and reusable steps. The idea is to help developers and not confuse them.

reference: Cucumber wiki - call step from step definition

h/t Taylor Mock

Capybara's RackTest driver limitation

By default, if you click a link through Capybara and it takes you into an external site, the current_url will work fine, but the content on the page does not change.

RackTest is the Capybara's default driver and it has its limitations.

you cannot use the RackTest driver to test a remote application, or to access remote URLs

If you really need to do that know other drivers features and try them.

h/t by Nick Palaniuk

Conveying Intent with RSpec subject

Today during a test refactor, I discovered RSpec's subject method. We used it a lot, because it helps convey the test's intent.

subject can be used as a convenience method, and to define the intent of a test. If you've ever seen a comment # This is what we are testing, subject would be an upgrade to that.

Here are three passing examples of this method in practice, from most to least verbose.

Explicit, with a named subject:

RSpec.describe Array, "with some elements, explicit named subject" do
  subject(:array) { [1, 2, 3] }
  it "should have the prescribed elements" do
    expect(array).to eq [1, 2, 3]
  end
end

Explicit, with no name:

RSpec.describe Array, "with some elements, explicit subject" do
  subject { [1, 2, 3] }
  it "should have the prescribed elements" do
    expect(subject).to eq [1, 2, 3]
  end
end

Implicit:

RSpec.describe Array, "with some elements, implicit subject" do
  it "should have the default elements" do
    expect(subject).to be_empty
  end
end

Substitute this for let on your described class. There are a ton of interesting uses and edges for this method, so dig into those docs.

h/t Mike Chau

Documentation

Test Validation Errors with RSpec

We can use RSpec unit tests to confirm model validations, like this:

# app/models/developer.rb
validates :email, format: { with: /foo/ }

# spec/models/developer_spec.rb
developer.email = 'bar'
expect(developer).to_not be_valid

But there's a false positive here; expect returns true, but we can't be sure why it returned true. It may have been an invalid email, because the regex doesn't match bar, or it may be invalid for many other reasons, such as an incomplete fixture.

Another method I re-learned today is this:

# spec/models/developer_spec.rb
developer.email = 'bar'
expect(developer).to_not be_valid
expect(developer.errors.message[:email]).to eq ['is invalid']

The first expect triggers the validations; the second proves the right error was included. This makes a difference when flash messages or other code depend on your errors stack.

Set custom user agent on #Capybara

Sometimes you want to emulate a request to your site from a device with a specific user-agent. This is particularly useful if you have a different behavior when users use those devices (e.g. wrapped mobile app using Cordova)

Setting the user agent can be accomplished by setting the header on the current session:

Capybara.current_session.driver.header('User-Agent', 'cordova')

Mark Test Pending With xit

Mark any RSpec test temporarily pending by changing it or specify to xit.

Take this test, from the 'Today I Learned' codebase:

# spec/models/channel_spec.rb

describe Channel do
  it 'should have a valid factory' do
    channel = FactoryGirl.build(:channel)
    expect(channel).to be_valid
  end
end

Change it to this:

# spec/models/channel_spec.rb

describe Channel do
  xit 'should have a valid factory' do
    channel = FactoryGirl.build(:channel)
    expect(channel).to be_valid
  end
end

Here's the test output:

$ rspec spec/models/channel_spec.rb:4
Run options: include {:locations=>{"./spec/models/channel_spec.rb"=>[4]}}

Channel
  should have a valid factory (PENDING: Temporarily skipped with xit)

...

Use this when refactoring code that breaks tests, to avoid deleting or commenting-out that hard-won test logic while you get back to green, test by test.

h/t Dorian Karter

Relish

Expect a Case-Insensitive Match

We had a testing issue this week when a button's text save changes! was rendering as SAVE CHANGES!, due to the CSS property text-transform: uppercase. How do you test that?

One technique is to use RSpec's match method, and include case insensitivity with i (explored earlier here):

> button_text = "foo"
=> "foo"
> expect(button_text).to match("FOO")
RSpec::Expectations::ExpectationNotMetError: expected "foo" to match "FOO"
# stuff
> expect(button_text).to match(/FOO/i)
=> true

We ultimately hard coded the expectation to match("FOO"), because allowing fOo and FoO seemed too permissive. But it remains an option.

Select A Select By Selector

Generally when using Capybara to select from a select input, I reference it by its name which rails associates with the label:

select("Charizard", from: "Pokemon")

However, not all forms are going to have a label paired with every select input. We don't want to let our test coverage suffer, so we are going to need a different way to select. Fortunately, Capybara allows us to chain select off a find like so:

find('#pokemon_list').select('Charizard')

Expecting A Selector and Its Content

In integration tests with Capybara, I find myself writing code like this a lot:

within 'h1' do
  expect(page).to have_content('Today I Learned')
end

Today I remembered that there is another way:

expect(page).to have_selector('h1', text: 'Today I Learned')

I prefer this because it captures the scope and the content in one line. RSpec integration tests in particular can get pretty long, and I think this is a good opportunity to keep them concise.

Testing Edit Forms

Today I found a way to assert that an edit form's inputs include a record's saved data. I think it strikes a good balance between broad and narrow scope.

# spec/features/user_edits_kit_spec.rb

within 'form' do
  expect(page).to have_selector("input[value='Default copy.']")
end

This asserts that some content is inside an input field in the form, rather than just anywhere on the page. You can narrow the scope as needed.

RSpec multiple formats with different out buffers

On RSpec there is a way to use multiple formatters from the command line and pipe each into a different output buffer (file/stdout).

To do this use rspec -f p -f j --out result.json spec

-f is alias for --format which takes the arguments

[p]rogress (default - dots)
[d]ocumentation (group and example names)
[h]tml
[j]son

--out will save the last formatter mentioned to a file.

The original formatter will print to $stdout so you end up having the normal display from RSpec and a JSON file which you can use to parse with your own script.

I wish I could cURL that network request

So OK, you've got this webpage that's making an xhr (XML http request) and you really want to iterate on the results you're getting back from that request. cURL seems like the answer there, but sometimes constructing a cURL request on the command line isn't worth the effort.

Enter Chrome.

In the network panel of Chrome's developer tools you can ctrl-click the request in question and choose Copy as cURL. Presto! Paste it into the cmd line and start iterating on that endpoint!

RSpec let!

RSpec's let method comes in two varieties.

let can be used to define a memoized helper method.

let(:current_user) {  FactoryGirl.create :user }

But let is lazily-evaluated, meaning it isn't evaluated until the first time the method is invoked. The value returned will be cached across multiple calls in the same example.

let! forces the method to be evaluated before each example in a before hook.

let!(:current_user) {  FactoryGirl.create :user, active_range: (Time.now)...(Time.now + 1.second) }

You might use `let! when you need a method to be invoked before each example, or when measuring precise time-dependent behavior where lazy evaluation could produce inconsistent results.

Source

Cleanup Cucumber Steps

Code that is never executed is bad. Try this on your brownfield test suite:

$ cucumber --dry-run -f stepdefs

The stepdefs flag reports the location of each step, and also which steps aren't being used by any test. Delete those dusty old steps!

--dry-run skips running the actual steps, a significant time-saver.

RSpec Specify

There are more ways to create a test block with RSpec than it. One is specify, and you can use it to make your tests more readable. Consider this block:

describe 'doing everything' do

  it 'in space' do
    # actions
  end

The sentence 'it in space' is awkward. You can replace it with 'specify', which brings us very close to a grammatically correct sentence. This is Ruby after all; we try to make it readable.

describe 'doing everything' do

  specify 'in space' do
    # actions
  end

RSpec Profile

RSpec includes an easy way to measure test speed, the -p (profile) flag. It takes an argument [COUNT] representing the number of blocks to measure, defaulting to 10.

Here's some sample output:

Top 3 slowest examples (0.03332 seconds, 57.9% of total time):
  Developer should have a valid factory
    0.0199 seconds ./spec/models/developer_spec.rb:4
  Post should have a valid factory
    0.00816 seconds ./spec/models/post_spec.rb:4
  Post should require a body
    0.00525 seconds ./spec/models/post_spec.rb:16

RSpec to HTML

With the RSpec -f (format) and -o (out) flags, and you can generate HTML representations of your test suite.

$ rspec spec/features/my_spec.rb -f html -o test_run.html
$ open test_run.html

Ready for a presentation, email, or archiving.

Assert No Selector

Capybara assertions can be very simple. Today I learned about the assert_no_selector and assert_selector methods.

Then 'assert_no_selector works' do
  assert_no_selector '.nonexistent-text'
end

Then 'assert_selector works' do
  assert_selector '.existent-text'
end

And here's the view that makes them pass:

# app/views/home/index.html.haml
.existent-text

No RSpec required!