Beyond Testing: Specs and Behaviour Driven Developmentrabble - evan henshaw plathblog - anarchogeek.comwork - entp.com

explicar quien soy

First Tests

First before we can talk about beyond testing we should talk about testing itself.

QA Testing

The first thing you think of when you hear testing is Quality Assurance, the QA test. A person sits and clicks on all the links and buttons. You know everything work if you don’t have a fatal crash.

It’s very useful to know when there are huge problems, but it’s an issue for those other people. But we aren’t other people, we’re programming, and for us the QA test is just extra work we don’t like. A huge list of bugs.

But there’s another kind of testing, the type which is in code. All the types of testing in code is to verify the functionality of an application automatically.

Regression Testing

But there’s another kind of testing, the type which is in code. All the types of testing in code is to verify the functionality of an application automatically.

Writing code to verify code. It’s recursive. You’ve got your application code, then tests to verify the application, then tests of the testing library, etc...

From the perspective of the programmer, regression testing is much more useful. It’s the process by which we can design the implementation of an application.

Unit Testing

The most common type of regression testing is the great UNIT TEST. Normally when you hear people talk about testing beyond QA they’re talking about unit tests.

What’s a unit? It’s the smallest piece of functionality of your application. It could be a method, or a class. Or perhaps a unit is a single line of code.

Testing forApplication Design

One thing which is distinctly different about regression testing vs QA testing is that regression testing is a technique for application design. Not designing the user interface but designing the code itself.

With testing we can come to a better understanding of how the parts of our application work. Once we know the parts we can use that to design the whole that emerges.

TDD - Test Driven Development

What is this TDD thing? It’s a philosophy of how we design code. Not the graphic design or interaction design of the interface, but the code, methods, classes, api, and the like. It’s important to know before we get in to TDD what it’s reacting to. The idea of the waterfall.

The waterfall method. In reality it’s the method which is still taught in universities, and which is certified by the ISO. It’s the core design philosophy of ENTERPRISE software.

The idea is we have a project, we do a planning process, create a map, a specification of how all of the classes and objects are going to work, design the whole API on paper. Then with this sacred document we can start to code based on gods plan.

The picture before is of Iguazú, a falls at the border of Argentina, Brazil, and Paraguay. But it’s not fair to use it. Because in reality even the waterfall method is about iteration. So perhaps it looks a little more like this. Each version’s release is followed by another planning stage. The idea is to make them short, but by short they mean a year or two.

But the process of software engineering is distinctly different from building a bridge. Every application is different. There are only a limited number of kinds of bridges. Sure there are differences, but they are variations on a theme. With software, by definition we are building something new every time. Because if it wasn’t new, we’d just reuse the old library or applicaiton.

Software is more like a puzzle. Something complicated, where we can never really know all the details. It’s also a continuous process, you’re not ever done. It’s telling that we say a software project is dead when it stops getting modified.

It’s perhaps better to think of software development as a river. The river is something which is alive. Sometimes it’s calm and slow moving, other times there are whitewater rapids, but it always is changing. Eventually the river reaches the ocean and dies. Living software is in a constant state of change. The architecture which was appropriate for one stage of an application’s life some times does not serve the next.

TDD - Test Driven Development

So with this we return to TDD. The concept is that we do regression tests, tests in code itself, as a way of helping us do software development.

TFD - Test First Developmentbefore or after?

When we talk about TDD a related concept often arises. When do we write the tests. Before, after, while coding, or while debugging? The reality is that it’s not that important the exact order of things. What is important is that testing is integrated in to the task of programming and not separated out. You can’t say, ok now we’re going to do 2 weeks of only writing tests.

Ok, so i’ve been saying that TDD is a better way of designing software. Now let’s get on to the details. What exactly does a unit test look like?

An Example

class Sum def sum a, b a + b endend

the code the testclass SumTest < Test::Unit::TestCase def test_sum s = Sum.new assert_equal 42, s.sum(30,12) assert_not_equal 5, s.sum(2,2) endend

Here’s a really simple example. Yes it’s stupid, yes it’s simple math. But remember unit tests are about testing all the really small parts. So ALL examples are likely to be stupid simple.

rabble@blackbook-4 ~/code $ ruby ejemplo_unit_test.rb Loaded suite ejemplo_unit_testStarted.Finished in 0.000345 seconds.

1 tests, 2 assertions, 0 failures, 0 errorsrabble@blackbook-4 ~/code $

the test

Now we simple run the unit test script and we can confirm that the test passes.

class Sumate def sum a, b a * b endend

class SumateTest < Test::Unit::TestCase def test_sum s = Sumate.new assert_equal 42, s.sum(30,12) assert_not_equal 5, s.sum(2,2) endend

the code the test

Now we go back to the sum method and we modify it. Really we’re attempting to refactor, where we change the implementation but NOT the functionality.

rabble@blackbook-4 ~/code $ ruby ejemplo_unit_test.rb Loaded suite ejemplo_unit_testStartedFFinished in 0.006554 seconds.

1) Failure:test_sum(SumateTest) [ejemplo_unit_test.rb:13]:<42> expected but was <360>.

1 tests, 2 assertions, 1 failures, 0 errors

the test

We re-run the test and now we can we have a failure. We know it failed, but it’s not so easy to see exactly what the problem is.

assert

assert

The fundamental building block of test::unit is the assert.

assert_equal expected, result

assert_equal for example, you provide what you expect to have and the actual result returned

assert is verification

What asserts do is verify code. And that is a good thing. We can know that there is a problem.

With this we can know if the system we are building is working or not. Unfortunately the failure messages don’t explain clearly what the problem is.

rabble@blackbook-4 ~/code $ ruby ejemplo_unit_test.rb Loaded suite ejemplo_unit_testStartedFFinished in 0.006554 seconds.

1) Failure:test_sum(SumTest) [ejemplo_unit_test.rb:13]:<42> expected but was <360>.

1 tests, 2 assertions, 1 failures, 0 errors

the test

With error messages like these it’s much harder to understand the problem. Really if your whole library is a single method it’s pretty easy to figure out, but when you’ve got many thousands of methods and tests, this kind of error is useless. What are the implications of failing to get 42 when we sum something?

Perhaps we’re testing the wrong things. We’re too close to the code. We need to test functionality and NOT the implementation.

The rhetoric around testing the unit has lead us astray. With the unit test all of our focus is on the small parts. But the units themselves change, and when they do we are forced to change the tests as well. When we have to update things two places that increases the workload and we can become confused about whether the problem is in the code or the test. We need to focus on what’s important, the functionality, the behaviour of each part of our application.

BDD: Behaviour Driven Developement

And with this, 33 slides in to the presentation, we finally arrive at BDD, Behaviour Driven Development.

the same difference

What’s BDD? Well the idea of TDD is good, but the language, the terms used, and it’s libraries are problematic. It encourages you to focus on the wrong things.

should vs assert

In BDD we use the method ‘should’ to explain how we want the application to behave, rather than verifying the particular implementation.

s.sum(30,12).should.be 42 s.sum(2,2).should.not.be 5

assert_equal 42, s.sum(30,12)assert_not_equal 5, s.sum(2,2)

in RSpec

in Test::Unit

Here’s an example of the same thing in test::unit and rspec. There’s not much difference in what it does. But moving from a test to a spec, represents a different way of thinking about the problem.

it “should add the numbers” do s = Sum.new s.sum(30,12).should.be 42 s.sum(2,2).should.not.be 5 end

def test_sum s = Sum.new assert_equal 42, s.sum(30,12) assert_not_equal 5, s.sum(2,2) end

in RSpec

in Test::Unit

la diferencia es poco, pero con diferente forma de pensar. desde test a spec.

describe Sum do it "should add the numbers" do s = Sum.new s.sum(30,12).should == 42 s.sum(2,2).should.not == 5 endend

the spec

And here we have the whole spec, you can see it does the same thing with a different approach.

rabble@blackbook-4 ~/code $ ruby ejemplo_rpsec.rb F1) 'Sum should add the numbers' FAILEDexpected: 42, got: 360 (using ==)ejemplo_rpsec.rb:10:

Finished in 0.007507 seconds

1 example, 1 failure

the spec

Now when we run the spec we can get a much better idea of what’s wrong when there is a failure.

rabble@blackbook-4 ~/code $ ruby ejemplo_unit_test.rb Loaded suite ejemplo_unit_testStartedFFinished in 0.006554 seconds.

1) Failure:test_sum(SumTest) [ejemplo_unit_test.rb:13]:<42> expected but was <360>.

1 tests, 2 assertions, 1 failures, 0 errors

the test

Remember back to the test.

rabble@blackbook-4 ~/code $ ruby ejemplo_rpsec.rb F1) 'Sum should add the numbers' FAILEDexpected: 42, got: 360 (using ==)ejemplo_rpsec.rb:10:

Finished in 0.007507 seconds

1 example, 1 failure

the spec

What’s the difference really? First it’s clearer, but more importantly it represents another way of thinking about tests.

What aboutreal world examples?

This is nice, but what about when applications get bigger, how do we translate a series of specs in to a narrative about how the application works.

examplesDirectMessagesController handling GET /direct_messages/1- should be successful- should change the status from unread- should render show template- should assign the found direct_message for the view- should not show a user's message to another user- should redirect if they try to view a deleted message

Now i’m going to go in to some more flushed out examples i stole from one of my work projects.

examples describe "handling GET /direct_messages/1" do define_models :direct_messages_controller before do login_as(:default) end def do_get get :show, :id => direct_messages(:default).id end it "should be successful" do do_get response.should be_success end it "should change the status from unread" do direct_messages(:default).unread?.should be_true do_get direct_messages(:default).reload.unread?.should be_false end it "should render show template" do do_get response.should render_template('show') end

it "should assign the found direct_message for the view" do do_get assigns[:direct_message].should == direct_messages(:default) end

Those english language specs of how the get method on the view action of the direct_messages controller should work is generated by this code.

As you can see we login the user before each spec, and we also have created a little helper method to call the action.

Each spec is very short.

examplesMembershipsController as a group administrator removing someone from a group- redirects to "group_path(@group)"- assigns @group- assigns @membership- assigns flash[:notice]- removes the membership

Here’s an example of the spec descriptions for a controller

examples describe "as a group administrator" do before do login_as :jack @group = groups(:default) @membership = memberships(:jane) end describe "removing someone from a group" do define_models :memberships act! { delete :destroy, :group_id => @group.permalink, :id => @membership.id } it_redirects_to { group_path(@group) } it_assigns :group, :membership, :flash => {:notice => :not_nil } it "removes the membership" do act! users(:jane).memberships.for(@group).should be_nil end end end

Here are the associated spec code for the controller. Again you can see we’ve built up a small dsl, domain specific language, around which we can keep our specs meaningful and very short.

examplesA user who is a moderator- should not be able to do admin functions- should be able to do moderator functions- should be able to do member functions- should be able to do nonmember functions

Here’s another example, describing a user model’s permissions

examples describe "who is a moderator" do define_models :permissions before do @user = users(:mod) @group = groups(:default) end it "should not be able to do admin functions" do @user.can?(ACTIONS[:admin], @group).should be_false end [:moderator, :member, :nonmember].each do |role| it "should be able to do #{role} functions" do @user.can?(ACTIONS[role], @group).should be_true end end end

And we have the code which impliments it.

narratives & stories

So we’ve described the idea of a narratives, the individual statements, combined in to description of the behaviour of each part of the application. Then you can combine those narratives in to a larger story.

St

narrative as a <role>i want to <activity>to do <a task>

as a useri want to publish a commentto directly participate in the forum

When we are making these spec style tests, we’re creating sentences which describe the functionality, we can use them to describe the use of the application. We already know the parts, it grows up from within the code as we write the parts. We can then string the specs together in to a use story executed in code.

example of a storyStory "View Home Page", %{ As a user I want to view my homepage To get an overview of the status of the system}, :type => RailsStory do Scenario "Publisher without vídeos" do Given "una empresa se llama", "Sin Vídeos" do |nombre| @empresa = Empresa.create! :nombre => nombre end And "un usario se llama", "novideos" do |login| @user = create_user login end And "el usuario es de", "empresa", "sin vídeos" do |klass, company_name| @user.update_attribute :empresa, klass.classify.constantize.find_by_name(empresa_nombre) end And "logged in as", "novideos" do |login| post "/sessions", :login => login, :password => "test" end

Here’s an example of stories written out in to code.. not entirely valid code.

Scenario "Basic user" do Given "A created user" And "two existing videos" When "visiting /videos" Then "both videos should be shown"end

an example of a story

other things

There are some other rspec / bdd related issues which i’ve skipped over and not covered.

mocks y stubs

Rspec has a mock library included and you can also use external ones. A Mock or Stub object replaces a real part of your system to make atomic testing simpler and faster.

shouldaBDD over Test::Unit

RSpec is perhaps to magical. The most popular alternative is to use Shoulda which lets you use an spec syntax.

matchyBDD over Test::Unit

Jeremy McAnally who i’m working with has an alternative implementation which he’s been working on.

Beyond Testing: specs and Behaviour Driven DevelopementRSpec - www.rspec.infoBDD - www.behaviour-driven.org

Thank you very much.

Creative CommonsPhotos Used• http://flickr.com/photos/foreversouls/4254487/

• http://flickr.com/photos/ejpphoto/2314610838/

• http://flickr.com/photos/mrpunto/114862457/

• http://flickr.com/photos/orvaratli/2713730155/

• http://flickr.com/photos/horizon/287190887/

• http://flickr.com/photos/freewine/478332550/

• http://flickr.com/photos/patrick_q/98179665/

• http://flickr.com/photos/gadl/284995199/

• http://flickr.com/photos/darko_pevec/2300487155

• http://flickr.com/photos/atouchofcolor/376242632/

• http://flickr.com/photos/bcnbits/363695635/

• http://flickr.com/photos/thomashawk/340185708/

• http://flickr.com/photos/thomashawk/268524287/