confitura 2012 bad tests, good tests

Bad Tests, Good Tests

Tomek Kaczanowski

http://twitter.com/#!/devops_borat

Tomek Kaczanowski

• Developer• Team lead• Blogger

• http://kaczanowscy.pl/tomek• Book author

• http://practicalunittesting.com

• Working at CodeWise (Krakow) ...we are hiring, wanna join us?

Before we begin

• Most of the examples are real but: Obfuscated

− to protect the innocents

Truncated

− imagine much more complex domain objects

• Asking questions is allowed ...but being smarter than me is not ;)

A little bit of history

Please... no more...

http://thedogatemycareplan.wordpress.com

Before we begin

• The tests were written in 2004-2006.• No automation, no CI.• Some tests do not compile.• In some tests you can read a comment that "WARNING:

test requires the divide param to be set to 20" but the code is so ugly, that there is no way to inject this value.

• Some test data are available in form of serialized objects (*.ser) that can not be currently deserialized, because the classes have changed.

• The project is now in maintenance.

Courtesy of Bartosz http://twitter.com/#!/bocytko

We don't need no stinkin' asserts!public void testAddChunks() {

System.out.println("*************************************");System.out.println("testAddChunks() ... ");ChunkMap cm = new ChunkMap(3);cm.addChunk(new Chunk("chunk"));

List testList = cm.getChunks("chunk",null);

if (testList.isEmpty())fail("there should be at least one list!");

Chunk chunk = cm.getActualChunk("chunk",null);if (chunk.getElements().isEmpty())

fail("there should be at least one element!");if (cm.getFinalChunkNr() != 1)

fail("there should be at least one chunk!");// iterate actual chunkfor (Iterator it = chunk.getElements().iterator();

it.hasNext();) {Element element = (Element) it.next();System.out.println("Element: " + element);

}showChunks(cm);System.out.println("testAddChunks() OK ");

}

Success is not an option...

/** * Method testFailure. */public void testFailure() { try { Message message = new Message(null,true); fail(); } catch(Exception ex) { ExceptionHandler.log(ExceptionLevel.ANY,ex); fail(); }}

What has happened? Well, it failed...public void testSimple() {

IData data = null;IFormat format = null;LinkedList<String> attr = new LinkedList<String>();attr.add("A");attr.add("B");

try {format = new SimpleFormat("A");data.setAmount(Amount.TEN);data.setAttributes(attr);IResult result = format.execute();System.out.println(result.size());Iterator iter = result.iterator();while (iter.hasNext()) {

IResult r = (IResult) iter.next(); System.out.println(r.getMessage());

...}catch (Exception e) {

fail();}

}

What has happened? Well, it failed...public void testSimple() {

IData data = null;IFormat format = null;LinkedList<String> attr = new LinkedList<String>();attr.add("A");attr.add("B");

try {format = new SimpleFormat("A");data.setAmount(Amount.TEN);data.setAttributes(attr);IResult result = format.execute();System.out.println(result.size());Iterator iter = result.iterator();while (iter.hasNext()) {

IResult r = (IResult) iter.next(); System.out.println(r.getMessage());

...}

catch (Exception e) {fail();

}}

data is still null here.Ready or not, NPE is coming.

Talk to me

//wait for messagesdo { input = ""; try { System.out.print(">"); read = System.in.read(buf); //convert characters to string input = new String(buf, 0, read - newline.length()); System.out.println(input);

if (input.equals("end") || input.equals("exit") || input.equals("stop") || input.equals("quit")) { System.out.println("Terminating Test please wait..."); System.out.println("******* Test terminated *******"); toStop = true; } else { System.out.println("Commands:" + newline + "'end', 'exit', 'stop' or 'quit' terminates this test "); } } catch (Exception e) { e.printStackTrace(); }} while (!toStop);

Tests are boring – let us autogenerate them!/*** Generated by JUnitDoclet, a tool provided by* ObjectFab GmbH under LGPL.* Please see www.junitdoclet.org, www.gnu.org* and www.objectfab.de for informations about* the tool, the licence and the authors.*/public class AdapterTest// JUnitDoclet begin extends_implementsextends TestCase// JUnitDoclet end extends_implements{ // JUnitDoclet begin class Adapter adapter = null; // JUnitDoclet end class

public AdapterTest(String name) { // JUnitDoclet begin method AdapterTest super(name); // JUnitDoclet end method AdapterTest }

public Adapter createInstance() throws Exception { // JUnitDoclet begin method testcase.createInstance return new Adapter(); // JUnitDoclet end method testcase.createInstance }

protected void setUp() throws Exception { // JUnitDoclet begin method testcase.setUp super.setUp(); adapter = createInstance(); // JUnitDoclet end method testcase.setUp }

protected void tearDown() throws Exception { // JUnitDoclet begin method testcase.tearDown adapter = null; super.tearDown(); // JUnitDoclet end method testcase.tearDown

public void testMain() throws Exception { // JUnitDoclet begin method testMain Adapter.main(new String [] {"ADAPTER"}); // JUnitDoclet end method testMain }

/** * JUnitDoclet moves marker to this method, if there is not match * for them in the regenerated code and if the marker is not empty. * This way, no test gets lost when regenerating after renaming. * Method testVault is supposed to be empty. */ public void testVault() throws Exception { // JUnitDoclet begin method testcase.testVault // JUnitDoclet end method testcase.testVault }

public static void main(String[] args) { // JUnitDoclet begin method testcase.main junit.textui.TestRunner.run(AdapterTest.class); // JUnitDoclet end method testcase.main }}

Tests are boring – let us autogenerate them!public void testSetGetTimestamp() throws Exception {

// JUnitDoclet begin method setTimestamp getTimestamp java.util.Calendar[] tests = {new GregorianCalendar(), null};

for (int i = 0; i < tests.length; i++) { adapter.setTimestamp(tests[i]); assertEquals(tests[i], adapter.getTimestamp()); } // JUnitDoclet end method setTimestamp getTimestamp }

public void testSetGetParam() throws Exception { // JUnitDoclet begin method setParam getParam String[] tests = {"a", "aaa", "---", "23121313", "", null};

for (int i = 0; i < tests.length; i++) { adapter.setParam(tests[i]); assertEquals(tests[i], adapter.getParam()); } // JUnitDoclet end method setParam getParam}

Bartosz, do you really work with such code?!

Conclusions

• Automation!• Running• Verification

• Tests are to be written not generated• You should be informed why your test failed• Master your tools

…at least learn the basics!

Few words about tests

Why bother with tests?

• System works as expected

• Changes do not hurt

• Documentation

http://twitter.com/#!/devops_borat

Tests help to achieve quality

Not sure when I saw this picture – probably in GOOS?

What happens if we do it wrong?

• Angry clients• Depressed developers

http://www.joshcanhelp.com

When I started out with unit tests, I was enthralled with the promise of ease and security that they would bring to my projects. In practice, however, the theory of sustainable software through unit tests started to break down. This difficulty continued to build up, until I finally threw my head back in anger and declared that "Unit Tests have become more trouble than they are worth."

Llewellyn Falco and Michael Kennedy, Develop Mentor August 09

http://chrispiascik.com/daily-drawings/express-yourself/

It is a full-time job

No smoke without testsclass SystemAdminSmokeTest extends GroovyTestCase {

void testSmoke() { def ds = new org.h2.jdbcx.JdbcDataSource( URL: 'jdbc:h2:mem:test;DB_CLOSE_DELAY=-1;MODE=Oracle', user: 'sa', password: '') def jpaProperties = new Properties() jpaProperties.setProperty( 'hibernate.cache.use_second_level_cache', 'false') jpaProperties.setProperty( 'hibernate.cache.use_query_cache', 'false') def emf = new LocalContainerEntityManagerFactoryBean( dataSource: ds, persistenceUnitName: 'my-domain', jpaVendorAdapter: new HibernateJpaVendorAdapter( database: Database.H2, showSql: true, generateDdl: true), jpaProperties: jpaProperties)

}

No smoke without testsclass SystemAdminSmokeTest extends GroovyTestCase {

void testSmoke() {// do not remove below code// def ds = new org.h2.jdbcx.JdbcDataSource(// URL: 'jdbc:h2:mem:test;DB_CLOSE_DELAY=-1;MODE=Oracle',// user: 'sa', password: '')//// def jpaProperties = new Properties()// jpaProperties.setProperty(// 'hibernate.cache.use_second_level_cache', 'false')// jpaProperties.setProperty(// 'hibernate.cache.use_query_cache', 'false')//// def emf = new LocalContainerEntityManagerFactoryBean(// dataSource: ds, persistenceUnitName: 'my-domain',// jpaVendorAdapter: new HibernateJpaVendorAdapter(// database: Database.H2, showSql: true,// generateDdl: true), jpaProperties: jpaProperties)

…some more code below, all commented out :(}

Let's follow the leader!

@Testpublic class ExampleTest {

public void testExample() {assertTrue(true);

}}

Uh-oh, I feel lonely...

@Testpublic class ExampleTest {

public void testExample() {assertTrue(true);

}}

Flickering tests

Asking for troubles...

LoggingPropertyConfigurator configurator

= mock(LoggingPropertyConfigurator.class);

BaseServletContextListener baseServletContextListener

= new BaseServletContextListener(configurator);

@Test public void shouldLoadConfigProperties() {

baseServletContextListener.contextInitialized();

verify(configurator).configure(any(Properties.class));

}

@Test(expected = LoggingInitialisationException.class)

public void shouldThrowLoggingException() {

System.setProperty("logConfig", "nonExistingFile");

baseServletContextListener.contextInitialized();

}

Should load some default config

Should load this specific file

Lets mock!

Mock'em All!public String getUrl(User user, String timestamp) {

String name=user.getFullName();String url=baseUrl

+"name="+URLEncoder.encode(name, "UTF-8")+"&timestamp="+timestamp;

return url;}

public String getUrl(User user) {Date date=new Date();Long time=(date.getTime()/1000); //convert ms to secondsString timestamp=time.toString();return getUrl(user, timestamp);

}

Mock'em All!public String getUrl(User user, String timestamp) {

String name=user.getFullName();String url=baseUrl

+"name="+URLEncoder.encode(name, "UTF-8")+"&timestamp="+timestamp;

return url;}

public String getUrl(User user) {Date date=new Date();Long time=(date.getTime()/1000); //convert ms to secondsString timestamp=time.toString();return getUrl(user, timestamp);

}

@Testpublic void shouldUseTimestampMethod() {

//givenUtil spyUtil = spy(util);

//whenspyUtil.getUrl(user);

//thenverify(spyUtil).getUrl(eq(user), anyString());

}

Use the front door

@Test

public void shouldAddTimestampToGeneratedUrl() {

//given

util = new ....

TimeProvider timeProvider = mock(TimeProvider.class);

when(timeProvider.getTime()).thenReturn("12345");

util.set(timeProvider);

//when

String url = util.getUrl(user);

//then

assertThat(url).contains("timestamp=12345");

}

Mock'em All!

@Test

public void shouldAddTimeZoneToModelAndView() {

//given

UserFacade userFacade = mock(UserFacade.class);

ModelAndView modelAndView = mock(ModelAndView.class);

given(userFacade.getTimezone()).willReturn("timezone X");

//when

new UserDataInterceptor(userFacade)

.postHandle(null, null, null, modelAndView);

//then

verify(modelAndView).addObject("timezone", "timezone X");

}

Use the front door

@Test

public void shouldAddTimeZoneToModelAndView() {

//given

UserFacade userFacade = mock(UserFacade.class);

ModelAndView modelAndView = new ModelAndView();

given(userFacade.getTimezone()).willReturn("timezone X");

//when

new UserDataInterceptor(userFacade)

.postHandle(null, null, null, modelAndView);

//then

assertThat(modelAndView).constains("timezone", "timezone X");

}

Single Responsibility Principle

SRP for tests

A test should have one and only one reason to fail.

P.S. This is definitely a good advice for unit tests, but rather not valid for integration and end-to-end tests.

Testing two things at once@DataProviderpublic Object[][] data() {

return new Object[][] { {"48", true}, {"+48", true}, {"++48", true}, {"+48503", true}, {"+4", false},{"++4", false}, {"", false},{null, false}, {" ", false}, };

}

@Test(dataProvider = "data")public void testQueryVerification(String query, boolean expected) {

assertEquals(expected, FieldVerifier.isValidQuery(query));}



}



Data

Algorithm / Logic



}



testQueryVerification1() {assertEquals(true, FieldVerifier.isValidQuery(„48”));

}testQueryVerification2() {

assertEquals(true, FieldVerifier.isValidQuery(„+48”));}testQueryVerification3() {

assertEquals(true, FieldVerifier.isValidQuery(„++48”));}testQueryVerification4() {

assertEquals(true, FieldVerifier.isValidQuery(„+48503”));}...

Testing two things at once

@DataProvider

public Object[][] data() {

return new Object[][] { {"48", true}, {"+48", true},

{"++48", true}, {"+48503", true}, {"+4", false},

{"++4", false}, {"", false},

{null, false}, {" ", false}, };

}

@Test(dataProvider = "data")

public void testQueryVerification(String query, boolean expected) {

assertEquals(expected, FieldVerifier.isValidQuery(query));

}

Concentrate on one feature@DataProviderpublic Object[][] validQueries() { return new Object[][] { {"48"}, {"48123"},

{"+48"}, {"++48"}, {"+48503"}};}

@Test(dataProvider = "validQueries")public void shouldRecognizeValidQueries( String validQuery) { assertTrue(FieldVerifier.isValidQuery(validQuery));}

@DataProviderpublic Object[][] invalidQueries() { return new Object[][] {

{"+4"}, {"++4"}, {""}, {null}, {" "} };

}

@Test(dataProvider = "invalidQueries")public void shouldRejectInvalidQueries( String invalidQuery) { assertFalse(FieldVerifier.isValidQuery(invalidQuery));}

Are you satisfied?

Happy path

testSum() {

assertEquals(Math.sum(2,2), 4);

}

http://mw2.google.com/mw-panoramio/photos/medium/68775332.jpg

Happy paths are for wimps

2 + 2

2 + -2

2 + -5

0 + 2

2 + 0

Integer.MAX_VALUE + something

etc.

http://kidskidskids.tumblr.com/post/1145294997

Avoiding happy paths

testSum() {

assertEquals(Math.sum(2,2), 4);

}

http://kidskidskids.tumblr.com/post/1145294997

sum(int x, int y) {

return 4;

}

And then listen to your code. Because it tells you something.

Start with one:

Do the simplest thing that works:

Avoiding happy paths

http://looneytunes09.files.wordpress.com/2010/07/lisa-yell.gif

You moron!Your test is so pathetic, that I can make it pass by doing such a silly thing. Try harder!

sum(int x, int y) {

return 4;

}

Readability is the king

Who the heck is “user_2” ?

@DataProvider

public static Object[][] usersPermissions() {

return new Object[][]{

{"user_1", Permission.READ},

{"user_1", Permission.WRITE},

{"user_1", Permission.REMOVE},

{"user_2", Permission.WRITE},

{"user_2", Permission.READ},

{"user_3", Permission.READ}

};

}

Ah, logged user can read and write...

@DataProvider

public static Object[][] usersPermissions() {


{ADMIN, Permission.READ},

{ADMIN, Permission.WRITE},

{ADMIN, Permission.REMOVE},

{LOGGED, Permission.WRITE},

{LOGGED, Permission.READ},

{GUEST, Permission.READ}

};

}

domain1, domain2, domain3, ...

Do not make me learn the API!

server = new MockServer(responseMap, true,

new URL(SERVER_ROOT).getPort(), false);




private static final boolean RESPONSE_IS_A_FILE = true;

private static final boolean NO_SSL = false;

server = new MockServer(responseMap, RESPONSE_IS_A_FILE,

new URL(SERVER_ROOT).getPort(), NO_SSL);




server = new MockServerBuilder()

.withResponse(responseMap)

.withResponseType(FILE)

.withUrl(SERVER_ROOT)

.withoutSsl().create();

What is really important?

What is really important?

@DataProvider

public Object[][] snapshotArtifacts() {


{"a", "b", "2.2-SNAPSHOT", Artifact.JAR },

{"c", "d", "2.2.4.6-SNAPSHOT", Artifact.JAR},

{"e", "f", "2-SNAPSHOT", Artifact.JAR}

};

}

@Test(dataProvider = "snapshotArtifacts")

public void shouldRecognizeSnapshots(

String groupId, String artifactId,

String version, Type type) {

Artifact artifact

= new Artifact(groupId, artifactId, version, type);

assertThat(artifact.isSnapshot()).isTrue();

}

Only version matters

@DataProvider

public Object[][] snapshotVersions() {


{"2.2-SNAPSHOT"},

{"2.2.4.6-SNAPSHOT"},

{"2-SNAPSHOT"}

};

}

@Test(dataProvider = "snapshotVersions")

public void shouldRecognizeSnapshots(String version) {

Artifact artifact

= new Artifact(VALID_GROUP, VALID_ARTIFACT_ID,

version, VALID_TYPE);

assertThat(artifact.isSnapshot()).isTrue();

}

Test method names

Naming is really important

Test methods names are important

• When test fails• Relation to focused tests

Test methods names are important

@Test

public void testOperation() {

configureRequest("/validate")

rc = new RequestContext(parser, request)

assert rc.getConnector() == null

assert rc.getOperation().equals("validate")

}

• testFindTransactionsToAutoCharge()• testSystemSuccess()• testOperation()

“should” is better than “test”

• shouldRejectInvalidRequests()• shouldSaveNewUserToDatabase()• constructorShouldFailWithNegativePrice()• shouldReturnOnlyUsersWithGivenName()

• testOperation()• testQuery()• testConstructor()• testFindUsersWithFilter()

“should” is better than “test”

• Starting test method names with “should” steers you in the right direction.

• “test” prefix makes your test method a limitless bag where you throw everything worth testing

http://www.greenerideal.com/

http://jochopra.blogspot.com/

Test methods names are important@Testpublic void testQuery(){ when(q.getResultList()).thenReturn(null); assertNull(dao.findByQuery(Transaction.class, q, false)); assertNull(dao.findByQuery(Operator.class, q, false)); assertNull(dao.findByQuery(null, null, false));

List result = new LinkedList(); when(q.getResultList()).thenReturn(result); assertEquals(dao.findByQuery(Transaction.class, q, false), result); assertEquals(dao.findByQuery(Operator.class, q, false), result); assertEquals(dao.findByQuery(null, null, false), null);

when(q.getSingleResult()).thenReturn(null); assertEquals(dao.findByQuery(Transaction.class, q, true).size(), 0); assertEquals(dao.findByQuery(Operator.class, q, true).size(), 0); assertEquals(dao.findByQuery(null, null, true), null);

when(q.getSingleResult()).thenReturn(t); assertSame(dao.findByQuery(Transaction.class, q, true).get(0), t); when(q.getSingleResult()).thenReturn(o); assertSame(dao.findByQuery(Operator.class, q, true).get(0), o); when(q.getSingleResult()).thenReturn(null); assertSame(dao.findByQuery(null, null, true), null);}

Assertions

Assertion part is freaking hugepublic void shouldPreDeployApplication() {

// givenArtifact artifact = mock(Artifact.class);when(artifact.getFileName()).thenReturn("war-artifact-2.0.war");ServerConfiguration config

= new ServerConfiguration(ADDRESS, USER, KEY_FILE, TOMCAT_PATH, TEMP_PATH);Tomcat tomcat = new Tomcat(HTTP_TOMCAT_URL, config);String destDir = new File(".").getCanonicalPath() + SLASH + "target" + SLASH;new File(destDir).mkdirs();

// whentomcat.preDeploy(artifact, new FakeWar(WAR_FILE_LENGTH));

//thenJSch jsch = new JSch();jsch.addIdentity(KEY_FILE);Session session = jsch.getSession(USER, ADDRESS, 22);session.setConfig("StrictHostKeyChecking", "no");session.connect();

Channel channel = session.openChannel("sftp");session.setServerAliveInterval(92000);channel.connect();ChannelSftp sftpChannel = (ChannelSftp) channel;

sftpChannel.get(TEMP_PATH + SLASH + artifact.getFileName(), destDir);sftpChannel.exit();

session.disconnect();

File downloadedFile = new File(destDir, artifact.getFileName());

assertThat(downloadedFile).exists().hasSize(WAR_FILE_LENGTH);}

Just say it

public void shouldPreDeployApplication() {

Artifact artifact = mock(Artifact.class);

when(artifact.getFileName())

.thenReturn(ARTIFACT_FILE_NAME);

ServerConfiguration config

= new ServerConfiguration(ADDRESS, USER,

KEY_FILE, TOMCAT_PATH, TEMP_PATH);

Tomcat tomcat = new Tomcat(HTTP_TOMCAT_URL, config);

tomcat.preDeploy(artifact, new FakeWar(WAR_FILE_LENGTH));

SSHServerAssert.assertThat(ARTIFACT_FILE_NAME)

.existsOnServer(config).hasSize(WAR_FILE_LENGTH);

}

Asserting using private methods

@Test

public void testChargeInRetryingState() throws Exception {

// given

TxDTO request = createTxDTO(RequestType.CHARGE);

AndroidTransaction androidTransaction = ...

// when

final TxDTO txDTO = processor.processRequest(request);

// then

assertState(request, androidTransaction,

CHARGED, CHARGE_PENDING, AS_ANDROID_TX_STATE,

ClientMessage.SUCCESS, ResultCode.SUCCESS);

}

Matchers vs. private methodsassertState(TxDTO txDTO, AndroidTransaction androidTransaction,

AndroidTransactionState expectedAndroidState, AndroidTransactionState expectedPreviousAndroidState, ExtendedState expectedState, String expectedClientStatus,

ResultCode expectedRequestResultCode) { final List<AndroidTransactionStep> steps

= new ArrayList<>(androidTransaction.getTransactionSteps()); final boolean checkPreviousStep = expectedAndroidState != null; assertTrue(steps.size() >= (checkPreviousStep ? 3 : 2));

if (checkPreviousStep) { AndroidTransactionStep lastStep = steps.get(steps.size() - 2); assertEquals(lastStep.getTransactionState(),

expectedPreviousAndroidState); }

final AndroidTransactionStep lastStep = steps.get(steps.size() - 1); assertEquals(lastStep.getTransactionState(), expectedAndroidState); assertEquals(lastStep.getMessage(), expectedClientStatus);

assertEquals(txDTO.getResultCode(), expectedRequestResultCode); assertEquals(androidTransaction.getState(), expectedAndroidState); assertEquals(androidTransaction.getExtendedState(), expectedState);

if (expectedClientStatus == null) { verifyZeroInteractions(client); }}

Matchers vs. private methods

@Test

public void testChargeInRetryingState() throws Exception {

// given

TxDTO request = createTxDTO(CHARGE);

AndroidTransaction androidTransaction = ...

// when

final TxDTO txDTO = processor.processRequest(request);

// then

assertThat(androidTransaction).hasState(CHARGED)

.hasMessage(ClientMessage.SUCCESS)

.hasPreviousState(CHARGE_PENDING)

.hasExtendedState(null);

assertEquals(txDTO.getResultCode(),

ResultCode.SUCCESS);

}

What is asserted?

@Test

public void testCompile_32Bit_FakeSourceFile() {

CompilerSupport _32BitCompilerSupport

= CompilerSupportFactory.getDefault32BitCompilerSupport();

testCompile_FakeSourceFile(_32BitCompilerSupport);

}

What is asserted?

@Test

public void testCompile_32Bit_FakeSourceFile() {

CompilerSupport _32BitCompilerSupport

= CompilerSupportFactory.getDefault32BitCompilerSupport();

testCompile_FakeSourceFile(_32BitCompilerSupport);

}

private void testCompile_FakeSourceFile(

CompilerSupport compilerSupport) {

compiledFiles

= compilerSupport.compile(new File[] { new File("fake") });

assertThat(compiledFiles, is(emptyArray()));

}

Asserting everything

public void invalidTxShouldBeCanceled() {

String fileContent =

FileUtils.getContentOfFile("response.csv");

assertTrue(fileContent.contains(

"CANCEL,123,123cancel,billing_id_123_cancel,SUCCESS,"));

}

Asserting everything




assertTrue(fileContent.contains(

"CANCEL,123,123cancel,billing_id_123_cancel,SUCCESS,"));

}




TxDTOAssert.assertThat(fileContent)

.hasTransaction("123cancel").withResultCode(SUCCESS);

}

Know your tool

Asynchronous Calls

@Test

public void updatesCustomerStatus() throws Exception {

// Publish an asynchronous event:

publishEvent(updateCustomerStatusEvent);

// Awaitility lets you wait

// until the asynchronous operation completes:

await()

.atMost(5, SECONDS)

.until(costumerStatusIsUpdated());

...

} http://code.google.com/p/awaitility/

Expected exceptions

@Test(expectedExceptions = SmsException.class)

public void shouldThrowException() throws SmsException {

try {

String s = gutExtractor.extractGut(„invalid gut”);

System.out.println(s);

} catch (SmsException e) {

e.printStackTrace();

throw e;

}

}

Expected exceptions

@Test(expectedExceptions = SmsException.class)


String s = gutExtractor.extractGut(„invalid gut”);

}

Expected exceptions (with catch-exception)

@Test


when(gutExtractor.extractGut(„invalid gut”));

then(caughtException())

.isInstanceOf(SmsException.class)

.hasMessage("Invalid gut")

.hasNoCause();

}

http://code.google.com/p/catch-exception/

Running SUT's code concurrently

@Test(threadPoolSize = 3, invocationCount = 10)

public void testServer() {

// this method will be run in parallel by 3 thread

// 10 invocations (in total)

}

Dependent test methods

@Test

public void shouldConnectToDB() {

// verifying that you can

// estabilish a connection with DB

}

@Test(dependsOnMethods = „shouldConnectToDB”)

public void should…() {

// some operations on DB

}

Know your tools

• Unit testing framework Use of temporary file rule

Listeners

Concurrency

@Before/@After

Parametrized tests

Test dependencies

Additional libraries

Hamcrest, FEST, Mockito, catch-exception, awaitility, …

• Build tool Parallel execution

CI

• IDE Templates

Shortcuts

What do you really want to test?

What do you really want to test?

@Test

public void shouldAddAUser() {

User user = new User();

userService.save(user);

assertEquals(dao.getNbOfUsers(), 1);

}

You wanted to see that the number increased

@Test

public void shouldAddAUser() {

int nb = dao.getNbOfUsers();

User user = new User();

userService.save(user);

assertEquals(dao.getNbOfUsers(), nb + 1);

}

Random

Doing it wrong

public void myTest() {SomeObject obj = new SomeObject(

a, b, c, productCode());// testing of obj here

}

private String productCode(){String[] codes =

{"Code A", "Code B", "Code C", "Code D"};

int index = rand.nextInt(codes.length); return codes[index];}

The dream of stronger, random-powered tests


randomName(), randomValue(), ....);// testing of obj here

}

Does it make your test stronger?

The dream of stronger, random-powered tests


randomName(), randomValue(), ....);// testing of obj here

}

Does it make your test stronger?

...or does it only bring confusion?

Test failedExpected

SomeObject(„a”, „b”, ....)but got

SomeObject(„*&O*$NdlF”, „#idSLNF”, ....)

Conclusions

There is more to it

• Integration / end-to-end tests which are not parametrized (so they all try to set up jetty on port 8080),

• Tests which should be really unit, but use Spring context to create objects,

• Tests with a lot of dependencies between them (a nightmare to maintain!),

• Tests which run slow• Tests which try to cover the deficiencies of production

code and end up being a total mess,• Tests which verify methods instead of verifying

responsibilities of a class• etc., etc.

Test-last? No!

• makes people not write tests at all• makes people do only happy-testing• tests reflect the implementation

For six or eight hours spread over the next few weeks I struggled to get the first test written and running. Writing tests for Eclipse plug-ins is not trivial, so it’s not surprising I had some trouble. [...] In six or eight hours of solid programming time, I can still make significant progress. If I’d just written some stuff and verified it by hand, I would probably have the final answer to whether my idea is actually worth money by now. Instead, all I have is a complicated test that doesn’t work, a pile of frustration, eight fewer hours in my life, and the motivation to write another essay.

Kent Beck, Just Ship, Baby

Always TDD?

Treat tests as the first class citizens

• do it everyday or forget about it

• use the right tool for the job

• and learn to use it!

• do not live with broken windows

• respect KISS, SRP, DRY (?)

• write good code, and you will also write good tests

• or rather write good tests and you will get good code for free

• code review your tests

• do more than happy path testing

• do not make the reader learn the API, make it obvious

• bad names lead to bad tests

• make tests readable using matchers, builders and good names

• test behaviour not methods

• be pragmatic about the tests you write

• TDD always?

• what is the best way to test it? unit/integration/end-to-end ?

• automate!

• always concentrate on what is worth testing

• ask yourself questions like: 'is it really important that X should send message Y to Z?'

• use the front door – state testing before interaction testing (mockc)

…questions?…rants?…hate speeches?…any other forms of expressing your ego?

Thank you for watching these slides! You can learn more about wirting high quality tests by reading my book – „Practical Unit Testing with TestNG and Mockito”.

You can also participate in writing of my new (free!) e-book devoted to bad and good tests.

Thank you!

https://github.com/tomekkaczanowski/bad-tests-good-tests

confitura 2012 bad tests, good tests

Technology