using xunit as a swiss-aarmy testing toolkit
DESCRIPTION
Modern Unit Testing practices act as a conduit for improved software designs that are more amenable to change and can be easily backed by automation for fast feedback on quality assurance. The necessity of reducing external dependencies forces us to design our modules with minimum coupling which can then be leveraged both at the module, component and subsystem levels in our testing. As we start to integrate our units into larger blocks and interface our resulting components with external systems we find ourselves switching nomenclature as we progress from Unit to Integration testing. But is a change in mindset and tooling really required? The xUnit testing framework is commonly perceived as an aid to Unit Testing but the constraints that it imposes on the architecture mean that it is an excellent mechanism for invoking arbitrary code in a restricted context. Tests can be partitioned by categorisation at the test and fixture level and through physical packaging leading to a flexible test code structure. Throw in its huge popularity and you have a simplified learning curve for expressing more that just unit tests. Using scenarios from his current system Chris aims to show how you can use a similar format and tooling for unit, component and integration level tests; albeit with a few liberties taken to work around the inherent differences with each methodology.TRANSCRIPT
![Page 1: Using xUnit as a Swiss-Aarmy Testing Toolkit](https://reader034.vdocuments.net/reader034/viewer/2022051411/540b1e698d7f72da6a8b460c/html5/thumbnails/1.jpg)
Using xUnit as a Swiss-Army Testing Toolkit
(Does ‘Unit’ Size Matter?)
ACCU Conference 2011
Chris [email protected]
![Page 2: Using xUnit as a Swiss-Aarmy Testing Toolkit](https://reader034.vdocuments.net/reader034/viewer/2022051411/540b1e698d7f72da6a8b460c/html5/thumbnails/2.jpg)
Stream of Consciousness
• Developer Driven Testing
• The Essence of (x)Unit Testing
• Those Pesky Dependencies
• Code & Test Evolution in Practice
![Page 3: Using xUnit as a Swiss-Aarmy Testing Toolkit](https://reader034.vdocuments.net/reader034/viewer/2022051411/540b1e698d7f72da6a8b460c/html5/thumbnails/3.jpg)
Stream of Consciousness
• Developer Driven Testing
• The Essence of (x)Unit Testing
• Those Pesky Dependencies
• Code & Test Evolution in Practice
![Page 4: Using xUnit as a Swiss-Aarmy Testing Toolkit](https://reader034.vdocuments.net/reader034/viewer/2022051411/540b1e698d7f72da6a8b460c/html5/thumbnails/4.jpg)
Text Book Teststring[][] tests = { { "3", "4", "+", "7" }, { "9", "1", "-", "8" }, { "2", "3", "*", "6" }, { "9", "3", "/", "3" },};
void run_tests(){ var calculator = new Calculator();
foreach(var test in tests) { var lhs = test[0]; var rhs = test[1]; var op = test[2];
var result = calculator(lhs, rhs, op);
assert(result == test[3]); }}
![Page 5: Using xUnit as a Swiss-Aarmy Testing Toolkit](https://reader034.vdocuments.net/reader034/viewer/2022051411/540b1e698d7f72da6a8b460c/html5/thumbnails/5.jpg)
Exercise Left for the Reader
External System 1 External System 2 External System 3
The System
42
ServicesDatabase
![Page 6: Using xUnit as a Swiss-Aarmy Testing Toolkit](https://reader034.vdocuments.net/reader034/viewer/2022051411/540b1e698d7f72da6a8b460c/html5/thumbnails/6.jpg)
Unit
Integration
System Component
Stress
Lexicon of Testing
End-to-End
Regression
White Box
Black Box
Characterisation
Exploration
![Page 7: Using xUnit as a Swiss-Aarmy Testing Toolkit](https://reader034.vdocuments.net/reader034/viewer/2022051411/540b1e698d7f72da6a8b460c/html5/thumbnails/7.jpg)
System
Integration
Component
Unit
Dependencies
Feedback
‘Unit’ Evolution
All Regression
![Page 8: Using xUnit as a Swiss-Aarmy Testing Toolkit](https://reader034.vdocuments.net/reader034/viewer/2022051411/540b1e698d7f72da6a8b460c/html5/thumbnails/8.jpg)
Stream of Consciousness
• Developer Driven Testing
• The Essence of (x)Unit Testing
• Those Pesky Dependencies
• Code & Test Evolution in Practice
![Page 9: Using xUnit as a Swiss-Aarmy Testing Toolkit](https://reader034.vdocuments.net/reader034/viewer/2022051411/540b1e698d7f72da6a8b460c/html5/thumbnails/9.jpg)
Test == Specification
public void Execute_Should_Elide_Agreement_When_No_Trades_Match(){ var trades = new List<Trade> { new Trade("trade-id", "product-a") }; var agreement = new Agreement("product-b"); var task = new PreparationTask(trades, agreement);
var result = task.Execute(s_services);
Assert.That(task.Agreement, Is.Null);}
![Page 10: Using xUnit as a Swiss-Aarmy Testing Toolkit](https://reader034.vdocuments.net/reader034/viewer/2022051411/540b1e698d7f72da6a8b460c/html5/thumbnails/10.jpg)
Consistent Stylepublic void a_c_sharp_test(){ var arrangement = new Arrangement();
var result = arrangement.action();
Assert.That(result, Is.EqualTo(expectation));}
create procedure a_sql_testas declare arrangement varchar(100), result varchar(100)
exec action @input = arrangement, @output = result
exec AssertAreEqual @result, "expectation"go
![Page 11: Using xUnit as a Swiss-Aarmy Testing Toolkit](https://reader034.vdocuments.net/reader034/viewer/2022051411/540b1e698d7f72da6a8b460c/html5/thumbnails/11.jpg)
Minimises Dependencies
MyService
External Service DatabaseFile System
Mock ExternalService
IExternalService IFileSystem IDatabase
Mock FileSystem
Mock Database
![Page 12: Using xUnit as a Swiss-Aarmy Testing Toolkit](https://reader034.vdocuments.net/reader034/viewer/2022051411/540b1e698d7f72da6a8b460c/html5/thumbnails/12.jpg)
Promotes Arbitrary Code Execution
public void Prepare_Should_Elide_Agreement_When_No_Trades_Match(){ var trades = new List<Trade> { new Trade("trade-id", "product-a") }; var agreement = new Agreement("product-b"); var task = new PreparationTask(trades, agreement);
var result = task.Execute(s_services);
Assert.That(task.Agreement, Is.Null);}
LibraryEXE Stub
Test Runner LibraryTestsDebugger
Custom TestHarness
![Page 13: Using xUnit as a Swiss-Aarmy Testing Toolkit](https://reader034.vdocuments.net/reader034/viewer/2022051411/540b1e698d7f72da6a8b460c/html5/thumbnails/13.jpg)
Automated Testing
• Lowers the barrier to running tests
• Regression testing is implicit
• Build server watches your back
![Page 14: Using xUnit as a Swiss-Aarmy Testing Toolkit](https://reader034.vdocuments.net/reader034/viewer/2022051411/540b1e698d7f72da6a8b460c/html5/thumbnails/14.jpg)
Stream of Consciousness
• Developer Driven Testing
• The Essence of (x)Unit Testing
• Those Pesky Dependencies
• Code & Test Evolution in Practice
![Page 15: Using xUnit as a Swiss-Aarmy Testing Toolkit](https://reader034.vdocuments.net/reader034/viewer/2022051411/540b1e698d7f72da6a8b460c/html5/thumbnails/15.jpg)
Pesky Dependencies
External System 1 External System 2 External System 3
The SystemService 2
Service 1
File-System
Database
![Page 16: Using xUnit as a Swiss-Aarmy Testing Toolkit](https://reader034.vdocuments.net/reader034/viewer/2022051411/540b1e698d7f72da6a8b460c/html5/thumbnails/16.jpg)
xUnit Abuse
• Fight the Shadow Cache
• Invoke TearDown from SetUp
• Test/build failure isn’t absolute
![Page 17: Using xUnit as a Swiss-Aarmy Testing Toolkit](https://reader034.vdocuments.net/reader034/viewer/2022051411/540b1e698d7f72da6a8b460c/html5/thumbnails/17.jpg)
File-System (Reading)
• Source Control directory
• Build server directory
• Resource files
![Page 18: Using xUnit as a Swiss-Aarmy Testing Toolkit](https://reader034.vdocuments.net/reader034/viewer/2022051411/540b1e698d7f72da6a8b460c/html5/thumbnails/18.jpg)
File-System (Writing)
• TEMP directory
• Output directory
![Page 19: Using xUnit as a Swiss-Aarmy Testing Toolkit](https://reader034.vdocuments.net/reader034/viewer/2022051411/540b1e698d7f72da6a8b460c/html5/thumbnails/19.jpg)
Database
• Per-user / per-branch workspace
• Only need schema not data (Integration)
• Can reuse existing unit test database
• Use same code revision for compatibility
• Use transactions to avoid residual effects
• Fake tables with CSV files
![Page 20: Using xUnit as a Swiss-Aarmy Testing Toolkit](https://reader034.vdocuments.net/reader034/viewer/2022051411/540b1e698d7f72da6a8b460c/html5/thumbnails/20.jpg)
Database Asserts
public void AddCustomer_Should_Persist_The_Customer(){ const id = 1234; const name = "name";
var customer = new Customer(. . .);
using (var connection = AcquireConnection()) { CustomerDataMapper.AddCustomer(customer, connection);
Assert.That(RowExists("dbo.Customer", " CustomerId = {0}" + " AND CustomerName = '{1}'", id, name), Is.True); }}
![Page 21: Using xUnit as a Swiss-Aarmy Testing Toolkit](https://reader034.vdocuments.net/reader034/viewer/2022051411/540b1e698d7f72da6a8b460c/html5/thumbnails/21.jpg)
Database SetUp/TearDown[TestFixture, TestCategory.DatabaseTest]public class SomeEntityTests : DatabaseTestBase{
[TestFixtureSetUp] public void FixtureSetUp { using(var connection = AcquireConnection()) { connection.Execute("insert into thingy_table values(1, 2, 3)");
connection.Execute("test.InsertThingy(1, 2, 3)"); } }
[TestFixtureTearDown] public void FixtureTearDown { using(var connection = AcquireConnection()) { connection.Execute("delete from thingy_table");
connection.Execute("test.DeleteAllThingys"); } }
}
![Page 22: Using xUnit as a Swiss-Aarmy Testing Toolkit](https://reader034.vdocuments.net/reader034/viewer/2022051411/540b1e698d7f72da6a8b460c/html5/thumbnails/22.jpg)
Helper Base Classpublic class DatabaseTestBase{ public ISqlConnection AcquireConnection() { return . . . }
. . .
public bool RowExists(string table, string where, string params[]) { string filter = String.Format(where, params);
string sql = String.Format( "select count(*) as [Count] from {0} where {1}" , table, filter);
using (var connection = AcquireConnection()) { var reader = connection.ExecuteQuery(sql);
return (reader.GetInt("Count") == 1); } }
. . .}
![Page 23: Using xUnit as a Swiss-Aarmy Testing Toolkit](https://reader034.vdocuments.net/reader034/viewer/2022051411/540b1e698d7f72da6a8b460c/html5/thumbnails/23.jpg)
External Systems
• Verify API behaviour
• Test internal façade
• Reliability varies (DEV vs PROD)
![Page 24: Using xUnit as a Swiss-Aarmy Testing Toolkit](https://reader034.vdocuments.net/reader034/viewer/2022051411/540b1e698d7f72da6a8b460c/html5/thumbnails/24.jpg)
Stream of Consciousness
• Developer Driven Testing
• The Essence of (x)Unit Testing
• Those Pesky Dependencies
• Code & Test Evolution in Practice
![Page 25: Using xUnit as a Swiss-Aarmy Testing Toolkit](https://reader034.vdocuments.net/reader034/viewer/2022051411/540b1e698d7f72da6a8b460c/html5/thumbnails/25.jpg)
System Architecture
Market Data Trade Data Analytics
The System
42
ServicesDatabase
![Page 26: Using xUnit as a Swiss-Aarmy Testing Toolkit](https://reader034.vdocuments.net/reader034/viewer/2022051411/540b1e698d7f72da6a8b460c/html5/thumbnails/26.jpg)
Initial System Test
MarketData Service
TradeData Service
Analytics Service
Calculator
Test Runner
System Tests
[Test, TestCategory.SystemTest]public void Calculate_Answer(){ . . .
var result = c.calculate();
Assert.Equal(result, 42);}
![Page 27: Using xUnit as a Swiss-Aarmy Testing Toolkit](https://reader034.vdocuments.net/reader034/viewer/2022051411/540b1e698d7f72da6a8b460c/html5/thumbnails/27.jpg)
Addressing External Risks
External MarketData Service API
External TradeData Service API
External MarketData Service Tests
External TradeData Service Tests
Test Runner
![Page 28: Using xUnit as a Swiss-Aarmy Testing Toolkit](https://reader034.vdocuments.net/reader034/viewer/2022051411/540b1e698d7f72da6a8b460c/html5/thumbnails/28.jpg)
Internal Service Design
External ServiceAPI
External ServiceTests
Internal Service
ExternalService Facade
Internal ServiceTests
MockExternal Services
PerformanceTest Runner
Mock Service
![Page 29: Using xUnit as a Swiss-Aarmy Testing Toolkit](https://reader034.vdocuments.net/reader034/viewer/2022051411/540b1e698d7f72da6a8b460c/html5/thumbnails/29.jpg)
Data Access Layer
DatabasePublic Interface
Database UnitTests
Data AccessLayer
Data AccessLayer Tests
Database APIMock Database
API
Mock DataAccess Layer
![Page 30: Using xUnit as a Swiss-Aarmy Testing Toolkit](https://reader034.vdocuments.net/reader034/viewer/2022051411/540b1e698d7f72da6a8b460c/html5/thumbnails/30.jpg)
DatabasePublic Interface
External AnalyticsService
External MarketData Service API
External MarketData Service API
System Evolution
Mock MarketData Service
Mock TradeData Service
Mock AnalyticsService
Calculator
Test Runner
Unit / Integration/ System Tests
[Test, TestCategory.SystemTest]public void Calc_Answer_For_ABC_Plc(){ . . .
var result = c.calculate();
Assert.Equal(result, 41.75);}
Mock DataAccess Layer
MarketData Service
TradeData Service
Analytics Service
Data AccessLayer