DIY (Do-it-yourself)

Testing

Ruby Engineer on the drivers team.

Based in Berlin.Maintain the gems:

bson mongo mongoid bson_ext mongo_kerberos

@EmStolfo

MongoDB DriversInterface to the server.

application driver MongoDBAPITCP

MongoDB DriversC C++ C# Java Node.js Perl PHP Python Ruby Scala

MongoDB Drivers

10 official drivers available.

Many started as OS projects.

History

No specifications.Divergent codebases

and features.

MongoDB DriversReality

Organizations can experience 1 interface.

Specifications guide design and provide documentation.

Rewrites with collective knowledge.

MongoDB DriversRevamp

Specifications

Equally valuable to the drivers team as to the community.

MongoDB Drivers

driver

MongoDB

MongoDB

10 ms

20 ms

10 ms

25 ms

(15 ms)

15 ms

"replica set"(15 ms)

The Spec Determine the "nearest" member as the one with the quickest response to a ping.“Choose a member randomly from those at most 15ms "farther" than the nearest member.

„I'm bored

How to validate compliance?

Requirements for validating compliance• Unit tests. • Integration tests. • Tests define as “data”. • Language-agnostic. • Reproducible scenarios.

define

Format - YAML Tests

Scenarios

Our approach:“DIY” testing

benefitsFormat: YAML

• Describes data. • Can translate to actions. • Most languages can use a YAML parsing

library. • Driver authors write a reusable harness. • Changes in the spec can be

communicated via additional YAML tests or changes to existing ones.

Test harness in each driver

1. Resource (mongodb) 2. Specification (parsed YAML) 3. Test (can be run)

3 types of tests

simple: (de)serialization

intermediate: driverhard: driver + mongodb

Simple: (de)serialization

BSON unit testsported from the General Decimal Arithmetic

Specification by Mike Cowlishaw http://speleotrove.com/decimal/dectest.html

Unit test (Decimal128)decimal128/decimal128-1.yml

SDAM unit tests

Intermediate: Driver

Server Discovery and Monitoring Spec

Logic required to make an application using MongoDB highly available.

Unit test #1 - YAMLsingle/direct_connection_standalone.yml

SDAM integration tests

Hard: Driver + server

Mongo Orchestration

Define clusters using JSON.Manipulate clusters via RESTful API.

Mongo OrchestrationHTTP server providing REST interface to manage

multiple MongoDB processes on the same machine.

Implemented in python.

Setup

The Ruby driver’s test harness

Resourcespec/support/mongo_orchestration/resource.rb

Specificationspec/support/mongo_orchestration/spec.rb

Testspec/support/mongo_orchestration/operation/client_operation.rb

YAML integration testrs/connection/primary-not-available.yml (I)

YAML integration testrs/connection/primary-not-available.yml (II)

How to build your own testing framework

Ask yourself1. Unit or integration tests? 2. Need to be language-agnostic? 3. Directly related to specs or

documentation?

Test framework entities

Resource Wrapper around

the external component.

Test framework entities

Specification The parsed YAML.

Test framework entities

State manipulator Processor of phases to get

to a certain state.

Test framework entities

Tests Can be executed.

Result is ‘pass’ or ‘fail’.

Make the test runner as generic as possible.

Install Mongo Orchestration