![Page 2: ESGF-test-suite ESGF-jenkins Sébastien Gardoll IPSL ......2018/12/06 · Sébastien Gardoll IPSL, software engineer sgardoll@ipsl.fr ESGF-test-suite ESGF-jenkins Dec 6th, 2018 2](https://reader030.vdocuments.net/reader030/viewer/2022041016/5ec9705c38000a687d6ec2e8/html5/thumbnails/2.jpg)
2
Continuous Integration (&delivery) system (CI)
Continuous Deployment system (CD)
Continuous Integration (CI)
Goal: automation of software production.
version controlsystem testing engine packaging
system
deploymentsystem
CI job
repositorysystem
building system
CD job
1/4
Typical architectureexample
![Page 3: ESGF-test-suite ESGF-jenkins Sébastien Gardoll IPSL ......2018/12/06 · Sébastien Gardoll IPSL, software engineer sgardoll@ipsl.fr ESGF-test-suite ESGF-jenkins Dec 6th, 2018 2](https://reader030.vdocuments.net/reader030/viewer/2022041016/5ec9705c38000a687d6ec2e8/html5/thumbnails/3.jpg)
3
Continuous Integration (&delivery) system (CI)
Continuous Deployment system (CD)
Continuous Integration (CI)
Typical case of CI workflow: management of Pull Requests.
version controlsystem testing engine packaging
system
deploymentsystem
CI job
repositorysystem
building system
CD job
1/4
1. Pull Request (into master)
![Page 4: ESGF-test-suite ESGF-jenkins Sébastien Gardoll IPSL ......2018/12/06 · Sébastien Gardoll IPSL, software engineer sgardoll@ipsl.fr ESGF-test-suite ESGF-jenkins Dec 6th, 2018 2](https://reader030.vdocuments.net/reader030/viewer/2022041016/5ec9705c38000a687d6ec2e8/html5/thumbnails/4.jpg)
4
Continuous Integration (&delivery) system (CI)
Continuous Deployment system (CD)
Continuous Integration (CI)
version controlsystem testing engine packaging
system
deploymentsystem
CI job
repositorysystem
building system
CD job
1/4
2. triggerjob
3.
PR pending until the successful completion of the CI job.
1. Pull Request (into master)
![Page 5: ESGF-test-suite ESGF-jenkins Sébastien Gardoll IPSL ......2018/12/06 · Sébastien Gardoll IPSL, software engineer sgardoll@ipsl.fr ESGF-test-suite ESGF-jenkins Dec 6th, 2018 2](https://reader030.vdocuments.net/reader030/viewer/2022041016/5ec9705c38000a687d6ec2e8/html5/thumbnails/5.jpg)
5
Continuous Integration (&delivery) system (CI)
Continuous Deployment system (CD)
Continuous Integration (CI)
version controlsystem testing engine packaging
system
deploymentsystem
CI job
repositorysystem
building system
CD job
1/4
2. triggerjob
3. new build
The application is built including the modifications introduced by the PR.
1. Pull Request (into master)
![Page 6: ESGF-test-suite ESGF-jenkins Sébastien Gardoll IPSL ......2018/12/06 · Sébastien Gardoll IPSL, software engineer sgardoll@ipsl.fr ESGF-test-suite ESGF-jenkins Dec 6th, 2018 2](https://reader030.vdocuments.net/reader030/viewer/2022041016/5ec9705c38000a687d6ec2e8/html5/thumbnails/6.jpg)
6
Continuous Integration (&delivery) system (CI)
Continuous Deployment system (CD)
Continuous Integration (CI)
version controlsystem testing engine packaging
system
deploymentsystem
CI job
repositorysystem
building system
CD job
1/4
2. triggerjob
3. new build
4.
Integration tests are run...
1. Pull Request (into master)
![Page 7: ESGF-test-suite ESGF-jenkins Sébastien Gardoll IPSL ......2018/12/06 · Sébastien Gardoll IPSL, software engineer sgardoll@ipsl.fr ESGF-test-suite ESGF-jenkins Dec 6th, 2018 2](https://reader030.vdocuments.net/reader030/viewer/2022041016/5ec9705c38000a687d6ec2e8/html5/thumbnails/7.jpg)
7
Continuous Integration (&delivery) system (CI)
Continuous Deployment system (CD)
Continuous Integration (CI)
version controlsystem testing engine packaging
system
deploymentsystem
CI job
repositorysystem
building system
CD job
1/4
2. triggerjob
3. new build
PR validated
4.
and the PR is validated only if the tests are passed.
1. Pull Request (into master)
![Page 8: ESGF-test-suite ESGF-jenkins Sébastien Gardoll IPSL ......2018/12/06 · Sébastien Gardoll IPSL, software engineer sgardoll@ipsl.fr ESGF-test-suite ESGF-jenkins Dec 6th, 2018 2](https://reader030.vdocuments.net/reader030/viewer/2022041016/5ec9705c38000a687d6ec2e8/html5/thumbnails/8.jpg)
8
Continuous Integration (&delivery) system (CI)
Continuous Deployment system (CD)
Continuous Integration (CI)
Then application is packaged...
version controlsystem testing engine packaging
system
deploymentsystem
CI job
repositorysystem
building system
CD job
1/4
2. triggerjob
3. new build
PR validated
4.
5.
1. Pull Request (into master)
new release
![Page 9: ESGF-test-suite ESGF-jenkins Sébastien Gardoll IPSL ......2018/12/06 · Sébastien Gardoll IPSL, software engineer sgardoll@ipsl.fr ESGF-test-suite ESGF-jenkins Dec 6th, 2018 2](https://reader030.vdocuments.net/reader030/viewer/2022041016/5ec9705c38000a687d6ec2e8/html5/thumbnails/9.jpg)
9
Continuous Integration (&delivery) system (CI)
Continuous Deployment system (CD)
Continuous Integration (CI)
and uploaded so as to be available for every person concerned.
version controlsystem testing engine packaging
system
deploymentsystem
CI job
repositorysystem
building system
CD job
1/4
2. triggerjob
3. new build
PR validated
4.
5. 6.
new releaseavailable
1. Pull Request (into master)
new release
![Page 10: ESGF-test-suite ESGF-jenkins Sébastien Gardoll IPSL ......2018/12/06 · Sébastien Gardoll IPSL, software engineer sgardoll@ipsl.fr ESGF-test-suite ESGF-jenkins Dec 6th, 2018 2](https://reader030.vdocuments.net/reader030/viewer/2022041016/5ec9705c38000a687d6ec2e8/html5/thumbnails/10.jpg)
10
Continuous Integration (&delivery) system (CI)
Continuous Deployment system (CD)
Continuous Integration (CI)
A successful CI job allows the PR to be merged into the code base.
version controlsystem testing engine packaging
system
deploymentsystem
CI job
repositorysystem
building system
CD job
1/4
2. triggerjob
3. new build
PR validated
4.
5. 6.
new releaseavailable
1. Pull Request (into master)
7. allow merging
new release
![Page 11: ESGF-test-suite ESGF-jenkins Sébastien Gardoll IPSL ......2018/12/06 · Sébastien Gardoll IPSL, software engineer sgardoll@ipsl.fr ESGF-test-suite ESGF-jenkins Dec 6th, 2018 2](https://reader030.vdocuments.net/reader030/viewer/2022041016/5ec9705c38000a687d6ec2e8/html5/thumbnails/11.jpg)
11
Continuous Integration (&delivery) system (CI)
Continuous Deployment system (CD)
Continuous Integration (CI)
CI job can optionally trigger a Continuous Deployment job...
version controlsystem testing engine packaging
system
deploymentsystem
CI job
repositorysystem
building system
CD job
1/4
2. triggerjob
3. new build
PR validated
4.
5. 6.
new releaseavailable
8. trigger job
1. Pull Request (into master)
7. allow merging
new release
![Page 12: ESGF-test-suite ESGF-jenkins Sébastien Gardoll IPSL ......2018/12/06 · Sébastien Gardoll IPSL, software engineer sgardoll@ipsl.fr ESGF-test-suite ESGF-jenkins Dec 6th, 2018 2](https://reader030.vdocuments.net/reader030/viewer/2022041016/5ec9705c38000a687d6ec2e8/html5/thumbnails/12.jpg)
12
Continuous Integration (&delivery) system (CI)
Continuous Deployment system (CD)
Continuous Integration (CI)
so as to deploy the new version of the application.
version controlsystem testing engine packaging
system
deploymentsystem
CI job
repositorysystem
building system
CD job
1/4
2. triggerjob
3. new build
PR validated
4.
5. 6.
new releaseavailable
8. trigger jobnew releaserunning
1. Pull Request (into master)
7. allow merging
new release
![Page 13: ESGF-test-suite ESGF-jenkins Sébastien Gardoll IPSL ......2018/12/06 · Sébastien Gardoll IPSL, software engineer sgardoll@ipsl.fr ESGF-test-suite ESGF-jenkins Dec 6th, 2018 2](https://reader030.vdocuments.net/reader030/viewer/2022041016/5ec9705c38000a687d6ec2e8/html5/thumbnails/13.jpg)
13
Continuous Integration (CI)2/4
The benefits of the CI (after some reasonable efforts):
● Prevent software regression: prevent faulty code from getting merged into the main repo.
● Less error prone: human out of the loop but everything is still manually operable.
● Improve reactive security: increase the frequency of releasing, from a CVE to a new release.
● Etc.
![Page 14: ESGF-test-suite ESGF-jenkins Sébastien Gardoll IPSL ......2018/12/06 · Sébastien Gardoll IPSL, software engineer sgardoll@ipsl.fr ESGF-test-suite ESGF-jenkins Dec 6th, 2018 2](https://reader030.vdocuments.net/reader030/viewer/2022041016/5ec9705c38000a687d6ec2e8/html5/thumbnails/14.jpg)
14
Continuous Integration (CI)3/4
ESGF-test-suite & ESGF-jenkins role:
ESGF-jenkins: Continuous Integration (&delivery) system (CI)
Continuous Deployment system (CD)
version controlsystem ESGF-test-suite
testing enginepackaging
system
deploymentsystem
CI job
repositorysystem
build system
CD job
![Page 15: ESGF-test-suite ESGF-jenkins Sébastien Gardoll IPSL ......2018/12/06 · Sébastien Gardoll IPSL, software engineer sgardoll@ipsl.fr ESGF-test-suite ESGF-jenkins Dec 6th, 2018 2](https://reader030.vdocuments.net/reader030/viewer/2022041016/5ec9705c38000a687d6ec2e8/html5/thumbnails/15.jpg)
15
Continuous Integration (CI)4/4
Notes:● ESGF-test-suite is not specific to Continuous
Integration and ESGF-docker (e.g. can check your production ESGF nodes).
● ESGF-jenkins is not specific to ESGF-docker: it welcomes any ESGF projects.
A CI example involving esgf-docker:
ESGF-jenkins: Continuous Integration (&delivery) system (CI)
Continuous Deployment system (CD)
githubversion control
systemESGF-test-suite
testing engine
docker-composepackaging
system
helm/kubernetesdeployment
system
CI job
dockerhubrepository
system
docker-composebuilding system
CD job
![Page 16: ESGF-test-suite ESGF-jenkins Sébastien Gardoll IPSL ......2018/12/06 · Sébastien Gardoll IPSL, software engineer sgardoll@ipsl.fr ESGF-test-suite ESGF-jenkins Dec 6th, 2018 2](https://reader030.vdocuments.net/reader030/viewer/2022041016/5ec9705c38000a687d6ec2e8/html5/thumbnails/16.jpg)
16
ESGF-test-suite1/2
Goals:● Gather the integration tests of the ESGF stack.● Developers of the ESGF stack are invited to implement/port their integration tests into
ESGF-test-suite.
Architecture: a well organized set of Python 2.7 scripts based on Nose2 (also based on Unittest). Git repository: https://github.com/ESGF/esgf-test-suite . Author: Nicolas Carenton.
For the time being, only fundamental integration tests has been implemented:● Pinging web pages (cog, thredds, stats, search, orp, etc.).● Cog/django login.● Http and gridftp downloading.
Features provided by esgf-test-suite:● Myproxy authentication (credential) routines.● High levels web crawling routines based on Selenium API, using Firefox.● High levels Thredds catalog routines.● Nice reporting (human readable, JSON, XML, etc.).● Selection of tests thanks to Nose.
ESGF-test-suite is standalone application with its dependencies wrapped into a singularity image (container system):● Install singularity, once.● Run a interactive shell within the image of the dependencies of ESGF-test-suite.● Run ESGF-test-suite.
![Page 17: ESGF-test-suite ESGF-jenkins Sébastien Gardoll IPSL ......2018/12/06 · Sébastien Gardoll IPSL, software engineer sgardoll@ipsl.fr ESGF-test-suite ESGF-jenkins Dec 6th, 2018 2](https://reader030.vdocuments.net/reader030/viewer/2022041016/5ec9705c38000a687d6ec2e8/html5/thumbnails/17.jpg)
17
ESGF-test-suite2/3
![Page 18: ESGF-test-suite ESGF-jenkins Sébastien Gardoll IPSL ......2018/12/06 · Sébastien Gardoll IPSL, software engineer sgardoll@ipsl.fr ESGF-test-suite ESGF-jenkins Dec 6th, 2018 2](https://reader030.vdocuments.net/reader030/viewer/2022041016/5ec9705c38000a687d6ec2e8/html5/thumbnails/18.jpg)
18
ESGF-test-suite3/3
Adding new integration tests:
1. Create a new file prefixed “test_” in the tests directory
ESGF-test-suite parent directory
tests
utils
test_… .py
2. Implement a Python class (set of tests) and some methods (tests) with annotations (ids):
inherits routine methods
using a routine method
ids of a set of tests
scope of a set of tests
unique id of a test
implementation of a test
![Page 19: ESGF-test-suite ESGF-jenkins Sébastien Gardoll IPSL ......2018/12/06 · Sébastien Gardoll IPSL, software engineer sgardoll@ipsl.fr ESGF-test-suite ESGF-jenkins Dec 6th, 2018 2](https://reader030.vdocuments.net/reader030/viewer/2022041016/5ec9705c38000a687d6ec2e8/html5/thumbnails/19.jpg)
19
ESGF-jenkins
mastercredentialmanager
CI jobwebUI
REST API
worker #1
CI job Docker
Singularity
worker #2
CI job Docker
Singularity
Goals:
● Run CI jobs (provide computation resources).
● Ease the interactions between CI jobs and other applications (Version Control System, Continuous Deployment systems, etc.).
ESGF-jenkins, a cluster of 3 Jenkins nodes:
● Scalable (by adding new worker nodes).● Job failover (across the nodes of a cluster).● Extensible (plugin system).● Nice web interface (https://esgf-build.ipsl.upmc.fr/jenkins/).● Point&Click configuration of Job.● CI Job description language (Jenkins pipeline script named Jenkinsfile).● GUI Job pipeline builder.● Secure credential manager.● Provides REST API.
Working capacities:
● Every node can run 2 jobs at the same time.
● Worker nodes can run 1 job using Docker and Singularity at the same time.
● Provided job subsystems (Docker, Singularity, Git, Maven, Ant, Gradle, Bash, etc.).
1/6
![Page 20: ESGF-test-suite ESGF-jenkins Sébastien Gardoll IPSL ......2018/12/06 · Sébastien Gardoll IPSL, software engineer sgardoll@ipsl.fr ESGF-test-suite ESGF-jenkins Dec 6th, 2018 2](https://reader030.vdocuments.net/reader030/viewer/2022041016/5ec9705c38000a687d6ec2e8/html5/thumbnails/20.jpg)
20
ESGF-jenkins
ESGF-jenkins
Github
Slack#esgf-docker-ci
Dockerhub
sends notifications (pushes, Pull Request creations, etc.)allows merging PR,
creates tags, etc.
sends notifications(image/tag pushed, etc.)
pushes containerimages
sends notifications
Examples of interactions
2/6
A complete and commented job script is available at:https://github.com/ESGF/esgf-docker/blob/master/Jenkinsfile
![Page 21: ESGF-test-suite ESGF-jenkins Sébastien Gardoll IPSL ......2018/12/06 · Sébastien Gardoll IPSL, software engineer sgardoll@ipsl.fr ESGF-test-suite ESGF-jenkins Dec 6th, 2018 2](https://reader030.vdocuments.net/reader030/viewer/2022041016/5ec9705c38000a687d6ec2e8/html5/thumbnails/21.jpg)
21
ESGF-jenkins3/6
Job status (failed, successful, presented like a weather forecast):
Job stage view:
![Page 22: ESGF-test-suite ESGF-jenkins Sébastien Gardoll IPSL ......2018/12/06 · Sébastien Gardoll IPSL, software engineer sgardoll@ipsl.fr ESGF-test-suite ESGF-jenkins Dec 6th, 2018 2](https://reader030.vdocuments.net/reader030/viewer/2022041016/5ec9705c38000a687d6ec2e8/html5/thumbnails/22.jpg)
22
ESGF-jenkins4/6
Exploring the log of a CI job:
![Page 23: ESGF-test-suite ESGF-jenkins Sébastien Gardoll IPSL ......2018/12/06 · Sébastien Gardoll IPSL, software engineer sgardoll@ipsl.fr ESGF-test-suite ESGF-jenkins Dec 6th, 2018 2](https://reader030.vdocuments.net/reader030/viewer/2022041016/5ec9705c38000a687d6ec2e8/html5/thumbnails/23.jpg)
23
ESGF-jenkins5/6
Builder of pipelines of jobs:
![Page 24: ESGF-test-suite ESGF-jenkins Sébastien Gardoll IPSL ......2018/12/06 · Sébastien Gardoll IPSL, software engineer sgardoll@ipsl.fr ESGF-test-suite ESGF-jenkins Dec 6th, 2018 2](https://reader030.vdocuments.net/reader030/viewer/2022041016/5ec9705c38000a687d6ec2e8/html5/thumbnails/24.jpg)
24
ESGF-jenkins6/6
Slack notifications (a PR successively failed then successfully tested):
Words in blue are hyperlinks that point to the jobs.
![Page 25: ESGF-test-suite ESGF-jenkins Sébastien Gardoll IPSL ......2018/12/06 · Sébastien Gardoll IPSL, software engineer sgardoll@ipsl.fr ESGF-test-suite ESGF-jenkins Dec 6th, 2018 2](https://reader030.vdocuments.net/reader030/viewer/2022041016/5ec9705c38000a687d6ec2e8/html5/thumbnails/25.jpg)
25
Thank you for your attention
Want an ESGF-jenkins account ?
Need help to implement integration tests for ESGF-test-suite ?
send me an email: [email protected]