continuous integration (ci) of fpga designs for cms...•using gitlabci/cd •git-based, fully...
TRANSCRIPT
![Page 1: Continuous Integration (CI) of FPGA Designs for CMS...•Using GitLabCI/CD •Git-based, fully featured platform for SW and FW development •FW requires CLI build system •Support](https://reader034.vdocuments.net/reader034/viewer/2022042219/5ec5664bfd680a5105410455/html5/thumbnails/1.jpg)
Continuous Integration (CI) of FPGA Designs for CMS
Topical Workshop on Electronics for Particle Physics (TWEPP) 2019
Robért Glein on behalf of the CMS Collaboration, CU Boulder, September 3rd 2019
![Page 2: Continuous Integration (CI) of FPGA Designs for CMS...•Using GitLabCI/CD •Git-based, fully featured platform for SW and FW development •FW requires CLI build system •Support](https://reader034.vdocuments.net/reader034/viewer/2022042219/5ec5664bfd680a5105410455/html5/thumbnails/2.jpg)
Problem: How to manage, build, and test FW designs?• Precondition: The firmware (FW) code is versioned with git
(or a similar versioning system)
• FW development• High degree of flexibility• Modular design often with various (git) submodules• Various tools for simulation (HDL), synthesis (netlist translation),
implementation (place and route), and test of FPGA designs • Time intensive (functional) verification using various methods
• Solution: Continuous integration (CI) of FW
09/03/19 Continuous Firmware Integration 2
![Page 3: Continuous Integration (CI) of FPGA Designs for CMS...•Using GitLabCI/CD •Git-based, fully featured platform for SW and FW development •FW requires CLI build system •Support](https://reader034.vdocuments.net/reader034/viewer/2022042219/5ec5664bfd680a5105410455/html5/thumbnails/3.jpg)
Outline
• Problem Description and Motivation
• Continuous Integration (CI) Overview
• Command Line Interface (CLI) FW Build Systems
• Continuous Integration / Continuous Deployment (CI/CD) Tools
• Demonstration
• Conclusion
Continuous Firmware Integration09/03/19 3
![Page 4: Continuous Integration (CI) of FPGA Designs for CMS...•Using GitLabCI/CD •Git-based, fully featured platform for SW and FW development •FW requires CLI build system •Support](https://reader034.vdocuments.net/reader034/viewer/2022042219/5ec5664bfd680a5105410455/html5/thumbnails/4.jpg)
Motivation of Continuous Integration for Firmware• Widely used for software• Especially useful with multiple developers working on the
same or interconnected FW• Automated check, build, simulation, and test from scratch
• Uniform build, simulation, and test environment• An error can be traced to the exact commit• Covers specified functionalities via simulation and test• No risk of human error (changed constants, submodule not
updated, specific tool not installed, …)• Manual verification (simulation and HW) of FW is minimized
• Especially if different FPGAs and/or configurations are supported
• Requires a command line interface (CLI) build system Continuous Firmware Integration09/03/19 4
![Page 5: Continuous Integration (CI) of FPGA Designs for CMS...•Using GitLabCI/CD •Git-based, fully featured platform for SW and FW development •FW requires CLI build system •Support](https://reader034.vdocuments.net/reader034/viewer/2022042219/5ec5664bfd680a5105410455/html5/thumbnails/5.jpg)
Continuous Integration (CI) Overview
Continuous Firmware Integration
Code:.vhd, .v, .xci, .bd,
…
Commit
Related code:build,
library, …
Simulation:ModelSim,
Vivado
Unit test
Build:synth, impl,
bitgen
Continuous Deployment (CD):
Review, stage, deploy
09/03/19 5
HW test
![Page 6: Continuous Integration (CI) of FPGA Designs for CMS...•Using GitLabCI/CD •Git-based, fully featured platform for SW and FW development •FW requires CLI build system •Support](https://reader034.vdocuments.net/reader034/viewer/2022042219/5ec5664bfd680a5105410455/html5/thumbnails/6.jpg)
Continuous Integration (CI) Overview
Continuous Firmware Integration
Code:.vhd, .v, .xci, .bd,
…
Commit
Related code:build,
library, …
Simulation:ModelSim,
Vivado
Unit test
Build:synth, impl,
bitgen
Continuous Deployment (CD):
Review, stage, deploy
09/03/19 6
HW test
• Manual code commit
•
•
![Page 7: Continuous Integration (CI) of FPGA Designs for CMS...•Using GitLabCI/CD •Git-based, fully featured platform for SW and FW development •FW requires CLI build system •Support](https://reader034.vdocuments.net/reader034/viewer/2022042219/5ec5664bfd680a5105410455/html5/thumbnails/7.jpg)
Continuous Integration (CI) Overview
Continuous Firmware Integration
Code:.vhd, .v, .xci, .bd,
…
Commit
Related code:build,
library, …
Simulation:ModelSim,
Vivado
Unit test
Build:synth, impl,
bitgen
Continuous Deployment (CD):
Review, stage, deploy
09/03/19 7
HW test
• Manual code commit
• Automated simulation (unit test), build, and HW tests
•
![Page 8: Continuous Integration (CI) of FPGA Designs for CMS...•Using GitLabCI/CD •Git-based, fully featured platform for SW and FW development •FW requires CLI build system •Support](https://reader034.vdocuments.net/reader034/viewer/2022042219/5ec5664bfd680a5105410455/html5/thumbnails/8.jpg)
Continuous Integration (CI) Overview
Continuous Firmware Integration
Code:.vhd, .v, .xci, .bd,
…
Commit
Related code:build,
library, …
Simulation:ModelSim,
Vivado
Unit test
Build:synth, impl,
bitgen
Continuous Deployment (CD):
Review, stage, deploy
09/03/19 8
HW test
• Manual code commit
• Automated simulation (unit test), build, and HW tests
• Manual review and automatic deployment
![Page 9: Continuous Integration (CI) of FPGA Designs for CMS...•Using GitLabCI/CD •Git-based, fully featured platform for SW and FW development •FW requires CLI build system •Support](https://reader034.vdocuments.net/reader034/viewer/2022042219/5ec5664bfd680a5105410455/html5/thumbnails/9.jpg)
Outline
• Problem Description and Motivation
• Continuous Integration (CI) Overview
• Command Line Interface (CLI) FW Build Systems
• Continuous Integration / Continuous Deployment (CI/CD) Tools
• Demonstration
• Conclusion
Continuous Firmware Integration09/03/19 9
![Page 10: Continuous Integration (CI) of FPGA Designs for CMS...•Using GitLabCI/CD •Git-based, fully featured platform for SW and FW development •FW requires CLI build system •Support](https://reader034.vdocuments.net/reader034/viewer/2022042219/5ec5664bfd680a5105410455/html5/thumbnails/10.jpg)
Firmware CLI Build System:Ruckus by SLAC• Well structured Makefile based CLI
• Supports incremental syn/impl feature (dcp), partial reconfiguration, and integration of HLS (all not tested by me)
• Some commands (right level of granularity):• make depend Vivado Project Creation• make sources Vivado Source Setup
• make xsim Vivado Simulation (CU contribution)• make syn Vivado Synthesis Only
• make bit Vivado Build
• make interactive Vivado Interactive (CLI)
• make gui Vivado GUI
• Project website: https://github.com/slaclab/ruckus
• Documentation: https://docs.google.com/presentation/d/1kvzXiByE8WISo40Xd573DdR7dQU4BpDQGwEgNyeJjTI/
Continuous Firmware Integration09/03/19 10
![Page 11: Continuous Integration (CI) of FPGA Designs for CMS...•Using GitLabCI/CD •Git-based, fully featured platform for SW and FW development •FW requires CLI build system •Support](https://reader034.vdocuments.net/reader034/viewer/2022042219/5ec5664bfd680a5105410455/html5/thumbnails/11.jpg)
Firmware CLI Build System:Investigated Alternatives • IP Bus Builder (IPBB)
• This Python tool is based on hierarchical dependency files *.dep in cfg folders to include sources
• Includes make-project, simulation, synth, impl, bit file creation, …• https://ipbus.web.cern.ch/ipbus/doc/user/html/firmware/ipbb-
primer.html
• HDL Make • Makefile based CLI but does not support block designs
(recursively)• https://www.ohwr.org/projects/hdl-make
• Vendor tools with CLI support (e.g. TCL)
Continuous Firmware Integration09/03/19 11
![Page 12: Continuous Integration (CI) of FPGA Designs for CMS...•Using GitLabCI/CD •Git-based, fully featured platform for SW and FW development •FW requires CLI build system •Support](https://reader034.vdocuments.net/reader034/viewer/2022042219/5ec5664bfd680a5105410455/html5/thumbnails/12.jpg)
Outline
• Problem Description and Motivation
• Continuous Integration (CI) Overview
• Command Line Interface (CLI) FW Build Systems
• Continuous Integration / Continuous Deployment (CI/CD) Tools
• Demonstration
• Conclusion
Continuous Firmware Integration09/03/19 12
![Page 13: Continuous Integration (CI) of FPGA Designs for CMS...•Using GitLabCI/CD •Git-based, fully featured platform for SW and FW development •FW requires CLI build system •Support](https://reader034.vdocuments.net/reader034/viewer/2022042219/5ec5664bfd680a5105410455/html5/thumbnails/13.jpg)
GitLab Continuous Integration / Continuous Deployment (CI/CD)
• Documentation: https://docs.gitlab.com/ee/ci/ and https://docs.gitlab.com/ee/ci/yaml/
• Can be used without any source files in the GitLabrepository (e.g. all code located in GitHub)
Continuous Firmware Integration
Source: https://docs.gitlab.com/ee/ci/README.html
09/03/19 13
![Page 14: Continuous Integration (CI) of FPGA Designs for CMS...•Using GitLabCI/CD •Git-based, fully featured platform for SW and FW development •FW requires CLI build system •Support](https://reader034.vdocuments.net/reader034/viewer/2022042219/5ec5664bfd680a5105410455/html5/thumbnails/14.jpg)
GitLab CI/CDInvestigated Alternatives
• https://about.gitlab.com/devops-tools/
• CircleCI: https://circleci.com
• Supposed to have the best GitHub integration
• After a glimpse, I did not find runner options (execute a script
on any workstation)
• Jenkins: https://jenkins.io
• Solid CI tool but not that powerful
• Travis, …
09/03/19 Continuous Firmware Integration 14
![Page 15: Continuous Integration (CI) of FPGA Designs for CMS...•Using GitLabCI/CD •Git-based, fully featured platform for SW and FW development •FW requires CLI build system •Support](https://reader034.vdocuments.net/reader034/viewer/2022042219/5ec5664bfd680a5105410455/html5/thumbnails/15.jpg)
Gitlab CI/CD Runner (Executing Workstation)
• https://docs.gitlab.com/ee/ci/runners/README.html
• Run the code defined in .gitlab-ci.yml (next slide)
• Isolated (virtual) machine: Pick up jobs through the coordinator (Gitlab server)
• Types:• Shared: Similar requirements between multiple projects (e.g. Python, GCC, …)• Specific: Special requirements (e.g. HW test workstation with connected boards)• Group: Multiple projects under one group (e.g. Xilinx tools like Vivado (HLS))
• GitLab server and runner are connected via token(s)
• gitlab-runner binary runs as gitlab-runner user on the workstation and can be configured (e.g. number of parallel jobs)
• Docker support for the build environment
09/03/19 Continuous Firmware Integration 15
![Page 16: Continuous Integration (CI) of FPGA Designs for CMS...•Using GitLabCI/CD •Git-based, fully featured platform for SW and FW development •FW requires CLI build system •Support](https://reader034.vdocuments.net/reader034/viewer/2022042219/5ec5664bfd680a5105410455/html5/thumbnails/16.jpg)
Gitlab CI/CD Configuration File: .gitlab-ci.yml• https://docs.gitlab.com/ee/ci/yaml/
• Highly configurable with a lot of options. Examples:
variables:VIVADO_PATH_SH: ‘/opt/Xilinx/Vivado/${VIVADO_VERSION}/settings64.sh'
stages:project-prepareproject-make
.job_template: &template_basetags:
- xilinx-toolsbefore_script:
- source $VIVADO_PATH_SH
project-prepare: # Actual job<<: *template_project_prepare # Calls *template_base (not shown)variables:
VIVADO_VERSION: "2019.1"
09/03/19 Continuous Firmware Integration 16
![Page 17: Continuous Integration (CI) of FPGA Designs for CMS...•Using GitLabCI/CD •Git-based, fully featured platform for SW and FW development •FW requires CLI build system •Support](https://reader034.vdocuments.net/reader034/viewer/2022042219/5ec5664bfd680a5105410455/html5/thumbnails/17.jpg)
Gitlab CI/CD Configuration File: .gitlab-ci.yml• https://docs.gitlab.com/ee/ci/yaml/
• Highly configurable with a lot of options. Examples:
variables:VIVADO_PATH_SH: ‘/opt/Xilinx/Vivado/${VIVADO_VERSION}/settings64.sh'
stages:project-prepareproject-make
.job_template: &template_basetags:
- xilinx-toolsbefore_script:
- source $VIVADO_PATH_SH
project-prepare: # Actual job<<: *template_project_prepare # Calls *template_base (not shown)variables:
VIVADO_VERSION: "2019.1"
09/03/19 Continuous Firmware Integration 17
![Page 18: Continuous Integration (CI) of FPGA Designs for CMS...•Using GitLabCI/CD •Git-based, fully featured platform for SW and FW development •FW requires CLI build system •Support](https://reader034.vdocuments.net/reader034/viewer/2022042219/5ec5664bfd680a5105410455/html5/thumbnails/18.jpg)
Gitlab CI/CD Configuration File: .gitlab-ci.yml• https://docs.gitlab.com/ee/ci/yaml/
• Highly configurable with a lot of options. Examples:
variables:VIVADO_PATH_SH: ‘/opt/Xilinx/Vivado/${VIVADO_VERSION}/settings64.sh'
stages:project-prepareproject-make
.job_template: &template_basetags:
- xilinx-toolsbefore_script:
- source $VIVADO_PATH_SH
project-prepare: # Actual job<<: *template_project_prepare # Calls *template_base (not shown)variables:
VIVADO_VERSION: "2019.1"
09/03/19 Continuous Firmware Integration 18
![Page 19: Continuous Integration (CI) of FPGA Designs for CMS...•Using GitLabCI/CD •Git-based, fully featured platform for SW and FW development •FW requires CLI build system •Support](https://reader034.vdocuments.net/reader034/viewer/2022042219/5ec5664bfd680a5105410455/html5/thumbnails/19.jpg)
Gitlab CI/CD Configuration File: .gitlab-ci.yml• https://docs.gitlab.com/ee/ci/yaml/
• Highly configurable with a lot of options. Examples:
variables:VIVADO_PATH_SH: ‘/opt/Xilinx/Vivado/${VIVADO_VERSION}/settings64.sh'
stages:project-prepareproject-make
.job_template: &template_basetags:
- xilinx-toolsbefore_script:
- source $VIVADO_PATH_SH
project-prepare: # Actual job<<: *template_project_prepare # Calls *template_base (not shown)variables:
VIVADO_VERSION: "2019.1"
09/03/19 Continuous Firmware Integration 19
![Page 20: Continuous Integration (CI) of FPGA Designs for CMS...•Using GitLabCI/CD •Git-based, fully featured platform for SW and FW development •FW requires CLI build system •Support](https://reader034.vdocuments.net/reader034/viewer/2022042219/5ec5664bfd680a5105410455/html5/thumbnails/20.jpg)
Gitlab CI/CD Configuration File: .gitlab-ci.yml• https://docs.gitlab.com/ee/ci/yaml/
• Highly configurable with a lot of options. Examples:
variables:VIVADO_PATH_SH: ‘/opt/Xilinx/Vivado/${VIVADO_VERSION}/settings64.sh'
stages:project-prepareproject-make
.job_template: &template_basetags:
- xilinx-toolsbefore_script:
- source $VIVADO_PATH_SH
project-prepare: # Actual job<<: *template_project_prepare # Calls *template_base (not shown)variables:
VIVADO_VERSION: "2019.1"
09/03/19 Continuous Firmware Integration 20
![Page 21: Continuous Integration (CI) of FPGA Designs for CMS...•Using GitLabCI/CD •Git-based, fully featured platform for SW and FW development •FW requires CLI build system •Support](https://reader034.vdocuments.net/reader034/viewer/2022042219/5ec5664bfd680a5105410455/html5/thumbnails/21.jpg)
CI Development I
Continuous Firmware Integration
FRW Code:.vhd, .v,
.xci, .bd, …
Commit:git add –u;git ci –m;git push
Related code:
ruckus, HLS, …
Simulation:make xsim
Unit test using TextIO: Implemented in CI job call
Build:make syn;make bit
Continuous Deployment (CD):
Review, stage, deploy
Use Vivado GUI between the steps: make gui
09/03/19 21
lib Code:.vhd, .v,
.xci, .bd, …
Commit:git add –u;git ci –m;git push
Stimuli: input.txt
Simulate the
testbench Result: output.txt
Reference: golden.txt
Com
pare
chec
ksum
s
HW test:VIO (trigger)
and ILA
![Page 22: Continuous Integration (CI) of FPGA Designs for CMS...•Using GitLabCI/CD •Git-based, fully featured platform for SW and FW development •FW requires CLI build system •Support](https://reader034.vdocuments.net/reader034/viewer/2022042219/5ec5664bfd680a5105410455/html5/thumbnails/22.jpg)
CI Development II:https://gitlab.cern.ch/GTT/fpga/-/jobs
• Template based CI stages/jobs
• Required SW is tagged
09/03/19 Continuous Firmware Integration 22
![Page 23: Continuous Integration (CI) of FPGA Designs for CMS...•Using GitLabCI/CD •Git-based, fully featured platform for SW and FW development •FW requires CLI build system •Support](https://reader034.vdocuments.net/reader034/viewer/2022042219/5ec5664bfd680a5105410455/html5/thumbnails/23.jpg)
CI Development III:Build Including Parallel Simulations
09/03/19 Continuous Firmware Integration 23
• https://gitlab.cern.ch/GTT/fpga/blob/master/.gitlab-ci.yml(template and instantiation)
• https://gitlab.cern.ch/GTT/fpga/pipelines/1056751 (visualization)
![Page 24: Continuous Integration (CI) of FPGA Designs for CMS...•Using GitLabCI/CD •Git-based, fully featured platform for SW and FW development •FW requires CLI build system •Support](https://reader034.vdocuments.net/reader034/viewer/2022042219/5ec5664bfd680a5105410455/html5/thumbnails/24.jpg)
Outline
• Problem Description and Motivation
• Continuous Integration (CI) Overview
• Command Line Interface (CLI) FW Build Systems
• Continuous Integration / Continuous Deployment (CI/CD) Tools
• Demonstration
• Conclusion
Continuous Firmware Integration09/03/19 24
![Page 25: Continuous Integration (CI) of FPGA Designs for CMS...•Using GitLabCI/CD •Git-based, fully featured platform for SW and FW development •FW requires CLI build system •Support](https://reader034.vdocuments.net/reader034/viewer/2022042219/5ec5664bfd680a5105410455/html5/thumbnails/25.jpg)
CI Demonstration
• https://gitlab.cern.ch/GTT/fpga/pipelines
• Access the GitLab repository
• Rerun implementation
• Watch the CI doing its job
• Explain different CI features• Pipeline• Jobs: Log and artifacts• Charts
09/03/19 Continuous Firmware Integration 25
![Page 26: Continuous Integration (CI) of FPGA Designs for CMS...•Using GitLabCI/CD •Git-based, fully featured platform for SW and FW development •FW requires CLI build system •Support](https://reader034.vdocuments.net/reader034/viewer/2022042219/5ec5664bfd680a5105410455/html5/thumbnails/26.jpg)
CI Conclusion
• Faster and reproducible results
• Easier troubleshooting (an error can be traced to the exact commit)
• Manual verification of FW is minimized• Using GitLab CI/CD
• Git-based, fully featured platform for SW and FW development• FW requires CLI build system• Support for the entire FPGA design process
• Plan, create, verify, package, release, configure, and monitor
• What happens if you do not use CI?• If you forget to test the entire chain, your algorithm embedded in the
infrastructure may cause errors (timing, not tested function, …)• Errors may be hidden and pile up until the system breaks down
Continuous Firmware Integration09/03/19 26
![Page 27: Continuous Integration (CI) of FPGA Designs for CMS...•Using GitLabCI/CD •Git-based, fully featured platform for SW and FW development •FW requires CLI build system •Support](https://reader034.vdocuments.net/reader034/viewer/2022042219/5ec5664bfd680a5105410455/html5/thumbnails/27.jpg)
CI Conclusion
• Faster and reproducible results
• Easier troubleshooting (an error can be traced to the exact commit)
• Manual verification of FW is minimized• Using GitLab CI/CD
• Git-based, fully featured platform for SW and FW development• FW requires CLI build system• Support for the entire FPGA design process
• Plan, create, verify, package, release, configure, and monitor
• What happens if you do not use CI?• If you forget to test the entire chain, your algorithm embedded in the
infrastructure may cause errors (timing, not tested function, …)• Errors may be hidden and pile up until the system breaks down• DON’T LET THIS HAPPEN TO YOU! à USE CI/CD!
Continuous Firmware Integration09/03/19 27
![Page 28: Continuous Integration (CI) of FPGA Designs for CMS...•Using GitLabCI/CD •Git-based, fully featured platform for SW and FW development •FW requires CLI build system •Support](https://reader034.vdocuments.net/reader034/viewer/2022042219/5ec5664bfd680a5105410455/html5/thumbnails/28.jpg)
Back Up
09/03/19 Continuous Firmware Integration 28
![Page 29: Continuous Integration (CI) of FPGA Designs for CMS...•Using GitLabCI/CD •Git-based, fully featured platform for SW and FW development •FW requires CLI build system •Support](https://reader034.vdocuments.net/reader034/viewer/2022042219/5ec5664bfd680a5105410455/html5/thumbnails/29.jpg)
https://about.gitlab.com/devops-tools/(From the Gitlab perspective)
09/03/19 Continuous Firmware Integration 29
![Page 30: Continuous Integration (CI) of FPGA Designs for CMS...•Using GitLabCI/CD •Git-based, fully featured platform for SW and FW development •FW requires CLI build system •Support](https://reader034.vdocuments.net/reader034/viewer/2022042219/5ec5664bfd680a5105410455/html5/thumbnails/30.jpg)
TMUX=1
Rx1, Rx2, …, Rx18 à TMUX=1Rx19, Rx20, …, Rx36 à TMUX=7Rx37, Rx38, …, Rx54 à TMUX=13
Track Stand-Alone Object (Global Track Trigger) in CMS: Overview
Continuous Firmware Integration
TSA#1 VU9P FPGA
GTYTF@25 Gbps
3009/03/19
Track Interface Rx
18x64 b
FIFOs+ optional conv.
96 b Converter
Track FIFOs2 Events
FW Modules for TMUX=13
FW Modules for TMUX=7
vld / rdy
vld / rdy
AlgorithmMET (HLS)
@ fclk,MET
Vertex (HLS)@ fclk,Vertex
Jet (Verilog)@ fclk,Jet
18x96 b
18x96 bObject Interface Tx
Converter with FIFOsvld / rdy
vld / rdy
vld / rdy
9x3 TSASer; 9x3 Muon;4xVtx PF;
2xJF&MET GT; 1 DAQ
Tack Sel.
![Page 31: Continuous Integration (CI) of FPGA Designs for CMS...•Using GitLabCI/CD •Git-based, fully featured platform for SW and FW development •FW requires CLI build system •Support](https://reader034.vdocuments.net/reader034/viewer/2022042219/5ec5664bfd680a5105410455/html5/thumbnails/31.jpg)
Doxygen: Automated Generation of Code (e.g. HDL) Documentation
09/03/19 Continuous Firmware Integration 31