MOTIVATION FULLY AUTOMATED ECU TEST IN CONTINUOUS INTEGRATION
Fully automated ECU test in Continuous Integration
Chassis Systems Control | CC-AS/EST2 | 29/11/2016 © Robert Bosch GmbH 2016. All rights reserved, also regarding any disposal, exploitation, reproduction, editing, distribution, as well as in the event of applications for industrial property rights.
2
Agenda
Overview Motivation Test focus Setup of test automation Summary and outlook
Fully automated ECU test in Continuous Integration
Chassis Systems Control | CC-AS/EST2 | 29/11/2016 © Robert Bosch GmbH 2016. All rights reserved, also regarding any disposal, exploitation, reproduction, editing, distribution, as well as in the event of applications for industrial property rights.
3
Overview
SCM
Jenkins
Test Environment Test Framework
Trigger SW-Build, Flashing, Start Tests
Control SCM, Test Environment, Test Campaigns
Reporting Test-Results to Jenkins, Artefact Repository
SW Build x SW Test SW-Config Testing-Config
MOTIVATION
Fully automated ECU test in Continuous Integration
Chassis Systems Control | CC-AS/EST2 | 29/11/2016 © Robert Bosch GmbH 2016. All rights reserved, also regarding any disposal, exploitation, reproduction, editing, distribution, as well as in the event of applications for industrial property rights.
5
Motivation Reduction loading and cost of complex HIL systems Verification of big amount of variants Enable feedback of quality with high level of automation
Increased flexibility small size and short setup time: ~5min multi project sharing
Enabler for enhanced test strategy early integration test quick regression tests on real HW
Fully automated ECU test in Continuous Integration
Chassis Systems Control | CC-AS/EST2 | 29/11/2016 © Robert Bosch GmbH 2016. All rights reserved, also regarding any disposal, exploitation, reproduction, editing, distribution, as well as in the event of applications for industrial property rights.
6
Challenge: Verification of huge amount of variants
Continuous Integration Server Test Server
…
…
? Project Build A
1-n variants SW-Build_1
?
?
?
SW-Config A
Software Configuration Management
SW-Config B
SW-Config n Project Build B
1-n variants SW-Build_1
Project Build n 1-n variants SW-Build_1
Testing
Testing
Testing
TEST FOCUS
Focus of testing Fully automated ECU test in Continuous Integration
Chassis Systems Control | CC-AS/EST2 | 29/11/2016 © Robert Bosch GmbH 2016. All rights reserved, also regarding any disposal, exploitation, reproduction, editing, distribution, as well as in the event of applications for industrial property rights.
8
Open loop testing SW/HW-Integration test
Black and white box test Basic tests of complete SW Detail tests of infrastructure SW
ECU (Device Under Test) Application SW
Hardware Plattform SW
OEM 3rd
Party SW
SWC2
RTE
SWC1 SWC…
XCP
Test-PC
Aurora/JTAG
USB
Bus-System: FlexRay, CAN, LIN,
Rx Tx
Tx Rx
VN8970
Hardware Abstraction Layer COM
Stack
Ethernet
VX1132
VN8970 VX1132 ECU = 1 Shelf
SW = Software SWC = Software Component
SETUP OF TEST AUTOMATION
Fully automated ECU test in Continuous Integration
Chassis Systems Control | CC-AS/EST2 | 29/11/2016 © Robert Bosch GmbH 2016. All rights reserved, also regarding any disposal, exploitation, reproduction, editing, distribution, as well as in the event of applications for industrial property rights.
10
Setup of test automation
Network Switch
Power Supply
Master Test-PC
Shelf 1
VN8970 VX1132 ECU
Shelf n:
CT-Manager + Dispatcher
Jenkins Project: A
Quick Test
CT Test Framework
Master-PC
Feature Test
Jenkins Project: A
SW Build x
SW Build y
Project: n Project: B Project: B
Test System SCM
Project: A Project: B
VN8970 VX1132 ECU
…
…
Project: n
Project: n
…
Fully automated ECU test in Continuous Integration
Chassis Systems Control | CC-AS/EST2 | 29/11/2016 © Robert Bosch GmbH 2016. All rights reserved, also regarding any disposal, exploitation, reproduction, editing, distribution, as well as in the event of applications for industrial property rights.
11
Workflow of test automation
Test System
CT INI CANoe Test … INI Databases SW Dev Stream: Project::A
Build Jenkins
SW Build X
VN8970
VX1132
Stand alone image
ECU
SCM
Master-PC
HEX/A2L
VN Image
1. Control devices
2. Flashing ECU
3. Download Image
4. Run tests
5. upload results
VN Image With test lists and RBS
Test-Env: Project::A
Fully autonomic test automation
CT-Manager + Dispatcher
CANoe full
Real time generation of config
CT Test Framework
Databases
Param-Files
Test-Env
Reports Reports
SW Build Y
HEX/A2L
Test Jenkins
Quick Test
Feature Test
Reports
Reports
SUMMARY AND OUTLOOK
Fully automated ECU test in Continuous Integration
Chassis Systems Control | CC-AS/EST2 | 29/11/2016 © Robert Bosch GmbH 2016. All rights reserved, also regarding any disposal, exploitation, reproduction, editing, distribution, as well as in the event of applications for industrial property rights.
13
Test campaigns for Continuous Testing Level 1: Testset_1 ~ 5min Test of flashing Basic network communication ECU load and task/event verification Basic diagnostic tests
Level 2: detail tests Tx-Test (Frame, PDU, CRC, ALIVE, Cycle Time) Timing tests, runtime dependencies Robustness and stress test Network management test (start up / shut down) Undervoltage test Physical Layer test …
Fully automated ECU test in Continuous Integration
Chassis Systems Control | CC-AS/EST2 | 29/11/2016 © Robert Bosch GmbH 2016. All rights reserved, also regarding any disposal, exploitation, reproduction, editing, distribution, as well as in the event of applications for industrial property rights.
14
Outlook 1
CI
ALM RTC-SCM
Test System
X y
Jenkins Project::A
Dev Stream: Project::A
Jenkins Project::A
Shelf 2
Shelf n
Shelf 1
CB CT
Test-Env: Project::A
CT-Manager + Dispatcher
Jenkins Project::A
Deploy
Deploy
DOORS: Project::A QM: Project::A
CD
Doors
QM
Artifact Rep.
HEX/A2L XML VN Image Manifest XUnit HTML Reports Artefact Repository
SW Build X
SW Build Y
X y
Quick Test
Feature Test
...
QM: Project::A
CT Test Framework
THANK YOU
Author: Markus Hauk Robert Bosch GmbH Robert-Bosch-Allee 1 74232 Abstatt