belle ii conditions database status · database status carlos fernando gamboa ([email protected]),...
TRANSCRIPT
![Page 1: Belle II Conditions Database status · Database status Carlos Fernando Gamboa (cgamboa@bnl.gov), Ruslan ... (database component) in numbers 950K rows 340MB 520 connections Day Number](https://reader034.vdocuments.net/reader034/viewer/2022042021/5e78e45fa222190861282b61/html5/thumbnails/1.jpg)
Belle II Conditions Database statusCarlos Fernando Gamboa ([email protected]), Ruslan Mashinistov, Benedikt Hegner. 32nd Belle II General Meeting, February 4th – 8th 2019
![Page 2: Belle II Conditions Database status · Database status Carlos Fernando Gamboa (cgamboa@bnl.gov), Ruslan ... (database component) in numbers 950K rows 340MB 520 connections Day Number](https://reader034.vdocuments.net/reader034/viewer/2022042021/5e78e45fa222190861282b61/html5/thumbnails/2.jpg)
Overview
• Status of Conditions Database (CDB) server. • Development activities and future work.
2
![Page 3: Belle II Conditions Database status · Database status Carlos Fernando Gamboa (cgamboa@bnl.gov), Ruslan ... (database component) in numbers 950K rows 340MB 520 connections Day Number](https://reader034.vdocuments.net/reader034/viewer/2022042021/5e78e45fa222190861282b61/html5/thumbnails/3.jpg)
Status of CDB server
3
Status of CDB server
3
![Page 4: Belle II Conditions Database status · Database status Carlos Fernando Gamboa (cgamboa@bnl.gov), Ruslan ... (database component) in numbers 950K rows 340MB 520 connections Day Number](https://reader034.vdocuments.net/reader034/viewer/2022042021/5e78e45fa222190861282b61/html5/thumbnails/4.jpg)
Belle II CDB Remote service accessibility (WAN/LAN)
June 19 2018 30th Belle II General Meeting 4
ConditionsCDB database
(Tier 1,US BNL)
WAN
LANTIER 1
BNLWorker Nodes
KEKDAQ
Tier 1sGE,CA,IT
Interactive Users
Interactive Users
KEKcc
Calibration cycle
![Page 5: Belle II Conditions Database status · Database status Carlos Fernando Gamboa (cgamboa@bnl.gov), Ruslan ... (database component) in numbers 950K rows 340MB 520 connections Day Number](https://reader034.vdocuments.net/reader034/viewer/2022042021/5e78e45fa222190861282b61/html5/thumbnails/5.jpg)
Belle II CDB Metadata servicereview
Architecture deployed using Kubernetes / Docker framework and controlled via puppetkubelet-1.9docker-1.12b2s(0.6.9.8): Java application on Payara micro 4.1.2
Database replicated for reliability purposesPostgres 9.6 hot standby replication
Database service
Read OnlyRead/Write
Node 1 Node 2
Belle2db service
5
![Page 6: Belle II Conditions Database status · Database status Carlos Fernando Gamboa (cgamboa@bnl.gov), Ruslan ... (database component) in numbers 950K rows 340MB 520 connections Day Number](https://reader034.vdocuments.net/reader034/viewer/2022042021/5e78e45fa222190861282b61/html5/thumbnails/6.jpg)
Production activities
Software Infrastructure• Upgraded Squid’s Docker image with Squid 3.5 version and package distribution
• Legacy PNNL’s container image uses an OSG non supported repository.• Deprecated redundant CDB web User Interface code.
CDB code and configuration updates• b2s: Enhanced protection for a Global Tag to prevent a change to its status once it is published.
b2s:0.6.9.6
• b2s:0.6.9.8 fix baseUrl dependency on HTTP/HTTPS and literal default port assignation.
• UI: Swagger interface using secure HTTP by default.• Password protected, contact me or DB coordinators to request access.
Authentication/Authorization for CDB write requests set in place.Isolation between delete and write/update requests.
1. cdbreader access to the swagger interface.2. commonDBuser have access to POST and PUT HTTP.3. cdbcoordinator restricted to HTTP DELETE method.
Updates applied on production system
6
![Page 7: Belle II Conditions Database status · Database status Carlos Fernando Gamboa (cgamboa@bnl.gov), Ruslan ... (database component) in numbers 950K rows 340MB 520 connections Day Number](https://reader034.vdocuments.net/reader034/viewer/2022042021/5e78e45fa222190861282b61/html5/thumbnails/7.jpg)
7
CDB metadata (database component)in numbers
950K rows
340MB
520 connections
Day
Number of table/database rows read 1/1/19 to 1/1/27
Usage driven by Read access
Day
103.4k READs
![Page 8: Belle II Conditions Database status · Database status Carlos Fernando Gamboa (cgamboa@bnl.gov), Ruslan ... (database component) in numbers 950K rows 340MB 520 connections Day Number](https://reader034.vdocuments.net/reader034/viewer/2022042021/5e78e45fa222190861282b61/html5/thumbnails/8.jpg)
Performance CDB service (belle2db services) The year so far CDB
CDB service resource usability within the capacity of resources provisioned.
8
Network CDB activity
CPU database activity
Transmitted Received
![Page 9: Belle II Conditions Database status · Database status Carlos Fernando Gamboa (cgamboa@bnl.gov), Ruslan ... (database component) in numbers 950K rows 340MB 520 connections Day Number](https://reader034.vdocuments.net/reader034/viewer/2022042021/5e78e45fa222190861282b61/html5/thumbnails/9.jpg)
9
Performance CDB service (belle2db services)The year so far CDBOverall service response in terms of HTTP response codes
Cumulative accounting of HTTP response HTTP response
Metadatabelle2db
HTTP response 2xx
HTTP response 4xxClient side
1.7M of HTTP 2XX succeeded
No significant HTTP 5xx server side errors
Need to identify/understand along with Belle2 database group and other experts root cause of client related errors.
Day
Day
![Page 10: Belle II Conditions Database status · Database status Carlos Fernando Gamboa (cgamboa@bnl.gov), Ruslan ... (database component) in numbers 950K rows 340MB 520 connections Day Number](https://reader034.vdocuments.net/reader034/viewer/2022042021/5e78e45fa222190861282b61/html5/thumbnails/10.jpg)
10
Performance CDB service (belle2db-file services)The year so far CDB
Cumulative accounting of HTTP response
Overall service response in terms of HTTP response codes
HTTP response 4xxClient side Errors
HTTP response 2xxsuccessful response 4M/requests
Day
Day
5M of HTTP 2XX succeeded
No significant HTTP 5xx server side errors
![Page 11: Belle II Conditions Database status · Database status Carlos Fernando Gamboa (cgamboa@bnl.gov), Ruslan ... (database component) in numbers 950K rows 340MB 520 connections Day Number](https://reader034.vdocuments.net/reader034/viewer/2022042021/5e78e45fa222190861282b61/html5/thumbnails/11.jpg)
Development activities and future work.
11
![Page 12: Belle II Conditions Database status · Database status Carlos Fernando Gamboa (cgamboa@bnl.gov), Ruslan ... (database component) in numbers 950K rows 340MB 520 connections Day Number](https://reader034.vdocuments.net/reader034/viewer/2022042021/5e78e45fa222190861282b61/html5/thumbnails/12.jpg)
Authentication filter
Authentication filter
● Extracts JWT from the Request’s header
● Verifies the signature● Parce the JWT’s
Payload● Confront User/Group
against requested API endpoint
API
Request+
JWT
● Developing the AuthenticationFilter
○ Intercept request and do some pre-processing before hit to the API
● JWT signature based on shared secret
● Future plans: Users/Groups managing
Development and Future work
12
Authentication filter
Authentication filter
● Extracts JWT from the Request’s header
● Verifies the signature● Parce the JWT’s
Payload● Confront User/Group
against requested API endpoint
API
Request+
JWT
● Developing the AuthenticationFilter
○ Intercept request and do some pre-processing before hit to the API
● JWT signature based on shared secret
● Future plans: Users/Groups managing
Json Web Token (JWT) technology been reviewed
Currently working in identifying a test scenario that allows full test integration with client and server.
![Page 13: Belle II Conditions Database status · Database status Carlos Fernando Gamboa (cgamboa@bnl.gov), Ruslan ... (database component) in numbers 950K rows 340MB 520 connections Day Number](https://reader034.vdocuments.net/reader034/viewer/2022042021/5e78e45fa222190861282b61/html5/thumbnails/13.jpg)
Development and Future work (cont.)• Secure HTTPs enabled
• To support dynamic protocol (HTTPS and HTTP) for CDB payloads.
• Consideration of database replication technology• Prototype for multi site standby replica of CDB
metadata.
13
![Page 14: Belle II Conditions Database status · Database status Carlos Fernando Gamboa (cgamboa@bnl.gov), Ruslan ... (database component) in numbers 950K rows 340MB 520 connections Day Number](https://reader034.vdocuments.net/reader034/viewer/2022042021/5e78e45fa222190861282b61/html5/thumbnails/14.jpg)
CDB readiness for real data takingRequesting feedback from Belle II Software and Computing communities:Are there any items that are critical to address before final data taking?
What are the items on the critical path?Otherwise,
Are we fully ready?
14
![Page 15: Belle II Conditions Database status · Database status Carlos Fernando Gamboa (cgamboa@bnl.gov), Ruslan ... (database component) in numbers 950K rows 340MB 520 connections Day Number](https://reader034.vdocuments.net/reader034/viewer/2022042021/5e78e45fa222190861282b61/html5/thumbnails/15.jpg)
Conclusion• CDB production service have been consistently working
stable.• Need to coordinate along with Belle II database group
and community to reduce client side related errors.• Robustness in terms of security has been increased
• Working along with the Belle II database group to evolve a granular secure Authentication/Authorization infrastructure.
• Replication of CDB metadata to external site being considered.
15
![Page 16: Belle II Conditions Database status · Database status Carlos Fernando Gamboa (cgamboa@bnl.gov), Ruslan ... (database component) in numbers 950K rows 340MB 520 connections Day Number](https://reader034.vdocuments.net/reader034/viewer/2022042021/5e78e45fa222190861282b61/html5/thumbnails/16.jpg)
Backup slides
16
![Page 17: Belle II Conditions Database status · Database status Carlos Fernando Gamboa (cgamboa@bnl.gov), Ruslan ... (database component) in numbers 950K rows 340MB 520 connections Day Number](https://reader034.vdocuments.net/reader034/viewer/2022042021/5e78e45fa222190861282b61/html5/thumbnails/17.jpg)
17
Performance CDB service (belle2db-file services)The year so far CDB
CDB files/payloadsbelle2db-files Network CDB activity
CPU database activity