![Page 1: BIG DATA ANALYTICS LARGE HADRON COLLIDER · BIG DATA ANALYTICS LARGE HADRON COLLIDER Manuel Martín Márquez, Senior Project Leader CERN –IT Department. 3 CERN EUROPEAN ORGANIZATION](https://reader034.vdocuments.net/reader034/viewer/2022042321/5f0b60757e708231d4303793/html5/thumbnails/1.jpg)
![Page 2: BIG DATA ANALYTICS LARGE HADRON COLLIDER · BIG DATA ANALYTICS LARGE HADRON COLLIDER Manuel Martín Márquez, Senior Project Leader CERN –IT Department. 3 CERN EUROPEAN ORGANIZATION](https://reader034.vdocuments.net/reader034/viewer/2022042321/5f0b60757e708231d4303793/html5/thumbnails/2.jpg)
BIG DATA ANALYTICS
LARGE HADRON COLLIDER
Manuel Martín Márquez, Senior Project Leader
CERN – IT Department
![Page 3: BIG DATA ANALYTICS LARGE HADRON COLLIDER · BIG DATA ANALYTICS LARGE HADRON COLLIDER Manuel Martín Márquez, Senior Project Leader CERN –IT Department. 3 CERN EUROPEAN ORGANIZATION](https://reader034.vdocuments.net/reader034/viewer/2022042321/5f0b60757e708231d4303793/html5/thumbnails/3.jpg)
3
CERN EUROPEAN ORGANIZATION FOR NUCLEAR RESEACH
A WORLDWIDE COLLABORATION
![Page 4: BIG DATA ANALYTICS LARGE HADRON COLLIDER · BIG DATA ANALYTICS LARGE HADRON COLLIDER Manuel Martín Márquez, Senior Project Leader CERN –IT Department. 3 CERN EUROPEAN ORGANIZATION](https://reader034.vdocuments.net/reader034/viewer/2022042321/5f0b60757e708231d4303793/html5/thumbnails/4.jpg)
4
FUNDAMENTAL RESEARCH
WHAT IS THE UNIVERSE MADE OF?
HOW DIT IT START?
![Page 5: BIG DATA ANALYTICS LARGE HADRON COLLIDER · BIG DATA ANALYTICS LARGE HADRON COLLIDER Manuel Martín Márquez, Senior Project Leader CERN –IT Department. 3 CERN EUROPEAN ORGANIZATION](https://reader034.vdocuments.net/reader034/viewer/2022042321/5f0b60757e708231d4303793/html5/thumbnails/5.jpg)
5
FUNDAMENTAL RESEARCH
Why do particle have mass?
![Page 6: BIG DATA ANALYTICS LARGE HADRON COLLIDER · BIG DATA ANALYTICS LARGE HADRON COLLIDER Manuel Martín Márquez, Senior Project Leader CERN –IT Department. 3 CERN EUROPEAN ORGANIZATION](https://reader034.vdocuments.net/reader034/viewer/2022042321/5f0b60757e708231d4303793/html5/thumbnails/6.jpg)
6
FUNDAMENTAL RESEARCH
Why is there no antimatter left in the Universe?
What is 95% of the Universe made of?
![Page 7: BIG DATA ANALYTICS LARGE HADRON COLLIDER · BIG DATA ANALYTICS LARGE HADRON COLLIDER Manuel Martín Márquez, Senior Project Leader CERN –IT Department. 3 CERN EUROPEAN ORGANIZATION](https://reader034.vdocuments.net/reader034/viewer/2022042321/5f0b60757e708231d4303793/html5/thumbnails/7.jpg)
7Manuel Martin MarquezIntel IoT Ignition Lab – Cloud and Big Data
Munich, September 17th
![Page 8: BIG DATA ANALYTICS LARGE HADRON COLLIDER · BIG DATA ANALYTICS LARGE HADRON COLLIDER Manuel Martín Márquez, Senior Project Leader CERN –IT Department. 3 CERN EUROPEAN ORGANIZATION](https://reader034.vdocuments.net/reader034/viewer/2022042321/5f0b60757e708231d4303793/html5/thumbnails/8.jpg)
8
CERN’s PARTICLE ACCELERATORS AND
EXPERIMENTS
![Page 9: BIG DATA ANALYTICS LARGE HADRON COLLIDER · BIG DATA ANALYTICS LARGE HADRON COLLIDER Manuel Martín Márquez, Senior Project Leader CERN –IT Department. 3 CERN EUROPEAN ORGANIZATION](https://reader034.vdocuments.net/reader034/viewer/2022042321/5f0b60757e708231d4303793/html5/thumbnails/9.jpg)
12/6/2019 Document reference 9
CERN Aerial View
World’s largest scientific instrument27km (16.8 miles) circumference, 6000+ superconducting magnets
Emptiest place in the solar system High vacuum inside the magnets
Hottest spot in the galaxy
During Lead ion collisions create temperatures 100 000x hotter than the heart of the sun;
Fastest racetrack on Earth
Protons circulate 11245 times/s (99.9999991% the speed of light)
![Page 10: BIG DATA ANALYTICS LARGE HADRON COLLIDER · BIG DATA ANALYTICS LARGE HADRON COLLIDER Manuel Martín Márquez, Senior Project Leader CERN –IT Department. 3 CERN EUROPEAN ORGANIZATION](https://reader034.vdocuments.net/reader034/viewer/2022042321/5f0b60757e708231d4303793/html5/thumbnails/10.jpg)
12/6/2019 Document reference 10
150 Million of sensorControl and detection sensors
Massive 3D camera
Capturing million of collisions per second
CMS Detector
![Page 11: BIG DATA ANALYTICS LARGE HADRON COLLIDER · BIG DATA ANALYTICS LARGE HADRON COLLIDER Manuel Martín Márquez, Senior Project Leader CERN –IT Department. 3 CERN EUROPEAN ORGANIZATION](https://reader034.vdocuments.net/reader034/viewer/2022042321/5f0b60757e708231d4303793/html5/thumbnails/11.jpg)
11
FUTURE CHALLENGES
HIGH-LUMINOSITY LHC
FUTURE CIRCULAR COLLIDER
![Page 12: BIG DATA ANALYTICS LARGE HADRON COLLIDER · BIG DATA ANALYTICS LARGE HADRON COLLIDER Manuel Martín Márquez, Senior Project Leader CERN –IT Department. 3 CERN EUROPEAN ORGANIZATION](https://reader034.vdocuments.net/reader034/viewer/2022042321/5f0b60757e708231d4303793/html5/thumbnails/12.jpg)
12
![Page 13: BIG DATA ANALYTICS LARGE HADRON COLLIDER · BIG DATA ANALYTICS LARGE HADRON COLLIDER Manuel Martín Márquez, Senior Project Leader CERN –IT Department. 3 CERN EUROPEAN ORGANIZATION](https://reader034.vdocuments.net/reader034/viewer/2022042321/5f0b60757e708231d4303793/html5/thumbnails/13.jpg)
Post-LHC accelerator projects (80-100 km)
![Page 14: BIG DATA ANALYTICS LARGE HADRON COLLIDER · BIG DATA ANALYTICS LARGE HADRON COLLIDER Manuel Martín Márquez, Senior Project Leader CERN –IT Department. 3 CERN EUROPEAN ORGANIZATION](https://reader034.vdocuments.net/reader034/viewer/2022042321/5f0b60757e708231d4303793/html5/thumbnails/14.jpg)
14
CERN’s BIG DATA AND MACHINE LEARNING
HIGH ENERGY PHYSICS
![Page 15: BIG DATA ANALYTICS LARGE HADRON COLLIDER · BIG DATA ANALYTICS LARGE HADRON COLLIDER Manuel Martín Márquez, Senior Project Leader CERN –IT Department. 3 CERN EUROPEAN ORGANIZATION](https://reader034.vdocuments.net/reader034/viewer/2022042321/5f0b60757e708231d4303793/html5/thumbnails/15.jpg)
15
![Page 16: BIG DATA ANALYTICS LARGE HADRON COLLIDER · BIG DATA ANALYTICS LARGE HADRON COLLIDER Manuel Martín Márquez, Senior Project Leader CERN –IT Department. 3 CERN EUROPEAN ORGANIZATION](https://reader034.vdocuments.net/reader034/viewer/2022042321/5f0b60757e708231d4303793/html5/thumbnails/16.jpg)
16
![Page 17: BIG DATA ANALYTICS LARGE HADRON COLLIDER · BIG DATA ANALYTICS LARGE HADRON COLLIDER Manuel Martín Márquez, Senior Project Leader CERN –IT Department. 3 CERN EUROPEAN ORGANIZATION](https://reader034.vdocuments.net/reader034/viewer/2022042321/5f0b60757e708231d4303793/html5/thumbnails/17.jpg)
17
![Page 18: BIG DATA ANALYTICS LARGE HADRON COLLIDER · BIG DATA ANALYTICS LARGE HADRON COLLIDER Manuel Martín Márquez, Senior Project Leader CERN –IT Department. 3 CERN EUROPEAN ORGANIZATION](https://reader034.vdocuments.net/reader034/viewer/2022042321/5f0b60757e708231d4303793/html5/thumbnails/18.jpg)
18
TRIGGER SYSTEM – FILTERING EVENTS & DATA
The trigger system selects approximately 1000 of the 1.7
billion collisions that occur each second in the centre of the
ATLAS detector.
![Page 19: BIG DATA ANALYTICS LARGE HADRON COLLIDER · BIG DATA ANALYTICS LARGE HADRON COLLIDER Manuel Martín Márquez, Senior Project Leader CERN –IT Department. 3 CERN EUROPEAN ORGANIZATION](https://reader034.vdocuments.net/reader034/viewer/2022042321/5f0b60757e708231d4303793/html5/thumbnails/19.jpg)
19
TRIGGER SYSTEM – FILTERING EVENTS & DATA
Improve efficiency, flexibility and quality filtering Remove the costly and rigid hardware based model
Reduce false positives rates
From rule based systems to Deep Learning Classifiers
![Page 20: BIG DATA ANALYTICS LARGE HADRON COLLIDER · BIG DATA ANALYTICS LARGE HADRON COLLIDER Manuel Martín Márquez, Senior Project Leader CERN –IT Department. 3 CERN EUROPEAN ORGANIZATION](https://reader034.vdocuments.net/reader034/viewer/2022042321/5f0b60757e708231d4303793/html5/thumbnails/20.jpg)
20
TRIGGER SYSTEM – ML PIPELINE
Data
IngestionFeature
PreparationModel
Development Training
Complex datasets
801x19 matrix
Files about 4TB
Data format preparation
19 Original Features
14 Derived from Domain
Knowledge (HLF)
Feed-Forward DNN
Recursive DNN - GRUs
Combined Models
Hyper-Parameter tuning
Scikit-learn-Keras-Spark (parallel)
Distributed Training
![Page 21: BIG DATA ANALYTICS LARGE HADRON COLLIDER · BIG DATA ANALYTICS LARGE HADRON COLLIDER Manuel Martín Márquez, Senior Project Leader CERN –IT Department. 3 CERN EUROPEAN ORGANIZATION](https://reader034.vdocuments.net/reader034/viewer/2022042321/5f0b60757e708231d4303793/html5/thumbnails/21.jpg)
21
TRIGGER SYSTEM – SCORING
Strict latency constraints Target level 1 trigger
Larger networks with longer latency. neutrino,
astronomical experiments, industrial
applications etc.
FPGAs Provide huge flexibility and allow us to cope
with response time required
Performance depends on how well you take
advantage of it
![Page 22: BIG DATA ANALYTICS LARGE HADRON COLLIDER · BIG DATA ANALYTICS LARGE HADRON COLLIDER Manuel Martín Márquez, Senior Project Leader CERN –IT Department. 3 CERN EUROPEAN ORGANIZATION](https://reader034.vdocuments.net/reader034/viewer/2022042321/5f0b60757e708231d4303793/html5/thumbnails/22.jpg)
22
QUANTUM COMPUTING – ANOTHER WEAPON?
Can Quantum Computing and Q-ML help Quantum Nearest Neighbors Clustering, PCA and SVM
But still hard to Get access to emulators and simulators
Get access to real devices, benchmark, compare results
Engineering aspects of QC installation, like cryogenics and material science
Use Cases: Track reconstruction in dense environments
Reconstruct neutrino interactions
Optimize Grid workflow
![Page 23: BIG DATA ANALYTICS LARGE HADRON COLLIDER · BIG DATA ANALYTICS LARGE HADRON COLLIDER Manuel Martín Márquez, Senior Project Leader CERN –IT Department. 3 CERN EUROPEAN ORGANIZATION](https://reader034.vdocuments.net/reader034/viewer/2022042321/5f0b60757e708231d4303793/html5/thumbnails/23.jpg)
23
CERN’s BIG DATA AND MACHINE LEARNING
CERN CONTROL SYSTEMS AND IOT
![Page 24: BIG DATA ANALYTICS LARGE HADRON COLLIDER · BIG DATA ANALYTICS LARGE HADRON COLLIDER Manuel Martín Márquez, Senior Project Leader CERN –IT Department. 3 CERN EUROPEAN ORGANIZATION](https://reader034.vdocuments.net/reader034/viewer/2022042321/5f0b60757e708231d4303793/html5/thumbnails/24.jpg)
24
CERN ACCELERATOR LOGGING SERVICE
+2M signals produce more than 2.5TB data per day.
From scalars to arrays of up-to 4 million elements.
Data diverse in nature: Accelerator running modes,
Equipment statuses,
Magnet currents,
Cryogenics temperatures,
Particle beam positions
![Page 25: BIG DATA ANALYTICS LARGE HADRON COLLIDER · BIG DATA ANALYTICS LARGE HADRON COLLIDER Manuel Martín Márquez, Senior Project Leader CERN –IT Department. 3 CERN EUROPEAN ORGANIZATION](https://reader034.vdocuments.net/reader034/viewer/2022042321/5f0b60757e708231d4303793/html5/thumbnails/25.jpg)
25
CERN ACCELERATOR LOGGING SERVICE
Control I-IoT data at CERN is disperse into several data silos
Current system optimized for real-time serving but not for data
exploration: Find hidden correlations
Anomalies detection
Post-mortem analysis
Root cause analysis (RCA)
Intelligent Alarm systems
Etc
![Page 26: BIG DATA ANALYTICS LARGE HADRON COLLIDER · BIG DATA ANALYTICS LARGE HADRON COLLIDER Manuel Martín Márquez, Senior Project Leader CERN –IT Department. 3 CERN EUROPEAN ORGANIZATION](https://reader034.vdocuments.net/reader034/viewer/2022042321/5f0b60757e708231d4303793/html5/thumbnails/26.jpg)
26
ENHANCE EXPLORATION - AUTONOMOUS TECH
Oracle Autonomous StrategyAutomated creation of required resources,
Administration, Patches,
Backups,
Memory handling
Flexibility to provisioning and scaling Easy Solution prototyping
Move fast from PoC to Production states
Cost Effective Solutions Hybrid systems integrating DB and External Object Storage
![Page 27: BIG DATA ANALYTICS LARGE HADRON COLLIDER · BIG DATA ANALYTICS LARGE HADRON COLLIDER Manuel Martín Márquez, Senior Project Leader CERN –IT Department. 3 CERN EUROPEAN ORGANIZATION](https://reader034.vdocuments.net/reader034/viewer/2022042321/5f0b60757e708231d4303793/html5/thumbnails/27.jpg)
27
DATA INTERFACE – NOTEBOOKS
The human factor
SWAN
Jupyter notebooks
Integrated with CERN
GPUs – Cloud
Oracle collaboration
![Page 28: BIG DATA ANALYTICS LARGE HADRON COLLIDER · BIG DATA ANALYTICS LARGE HADRON COLLIDER Manuel Martín Márquez, Senior Project Leader CERN –IT Department. 3 CERN EUROPEAN ORGANIZATION](https://reader034.vdocuments.net/reader034/viewer/2022042321/5f0b60757e708231d4303793/html5/thumbnails/28.jpg)