defcon: high-performance event processing with ......jvm1 isolation with separate jvms •comparison...
TRANSCRIPT
![Page 1: DEFCon: High-Performance Event Processing with ......JVM1 Isolation with Separate JVMs •Comparison with Marketcetera (Open Source trading platform) – One JVM per investor • Throughput:](https://reader033.vdocuments.net/reader033/viewer/2022050202/5f560cab283d6e731118bd3f/html5/thumbnails/1.jpg)
Peter R. Pietzuch
DEFCon: High-Performance Event Processing with Information Security
Matteo Migliavacca, Ioannis Papagiannis, Peter Pietzuch Imperial College London
David M. Eyers, Jean Bacon Cambridge Computer Laboratory
Brian Shand National Health Service, UK
![Page 2: DEFCon: High-Performance Event Processing with ......JVM1 Isolation with Separate JVMs •Comparison with Marketcetera (Open Source trading platform) – One JVM per investor • Throughput:](https://reader033.vdocuments.net/reader033/viewer/2022050202/5f560cab283d6e731118bd3f/html5/thumbnails/2.jpg)
Event Stream Processing Needs Strong Security
• Event processing – Stream of messages transformed
in near real-time by processing units – Confidential information:
healthcare, social networks, finance
• Problem: incorrect event flows – Lead to security violations – Within application , with the environment – Possible causes: bugs, security attacks, third party
code, malicious code 1
log
access control
1
2
1 2
![Page 3: DEFCon: High-Performance Event Processing with ......JVM1 Isolation with Separate JVMs •Comparison with Marketcetera (Open Source trading platform) – One JVM per investor • Throughput:](https://reader033.vdocuments.net/reader033/viewer/2022050202/5f560cab283d6e731118bd3f/html5/thumbnails/3.jpg)
Financial Processing: Security and Latency
• market data processing and local brokering
• Security is important – Data is valuable: banks fined for exploiting client information
• Performance constraints – Latency, Throughput
• Shared Platform – Processing near stock exchanges costly:
share resources, reduce entry costs for small firms – Local brokering to avoid transaction fees and trade anonymously 2
Bank Investor Monitor
monitor
match
order
Broker deal Stock Ticker
tick
Client Investor Monitor
1
log 2
![Page 4: DEFCon: High-Performance Event Processing with ......JVM1 Isolation with Separate JVMs •Comparison with Marketcetera (Open Source trading platform) – One JVM per investor • Throughput:](https://reader033.vdocuments.net/reader033/viewer/2022050202/5f560cab283d6e731118bd3f/html5/thumbnails/4.jpg)
Security Approach: Information Flow Control
• Protect data end-to-end: Information Flow Control (IFC): – Don’t try to eliminate all bugs and (hard!) – Track and control information flows in application – Previously applied to operating systems and programming languages
Goal: apply IFC to current high-performance event processing systems
3
Stock Ticker
Bank Investor
Client Investor
Monitor
Monitor
Broker
monitor
match
tick order
deal 1
log 2
2 1
![Page 5: DEFCon: High-Performance Event Processing with ......JVM1 Isolation with Separate JVMs •Comparison with Marketcetera (Open Source trading platform) – One JVM per investor • Throughput:](https://reader033.vdocuments.net/reader033/viewer/2022050202/5f560cab283d6e731118bd3f/html5/thumbnails/5.jpg)
Contributions and Overview
• Decentralized Event Flow Control (DEFC) model – IFC applied to event processing
• DEFCon high-performance implementation – Safe and efficient event flows in Java
• Practical isolation methodology – Secure production-level language runtimes with low effort (OpenJDK 6)
• Evaluation – Throughput and latency overhead
4
![Page 6: DEFCon: High-Performance Event Processing with ......JVM1 Isolation with Separate JVMs •Comparison with Marketcetera (Open Source trading platform) – One JVM per investor • Throughput:](https://reader033.vdocuments.net/reader033/viewer/2022050202/5f560cab283d6e731118bd3f/html5/thumbnails/6.jpg)
DEFCon
Event Processing in DEFC
5
Bank Investor Client Monitor Client Investor 77
name data S (confidentiality)
… … {client77}
… … {client77}
S:{client77} S:{}
Event
parts
name data
command monitor
stock IBM
name data S (confidentiality) command monitor { }
stock IBM {client77}
1 ?
2
log
S:{client77}
unit can output part iff
€
S(unit)⊆ S(part)
unit can input part iff
€
S(part)⊆ S(unit)
![Page 7: DEFCon: High-Performance Event Processing with ......JVM1 Isolation with Separate JVMs •Comparison with Marketcetera (Open Source trading platform) – One JVM per investor • Throughput:](https://reader033.vdocuments.net/reader033/viewer/2022050202/5f560cab283d6e731118bd3f/html5/thumbnails/7.jpg)
DEFC Privileges
Clearance privilege: receiving confidential information – Allows units to add tag to its label
Declassification privilege: making confidential data public : – Allows units to remove tag from its label
6
Bank Investor Client Monitor Client Investor 77
S:{client77} S:{} S:{client77}
can receive confidential information
cannot receive confidential information
client77+
client77+
client77+
client77-
client77+, client77-
cannot make confidential information public
can make confidential information public
![Page 8: DEFCon: High-Performance Event Processing with ......JVM1 Isolation with Separate JVMs •Comparison with Marketcetera (Open Source trading platform) – One JVM per investor • Throughput:](https://reader033.vdocuments.net/reader033/viewer/2022050202/5f560cab283d6e731118bd3f/html5/thumbnails/8.jpg)
An Example of Leaks to Avoid
• Untainted unit tries to read tainted part – First try: return access denied
7
… Client Monitor
name data S (confidentiality) … … {client77}
S:{client77}
Access Denied
S:{}
![Page 9: DEFCon: High-Performance Event Processing with ......JVM1 Isolation with Separate JVMs •Comparison with Marketcetera (Open Source trading platform) – One JVM per investor • Throughput:](https://reader033.vdocuments.net/reader033/viewer/2022050202/5f560cab283d6e731118bd3f/html5/thumbnails/9.jpg)
An Example of Leaks to Avoid
• Untainted unit tries to read tainted part – First try: return access denied
• Leaks name of secret parts
8
… Client Monitor
name data S (confidentiality) … … {client77}
S:{client77}
name data S (confidentiality) FirstLetterIsI … {client77}
Access Denied
FirstLetter = I ?
stock=IBM
Bank Investor S:{}
FirstLetterIsI
S:{}
![Page 10: DEFCon: High-Performance Event Processing with ......JVM1 Isolation with Separate JVMs •Comparison with Marketcetera (Open Source trading platform) – One JVM per investor • Throughput:](https://reader033.vdocuments.net/reader033/viewer/2022050202/5f560cab283d6e731118bd3f/html5/thumbnails/10.jpg)
An Example of Leaks to Avoid
• Untainted unit tries to read tainted part – First try: return access denied
• Leaks name of secret parts – Second try: update unit label to part label
9
… Client Monitor
name data S (confidentiality) … … {client77}
S:{client77}
name data S (confidentiality) FirstLetterIsI … {client77}
Access Denied
FirstLetter = I ?
stock=IBM
Bank Investor S:{}
FirstLetterIsI
Ok, label change
S:{} S:{client77}
![Page 11: DEFCon: High-Performance Event Processing with ......JVM1 Isolation with Separate JVMs •Comparison with Marketcetera (Open Source trading platform) – One JVM per investor • Throughput:](https://reader033.vdocuments.net/reader033/viewer/2022050202/5f560cab283d6e731118bd3f/html5/thumbnails/11.jpg)
An Example of Leaks to Avoid
• Untainted unit tries to read tainted part – First try: return access denied
• Leaks name of secret parts – Second try: update unit label to part label
• Secret inferred by absence of communication
10
… Client Monitor
name data S (confidentiality) … … {client77}
S:{client77}
name data S (confidentiality) FirstLetterIsI … {client77}
Access Denied
FirstLetter = I ?
stock=IBM
Bank Investor S:{}
FirstLetterIsI
Ok, label change
FirstLetter = J ?
S:{}
Not Found
FirstLetterIsNotJ S:{} S:{client77}
![Page 12: DEFCon: High-Performance Event Processing with ......JVM1 Isolation with Separate JVMs •Comparison with Marketcetera (Open Source trading platform) – One JVM per investor • Throughput:](https://reader033.vdocuments.net/reader033/viewer/2022050202/5f560cab283d6e731118bd3f/html5/thumbnails/12.jpg)
An Example of Leaks to Avoid
• Untainted unit tries to read tainted part – First try: return access denied
• Leaks name of secret parts – Second try: update unit label to part label
• Secret inferred by absence of communication
– Solution: avoid implicit label changes, return part not found
• Result: all unit label changes must be explicit – First update label, then read part 11
… Client Monitor
name data S (confidentiality) … … {client77}
S:{client77}
name data S (confidentiality) FirstLetterIsI … {client77}
Access Denied
FirstLetter = I ?
stock=IBM
Bank Investor S:{}
FirstLetterIsI
Ok, label change
FirstLetter = J ?
S:{}
Not Found
FirstLetterIsNotJ
Not Found
??? S:{} S:{client77}
![Page 13: DEFCon: High-Performance Event Processing with ......JVM1 Isolation with Separate JVMs •Comparison with Marketcetera (Open Source trading platform) – One JVM per investor • Throughput:](https://reader033.vdocuments.net/reader033/viewer/2022050202/5f560cab283d6e731118bd3f/html5/thumbnails/13.jpg)
Contributions and Overview
• Decentralized Event Flow Control (DEFC) model – IFC applied to event processing
• DEFCon high-performance implementation – Safe and efficient event flows in Java
• Practical isolation methodology – Secure production-level language runtimes with low effort (OpenJDK 6)
• Evaluation – Throughput and latency overhead
12
![Page 14: DEFCon: High-Performance Event Processing with ......JVM1 Isolation with Separate JVMs •Comparison with Marketcetera (Open Source trading platform) – One JVM per investor • Throughput:](https://reader033.vdocuments.net/reader033/viewer/2022050202/5f560cab283d6e731118bd3f/html5/thumbnails/14.jpg)
• DEFC assumes units communicate through labelled events • How to control communication between units?
– VM or OS processes: heavy, require copying of data – Use threads: sharing data in single address space – Java: mature, pervasive, good performance
• How to control communication between Java threads?
DEFCon: Controlling Event Flows
DEFCon
Bank Investor Client Monitor
Client Investor
DEFCon label check
1
2
13
?
![Page 15: DEFCon: High-Performance Event Processing with ......JVM1 Isolation with Separate JVMs •Comparison with Marketcetera (Open Source trading platform) – One JVM per investor • Throughput:](https://reader033.vdocuments.net/reader033/viewer/2022050202/5f560cab283d6e731118bd3f/html5/thumbnails/15.jpg)
Communication: Threads Share Immutable Data
• Unit Threads create new objects to put in events • Problem: how to deliver them to receiving units?
– Copy objects in events • Slow
14 DEFCon
Stock:IBM
Bank Investor Client Monitor
Stock:IBM
S:{} S:{}
![Page 16: DEFCon: High-Performance Event Processing with ......JVM1 Isolation with Separate JVMs •Comparison with Marketcetera (Open Source trading platform) – One JVM per investor • Throughput:](https://reader033.vdocuments.net/reader033/viewer/2022050202/5f560cab283d6e731118bd3f/html5/thumbnails/16.jpg)
Communication: Threads Share Immutable Data
• Unit Threads create new objects to put in events • Problem: how to deliver them to receiving units?
– Copy objects in events • Slow
– Transfer references to shared objects
15 DEFCon
Stock:IBM
Bank Investor Client Monitor
S:{} S:{}
![Page 17: DEFCon: High-Performance Event Processing with ......JVM1 Isolation with Separate JVMs •Comparison with Marketcetera (Open Source trading platform) – One JVM per investor • Throughput:](https://reader033.vdocuments.net/reader033/viewer/2022050202/5f560cab283d6e731118bd3f/html5/thumbnails/17.jpg)
Communication: Threads Share Immutable Data
• Unit Threads create new objects to put in events • Problem: how to deliver them to receiving units?
– Copy objects in events • Slow
– Transfer references to shared objects • Problem if unit labels change
16 DEFCon
Stock:IBM
Bank Investor Client Monitor
S:{} S:{} S:{client77} ?
![Page 18: DEFCon: High-Performance Event Processing with ......JVM1 Isolation with Separate JVMs •Comparison with Marketcetera (Open Source trading platform) – One JVM per investor • Throughput:](https://reader033.vdocuments.net/reader033/viewer/2022050202/5f560cab283d6e731118bd3f/html5/thumbnails/18.jpg)
Communication: Threads Share Immutable Data
• Unit Threads create new objects to put in events • Problem: how to deliver them to receiving units?
– Copy objects in events • Slow
– Transfer references to shared objects • Problem if unit labels change
• Shared state allows unrestricted communication – Solution: only allow immutable objects in event parts
17 DEFCon
Bank Investor Client Monitor
ImmutableStock:IBM
S:{} S:{}
![Page 19: DEFCon: High-Performance Event Processing with ......JVM1 Isolation with Separate JVMs •Comparison with Marketcetera (Open Source trading platform) – One JVM per investor • Throughput:](https://reader033.vdocuments.net/reader033/viewer/2022050202/5f560cab283d6e731118bd3f/html5/thumbnails/19.jpg)
Class Library
Communication: Shared State in Runtimes
18
JVM
Client Monitor Client Investor Bank Investor
DEFCon
static fields
native methods
Native OS
~4000
~2000
OpenJDK 6
![Page 20: DEFCon: High-Performance Event Processing with ......JVM1 Isolation with Separate JVMs •Comparison with Marketcetera (Open Source trading platform) – One JVM per investor • Throughput:](https://reader033.vdocuments.net/reader033/viewer/2022050202/5f560cab283d6e731118bd3f/html5/thumbnails/20.jpg)
Isolation Methodology Overview
• Goal – Provide isolation between Java Threads – Secure potentially dangerous targets: static fields and native
methods
• Previous Java isolation approaches – Do not support fast message passing between isolates (MVM) – Use custom Class Libraries and/or JVMs (I-JVM) – Require extensive analysis of Class Library (KaffeOS, Joe-E)
• Our approach 1. Identify potentially dangerous targets using static analysis 2. Modify runtime behaviour of targets using aspect oriented
programming (AOP) 3. White-list safe targets
19
![Page 21: DEFCon: High-Performance Event Processing with ......JVM1 Isolation with Separate JVMs •Comparison with Marketcetera (Open Source trading platform) – One JVM per investor • Throughput:](https://reader033.vdocuments.net/reader033/viewer/2022050202/5f560cab283d6e731118bd3f/html5/thumbnails/21.jpg)
Class Library
1. Static Analysis
20
Client Monitor Client Investor Bank Investor
DEFCon
JVM Native
OS
static fields
native methods
~4000
~2000
OpenJDK 6
![Page 22: DEFCon: High-Performance Event Processing with ......JVM1 Isolation with Separate JVMs •Comparison with Marketcetera (Open Source trading platform) – One JVM per investor • Throughput:](https://reader033.vdocuments.net/reader033/viewer/2022050202/5f560cab283d6e731118bd3f/html5/thumbnails/22.jpg)
Class Library
1. Static Analysis
21
Client Monitor Client Investor Bank Investor
DEFCon
JVM Native
OS
removed
static fields
native methods
~4000
~2000
~2000
~1000
OpenJDK 6
![Page 23: DEFCon: High-Performance Event Processing with ......JVM1 Isolation with Separate JVMs •Comparison with Marketcetera (Open Source trading platform) – One JVM per investor • Throughput:](https://reader033.vdocuments.net/reader033/viewer/2022050202/5f560cab283d6e731118bd3f/html5/thumbnails/23.jpg)
Class Library
1. Static Analysis
22
Client Monitor Client Investor Bank Investor
DEFCon
reachable
JVM Native
OS
removed
static fields
native methods
~4000
~2000
~2000 ~900
~1000 ~300
OpenJDK 6
![Page 24: DEFCon: High-Performance Event Processing with ......JVM1 Isolation with Separate JVMs •Comparison with Marketcetera (Open Source trading platform) – One JVM per investor • Throughput:](https://reader033.vdocuments.net/reader033/viewer/2022050202/5f560cab283d6e731118bd3f/html5/thumbnails/24.jpg)
Class Library
2. AOP Runtime Injection
23
Client Monitor Client Investor Bank Investor
DEFCon
removed
reachable
JVM Native
OS
static fields
native methods
~4000
~2000
~2000 ~900
~1000 ~300
transparent duplication
security checks
OpenJDK 6
![Page 25: DEFCon: High-Performance Event Processing with ......JVM1 Isolation with Separate JVMs •Comparison with Marketcetera (Open Source trading platform) – One JVM per investor • Throughput:](https://reader033.vdocuments.net/reader033/viewer/2022050202/5f560cab283d6e731118bd3f/html5/thumbnails/25.jpg)
Class Library
3. White-listing
24
Client Monitor Client Investor Bank Investor
DEFCon
removed
reachable
JVM Native
OS
static fields
native methods
~4000
~2000
~2000 ~900
~1000 ~300
transparent duplication
security checks
white-listing
OpenJDK 6
![Page 26: DEFCon: High-Performance Event Processing with ......JVM1 Isolation with Separate JVMs •Comparison with Marketcetera (Open Source trading platform) – One JVM per investor • Throughput:](https://reader033.vdocuments.net/reader033/viewer/2022050202/5f560cab283d6e731118bd3f/html5/thumbnails/26.jpg)
Class Library
3. White-listing
25
Client Monitor Client Investor Bank Investor
DEFCon
removed
reachable
JVM Native
OS
static fields
native methods
~4000
~2000
~2000 ~900
~1000 ~300
transparent duplication
security checks
white-listing
Target type
Manually white-listed
for unit execution for performance
static fields 27 6
native methods 15 9 OpenJDK 6
![Page 27: DEFCon: High-Performance Event Processing with ......JVM1 Isolation with Separate JVMs •Comparison with Marketcetera (Open Source trading platform) – One JVM per investor • Throughput:](https://reader033.vdocuments.net/reader033/viewer/2022050202/5f560cab283d6e731118bd3f/html5/thumbnails/27.jpg)
Isolation Summary
• What we achieved – Secured OpenJDK 6 for running financial scenario – Required few days of manual work – Easily applicable to new versions/different JDKs
• Limitations – Assumes knowledge of unit bytecode for static analysis
• Might need additional effort for new units
– Manual code auditing subject to human errors
26
![Page 28: DEFCon: High-Performance Event Processing with ......JVM1 Isolation with Separate JVMs •Comparison with Marketcetera (Open Source trading platform) – One JVM per investor • Throughput:](https://reader033.vdocuments.net/reader033/viewer/2022050202/5f560cab283d6e731118bd3f/html5/thumbnails/28.jpg)
Contributions and Overview
• Decentralized Event Flow Control (DEFC) model – IFC applied to event processing
• DEFCon high-performance implementation – Safe and efficient event flows in Java
• Practical isolation methodology – Secure production-level language runtimes with low effort (OpenJDK 6)
• Evaluation – Throughput and latency overhead
27
![Page 29: DEFCon: High-Performance Event Processing with ......JVM1 Isolation with Separate JVMs •Comparison with Marketcetera (Open Source trading platform) – One JVM per investor • Throughput:](https://reader033.vdocuments.net/reader033/viewer/2022050202/5f560cab283d6e731118bd3f/html5/thumbnails/29.jpg)
Evaluation: Performance Overhead
• Overhead of security (labels and isolation • Measure overhead
– Rate of processed ticks – Latency of produced deals
• Synthetic traces on 6k stock symbols – Prices set to trigger a deal every 10 ticks
• Experiments on dual Intel Xeon E5540 2.53GHz
28
DEFCon
Stock Ticker
Client Investor
Bank Investor
Monitor
Monitor
Broker
monitor
match
tick order
deal
![Page 30: DEFCon: High-Performance Event Processing with ......JVM1 Isolation with Separate JVMs •Comparison with Marketcetera (Open Source trading platform) – One JVM per investor • Throughput:](https://reader033.vdocuments.net/reader033/viewer/2022050202/5f560cab283d6e731118bd3f/html5/thumbnails/30.jpg)
Acceptable Reduction on Throughput
• Label checks: marginal overhead • Isolation: ~20% overhead
29
![Page 31: DEFCon: High-Performance Event Processing with ......JVM1 Isolation with Separate JVMs •Comparison with Marketcetera (Open Source trading platform) – One JVM per investor • Throughput:](https://reader033.vdocuments.net/reader033/viewer/2022050202/5f560cab283d6e731118bd3f/html5/thumbnails/31.jpg)
Low Impact on Latency
• Label checks: ~0.5 ms overhead • Isolation: ~1ms overhead
30
![Page 32: DEFCon: High-Performance Event Processing with ......JVM1 Isolation with Separate JVMs •Comparison with Marketcetera (Open Source trading platform) – One JVM per investor • Throughput:](https://reader033.vdocuments.net/reader033/viewer/2022050202/5f560cab283d6e731118bd3f/html5/thumbnails/32.jpg)
JVM2
JVM1
Isolation with Separate JVMs
• Comparison with Marketcetera (Open Source trading platform) – One JVM per investor
• Throughput: – Comparable with DEFCon with few investors – Does not scale
• Latency: around 8 ms 31
Stock Ticker
Client Investor
Bank Investor
Monitor
Monitor
Broker
monitor
match
tick order
deal
![Page 33: DEFCon: High-Performance Event Processing with ......JVM1 Isolation with Separate JVMs •Comparison with Marketcetera (Open Source trading platform) – One JVM per investor • Throughput:](https://reader033.vdocuments.net/reader033/viewer/2022050202/5f560cab283d6e731118bd3f/html5/thumbnails/33.jpg)
Future Work
• Distribution – Performance limited by number of cores – Scale DEFCon to multiple engines
• Usability – Correctly assigning labels is hard – Tools to help design and automatically check labelling
• Performance isolation – Units compete for resources – Prevent uncooperative behaviours
32
![Page 34: DEFCon: High-Performance Event Processing with ......JVM1 Isolation with Separate JVMs •Comparison with Marketcetera (Open Source trading platform) – One JVM per investor • Throughput:](https://reader033.vdocuments.net/reader033/viewer/2022050202/5f560cab283d6e731118bd3f/html5/thumbnails/34.jpg)
Conclusion
• Event processing requires security and low latency
• DEFC model – Provides strong and fine-grained security by
applying Information Flow Control to event processing
• DEFCon implementation – Processes events in single address space for performance – Provides isolation on production-level language runtimes
Tracking and enforcing security of event flows can be done with reasonable overhead
• Thank You! … Questions? • ([email protected])
33
![Page 35: DEFCon: High-Performance Event Processing with ......JVM1 Isolation with Separate JVMs •Comparison with Marketcetera (Open Source trading platform) – One JVM per investor • Throughput:](https://reader033.vdocuments.net/reader033/viewer/2022050202/5f560cab283d6e731118bd3f/html5/thumbnails/35.jpg)
END
• END 34