java one 2015 - v1
TRANSCRIPT
Michael Dawson
October 2015
Leveraging Java Optimizations to Improve Density in Cloud Environments
About Michael Dawson Loves the web and building software
2
Senior Software Developer @ IBM23 years development experience11 years in runtime development
Contact me:
[email protected]: @mhdawson1https://www.linkedin.com/pub/michael-dawson/2/128/605
3
• Motivation
• Shared Classes
• Cloud Environments• Docker• Cloud Foundry
• Leveraging SC in Cloud
Agenda
4
• Startup affects customers
• - Restart
• - Scale out
• Footprint affects cost• Lots of small apps
Motivation
IBM SDK for Java
java -Xshareclasses:name=myHelloApp -jar
5
Shared Classes
No -Xshareclasses -Xshareclasses(1st run) -Xshareclasses
0
200
400
600
800
1000
1200
1400
1600
Java 70 Tomcat Startup
Linux x86_64
Tim
e (
ms
)
Faster Startup Smaller Footprint
6
Shared Classes
• What• Read only part of Class
• Class File Bytes• String De-duplication• Jit Data (AOT)• Class Debug Data (separate)
• How• SysV Shared Memory (e.g. on z/OS)• POSIX mmap (e.g. on Linux and AIX)• CreateFileMapping (e.g. on Windows)
7
Shared Classes – Faster Startup• Classloading Time
• Read from disk• Parse• Create in-memory representation
• JIT artifacts• Methods initially run interpreted• 1000’s of methods are JIT compiled• Start-up methods same run to run• Why compile over and over ?• Metadata to guide compilation
8
Shared Classes – Lower Footprint
HeapObject Instances
(e.g. tmp)
JITCode & Data Cache
Shared Class Data
Byte Codes(e.g. MyObject)
JVM Process Memory
HeapObject Instances
(e.g. tmp)
JITCode & Data Cache
Shared Class DataByte Codes
(e.g. MyObject)
JVM Process Memory
Shared Class Data
Byte Codes(e.g. MyObject)
9
Cloud Environments
Bare Metal
Virtual Machines
Docker
PaaS
10
Cloud Environments – Bare Metal
Multiple Apps Common
Shared storage and memory
Shared Classes works by default
APP1
APPN
Faster Startup
Lower Footprint
11
Cloud Environments – Virtual Machine
Smaller deployment units (+)
Memory not shared across VMs
Storage not shared across VMs
APP1
APPN
VM1 VMn
Shared Classes – Reduced Effectiveness
Faster Startup
Lower Footprint
12
Cloud Environments - Docker
Base OS (ex Ubuntu 14)
MiddlewareApplication
Application
https://hub.docker.com/_/websphere-liberty/
https://hub.docker.com/_/ubuntu/
APP1
APPN
CONT1 CONTn
Middleware
Base OS
Images
Application
State1 Staten
Containers
13
Cloud Environments – Docker Demo
Docker images
Docker ps
Start/stop show state not preserved
14
Cloud Environments – Docker
Even smaller deployment units (++)
Memory not shared across VMs
Storage not persistent
APP1
APPN
CONT1 CONTn
Shared Classes – Reduced Effectiveness
Faster Startup
Lower Footprint
15
Cloud Environments – PaaS – Cloud Foundry
Staging -> buildpack creates Droplet
Start or Scale
Droplet to DEA, Extract Droplet
Warden with droplet contents
Droplet size affects push/startup times
16
Cloud Environments – Cloud Foundry
17
Cloud Environments – PaaS
Even smaller deployment units (+++)
Memory not shared across VMs
Storage not persistent
APP1
APPN
Warden1
Shared Classes – Reduced Effectiveness
Faster Startup
Lower Footprint
WardenN
18
Cloud Environments – What’s different
Persistent
Storage
Shared
Storage
Shared
Memory
Bare Metal X X X
Virtual
Machines
X
Docker
PaaS
19
Cloud Environments – What can we do ?
Start-up
Persistent Storage ->Generate Dynamically
Add cache sharing mechanism
Shared storage
Cache server
Bundle pre-built cache
Populate through warm up
Extra disk footprint
20
Cloud Environments – What can we do ?
Footprint
Transparent Page sharing at OS
Bundled pre-built cache
Cache sharing mechanism
Guided build
Share Memory across containers/wardens
Virt1
Virt2
VirtN
phys
21
Cloud Environments – Virtual Machines
Persistent storage – Startup benefit by default
Memory - build Cache to Enable sharing through TPS
Pre-created, bundled or cache server
Guide creation to end up with same result
To read more:
http://www.computer.org/csdl/proceedings/ispass/2013/5
776/00/06557144.pdf
22
Cloud Environments - Docker
Startup – shared cache pre-built (read-only)
Footprint – Share memory across container !
Shared Cache disk file from shared layer
Memory mapping results in shared memory
Validated through lsof and /proc/pid/smaps
23
Cloud Environments - Docker
APP1
APPN
CONT1 CONTn
Liberty, JVM, Cache
Base OS
Application
State1 Staten
In Memory cache !!
Middleware Image with:
JVM
Liberty
Pre-built Shared
Cache
Cache Read-only
File system back end
with right properties (ex
aufs)
24
1 2 3 4 5 6 7 8 9 10
0
200
400
600
800
1000
1200
1400
Used Memory (PSS)
Share
NoShare
NoShareNoCache
Cloud Environments – Docker
1 2 3 4 5 6 7 8 9 10
0
5
10
15
20
25
Startup Times
Share
NoShare
NoShareNoCache
Share: Shared cache pre-created and including in base layer –-Xshareclasses:cacheDir=/usr/lib/liberty-java/caches,name=cache1,enableBCI
NoShare: cached created dynamically -Xshareclasses
NoShareNoCache: No cache -Xshareclasses:none
Faster Startup
Lower Footprint
25
Cloud Environments – Cloud Foundry
Experimented with 2 optimizations so far
Share memory across wardens
Requires changes to CF installation
Cache server
Joint work with CAS Atlantic
http://www.unb.ca/research/casatlantic/index.html
Panagiotis Patros, Kenneth Kent
26
Cloud Environments – Cloud Foundry – Share Memory
JVM, Liberty Cashed installed on DEA
Mapping options
Same File available in all Wardens
Memory map of same file -> shared memory across Wardens
R/O due to security concerns
- src_path: /var/vcap/packages/ibmjvm64dst_path: /tmp/jvms/ibmjvm64mode: ro
src_path: /var/vcap/packages/ibmClassCachesdst_path: /tmp/jvms/ibmClassCachesmode: ro- src_path: /var/vcap/packages/libertydst_path: /tmp/libertymode: ro
27
Cloud Environments – Cloud Foundry – Share Memory
0
2
4
6
8
10
12
14
16
18
20
No Sharing Sharing Libraries Sharing Libraries andCache
Start times (seconds)
0
20
40
60
80
100
120
140
160
180
200
No Sharing Sharing Libraries Sharing Libraries andCache
Resident Set Size (MB)
Faster Startup
Lower Footprint
28
Deployed as Bluemix App
No changes to CF Infrastructure
Cache Created Dynamically
Zeroth instance creates cache
Restage after cache creation
Cache bundled into droplet
Avoids “pull time”
Avoids manual cache creation
Extended to other artifacts (JSP precompile)
Cloud Environments – Cloud Foundry – Cache Server
The DCAS technique brings the second (scaled) instance up faster
DCAS speedup over default is 1.37: around 3 seconds faster
Request to restart at 120s
Restarts happen when a new service is installed
DCAS brings up the server online faster
The DCAS technique brings the second (restarted) instance up faster
DCAS speedup over default is again 1.37: around 4 seconds faster
33
Questions
Copyrights and Trademarks
© IBM Corporation 2015. All Rights Reserved
IBM, the IBM logo, ibm.com are trademarks or registered
trademarks of International Business Machines Corp.,
registered in many jurisdictions worldwide. Other product and
service names might be trademarks of IBM or other companies.
A current list of IBM trademarks is available on the Web at
“Copyright and trademark information” at
www.ibm.com/legal/copytrade.shtml
Node.js is an official trademark of Joyent. IBM SDK for Node.js is not formally
related to or endorsed by the official Joyent Node.js open source or
commercial project.
Java, JavaScript and all Java-based trademarks and logos are trademarks or
registered trademarks of Oracle and/or its affiliates.