financial services analytics on aws
TRANSCRIPT
![Page 1: Financial Services Analytics on AWS](https://reader033.vdocuments.net/reader033/viewer/2022042907/5878fc791a28ab49608b6ccd/html5/thumbnails/1.jpg)
Financial Services Analytics on AWS
![Page 2: Financial Services Analytics on AWS](https://reader033.vdocuments.net/reader033/viewer/2022042907/5878fc791a28ab49608b6ccd/html5/thumbnails/2.jpg)
Infrastructure Regions Availability Zones Points of Presence
EnterpriseApplications Virtual Desktops Sharing & Collaboration
Core Services Storage(Object, Block and Archival)
Compute(VMs, Auto-scaling and Load Balancing)
Databases(Relational, NoSQL, Caching)
Networking(VPC, DX, DNS)
CDN
Access Control
Usage & Resource Tracking
Monitoring and Logs
Administration & Security
Key Storage & Management
IdentityManagement
Service Catalog
Platform Services
Deployment & ManagementOne-click web app deploymentDev/ops resourcemanagement
Resource Templates
PushNotifications
Mobile Services
Identity
Sync
Mobile Analytics
App ServicesQueuing &Notifications
Workflow
App streaming
Transcoding
Search
Analytics
Hadoop
Data Pipelines
Data warehouse
Real-timeStreaming Data
Code Deploy
Code Pipeline
Code Commit
Machine Learning
![Page 3: Financial Services Analytics on AWS](https://reader033.vdocuments.net/reader033/viewer/2022042907/5878fc791a28ab49608b6ccd/html5/thumbnails/3.jpg)
US-WEST (Oregon) EU-WEST (Ireland)ASIA PAC (Tokyo)
ASIA PAC (Singapore)
US-WEST (San Francisco)
SOUTH AMERICA (Sao Paulo)
US-EAST (Virginia)
GOV CLOUD
ASIA PAC (Sydney)
Global Infrastructure
CHINA (beta)
EU-CENTRAL(Germany)
Current Region
Announced Region
Hong Kong
Ohio
London
India
![Page 4: Financial Services Analytics on AWS](https://reader033.vdocuments.net/reader033/viewer/2022042907/5878fc791a28ab49608b6ccd/html5/thumbnails/4.jpg)
Availability Zone
Global Infrastructure
![Page 5: Financial Services Analytics on AWS](https://reader033.vdocuments.net/reader033/viewer/2022042907/5878fc791a28ab49608b6ccd/html5/thumbnails/5.jpg)
![Page 6: Financial Services Analytics on AWS](https://reader033.vdocuments.net/reader033/viewer/2022042907/5878fc791a28ab49608b6ccd/html5/thumbnails/6.jpg)
Requirements
Store Any Amount of DataWithout Capacity Planning
Perform Complex Analysis on Any Data
Scale on Demand
Store Data SecurelyMove to Real Time
Realised Value
Agile Analytics, DevOps in the WarehouseDecrease Time to Market
Build Environments Quickly
Reduce CostsReduce Capital Expenditure
Enable Global Reach
![Page 7: Financial Services Analytics on AWS](https://reader033.vdocuments.net/reader033/viewer/2022042907/5878fc791a28ab49608b6ccd/html5/thumbnails/7.jpg)
87% now will consider cloud
for their big data Advanced analytics
closing-in on BI
Issues beyond security (reality, perception, regulation) being
addressed by march of technology
Building & deploying Big Data analytics or processing applications in the cloud can reduce complexity and time to market
Source: Gigaom Research data warehous ing survey 2014
![Page 8: Financial Services Analytics on AWS](https://reader033.vdocuments.net/reader033/viewer/2022042907/5878fc791a28ab49608b6ccd/html5/thumbnails/8.jpg)
Ingestion…
Integration…
Retention
![Page 9: Financial Services Analytics on AWS](https://reader033.vdocuments.net/reader033/viewer/2022042907/5878fc791a28ab49608b6ccd/html5/thumbnails/9.jpg)
STORAGECOMPUTECOMPUTE COMPUTE
COMPUTECOMPUTE
COMPUTE
COMPUTE
COMPUTECOMPUTE
COMPUTE
![Page 10: Financial Services Analytics on AWS](https://reader033.vdocuments.net/reader033/viewer/2022042907/5878fc791a28ab49608b6ccd/html5/thumbnails/10.jpg)
Availability99.99%
Durability 99.999999999%
A Distributed Object StoreNot a file system
No Single Points of FailureEventually consistent
Paradigm Object storePerformance Very FastRedundancy Across Availability Zones
Security Public Key / Private KeyPricing $0.03/GB/month
Typical use case Write once, read many
Simple Storage Service
Highly scalable object storage for the internet
1 byte to 5TB in size99.999999999% durability
![Page 11: Financial Services Analytics on AWS](https://reader033.vdocuments.net/reader033/viewer/2022042907/5878fc791a28ab49608b6ccd/html5/thumbnails/11.jpg)
S3 – Standard S3 – Infrequent Access Amazon Glacier
![Page 12: Financial Services Analytics on AWS](https://reader033.vdocuments.net/reader033/viewer/2022042907/5878fc791a28ab49608b6ccd/html5/thumbnails/12.jpg)
34 secs per terabyte
GB/Second
Read
er C
onne
ctio
ns
Amazon S3 provides near linear scalability
S3 Streaming Performance100 VMs; 9.6GB/s; $26/hr 350 VMs; 28.7GB/s; $90/hr
S3 Performance & Scalability
![Page 13: Financial Services Analytics on AWS](https://reader033.vdocuments.net/reader033/viewer/2022042907/5878fc791a28ab49608b6ccd/html5/thumbnails/13.jpg)
AWS Security Services
Compute Storage
AWS Global Infrastructure
Database
App Services
Deployment & Administration
Networking
Analytics
![Page 14: Financial Services Analytics on AWS](https://reader033.vdocuments.net/reader033/viewer/2022042907/5878fc791a28ab49608b6ccd/html5/thumbnails/14.jpg)
IAM
Users
AWS
Directory Service
AD Connector
Direct Connect
Hardware VPN
![Page 15: Financial Services Analytics on AWS](https://reader033.vdocuments.net/reader033/viewer/2022042907/5878fc791a28ab49608b6ccd/html5/thumbnails/15.jpg)
Amazon KinesisManaged Service for Real Time Big Data ProcessingCreate Streams to Produce & Consume DataElastically Add and Remove Shards for PerformanceUse Kinesis Worker Library to Process DataIntegration with S3, Redshift and Dynamo DB
Compute Storage
AWS Global Infrastructure
Database
App Services
Deployment & Administration
Networking
Analytics
Application Services
![Page 16: Financial Services Analytics on AWS](https://reader033.vdocuments.net/reader033/viewer/2022042907/5878fc791a28ab49608b6ccd/html5/thumbnails/16.jpg)
Data Sources
App.4
[Machine Learning]
AWS E
ndpo
int
App.1
[Aggregate & De-‐Duplicate]
Data Sources
Data Sources
Data Sources
App.2
[Metric Extraction]
S3
DynamoDB
Redshift
App.3[Sliding Window Analysis]
Data Sources
Availability Zone
Kinesis Streams
Availability Zone
Availability Zone
Shard 1Shard 2Shard N
![Page 17: Financial Services Analytics on AWS](https://reader033.vdocuments.net/reader033/viewer/2022042907/5878fc791a28ab49608b6ccd/html5/thumbnails/17.jpg)
without writing an application managing infrastructure
Batch compress encryptin as little as 60 secs
Capture and submit streaming data to Firehose
Firehose loads streaming data continuously into S3 and Redshift
Analyze streaming data using your favorite BI tools
Kinesis Firehose
![Page 18: Financial Services Analytics on AWS](https://reader033.vdocuments.net/reader033/viewer/2022042907/5878fc791a28ab49608b6ccd/html5/thumbnails/18.jpg)
![Page 19: Financial Services Analytics on AWS](https://reader033.vdocuments.net/reader033/viewer/2022042907/5878fc791a28ab49608b6ccd/html5/thumbnails/19.jpg)
Traditional Business Intelligence…
OLAP…
Data Sources for ML
![Page 20: Financial Services Analytics on AWS](https://reader033.vdocuments.net/reader033/viewer/2022042907/5878fc791a28ab49608b6ccd/html5/thumbnails/20.jpg)
Relational Database ServiceManaged Database-as-a-ServiceNo need to install or manage database instancesAutomated Backup/Recover, Patching & UpgradeScalable and fault tolerant configurations6TB & 30,000 IOPS
Managed Database
RDS Dynamo DB
Redshift ElastiCache
Compute Storage
AWS Global Infrastructure
Database
App Services
Deployment & Administration
Networking
Analytics
![Page 21: Financial Services Analytics on AWS](https://reader033.vdocuments.net/reader033/viewer/2022042907/5878fc791a28ab49608b6ccd/html5/thumbnails/21.jpg)
Managed Data Warehouse
RedshiftManaged Massively Parallel Petabyte Scale Data WarehouseStreaming Backup/Restore to S3Load data from S3, DynamoDB and EMRExtensive Security FeaturesScale from 160 GB -> 1.6 PB Online
RDS Dynamo DB
Redshift ElastiCache
Compute Storage
AWS Global Infrastructure
Database
App Services
Deployment & Administration
Networking
Analytics
![Page 22: Financial Services Analytics on AWS](https://reader033.vdocuments.net/reader033/viewer/2022042907/5878fc791a28ab49608b6ccd/html5/thumbnails/22.jpg)
Redshift lets you start small and grow bigExtra Large Node (dc1.xl & ds2.xl)3 spindles, 15-30GiB RAM 2 or 4 virtual cores, 10GigE
Single Node (160GB SSD or 2TB Magnetic)
Cluster 2-32 Nodes (320GB SSD – 64TB Magnetic)
8 Extra Large Node (dc1.8xl & ds2.8xl)24 spindles, 120-244GiB RAM, 2.56TB SSD or 16TB Magnetic, 16 or 32 virtual cores, 10GigE
Cluster 2-100 Nodes (5TB SSD – 1.6PB Magnetic)
8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL
8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL
8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL
8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL
8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL
8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL
8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL
8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL
8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL
8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL
XL
XLXLXLXL
XLXLXLXL
XLXLXLXL
XLXLXLXL
XLXLXLXL
XLXLXLXL
XLXLXLXL
XLXLXLXL
![Page 23: Financial Services Analytics on AWS](https://reader033.vdocuments.net/reader033/viewer/2022042907/5878fc791a28ab49608b6ccd/html5/thumbnails/23.jpg)
23
LEADING INDEX PROVIDER WITH 41,000+ INDEXES
ACROSS ASSET CLASSES AND GEOGRAPHIES
Over 10,000 Corporate Clients in 60 countries
Our technology powers over
70 MARKETPLACES,
regulators, CSDs and clearing-houses
in over 50 COUNTRIES100+ DATA
PRODUCT OFFERINGSsupporting 2.5+ millioninvestment professionals
and users IN 98 COUNTRIES
26 Markets 3 Clearing Houses5 Central Securities
Depositories
Lists more than 3,500 companies in 35 countries,
representing more than $8.8 trillion in total market value
![Page 24: Financial Services Analytics on AWS](https://reader033.vdocuments.net/reader033/viewer/2022042907/5878fc791a28ab49608b6ccd/html5/thumbnails/24.jpg)
![Page 25: Financial Services Analytics on AWS](https://reader033.vdocuments.net/reader033/viewer/2022042907/5878fc791a28ab49608b6ccd/html5/thumbnails/25.jpg)
![Page 26: Financial Services Analytics on AWS](https://reader033.vdocuments.net/reader033/viewer/2022042907/5878fc791a28ab49608b6ccd/html5/thumbnails/26.jpg)
![Page 27: Financial Services Analytics on AWS](https://reader033.vdocuments.net/reader033/viewer/2022042907/5878fc791a28ab49608b6ccd/html5/thumbnails/27.jpg)
Exploratory Analytics…
Data Cleansing…
Advanced Data Science
![Page 28: Financial Services Analytics on AWS](https://reader033.vdocuments.net/reader033/viewer/2022042907/5878fc791a28ab49608b6ccd/html5/thumbnails/28.jpg)
Elastic MapReduceManaged, elastic Hadoop (1.x & 2.x) clusterIntegrates with S3, DynamoDB and RedshiftInstall End User Tools Automatically (Spark, Presto, Impala)Support for EC2 Spot InstancesTransient or Always on Clusters
Managed Big Data
Elastic MapReduce
Compute Storage
AWS Global Infrastructure
Database
App Services
Deployment & Administration
Networking
Analytics
![Page 29: Financial Services Analytics on AWS](https://reader033.vdocuments.net/reader033/viewer/2022042907/5878fc791a28ab49608b6ccd/html5/thumbnails/29.jpg)
Try different configurations to find the optimal cost/performance balance
CPUc4 family
cc2.8xlarged2 family
Memorym2 familyr3 family
Disk/IOd2 familyi2 family
Generalm3 family
Choose your instance types
ETL Machine Learning Spark HDFS
![Page 30: Financial Services Analytics on AWS](https://reader033.vdocuments.net/reader033/viewer/2022042907/5878fc791a28ab49608b6ccd/html5/thumbnails/30.jpg)
Weather Insurance for Farms
Challenge:Volatile weather is deadly to crops like grapes
60 years of crop data
200 TB of S3 Data
1M government Doppler radar points
Solution:Built a predictive model based on freely available data:
150B Soil Observations 850K Precision Rainfall Grids Tracked
3M Daily Weather Measurements
50 EMR clusters process new data as it comes into S3
each day, continuously updating the model
![Page 31: Financial Services Analytics on AWS](https://reader033.vdocuments.net/reader033/viewer/2022042907/5878fc791a28ab49608b6ccd/html5/thumbnails/31.jpg)
$10-20M Savings by moving Platform to AWS
![Page 32: Financial Services Analytics on AWS](https://reader033.vdocuments.net/reader033/viewer/2022042907/5878fc791a28ab49608b6ccd/html5/thumbnails/32.jpg)
![Page 33: Financial Services Analytics on AWS](https://reader033.vdocuments.net/reader033/viewer/2022042907/5878fc791a28ab49608b6ccd/html5/thumbnails/33.jpg)
Predictive Analytics…
![Page 34: Financial Services Analytics on AWS](https://reader033.vdocuments.net/reader033/viewer/2022042907/5878fc791a28ab49608b6ccd/html5/thumbnails/34.jpg)
Easily create machine learning models
Visualize and optimize models
Put models into production in seconds
Battle-hardened technologyMachine Learning
SoftwareDevelopment
Introducing Amazon Machine Learning
![Page 35: Financial Services Analytics on AWS](https://reader033.vdocuments.net/reader033/viewer/2022042907/5878fc791a28ab49608b6ccd/html5/thumbnails/35.jpg)
Developing with Amazon Machine Learning
Buildmodel
Validate &optimize
Make predictions
1 2 3
![Page 36: Financial Services Analytics on AWS](https://reader033.vdocuments.net/reader033/viewer/2022042907/5878fc791a28ab49608b6ccd/html5/thumbnails/36.jpg)
Use existing data in S3, Redshift and RDS
Automatic data visualization & exploration
Descriptive and summary statistics
Your data doesn’t have to be perfect
Missing data, malformed data records, type validation
Building a Predictive Model
![Page 37: Financial Services Analytics on AWS](https://reader033.vdocuments.net/reader033/viewer/2022042907/5878fc791a28ab49608b6ccd/html5/thumbnails/37.jpg)
Model Validation and Optimization Tools
![Page 38: Financial Services Analytics on AWS](https://reader033.vdocuments.net/reader033/viewer/2022042907/5878fc791a28ab49608b6ccd/html5/thumbnails/38.jpg)
Batch predictionsAsynchronous predictions with trained model
Real time predictionsSynchronous, low latency, high throughputMount API end-point with a single click
Making Predictions
![Page 39: Financial Services Analytics on AWS](https://reader033.vdocuments.net/reader033/viewer/2022042907/5878fc791a28ab49608b6ccd/html5/thumbnails/39.jpg)
![Page 40: Financial Services Analytics on AWS](https://reader033.vdocuments.net/reader033/viewer/2022042907/5878fc791a28ab49608b6ccd/html5/thumbnails/40.jpg)
Data Visualiation…
![Page 41: Financial Services Analytics on AWS](https://reader033.vdocuments.net/reader033/viewer/2022042907/5878fc791a28ab49608b6ccd/html5/thumbnails/41.jpg)
Old-guard BI
Costs Too Much
Pay $ million before seeing first analysis3 year TCO $150 to $250 per user per month
Takes Too Long
Spend 6 to 12 months of consulting and SW implementation time
![Page 42: Financial Services Analytics on AWS](https://reader033.vdocuments.net/reader033/viewer/2022042907/5878fc791a28ab49608b6ccd/html5/thumbnails/42.jpg)
![Page 43: Financial Services Analytics on AWS](https://reader033.vdocuments.net/reader033/viewer/2022042907/5878fc791a28ab49608b6ccd/html5/thumbnails/43.jpg)
A very fast, cloud-powered, BI service for 1/10th the cost of old-guard BI software
![Page 44: Financial Services Analytics on AWS](https://reader033.vdocuments.net/reader033/viewer/2022042907/5878fc791a28ab49608b6ccd/html5/thumbnails/44.jpg)
$9 per user per month
With 1 year commitment
![Page 45: Financial Services Analytics on AWS](https://reader033.vdocuments.net/reader033/viewer/2022042907/5878fc791a28ab49608b6ccd/html5/thumbnails/45.jpg)
Business user
Sign-in
First analysis in about 60 seconds
Register for preview beginning Oct 7 at aws.amazon.com/quicksight
![Page 46: Financial Services Analytics on AWS](https://reader033.vdocuments.net/reader033/viewer/2022042907/5878fc791a28ab49608b6ccd/html5/thumbnails/46.jpg)
![Page 47: Financial Services Analytics on AWS](https://reader033.vdocuments.net/reader033/viewer/2022042907/5878fc791a28ab49608b6ccd/html5/thumbnails/47.jpg)
Business User
QuickSight API
Data Prep Metadata SuggestionsConnectors SPICE
Business User
QuickSight UI
Mobile Devices Web Browsers
Partner BI products
AmazonS3
Amazon Kinesis
Amazon DynamoDB
Amazon EMR
Amazon Redshift Amazon RDSFiles Third-party
![Page 48: Financial Services Analytics on AWS](https://reader033.vdocuments.net/reader033/viewer/2022042907/5878fc791a28ab49608b6ccd/html5/thumbnails/48.jpg)
![Page 49: Financial Services Analytics on AWS](https://reader033.vdocuments.net/reader033/viewer/2022042907/5878fc791a28ab49608b6ccd/html5/thumbnails/49.jpg)
![Page 50: Financial Services Analytics on AWS](https://reader033.vdocuments.net/reader033/viewer/2022042907/5878fc791a28ab49608b6ccd/html5/thumbnails/50.jpg)
![Page 51: Financial Services Analytics on AWS](https://reader033.vdocuments.net/reader033/viewer/2022042907/5878fc791a28ab49608b6ccd/html5/thumbnails/51.jpg)
![Page 52: Financial Services Analytics on AWS](https://reader033.vdocuments.net/reader033/viewer/2022042907/5878fc791a28ab49608b6ccd/html5/thumbnails/52.jpg)
Native mobile experienceiOS, Android
Full experience on tablets
Consumption experience on
smart phones
Very fast response
![Page 53: Financial Services Analytics on AWS](https://reader033.vdocuments.net/reader033/viewer/2022042907/5878fc791a28ab49608b6ccd/html5/thumbnails/53.jpg)
![Page 54: Financial Services Analytics on AWS](https://reader033.vdocuments.net/reader033/viewer/2022042907/5878fc791a28ab49608b6ccd/html5/thumbnails/54.jpg)
![Page 55: Financial Services Analytics on AWS](https://reader033.vdocuments.net/reader033/viewer/2022042907/5878fc791a28ab49608b6ccd/html5/thumbnails/55.jpg)
$9
$18
Per user per month
Per user per month
![Page 56: Financial Services Analytics on AWS](https://reader033.vdocuments.net/reader033/viewer/2022042907/5878fc791a28ab49608b6ccd/html5/thumbnails/56.jpg)
![Page 57: Financial Services Analytics on AWS](https://reader033.vdocuments.net/reader033/viewer/2022042907/5878fc791a28ab49608b6ccd/html5/thumbnails/57.jpg)
Integrated Analytics
![Page 58: Financial Services Analytics on AWS](https://reader033.vdocuments.net/reader033/viewer/2022042907/5878fc791a28ab49608b6ccd/html5/thumbnails/58.jpg)
![Page 59: Financial Services Analytics on AWS](https://reader033.vdocuments.net/reader033/viewer/2022042907/5878fc791a28ab49608b6ccd/html5/thumbnails/59.jpg)
![Page 60: Financial Services Analytics on AWS](https://reader033.vdocuments.net/reader033/viewer/2022042907/5878fc791a28ab49608b6ccd/html5/thumbnails/60.jpg)
![Page 61: Financial Services Analytics on AWS](https://reader033.vdocuments.net/reader033/viewer/2022042907/5878fc791a28ab49608b6ccd/html5/thumbnails/61.jpg)
![Page 62: Financial Services Analytics on AWS](https://reader033.vdocuments.net/reader033/viewer/2022042907/5878fc791a28ab49608b6ccd/html5/thumbnails/62.jpg)
Validate records, recordsets or datasets
Store validation status
Manage validation rules
Abide data store
Validation rules
Validation results / log
Manage ingestion rules
Split data into records
Assign record identifiers Output records
Store event details – rule, stamp etc
Assign record metadata
Check record format
Transform to common format
Ingestion rules
Ingestion audit log
Get data
Manage input queue
Manage receive rules
Assign dataset identifier
Assign dataset metadata
Store original data
Store event details
Receive rules
Receive audit log
Original data store
Data service endpoints Fetch data set Perform
calculations Save datasets Re-‐validate
Store event details
Manage processing rules
Processing audit log
Processing rules
Format data Check data
Store event details
Manage output rules
Send output
Output audit log
Output rules
Data service endpoints
Storage
Service endpoint
Function
Rules
Receive
Ingest
Validate
Process
Output
Raw data
Common format
Validated
Processed
Output format
Data path
Events & logic
Optional data path
Raw data*
With dataset metadata*
* Visio 2013 only © Abide Financial
![Page 63: Financial Services Analytics on AWS](https://reader033.vdocuments.net/reader033/viewer/2022042907/5878fc791a28ab49608b6ccd/html5/thumbnails/63.jpg)
![Page 64: Financial Services Analytics on AWS](https://reader033.vdocuments.net/reader033/viewer/2022042907/5878fc791a28ab49608b6ccd/html5/thumbnails/64.jpg)
![Page 65: Financial Services Analytics on AWS](https://reader033.vdocuments.net/reader033/viewer/2022042907/5878fc791a28ab49608b6ccd/html5/thumbnails/65.jpg)
![Page 66: Financial Services Analytics on AWS](https://reader033.vdocuments.net/reader033/viewer/2022042907/5878fc791a28ab49608b6ccd/html5/thumbnails/66.jpg)
![Page 67: Financial Services Analytics on AWS](https://reader033.vdocuments.net/reader033/viewer/2022042907/5878fc791a28ab49608b6ccd/html5/thumbnails/67.jpg)
![Page 68: Financial Services Analytics on AWS](https://reader033.vdocuments.net/reader033/viewer/2022042907/5878fc791a28ab49608b6ccd/html5/thumbnails/68.jpg)
![Page 69: Financial Services Analytics on AWS](https://reader033.vdocuments.net/reader033/viewer/2022042907/5878fc791a28ab49608b6ccd/html5/thumbnails/69.jpg)
Thank You!