make your own cloud security monitoring solution · common intrusion detection!6. actual response...
TRANSCRIPT
![Page 1: Make Your Own Cloud Security Monitoring Solution · Common Intrusion Detection!6. Actual Response Unit!7. For this talk, we will focus on the CIDF components provided by AWS and GCP](https://reader034.vdocuments.net/reader034/viewer/2022042417/5f338b15de182847ed6f4dd0/html5/thumbnails/1.jpg)
Make Your Own Cloud Security
Monitoring Solution
�1
![Page 2: Make Your Own Cloud Security Monitoring Solution · Common Intrusion Detection!6. Actual Response Unit!7. For this talk, we will focus on the CIDF components provided by AWS and GCP](https://reader034.vdocuments.net/reader034/viewer/2022042417/5f338b15de182847ed6f4dd0/html5/thumbnails/2.jpg)
About The Presenter
Security Researcher
Former ISS X-Force Member
Currently Staff Engineer at Datadog* *These opinions are mine and not my employer’s
John Ventura
@JohnAVentura!2
![Page 3: Make Your Own Cloud Security Monitoring Solution · Common Intrusion Detection!6. Actual Response Unit!7. For this talk, we will focus on the CIDF components provided by AWS and GCP](https://reader034.vdocuments.net/reader034/viewer/2022042417/5f338b15de182847ed6f4dd0/html5/thumbnails/3.jpg)
Alerting in the Cloud
• Build an alerting system for GCP or AWS
• Associated dangers
• Overcoming these dangers
!3
![Page 4: Make Your Own Cloud Security Monitoring Solution · Common Intrusion Detection!6. Actual Response Unit!7. For this talk, we will focus on the CIDF components provided by AWS and GCP](https://reader034.vdocuments.net/reader034/viewer/2022042417/5f338b15de182847ed6f4dd0/html5/thumbnails/4.jpg)
Why Alerting?
Do you have policies?
Do people make mistakes?
Do you get attacked?
!4
![Page 5: Make Your Own Cloud Security Monitoring Solution · Common Intrusion Detection!6. Actual Response Unit!7. For this talk, we will focus on the CIDF components provided by AWS and GCP](https://reader034.vdocuments.net/reader034/viewer/2022042417/5f338b15de182847ed6f4dd0/html5/thumbnails/5.jpg)
Build An Alerting System
You already have the components!
!5
![Page 6: Make Your Own Cloud Security Monitoring Solution · Common Intrusion Detection!6. Actual Response Unit!7. For this talk, we will focus on the CIDF components provided by AWS and GCP](https://reader034.vdocuments.net/reader034/viewer/2022042417/5f338b15de182847ed6f4dd0/html5/thumbnails/6.jpg)
CIDF defines four categories of components:
•Event generators •Analyzers •Databases •Response units
https://tools.ietf.org/html/draft-staniford-cidf-data-formats-00
FrameworkCommon Intrusion Detection
!6
![Page 7: Make Your Own Cloud Security Monitoring Solution · Common Intrusion Detection!6. Actual Response Unit!7. For this talk, we will focus on the CIDF components provided by AWS and GCP](https://reader034.vdocuments.net/reader034/viewer/2022042417/5f338b15de182847ed6f4dd0/html5/thumbnails/7.jpg)
Actual Response Unit
!7
![Page 8: Make Your Own Cloud Security Monitoring Solution · Common Intrusion Detection!6. Actual Response Unit!7. For this talk, we will focus on the CIDF components provided by AWS and GCP](https://reader034.vdocuments.net/reader034/viewer/2022042417/5f338b15de182847ed6f4dd0/html5/thumbnails/8.jpg)
For this talk, we will focus on the CIDF components provided by AWS and GCP.
AWS and GCP
!8
![Page 9: Make Your Own Cloud Security Monitoring Solution · Common Intrusion Detection!6. Actual Response Unit!7. For this talk, we will focus on the CIDF components provided by AWS and GCP](https://reader034.vdocuments.net/reader034/viewer/2022042417/5f338b15de182847ed6f4dd0/html5/thumbnails/9.jpg)
Public cloud environments provide some CIDF components as services:
•Easy to configure •Can feed into 3rd party tools •Accessible by API Calls (EVERYTHING is an API CALL)
CIDF In Public CloudEnvironments
!9
![Page 10: Make Your Own Cloud Security Monitoring Solution · Common Intrusion Detection!6. Actual Response Unit!7. For this talk, we will focus on the CIDF components provided by AWS and GCP](https://reader034.vdocuments.net/reader034/viewer/2022042417/5f338b15de182847ed6f4dd0/html5/thumbnails/10.jpg)
AWS (CloudTrail) • Easily configurable • Logs ALL API calls to S3 or Lambda • Easily consumable by third party tools
Native Event Generators
!10
![Page 11: Make Your Own Cloud Security Monitoring Solution · Common Intrusion Detection!6. Actual Response Unit!7. For this talk, we will focus on the CIDF components provided by AWS and GCP](https://reader034.vdocuments.net/reader034/viewer/2022042417/5f338b15de182847ed6f4dd0/html5/thumbnails/11.jpg)
AWS (CloudWatch) • Easily configurable • Logs filtered API calls to anywhere • Easily consumable by third party tools
Native Event Generators
!11
![Page 12: Make Your Own Cloud Security Monitoring Solution · Common Intrusion Detection!6. Actual Response Unit!7. For this talk, we will focus on the CIDF components provided by AWS and GCP](https://reader034.vdocuments.net/reader034/viewer/2022042417/5f338b15de182847ed6f4dd0/html5/thumbnails/12.jpg)
Stackdriver (GCP) • GCP log management platform • Collects GCP event data • Includes features that facilitate data
management
Native Event Generators
!12
![Page 13: Make Your Own Cloud Security Monitoring Solution · Common Intrusion Detection!6. Actual Response Unit!7. For this talk, we will focus on the CIDF components provided by AWS and GCP](https://reader034.vdocuments.net/reader034/viewer/2022042417/5f338b15de182847ed6f4dd0/html5/thumbnails/13.jpg)
Native Analyzers
AWS and GCP both provide native analyzers
• Accessible via the GUI
• Accessible via APIs
• Data presentations may vary
considerably based on logged calls
!13
![Page 14: Make Your Own Cloud Security Monitoring Solution · Common Intrusion Detection!6. Actual Response Unit!7. For this talk, we will focus on the CIDF components provided by AWS and GCP](https://reader034.vdocuments.net/reader034/viewer/2022042417/5f338b15de182847ed6f4dd0/html5/thumbnails/14.jpg)
Create a filter!
GCP’s Native Analyzer
!14
![Page 15: Make Your Own Cloud Security Monitoring Solution · Common Intrusion Detection!6. Actual Response Unit!7. For this talk, we will focus on the CIDF components provided by AWS and GCP](https://reader034.vdocuments.net/reader034/viewer/2022042417/5f338b15de182847ed6f4dd0/html5/thumbnails/15.jpg)
Create a policy!
GCP’s Native Response Unit
!15
![Page 16: Make Your Own Cloud Security Monitoring Solution · Common Intrusion Detection!6. Actual Response Unit!7. For this talk, we will focus on the CIDF components provided by AWS and GCP](https://reader034.vdocuments.net/reader034/viewer/2022042417/5f338b15de182847ed6f4dd0/html5/thumbnails/16.jpg)
AWS’ Native Analyzer
!16
![Page 17: Make Your Own Cloud Security Monitoring Solution · Common Intrusion Detection!6. Actual Response Unit!7. For this talk, we will focus on the CIDF components provided by AWS and GCP](https://reader034.vdocuments.net/reader034/viewer/2022042417/5f338b15de182847ed6f4dd0/html5/thumbnails/17.jpg)
AWS Native Response Unit
AWS offers native support for:
•SMS through SNS •E-mail through Simple Email Service •Much more…
!17
![Page 18: Make Your Own Cloud Security Monitoring Solution · Common Intrusion Detection!6. Actual Response Unit!7. For this talk, we will focus on the CIDF components provided by AWS and GCP](https://reader034.vdocuments.net/reader034/viewer/2022042417/5f338b15de182847ed6f4dd0/html5/thumbnails/18.jpg)
Simple AWS Alerting
!18
![Page 19: Make Your Own Cloud Security Monitoring Solution · Common Intrusion Detection!6. Actual Response Unit!7. For this talk, we will focus on the CIDF components provided by AWS and GCP](https://reader034.vdocuments.net/reader034/viewer/2022042417/5f338b15de182847ed6f4dd0/html5/thumbnails/19.jpg)
AWS CIDF Summary
• Configurable
• Consumable
• Spread across multiple services
• Intended for programatic access
• Log formats have inconsistent schema
!19
![Page 20: Make Your Own Cloud Security Monitoring Solution · Common Intrusion Detection!6. Actual Response Unit!7. For this talk, we will focus on the CIDF components provided by AWS and GCP](https://reader034.vdocuments.net/reader034/viewer/2022042417/5f338b15de182847ed6f4dd0/html5/thumbnails/20.jpg)
GCP CIDF Summary
• Configurable
• Consolidated in single service
• Heavy focus on GUI
• Log formats have inconsistent schema
!20
![Page 21: Make Your Own Cloud Security Monitoring Solution · Common Intrusion Detection!6. Actual Response Unit!7. For this talk, we will focus on the CIDF components provided by AWS and GCP](https://reader034.vdocuments.net/reader034/viewer/2022042417/5f338b15de182847ed6f4dd0/html5/thumbnails/21.jpg)
Incompleteness Theorem
BEWARE!
!21
![Page 22: Make Your Own Cloud Security Monitoring Solution · Common Intrusion Detection!6. Actual Response Unit!7. For this talk, we will focus on the CIDF components provided by AWS and GCP](https://reader034.vdocuments.net/reader034/viewer/2022042417/5f338b15de182847ed6f4dd0/html5/thumbnails/22.jpg)
gcloud logging [metrics | sinks] list
gcloud logging metrics update [metric_name] —log-filter = “SOMETHING”
Information for Red Teams!
!22
![Page 23: Make Your Own Cloud Security Monitoring Solution · Common Intrusion Detection!6. Actual Response Unit!7. For this talk, we will focus on the CIDF components provided by AWS and GCP](https://reader034.vdocuments.net/reader034/viewer/2022042417/5f338b15de182847ed6f4dd0/html5/thumbnails/23.jpg)
Information for Red Teams!
!23
![Page 24: Make Your Own Cloud Security Monitoring Solution · Common Intrusion Detection!6. Actual Response Unit!7. For this talk, we will focus on the CIDF components provided by AWS and GCP](https://reader034.vdocuments.net/reader034/viewer/2022042417/5f338b15de182847ed6f4dd0/html5/thumbnails/24.jpg)
Information for Red Teams!
•aws events list-rules # for CloudWatch rules
•aws events [disable-rule | enable-rule] —name [rule name]
•aws cloudtrail describe-trails
!24
![Page 25: Make Your Own Cloud Security Monitoring Solution · Common Intrusion Detection!6. Actual Response Unit!7. For this talk, we will focus on the CIDF components provided by AWS and GCP](https://reader034.vdocuments.net/reader034/viewer/2022042417/5f338b15de182847ed6f4dd0/html5/thumbnails/25.jpg)
Information for Red Teams!
!25
![Page 26: Make Your Own Cloud Security Monitoring Solution · Common Intrusion Detection!6. Actual Response Unit!7. For this talk, we will focus on the CIDF components provided by AWS and GCP](https://reader034.vdocuments.net/reader034/viewer/2022042417/5f338b15de182847ed6f4dd0/html5/thumbnails/26.jpg)
•Transparent to attackers
•Easily clobbered by admins (including you)
•Metrics/Exports/Filters can be imprecise
Native Alerting Limitations
!26
![Page 27: Make Your Own Cloud Security Monitoring Solution · Common Intrusion Detection!6. Actual Response Unit!7. For this talk, we will focus on the CIDF components provided by AWS and GCP](https://reader034.vdocuments.net/reader034/viewer/2022042417/5f338b15de182847ed6f4dd0/html5/thumbnails/27.jpg)
What Can We Do?
!27
![Page 28: Make Your Own Cloud Security Monitoring Solution · Common Intrusion Detection!6. Actual Response Unit!7. For this talk, we will focus on the CIDF components provided by AWS and GCP](https://reader034.vdocuments.net/reader034/viewer/2022042417/5f338b15de182847ed6f4dd0/html5/thumbnails/28.jpg)
Build your own Alerting!
Advantages
• Less transparent to attackers
• Allows for more complicated filtering
• Enabled third-party technology
• Storage and retention…
!28
![Page 29: Make Your Own Cloud Security Monitoring Solution · Common Intrusion Detection!6. Actual Response Unit!7. For this talk, we will focus on the CIDF components provided by AWS and GCP](https://reader034.vdocuments.net/reader034/viewer/2022042417/5f338b15de182847ed6f4dd0/html5/thumbnails/29.jpg)
Case Study
In this scenario
• AWS is our primary cloud
• GCP is our secondary cloud
• Shipping data out of GCP to other CIDF components
• Custom shipper “GoodCoP”
!29
![Page 30: Make Your Own Cloud Security Monitoring Solution · Common Intrusion Detection!6. Actual Response Unit!7. For this talk, we will focus on the CIDF components provided by AWS and GCP](https://reader034.vdocuments.net/reader034/viewer/2022042417/5f338b15de182847ed6f4dd0/html5/thumbnails/30.jpg)
Stackdriver API
•Continuous Polling
•Lose some convenience
•Protobuf support*
*https://github.com/googleapis/googleapis/blob/master/google/cloud/audit/audit_log.proto
!30
![Page 31: Make Your Own Cloud Security Monitoring Solution · Common Intrusion Detection!6. Actual Response Unit!7. For this talk, we will focus on the CIDF components provided by AWS and GCP](https://reader034.vdocuments.net/reader034/viewer/2022042417/5f338b15de182847ed6f4dd0/html5/thumbnails/31.jpg)
Application Flow
from google.cloud.logging import Client, ASCENDING, DESCENDING
client = Client(project = projectName) while true: timeFilter = GetFilter(LastScanTime) entries = False while not entries: entries = client.list_entries(order_by=DESCENDING, filter_ = timeFilter) for entry in entries: DoSomething(entry.payload) UpdateScanTime(entry.timestamp)
!31
![Page 32: Make Your Own Cloud Security Monitoring Solution · Common Intrusion Detection!6. Actual Response Unit!7. For this talk, we will focus on the CIDF components provided by AWS and GCP](https://reader034.vdocuments.net/reader034/viewer/2022042417/5f338b15de182847ed6f4dd0/html5/thumbnails/32.jpg)
GoodCoP Configuration
!32
![Page 33: Make Your Own Cloud Security Monitoring Solution · Common Intrusion Detection!6. Actual Response Unit!7. For this talk, we will focus on the CIDF components provided by AWS and GCP](https://reader034.vdocuments.net/reader034/viewer/2022042417/5f338b15de182847ed6f4dd0/html5/thumbnails/33.jpg)
Flip the Script
What if AWS alerting data flows into GCP?
!33
![Page 34: Make Your Own Cloud Security Monitoring Solution · Common Intrusion Detection!6. Actual Response Unit!7. For this talk, we will focus on the CIDF components provided by AWS and GCP](https://reader034.vdocuments.net/reader034/viewer/2022042417/5f338b15de182847ed6f4dd0/html5/thumbnails/34.jpg)
Considerations for AWS
• CloudTrail - Dump all your API event to S3 or Lambda
• CloudWatch - Dump filtered API events to Lambda, Kinesis, SQS, SNS, or elsewhere
Consider separate accounts!!34
![Page 35: Make Your Own Cloud Security Monitoring Solution · Common Intrusion Detection!6. Actual Response Unit!7. For this talk, we will focus on the CIDF components provided by AWS and GCP](https://reader034.vdocuments.net/reader034/viewer/2022042417/5f338b15de182847ed6f4dd0/html5/thumbnails/35.jpg)
Adding Third Parties
Several third party CIDF components are available.
!35
![Page 36: Make Your Own Cloud Security Monitoring Solution · Common Intrusion Detection!6. Actual Response Unit!7. For this talk, we will focus on the CIDF components provided by AWS and GCP](https://reader034.vdocuments.net/reader034/viewer/2022042417/5f338b15de182847ed6f4dd0/html5/thumbnails/36.jpg)
Open Source Searching and Alerting
•Elasticsearch
•Elastalert
•Streamalert
!36
![Page 37: Make Your Own Cloud Security Monitoring Solution · Common Intrusion Detection!6. Actual Response Unit!7. For this talk, we will focus on the CIDF components provided by AWS and GCP](https://reader034.vdocuments.net/reader034/viewer/2022042417/5f338b15de182847ed6f4dd0/html5/thumbnails/37.jpg)
Third Party Response Units
•Slack — https://api.slack.com/
•PagerDuty — https://v2.developer.pagerduty.com/
•SMS — Several available
•SMTP Email — Several available
•Smart bulbs — https://www.developers.meethue.com/philips-hue-api
!37
![Page 38: Make Your Own Cloud Security Monitoring Solution · Common Intrusion Detection!6. Actual Response Unit!7. For this talk, we will focus on the CIDF components provided by AWS and GCP](https://reader034.vdocuments.net/reader034/viewer/2022042417/5f338b15de182847ed6f4dd0/html5/thumbnails/38.jpg)
Simple AWS Alerting
!38
![Page 39: Make Your Own Cloud Security Monitoring Solution · Common Intrusion Detection!6. Actual Response Unit!7. For this talk, we will focus on the CIDF components provided by AWS and GCP](https://reader034.vdocuments.net/reader034/viewer/2022042417/5f338b15de182847ed6f4dd0/html5/thumbnails/39.jpg)
Datadog as Datastore
!39
![Page 40: Make Your Own Cloud Security Monitoring Solution · Common Intrusion Detection!6. Actual Response Unit!7. For this talk, we will focus on the CIDF components provided by AWS and GCP](https://reader034.vdocuments.net/reader034/viewer/2022042417/5f338b15de182847ed6f4dd0/html5/thumbnails/40.jpg)
Datadog as Analyzer
!40
![Page 41: Make Your Own Cloud Security Monitoring Solution · Common Intrusion Detection!6. Actual Response Unit!7. For this talk, we will focus on the CIDF components provided by AWS and GCP](https://reader034.vdocuments.net/reader034/viewer/2022042417/5f338b15de182847ed6f4dd0/html5/thumbnails/41.jpg)
Datadog Response Units
!41
![Page 42: Make Your Own Cloud Security Monitoring Solution · Common Intrusion Detection!6. Actual Response Unit!7. For this talk, we will focus on the CIDF components provided by AWS and GCP](https://reader034.vdocuments.net/reader034/viewer/2022042417/5f338b15de182847ed6f4dd0/html5/thumbnails/42.jpg)
Go Build It!
!42
• CIDF components are out there
• These systems can be fragile
• Good luck!
![Page 43: Make Your Own Cloud Security Monitoring Solution · Common Intrusion Detection!6. Actual Response Unit!7. For this talk, we will focus on the CIDF components provided by AWS and GCP](https://reader034.vdocuments.net/reader034/viewer/2022042417/5f338b15de182847ed6f4dd0/html5/thumbnails/43.jpg)
•Twitter: @JohnAVentura
•Github: https://github.com/johnaventura/
Thank you
!43