amazon elastic file system (amazon efs) introduction & demo
TRANSCRIPT
![Page 1: Amazon Elastic File System (Amazon EFS) Introduction & Demo](https://reader031.vdocuments.net/reader031/viewer/2022021812/589daf561a28ab21728b6113/html5/thumbnails/1.jpg)
Amazon Elastic File System
(Amazon EFS)
Introduction & Demo
Danilo Poccia – AWS Technical Evangelist
@danilop danilop
![Page 2: Amazon Elastic File System (Amazon EFS) Introduction & Demo](https://reader031.vdocuments.net/reader031/viewer/2022021812/589daf561a28ab21728b6113/html5/thumbnails/2.jpg)
Agenda
1. Provide an overview of Amazon EFS
2. Introduce Amazon EFS technical concepts
3. Conduct a walkthrough of creating a file system
4. Discuss file system security mechanisms
5. Explore the Amazon EFS regional availability and
durability model
6. More advances use cases
![Page 3: Amazon Elastic File System (Amazon EFS) Introduction & Demo](https://reader031.vdocuments.net/reader031/viewer/2022021812/589daf561a28ab21728b6113/html5/thumbnails/3.jpg)
Overview of Amazon EFS
![Page 4: Amazon Elastic File System (Amazon EFS) Introduction & Demo](https://reader031.vdocuments.net/reader031/viewer/2022021812/589daf561a28ab21728b6113/html5/thumbnails/4.jpg)
The AWS storage portfolio
Amazon S3• Object storage: Data presented as buckets of objects
• Data access via APIs over the Internet
Amazon
EFS• File storage (analogous to NAS): Data presented as a file system
• Shared low-latency access from multiple EC2 instances
Amazon
Elastic Block
Store
• Block storage (analogous to SAN): Data presented as disk volumes
• Lowest-latency access from single Amazon EC2 instances
Amazon
Glacier• Archival storage: Data presented as vaults/archives of objects
• Lowest-cost storage, infrequent access via APIs over the Internet
![Page 5: Amazon Elastic File System (Amazon EFS) Introduction & Demo](https://reader031.vdocuments.net/reader031/viewer/2022021812/589daf561a28ab21728b6113/html5/thumbnails/5.jpg)
What is Amazon EFS?
• Fully managed file system for EC2 instances
• Provides standard file system semantics
• Works with standard operating system APIs
• Sharable across thousands of instances
• Elastically grows to petabyte scale
• Delivers performance for a wide variety of workloads
• Highly available and durable
• NFS v4–based
![Page 6: Amazon Elastic File System (Amazon EFS) Introduction & Demo](https://reader031.vdocuments.net/reader031/viewer/2022021812/589daf561a28ab21728b6113/html5/thumbnails/6.jpg)
Amazon EFS is designed for a broad range of
use cases, such as:
• Content repositories
• Development environments
• Home directories
• Big data
![Page 7: Amazon Elastic File System (Amazon EFS) Introduction & Demo](https://reader031.vdocuments.net/reader031/viewer/2022021812/589daf561a28ab21728b6113/html5/thumbnails/7.jpg)
Operating shared file storage today is a pain
Application owner
or developer
IT administrator
Business owner
• Estimate demand
• Procure hardware
• Set aside physical space
• Set up and maintain hardware (and network)
• Manage access and security
• Provide demand forecasts/business case
• Add lead times and extra coordination to your schedule
• Limit your flexibility and agility
• Make up-front capital investments, over buy, stay on a
constant upgrade/refresh cycle
• Sacrifice business agility
• Distract your people from your business’s mission
![Page 8: Amazon Elastic File System (Amazon EFS) Introduction & Demo](https://reader031.vdocuments.net/reader031/viewer/2022021812/589daf561a28ab21728b6113/html5/thumbnails/8.jpg)
We focused on changing the game
Amazon EFS
is simple
Amazon EFS
is elastic
Amazon EFS
is scalable
1 2 3
![Page 9: Amazon Elastic File System (Amazon EFS) Introduction & Demo](https://reader031.vdocuments.net/reader031/viewer/2022021812/589daf561a28ab21728b6113/html5/thumbnails/9.jpg)
Amazon EFS is simple
• Fully managed
– No hardware, network, file layer
– Create a scalable file system in seconds!
• Seamless integration with existing
tools and apps
– NFS v4—widespread, open
– Standard file system semantics
– Works with standard OS file system APIs
• Simple pricing = simple forecasting
1
![Page 10: Amazon Elastic File System (Amazon EFS) Introduction & Demo](https://reader031.vdocuments.net/reader031/viewer/2022021812/589daf561a28ab21728b6113/html5/thumbnails/10.jpg)
Amazon EFS is elastic
• File systems grow and shrink
automatically as you add and remove
files
• No need to provision storage capacity
or performance
• You pay only for the storage space you
use, with no minimum fee
2
![Page 11: Amazon Elastic File System (Amazon EFS) Introduction & Demo](https://reader031.vdocuments.net/reader031/viewer/2022021812/589daf561a28ab21728b6113/html5/thumbnails/11.jpg)
• File systems can grow to petabyte
scale
• Throughput and IOPS scale
automatically as file systems grow
• Consistent low latencies regardless
of file system size
• Support for thousands of concurrent
NFS connections
Amazon EFS is scalable3
![Page 12: Amazon Elastic File System (Amazon EFS) Introduction & Demo](https://reader031.vdocuments.net/reader031/viewer/2022021812/589daf561a28ab21728b6113/html5/thumbnails/12.jpg)
Why does this matter…
… to app owners
and developers?
… to your
business?
• Easy to move existing code, applications, and tools
used today with existing NFS servers to the AWS cloud
• Simple shared file storage solution for new cloud-native
applications
• Predictable pricing with no up-front investment
• Increased agility
• Spend less time managing file storage and more
time focusing on your business
… to IT
administrators?
• Eliminates need to manage and maintain file system
storage at scale
![Page 13: Amazon Elastic File System (Amazon EFS) Introduction & Demo](https://reader031.vdocuments.net/reader031/viewer/2022021812/589daf561a28ab21728b6113/html5/thumbnails/13.jpg)
Diving In
![Page 14: Amazon Elastic File System (Amazon EFS) Introduction & Demo](https://reader031.vdocuments.net/reader031/viewer/2022021812/589daf561a28ab21728b6113/html5/thumbnails/14.jpg)
Some key AWS concepts to understand
• Region
• Availability Zone (AZ)
• Amazon Virtual Private Cloud (VPC)
![Page 15: Amazon Elastic File System (Amazon EFS) Introduction & Demo](https://reader031.vdocuments.net/reader031/viewer/2022021812/589daf561a28ab21728b6113/html5/thumbnails/15.jpg)
Region
• Geographic area where
AWS services are available
• Customers choose
region(s) for their AWS
resources
• 11 regions worldwide
REGION
![Page 16: Amazon Elastic File System (Amazon EFS) Introduction & Demo](https://reader031.vdocuments.net/reader031/viewer/2022021812/589daf561a28ab21728b6113/html5/thumbnails/16.jpg)
Availability Zone (AZ)
• Each region has multiple,
isolated locations known as
Availability Zones
• Low-latency links between
AZs in a region
• When launching an EC2
instance, a customer
chooses an AZ AVAILABILITY ZONE 3
EC2
AVAILABILITY ZONE 2
AVAILABILITY ZONE 1
EC2EC2
EC2
REGION
![Page 17: Amazon Elastic File System (Amazon EFS) Introduction & Demo](https://reader031.vdocuments.net/reader031/viewer/2022021812/589daf561a28ab21728b6113/html5/thumbnails/17.jpg)
Amazon VPC
• Logically isolated section
of the AWS cloud, virtual
network defined by the
customer
• When launching instances
and other resources,
customers place them in a
VPC
• All new customers have a
default VPC
AVAILABILITY ZONE 1
REGION
AVAILABILITY ZONE 2
AVAILABILITY ZONE 3
VPC
EC2EC2
EC2
EC2
![Page 18: Amazon Elastic File System (Amazon EFS) Introduction & Demo](https://reader031.vdocuments.net/reader031/viewer/2022021812/589daf561a28ab21728b6113/html5/thumbnails/18.jpg)
What is a file system?
• The primary resource in Amazon EFS
• Where you store files and directories
![Page 19: Amazon Elastic File System (Amazon EFS) Introduction & Demo](https://reader031.vdocuments.net/reader031/viewer/2022021812/589daf561a28ab21728b6113/html5/thumbnails/19.jpg)
How to access a file system from an instance
• You “mount” a file system on an EC2 instance
(standard command); the file system appears like a
local set of directories and files
• An NFS v4 client is standard on Linux distributions
mount –t nfs4
[file system DNS name]:/
/[user’s target directory]
![Page 20: Amazon Elastic File System (Amazon EFS) Introduction & Demo](https://reader031.vdocuments.net/reader031/viewer/2022021812/589daf561a28ab21728b6113/html5/thumbnails/20.jpg)
What is a mount target?
• To access your file system from instances in a VPC, you create mount targets in the VPC
• A mount target is an NFS v4 endpoint in your VPC
• A mount target has an IP address and a DNS name you use in your mount command
AVAILABILITY ZONE 1
REGION
AVAILABILITY ZONE 2
AVAILABILITY ZONE 3
VPC
EC2EC2
EC2
EC2
Mount
target
![Page 21: Amazon Elastic File System (Amazon EFS) Introduction & Demo](https://reader031.vdocuments.net/reader031/viewer/2022021812/589daf561a28ab21728b6113/html5/thumbnails/21.jpg)
How does it all fit together?
AVAILABILITY ZONE 1
REGION
AVAILABILITY ZONE 2
AVAILABILITY ZONE 3
VPC
EC2EC2
EC2
EC2
Customer’s file
system
![Page 22: Amazon Elastic File System (Amazon EFS) Introduction & Demo](https://reader031.vdocuments.net/reader031/viewer/2022021812/589daf561a28ab21728b6113/html5/thumbnails/22.jpg)
There are three ways to set up and
manage a file system
• AWS Management Console
• AWS Command Line Interface (CLI)
• AWS Software Development Kit (SDK)
![Page 23: Amazon Elastic File System (Amazon EFS) Introduction & Demo](https://reader031.vdocuments.net/reader031/viewer/2022021812/589daf561a28ab21728b6113/html5/thumbnails/23.jpg)
The AWS Management Console, CLI, and SDK each
allow you to perform a variety of management tasks
• Create a file system
• Create and manage mount targets
• Tag a file system
• Delete a file system
• View details on file systems in your AWS account
![Page 24: Amazon Elastic File System (Amazon EFS) Introduction & Demo](https://reader031.vdocuments.net/reader031/viewer/2022021812/589daf561a28ab21728b6113/html5/thumbnails/24.jpg)
Setting up and mounting a file system takes
less than a minute
1. Create a file system
2. Create a mount target in each AZ from which
you want to access the file system
3. Enable the NFS client on your instances
4. Run the mount command
![Page 25: Amazon Elastic File System (Amazon EFS) Introduction & Demo](https://reader031.vdocuments.net/reader031/viewer/2022021812/589daf561a28ab21728b6113/html5/thumbnails/25.jpg)
Demo – Part 1
![Page 26: Amazon Elastic File System (Amazon EFS) Introduction & Demo](https://reader031.vdocuments.net/reader031/viewer/2022021812/589daf561a28ab21728b6113/html5/thumbnails/26.jpg)
Securing Your File System
![Page 27: Amazon Elastic File System (Amazon EFS) Introduction & Demo](https://reader031.vdocuments.net/reader031/viewer/2022021812/589daf561a28ab21728b6113/html5/thumbnails/27.jpg)
Several security mechanisms
• Control network traffic to and from file systems (mount
targets) by using VPC security groups and network ACLs
• Control file and directory access by using standard
Linux/Windows directory-level/file-level permissions
• Control administrative access (API access) to file systems
by using AWS Identity and Access Management (IAM)
![Page 28: Amazon Elastic File System (Amazon EFS) Introduction & Demo](https://reader031.vdocuments.net/reader031/viewer/2022021812/589daf561a28ab21728b6113/html5/thumbnails/28.jpg)
Only EC2 instances in the VPC you specify can access
your Amazon EFS file system
Customer’s file
system
VPC
EC2EC2
EC2
EC2
VPC
EC2EC2
EC2
EC2
![Page 29: Amazon Elastic File System (Amazon EFS) Introduction & Demo](https://reader031.vdocuments.net/reader031/viewer/2022021812/589daf561a28ab21728b6113/html5/thumbnails/29.jpg)
VPC
EC2
EC2
Security groups control which instances in your VPC
can connect to your mount targets
Customer’s file
system
Security group:
sg-allowed
Security group:
Permit inbound traffic
from “sg-allowed”
Security group:
sg-not-allowed
![Page 30: Amazon Elastic File System (Amazon EFS) Introduction & Demo](https://reader031.vdocuments.net/reader031/viewer/2022021812/589daf561a28ab21728b6113/html5/thumbnails/30.jpg)
Amazon EFS supports user-level file and directory
access permissions
• Set file/directory permissions to specify read-write-
execute permissions for users and groups
![Page 31: Amazon Elastic File System (Amazon EFS) Introduction & Demo](https://reader031.vdocuments.net/reader031/viewer/2022021812/589daf561a28ab21728b6113/html5/thumbnails/31.jpg)
Integration with IAM provides administrative security
• Use IAM policies to control who can use
the administrative APIs to create, manage,
and delete file systems
• Amazon EFS supports action-level and
resource-level permissions
![Page 32: Amazon Elastic File System (Amazon EFS) Introduction & Demo](https://reader031.vdocuments.net/reader031/viewer/2022021812/589daf561a28ab21728b6113/html5/thumbnails/32.jpg)
Regional Availability and Durability
![Page 33: Amazon Elastic File System (Amazon EFS) Introduction & Demo](https://reader031.vdocuments.net/reader031/viewer/2022021812/589daf561a28ab21728b6113/html5/thumbnails/33.jpg)
In what regions can I use Amazon EFS?
• US-West (Oregon) *Preview
• US-East (Northern Virginia)
• EU (Ireland)
![Page 34: Amazon Elastic File System (Amazon EFS) Introduction & Demo](https://reader031.vdocuments.net/reader031/viewer/2022021812/589daf561a28ab21728b6113/html5/thumbnails/34.jpg)
Data is stored in multiple AZs for high availability
and durability
• Every file
system object
(directory, file,
and link) is
redundantly
stored across
multiple AZs in
a region
AVAILABILITY
ZONE 1
REGION
AVAILABILITY
ZONE 2
AVAILABILITY
ZONE 3
Amazon
EFS
![Page 35: Amazon Elastic File System (Amazon EFS) Introduction & Demo](https://reader031.vdocuments.net/reader031/viewer/2022021812/589daf561a28ab21728b6113/html5/thumbnails/35.jpg)
Data can be accessed from any AZ in the region
while maintaining full consistency
• Your EC2 instances can connect to your Amazon EFS file system from any AZ in a region
• All reads and writes will be fully consistent in all AZs; that is, a read in one AZ is guaranteed to have the latest data, even if the data is being written in another AZ
AVAILABILITY
ZONE 1
REGIONVPC
EC2EC2
EC2
AVAILABILITY
ZONE 2
AVAILABILITY
ZONE 3
EC2
Write
Read
![Page 36: Amazon Elastic File System (Amazon EFS) Introduction & Demo](https://reader031.vdocuments.net/reader031/viewer/2022021812/589daf561a28ab21728b6113/html5/thumbnails/36.jpg)
Demo – Part 2
![Page 37: Amazon Elastic File System (Amazon EFS) Introduction & Demo](https://reader031.vdocuments.net/reader031/viewer/2022021812/589daf561a28ab21728b6113/html5/thumbnails/37.jpg)
Use Cases
![Page 38: Amazon Elastic File System (Amazon EFS) Introduction & Demo](https://reader031.vdocuments.net/reader031/viewer/2022021812/589daf561a28ab21728b6113/html5/thumbnails/38.jpg)
Producer
Consumer
![Page 39: Amazon Elastic File System (Amazon EFS) Introduction & Demo](https://reader031.vdocuments.net/reader031/viewer/2022021812/589daf561a28ab21728b6113/html5/thumbnails/39.jpg)
Producer
Producer
Producer
ConsumerConsumer
Consumer
File
System
![Page 40: Amazon Elastic File System (Amazon EFS) Introduction & Demo](https://reader031.vdocuments.net/reader031/viewer/2022021812/589daf561a28ab21728b6113/html5/thumbnails/40.jpg)
/efs/input
/efs/work
/efs/output
![Page 41: Amazon Elastic File System (Amazon EFS) Introduction & Demo](https://reader031.vdocuments.net/reader031/viewer/2022021812/589daf561a28ab21728b6113/html5/thumbnails/41.jpg)
/efs/input
/efs/work/work-<n>
/efs/output
![Page 42: Amazon Elastic File System (Amazon EFS) Introduction & Demo](https://reader031.vdocuments.net/reader031/viewer/2022021812/589daf561a28ab21728b6113/html5/thumbnails/42.jpg)
Amazon SQS
Fast, reliable, scalable, fully managed message
queuing service
Amazon Kinesis
Real-time data processing over large,
distributed data streams
![Page 43: Amazon Elastic File System (Amazon EFS) Introduction & Demo](https://reader031.vdocuments.net/reader031/viewer/2022021812/589daf561a28ab21728b6113/html5/thumbnails/43.jpg)
Shared Website
![Page 44: Amazon Elastic File System (Amazon EFS) Introduction & Demo](https://reader031.vdocuments.net/reader031/viewer/2022021812/589daf561a28ab21728b6113/html5/thumbnails/44.jpg)
Web Server Web ServerWeb Server
Load
Balancer
File
System
![Page 45: Amazon Elastic File System (Amazon EFS) Introduction & Demo](https://reader031.vdocuments.net/reader031/viewer/2022021812/589daf561a28ab21728b6113/html5/thumbnails/45.jpg)
/efs/web/prod/
/efs/web/new/
/efs/web/old/
/efs/web/prod/
![Page 46: Amazon Elastic File System (Amazon EFS) Introduction & Demo](https://reader031.vdocuments.net/reader031/viewer/2022021812/589daf561a28ab21728b6113/html5/thumbnails/46.jpg)
AWS CodeDeploy
Automates code deployments and helps you
avoid downtime
AWS CodePipeline
Continuous Delivery service for fast and reliable
application updates
![Page 47: Amazon Elastic File System (Amazon EFS) Introduction & Demo](https://reader031.vdocuments.net/reader031/viewer/2022021812/589daf561a28ab21728b6113/html5/thumbnails/47.jpg)
Wrapping Up
![Page 48: Amazon Elastic File System (Amazon EFS) Introduction & Demo](https://reader031.vdocuments.net/reader031/viewer/2022021812/589daf561a28ab21728b6113/html5/thumbnails/48.jpg)
Simple and predictable pricing
• With Amazon EFS, you pay only for the storage space you use
– No minimum commitments or up-front fees
– No need to provision storage in advance
– No other fees, charges, or billing dimensions
• Amazon EFS price: $0.30/GB-month
![Page 49: Amazon Elastic File System (Amazon EFS) Introduction & Demo](https://reader031.vdocuments.net/reader031/viewer/2022021812/589daf561a28ab21728b6113/html5/thumbnails/49.jpg)
What to do next?
• Learn more at aws.amazon.com/efs
• Request an invite for our preview
Timothy Harder
![Page 50: Amazon Elastic File System (Amazon EFS) Introduction & Demo](https://reader031.vdocuments.net/reader031/viewer/2022021812/589daf561a28ab21728b6113/html5/thumbnails/50.jpg)
aws.amazon.com/blogs/aws
aws.amazon.com/new
@AWScloud @AWS_UKI
![Page 51: Amazon Elastic File System (Amazon EFS) Introduction & Demo](https://reader031.vdocuments.net/reader031/viewer/2022021812/589daf561a28ab21728b6113/html5/thumbnails/51.jpg)
Thank You
Danilo Poccia – AWS Technical Evangelist
@danilop danilop