aws april webinar series - amazon efs: scalable, shared file storage for amazon ec2

46
© 2015, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Edward Naim, Principal Product Manager April 29, 2015 Amazon EFS Webinar Introduction to Amazon Elastic File System (EFS)

Upload: amazon-web-services

Post on 18-Jul-2015

643 views

Category:

Technology


4 download

TRANSCRIPT

Page 1: AWS April Webinar Series - Amazon EFS: Scalable, Shared File Storage for Amazon EC2

© 2015, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

Edward Naim, Principal Product Manager

April 29, 2015

Amazon EFS WebinarIntroduction to Amazon Elastic File System (EFS)

Page 2: AWS April Webinar Series - Amazon EFS: Scalable, Shared File Storage for Amazon EC2

Goals and expectations for this session

Overall goal: Introduce you to Amazon EFS (what it is, features,

how it can help you)

Webinar intended for all levels: We’ll cover both beginner topics

and more advanced concepts

We’ll do Q&A at the end: Submit questions during presentation

Page 3: AWS April Webinar Series - Amazon EFS: Scalable, Shared File Storage for Amazon EC2

Agenda

1. Provide overview of EFS

2. Introduce EFS technical concepts

3. Walk through experience of creating a file system

4. Discuss file system security mechanisms

5. Explore the EFS regional availability and durability model

Page 4: AWS April Webinar Series - Amazon EFS: Scalable, Shared File Storage for Amazon EC2

Overview of Amazon EFS

Page 5: AWS April Webinar Series - Amazon EFS: Scalable, Shared File Storage for Amazon EC2

Amazon S3• Object storage: data presented as buckets of objects

• Data access via APIs over the Internet

Amazon EFS• File storage (analogous to NAS): data presented as a file system

• Shared low-latency access from multiple EC2 instances

Amazon

Elastic Block

Store

• Block storage (analogous to SAN): data presented as disk volumes

• Lowest-latency access from single Amazon EC2 instances

Amazon

Glacier

• Archival storage: data presented as vaults/archives of objects

• Lowest-cost storage, infrequent access via APIs over the Internet

The AWS storage portfolio

Page 6: AWS April Webinar Series - Amazon EFS: Scalable, Shared File Storage for Amazon EC2

Fully managed file system for EC2 instances

Provides standard file system semantics

Works with standard operating system APIs

Sharable across thousands of instances

Elastically grows to petabyte scale

Delivers performance for a wide variety of workloads

Highly available and durable

NFS v4–based

What is Amazon EFS?

Page 7: AWS April Webinar Series - Amazon EFS: Scalable, Shared File Storage for Amazon EC2

EFS is designed for a broad range of use

cases, such as…

Content repositories

Development environments

Home directories

Big data

Page 8: AWS April Webinar Series - Amazon EFS: Scalable, Shared File Storage for Amazon EC2

Operating shared file storage today is a pain

Application owner

or developer

IT administrator

Business owner

• Estimate demand

• Procure hardware

• Set aside physical space

• Set up and maintain hardware (and network)

• Manage access and security

• Provide demand forecasts/business case

• Add lead times and extra coordination to your schedule

• Limit your flexibility and agility

• Make up-front capital investments, over-buy, stay on a

constant upgrade/refresh cycle

• Sacrifice business agility

• Distract your people from your business’s mission

Page 9: AWS April Webinar Series - Amazon EFS: Scalable, Shared File Storage for Amazon EC2

We focused on changing the game

EFS is

simpleEFS is

elasticEFS is

scalable

1 2 3

Page 10: AWS April Webinar Series - Amazon EFS: Scalable, Shared File Storage for Amazon EC2

EFS is simple

Fully managed

- No hardware, network, file layer

- Create a scalable file system in seconds!

Seamless integration with existing tools and apps

- NFS v4—widespread, open

- Standard file system semantics

- Works with standard OS file system APIs

Simple pricing = simple forecasting

1

Page 11: AWS April Webinar Series - Amazon EFS: Scalable, Shared File Storage for Amazon EC2

EFS is elastic

File systems grow and shrink automatically

as you add and remove files

No need to provision storage capacity or

performance

You pay only for the storage space you use,

with no minimum fee

2

Page 12: AWS April Webinar Series - Amazon EFS: Scalable, Shared File Storage for Amazon EC2

File systems can grow to petabyte scale

Throughput and IOPS scale automatically

as file systems grow

Consistent low latencies regardless of file

system size

Support for thousands of concurrent NFS

connections

EFS is scalable3

Page 13: AWS April Webinar Series - Amazon EFS: Scalable, Shared File Storage for Amazon EC2

Why does this matter?...

… to app owners

and developers?

… to your

business?

• Easy to move existing code, applications, and tools

used today with existing NFS servers to the AWS cloud

• Simple shared file storage solution for new cloud-native

applications

• Predictable pricing with no up-front investment

• Increased agility

• Spend less time managing file storage and more

time focusing on your business

… to IT

administrators?

• Eliminates need to manage and maintain file system

storage at scale

Page 14: AWS April Webinar Series - Amazon EFS: Scalable, Shared File Storage for Amazon EC2

Diving In

Page 15: AWS April Webinar Series - Amazon EFS: Scalable, Shared File Storage for Amazon EC2

Some key AWS concepts to understand

Region

Availability Zone (AZ)

Amazon Virtual Private Cloud (VPC)

Page 16: AWS April Webinar Series - Amazon EFS: Scalable, Shared File Storage for Amazon EC2

Region

Geographic area where AWS services are available

Customers choose region(s) for their AWS resources

Eleven regions worldwide

REGION

Page 17: AWS April Webinar Series - Amazon EFS: Scalable, Shared File Storage for Amazon EC2

Availability Zone (AZ)

Each region has multiple,

isolated locations known as

Availability Zones

Low-latency links between

AZs in a region

When launching an EC2

instance, a customer chooses

an AZ

AVAILABILITY ZONE 3

EC2

AVAILABILITY ZONE 2

AVAILABILITY ZONE 1

EC2EC2

EC2

REGION

Page 18: AWS April Webinar Series - Amazon EFS: Scalable, Shared File Storage for Amazon EC2

Virtual Private Cloud (VPC)

Logically isolated section of the AWS cloud, virtual network defined by the customer

When launching instances and other resources, customers place them in a VPC

All new customers have a default VPC

AVAILABILITY ZONE 1

REGION

AVAILABILITY ZONE 2

AVAILABILITY ZONE 3

VPC

EC2EC2

EC2

EC2

Page 19: AWS April Webinar Series - Amazon EFS: Scalable, Shared File Storage for Amazon EC2

What is a file system?

The primary resource in EFS

Where you store files and directories

Can create unlimited file systems per account

Page 20: AWS April Webinar Series - Amazon EFS: Scalable, Shared File Storage for Amazon EC2

How to access a file system from an instance

You “mount” a file system on an EC2 instance (standard command) — the file system will appear like a local set of directories and files

An NFS v4 client is standard on Linux distributions

mount –t nfs4

[file system DNS name]:/

/[user’s target directory]

Page 21: AWS April Webinar Series - Amazon EFS: Scalable, Shared File Storage for Amazon EC2

What is a mount target?

To access your file system from instances in a VPC, you create mount targets in the VPC

A mount target is an NFSv4 endpoint in your VPC

A mount target has an IP address and a DNS name you use in your mount command

AVAILABILITY ZONE 1

REGION

AVAILABILITY ZONE 2

AVAILABILITY ZONE 3

VPC

EC2EC2

EC2

EC2

Mount

target

Page 22: AWS April Webinar Series - Amazon EFS: Scalable, Shared File Storage for Amazon EC2

How does it all fit together?

AVAILABILITY ZONE 1

REGION

AVAILABILITY ZONE 2

AVAILABILITY ZONE 3

VPC

EC2EC2

EC2

EC2

Customer’s file

system

Page 23: AWS April Webinar Series - Amazon EFS: Scalable, Shared File Storage for Amazon EC2

There are three ways to set up and

manage a file system

AWS Management Console

AWS Command Line Interface (CLI)

AWS Software Development Kit (SDK)

Page 24: AWS April Webinar Series - Amazon EFS: Scalable, Shared File Storage for Amazon EC2

The AWS Management Console, CLI, and SDK each

allow you to perform a variety of management tasks

Create a file system

Create and manage mount targets

Tag a file system

Delete a file system

View details on file systems in your AWS account

Page 25: AWS April Webinar Series - Amazon EFS: Scalable, Shared File Storage for Amazon EC2

Setting up and mounting a file system takes

under a minute

1. Create a file system

2. Create a mount target in each AZ from which you want

to access the file system

3. Enable the NFS client on your instances

4. Run the mount command

Page 26: AWS April Webinar Series - Amazon EFS: Scalable, Shared File Storage for Amazon EC2
Page 27: AWS April Webinar Series - Amazon EFS: Scalable, Shared File Storage for Amazon EC2
Page 28: AWS April Webinar Series - Amazon EFS: Scalable, Shared File Storage for Amazon EC2
Page 29: AWS April Webinar Series - Amazon EFS: Scalable, Shared File Storage for Amazon EC2
Page 30: AWS April Webinar Series - Amazon EFS: Scalable, Shared File Storage for Amazon EC2
Page 31: AWS April Webinar Series - Amazon EFS: Scalable, Shared File Storage for Amazon EC2
Page 32: AWS April Webinar Series - Amazon EFS: Scalable, Shared File Storage for Amazon EC2
Page 33: AWS April Webinar Series - Amazon EFS: Scalable, Shared File Storage for Amazon EC2

Securing Your File System

Page 34: AWS April Webinar Series - Amazon EFS: Scalable, Shared File Storage for Amazon EC2

Several security mechanisms

Control network traffic to and from file systems (mount

targets) by using VPC security groups and network ACLs

Control file and directory access by using standard

Linux/Windows directory-/file-level permissions

Control administrative access (API access) to file systems

by using AWS Identity and Access Management (IAM)

Page 35: AWS April Webinar Series - Amazon EFS: Scalable, Shared File Storage for Amazon EC2

Only EC2 instances in the VPC you specify can access

your EFS file system

VPC

EC2EC2

EC2

EC2

VPC

EC2EC2

EC2

EC2

Customer’s file

system

Page 36: AWS April Webinar Series - Amazon EFS: Scalable, Shared File Storage for Amazon EC2

VPC

EC2

EC2

Security groups control which instances in your VPC

can connect to your mount targets

Customer’s file

system

Security group:

sg-allowed

Security group:

Permit inbound traffic

from “sg-allowed”

Security group:

sg-not-allowed

Page 37: AWS April Webinar Series - Amazon EFS: Scalable, Shared File Storage for Amazon EC2

EFS supports user-level file and directory

access permissions

Set file/directory permissions to specify read-write-execute

permissions for users and groups

Page 38: AWS April Webinar Series - Amazon EFS: Scalable, Shared File Storage for Amazon EC2

Use IAM policies to control who can use the

administrative APIs to create, manage, and

delete file systems

EFS supports action-level and resource-level

permissions

Integration with IAM provides administrative

security

Page 39: AWS April Webinar Series - Amazon EFS: Scalable, Shared File Storage for Amazon EC2

Regional Availability

and Durability

Page 40: AWS April Webinar Series - Amazon EFS: Scalable, Shared File Storage for Amazon EC2

In what regions can I use EFS?

US-West (Oregon)

US-East (Northern Virginia)

EU (Ireland)

Page 41: AWS April Webinar Series - Amazon EFS: Scalable, Shared File Storage for Amazon EC2

Data is stored in multiple AZs for high availability

and durability

Every file

system object

(directory, file,

and link) is

redundantly

stored across

multiple AZs in

a region

AVAILABILITY

ZONE 1

REGION

AVAILABILITY

ZONE 2

AVAILABILITY

ZONE 3

Amazon

EFS

Page 42: AWS April Webinar Series - Amazon EFS: Scalable, Shared File Storage for Amazon EC2

Data can be accessed from any AZ in the region

while maintaining full consistency

Your EC2 instances can connect to your EFS file system from any AZ in a region

All reads will be fully

consistent in all AZs—that

is, a read in one AZ is

guaranteed to have the

latest data, even if the data

is being written in another

AZ

AVAILABILITY

ZONE 1

REGIONVPC

EC2EC2

EC2

AVAILABILITY

ZONE 2

AVAILABILITY

ZONE 3

EC2

Write

Read

Page 43: AWS April Webinar Series - Amazon EFS: Scalable, Shared File Storage for Amazon EC2

Wrapping Up

Page 44: AWS April Webinar Series - Amazon EFS: Scalable, Shared File Storage for Amazon EC2

Simple and predictable pricing

With EFS, you pay only for the storage space you use

• No minimum commitments or up-front fees

• No need to provision storage in advance

• No other fees, charges, or billing dimensions

EFS price: $0.30/GB-month

Page 45: AWS April Webinar Series - Amazon EFS: Scalable, Shared File Storage for Amazon EC2

What to do next?

Learn more at aws.amazon.com/efs

Request an invite for our Preview

Page 46: AWS April Webinar Series - Amazon EFS: Scalable, Shared File Storage for Amazon EC2

Poll and Q&A next

Thank you!