instant access. infinite data. one solution.mercury.pr.erau.edu/.../solve-technical-overview.pdf ·...

NCAR - Computational Information Systems Laboratory Seminar Series 1

Atrato SOLVE - Scalable Offload Logical Volume Engine Dr. Sam Siewert

NCAR - Computational Information Systems Laboratory Seminar Series

Atrato® Velocity Series

SAID - Data Enclosure Highly parallel access – high speed Extremely dense – high capacity Architecture scales with new technologies Self-maintaining, low power

Data Access Controller Highly redundant/available configurations Seamless integration into existing data centers Host for ApplicationSmart

Atrato Virtualization Software (AVS) New Levels of Storage Virtualization Application intelligence and autonomics Industry’s first automatic tiering application aware

2


Atrato Integration Options NAS (Network Attached Storage) SAN (Storage Area Network)

Servers

SAN

Storage

NAS File Level Commands (NFS, CIFS) Block Level

Commands

Business Problems We Solve Data / Content Access – slow Storage Access Cost – too high Content search – too slow

Security/Encryption - unsecure Scalability – expensive Maintenance & Power – too costly

NAS Clients

3

NAS Servers

Block Level Commands

SAN

Storage


Extreme Capacity, Performance Density

• Highest Disk Density Per Rack

• 8x Acceleration of Tier 0 and Tier 1

• Up to 100x Lower Latency

• Optional Tiering Down for Capacity

Lowest TCO – Most Green

• 50%-80% Less Power & Cooling

• 75% Less Data Center Space

• No Overbuying

• Reduced Maintenance Costs

Atrato Delivers Data Center Value

Simplified, Intelligent Management

•Self-Healing

Self-Maintaining

Self-Optimizing

•Automatic Tiering

•Aligns Resources to Business Needs

•Frees IT staff


Atrato Storage Markets Digital Media High Performance

IPTV / VOD

Video Surveillance Analytics

Enterprise Database / Microsoft Exchange®

Oil & Gas Exploration

Financial Analysis “Monte Carlo”

Digital Studio Pre / Post Production

Social Networks Web 2.0


ApplicationSmart Tiered Management Integration with NAS FusionIO


Storage Cache Problem: Managed by OS, tracks data access to prioritize order its cache directory

ApplicationSmart replaces the need for Cache: ApplicationSmart manages cacheable data in real-time

New Data Center Storage ApplicationSmart Self-Optimization

7

Data Storage Random Data Access Atrato V1000 Profile Monitors,

Access Change Detection and Adaptation, Multi-modal

SSD

SAID

Tier 0 Replication (Intelligent Block Mgr)

If Data is Completely Random

Too Small – GB, need TB • Systems running cache • Static data only • Limited memory

Traditional Storage Arrays

Imprecise, Not Adaptive • No random access • Expensive • Cache misses = slow access

Linear Cache

Missed Penalty, Too High • Over provisioning • Increase rack space • Added power & cooling costs

Add More Storage Spindles

Identify Cacheability by Initiator and Application, Compute Speed-up

Terascale

Petascale


Self-Optimizing Intelligence

• Adaptive histogram, highly compressed, scales to petabytes • Accelerates IO for high access content

• Tuned for RAID access (FIFO with back-end IO reforming) • Lower latency, higher throughput, higher access

Ingest Accelerator

• Dynamic block migration with access pattern change • Mix profiling: block level and file level, very precise

TME (Tiered Management Engine)

• Detector for sequential/random initiator streams • Adjustment for read-ahead cache

Egress Accelerator

• Full AVS VLUN creation and management • SSD storage pool, data lifetime protection options

SLM (SSD LUN Manager)

Access Profiler

ApplicationSmart

8

AVS included features

Upgrade features


Spectrum of Workloads and ApplicationSmart Acceleration

Sequential Hot-Spots Random

Ingest IO reforming

egress IO read-ahead

Fully Predictable (SLC/RAM FIFOs)

Semi-Predictable (Scalable MLC Flash)

Non-Cacheable (Solved by SAID spindle density)


How Is ApplicationSmart Different?

Histogram Analysis Identifies access hot-spots Notes when access changes

are statistically significant Mapping integrates with

virtualization engine

Histogram Groupings Accelerates IO for high

access content Replicates blocks when

statistically significant Virtualization allows for

real-time updates End user customizable

10

Actual VoD Application with Atrato ApplicationSmart Cache Data Profiler


JBOF 2TB, 1RU

LSI MegaRAID 8888

Mellanox 2 port, 20G

AVS

SAN-Scalable JBOF (Petascale) • Infiniband or 10G FCoE/iSCSI Scaling – 20 Gbps Minimum • 2TB Per Unit, Scaled on SAN Rather Than Controller • Infinite Linear Tier-0 Scaling with V1000’s

JBOF 2TB, 1RU

LSI MegaRAID 8888

Mellanox 2 port, 20G

AVS SAN


Ingest FIFO IO Reforming

AVE

2115K Single IO

Single IO Completion

RAID0 SATA ROC

Tier0 Array

SAID

2115K Single IO

Single IO Completion

16x128K Threaded IOs

16x128K Threaded IOs

33rd Single 67K IO


V1000 Random & Sequetial IO Performance Measured (50TB SAID)

13

01002003004005006007008009001000110012001300140015001600170018001900200021002200

0100020003000400050006000700080009000

10000110001200013000140001500016000170001800019000200002100022000

4 8 16 32 64 128

256

512

1024

256

1870

MBy

tes/

sec

IOs/

sec

IO Transfer Size (KBytes)

DL580 G5 Controller, 1SAID with 320GB, 7200RPM HDDs, AVS2.0, R10, aio-stress, 11/13/08

Sum of IOPS - rnd - read Sum of IOPS - rnd - wrt Sum of IOPS - seq - read Sum of IOPS - seq - wrt

Sum of MB/s - rnd - read Sum of MB/s - rnd - wrt Sum of MB/s - seq - read Sum of MB/s - seq - wrt


Scaled-out 10TB SSD + 20TB HDD Test Configuration

• 21U Total • 5 Controllers • 5 25 SSD JBOFs • 10 SSDs/Controller • 1 SAID 320 • 20 10GE Ports

14


Why this is Interesting to HPC

• Scalable Meta-Data Solid-State and/or Hybrid VLUNs • TSM for Deep Archive • Oracle RAC Linux Index/Log Meta-data • IBRIX Meta-data Cluster, GPFS, Lustre MDS

• Scalable HDD and/or Hybrid VLUNs • DB Store • Lustre OSTs • IBRIX, GPFS, Exanet Volumes

• SAN Scalable SSD and HDD • FC, Infiniband (SRP), 10GE iSCSI TOE • Solid-State and HDD Storage Pools with Unified VLUN Management • All SSD, Hybrid SSD+HDD, All HDD Co-Managed • Linear Infinite Scaling Through SAN and NAS Volume Managers and Servers

• Can Surface a VLUN Locally on a Controller Node for Integrated NAS/SAN • OEM Integration Only • NAS Partner Integration for Fully Featured NAS or Parallel File Systems

15


Hybrid Integration of SSDs drive IOPS and expand the performance envelope

16

400 K

20 K

IOPs

2 GB/s

MB

/sec

HDD

SSD

Bandwidth

Block Size


0

500

1000

1500

2000

2500

0

50000

100000

150000

200000

250000

300000

350000

400000

450000

0 50 100 150 200 250

IOPs

IO Request Size (KBytes)

SSD+HDD Hybrid VLUN Performance Synergy

IOPs required

HDD IOPs

SSD IOPs

HDD BW (MB/sec)

SSD BW (MB/sec)

Hybrid BW

ACTUAL Performance of Hybrid VLUN from 2U SSD and SAID


1M IOPs Test install at MS

18

2U Server: • Peak 710K IOPs (512b

Random Read) off ONE iSCSI initiator

• Saturated 2 Quad CPUs using 2 10G iSCSI Intel X-520 links

• Back-end at 1/5th loading possible (3.5M IOPs back-end test)

4U Server Test (TBD): • 3650 M2 with Six-Core

Westmere • 3850 M3 3.2Ghz Quad-

Socket Nehalem/Tylersburg

• Expect to hit 1.4M IOPs off ONE initiator


Stressing Windows Capability Win2k8 R2 SINGLE Initiator Near 1M IOPs

19


The Bottom Line -- Hybrid Storage Delivers the Flexibility to Solve Problems

20

RAM Scaling $$$$$

Perf

orm

ance

Scalability

Add HDD Back-end

Add

SSD

TM

E /

IA

Add

RAM

TM

E /

EA

SSD Scaling

$$$

HDD Scaling

$

SSD + HDD Scaling

$$

Fundamental Storage Customer Requirements

Performance Capacity Scalability Cost


Virtualization Engine – Tiered Ingest/Egress Details

Virtualization Engine

Front-End IO Interface (SCSI Target Mode

Transport and Processing)

Back-end IO

IO Request Interface

ITL-Nexus IO-mapper

RAID-10 Mapping RAID-50 Mapping

Customer Initiator

HDD SAID SSD JBOF

Ingest Egress

FIFO VLUN1

Tier Manager

Tier-0 Analyzer

Tier-1 Analyzer

VLUN1 VLUN-n

IO request

Ingest IO Reforming Egress IO Read-ahead

Ingest Egress

FIFO VLUN-n

Tier-0 Cache VLUN2

VLUN2


Virtualization Engine – Tiered Cache Write-Back on Read Details




Back-end IO


ITL-Nexus IO-mapper


Customer Initiator

HDD SAID SSD JBOF

Ingest Egress

FIFO VLUN1

Tier Manager

Tier-0 Analyzer

Tier-1 Analyzer

VLUN1 VLUN-n

IO request


Ingest Egress

FIFO VLUN-n

Tier-0 Cache VLUN2

VLUN2


Virtualization Engine – Tiered Cache Write-Through Details




Back-end IO


ITL-Nexus IO-mapper


Customer Initiator

HDD SAID SSD JBOF

Ingest Egress

FIFO VLUN1

Tier Manager

Tier-0 Analyzer

Tier-1 Analyzer

VLUN1 VLUN-n

IO request


Ingest Egress

FIFO VLUN-n

Tier-0 Cache VLUN2

VLUN2


Virtualization Engine – Tiered Cache Read-Hit Details




Back-end IO


ITL-Nexus IO-mapper


Customer Initiator

HDD SAID SSD JBOF

Ingest Egress

FIFO VLUN1

Tier Manager

Tier-0 Analyzer

Tier-1 Analyzer

VLUN1 VLUN-n

IO request


Ingest Egress

FIFO VLUN-n

Tier-0 Cache VLUN2

VLUN2


Virtualization Engine – Tiered Cache Read-Miss Details




Back-end IO


ITL-Nexus IO-mapper


Customer Initiator

HDD SAID SSD JBOF

Ingest Egress

FIFO VLUN1

Tier Manager

Tier-0 Analyzer

Tier-1 Analyzer

VLUN1 VLUN-n

IO request


Ingest Egress

FIFO VLUN-n

Tier-0 Cache VLUN2

VLUN2


Histogram Resolution Example

410

700

128 MB Cells

170 170

Level-1 Hot-Spot(s)

Level-2 Hot-Spot(s)

(e.g. 7.5GB)

(e.g. 4.5MB)

Note: Hot-Spot Regions Not Likely to Be Contiguous, so diagram is figurative

Full Capacity HDD FV Counter Array

Top 10% FV Counter

Array Background Migration Process (AVE Region/Block Mapping Update)

SSD 5% Block Cache

SSD FV Counter

Array Eviction Candidates


FV Stability Formulation

][][

0.2])[][(][,

_

][][,

___

1

_

02

12

1

_))_((

))_((1

iFviFvShape

iFviFvabsiFvi

SamplesTotal

jBiniFvi

DimensionFvBinsNumSizeFv

t

SizeFvi

it

tt

t

SizeFvsizeFvij

sizeFvijt

∆−∆=∆

−=∆∀

=∀

=

∑

∑

<

=

+<

=

FV_Size=number of counters lumped in dimension

Num_Bins=Total counters or number of regions

FV_Dimension=number of elements in vector

Summation of Normalized Histogram taken at epoch t1, |Fv| < 1.0

Fv Change between epoch t2 and t1, where |DFv < 1.0|

0.0 ≤ DShape ≤ 1.0 ΔFV=0.0 No Shape Change ΔFV=1.0 Max Shape Change -

Unstable


Hash Mapping Example

410

700

128 MB Cells Bitmap Array to Indicate Whether Region has Tier-0

Cached Data (Ptr=yes, Null Ptr=none)


2.2sec)100004.0(sec)8006.0(

sec10000_

)__)_1(()___(___

_

6.00.1_0__0_____0__0

][____

][____0

__)0][__(__0

__

_

)1___(

0

)1__0(

0

)1___(

0

=×+×

=

×−+×=

+=

×=×=

=

=

=

×

>=

∑

∑

∑

−

=

−

=

−

=

µµµupspeed

latencyHDDaveratehitlatencySSDaveratehitlatencyHDDaveupspeed

TTT

upspeed

efficiencytierfitaccesstierratehitIOssortedtotalIOshostedtierfitaccesstier

icountsaccesssortedIOssortedtotal

icountsaccesssortedIOshostedtier

sizesetLBAicountsaccesssortedevaluatesizeLBAtier

missHDDhitSSD

onlyHDD

countsaccesssortedsizeof

i

setsLBAtier

i

countsaccesssortedsizeof

i

instant access. infinite data. one solution.mercury.pr.erau.edu/.../solve-technical-overview.pdf ·...

Documents