stonehenge education stonehenge team © 2003 ibm corporation

Stonehenge Education

Stonehenge Team © 2003 IBM Corporation



File Services from a Customer View

Customer concerns:• High availability

• Lower management overhead

• Scalability without disruption

• Security

• ...

A glance at typical customer installations >>>



Typical Server Environment, w/wo. SAN

Customer concern: If server fails, data is inaccessible. If storage 'dies', changed data since last backup is lost.

LAN

SAN

'Office' clients

Local disks

Note: Disk drives storeblocks of bytes; servers interpret those as files.



Providing High Availability

Clustering Fileserver with local or SAN storage Active/Standby or Active/Active (mutual failover)

LAN

SAN

Active Standby Active' Active"

Disk' Disk"

Same, but vice-versa

LAN

local

Standby server takes overdisk + service + IP addressduring active server failure.



Customer Headache

Needs a cluster for every new service ($$) Complex setup and maintenance No easy scalabililty

LAN

SAN



Scaling Servers, Migrating Data

Customers needs a bigger file server... A: Copying data, shutdown, startup new B: Shutdown, re-assigning SAN disk, startup

SAN

A

B

Always disruptive!

LAN



Migrating Data between Locations

Data needs to be copied over a thin WAN connection Original data is subject to change while being copied => Servers have to be shut down for significant time

WAN

100km+

Always disruptive!



Customer Whish List

Make services less hardware-dependent Consolidate hardware Migrate & scale without disruption

LAN

File service Web service Print service

Enough hardware to handle it

Just enough storage



Stonehenge Architecture

Services (Samba, Apache, Print, …) are "virtual servers" Each virtual server has it's own IP address

LAN

File

Distributed file storage layer

IP1

web

IP2

print

IP3



Stonehenge Architecture (2)

Virtual servers are implemented on physical machineswithout (!) locally stored data


File

IP1

print

IP3

web

IP2

web

IP2

LAN

Virtual servers can 'hop' between nodes

Several virtual servers residing on

one node



Stonehenge Architecture (3)

Data is stored on hidden backend servers (SAN recommended)

Cell Cell

LAN

File

IP1

web

IP2

print

IP3

AFS server

AFS server

AFS server

Private LAN

"Disk owner" layer

Each virtual server can access any AFS server



Stonehenge Ingredients

Where they go:

File

IP1

web

IP2

print

IP3

AFS server

AFS server

AFS server

Linux

OpenAFS client

Endpoint'

Apache

Samba

AIX / Linux

OpenAFS server

Endpoint"

Linux

OpenAFS client

MySQL

Admin server

LDAP

Admin



File

IP1

web

IP2

print

IP3

AFS server

AFS server

AFS server

Admin Console View



Example: Datacenter + Branch Offices Setup

WAN

User LAN User LAN User LAN

Private LAN

AFS(x/p-Series)

Physical(xSeries)

Disk cache

WAN

SAN

Branch Type 2

Site with local storage as well as cached headoffice

access.

Reading, writing and replicating to & from

Tape backup of branch sites using

replica

Branch Type 1

Site with cached access to headoffice

data; local speed after first access.

Mainly for reading

Disk cache



Stonehenge TCO Savings: Where?

Scaling Migrating Backing up/Restoring Securing/Mirroring Administrating



Scaling Stonehenge

Adding services (e.g. new file servers) affects the front layer Add node hardware only if required


File

IP1

Fileweb

IP3

LAN

Add hardware, redistribute services:

No Downtime

IP5

print

IP2

File

IP4

File +



Migrating Data within Stonehenge

Moving data is done in the backend Services are not affected

Cell CellAs soon as data is

synchronized, mirror copy becomes original:

LAN

File

IP1

web

IP2

print

IP3

AFS server

AFS server

AFS server

— Private LAN or WAN



Example1: Datacenter adds Webservers

Traditional method:• Move servers in, attach storage

• install & configure web services

• Transfer data (1 day)

With Stonehenge:• Define new virtual web server on hardware with resources left

• Add access rights to web files, configure web (15 min) Later:

• Add hardware when resources are low



Example2: Company opens a new Subsidiary

Traditional method:• Move servers into subsidiary, install storage

• install services

• Replay data from tape, fetch latest changes through WAN (1

week)

With Stonehenge:• Move server hardware in, connect to wide area network

• Create or Move virtual servers from remote (1 day)

• Cached access to company data ok Later:

• Add backend server + storage for enabling replica etc.



Backup Strategy in Stonehenge with AFS

AFS server

AFS server

AFS server

AFS server

AFS server

Point-in-time copy of volumes (backup volume)

Replicate snapshot/backup volume to other location

Backup to tape if requried

"Snapshot" at T1

AFS server

Copy of "Snapshot"



Tapeless Backup in Stonehenge with AFS

AFS server

AFS server

AFS server

Dump volume content onto disk archive file (full or incremental)

Keep dump archive files on disks for quick restore

Dumping eliminates TSM "small files" performance bottleneck

1 2

This can be cheaper"nearline" storage



Securing/Mirroring Data in Stonehenge with AFS

AFS server

AFS server

AFS server

AFS server

AFS server

AFS server

Asynchronous Mirror Scheduled Replica

Synchronous local Mirror Sync. SAN Failover Mirror

• Logical Volume Manager feature = =



Stonehenge User Management

Integration Automation Accounting



User/Client Management Integration

Single Sign-on in existing Windows® environment

W2K Active Directory is managed from Stonehenge• User Access rights used for AFS / Samba

• Samba DFS is integrated in Stonehenge

OpenLDAP may be used instead of external Active Directory• Option: Active Directory may be synchronized with external LDAP

ACL management: Yes for Samba/Web; No for NFS access• But: Attaching NFS clients through OpenAFS client plugin* offers

enhanced ACLs, better network utilization, persistent caching etc.

*http://www.openafs.org/release/latest.html



User Management Automation: LDAP Sync

Employee newly hired:

Employee re-assigned:

Employee retires:

Minimum Admin Involvement!

Bluepages User listLDAP Sync

Bluepages Access rights

Bluepages Access rights

× ×

+



Economies in User Management

Each user has a home volume (Z:\ or //mainz/afs/home_065432) ...with optional subfolders with older versions:

Z:\MyDocuments MyDailyWork.doc 5.4.2003.oldfiles MyDailyWork.doc 4.4.2003 < previous document version Unchanged_file.doc 1.2.2003 < unchanged = only a pointer

Minimizes restore requests for accidentially deleted files!



Easy Planning & Accounting

Charge service users according to their virtual service traffic:



Information & Sales Material Intranet + Event calendar

w3.ais.mainz.de.ibm.com/stonehenge/

Internet www-5.ibm.com/services/de/its/filestore.html/

Support for new opportunities, sizing, etc. [email protected]

Sales Material• 2-page Customer Flyer (english)• Several presentations; see website• 1-page ITS Customer Flyer (german)

Demo software for Thinkpads: Offline GUI emulation



Questions ?



Additional Information



Sample accounting output

Storage space used:& maximum quota

Bandwidth used:local & remote

Recent accesses:

afsfs01.sc.ais.mainz.de.ibm.com /vicepaa RWrite 536871744 ROnly 0 Backup 0 MaxQuota 10240000 K Creation Tue Jul 23 13:22:01 2002 Last Update Fri Mar 14 14:41:21 2003 0 accesses in the past day (i.e., vnode references)

Raw Read/Write Stats |-------------------------------------------| | Same Network | Diff Network | |----------|----------|----------|----------| | Total | Auth | Total | Auth | |----------|----------|----------|----------|Reads | 1719 | 1719 | 913 | 913 |Writes | 190 | 190 | 0 | 0 | |-------------------------------------------|

Writes Affecting Authorship |-------------------------------------------| | File Authorship | Directory Authorship| |----------|----------|----------|----------| | Same | Diff | Same | Diff | |----------|----------|----------|----------|0-60 sec | 36 | 0 | 45 | 0 |1-10 min | 0 | 0 | 1 | 0 |10min-1hr | 0 | 0 | 0 | 0 |1hr-1day | 0 | 0 | 0 | 0 |1day-1wk | 0 | 0 | 3 | 0 |> 1wk | 0 | 0 | 1 | 0 | |-------------------------------------------|

RWrite: 536871744 number of sites -> 1 server afsfs01.sc.ais.mainz.de.ibm.com partition /vicepaa RW Site

afsfs01.sc.ais.mainz.de.ibm.com /vicepaa RWrite 536871744 ROnly 0 Backup 0 MaxQuota 10240000 K Creation Tue Jul 23 13:22:01 2002 Last Update Fri Mar 14 14:41:21 2003 0 accesses in the past day (i.e., vnode references)

Raw Read/Write Stats |-------------------------------------------| | Same Network | Diff Network | |----------|----------|----------|----------| | Total | Auth | Total | Auth | |----------|----------|----------|----------|Reads | 1719 | 1719 | 913 | 913 |Writes | 190 | 190 | 0 | 0 | |-------------------------------------------|

Writes Affecting Authorship |-------------------------------------------| | File Authorship | Directory Authorship| |----------|----------|----------|----------| | Same | Diff | Same | Diff | |----------|----------|----------|----------|0-60 sec | 36 | 0 | 45 | 0 |1-10 min | 0 | 0 | 1 | 0 |10min-1hr | 0 | 0 | 0 | 0 |1hr-1day | 0 | 0 | 0 | 0 |1day-1wk | 0 | 0 | 3 | 0 |> 1wk | 0 | 0 | 1 | 0 | |-------------------------------------------|

RWrite: 536871744 number of sites -> 1 server afsfs01.sc.ais.mainz.de.ibm.com partition /vicepaa RW Site



When you should consider Stonehenge

1 Running a Windows® or DCE-DFS office environment

2 Looking for file- & print-server consolidation

3 Looking for scalable NAS with disaster recovery

4 Managing many users (min > 500)

5 Running branch offices that need interconnection

6 Concerned with storage migration

7 NOT running databases on fileservers or NAS!



Product availability

Public beta in Germany since 1/2003

Preview at CeBIT 3/2003

Product availability since 8/2003

Western and Northern Europe regional support in Q1-2004

IBM service organization ITS offers standardized bundles • Hardware, software, implementation, service

• Storage environment managed from remote



Beta-installation outside IBM (Jan.2003)

Large German public sector federal bureau• Stonehenge for 5000 users

Customer's reason for choosing Stonehenge• Security, inherent hack-proof

• Savings potential on daily administrative tasks



Upcoming reference installations outside IBM

Large German Bank• Pilot with few 1000 users, later extending to subsidiaries

German Energy Provider• With IT partner, migrating all WinNT users to Stonehenge

Multinational Electronics & Power Manufacturer• Managed Storage, hardware in-house / management remote

University• Pilot for joint German universities project• Many users

...



Traditional IT services:

Virtual Hosts

Shared Data Layer

Unix W2KWeb DNS LDAP...DHCP

Print

WebW2K

Mail

Print LDAP DNS DHCP

Unix

Files

FTP

FTP

Robust load balancing

Stonehenge Goal:



Stonehenge Ingredients Linux (SuSE, RedHat) Samba "Windows server under Linux", SMB protocol Apache Webserver for Linux OpenAFS Andrew File System …

MySQL Database, Sequential Query Language OpenLDAP 'Index', Lightweight Directory Access Protocol Kerberos Security system developed by MIT ...

Stonehenge Admin server Endpoints (per node type) GUI Core Components

Assisted Components

Helper Applications



Stonehenge, VMware, SVC, Storage Tank

Stonehenge– is a fileserver virtualization architecture including ACL, User-, DNS and

DFS management, storage migration and mirroring tools. Use for many small (Windows etc.) office clients.

VMware– is a partitioning software for several virtual OS spaces on one Intel

machine. Use for any application, but within one machine. SAN Volume Controller (Lodestone)

– is a storage re-organizer on LUN level, totally transparent to any application or OS. Includes migration, resizing, copying and mirroring tools. Performance and I/O handling is ok for databases.

SAN Filesystem (Storage Tank)– is a network filesystem where clients access shared files not through

LAN, but through SAN. Clients need fibrechannel and a driver that handles file placement using policies. Use for high duty fibre clients.



The sizing data contained herein was obtained in a controlled environment based on the use of specific data. Actual results that may be obtained in other operating environments may vary significantly. These values do not constitute a guarantee of performance.

Product data is accurate as of initial publication and is subject to change without notice.

No part of this presentation may be reproduced or transmitted in any form without written permission from IBM Corporation.

References in this document to IBM products, programs, or services do not imply that IBM intends to make these available in all countries in which IBM operates. Any reference to an IBM program product in this document is not intended to state or imply that only IBM's program product may be used. Any functionally equivalent program may be used instead.

The information provided in this document has not been submitted to any formal IBM test and is distributed "As Is" basis without any warranty either express or implied. The use of this information or the implementation of any of these techniques is a customer responsibility and depends on the customer's ability to evaluate and integrate them into their operating environment.

Java is a trademark of Sun Microsystems, Inc. in the United States and other countries. Windows is a registered trademark of Microsoft Corporation in the United States and other countries.

Disclaimer/Trademarks

stonehenge education stonehenge team © 2003 ibm corporation

Documents

ibm corporation slide

ibm corporation file

storage slide

afs server slide

node slide

locations data needs

wan connection original

san customer concern