swift install workshop - openstack conference spring 2012
DESCRIPTION
OpenStack Swift is a highly-available distributed object storagesystem which supports highly concurrent workloads. Swift is thebackbone behind Cloud Files, Rackspace's storage-as-a-serviceoffering.In this workshop, which will be hosted by members of SwiftStack, Inc.,we'll walk you through deployment and use of OpenStack Swift. We'llbegin by showing you how to install Swift from the ground up.You'll learn:- what you should know about Swift's architecture- how to bootstrap a basic Swift installationAfter that, we'll cover how to use Swift, including information on:- creating accounts and users- adding, removing, and managing data- building applications on top of SwiftBring your laptop (with virutalization extensions enabled in the BIOS)and we will walk through setting up Swift in a virtual machine. We'llalso build an entire application on top of Swift to illustrate how touse Swift as a storage service. This is a workshop you won't want tomiss!TRANSCRIPT
![Page 1: Swift Install Workshop - OpenStack Conference Spring 2012](https://reader030.vdocuments.net/reader030/viewer/2022020116/556424d7d8b42a73298b50dc/html5/thumbnails/1.jpg)
OpenStack Swift Workshop
Joe Arnold
April 20, 2012San Francisco, CA
@joearnold swiftstack.com
Darrell [email protected]
@swifterdarrell
![Page 2: Swift Install Workshop - OpenStack Conference Spring 2012](https://reader030.vdocuments.net/reader030/viewer/2022020116/556424d7d8b42a73298b50dc/html5/thumbnails/2.jpg)
Getting Ready. . .Creating Virtualbox Environment
Virtualbox->File->Import Appliance...Start ‘Swift VB 32bit Demo’
ssh -p 3333 -l demo localhost
Username: “demo”
password: “password”
![Page 3: Swift Install Workshop - OpenStack Conference Spring 2012](https://reader030.vdocuments.net/reader030/viewer/2022020116/556424d7d8b42a73298b50dc/html5/thumbnails/3.jpg)
Uses of Swift
Overview of Architecture9 minutes
Deployment Workshop 47 minutes
![Page 4: Swift Install Workshop - OpenStack Conference Spring 2012](https://reader030.vdocuments.net/reader030/viewer/2022020116/556424d7d8b42a73298b50dc/html5/thumbnails/4.jpg)
Uses of Swift
Overview of Architecture9 minutes
Deployment Workshop 47 minutes
![Page 5: Swift Install Workshop - OpenStack Conference Spring 2012](https://reader030.vdocuments.net/reader030/viewer/2022020116/556424d7d8b42a73298b50dc/html5/thumbnails/5.jpg)
Data Storage
HTTP API
Swi ft is Similar to Amazon S3
NOT blocksNOT filesystemNOT SAN/NAS/DAS
![Page 6: Swift Install Workshop - OpenStack Conference Spring 2012](https://reader030.vdocuments.net/reader030/viewer/2022020116/556424d7d8b42a73298b50dc/html5/thumbnails/6.jpg)
Swift is laser focusedSwift Attributes
Scalable High concurrency
easy to useDurable
![Page 7: Swift Install Workshop - OpenStack Conference Spring 2012](https://reader030.vdocuments.net/reader030/viewer/2022020116/556424d7d8b42a73298b50dc/html5/thumbnails/7.jpg)
jclouds
![Page 8: Swift Install Workshop - OpenStack Conference Spring 2012](https://reader030.vdocuments.net/reader030/viewer/2022020116/556424d7d8b42a73298b50dc/html5/thumbnails/8.jpg)
Operating System + Kernel
OS, Node & Drive Monitoring
Swift Runtime Monitoring
OpenStack Object Storage
SSL Termination Authentication
Load Balancing
SwiftStack Distribution
![Page 9: Swift Install Workshop - OpenStack Conference Spring 2012](https://reader030.vdocuments.net/reader030/viewer/2022020116/556424d7d8b42a73298b50dc/html5/thumbnails/9.jpg)
Controller
SwiftStack Controller
![Page 10: Swift Install Workshop - OpenStack Conference Spring 2012](https://reader030.vdocuments.net/reader030/viewer/2022020116/556424d7d8b42a73298b50dc/html5/thumbnails/10.jpg)
Uses of Swift
Overview of Architecture9 minutes
Deployment Workshop 47 minutes
![Page 11: Swift Install Workshop - OpenStack Conference Spring 2012](https://reader030.vdocuments.net/reader030/viewer/2022020116/556424d7d8b42a73298b50dc/html5/thumbnails/11.jpg)
Proxy
The Ring
Zones
Swi ft ArchitectureOverview
![Page 12: Swift Install Workshop - OpenStack Conference Spring 2012](https://reader030.vdocuments.net/reader030/viewer/2022020116/556424d7d8b42a73298b50dc/html5/thumbnails/12.jpg)
Proxyhandles incoming requests
Scales Horizontally
![Page 13: Swift Install Workshop - OpenStack Conference Spring 2012](https://reader030.vdocuments.net/reader030/viewer/2022020116/556424d7d8b42a73298b50dc/html5/thumbnails/13.jpg)
The Ringmaps data to disk
![Page 14: Swift Install Workshop - OpenStack Conference Spring 2012](https://reader030.vdocuments.net/reader030/viewer/2022020116/556424d7d8b42a73298b50dc/html5/thumbnails/14.jpg)
Storage Zonesisolate physical failures
![Page 15: Swift Install Workshop - OpenStack Conference Spring 2012](https://reader030.vdocuments.net/reader030/viewer/2022020116/556424d7d8b42a73298b50dc/html5/thumbnails/15.jpg)
Quorum Writes
![Page 16: Swift Install Workshop - OpenStack Conference Spring 2012](https://reader030.vdocuments.net/reader030/viewer/2022020116/556424d7d8b42a73298b50dc/html5/thumbnails/16.jpg)
Single-Disk Reads
![Page 17: Swift Install Workshop - OpenStack Conference Spring 2012](https://reader030.vdocuments.net/reader030/viewer/2022020116/556424d7d8b42a73298b50dc/html5/thumbnails/17.jpg)
Replicationprocess runs continuously
![Page 18: Swift Install Workshop - OpenStack Conference Spring 2012](https://reader030.vdocuments.net/reader030/viewer/2022020116/556424d7d8b42a73298b50dc/html5/thumbnails/18.jpg)
Uses of Swift
Overview of Architecture9 minutes
Deployment Workshop 47 minutes
![Page 19: Swift Install Workshop - OpenStack Conference Spring 2012](https://reader030.vdocuments.net/reader030/viewer/2022020116/556424d7d8b42a73298b50dc/html5/thumbnails/19.jpg)
Workshop Philosophy
Learn the Guts step-by-step by typing.
(NOT run our magic deployment tool.)
Follow along
-or-
Sit back and watch some typing
![Page 20: Swift Install Workshop - OpenStack Conference Spring 2012](https://reader030.vdocuments.net/reader030/viewer/2022020116/556424d7d8b42a73298b50dc/html5/thumbnails/20.jpg)
Getting Ready. . .Creating Virtualbox Environment
Virtualbox->File->Import Appliance...Swift VB 32bit Demo.ovaStart ‘Swift VB 32bit Demo’ VMOn host OS, SSH to localhost, port 3333 (port forwards to guest OS)
SSH to (host) localhost, port 3333$ ssh -p 3333 -l demo localhost
Username: “demo”Password: “password”
![Page 21: Swift Install Workshop - OpenStack Conference Spring 2012](https://reader030.vdocuments.net/reader030/viewer/2022020116/556424d7d8b42a73298b50dc/html5/thumbnails/21.jpg)
Formatting Devices$ sudo su -# mkfs.xfs -f -i size=512 -L d1 /dev/sdb# mkfs.xfs -f -i size=512 -L d2 /dev/sdc# mkfs.xfs -f -i size=512 -L d3 /dev/sdd# mkfs.xfs -f -i size=512 -L d4 /dev/sde
H a n d y C o m m a n d sMounted Filesystems# df
Formatted Filesystems# blkid
![Page 22: Swift Install Workshop - OpenStack Conference Spring 2012](https://reader030.vdocuments.net/reader030/viewer/2022020116/556424d7d8b42a73298b50dc/html5/thumbnails/22.jpg)
Mounting Drives# mkdir -p /srv/node/d1# mkdir -p /srv/node/d2# mkdir -p /srv/node/d3# mkdir -p /srv/node/d4
# mount -t xfs -L d1 /srv/node/d1# mount -t xfs -L d2 /srv/node/d2# mount -t xfs -L d3 /srv/node/d3# mount -t xfs -L d4 /srv/node/d4
H a n d y C o m m a n d s# for i in 1 2 3 4; do> mkdir -p /srv/node/d$i> mount -t xfs -L d$i /srv/node/d$i> done# chown -R swift:swift /srv/node
![Page 23: Swift Install Workshop - OpenStack Conference Spring 2012](https://reader030.vdocuments.net/reader030/viewer/2022020116/556424d7d8b42a73298b50dc/html5/thumbnails/23.jpg)
Installing Swift# apt-get install curl gcc bzr python-configobj python-coverage python-dev python-nose python-setuptools python-simplejson python-xattr python-webob python-eventlet python-greenlet debhelper python-sphinx python-all python-openssl python-pastedeploy python-netifaces bzr-builddeb xfsprogs(get and install Swift)# apt-get update
...those should be already installed
![Page 24: Swift Install Workshop - OpenStack Conference Spring 2012](https://reader030.vdocuments.net/reader030/viewer/2022020116/556424d7d8b42a73298b50dc/html5/thumbnails/24.jpg)
Creating the Builder Files
# cd /etc/swift# swift-ring-builder account.builder create 18 3 1# swift-ring-builder container.builder create 18 3 1# swift-ring-builder object.builder create 18 3 1
swift-ring-builder (account|container|object).builder create <part_power> <replicas> <min_part_hours>
![Page 25: Swift Install Workshop - OpenStack Conference Spring 2012](https://reader030.vdocuments.net/reader030/viewer/2022020116/556424d7d8b42a73298b50dc/html5/thumbnails/25.jpg)
Creating the Builder Files
Builder files contain a record of all devices in the cluster
AccountsContainers
Objects
![Page 26: Swift Install Workshop - OpenStack Conference Spring 2012](https://reader030.vdocuments.net/reader030/viewer/2022020116/556424d7d8b42a73298b50dc/html5/thumbnails/26.jpg)
Creating the Builder FilesSwift Partitions
![Page 27: Swift Install Workshop - OpenStack Conference Spring 2012](https://reader030.vdocuments.net/reader030/viewer/2022020116/556424d7d8b42a73298b50dc/html5/thumbnails/27.jpg)
Creating the Builder FilesHow many Swift Partitions?
How big will you be when you grow up?Rule of thumb: 100 partitions * the number of drives that you think you will ever have, carried up to the nearest power of 2.
24
24
24
24
96 Drives * 100 = 9600
2^13 = 8192
2^14 = 16384
![Page 28: Swift Install Workshop - OpenStack Conference Spring 2012](https://reader030.vdocuments.net/reader030/viewer/2022020116/556424d7d8b42a73298b50dc/html5/thumbnails/28.jpg)
Creating the Builder FilesReplicas
Go with 3.
![Page 29: Swift Install Workshop - OpenStack Conference Spring 2012](https://reader030.vdocuments.net/reader030/viewer/2022020116/556424d7d8b42a73298b50dc/html5/thumbnails/29.jpg)
Creating the Builder FilesMini Part Hours
Ensures that only one replica is in flight
A good default configuration setting is 24 hours.
![Page 30: Swift Install Workshop - OpenStack Conference Spring 2012](https://reader030.vdocuments.net/reader030/viewer/2022020116/556424d7d8b42a73298b50dc/html5/thumbnails/30.jpg)
Creating the Builder Files$ ls /etc/swiftaccount.builder backups container.builder object.builder (+ *.conf)
E x t r a C r e d i t# pythonPython 2.7.1+ (r271:86832, Apr 11 2011, 18:05:24)[GCC 4.5.2] on linux2Type "help", "copyright", "credits" or "license" for more information.
>>> import pickle>>> print pickle.load(open('object.builder')){'_replica2part2dev': None, '_last_part_gather_start': 0, 'min_part_hours': 1, 'replicas': 3, 'parts': 1024, 'part_power': 10, 'devs': [], 'devs_changed': False, 'version': 0, '_last_part_moves_epoch': None, '_last_part_moves': None, '_remove_devs': []}
![Page 31: Swift Install Workshop - OpenStack Conference Spring 2012](https://reader030.vdocuments.net/reader030/viewer/2022020116/556424d7d8b42a73298b50dc/html5/thumbnails/31.jpg)
Creating the Rings: Zonesswift-ring-builder (account|container|object).builder add z<zone>-<ip>:<port>/<device_name> <weight>
![Page 32: Swift Install Workshop - OpenStack Conference Spring 2012](https://reader030.vdocuments.net/reader030/viewer/2022020116/556424d7d8b42a73298b50dc/html5/thumbnails/32.jpg)
Creating the Rings: Weights
100 150
2 3 3 32 2
PartitionsPartitions
DisksDisks
swift-ring-builder (account|container|object).builder add z<zone>-<ip>:<port>/<device_name> <weight>
![Page 33: Swift Install Workshop - OpenStack Conference Spring 2012](https://reader030.vdocuments.net/reader030/viewer/2022020116/556424d7d8b42a73298b50dc/html5/thumbnails/33.jpg)
Creating the Rings: Adding Devices
Validate your rings# swift-ring-builder account.builder# swift-ring-builder container.builder# swift-ring-builder object.builder
Note that there are no partitions assigned to any drives yet. We must use “rebalance” to assign partitions and create the actual ring files.
![Page 34: Swift Install Workshop - OpenStack Conference Spring 2012](https://reader030.vdocuments.net/reader030/viewer/2022020116/556424d7d8b42a73298b50dc/html5/thumbnails/34.jpg)
Creating the Rings: Do It (Rebalance)!# cd /etc/swift# swift-ring-builder account.builder rebalance# swift-ring-builder container.builder rebalance# swift-ring-builder object.builder rebalance
# ls /etc/swift/*.ring.gzaccount.ring.gz container.ring.gz object.ring.gz
Validate your rings# swift-ring-builder account.builder# swift-ring-builder container.builder# swift-ring-builder object.builder
![Page 35: Swift Install Workshop - OpenStack Conference Spring 2012](https://reader030.vdocuments.net/reader030/viewer/2022020116/556424d7d8b42a73298b50dc/html5/thumbnails/35.jpg)
Let’s Start Swift!
# swift-init main restart
H a n d y C o m m a n d sTail the log file$ tail -f /var/log/swift/all.log
![Page 36: Swift Install Workshop - OpenStack Conference Spring 2012](https://reader030.vdocuments.net/reader030/viewer/2022020116/556424d7d8b42a73298b50dc/html5/thumbnails/36.jpg)
Get a Token!$ curl -v -H 'X-Auth-User: admin:admin' -H 'X-Auth-Key: admin' http://127.0.0.1:8080/auth/v1.0/
< HTTP/1.1 200 OK< X-Storage-Url: http://127.0.0.1:8080/v1/AUTH_admin< X-Storage-Token: AUTH_tk265318ae5e7e46f1890a441c08b5247f< X-Auth-Token: AUTH_tk265318ae5e7e46f1890a441c08b5247f< X-Trans-Id: txc75adf112791425e82826d6e3989be4d< Content-Length: 0
Extra Credit: Token in memcache?# python>>> import swift.common.memcached as memcached>>> memcache = memcached.MemcacheRing(['127.0.0.1:11211'])>>> print memcache.get('AUTH_/user/admin:admin')AUTH_tk265318ae5e7e46f1890a441c08b5247f>>> print memcache.get('AUTH_/token/AUTH_tk265318ae5e7e46f1890a441c08b5247f')(1308804765.9103661, 'admin,admin:admin')>>> (Hit Ctrl-D to exit)
![Page 37: Swift Install Workshop - OpenStack Conference Spring 2012](https://reader030.vdocuments.net/reader030/viewer/2022020116/556424d7d8b42a73298b50dc/html5/thumbnails/37.jpg)
Use a Token!$ curl -v -H 'X-Auth-Token: <your x-auth-token>' <your x-storage-url>
< HTTP/1.1 204 No Content< X-Account-Object-Count: 0< X-Account-Bytes-Used: 0< X-Account-Container-Count: 0< Accept-Ranges: bytes< X-Trans-Id: txafe3c83ed76e46d2a9536dd61d9fcf09< Content-Length: 0
![Page 38: Swift Install Workshop - OpenStack Conference Spring 2012](https://reader030.vdocuments.net/reader030/viewer/2022020116/556424d7d8b42a73298b50dc/html5/thumbnails/38.jpg)
Uploading
$ swift -U admin:admin -K admin -A http://127.0.0.1:8080/auth/v1.0/ upload test_container /vmlinuz
$ swift -U admin:admin -K admin -A http://127.0.0.1:8080/auth/v1.0/ list test_container
H a n d y C o m m a n dLet’s use the command ‘swift’ from here on out.
Where is the object?# cd /srv/node# find . -name '*.data'# swift-object-info /full/path/to/file.data# swift-get-nodes /etc/swift/object.ring.gz \ AUTH_admin test_container vmlinuz
![Page 39: Swift Install Workshop - OpenStack Conference Spring 2012](https://reader030.vdocuments.net/reader030/viewer/2022020116/556424d7d8b42a73298b50dc/html5/thumbnails/39.jpg)
Replication
Start Swift replicator services# swift-init account-replicator start# swift-init container-replicator start# swift-init object-replicator start
Is rsync running?$ rsync localhost::$ rsync localhost::account$ rsync localhost::container$ rsync localhost::object
![Page 40: Swift Install Workshop - OpenStack Conference Spring 2012](https://reader030.vdocuments.net/reader030/viewer/2022020116/556424d7d8b42a73298b50dc/html5/thumbnails/40.jpg)
DESTROY!!!!Delete all data in a zone!
# cd /srv/node/d1# find .# rm -rf *; find .# sleep 20# find .
H a n d y C o m m a n d sTail the log file# tail -f /var/log/swift/all.log# tail -f /var/log/rsyncd.log
![Page 41: Swift Install Workshop - OpenStack Conference Spring 2012](https://reader030.vdocuments.net/reader030/viewer/2022020116/556424d7d8b42a73298b50dc/html5/thumbnails/41.jpg)
24-36 disk systemsSATA desktop drives24-48 GB RAM
![Page 42: Swift Install Workshop - OpenStack Conference Spring 2012](https://reader030.vdocuments.net/reader030/viewer/2022020116/556424d7d8b42a73298b50dc/html5/thumbnails/42.jpg)
Zone
Zone
Zone
Zone
ProxyProxy
100 Terabyte 2 Proxy/LB Servers4 Object Stores (36-drive, 3TB)
![Page 43: Swift Install Workshop - OpenStack Conference Spring 2012](https://reader030.vdocuments.net/reader030/viewer/2022020116/556424d7d8b42a73298b50dc/html5/thumbnails/43.jpg)
Zone
Zone
Zone
Zone
Zone
Agg
ToR ToR ToR ToR ToR
Prox
y
1.5 Petabyte
2 Agg Switches6 Proxy/LB Servers5 ToR Switches50 Object Stores (36-drive / 3TB
![Page 44: Swift Install Workshop - OpenStack Conference Spring 2012](https://reader030.vdocuments.net/reader030/viewer/2022020116/556424d7d8b42a73298b50dc/html5/thumbnails/44.jpg)
Thank You!This presentation + VBox Image
Swift technical white paper
Contact: [email protected]
[email protected]@swiftstack.com