Download - An Open Source Approach to Replication And
-
8/14/2019 An Open Source Approach to Replication And
1/14
An Open Source approach to replication and
reco
-
8/14/2019 An Open Source Approach to Replication And
2/14
Current Challenges:Miniscule IT Budgets.Backup Tapes dont meet capacity needs or are costly.Most RAIDs dont protect from certain data loss.Quick recovery of virtual disks and OS.Stuck with same vendor for offsite storage.
-
8/14/2019 An Open Source Approach to Replication And
3/14
Vendor Specific SolutionsCostly; upwards of $60,000 for infrastructure.Offers features only available to like systems.Choose Two out of Three: Fast, Reliable, Cheap.
Yearly support costs can damper most IT budgets. Typical useable storage capacity: 2.8TB with a single shelf, therest for snapshots and filesystem overhead.
-
8/14/2019 An Open Source Approach to Replication And
4/14
An Open Source SolutionOpen Solaris with ZFS and iscsitargetsoftware.Relatively inexpensive compared to 3 rd
vendorsCan mix and match hardware.ZFS: Virtually unlimited capacity!
Technical creativity when building scripts.
-
8/14/2019 An Open Source Approach to Replication And
5/14
Cons about OpensolarisiSCSI
Not supported by VMware. (yet)OS and Filesystem Learning curveLimited to 1gb/s of bandwidth per SAN* (until
10gb/s is released)No support for MTU 9000 (Jumbo Frames)
-
8/14/2019 An Open Source Approach to Replication And
6/14
Necessary Tools forsuccess.
ZFS File System; /sbin/zpool and /sbin/zfs;snapshotsIscsitadm (pkg add)SSH keygen/pgp
A replication script (e.g. zfs-replicate.sh)CronMailEnable Vmware LVM/snapshot
-
8/14/2019 An Open Source Approach to Replication And
7/14
What is Zpool/ZFS?ZFS is a file system designed by Sun Microsystems for the SolarisOperating System. Features include support for high storagecapacities (16 exabytes per pool or 16 million terrabytes), snapshotsand copy-on-write clones, continuous integrity checking (256bit CRCchecks of every block) and automatic repair (scrubbing), RAID-Z andACLs. (Found on Wiki)http://opensolaris.org/os/community/zfs/whatis/RAID 0 5: No protection against silent disk corruption and bit rot.http://blogs.sun.com/bonwick/entry/raid_zZpool: disk pool creation, iostatus, and health display (build RAIDZ,RAIDZ2, Jbods, stripes, mirrors, striped mirrors, etc)
http://opensolaris.org/os/community/zfs/whatis/http://blogs.sun.com/bonwick/entry/raid_zhttp://blogs.sun.com/bonwick/entry/raid_zhttp://opensolaris.org/os/community/zfs/whatis/ -
8/14/2019 An Open Source Approach to Replication And
8/14
Open Solaris iscsitargetUses industry standard iscsi target and initiator callsExtremely easy to use and seamlessly integrates withZpool/ZFSAllows for the creation of soft provisioned disks.
ACL support, deny unwanted initiators.Iscsitadm list targets v : details of each currently activetargets
-
8/14/2019 An Open Source Approach to Replication And
9/14
Replication Script; Mail;Cron
Many available on the internetModified a script from the web (author unknown) to workaccording to my requirements. Still in progress butworks . (zfs-replicate.sh)
Use Cron to execute zfs-replicate on a schedule.Mail the results to your disk admins.
-
8/14/2019 An Open Source Approach to Replication And
10/14
Vmware EnterpriseConfigure VMWare for iSCSI supportAdd targets to Vmware host adapters.Enable LVM snapshot support: GUI far more
simpler than console.
-
8/14/2019 An Open Source Approach to Replication And
11/14
Snapshots: How to?Reserve up to 40% of maximum pool capacity for snapshots,i.e. 10TB pool, 6TB data, 4TB snapshots. Admin discretion andmay be less depending on LUN configuration.Snapshots are the size of the used capacity of theLUN/partition.Making a snapshot is a breeze!
/sbin/zfs snapshot pool/partition@snapshotSend the snapshot over to the remote site.
/sbin/zfs send pool/partition@snapshot | ssh username@host \/sbin/zfs recv pool/partition
-
8/14/2019 An Open Source Approach to Replication And
12/14
Replication typesZFS send/recv with SSH (avg. speed: 25MB/s at 256bit encryption)Slowest replication, doesnt require dedicated network and secure.
Different algorithms provide different data throughput.Stunnel may be quicker!
ZFS send/recv with mbuffer (speeds: 80 to 120MB/s Avg: 360Gig/hr)lev@siscsi-sas:~$ zfs send sd/Linux@1 | mbuffer -b1024k -m 1500m -O 172.32.66.10:9999;LOCAL SITElev@piscsi-sas:~$ mbuffer -m 1500M -s1024k -I 172.32.66.11:9999 | zfs recv pdrive/Linux;REMOTE SITE
http://www.maier-komor.de/mbuffer.htmlMust have enough system memory (+4GB) to support large buffers.Non encrypted, requires network paths to be safe.
ZFS send/recv with netcat/rsh (avg. speed: 35MB/s)Insecure data copy, just for comparison.
http://www.maier-komor.de/mbuffer.htmlhttp://www.maier-komor.de/mbuffer.html -
8/14/2019 An Open Source Approach to Replication And
13/14
Data RecoveryActivate offsite VM ESX Host.Access offsite disk storage LUNs.
add iSCSI remote host to Virtual CenterRe-add VM Disk (LUNs) to offsite VM Infrastructure.
GUI: Advanced Features: LVM: EnableResignatureCLI: /proc/vmware/config/LVM/EnableResignature
Resignatured LUNs will appear automatically in the storage section of VirtualCenter.
Re-add VM Guests from resignatured LUNsOffsite LUNs can be added to primary VM site for data recovery ortesting...
-
8/14/2019 An Open Source Approach to Replication And
14/14
Questions?
Tano Simonian
Email: [email protected](ofc) 310 794 9669
mailto:[email protected]:[email protected]