technical brief › docs › tiger-bridge › getbridge › ti… · hybrid architecture capable of...

Technical Brief

Policy based Software only connector blending NTFS with cloud, NAS, and tape into a single global file system. Enables seamless and transparent cloud integration for legacy workflows using existing hardware and infrastructure.

Tiger Bridge Scale-Deep Lifecycle Manager

v. 4.1 – June 28th 2018

Alexander Lefterov Vesselin Batzarov Nikola Apostolov

Bernard Lamborelle Lance Kelson

1 © 2017-2018 Tiger Technology – Specifications subject to change without notice.

Tiger Technology AD 54 G.M.Dimitrov blvd. - 1125 Sofia, BULGARIA

Telephone: 514 700 1230 • [email protected]

TABLE OF CONTENT

Introduction 3

Hierarchical vs. Object storage 4 The challenge of using object-storage with traditional applications 5

Tiger Bridge: the perfect hybrid solution 6 A seamless workflow 7 Unique Azure Integration 8 Keeping multiple servers in sync 8 Leveraging the scalable processing power of the cloud 9 Key features 10

Three simple phases for transitioning to the cloud 11 Phase 1 - Replicate to the cloud 11 Phase 2 - Operate an hybrid setup 11 Phase 3 - Complete the migration 12

Applications 12 Seamless Capacity Expansion 13 Easy Storage Migration/Decommissioning 13 Quick-Restore Backup 13 Zero-Downtime Disaster Recovery 13 Minimizing Block-Level Storage Utilization in the Cloud 14 Manipulating and Accessing Big Data 15 Leveraging HSM Storage in the Cloud 15 Rich Media & Entertainment 15 Video Surveillance 16

Conclusion 18


mailto:[email protected]

Introduction

The dramatic explosion of content is causing IT departments around the world to play catch up. Traditional file systems, which have been universal data repositories for virtually all applications, are now revealing their limitations in large scale deployments, where object-storage has proven its intrinsic value. Unfortunately, interfacing unstructured object-storage with current applications is challenging because the applications were designed for hierarchical storage. While the cloud offers amazing benefits, the migration of legacy servers has proven to be slow, difficult and challenging. There is a clear need for a new hybrid architecture capable of facilitating the integration and migration of large scale Windows NTFS cloud deployments while reducing cost & complexity. In addition, the bulk of data stored on file servers is stale data that accumulates over time. In many cases, only a small portion of all files are being actively worked on at any given time. While users need easy access to their data, admins must manage performance and capacity growth, security, backups, long-term archives, and deal with multiple locations as cost effectively and efficiently as possible. They simply can’t afford any excess downtime and must stay within budget. Tiger Bridge extends local storage to DAS, NAS, Tape and Cloud targets. It installs as a software connector on existing servers in order to extend local drives into the target by seamlessly replicating and tiering data across multiple locations. Best of all, Tiger Bridge enables cloud processing to be easily performed on that very same data. With Tiger Bridge, there is no need to change workflows, train users or reconfigure applications. Simply connect your existing file system to your preferred target.


Hierarchical vs. Object storage

The idea of using a file system to store data on a volume has been ubiquitous in computer technology for decades. Most applications today, including IoT devices, medical scanners, surveillance cameras, word processors, image processing - were designed to work with hierarchical file systems. They can store or modify data on structured file systems only (DAS, NAS, SAN). Structured file systems can be thought of as hierarchical indexes, containing folders, sub-folders and files. Using an absolute pathname, one can access any particular file, as well as part of it. File systems make it easy to keep things organized. It is also the most natural way for people to store, search and access content. The challenge associated with hierarchical storage is that file systems do not grow easily. Because file systems rely on specific addresses and locations, it is difficult to move data around without introducing broken links. In addition, hierarchical file systems typically have less means of protecting themselves against corruption. If a segment of the hierarchy gets corrupted, it might no longer be possible to access some of the information below. Object Storage systems are one of the key components of the modern cloud infrastructure. Data is stored as an object and the application storing it receives a

global unique identifier (aka GUID). This means that unlike a structured file system, there is no way to know where a particular file is actually stored. In order to access a file, it is necessary to provide the object’s GUID. Object storage is designed to hold massive amounts of unstructured data and offers great benefits including: scalability, advanced data protection

techniques like self healing, automated replication including replication between geographically dispersed data centers, high availability, and improved search capabilities compared to hierarchical file systems. The RESTful/S3 API web access to object storage makes data stored accessible from anywhere in the world over the Internet. Object storage is used by Google and Facebook to keep your photos, as well as by Office 365 and G Suite.


The benefits of object storage make them suitable not only for the Internet application giants, but increasingly for small, midsize and enterprise-class organizations. More and more businesses use or consider using object storage. Many cloud services offer relatively inexpensive public or private repositories, or on-premises private cloud solutions. The fact that object storage can easily be accessed remotely through the Internet makes it an attractive option for offsite data backup, or as a platform for various data services.

The challenge of using object-storage with traditional applications

When using object-storage, the burden of organizing and presenting the information in a logical and intuitive way is left to the individual applications. This is why smartphones don’t store content using a global directory structure, but let each app organize data in its own way. While object storage is great for storing and searching unstructured data, it is more suitable for Write Once Read Many (WORM) scenarios. In most cases, object storage cannot handle modifications of data in place. Objects have to be manipulated as a whole unit. Usually modifying an object involves retrieving the existing object, modifying it locally with the respective application and re-uploading it as a new object. This can have performance implications as well as cost related to the price associated with uploading and downloading objects.

Hierarchical Storage Object Storage

Price $$$ $ Data-type Structured Semi Structured + Unstructured

Typical use Millions of files/Terabytes Billions of objects/Exabytes Protocols Block-level (e.g. SCSI) HTTP REST API

Optimized for Fast random access Capacity Reads and writes on small records, and ability to modify data in place.

Scalability

Storage for hypervisors Geo-distribution Web accessibility

Typical apps Word processors, content creation apps, CAD systems...

Social media, video, photos, log files, web content, large data sets


Approach File System Entry = Datapath + file metadata → data stored at fixed location

Object = GUID (Global Unique Identifier) → data + associated metadata stored somewhere

Object storage is typically not suitable for random access of data in an object. For example, if a movie file is stored as an object, an application cannot read the data from the middle of it. It is necessary to download locally the entire file first and then access the required portion of data. Object storage cannot be accessed directly as a mounted drive in the operating system without significant performance degradation. In order to facilitate the integration of object-storage within traditional IT environments, vendors have developed gateway solutions that expose the cloud storage as an SMB/CIFS/NFS share (or iSCSI target). In this case, a hierarchical file system front-end gives access to the cloud storage. The StorSimple appliance provides such a gateway to Microsoft Azure. Some challenges associated with interfacing with Object storage using gateway servers include:

- The Object storage is presented as yet a new “silo” of data (i.e. a new mount point) that must be integrated into existing processes and workflows.

- Performance and/or compatibility of the SMB/CIFS/NFS protocol is often sub-optimal.

- Multiple gateways can be required where performance requirements are high.

- Internal caching may create racing conditions and latency issues. - Sending content over an SMB/CIFS/NFS protocol is not secure. - Many applications do not support, or work well using SMB/CIFS/NFS

protocols.

Tiger Bridge: the perfect hybrid solution

Tiger Bridge makes it easy to seamlessly extend local drives into the cloud in a way that not only facilitates migration but also provides geo synchronization.


By offering the ideal integration of hierarchical storage and object storage, Tiger Bridge does not just keep an index of the object-content (as is the case when accessing the SMB/CIFS/NFS share of an object-storage), but also acts as a local cache.

In this example with Tiger Bridge, the local storage mirrors the content in the object storage. Importantly this content is now associated with filenames and folders. Thus users can easily “browse” the contents of the object storage and move files using the file browser in their operating system. By analyzing access to files and by relying on simple policies, such as last access date, file size, and how much free space should be retained on the

primary server, Tiger Bridge determines when a newly created or updated file must be replicated or when a stale file must be tiered.

A seamless workflow Users and applications are not affected when files are migrated from the local server to the object storage as stub files are created to replace any files that were moved. Stub files are unique and contain all metadata from the original file (size, permissions, last access date, etc.). Stub files are accessed by users and applications just like any other file. The state (i.e. normal, replicated or offline) of a file is clearly visible to users through a file system extension icon, something not possible when going through an external gateway. When a stub file is accessed, the original data is automatically and transparently restored back to the local file system and handed to the application, just as it normally would. Therefore, users and applications do not experience any disruption in their workflow. Because Tiger Bridge runs directly on a Windows server and manages the existing NTFS file system, there is no need to reformat the storage, give up


performance or compromise security by going through an external gateway. Tiger Bridge ensures strict adherence to Active Directory preserving security throughout. Data is moved between primary and secondary tiers of storage using SMB/CIFS/NFS protocol or a RESTful/S3 API. When connecting to a target over a RESTful/S3 API, data is encrypted directly on the server via https. This is accomplished before it is sent to the cloud ensuring the highest level of security.

Tiger Bridge extends the range of applications that can make use of cloud storage.

In environments where applications do not run on Windows servers, it is extremely easy to turn Tiger Bridge into a traditional cloud gateway appliance. In this case the hierarchical storage still resides on the Windows server, but is made available via standard file sharing protocols such as SMB/CIFS/NFS. Multiple users and clients can then access the object storage data transparently while enjoying all the benefits of hierarchical storage.

Unique Azure Integration In addition to supporting DAS, NAS, SAN and Tape targets, Tiger Bridge supports the most popular cloud providers (Amazon, Azure, Backblaze, DDN, Wasabi, etc.). The level of support and integration can vary from one cloud provider to another. For instance, Tiger Bridge supports seamless data movement between Microsoft Azure’s three blob layers (Hot, Cool, Archive) while preserving Active Directory. Bridge also fully supports Azure Snapshots and Soft delete features providing the most powerful and transparent interface to Azure.


Keeping multiple servers in sync Tiger Bridge makes it easy to keep any number of servers in sync. These servers can be local or across the globe (Geo Synchronization). Servers only need to connect to the same target (NAS, SAN or Cloud) in order for their file systems to automatically be kept in sync. Content that is written on any one server will not only be replicated to the target, but will also automatically appear - like magic - locally on all other servers. By default, only a stub-file will be created on remote servers (data will only be downloaded when the stub-file is opened), but it is also possible to configure Tiger Bridge to perform auto-retrieval of all files. This unique and advanced sync mechanism lets you maintain two - or hundreds of servers - in perfect sync.

Leveraging the scalable processing power of the cloud Some popular cloud gateway solutions push blocks of data to the cloud instead of files. While this is transparent to local users and servers, such an approach requires block level storage to be pre-allocated and formatted as volumes of fixed sizes in the cloud. Block storage often provide for deduplication capabilities, but is expensive and lacks flexibility. And although deduplication makes sense for storing databases and virtual machines, it offers very little value when working with video and pictures. More importantly, block storage precludes any virtual machine running inside the cloud to gain access to this data set for processing and analytics. For all intents and purposes the data is stored inside block storage can only be accessed by the local server that is connected to it. From the cloud perspective, this data is proprietary and unusable.

Because Tiger Bridge pushes files to the cloud rather than blocks, it enables a far more powerful hybrid architecture to be deployed. In addition to serving as the extended native storage of a local server, Tiger Bridge makes it possible to fully


leverage the power of cloud computing to process this shared data. Complex tasks such as big data analytics, facial recognition, and video transcoding can take place in the cloud, while the local server maintains access to all the files for local use. And given the file structure is identical on both the local server and the cloud, it is very easy to automate such complex processes. As such, Tiger Bridge stands out as the ideal solution for anyone working with digital content that wants to benefit from scalable cloud processing.

Key features ● Runs on any Windows 7, Windows 10 or Windows Server 2008/2012 R2/2016 ● Installs as an add-on software connector on existing file servers ● Content pushed to the cloud can be manipulated inside the cloud by virtual

machines ● Provides visual status of files through a custom icon extension to the file system ● Does not affect server performance ● Does not need to crawl the file system to keep track of changes ● Primary storage can be NTFS file systems (as well as Tiger Store and Tiger

Pool) ● Supports native Active Directory security authentication ● Capable of Full and Partial file retrieval ● Real-time file system monitoring ● Optimized for high performance environments ● Restores data and metadata from object storage to local file system ● Supports Geo Synchronization of any number of servers ● Supports a wide variety of object storage through the RESTful/S3 API

▪ Amazon AWS


▪ DataDirect Networks WOS ▪ IBM Cloud Object Storage ▪ Microsoft Azure ▪ Spectra Logic BlackPearl ▪ ...and additional providers

Three simple phases for transitioning to the cloud Tiger Bridge helps organizations transition their legacy infrastructure to the cloud in three easy and seamless phases. If you want to transition quickly, just proceed through these various phases. You will be comforted to know that you can easily test and pause anywhere along the way without the fear of experiencing downtime or frustration.

The above diagram represents a typical on-premises file server with local storage and clients. This is what a typical setup looks like prior to any cloud integration.

Phase 1 - Replicate to the cloud The first phase of your migration should consist of replicating your existing data to the cloud using blob storage. Your server and clients remain on-premises.

At this stage, you already benefit from a reliable backup to the cloud.


Phase 2 - Operate an hybrid setup The second phase consists in provisioning a virtual server with a minimum amount of block-level storage and a few clients in the cloud.

Using Tiger Bridge, the virtual server can remain in perfect sync with the on-premises server. The bulk of your data is stored on blob storage. You can now start moving some of your operations and processes to the cloud, while continuing to run the remaining tasks on-premises without interruption.

Phase 3 - Complete the migration When you are ready to finalize your transition, just move your remaining processes and cpu to the cloud.

When your migration to the cloud is complete, you can decommission your old hardware.

Applications

Tiger Bridge is very flexible and powerful enabling it to be used in a myriad of ways. Whether seamlessly expanding, migrating or decommissioning aging storage, facilitating collaboration between multiple branches and offices, or creating a hybrid infrastructure to enable Big Data analytics, Tiger Bridge does it all. Best of all, Tiger Bridge does it seamlessly and without disrupting your existing workflow. Here are just a few examples of what you can do with Tiger


Bridge:

Seamless Capacity Expansion Increasing storage capacity of an existing server can be a daunting task. as scheduling the required downtime to copy and migrate data to a new drive is always problematic. The alternative is to leave the current volume untouched and configure an additional volume. This can be difficult as well as it requires notifying users or reconfiguring services to make use of the new storage. Using Tiger Bridge, you can seamlessly expand your existing volume into a new, larger volume. It is literally like creating an overflow path between your pond and the ocean. Users can continue filling up the pond as it will never overflow.

Easy Storage Migration/Decommissioning Migrating from or decommissioning an aging storage infrastructure is another challenging task facing IT personnel, especially in environments operating 24/7. Using Tiger Bridge, you can seamlessly expand your existing volume into a new volume. Even if it takes a few days to replicate the entire content, you will only need to prevent writing during a few minutes to ensure all recently written data will propagate to the target. Once content is up to date on the target, you can simply decommission the old source volume and set the target as your new volume. Total downtime can be reduced to just a few minutes.

Quick-Restore Backup A simple backup solution is easily implemented with Tiger Bridge. Instead of relying on a fixed backup window (e.g. 3 AM), Tiger Bridge will perform an ongoing backup throughout the day and night, according to the replication delay that was specified. In the event of a system or drive failure, Tiger Bridge makes it easy to quickly restore all file system metadata and resume operations. This process is significantly faster than restoring the actual data because only file metadata needs to be restored. Once stub-files have been restored, operations can resume. Only data that is accessed gets restored; ensuring fast recovery from failure. Note that it is also possible to continue restoring all (or selected) data in the background as well, thereby providing a prioritized restoration process.


Zero-Downtime Disaster Recovery Restoring data from a traditional backup prevents an organization from resuming operations until all files have been restored. This can easily translate into days of downtime. In sharp contrast, Tiger Bridge provides virtually zero downtime when recovering from a disaster. When a disaster strikes, the replication site is ready to operate. Thanks to Tiger Bridge’s Geo Synchronization features, stub-files are already created and ready on the DR server and can be immediately accessed. Tiger Bridge running on the Primary site automatically replicates newly created content to the cloud. Seconds later, corresponding stub-files appear on the Tiger Bridge file system that is running at the Recovery site (both sites need to be connected to the same object storage). Data itself does not need to be replicated on the Recovery site for operations to resume, as the Recovery site can immediately start accessing file metadata. Only files that are needed at that very moment are retrieved. Given the bulk of data is usually stale, a complete restore of all data may never be needed on the Recovery site. As such, significant savings can be realized by reducing the size of the storage infrastructure required at the Recovery site.

To further speed the recovery process, Tiger Bridge’s Partial Restore functionality ensures immediate access to early blocks of data being requested by the applications while larger files continue to be loaded. When paying for data transfer to and from the cloud, substantial savings can be realized using this approach as only files being accessed are recovered.


Minimizing Block-Level Storage Utilization in the Cloud When running traditional applications in the cloud, a certain amount of block-level storage is required for deployment of a traditional file system. For Azure this storage is referred to as Managed Disks; for Amazon it is called Elastic Block Store (EBS). This type of storage is considerably more expensive and difficult to provision. Using Tiger Bridge on virtual servers in the cloud (as opposed to connecting to the cloud) enables substantial savings by minimizing the amount of block-level storage required while expanding seamlessly into less expensive and more manageable object storage.

Manipulating and Accessing Big Data Exposing existing cloud data sets over a traditional file system can prove challenging when dealing with Big Data. The major concern is the time it takes to scan petabytes of data in the cloud to map them onto a file system can take a vast amount of time. Tiger Bridge FastXploreTM is a unique feature that allows dynamically building the file system as users and applications navigate through the cloud data. Since only the objects of immediate interest are being scanned, one can quickly connect and start browsing extremely large cloud data sets through a file system interface.

Leveraging HSM Storage in the Cloud Microsoft Azure Archive provides multiple tiers of object-storage storage for Hierarchical Storage Management (HSM). Tiger Bridge features the ability to automate data movement between various tiers of cloud storage (as well as between various providers). This transparent and optimized lifecycle management translates into significant cost savings.

Rich Media & Entertainment New resolutions keep driving the need for more speed and storage, as well as for ways to locate material and collaborate more efficiently over different geographic locations. Online, production-grade storage always comes at a premium because it is designed for high-speed. Instead of using it for holding dormant projects, Tiger Bridge enables rich media organizations to minimize the high-speed storage they actually need for their active jobs and push dormant projects to more cost-effective and resilient object-storage. Tiger Bridge enables the syncing of multiple servers (through their local file system) when connected to the same object-storage. In the diagram below, Tiger Store also runs on the Tiger Bridge


server and acts as a file-level SAN management metadata controller. This provides all edit suites with simultaneous access to the same LUN over Fibre Channel for maximum performance.

Once clips have been pushed to the cloud, a network of virtual machines can be used to perform complex tasks such as object recognition, advanced visual processing and transcoding.

Video Surveillance Booming demand for security means it is no longer exceptional to find thousands of cameras on a single location sending all their feeds to a few recording servers. However, the bulk of video surveillance data has no value… until some critical evidence must be retrieved from it. By presenting the object-storage as an extension of the local NTFS file system – as opposed to an external SMB/CIFS/NFS share, Tiger Bridge enables surveillance applications to interact with object storage in a completely transparent way.


Object storage is ideal for video surveillance because it allows evidence to be preserved off-premises and accessed from anywhere. Tiger Bridge allows traditional network video servers to write video feeds to a local drive (or SAN) before being replicated to the object storage. When object storage is off-premises, this buffer ensures that all evidence is recorded prior to being encrypted and transferred to the cloud. As Tiger Bridge runs directly in the server’s memory, https encryption takes place before any critical data goes out on the network, thereby ensuring the highest level of security.


Once video has been pushed to the cloud, virtual machines can manipulate the data and perform tasks such as facial and object recognition.

Conclusion

Tiger Bridge is a high-performance, secure and flexible software connector that enables object based storage to be integrated as an extension of familiar block-level storage. By overcoming the limitations associated with traditional gateway technologies, Tiger Bridge lets organizations of all sizes immediately and easily benefit from the scalability, resilience and cost savings that are driving the rapid adoption of object based infrastructures. Any organization can now benefit from an optimized workflow experience both locally and by enabling data to be submitted to cloud-based computing while significantly reducing overall costs.


technical brief › docs › tiger-bridge › getbridge › ti… · hybrid architecture capable of...

Documents