Alternative Storage Technologies

Whitepaper

THE COMPLEX NEEDS OF PRESERVATIONThe world is overflowing with digital data; in fact the digital universe is doubling every two years according to IDC1. A share of this digital universe has value in the long term.

This is for instance data that is considered part of our cultural heritage, strategic business infor- mation or data that must be preserved for legal reasons. In order to ensure long term survival, these data need to be treated differently than short-lived information.

Do the existing storage technologies meet the complex requirements involved in long-term pre- servation of data? What are the unique character- istics of the available alternatives? These are some of the issues we will look into. In our evaluation of the alternatives we will in particular focus on the following aspects:• How well is data security and integrity maintained?• How easy will it be to access the data in the future?• How cost-efficient is the technology in a long term perspective?

Alternative Storage TechnologiesWhitepaper

What does «digital» really mean?

When a file is stored digitally, the data is stored as binary codes, i.e. sequences of numbers (0 and 1,) which are written on to a storage medium. By decoding these numbers, a perfect copy of the original file is brought back. “Digital storage”, including the Cloud, normally means digital application of analogue storage medium.

1IDC, “The Digital Universe of Opportunities”, April 2014www.emc.com/leadership/digital-universe/index.htm

Alternative Storage TechnologiesWhitepaper

Linear Tape Open (LTO) is a magnetic tape storage open technology that was developed in the late 1990s by HP, IBM and Certance (now Quantum). LTO is widely used for mass storage, and the latest generations (LTO 7) can store up to 6 terabyte. For long-term preservation purposes, it is important to be aware that data capacity increases for each generation, but this has a negative impact on the expected lifetime. As for other magnetic tapes, LTO requires a migration-based archiving strategy in order to ensure data safety and accessibility.

2Fujifilm, “LTO Ultrium Technology” 2012, www.fujifilmusa.com/shared/bin/LTO_Data_Tape_Seminar_2012.pdf

3LTO Ultrium readers are two generations backward compatible

MAGNETIC TAPEWhat it is: Magnetic tape is an analogue medium for magnetic recording, made of a thin magne-tizable coating on a long, narrow strip of plastic film. The plastic film is coated with magnetic material, i.e. ferric oxide powder which is mixed with a binder to attach it to the plastic. It includes a dry lubricant to avoid damaging the tape drive. The magnetic coating makes it possible to retain electronically encrypted data in digital format. The tape is normally packaged in a plastic cartridge for protection and tape drives are used to read and write the data.

Cheap mass storage: Magnetic tape allows largeamounts of digital data to be stored at a relatively low cost, making it well suited for a mass storage purpose.

Handle with care: When using tape for archiving purposes one has to bear in mind that tape is a sensitive material that requires proper handling and environmental conditions. For instance, mag-netic fields are obviously a threat to any magnetic storage, and smoke and small particles can cause damage and loss of data. It is therefore imperative to have clean operating conditions, with the right temperature and humidity levels. With minimal usage, handled properly and stored in optimal conditions, magnetic tapes have a lifetime expec-tancy ranging between 10 to 30 years.2 However, due to the delicate material and its high density of storage, the risk of damage and failure increases each time the tape is used. Consequently, the usage should be

kept to a minimum, and it is considered “best prac-tice” to migrate the data at least every 5-10 years to newer tape formats.

Migrations and vendor lock-in: A drawback withmagnetic tape is that data retrieval is depen-dent on specific reading devices, i.e. tape drives, which need to be maintained. New tape drives are normally not able to bring back data from older generations of tapes3. This means that when using magnetic tape for long term preservation, regular and endless investments in migration is needed.

OPTICAL DISCSWhat it is: Optical discs are flat circular disks that encode digital data on one of its flat surfaces. They are usually made of aluminium, but for preserva-tion purposes it is preferred that the reflective lay-er of the optical disk is gold4. Optical disc drives(ODD) with laser beams are used for engraving the data onto the disc, and the laser light is also used for reading back the data whilst the disk spins at high speeds.

CDs and DVDs: The two main types of optical discs used by archivists are CDs and DVDs. These can be further split into:• CD-R and DVD-R: Write once, read many• CD-RW and DVD-RW: Rewritable• DVD-RAM: Rewritable discs formatted for random access (like a computer hard disc)

Ease of use: Optical discs such as CDs and DVDs are easy to use, and both the media and the hardware are relatively inexpensive to purchase. In addition, the laser-based device reader is not in contact with the disc which makes mechanical failure less likely, and most of the readers are backward compatible. These factors have made optical discs widely used by archivists.

High failure rates: Originally developed primarily as a mass consumer product, elaborate measures are required when using optical discs for long term preservation. Optical discs have a relatively high risk of failure5, and it is strongly recommended to always do thorough testing to ensure required standards are met. Scratching and environmental factors such as dust, heat or UV light can cause severe damage to the disks. Optical discs also offer low data capacity compared to alternative tech-nologies.

Frequent migrations: It is challenging to reliablyestimate the real lifetime of optical discs. They arefragile, and the life span depends on factors such as manufactured quality, how well it is recorded and its physical handling and storage. To lower the risk of failure, best practice is to frequently migrate onto newer formats. This is in sharp contrast to manufacturers who tend to claim an expected lifetime of up to several hundred years.

Alternative Storage TechnologiesWhitepaper

Blu-Ray is an optical disc designed to supersede the DVD format with a higher capacity. However, it is more popular among households than professional archivists.

M-Disk is an optical disc which was introduced in 2013, claiming a lifetime of 1000 years. However, a Blu-Ray reader is needed for data retrieval, and the future accessibility of these is a challenge the manu-facturer does not give an answer to.

4DPC; “Handbook in Digital Preservation”; www.dpconline.org/advice/preservationhandbook/media-and-formats/media

5 “Risks Associated with the Use of Recordable CDs and DVDs as a Reliable Storage Medium in Archival Collections – Strategies and Alternatives”, Memory of the World Programme, UNESCO, 2006; http://unesdoc.unesco.org/images/0014/001477/147782E.pdf

Alternative Storage TechnologiesWhitepaper

HARD DISK DRIVE (HDD)What it is: A hard disk is a rapidly rotating disk used for storing digital information. The data is recorded by magnetizing a thin film of ferromagnetic mate-rial. A read/write head on an arm accesses the data when the disk is spinning.

Instant access: Hard disks provide instant access to the data and the reading and writing device is an integrated part of the storage medium. They enable cost-efficient storage of large amounts of data, with a storage capacity of up to 10 terabytes per disk (2016).

Short durability: High failure rates make hard disksinappropriate for long-term preservation. A recent study projects an average lifespan of six years.6

It is therefore common to combine a number of disks for redundancy, also known as a RAID. From a mechanical point-of-view, a frequent issue is that the disk will not spin, and the failure rates increase with age. Another common reason for failure is

that the read/write head scrapes the rotating platter and thereby causes damage and data loss. Furthermore, a hard drive is a magnetic storage technology and its magnetic strength will natural-ly deteriorate over time when placed offline for archiving upon a shelf.

Solid-State Drives (SSD) use flash memory for digital storage, unlike HDD’s magnetic technology. They are more mechanically robust than HDDs and require less power, but have less data capacity and are more expensive in use.

SSD is well suited for portable devices and applications requiring fast data access, but are not considered to have the attributes required for long-term digital preservation.

6 http://blog.backblaze.com/2013/11/12/how-long-do-disk-drives-last, retrieved November 21st, 2013

THE CLOUDWhat it is: “The Cloud” is a term used for storage services where the data is stored in virtualized pools operated by third parties. These hosting companies operate large data centres, filled with servers, which could be located anywhere in the world. They are normally subject to strict security measures such as restricted access, environmental control and emergency backup power supply.

Store and ignore: For the end user, the Cloud offers clear advantages such as instant access, ease of use with no need for maintenance, and it is inexpensive to buy or lease storage space. This makes it well suited for basic storage needs. How-ever, for the valuable data that requires secure,

long-term preservation the Cloud involves some major concerns related to security, privacy and ownership.

Ownership and privacy concerns: The Cloud in-volves entrusting a third party to store the data. This means the service provider has access to the data and may accidentally or deliberately disclose or alter the information. The service provider may also go out of business, posing questions as to who owns the servers and the data on them. The data owner will in this situation have limited control, and the data is inaccessible from the moment the network connection or power supply is switched off.

Security issues: Data accessible through the web is vulnerable to hacking. Although the service providers strive to increase the security level, there is no guarantee that the data is out of reach of individual hackers, companies or national security agencies that might have an interest in accessing the information.

Some data centres offer offline back-up, normally in the form of magnetic tape, in addition to cloud storage. This makes the data more secure, but a migration-based archiving strategy is then needed when the time perspective is long.

Amazon Glacier is an online file storage service developed for long-term storage of data that does not require instant access. Like other Cloud ser-vices, Glacier is based on hard drives, but the data retrieval time is 3 to 5 hours according to Amazon. Storage is cheap, but users are charged for retriev-ing the data hence the risk of vendor lock-in should be considered. The privacy, security and ownership concerns related to cloud storage also apply to Amazon Glacier.

Data centre with servers

Alternative Storage TechnologiesWhitepaper

MICROFILMWhat it is: Microfilm is 16mm or 35mm film stored on open reels or in cassettes. It contains small images, usually in black and white, although some microfilm formats also support digital data. The two main types of microfilms are:

• Silver halide film: The recommended alternative for long-term preservation. The image is trans-ferred to the film by using silver emulsion on a polyester strip.

• Vesicular film: Creates the image on the polyester strip by using microscopic bubbles instead of silver emulsion, making it a less expensive but also a less durable solution.

Special cameras capable of photographing at reduced size are used in order to transfer an image to microfilm. The image is then printed on the film and chemically processed in a laboratory. The film processing makes the recording process more complex compared with some of the alter-native technologies. In spite of this, microfilm is widely used for long-term preservation of data, for a number of reasons.

Long durability and no need for decoding: Microfilm has a lifetime expectancy of up to several hundred years when stored properly at the recommended temperature and humidity levels. Additionally, as the data is normally analogue, data stored on microfilm can be read back by simply using a magnifying glass. Hence data retrieval is independent of specific reading devices, and the images on the film require no software decoding. Desktop readers with large screens and zooming lenses are normally used for data retrieval. The downside is that it is a manual and cumbersome process to access and reproduce data stored on traditional microfilm.

A migration-free WORM: The durability and secure read-back of information written as images make microfilm a migration-free storage alternative, potentially creating large cost savings and elim-inating the risk of migration-related data loss. By being an analogue medium stored offline, micro-

film is also a secure solution. As a true WORM (Write Once, Read Many) it is literally impossible to edit or delete the information that is written to the film. Hence microfilm meet many of the criteria for long-term preservation.

Manual handling: The major drawback with micro- film is that the workflow for reproducing data is time-consuming and largely manual. Microfilm was designed for storing analogue images, and in a world overflowing with digital data, traditional microfilm is simply not an efficient alternative.

Microfilm reader

A microfiche has many of the same attributes as microfilm, but is formatted as a card rather than as a film reel. They are normally stored in open top envelopes in drawers or boxes.

An ultrafiche is a compact version of a microfiche, storing images at much higher densities and often made directly from computers.

Alternative Storage TechnologiesWhitepaper

Alternative Storage TechnologiesWhitepaper

PIQL’S PRESERVATION TECHNOLOGYWhat it is: Piql offers Preservation Services based on a turnkey system designed to comply with the needs of long-term digital preservation. The result is a technology that allows digital data to be preserved safely and efficiently, and easily retriev-able independent of future access to specific technologies or vendors.

Piql uses high-resolution photosensitive film as a digital storage medium. Data is written to film as large QR-codes, each containing 8.8 million pixels. This allows any kind of data to be preserved offline, on a storage medium with a documented lifetime of 500 years.

Piql has developed all software and hardware needed to write and read back data. Yet there is no vendor lock-in as the decoding software is open source and the data can be retrieved by using any digital camera and computer available in the future.

Future-proof WORM: Piql’s solution offers many of the same preservation qualities as microfilm, but applied to digital data. As a true WORM stored offline, it is impossible to manipulate or delete data once it is written. Digital data written as 0 and 1 obviously need decoding to get back the original file. To make data retrieval future-proof, explanations of how to decode and retrieve the information is written as human readable text on

Data preserved in digital and visual formats

Alternative Storage TechnologiesWhitepaper

the storage medium. By being self-contained with a documented lifetime of 500 years, valuable data can be preserved for the unforeseeable future.

The technology is also migration-free, both in terms of storage medium, reading device and file formats (provided that archival formats are used). The latter is due to the fact that file format specifications can be written in readable text on the storage medium.

For digital and visual formats: Piql’s technology differs from traditional microfilm in enabling digital data to be stored alongside visual images, making it suitable for the digital era we have entered. The flexibility in combining digital and visual formats opens up for new possibilities and additional security, such as visual previews of digital files.

Search and access files: Piql’s technology allows for metadata searches and accessibility on a different level than traditional microfilm. As a seamless element within a standardIT infrastructure, users can search for the requested file and get it back in original format within minutes

(retrieval time will vary according to each client’s preferences).

Migration-free long-term preservation: Piql’s preservation technology is not recommended for data that needs to be instantly accessible to ensure business continuity. It is rather a secure, migrationfree and future-proof option for valuable data that needs to be preserved long-term.

Piql’s film and box are tested to ensure the preserved data will be kept intact for

hundreds of years.

DIFFERENT TECHNOLOGIES FOR DIFFERENT NEEDS

We have seen that each of the storage technologies have different characteristics, making them suitable for different needs. Data owners need to thoroughly consider their requirements in terms of e.g. acceptable data retrieval time, security level and budget limitations during the archival period.

The following table provides an overview of some of the attributes of each of the storage technol-ogies. It is important to keep in mind that this is a general overview of technologies constantly in development. Within each category, there are differences between vendors and products.

Optical Disks

Magnetic Tape

Hard Disk Drive

Cloud Microfilm Piql Preservation Services

Projected lifetime7 30 100 6 N/A 500 500+

Digital data

Analogue data8

True WORM9

Offline format11

Instant retrieval

Vendor-neutral reading device

Migration-free storage medium

True archival format12

10

7General figures, there are variations from vendor to vendor

8Human readable text or pictures

9Write Once, Read Many, meaning the data is unalterable after being recorded on the storage medium

10CD-R and DVD-R only

11Does not require any maintenance of the medium to ensure data integrity

12Self-contained format: descriptions of how the data is stored, file formats, information on how to decode the data, etc.

Piql secures long-term future access to valuable digital data. Head-quartered in Norway since 2002, Piql Preservation Services are offered to clients around the world by the use of technology that converts high-resolution film into a digital preservation medium for the future.

Piql AS • Grønland 56, N-3045 Drammen, Norway • tel +47 905 33 432 • office@piql.com • www.piql.com