toward digitizing all forms of documentation

8/4/2019 Toward Digitizing All Forms of Documentation

http://slidepdf.com/reader/full/toward-digitizing-all-forms-of-documentation 1/11

Search | Back Issues | Author Index | Title Index | Contents

D-Lib Magazine

March/April 2009

Volume 15 Number 3/4

ISSN 1082-9873

Toward Digitizing All Forms of Documentation

George V. LandonEastern Kentucky University<[email protected]>

Abstract

Large-scale book scanning projects are delivering unprecedented access to the majority of library holdings bygiving users unparalleled access to vast collections of books. However, these efforts have focused on typical

bound books, which leaves many forms of documentation out of the digitization framework. We presenttechniques that we have developed to digitize numerous other forms of documentation, including deterioratedmanuscripts and photography. These technologies are engineered in conjunction with budgetary and physicalconstraints often placed on digitization projects.

Introduction

Recent large-scale document digitization initiatives have created new modes of access to modern librarycollections with the development of new hardware and software technologies. Most commonly, thesedigitization projects focus on accurately scanning bound texts, some reaching an efficiency of more than onemillion volumes per year. While vast digital collections are changing the way users access texts, currentscanning paradigms cannot handle many non-standard materials. Documentation forms such as manuscripts,scrolls, codices, deteriorated film, epigraphy, and rock art all contain a wealth of human knowledge in physicalforms that are not accessible by standard book scanning technologies. This great omission motivates our development of new technology for digitizing deteriorated bound works, damaged manuscripts, anddisintegrating photographic negatives that remains cost-effective and easily utilized by non-technical staff.

After a brief overview of the current progress that large-scale digitization initiatives are making, we willaddress particular issues that arise when scanning damaged and/or deteriorating works. Furthermore, we

demonstrate new methods we have developed for digitizing and virtually restoring two disparatedocumentation forms: manuscripts and photographic negatives.

Large-Scale Digitization Initiatives

The quest for amassing all of the printed works in the world is nothing new. The famous Royal Library of Alexandria, 300BC, began with the goal of acquiring all of the known world's knowledge (Rollin, 1815). Withan estimated 400,000 to 700,000 volumes in the collection before the library's destruction, this goal seemedattainable.

According to the Online Computer Library Center (OCLC) WorldCat bibliographic database, there are 32

million records describing printed books in American libraries (Lavoie, Connaway, & Dempsey, 2005). Thistranslates into at least 32 million unique books that must be digitized in the United States alone.

From a global perspective, attempts have been made to estimate the number of books printed in the world.Through methodical research and a fair number of extrapolations, it is thought that the world inventory of

ward Digitizing All Forms of Documentation http://www.dlib.org/dlib/march09/landon/03landon.html

11 21/05/2011 11:54 PM



original books might be between 74 million and 175 million books (Lyman & Varian, 2003). However, thisapproximation does not include scrolls, manuscripts, and other documentation forms.

For the millions of books currently available on library shelves, recent projects have begun the daunting task of providing access to these volumes through new digitization and delivery methods. These projects havesuccessfully started a trend to provide the world easy access to all printed materials. However, the world'sdigitization efforts should not be focused solely on the textual information of written or typed volumes. Tofully understand what is lacking, one must first recognize the accomplishments that have already been made.

Google Book Search

In 2004, Google Inc. began Google Book Search, an effort to digitize and provide online access to virtually allof the world's books. The project claims to be able to digitize one million books per year ( Google, Inc., 2007)with approximately 7 million books digitized to date (Rich, 2009).

The goal of Google Book Search is to make textual content in books searchable. This requires a great deal of image processing to accurately segment text from the background or illustrations. However, by convertingthese document images into easily segmented text, much of the contextual detail, such as graphics andillustrations, are greatly compromised.

An example scan from Google Book Search.

The original imagedtitle page.

The post-processedversion included in the

digital library.

The region showing compromised contextualdetail.

While the omission of this content is understandable in practice, it does limit the overall presentation of avolume. Moreover, it should be noted that Google's digitization technology is not without restrictions. Fromthe Google Book Search Policy (Google, Inc., 2007):

Google developed innovative technology to scan the content without harming the books. Any book deemed too fragile will not be scanned by Google, but may be treated by expert library staff.Once scanned, all print volumes are returned to the library collections.

Unfortunately, it appears that Google's digital library will not include many fragile works, which includes older volumes that have deteriorated with age.

Open Content Alliance

Started as a venture between the Internet Archive and Yahoo!, Inc. in early 2005, the Open Content Allianceis also working to make digital collections available. Focusing on improving availability of the content, theOCA has attracted contributors that are willing to donate their already digitized collections. Therefore, much


11 21/05/2011 11:54 PM



of the collections are determined by the contributors as is the digitization technology employed.

Europeana

The European digital library, Europeana, was started in July 2007. While the initial digital collection containsroughly 2 million texts, images, audio files, and movies, the number is projected to reach 6 million items by2010. The goal of Europeana is to amass all of Europe's documented cultural heritage by partnering with thecontent holders. Consequently, this initiative relies mainly on partner organizations to provide the digitized

documents and metadata.

Digitization is not a One-Size-Fits-All Process

As the Google Book Search Project suggests, some texts are "deemed too fragile" to be handled and scanned by technicians, but "may be treated by expert library staff (Google, Inc., 2007)." To remove this extradistinction between documents, it would be more appropriate to develop systems accessible to the expertlibrary staff. It is in the best interest of all parties involved in digitization projects to leave the handling of

priceless works to those who are charged over them. Therefore, digitization is a task that should be available tolibrarians and curators.

Expanding notions of which documents need digitization and preservation also changes the perception of content for standard textual documents. Consequently, projects that focus mainly on document text lose muchof the meaning and contextual information. This importance was addressed by the Library of CongressManuscript Digitization Demonstration Project: Final Report (Library of Congress, 1998).

The committee discussed the degree to which an image need replicate the look and feel of anoriginal manuscript (or other archival) artifact. Although most members agreed that Library of Congress treasures, e.g., drafts of the Gettysburg address, warranted a kind of museum-qualityfacsimile, there was less consensus that such treatment was warranted for routine documents,especially the kind of twentieth-century typescripts that form the greatest portion of the FederalTheater Project collection.

This argument resounds for the digitization of all historical documents. Therefore, document digitizationcannot be viewed as a standard procedure that excludes many document types and compromises what contentis preserved. Consequently, this drives our research to accurately capture, and in many cases restore,documentation in all its multifarious forms. A discussion of this research follows here.

Effects of Digitization Error

To engage in the discussion of document digitization, one must first understand the main problem that arises indigitizing collections. New technology has focused on scanning documents that are not compatible with normaldigitization technologies. Due to deterioration and damage, it is impossible to place many older texts in a

flatbed or book scanner. Consequently, these degradations not only affect the digitization technology but alsogreatly reduce the quality of the acquired image.

There are two main sources of error commonly encountered during damaged document digitization: photometric and geometric. We will now discuss recent developments to provide corrections for these types of digitization errors.

Error Cause Effect

Photometric Non-uniform lighting Inconsistent shading of content

Geometric Warping of the substrate Skewing of the content

Photometric Correction of Documents

Photometry measures the perceivable intensities of light by the human eye. The term photometric correction,


11 21/05/2011 11:54 PM



Newspaper with light probe

Close-up of original Close-up of correcteddocument

The paper light probe is a simple piece of white letter paper with uniform folds as shown at the bottom of theleft figure. We introduce this form of a light probe due to its ease of construction and readily availableconstruction material. The 3D shape information of the probe will provide enough data to calculate the 3D

position and color of the light source illuminating the scene.

A 3D scanner is used to acquire the shape and color information of both the light probe and the document to be digitized. To restore the color information, the light position and color are calculated using the knownsurface normals, imaged color, and known color of the paper light probe to solve the general irradianceequation. Once the light source color and position are known, the intrinsic color of the document can berestored using the document's surface normals and imaged color.

The entire system uses the following components:

3D scanner ( NextEngine, Inc., 2009)

laptop computer

folded piece of white paper

single high-intensity light

The process is performed automatically with a total equipment cost of less than $5,000. Therefore, library staff can acquire digital scans of a warped document and easily restore its color information without additionalintensive training or organization resources.

When the document is corrected photometrically, the color distortions perceivable by the observer arediminished. However, for many document applications, the uncorrected geometric distortions still reducecontent detection and legibility.

Geometric Correction of Documents

Acquisition and restoration of general document types has been given focus by many groups who have made agreat deal of progress in creating fast and accurate digitization systems.

While others have relied on assumed document shapes to provide photometric and geometric corrections for objects such as bound books (Cao, Ding, & Liu, 2003) and folded documents (Brown & Tsoi, 2006), someworks have focused mainly on geometric correction of arbitrarily distorted documents (Zhang, Zhang, & Tan,2008) (Brown & Seales, 2004).


11 21/05/2011 11:54 PM



Illuminated Latin Bible 1407 AD by Gerard Brils, Belgium.(Photo Courtesy A. Pingstone 2005)

The Latin Bible, shown above, contains low-frequency warpage, which reduces legibility for some areas of thetext.

Discussion of the geometry of a document simply refers to its physical shape. Digitizing non-planar documentswith standard methods introduces shearing and skewing distortions of the content. For modern texts, damaged

volumes can be physically manipulated to reduce these geometric distortions. However, for conserved texts,this type of handling is not an option.

There have been two main approaches to correcting geometric distortions in document images.

First, image-based methods rely on a combination of assumptions about the content/context with knownconstraints on the document. These methods look either at the actual content of the document to determinewhere distortions occur or at the document composition such as distorted paper edges or bindings. A model isfit to these distortions, which can be reversed. This inversion effectively reduces these distortions for manydocuments.

Second are physically based simulations of document unwarping. Using the complete three-dimensionalgeometry of the document, the surface is modeled as malleable material that will deform when interacting withexternal forces. Many examples exist where the geometry and composition of the document surface iscomplex enough to prohibit accurate three-dimensional and image-based acquisition. Moreover, manycommercial 3D modeling applications now include these advanced simulations under the guise of cloth

modeling . We have had particularly good results flattening a document with the Autodesk 3D studio Maxcloth simulation tools.

The combined application of the photometric and geometric restoration techniques corrects imaging errors for many documents. However, as technology stands today, there are a large number of documents excluded fromthis acquisition and restoration technique. For example, paper with fine folds and ridges is often difficult toaccurately digitize with scanning equipment available to the practitioner.

A New Paradigm for Document Scanning

Many historical documents contain deteriorations in the surface, such as wrinkles and creases that havedetails, which are undetectable by many 3D imaging devices. In these cases, we have developed an advancedmodel for each digitized pixel in the scanned image, which provides a technique to compensate for lack of resolution in the document sampling.


11 21/05/2011 11:54 PM



New document scanner uses onlya laptop computer and digital camera.

This pixel-wise image-based acquisition and restoration methodology exploits the transmissiveness of documents to obtain the intrinsic color and distorted shape (Landon, Seales, & Clarke, 2008). The digitization

process uses

an LCD monitor, and

a Digital video camera

both controlled by the same computer to effectively scan the object.

The document is placed on top of the display, and light patterns are rear-illuminated on the document. Asthese patterns are displayed, the camera observes the illumination distortions introduced by the document, and

an overall model is generated to compensate for these discrepancies. An assumption is made that the scanneddocument transmits light and does so in a diffuse manner, which means that light leaves the surface of thedocument at a uniform intensity, at all angles, from the surface.

An initial, one-time-calibration scan is performed to build a baseline dataset for the scanner. Then documentsare placed on the laptop display, and the light patterns are repeated. It is the change in light transmission whenthe document is in place that generates the model for the document content.

The generated model for each pixel allows the photometrically correct content to be extracted directly. Inaddition, the model provides document shape information that can be used to restore the shape of ageometrically warped substrate with geometric restoration procedures.

Deteriorated phonedirectory

Acquired shapeinformation

Close-up of aregion of the

directory

Acquired documentcontent

Our image-based technique provides a direct way to generate restored images without knowing any additionalinformation about the document or the scene set-up. Consequently, this allows full access to the scanningtechnology in terms of required skill set and cost effectiveness. The operator needs to roughly align the


11 21/05/2011 11:54 PM



For an overview of photograph restoration techniques, the reader may refer to (Stanco, Ramponi, & de Polo,2003). However, these methods generally focus on a scanned image of a photograph and usually only handlestandard photographic prints. One project that does work directly with glass plate negatives (Stanco, Tenze,Ramponi, & de Polo, 2004) uses rigid transformations to assemble broken photographs.

The standard restoration technique for acetate negatives requires a chemical-based physical separation of layers in a permanent and destructive manner. While this technique provides the restored image, it requires a

vast amount of expert time and leaves the photograph forever altered. It should also be noted that thedegradation of these negatives cannot be stopped; only slowed. Therefore, digitizing these historic photo-negatives for virtual restoration is a technology that must be researched and developed.

Imaged negative of a grave marker, Lexington, KY Positive image

Automatically acquired surface information Automatically acquired photographic content

Present digitization procedures are not deployable for these multi-layer documents. Due to a great deal of non-uniform separation between layers, the final photographic image contains many channeled photometricdistortions. However, our transmissive document scanner, already discussed, has been demonstrated toeffectively digitize these non-standard materials. Since this scanner works in a fully image-based modality, italleviates many complications that arise with complex materials stacked layer upon layer such as accuratethree-dimensional acquisition or feature detection. We show examples of negatives from the University of Kentucky Libraries Special Collections and Digital Programs.

As we show in the figure on the left below, the damage due to the so-called vinegar syndrome of acetatenegatives greatly reduces the quality of the negative print. However, the results of our scanning andrestoration method move much closer to providing the undistorted content of the original emulsion layer. By

analyzing the difference in light attenuation between the empty scanner and the scanner with a negative in place, we are able to extract two unique forms of data in addition to the typical digitized print. First, the ratioof light from the display area light transmitted through the empty medium to light from the display areatransmitted through the negative gives an estimation of intrinsic photographic content from the emulsion layer.Second, the shift in the perceived light source through the differing mediums gives an estimation of surface


11 21/05/2011 11:54 PM



angle. The figure on the right below shows all three of the distinct datasets acquired from a single negativescan using our scanner.

Zoom of uncorrected

negative scan (whitearea signifies data

loss due tocracking).

Photometricallycorrected content

with geometricflattening applied.

Once these datasets are performed, we are left with a data-set similar to the previously discussed method thatuses an off-the-shelf 3D scanner. Therefore, we can now apply the same geometric correction technique toremove shape distortions in the photometrically correct photographic content.

Conclusion

This work presents our research to develop new methods for document acquisition and restoration. Thesetechnologies will enable both large-scale and intra-institutional digitization initiatives to include deterioratingand damaged document types while retaining high-quality representations of the content. Documents that wereonce deemed too fragile or damaged to be digitized now have a solution to allow inclusion in digitization

pipelines.

Acknowledgements

The University of Kentucky Libraries Special Collections and Digital Programs generously provided access tothe deteriorating acetate negatives shown here.

Bibliography

Brown, M., & Seales, W. B. (2004). Image restoration of arbitrarily warped documents. IEEE Transactions on

Pattern Analysis and Machine Intelligence, 26 (10), 1295-1306. <doi:10.1109/TPAMI.2004.87>.

Brown, M., & Tsoi, Y. (2006). Geometric and shading correction for images of printed materials using boundary. IEEE Trans. Image Proc., 15 (6), 1544-1554. <doi:10.1109/CVPR.2004.115>.

Cao, H., Ding, X. Q., & Liu, C. (2003). A cylindrical surface model to rectify the bound document image. ICCV '03: Proceedings of the Nineth IEEE International , (pp. 228-233).<doi:10.1109/ICCV.2003.1238346>.

Google, Inc. (2007) CIC/Google Book Search Project: Frequently Asked Questions <http://www.news-releases.uiowa.edu/2007/june/060607google-faqs.pdf >.

Harry Ransom Center: The University of Texas at Austin. (n.d.). The Digital Gutenberg Project . RetrievedJanuary 15, 2009, from <http://www.hrc.utexas.edu/educator/modules/gutenberg/books/legacy/>.

Horvath, D. G. (1987). The Acetate Negative Survey: Final Report . The University of Louisville, EkstromLibrary, Photographic Archives, Louisville, KY 40292, February 1987.


f 11 21/05/2011 11:54 PM



Landon, G. V., Seales, W. B., & Clarke, D. (2008). A new system to acquire and restore document shape andcontent. Proceedings of the 5th ACM/IEEE international Workshop on Projector Camera Systems (pp. 1-8).Marina del Rey: ACM. <doi:10.1145/1394622.1394638>.

Landon, G., Lin, Y., & Seales, W. (2007). Towards Automatic Photometric Correction of Casually IlluminatedDocuments. IEEE Conference on Computer Vision and Pattern Recognition, 2007. CVPR '07 ., (pp. 1-8).<doi:10.1109/CVPR.2007.383254>.

Lavoie, B., Connaway, L. S., & Dempsey, L. (2005). Anatomy of aggregate collections: The example of

google print for libraries 11 (9). D-Lib Magazine. <doi:10.1045/september2005-lavoie >.

Library of Congress. (1998) Library of congress manuscript digitization demonstration a project: Final report.October 1998. <http://lcweb2.loc.gov/ammem/pictel/ >.

Lyman, P. & Varian, H. R. (2003). How Much Information, Retrieved November, 14, 2008, from<http://www.sims.berkeley.edu/how-much-info-2003 >.

NextEngine, Inc. (2009). Retrieved January 15, 2009, from <https://www.nextengine.com/indexSecure.htm>.

Rich, M. (2009, January 5). Google Hopes to Open a Trove of Little-Seen Books. Retrieved January 15, 2009,from The New York Times: <http://www.nytimes.com/2009/01/05/technology/internet/05google.html>.

Rollin, C. (1815). The ancient history of the Egyptians, Carthaginians, Assyrians, Babylonians, Medes &

Persians, Macedonians, and Grecians. Silas Andrus.

Stanco, F., Ramponi, G., & de Polo, A. (2003). Towards the automated restoration of old photographic prints:a survey. EUROCON 2003. Computer as a Tool ., (pp. 370-37). <doi:10.1109/EURCON.2003.1248221>.

Stanco, F., Tenze, L., Ramponi, G., & de Polo, A. (2004). Virtual restoration of fragmented glass plate photographs. 12th IEEE Mediterranean Electrotechnical Conference, 2004. MELECON 2004, (pp. 243-246).<doi:10.1109/MELCON.2004.1346819>.

Wada, T., & Matsuyama, T. (1992). Shape from Shading on Textured Cylindrical Surface: Restoring DistortedScanner Images of Unfolded Book Surfaces. IAPR Workshop on Machine Vision Applications, (pp. 591-594).Retreived February 3, 2008 from <http://b2.cvl.iis.u-tokyo.ac.jp/mva/proceedings/CommemorativeDVD/1992/papers/1992591.pdf >.

Zhang, L., Zhang, Y., & Tan, C. (2008). An improved physically-based method for geometric restoration of distorted document images. IEEE Transactions on Pattern Analysis and Machine Intelligence , 30 (4),728-734. <doi:10.1109/TPAMI.2007.70831>.

Copyright © 2009 George V. Landon

Top | ContentsSearch | Author Index | Title Index | Back Issues

Previous Article | First Conference ReportHome | E-mail the Editor

D-Lib Magazine Access Terms and Condit ions

doi:10.1045/march2009-landon
