Monthly Archives: October 2016

The Complexity of Color – Creating Digital Surrogates

Researchers and students today have an increasing expectation of being able to find needed materials and information online. Libraries and other cultural heritage institutions have responded by focusing on digitization for both preservation and access to these cultural heritage materials. The Digital Imaging Lab, as part of the Preservation and Digital Conversion Division, is at the center of the creation of digital surrogates for preservation and online access at Columbia University.

Digital surrogates are “faithful” reproductions of an object at the time it is photographed. It is an accurate record of the item’s appearance and condition at a singular point in time. Since we want to handle the material as little as possible, a use-neutral file is created that is large enough to cover the many expected uses of an image file, from viewing on a computer monitor to reproduction in books and journals. But how do we know what is large enough? Back in 2010, in order to ensure a certain level of quality, federal agencies set about writing a set best practices for all those involved with the creation of digital surrogates for the federal government. This became known as The Federal Agencies Digitization Guidelines Initiative or FADGI. These guidelines became the standard for cultural heritage institutions here in America. European institutions adopted similar standards, known as Metamorfoze. In the Digital Imaging Lab we strive to reach the 3 and 4 star ratings parameters set forth in these guidelines (page 10 of the FADGI guidelines has an explanation of the star rating system).

Creating a digital surrogate.

Creating a digital surrogate. Photo credit: David Ortiz.

By far the biggest challenge faced by any institution in the creation of digital surrogates is the capture of accurate color. Color is a highly complex subject. It is thought that the perception of color takes place in the mind as an endpoint of a sequence of physical, physiological and psychological events. Each individual perceives color in his or her own unique way. Just like individuals, every device used in our lab, from the monitors we use to the cameras to the printers, capture and display color differently. Luckily scientists have been studying color for a long time and there exist several models for the mathematical description of color. In the digital realm we use a color model called RGB, named for the red, green and blue light that devices use in capturing and displaying color. Certain amounts of each of these lights (on a scale of 0 to 255) will produce almost all of the colors we can perceive.

Before we can use any device in the Digital Imaging Lab we must first find out how that device captures or displays color, and then, using software, we can bring it to a known state of color capture or display. The process in a nutshell is this:  using scientifically created and widely recognized color targets that have patches of mathematically defined colors, we have the device capture an image of the target (or for a monitor, display a set of defined colors). We note how far off the device is and then create a set of offsets (or corrections) for the device. These offsets are what are typically called color profiles.

This is a very simplified version of what goes into color profiles. The color of the walls, ambient lighting, the position and type of lights being used for photography, all have unique characteristics that must be dealt with when getting ready to create a digital surrogate. Even the pigments in the object being imaged can affect the sensor in unique and surprising ways. Contrary to the wishes of most cultural heritage institutions, creating high quality digital surrogates that have a good chance of persisting long into the future, is never as simple as pushing a button.

0009529

Digital Surrogate of a page from a medieval manuscript, at bottom is a Kodak color target.

A Brief History of the Preservation Reformatting Department

Columbia’s Preservation Reformatting Department (PRD) began as a reprographic services unit back in the 1930s. In the 1970s-1980s, the department gradually became a reprogaphy unit with an emphasis on the preservation of brittle and deteriorating materials.

While the Preservation Division was taking shape, the world was just beginning to understand the slow moving disaster headed our way: the acids within wood pulp paper, which would eventually consume our books and documents. A number of studies done as early as the 1930s had found that an overwhelming percentage of research collections were printed on acidic paper which, under less than pristine conditions, would eventually become embrittled, ensuring the eventual destruction of more than a century of scholarly works.

Daily Spectator

Columbia Daily Spectator, Dec. 8, 1941

In an attempt to cope with this looming catastrophe, the National Endowment for the Humanities (NEH) issued a number of grants to research libraries throughout the U.S., including CUL. These funds, along with emerging best practices established by the Research Library Group (RLG), provided us with the means of moving nascent reformatting projects into large scale reformatting programs, which endure to this day, albeit in a much evolved form.

As of 2016, PRD has transformed itself in many ways; a reflection of the revolutionary technological changes happening outside and within the library doors. We continue to prioritize materials in demand but have expanded our capacity. You may be surprised to hear that we still do send out shipments of microfilm for brittle, circulating collections, primarily due to copyright restrictions. We also still create preservation photocopies for materials for which we really need physical copies on the shelves, such as music scores and reference materials.

In addition, we have a fully developed program for ebook creation for public domain materials and PRD staff is responsible for every step of this process, as they have been for many years with microfilming and photocopying. The staff collates items, searches for existing copies, creates copy catalogued records for the new formats, sends and receives vendor reformatted materials, and is responsible for all QC, image processing and uploading and organizing on Internet Archive.

ldpd_10993010_000_00000001

A pamphlet from the Missionary Research Library (Burke Library)

A future blog post will explore this process and some of the customizations and enhancements that PRD has come up with over the years.

Finally, we are also responsible for the front end of patron services and for numerous special projects, such as the Columbia Spectator digitization project and Burke’s Missionary Research Library digitization, images of which are included in this post.