Camera system for multispectral imaging of documents

Camera system for multispectral imaging of documents William A. Christens-Barry*a, Kenneth Boydstonb, Fenella G. Francec, Keith T. Knoxd, Roger L. Easton, Jr.e, and Michael B. Tothf a Equipoise Imaging, LLC, 4009 St. Johns Lane, Ellicott City, MD 21042 b MegaVision, Inc., PO Box 60158, Santa Barbara, CA 93160 c Preservation Research and Testing Division, Library of Congress, Washington, DC, 20540 d Boeing LTS, 535 Lipoa Pwky., Kihei, HI 96753 e Chester F. Carlson Center for Imaging Science, Rochester Inst. of Tech., Rochester, NY 14623 f R.B. Toth Associates, 10606 Vale Road, Oakton, VA 22124 ABSTRACT A spectral imaging system comprising a 39-Mpixel monochrome camera, LED-based narrowband illumination, and acquisition/control software has been designed for investigations of cultural heritage objects. Notable attributes of this system, referred to as EurekaVision, include: streamlined workflow, flexibility, provision of well-structured data and metadata for downstream processing, and illumination that is safer for the artifacts. The system design builds upon experience gained while imaging the Archimedes Palimpsest and has been used in studies of a number of important objects in the LOC collection. This paper describes practical issues that were considered by EurekaVision to address key research questions for the study of fragile and unique cultural objects over a range of spectral bands. The system is intended to capture important digital records for access by researchers, professionals, and the public. The system was first used for spectral imaging of the 1507 world map by Martin Waldseemüller, the first printed map to reference “America.” It was also used to image sections of the Carta Marina 1516 map by the same cartographer for comparative purposes. An updated version of the system is now being utilized by the Preservation Research and Testing Division of the Library of Congress. Keywords: spectral imaging, EurekaVision, LED spectral illumination, Waldseemüller map, cultural studies, maps

1. INTRODUCTION The rapid development of modern imaging technologies has had a profound effect on the study of historical artifacts over the last decade or so, to the point where digital systems are commonly used to provide images for scholarly study.1 Spectral imaging of manuscripts has been developed in parallel to improve the readability of documents and assist assessment of their condition.2-4 The EurekaVision system reported here was developed to collect spectral images of a printed artifact at the Library of Congress prior to scheduled long-term encasement and exhibition. The images provide a baseline for the condition of the artifact for comparison and assessment of any changes. The primary subject imaged was the Universalis cosmographia secundum Ptholomaei traditionem et Americi Vespucii alioru[m]que lustrations, a world map by Martin Waldseemüller first printed from woodblocks in 1507.4 The same woodblocks likely were used for subsequent printings for several years. The earliest known document to name “America,” the map is printed on 12 sheets, each approximately 620mm × 460mm. All sheets bear one of two versions of a watermark containing a threepointed crown and two of the twelve sheets exhibit red-ink grids. The imaging study of the Waldseemüller 1507 map was intended to create a series of baseline images to inform researchers and make available and accessible information about the Waldseemüller since the map was to be encased for 25-30 years. A team of imaging and preservation scientists, supported by information technology and conservation professionals, constructed an imaging system with reflected, transmitted, and raking illumination from banks of light-emitting diodes (LEDs) to produce spectral images of both sides of each of 12 map sheets of the 1507 map under all available illuminations at 300dpi. The system was also used to capture images of both sides of each sheet at 600 dpi and stitched to create large-format images of all pages. In addition, 1200dpi images were collected of key areas of specific interest, e.g., paper watermarks. The utilization of the spectral imaging system therefore allowed a greater range of data to be collected for preservation research and scholarly studies than that collected by simple digital document imaging.

Sensors, Cameras, and Systems for Industrial/Scientific Applications X, edited by Erik Bodegom, Valérie Nguyen, Proc. of SPIE-IS&T Electronic Imaging, SPIE Vol. 7249, 724908 © 2009 SPIE-IS&T · CCC code: 0277-786X/09/$18 · doi: 10.1117/12.815374 SPIE-IS&T Vol. 7249 724908-1

Because of the impending deadline for encasement of the map, the EurekaVision system originally was assembled from existing components. For example, the LED illumination system had originally been constructed for imaging of the Archimedes Palimpsest in August 2007 by a team that included several of the authors of this paper. The camera and LED system were integrated shortly before the imaging session at the Library of Congress, which was conducted over a 10day span in November 2007. Additional objects were imaged during the same session, including several items of interest on Waldseemüller’s printed map from 1516 known as the Carta Marina (“Map of the Sea”): two entire sheets (9, 10) from the set of twelve to compare the red grid feature, a paper pastedown on sheet 9 that was thought to be obscuring text beneath, and watermarks on sheets 1 and 11. The pastedowns and watermarks were imaged in transmission. Finally, the team also imaged a parchment map from the collection of Mr. Marcian F. Rossi that was given to the Library of Congress in 1935. It is sometimes called “Map with Ship” because of a sketch on the map is thought by some to be a very early map of the Pacific Ocean. Besides delivering the spectral images of each sheet, the imaging team performed multispectral image processing, including pseudocolor renderings, of images in areas of key interest on the map. The processing highlighted the different spectral responses observed with wavelength analysis from paper, parchment, inks and colorants. Future additional processing will be used to analyze colorants, inks, and other components. In addition to installation, integration, and operation of the imaging equipment, the imaging team also provided program and data management to ensure the image and data products and services met the needs of the Library of Congress Preservation Research and Testing Division.

2. THE EUREKAVISION SYSTEM The integrated monochrome E6 camera from MegaVision, Inc., utilizes a 39-megapixel Kodak CCD sensor array with 7216 × 5412 pixels, each with linear dimension of 6.8 microns. The camera produces images quantized to a dynamic range of 12 bits per channel. The EurekaLight™ LED illuminators and SideLong™ raking-light illuminators from Equipoise Imaging, LLC were used to collect images in twelve distinct spectral bands. The SideLong illuminators generate two spectral bands of illumination (blue and infrared) from either side of the object. 2.1 Illumination System Narrowband illumination in the ultraviolet (UV), visible (VIS), and near-infrared (NIR) was provided by a pair of EurekaLight illuminator panels. These panels utilize high power LEDs that emit in narrow spectral bands over ranges of wavelengths from the near ultraviolet to the near infrared. The lighting system includes multiple LEDs at each of 12 wavelengths, as shown in the table: λ0 [nm]

Δλ (FWHM, nm)

Relative Peak Power (arbitrary units) 365 25 0.68 445 40 0.45 470 40 0.45 505 40 0.79 530 40 0.55 570 35 0.84 617 40 0.92 625 35 1.00 700 30 0.89 735 35 0.89 780 35 0.63 870 45 0.79 Lenses are fitted to the individual LEDs to match the spot size of the illumination to the dimensions of the imaging subjects. Diffusers are mounted on each individual ultraviolet and visible LED, while a single large diffuser was used with each multiple wavelength bank of infrared LEDs. Materials and optical properties of the lenses and diffusers were selected to match the wavelengths and angular distributions of light emitted by the different types of LEDs that were used. In particular, the lenses and diffusers used with the UV LEDs transmit the 365nm emission with little loss and without fluorescence.

SPIE-IS&T Vol. 7249 724908-2

Our principal technique for acquiring images utilized a pair of EurekaLight illuminators mounted on adjustable stands above the manuscript and placed 1–2 meters from the optical axis of the camera to provide illumination. This resulted in illumination directed downward at incidence angles between 30 and 60 degrees. While this configuration is termed “reflectance” imaging, it should be noted that images illuminated at λ0 = 365nm (ultraviolet) are due to the emission of visible (and potentially near-infrared) light due to fluorescence of the paper or parchment of the subject. Any ultraviolet radiation reflected from the subject was blocked from the camera CCD by the glass optics of the camera lens, which effectively acted as a longpass filter at this wavelength. While quartz optics may be used in principle to capture reflectance images at ultraviolet wavelengths, we did not have an appropriate quartz lens capable of filling the large CCD. Also, images under ultraviolet illumination would include contributions due to both ultraviolet reflectance and visible emission from fluorescence that should be removed by an appropriate cutoff filter. For the analyses carried out in this study, broadband fluorescence emission (largely confined to the blue-green region of the visible spectrum) due to UV illumination was treated simply as an additional reflectance band in our wavelength image cube. Precise adjustment of the locations and incidence angles of the illuminators was performed in conjunction with the capture of a test series of images used for flat field calibration and white balance procedures. Raking illumination at specified angles of incidence may be used to enhance the visibility of surface relief structures. Our system employed raking illumination at 470nm and 910nm provided by a pair of SideLong raking illuminators. These linear LED arrays, which are lensed to provide a beam that is rectangular in shape and with small angular spread (~4º half angle), are positioned on the flanks of the subject material and adjusted to the desired angle of incidence. This proved useful in some of the typography of the watermark and papermaking features, such as chain and laid lines. Transmitted-light imaging was invaluable for imaging watermarks and manuscript content within the substrate. The illuminators were placed beneath a translucent diffusive platen of the copy stand and directed upward toward the subject (angle of incidence of 180º). Transmissive images at all available LED wavelengths were collected. The system had to be rearranged to change from transmissive mode to reflective mode, which means that the images from the two modes are not registered and may not be easily combined from the results of this study.

MegaVision E6 Monoebrome Digital Back

Figure 1: Eureka Vision system at the Library of Congress set up for spectral imaging at 600dpi, showing the camera with MegaVision E6 digital back, two Eureka Light LED panels and two SideLong LED panels for raking illumination.

The illuminators contain integral microcontroller, logic, and power electronics that interface with the host software via USB. Each of the illuminators plugs into a single interface unit containing an LED power supply and a USB hub that is connected to a single USB port of the host computer. Digital Camera System An integrated digital camera system comprising a MegaVision monochrome E6 39-megapixel back, technical camera body, digitally controlled shutter and lens, is at the core of image capture.

SPIE-IS&T Vol. 7249 724908-3

Digital Sensor: The MegaVision 39-megapixel E6 Monochrome back provides highly linear data with excellent quantum efficiency and a 12-bit signal-to-noise ratio (S/N) (up to 4,000 to 1). Dual taps enable data rate readout at 20 megapixels per second with very low read noise. Because there is no decal color filter array (e.g., Bayer pattern), sensitivity is improved, the cultural object is exposed to less light, and resolution is improved because no anti-aliasing filter is required. Pixel response is 100% from pixel to pixel. A removable IR filter is normally removed for in-studio work. Besides enabling infrared image capture, the removal of the infrared filter significantly improves red sensitivity, thereby reducing the required exposure. Specifications: Sensor: Kodak KAF 39000 CCD array, 7216 × 5412 pixels, 6.8micron × 6.8micron, uncooled Sensor size: 49mm × 37mm Dynamic range: 12 bit (14-bit ADC) Equivalent film speeds: 100, 200, 400 ISO Interface: IEEE 1394 FireWire 400j Capture rate: 3 seconds per frame Camera Body: A technical camera capable of accepting a copal shutter and lens system provides the mounting for the back, shutter, and lens subsystems. For maximum flexibility, a Toyo VX23D 2×3 view camera is provided. The VX23D is light enough to be very portable and utilize a modest tripod, yet rigid enough to work on a copy stand in a production environment. All view camera motions, rise/fall and swing/tilt are available on both front and rear standards. Other camera bodies and bodies with fixed focal plane/lens axes are available for use in more restricted applications. The digital back is easily customized to specialty camera body requirements. Shutter/aperture: A key component of the camera system is the shutter. Since each multispectral image typically includes a dozen or more exposures, the wear on the shutter is an order of magnitude or more than it would for single-exposure captures. The Schneider digitally controlled copal shutter provides more than one million releases with high repeatability. The possible exposure range is 17msec to 60 seconds in increments of 1/10 stop. The digitally controlled aperture can also be set in 1/10 stop increments. The shutter range is a function of the lens. Shutter settings are reported and logged into the EXIF metadata of the image file. The shutter is controlled and powered by a USB interface to the host computer. Lens: While any Copal 0 lens may be used with the system, care was taken to match the lens to the application to ensure that the lens resolved 6.8-micron pixels at the focal plane. Since there is no compromising color filter array and since the pixel response is 100%, the lens normally is the resolution-limiting element in the system. A Schneider Digitar Apo Macro 120 lens was selected for this work on the basis of its good sharpness, low distortion, and low scatter at working distances of about 0.5–2 meters. This range combined with the 6.8-micron pixels of the E6 back results in scene resolutions that vary from less than 300 dpi to over 1,000 dpi. File Sizes: 78 Mbyte at 16 bits, 39 Mbyte at 8 bits 114 Mbyte color composite at 8 bits/channel 1 Gigabyte typical for 12-band spectral stack at 16 bits/band. Format: single channel DNG or TIFF Metadata: header-embedded, extensible EXIF and IPTC

SPIE-IS&T Vol. 7249 724908-4

The camera was mounted on a TTI 3040 motorized copystand, with illuminators placed in left and right flanking positions at distances of 0.5-1 meter. Working distances ranging between approximately 450mm and 1900mm were used to achieve spatial resolutions in the range of 1200dpi to 300dpi, depending on the required resolution for the specific situation. The size of the individual map pages was well matched to the size of the sensor for imaging at 300dpi. At 600dpi and higher resolutions, a computer-controlled x-y translation table was used to precisely position the map page beneath the camera. Sufficient overlap of the images was maintained to allow digital stitching of the images to create full-size images at this resolution. PhotoShoot Software Management of the large volume of spectral image data proved critical to the project. All imaging data had to be readily accessible along with required metadata for image processing and analysis. Effective use of this imaging system for scientific and preservation studies required establishment of standardized data elements and software for storing image data and metadata during the imaging campaign. The PhotoShoot software package is what its name implies: an application for capturing pictures. PhotoShoot intimately couples with the camera and the lights to enable precise control of each. It controls all elements of image capture: digital back, shutter, aperture, and lighting. Tools in Photoshoot enable composition, focusing, and lighting setup in addition to image capture. Exposures are precisely established and maintained to ensure maximum signal-to-noise ratio (S/N) for each capture. Dark noise is routinely assessed and temporally persistent noise is automatically subtracted pixel-by-pixel from every exposure. Images are automatically corrected pixel-by-pixel for variations in gain. PhotoShoot references the image data to objective imaging standards, such as ISO rating, and enables rapid and convenient inspection of captured images to verify image quality. PhotoShoot treats the digital images in a manner similar to photographic emulsions, with developers appropriate to specific targets. Selection of developer allows flexibility in density ranges and ISO settings for the desired target. Metadata are embedded in the header of the picture file to facilitate image storage and retrieval, processing and analysis; PhotoShoot metadata support is based upon EXIF and IPTC standards that may be customized to embed database information directly into the image files. The metadata formats were developed from broadly accepted international standards and protocols to ensure searchable information was included in the file headers in a format that was persistent over time and on a range of computer systems. The imaging team’s data manager, Doug Emery of Emery IT, developed the data management and metadata models prior to imaging. These metadata formats are adapted to allow searchable information to be included in the file headers. PhotoShoot enables automatic naming, routing, and organization of captured images to be based on embedded metadata that are captured during the imaging. Calibration Imaging Because the monochrome camera lacks chromatic filters (other than the glass lens that blocks wavelengths λ ≤ 400nm), light from all wavelengths within its spectral sensitivity range (up to λ ~ 1100nm) contribute to the acquired image. Consequently, stray light must be carefully limited and accounted for to ensure maximal use of the available dynamic range. Stray background light was carefully eliminated and screened; images with illumination sources turned off were collected at regular intervals using the same exposure durations as used for each wavelength during imaging of the subject materials. Dark images were recorded at regular intervals for use by the noise reduction routines employed by the PhotoShoot software. For these, series of images were acquired without illumination and with the lens cap on, using fixed exposure durations corresponding to the exposure durations used for actual subject imaging. The PhotoShoot software calculated noise statistics at each pixel for each such set and applied these statistics to the subject images in order to reduce noise. A Macbeth™ color chart, consisting of an array of color swatches with know color coordinates, was routinely placed next to the subject and captured as part of each image. The inclusion of internal color standards ensured that color calibration could be performed using the actual lighting and camera conditions employed for each image. Images were collected of both sides of all sheets at resolutions of 300 dpi (approximately 12 pixels per mm), requiring one image per wavelength per sheet, and at 600 dpi, requiring six images per wavelength per sheet.

SPIE-IS&T Vol. 7249 724908-5

3. IMAGING AND IMAGE PROCESSING Images at 300dpi of each full sheet were collected by illuminating with each LED waveband in turn and exposing for the predetermined proper time. All images were collected at the same focus setting and the same f/stop. The image sequences were controlled by the system computer. Each spectral sequence required of the order of one minute. For capturing the high-resolution images (600dpi), the sheet was placed on a computer-controlled x-y table (Figure 2). After completion of each spectral sequence, the table was translated to the next position and the sequence repeated.

\ IA

:ir Figure 2: Universalis cosmographia sheet 9 (lower-left corner of map) on computer-controlled positioning platen for imaging at 600dpi, also showing LED panel for raking-light illumination at upper right.

The images captured by the EurekaVision system were enhanced to bring out specific details, using software developed for the Archimedes Palimpsest project1,2. The techniques used were directed at specific tasks and items of interest, including possible impressions of the page due to the printing, red grid lines, text beneath pastedowns, and watermarks. In addition, we used a commercial software tool to stitch the individual 600-dpi images of each sheet in each color to create images of full sheets at this resolution. 3.1 Embossing The raking-light images were intended to capture the surface topography of the map. The surface roughness shows clearly in the raking images, in the form of tiny shadows. By combining the raking images illuminated from the left and right sides, individual bumps in the surface can be outlined. Any printing impressions surrounding the characters due to the pressure from the woodblock would be hidden by the ink of the characters themselves. To produce an image with the ink suppressed, the numerical difference between the blue and infrared raking images from each side was computed. Prior to differencing, the local contrast of both images was balanced. First, the mean and standard deviation within a 401-pixel square window was computed. The gray value of the center pixel in this window was changed to reflect the deviation of the original value from the mean on a scale based upon the standard deviation. As a result, the gray-level means and standard deviations are matched for every window across the image. After this adjustment, details that are the same in the images at both wavelengths, such as the dark ink or the lighter paper, have the same value in the balanced images, so these details become a uniform gray in the difference image. Due to the longer wavelength and the focus setting of the lens, the infrared raking image is less sharp than the blue raking image. When a blurred version of an image is subtracted from a sharp version of the image, the edges are enhanced and the uniform areas are suppressed. This is similar to the edge-enhancement process called unsharp masking5 that is commonly used in the printing industry.

SPIE-IS&T Vol. 7249 724908-6

The result of this processing can be seen in the left-hand image in Figure 3. After enhancing the edges and attenuating the uniformly gray areas, the image conveys the perception of embossing, which the scholars find to be useful. 3.2 Red Grid Lines Due to fading of the colorant over the last 500 years, the red grid lines on sheets 6 and 7 are now barely visible. The interest in this feature was twofold: 1) to enhance the grid lines to better visualize the start, finish, and placement of these lines, thus possibly revealing the original intent of the printer, and 2) to allow scholars and researchers to gain greater knowledge of the thought processes of the cartographers working 500 years ago. The right-hand image of Figure 3 is an enhanced version of a section of sheet 6 with faded red grid lines. The color of the lines was enhanced by computing color differences from the multispectral image set. Two composite images were created – a blue image from the average of the three blue wavelengths (445nm, 470nm, and 505nm), and an infrared image from the average of the infrared wavelengths (735nm, 780nm, and 870nm). The two composite images were then enhanced using the pseudocolor technique developed for the Archimedes Palimpsest project2. The resulting enhancement of the colored red grid lines is seen in the right-hand image of Figure 3; note that the red grid lines also are visible in the left-hand “embossed” image of Figure 3.

3

_t

H Figure 3: Results of two different processing algorithms on the sheet with red grid lines: (left) after image processing to enhance edges, giving perception of “embossing;” (right) after pseudocolor processing to enhance red grid lines.

3.3 Infrared Text Imaging of Pastedowns and Watermarks As noted previously, sheets 9 and 12 on the Waldseemüller and Carta Marina maps contained pastedowns – small pieces of paper affixed to the printed sheet to append (or to obscure) supplementary information. The pastedown on sheet 9 of the Carta Marina contained a list of errata that presumably were to be corrected on the woodblock before further printings. Analysis of this copy revealed that the changes were made on all but one sheet (sheet 6, of which 2 copies exist – one paper and one parchment). It is suggested that the parchment sheet may be one of the original proof sheets. To assess the potential value of using EurekaVision to characterize paper watermarks, examples were imaged at high resolution (1200dpi) in transmission. Watermarks are very important in the analysis of papers; they allow researchers to trace the paper to a specific location of its production. Prior to 1800, all paper was made by hand using a mold consisting of a frame with a fine mesh of wires running parallel to the long sides of the mold. Thicker wires – the chain lines –ran parallel to the shorter sides of the mold. Water with suspended paper fibers was evenly distributed over the wire mesh and then the water was drained off. The mold was then turned over. This created two distinct sides to the paper; the mold side carries the impressions of the wire and chain lines. The identification and study of watermarks is critical to analysis and establishment of the provenance of library and archive documents.

SPIE-IS&T Vol. 7249 724908-7

Until safety requirements precluded its use, beta radiography had been the accepted method for studying watermarks. Transmission images collected with the EurekaVision may provide a useful alternative for studying watermarks, as well as for revealing features of the paper‘s construction. Very little image processing was required to read the text in the pastedowns (Figure 4) or on the images of the watermark (Figure 5). The images of both subjects were taken in transmission with infrared light at 910 nm and required only simple linear adjustments of contrast to emphasize the important image details. 4aW

ofW,

Iand: 3nocaflO &IThano bcpo*t nni mfug1an4. I. cu bid'Iti picu ct'(.wp occafl6 b(pcx1 im ci lQ4i *lk4 prd p0pc b1dz pa4tii I1 po?rog1in pvot bM.n raa mi$napp

pstatcma, (wcm61iiir ue

afa .p c133l1 i5pfScuaIdl 7anuofui*k riva futa pie e d,4.4 oun* ppc 3i(mar la r ion ,iscn

ctô Li tui rn.4,p i(uic,4ppe Camu1l 3T5 IUi f ominio th.i t ThnCp c6i rtic wtinc d mbjep tMT p.aai rn NbuIt ZaranC .'rciI kp a1dii poz sprn ourn rabula cZt'ipukic tc.i ,ar5 .c no4 uta

j

m(pod,i p (mpflIrrr:m & tirnnp ofIn M iula fub cuur, CMilu k1 Ipat.

oe p fmo üi cad& a'tw4w fu Thç L1 Ino: rUe.

O

.4afllt pi'scU

gj p Dizitht

fl1XU.)fl rwq

inØ pie pii*,.

atm mac

Figure 4: (left) visual appearance of "pastedown" on sheet 9 of the Carta Marina; (right) image using transmitted illumination at 910nm . The transmitted-light image has been reversed left to right, showing text on the underside of the pastedown.

1k, !NAI

Figure 5: Image of the watermark on one sheet of the 1507 map in transmitted light at 910nm.

3.4 Digital Stitching of 600dpi Images Finally, images of both sides of each complete sheet at 600dpi were created by digital stitching using the Autosync utility in ERDAS Imagine (from Leica GeoSystems), as shown in Figure 6. The size of each stitched file was approximately 354 Mbytes per band.

SPIE-IS&T Vol. 7249 724908-8

24 inches

Azimuth 5.4 12 px