System considerations for multispectral image compression designs

System Considerations for Multispectral Image Compression Designs VAL D. VAUGHN and TIMOTHY S . WlLKlNSON

E

volving remote sensing systems for today’s imaging applications are being called upon to collect data of higher spatial resolution in wider coverage areas, and an increasing number of spectral bands. Much of this migration is being dictated by military, commercial and civil reJANUARY 1995

quirements. In 1992, Congress passed the Land Remote Sensing Policy Act [ l ] in an attempt to maintain the Landsat program as a leader in the collection of multispectral imagery (MSI). This act formed a joint alliance between DOD and NASA, whose goal was to take Landsat into the 2 1 st century

IEEE SIGNAL PROCESSING MAGAZINE 1053-5888/Y5/ls4.0oolYY5lEEE

19

such a diverse set of applications comes a wide variety of system requirements. The challenge of including MSI compression in a system is to ensure sufficient exploitation integrity of the imagery, while providing compression ratios large enough to allow high data rate collection. The following items are factors that increase data rates and motivate MSI compression.

Spatial Resolution

I . Multispectral color composite images.

by developing advanced technology that would facilitate the collection of high resolution imagery. (The alliance was dissolved in 1994 and Landsat development continues with NASA). With an expected increase in data volume and a lack of viable high rate data transmission channels, designers’ initiated development of multispectral image compression techniques to meet the anticipated data flow requirements. As a part of the DOD effort, the Defense Landsat Program Office began to develop a cost-effective multispectral remote sensing technology to follow Landsat 7. The Advanced Land Remote Sensing System (ALRSS) development included definition studies for system concepts, advanced focal plane design, and multispectral compression development. From the compression studies came many “lessons learned” that will be addressed in this article. Rather than describe in detail the actual compression technology developed under this study, or advocate specific compression technologies as the best solution, we will first address some typical user requirements and then compare these requirements to existing and future systems. This discussion will bring forth the importance of including MSI compression in future system designs. The MSI collection chain and its impact on compression will be examined next, followed by a discussion of specific compression properties of MSI data. Finally, we’ll explore methods of exploiting spatial and spectral information for compression gain, followed by simulation and evaluation results from the ALRSS compression development.

Understanding the Users’ Needs MSI can be used in a myriad of applications covering areas in global environmental monitoring, mapping, charting, geodesy, and land use planning. There is also the area of natural resource management, including forestry, agriculture, water quality monitoring, and wildlife habitat management. With 20

The ground sample distance (GSD) of a sensor limits the size of objects that can be resolved in a digital image. For example, measuring the extent of deforestation requires 30 meter GSD imagery, but measuring vegetation type or stress requires 10 meter spacing. In contrast, the GSD necessary to detect and then identify specific targets such as surface ships is less than 10 meter GSD, and to identify the type of ship needs less than 4 meter spacing [ 2 ] .Detecting and identifying smaller objects such as vehicles requires even finer resolution. To emphasize the difference in utility as a function of GSD, two scenes of the same area collected at different resolutions, using the M7 sensor [3],are shown in Fig. 1. A 3 band color composite is shown in Fig. la. This composite was selected to emphasize vegetation of an agricultural area, which was sampled at 10 meters GSD with a small area zoomed and interpolated to show more detail. Figure l b shows the same small area, which was collected at 1 meter resolution. Comparing the 1 meter GSD with the zoomed 10 meter GSD imagery, we see that, for applications which require detail to identify objects such as buildings and vehicles, higher resolution is better. Several system concepts have been formulated for ALRSS that address resolution. The concept studies projected that higher resolution MSI (55 meters), would enhance current applications and spur new users and applications. Unfortunately, the data collection rate increases quadratically as the GSD is decreased, and in turn adds to the cost of building and operating such a system. Spectral Resolution The spectral resolution of a remote sensing image system refers to the number and widths of the spectral bands that are collected. The wavelengths can span the spectrum from the ultraviolet through the infrared. Multispectral sensors typically collect many separate bands with spectral characteristics selected to emphasize or delineate particular spectral features. In the extreme, hyperspectral and ultraspectral systems collect hundreds if not thousands of narrow adjacent spectral bands. Typically, most general exploitation tasks such as categorizing terrain can be done sufficiently with at least 4 appropriately selected spectral bands [4]. However, increasing the number of non-overlapping bands allows the user to discem a larger variety of targets, identify man-made structures, measure water depth and condition, map terrain features, and measure small differences in crop types. The system impact of spectral resolution is that the data collection rate increases in proportion to the number of bands collected (assuming constant swath and GSD).

IEEE SIGNAL PROCESSING MAGAZINE

JANUARY 1995

Radiometric Precision Radiometric precision, e. g., the allocation of bits to quantizing dynamic range, is important for many scientific applications. High radiometric precision is especially necessary when measuring minute changes in observables. The current Landsat Thematic Mapper quantizes each pixel to 8 bits (256 counts), but precision up to 12 bits per pixel (4096 counts) is desired for many remote sensing applications. Increasing the radiometric resolution of the data may enhance analysis, but also results in a concommitant increase in collection data rate; the data rate is directly proportional to the number of bits per pixel quantization. Area Collection Users are typically interested in collecting the largest ground coverage possible. Collecting wide swaths improves the overall ground coverage, and provide larger, more complete data sets with which to work. Larger areas of collection also improve revisit time, i.e., the time between subsequent collections covering the same geographic location. Although revisit time is also a function of vehicle orbit, wider swaths decrease the number of revolutions necessary to cover a given area). Today’s technology allows collection of very wide swaths, yet, in many situations, transmission rates limit the swath width that can be collected for a given GSD. Data collection rates increase linearly with swath width, and a tradeoff exists between the swath width and the GSD. Sharpening (Pan) and Stereo Imagery Visual exploitation can be greatly enhanced by combining MSI with a higher resolution sharpening band [5].The sharpening band is typically a single panchromatic band whose spectral coverage overlaps other multispectral bands. Collecting a sharpening band with twice the resolution of the rnultispectral bands uses as much bandwidth as four individual multispectral bands, and greatly impacts the collection budget. Stereo imagery is essential for making contour maps and is useful for photointerpretation. But since stereo imagery is collected by imaging the same swath from a second angle, the data volume is double that required to collect only monoscopic imagery.

-

2 . Model of remote sensing system chain

sensors that are designed to collect higher resolution imagery (less than 100 meter GSD). Table 1 lists the collection parameters of four systems. With the demise of Landsat 6, which was lost during launch in 1993, the U. S. depends upon the aging Landsat 4 and Landsat 5 aystems for its source of collection. Current development of the Landsat 7 system is ongoing but launch is not expected any earlier than 1997. With the increasing need for higher resolution MSI by civil and military users alike, it follows that a system with larger collection capability is necessary for the future. To motivate the need for compression, the following two systems are presented to give what might be considered a minimum and maximum requirement for data compression. The minimum requirement is derived from the High Resolution Multispectral Stereo Imager (HRMSI), which was proposed originally as an option with the Enhanced Thematic

Comparing Current and Future MSI System Data Rates There are numerous space based MSI sensors that are currently deployed and actively collecting imagery. For this discussion, we will focus on the space-bom multispectral Table 1: C

1

3. Atmospheric absorption features. JANUARY 1995

IEEE SIGNAL PROCESSING MAGAZINE

21

swath width of greater than 100 kilometer to facilitate wide area collection. Such a system could potentially generate 4.28 Gbps of data. In order for this system to transmit in real time through current crosslink channels such as TDRSS, or an advanced TDRSS (600 Mbps), would require an average compression rate of 14 to 1. A direct downlink with a common data link would require even more compression.

MSI Collection Chain

I One of the ways to study and optimize the compression algorithm performance is to simulate the compression algorithm in software and test it on representative imagery. For the simulation of on-board compression, it is important to use example imagery that is representative of that coming from the focal plane. Understanding each of the processes in the image collection chain and its effect on the imagery will allow definition of reasonable system requirements and proper simulation of compression. A basic configuration of a remote image collection system is shown in Fig. 2. It is useful to follow the image formation from the source to completion and describe some of the impacts that each will have on compression.

4. Amplifier failure simulation and resulting compression errors.

Mapper Plus (ETM+) instrument for Landsat 7. The HRMSI design met many of the current DOD requirements. The maximum requirement is derived from projected future focal plane designs. This design was derived by estimating where space qualified focal plane technology would be around the year 2000. HRMSI-like System The HRMSI instrument as proposed had 4 multispectral bands (current Landsat wavelengths) and a pan band which could be pointed fore and aft to image stereo pairs. The multispectral bands were 10 meter GSD with the pan band collecting 5 meter GSD. The pixel dynamic range was 9 bits for the panchromatic imagery, 10 bits for the multispectral bands. Swath width would be at minimum 40 kilometers (the actual HRMSI design had a variable width swath of less than 25 kilometers). Such a system would generate approximately 225 Mbps of data, which could fit through a TDRSS crosslink or a common data link for transmission without compression (the TDRSS cross link capacity is 300 Mbps, the common data link capacity 274 Mbps). In this case, compression is only necessary to economize the long term transmission costs. Advanced System The state of current and future focal plane technology may allow a space based system to be designed that carries up to 18 spectral bands, ranging from 5 to 40 meter GSD, with a single pan band of 2.5 meter GSD [7]. Each pixel would have 11 bits of useful dynamic range. Pushbroom technology, which scans parallel to the vehicle track (Fig. 2), will allow a 22

Atmosphere In the ideal case, an image would be an exact measure of the light energy (radiance) that comes from the surface of the earth. In reality, the light must travel through the Earth’s atmosphere which, through absorption and scattering, can change the spectral characteristics. Each multispectral band is affected differently by the dynamics of the atmosphere. Figure 3 shows the spectral transmission characteristics of the atmosphere. Note that there are several wavelength regions where light energy is strongly absorbed. Imagery collected in such wavelength regions will be noisy and difficult to compress. Optics and MTF The optical system itself affects the sensed light. The system’s best case resolution is limited by contributions from several sources, including the optical system, temporal aperture smear, velocity mismatch smear, diffusion, and detector aperture. The combined effect is to reduce high frequency spatial information. Any degradation of the system MTF decreases the ideal image quality. Regardless of the focal plane sample distance, the true resolvable distance may be limited by the system MTF. For the case of multispectral imagery, the optical MTF also changes as a function of wavelength. The system MTF strongly influence the spatial correlation of an image and thereby affects the performance of spatial compression algorithms. Analog Electronics The focal plane is the first stage of converting the continuous light source into a sampled image. The analog electronics, from the focal plane to the A/D converter, introduce various levels of noise to the image signal [8]. This noise comes from different sources including detector response, photon noise,

IEEE SIGNAL PROCESSING MAGAZINE

JANUARY 1995

Band Number

~

1

1

2 4 ~~

5

6 7 8

Wavelength Range (nm) 360-380 4 10-440 440-460 460-480 480-5 10 510-550 550-600 600-670 830-1020

__.____.

12

and dark current, detector defects, saturation, and dropouts. Sensor noise is realized as a spatial nonuniformity (striping), banding, pixel saturation, or line dropout. Their effect is to increase the high frequency element of the signal and reduce redundancy. Since the striping and artificial edges caused by dropouts must be removed with ground processing, the defects themselves must be preserved with a certain level of detail. On-board calibration and processing can eliminate many of the defects, but because of the additional computation it is not always feasible. The sensitivity of compression algorithms to sensor defects will ultimately help to determine the amount of calibration that is necessary. Most focal plane designs use an amplifier configuration that directs odd and even columns of pixels through different amplifiers. This configuration ensures that if a single amplifier fails, only the odd or even columns of pixels will be lost and a lower resolution image can be recovered. An example of a simulated failed amplifier in a single band is shown in Fig. 4a. Figure 4b shows errors from compressing the defective imagery using a spectral/spatial linear prediction-base compression algorithm. The absolute errors in Fig. 4 have been pseudocolor mapped to better visualize their magnitude. The increase in the error is evident at the amplifier dropout boundary. Since this particular linear prediction technique uses adjacent spectral pixels, errors from an adjacent band due to propagation of prediction errors from the corrupted band are also affected (Fig. 4c).

On-board Band Registration Maximum use of spectral redundancy requires that individual pixels from each spectral band represent the same location on the ground. In reality, spectral pixels are often misregistered from each other. Misregistered pixels, even if offset by only fractional amounts, mix the spectral signature and decrease the ability to use adjacent bands for decorrelation or prediction [9]. Sources of band misregistration at the sensor include misalignment of detector arrays, misalignment between focal JANUARY 1995

planes, variations in sensor sample rates, platform motion, and yaw tracking errors due to focal plane displacement. Additional misregistration from geometric distortions include earth curvature, earth motion, and panoramic effects due to wide field of view. The misregistration sources are either predictable (e. g., sensor focal plane geometry) or “scene dependent” (e. g., varying with geometry parameters of scene collection). Compensation for predictable sources can be performed with ground processing, but it is difficult to perform such corrections on-board. If spectral decorrelation is used, the compression algorithm must be designed to operate within the system specifications assuming the maximum tolerable misregistration. Onboard Compression The compression system itself introduces quantization noise and MTF changes. These changes are determined by the type of compression algorithm used and also by the amount of compression applied. Typically, compression algorithms may introduce a smoothing from pixel to pixel, blocking artifact (aliasing), and artificially decrease the signal to noise ratio through quantization. The image quality at the compressor input will effect the compression performance and likewise, the reduction in quality of the compressed image may overwhelm the intended system image quality budget and requirements. Therefore, system requirements must take into account the performance regime of the compression algorithm. Communications Channel The compressed data is transmitted to the ground via a communications data link. Typical data links are subject to transmission noise, which causes bit errors. If appropriate error correction safeguards are not imposed, this will introduce further changes in the data due to incorrect decompression of the corrupted data. Note that the introduction of bit errors may affect the outcome of decompressing only a few values or, in the case of variable length coding, many pixels [IO]. Ground Processing The compressed image data is decompressed on the ground and then run through several different processes which introduce additional changes. Many of the ground processing corrections, (e. g., geometric correction, radiometric balancing, band-to-band misregistration), are corrected using an interpolation/resampling process. These processes, in effect, introduce additional MTF-like losses, which change the image characteristics of the original raw sensor data. To compensate for this, simulation imagery can be generated with a more representative system MTF by resampling high resolution imagery that has been convolved with a known system MTF.

Multispectral Compression Design Concepts A simplified version of most compression systems can be easily described by a conceptual flow through three different stages: transform, quantization and compression. This con-

IEEE SIGNAL PROCESSING MAGAZINE

23

cept is useful in describing the functional parts of a compression algorithm. The first step, a transform, represents a method to convert image pixels into a form that is more amenable to quantization. This transform can take on many different forms, from linear prediction or orthogonal transform, to vision modeling or a simple pixel remap. This step is typically considered a “decorrelation” step because, in the case of unitary transformations, the resulting transform coefficients are relatively uncorrelated. The primary purpose of this is to allow each of the coefficients to be quantized independently of the others. Most unitary transforms also have the tendency to compact the image signal energy into relatively few coefficients. In this case, many of the coefficients hold very little energy and can be highly compressed. Other transforms, such as linear prediction, also act as a decorrelation step, but in a different way. By exploiting the pixel-to-pixel redundancy that exists between image pixels, a pixel can be closely predicted by its surrounding values. The resulting prediction errors are typically well behaved, decorrelated residuals that are easier to quantize. Following the transform is the quantization step. The goal of this step is to quantize the transformed data selectively to reduce the entropy of the data, while ensuring that unwanted distortion is minimized. This step reduces or changes the number of possible values of the transformed data. It therefore may change the coefficient values and introduce noise into the decompressed imagery. The last step of the process, the coding step, is where the actual compression takes place. Having gone through an entropy reducing step, the resultant quantized values can be coded to reduce their overall average symbol length. This step can be as simple as relabeling quantized values, but usually relies upon entropy codes [IO] to represent the data efficiently.

MSI Properties Relevant to Compression The volumes of data that will become available with future multi- and hyperspectral systems dictate the use of some type of data compression. Since most lossless compression schemes can achieve at best a 2: 1 reduction in the number of bits per pixel [ 111, lossy algorithms are likely to be required. Lossy compression introduces distortion into the pixel values and will therefore impact exploitation of the data. Radiometric calculations will be affected as will be a variety of automated classification algorithms now available. Though the nature of the distortions introduced into the data varies with the type of algorithm, all data compression schemes operate by exploiting redundancies inherent to the data. Single-band image compression relies mainly upon the spatial correlation of the image pixels to accomplish this task. Multiple, correlated spectral bands provide a third dimension in which redundancy can be exploited. A thorough understanding of the correlation properties of the data is critical to the design of any compression algorithm. First, however, we examine properties of the data sets.

I 00 -6

-

- J

-4 x

-

- 1

- -

-2 0 2 d stonce from o u t o c o i r e l o t on or14 n

4

1 6

5. Cross-scan slice C(s,U) of the autocorrelafionfunction C(m,n) of AVIRIS hyperspectral data.

’I

-I

I

r Or8ginal AVIRIS doto Dota oggregated t o m 7 band response

500

1000 1500 Band c e n t e r wavelength. n m

2000

2500

1

6 . Values C(I ,U) of the autocorrelation,function of AVIRIS hyperspectral data. 24

IEEE SIGNAL PROCESSING MAGAZINE

JANUARY 1995

Experimental MSI Data Both hyperspectral data sources and multispectral data sources are of interest in this examination. The former are generally characterized by a large number of narrow, contiguous spectral bands derived from an imaging spectrometer. The latter are generally characterized by a smaller number of bands of wider spectral response, e.g., Landsat or SPOT. The hyperspectral data source used in this work is the Airbome Visible Infrared Imaging Spectrometer (AVIRIS) [ 121. The spectral bands are spaced at roughly 10 nanometers (nm) beginning at a wavelength of 440 nm. Each pixel has a radiometric precision (dynamic range) of 12 bits and represents a GSD of 20 meters. Although the sensor collects 224 bands, some overlap in band centers is found where the individual spectrometers adjoin. For this analysis, a total of 204 unique spectral bands were retained in each scene. One scene contains the city of Corvallis and surrounding area in Oregon. Notable features include clearly defined urban boundaries, some small fields, and sinuous rivers and streams. The second scene is an agricultural area in Maricopa, Arizona. It contains mostly farmlands, with two large roads dividing the major land plots. The multispectral data source referred to in this work is the M7 scanner. Depending upon the acquisition conditions, up to 16 bands of data are available. Table 2 lists the 50% bandpass wavelengths for each of the first 13 bands: the remaining 3 bands are not used in this work. Note that the first two bands are collected at wavelengths shorter than the first AVIRIS band. Each M7 scene has been processed to a precision of 8 bits per pixel (bpp). The first M7 scene is a riverside industrial area in Detroit, Michigan. It contains a number of industrial plants, parking lots, paved roads, and rail lines. The second scene is a mountainous area in southern Califomia. It contains many geological features and some urban areas with buildings and paved roads. Two versions of each of these scenes are used: the original 5 meter GSD version, and a second version that has been degraded to a GSD of 10 meters.

0

500

1000 1500 Bond center wavelength. nm

2000

All of the data used in this work are cloud-free and have been collected under clear viewing conditions. All bands have been processed to remove sensor anomalies and spatially registered. Note that these corrections impact data compression algorithms, as misregistered and/or anomalous data may reduce both spatial and spectral correlations [9]. Such impacts merit further investigation as they are critical to both airborne and ground segment compression implementations. Spatial Properties Pixel-to-pixel correlation is the form of spatial redundancy that is most often exploited by single band image compression algorithms. The normalized autocorrelation function for pixels I(m,n) in an image is defined by M-1 N-1

where

-x

M-1 N-1

,LlI =

1 MNm=O

Cl(w4

n=O

and M- 1

2

1 0,= MN

-XI

2

2

(m,n) - pl

m=O (3) The behavior of the autocorrelation function near the origin reveals the degree of local redundancy present in the data. For example, differential pulse code modulation (DPCM) exploits this redundancy by forming a prediction of the next pixel value based on past values, and coding the residual error of the prediction. In addition, since high spatial correlation usually implies a lack of high spatial frequency content,

2500

7. Changes in entropy after spatial prediction for two AVIRIS hyperspectral scences (solid and dotted lines),and AVIRIS data aggregated to M 7 response (asterisks and crosshatches). JANUARY 1995

IEEE SIGNAL PROCESSING MAGAZINE

25

Or8ginaI bond entropy Entropy a f t e r predict8on from band 3 7 0 n m away Entropy o f t e r pred#ct#onfrom band 5 0 n m away

0 0

500

1000 1500 Wovelength, nm

2000

2500

-

0

_ 500

~

~. 1000

.

1500 Wavelength. nm

~. - , - , 2000

I~

~

2500

8. Band-to-band cor-relution of AVIRIS hyperspectral datu for selected wavelengths.

9. AVIRIS hyperspectral data coding gain due to band prediction ut different spectral distances fr-om target hand.

unitary transform based schemes may exploit this redundancy by carefully maintaining the more prevalent low frequency components. For the two AVIRIS test scenes, the behavior of the autocorrelation function in the cross-scan direction is shown in Fig. 5. The different curves represent different band wavelengths. There is little difference between the profiles for wavelengths of 784 nm and 1560nm, and each resembles the typical autocorrelation profile of a visible image. The rapid decay of the profile at 1362 nm indicates weak in-band correlation: this is not surprising since that wavelength lies in an atmospheric absorption region of the spectrum (see Fig. 3). The weak correlation that is present can be attributed to the smoothing from the interaction of the spatial modulation transfer function (MTF) of the sensor with the colored noise propagated to the sensor through the atmosphere. The profile in the along-scan direction is slightly higher: compression algorithm designers should recognize that significant correlation differences may exist due to the physics of electronic

scanning. The high correlation one pixel away from the origin extends through most of the AVIRIS spectrum, as shown in Fig. 6. Note again that most bands have a single pixel correlation value, equivalent to the parameter p for a first-order Markov image model, of approximately 0.9. This suggests that any individual band, except one in an absorption region, has spatial properties similar to those of visible imagery. The asterisks in Fig. 6 show the effects of aggregating (accumulating without spectral MTF correction) the individual AVIRIS bands falling within each M7 band’s 50% response range. The effect is to raise the in-band correlation slightly above that of the individual AVIRIS bands. While it is not possible to compare the correlation behavior of the M7 data directly with the AVIRIS data, it is similar. Table 3 gives the average single pixel correlation values for the two M7 scenes. Note that the correlation values increase as the GSD decreases. These correlation values at either resolution are sufficiently high to validate a Markov image model for each individual band. Successfully exploitating the correlation in data compression can be demonstrated through entropy analysis. For this work, the efficiency of a simple two-dimensional spatial predictor applied within each band is measured by the change in entropy between the original band and the prediction residuals. While this measure is most critical to lossless compression schemes, it also indicates promising areas for the controlled introduction of distortions by lossy algorithms. Figure 7 shows the effects of simple in-band prediction for the two AVIRIS scenes, and illustates several points. First, the gains obtained by prediction (and other redundancy removal schemes) are highly scene-dependent. For 12 bpp original data, these gains are significant given such a simple predictor. Note that as expected, no gain is achieved in the strong absorption regions. The bands aggregated to the M7 range show slightly higher coding gain than their original AVIRIS counterparts. This phenomenon is expected given their slightly higher correlation values as shown in Fig. 7. The M7 scenes show similar behavior, as noted in Table 4. While the coding gains differ between scenes and bands,

26

IEEE SIGNAL PROCESSING MAGAZINE

JANUARY 1995

r

i

~~

~

~

Table 6: M7 Multispectral Data Coding Gain Due to Band Prediction from Most Correlated Band

I

they are again substantial, given the original 8 bpp dynamic range and a simple predictor. The S meter GSD scenes experience greater prediction gain than their 10 meter counterparts, which is consistent with the correlation observations of Table 3.

Spectral PI-operties While spatial correlation addresses redundancy present in a single image band only, MSI compression techniques can be designed to exploit further redundancies that exist between spectral bands. Provided that the bands are completely registered, significant band-to-band correlation is present. even across large differences in wavelength. Figure 8 shows how three different AVIRIS bands correlate with other bands across the wavelength range. In-band, single pixel spatial correlation is typically near 0.9: however, there are extensive wavelength ranges over which band-to-band correlation exceeds 0.9 for any particular band (excluding those i n the absorption regions). For the 1016 nm band, a correlation greater than 0.9 exists across more than 400 nm of the spec-

trum. Large spectral correlations can even cross absorption regions, as shown by the 17 18 nm band. The spectral correlation of the M7 data. given in Table 5 , shows the same trends. The 5 meter GSD bands often have adjacent bands with which correlation exceeds 0.9. A dramatic increase in the spectral correlation is seen as the GSD is degraded to 10 meters. Not only are spectral correlations with adjacent bands higher, but spectral correlations with all bands in the M7 suite are increased. This result is due to the spatial MTF which blurs edges within the bands and therefore smooths the spectral profile of any given pixel. Once again, to be of benefit, the spectral correlations must be translated into gains in entropy. For this analysis, a simple linear function of a given band was used to estimate the pixels in a second band. and the entropy of the prediction residuals was compared to the entropy of the raw data. For the AVIRIS hyperspectral data, as shown in Fig. 9, spectral predictors were formed for bands SO nm apart and 370nm apart in the spectrum. Not unexpectedly, the entropy of the residuals decreases as the band used for spectral prediction is moved closer to the estimated band. Nonetheless, significant entropy gains can be obtained by prediction across large wavelength differences. In the case of the M7 data, summarized in Table 6, the spectral predictor of a given band was a linear function of the other most highly correlated band. The table again shows that significant gains can be obtained with simple spectral prediction. It also shows little additional entropy change when the GSD is raised. This is an apparent paradox. since Table 5 shows large increases in spectral correlation with GSD. Note, however, that the spectral correlation gains are largest for the most poorly correlated 5 meter bands. Since the predictor is based on the band with the best spectral correlation, little difference is seen in the entropy gained with changing GSD. This data does suggest, though, that the issue of band selection for spectral prediction may be less critical for lower spatialispectral resolution data. Finally, to illustrate the tradeoffs between spatial and spectral prediction, a mixed predictor equal to half of the sum

I b

;-

4

-

2a i[ L Y a y1

z

G

--

___

Spectral prediction only Mixed spattol/spectral predNct8on

4. I

.

-

.

-,

3-

t

i

C n '

-

5 2 a h Y

'

lt

- __

E __ 0

LI

Spectiol prediction only Mixed spotial/spectral

__~

0

1

PredlctiT-

500

!

-~

1000

1500

2000

2500

Wavelength, nm

JANUARY 1995

IEEE SIGNAL PROCESSING MAGAZINE

500

1x10

1000 Wavelenqth, nm

2000

2500

21

of the individual predictors was tested. Figure 10a compares the entropy decrease obtained with spectral prediction alone, from a band 10 nm earlier in the spectrum, with that obtained from a mixed spatial/spectral prediction, where the spatial prediction is in-band and the spectral prediction is also 10 nm earlier in the spectrum. Note for this extremely small spectral separation, the strictly spectral predictor produces consistently higher entropy changes (coding gain). In fact, comparison of the solid line in Fig. 10a with the coding gain for scene 1 due to spatial prediction alone (Fig. 7), indicates the spectral prediction is much more effective. When the wavelength interval is increased to 90 nm, however, as in Fig. lob, the coding gains of the spectral and mixed techniques are comparable. Evidently, for hyperspectral data, the preferred direction of prediction is a strong function of the spectral band separation. For the M7 data, the results of the two prediction methods are shown in Table 7. In this case, the mixed predictor increases the coding gains at the higher band numbersregimes in which the spectral predictor alone produces relatively small gains. On the other hand, the mixed predictor reduces the coding gains in some cases where the spectral predictor alone performs quite well. This observations suggest that an adaptive predictor, or one with selectable spatial/spectral regions, would perform well. Implementation of such a predictor is not trivial, however, as it simultaneously constrains and is constrained by the design of the imaging optics and scanning electronics.

Examples- Techniques for Spectral and Spatial Decorrelation We’ve previously shown that certain spectral bands, especially in the visible range of the spectrum, are correlated. This band to band redundancy permits one to apply various redundancy removing techniques. There are many different approaches to spectral decorrelation. Two such applications are described here and were utilized in the ALRSS compression studies. Other techniques were investigated independently and also will briefly be described. Band-by-Band Linear or Nonlinear Prediction With high band to band redundancy, there is an enhanced capability to predict the pixels in one band, given knowledge of the adjacent bands. Three dimensional prediction structures can be constructed by weighting neighboring pixels from within the same band and from adjacent bands to predict the current pixel value. This principle was used in the development of two different algorithms. A more complete description of each algorithm is found in [ 13,141. These two approaches are similar because they use the adjacent spectral bands for prediction purposes. They differ, however, in the quantization and compression strategies. In [ 131,the prediction is truly three-dimensional, with neighboring pixels and adjacent pixels from adjacent bands being used to form the prediction. The residual value between the predicted and the actual value is then quantized and compressed as in other 28

Table 7: Comparisonof Spectral and SpatiaUSpectral

DPCM techniques (3D DPCM). In [ 141, spectral prediction preceeds a separate spatial decorrelation step. The prediction occurs using adjacent spectral pixels only, followed by a spatial block discrete cosine transform compression of the prediction residual data. The spectral prediction technique relies upon a nonlinear affine transform (affine/DCT). Thus, the prediction residual values are treated as single band images. In both cases, significant coding gain has been achieved using the spectral prediction technique. Spectral Decorrelation Transforms The optimal decorrelation transformation is the Karhunen-Loeve transform (KLT). The KLT is often used in multiple band imagery to decorrelate the different spectral bands. With the KLT, a vector consisting of a set of pixels from adjacent spectral bands is decorrelated in the spectral dimension only. This decorrelation transformation compacts a majority of the spectral signal energy into a relatively small number of coefficients. The resulting transformed bands, herefore referred to as eigenbands, can then be treated similar to individual image bands and compressed using the traditional single band image compression techniques. Significant gains are obtained by compressing each eigenband in proportion to its relative importance. For instance, the first eigenband holds the most signal energy and therefore should be compressed with the least amount of distortion. As the relative importance of each eigenband decreases, the bit allocation to that band should also decrease. In the limit, some of the eigenbands could be represented only by their mean values. One approach to spectral decorrelation and quantization is outlined in more detail in [15]. This algorithm development used a block discrete cosine transform based compression algorithm to compress the eigenbands (KLT/DCT). Though the KLT adds significant complexity to the compression algorithm, the gains in compression performance are also significant. Unfortunately, no fast version of the KLT exists. Other fast

IEEE SIGNAL PROCESSING MAGAZINE

JANUARY 1995

algorithms, such as the cosine transform are not effective on spectral data. Therefore, this spectral decorrelation technique can be computationally intense.

showed change. Figures 1 I , 12 and 13 show examples of color composites of three different images from the visual assessment. Figure

Other MSI Compression Technology Numerous other MSI compression techniques have been investigated independently of the ALRSS studies. Several methods look promising and warrant mention here. In recent years, the wavelet transform has emerged as a promising decorrelation transform. Several applications of the 2-dimensional and 3-dimensional wavelet transform have been simulated with promising results. Using thc KLT in tandem with a 2-D wavelet compression algorithm 01-3-D wavelets was also investigated [ 16, 17, 181. Vector quantization (VQ) has also been applied to MSI with significant success. The vector nature of registered MSI makes it naturally favorable for VQ. Several variations of MSI VQ have also been investigated [ 19. 20, 211.

Simulation Results and Artifacts The ALRSS compression studies were designed to investigate the nature of compression distortion and its impact on typical exploitation activities. Many different metrics were developed in an effort to identify and understand the interaction of compression and exploitation. Two exploitation approaches were evaluated: visual, and machine. A series of metrics were developed to characterize the statistical changes in the compressed imagery. In addition, the impact of the compression on exploitation was evaluated for several exploitation processes by comparing results obtained using uncompressed and compressed images. The goal was to identify those metrics which were most correlated to the exploitation task. This approach has proven more difficult in the past when trying to correlate metrics to visual fidelity. This is mostly due to the nonlinear nature of vision. But machine exploitation algorithms may be more correlated to standard statistical nietrics.

I . Oricqitiuluncl I .20 bpp. 3 - 0 DPCM compressed inia,qe.

Visiiul Assessmc~tit

For visual quality, 20 different images were ranked and compared in a formal evaluation. The MSI scenes were processed with two different compression algorithms (KLT/DCT and 3D DPCM) at compression rates ranging from 2.2 bpp down to 0.15 bpp (total of 240 images used in the evaluation). The original imagery had a variety of scene content. band selections. and sensor characteristics (e.g., MTF, SNR, defects). Twelve users from the scientific community and others trained to visually exploit imagery were asked to compare the rate and rank the original and compressed imagery in a formal controlled evaluation. The ranking was done on a basis of quality as compared to the original image. The results of this evaluation showed that quality and interprerability of MSI is not affected for 80% of the examples for compression rates down to 1.2 bpp (KLTDCT), and 2.2 bpp (3-D DPCM) [22].

JANUARY 1995

IEEE SIGNAL PROCESSING MAGAZINE

29

1 1 a shows an original, true color composite of an industrial and residential site with a resolution of 2.5 meters. Figure 11b is the same image compressed with the 3D DPCM algorithm from 8 bpp to 1.2 bpp. Figure 12a is an infrared color composite, emphasizing vegetation. The scene, with a resolution of 5 meters, includes an airfield, water treatment plant, freeway, and residential areas. The compressed version in Fig. 12b simulates the affineDCT algorithm at 0.74 bpp. Figure 13a is a true color composite of a coastline residential area, collected at 10 meters. Figure 13b shows the same scene compressed to 0.79 bpp with the KLTDCT algorithm. These scenes were selected as examples to show both the quality and the contrasting distortions that occur with each compression algorithm. Metrics and Machine Exploitation Results Many metrics were developed and compared against true exploitation results in an attempt to measure the potential performance of the compression algorithms against a multispectral exploitation task [23] and also to obtain metrics that can be used to predict performance wilhout actually completing lengthy exploitation tasks. Along with the typical error metrics that are applied to imagery, several MSI specific and image feature specific metrics were applied to the compressed imagery. The MSI specific metrics include likelihood ratio, principle component analysis, simultaneous diagonalization, PCICE [24], and spectral vector differences. These metrics were supplemented with several typical exploitation activities, including supervised and unsupervised classification. The metrics were used to evaluate a set of 23 images, many of which were used in the visual evaluation. Two algorithms simulating 7 compression rates were examined, and many initial observations were drawn from the process. In general, the metric evaluation process showed that lossy compression can be applied to MSI without significantly changing the exploitation results. Specific observations showed (1) the two compression algorithms that were tested introduced insignificant measurable loss down to a compression rate of just below 2 bpp; (2) there were no real significant globally measured differences between the two algorithms until the compression reached approximately 1.5 bpp. At that point, the more complex KLTDCT algorithm always measured less distortion; ( 3 ) though not included in the original evaluation because the development contract started later and the affineDCT algorithm developed under the ALRSS study, which uses spectral affine prediction followed with spatial compression, measured almost equivalent statistical errors to the KLTDCT algorithm. Though further evaluation is warranted, using the affine predictor as a substitute for a KLT is significant because the KLT introduces significant complexity to the compression algorithm design and may not be as ammenable to spacecraft implementation.

Summary MSI compression has evolved into a viable solution for band limited communications problems in current and future re30

e

E

13. Original and 0.79 bpp KLTlDCT cornpressed image

mote sensing systems. MSl compression technology continues to mature as research identifies the interaction of compression distortion and typical multispectral exploitation tasks. Understanding of both compression artifacts and exploitation techniques must proceed in parallel because sensitivity to errors (distortion) must be addressed for a much larger usage base. This article has provided an introduction to the expcrience gained from the ALRSS compression development, and an insight into the challenges of MSl and space-based compression algorithm design. The ALRSS studies provide an initial look at the challenges of designing and evaluating MSI compression systems. The results of these studies have shown that compression rates between 2.2 and 1.2 bpp are viable and feasible for space-based applications today. MSI systems can be designed to include compression without changing the significance of the final image product. The authors hope that this information will not only assist those interested in compression design, but also that the remote sensing community as a whole will better understand and accept the potential benefits that come with compression.

Acknowledgement The authors would like to thank Lt. Col. Kevin Rose and Capt. David Ehrhard of the Defense Landsat Program Office for their support and interest in pursuing MSI compression. A portion of this work was also supported by The Aerospace Corporation Sponsored Research program.

IEEE SIGNAL PROCESSING MAGAZINE

JANUARY 1995

Val D. Vaughn and Timothy S. Wilkinson are with The Aerospace Corporation, El Segundo. CA. Author Vaughn may be reached at The Aerospace Corp.. 13873 Park Center Road, Herndon. VA 2207 1.

References

JANUARY 1995

IEEE SIGNAL PROCESSING MAGAZINE

31