A digital image system for atmospheric research

Cornput.d~Elect.EngngVol.5, pp. 345-364 © PergamonPressLtd.. 1978. Printedin GreatBritain

0045-79~17811201--0M5/$02.0010

A DIGITAL IMAGE SYSTEM FOR ATMOSPHERIC RESEARCH? MICHAEL ANDREWS~ a n d ROBERT FITCH Department of Electrical Engineering, Colorado State University, Fort Collins, CO 80523, U.S.A. (Received 5 April 1978; received for publication 21 September 1978) Abstract--This paper describes the architecture of a medium scale digital image processing system developed as a research tool for analysis of meteorological data. The system is also being used for research on efficient image processing systems. Four qualitative performance measures for any image processor are introduced with specific application to the present machine. Preliminary results with noise reduction algorithms in satellite data are presented. Lastly, the versatility of the machine as a test bed for architectural studies of the computational structure of image processors with a microprogrammable control unit is discussed.

1. I N T R O D U C T I O N

Our stratosphere, residence of the world's weather, has largely been observed beneath the clouds. Today, however, the launch of the fifth Synchronous Meteorological Satellite (SMS-5) will herald a new dimension of contiguous weather observation from above when the five satellites will each scan 20% of the earth and transmit on a downlink visible and IR intensity data via binary-coded pixels every half-houri1]. Unfortunately, at present such voluminous data is received at transmission rates which rapidly become archived, mainly due to primitive real-time data processing systems. Only recently have we been able to rapidly use data for current analysis. In addition, we now can interpret complex weather behavior at mesoscale levels, in severe storms and hemispheric cloud motions because of recent advances in digital display systems, one of which is described in this paper[2]. These new analytical tools greatly assist in the current pursuit[3-12] for reliable and cogent weather prediction. Typically, such systems employ digital imaging techniques which have (a) high resolution, (b) stable image displays for microscopic analysis, (c) high speed image sequencing for temporal studies, (d) image enhancement for highlighting specific intensity regions (contour banding), and (e) real-time cursors, zoom, translate, and rotate for interactive capability. The need for such requirements and quantitative measures for evaluation have been proposed by [13, 14] and discussed in [15] and will not be repeated here. Of considerable interest to atmospheric research is the ability to replicate weather maps transmitted in real-time from satellites and portrayed as high fidelity Video scenes coupled with flexible graphic overlays for political and geographical registration. However, in the past, these techniques have required manual generation of picture composition by photomosaics necessitating extensive wet chemistry support (for developing, enlarging, etc.). Now, we can employ recent advances in solid state electronic technology (semiconductor memory, microprocessors and high resolution video monitors) which virtually automate all phases of the weather map generation. Other attractive features, now feasible, include enhancing invisible scenes (hot air masses), etching details by pseudocolor and overlaying topographical as well as geographical features of landscape. Such capability which exists to some extent in all large scale systems[6--9] relies heavily upon menu-driven software packages desirable for reducing inexperienced-user training. In essence, several modular routines are provided for the user via either a video screen tabloid entry[6], trackball/cursor[9], or special function keys[7, 8]. Such software coupled with annotated cursor, zoom, translate and rotate capabilities in the above systems as well as in small systems [10, 11] enhances the human manipulative skills in a tight man-machine interactive loop. However, to ensure rapid analysis and preserve continuity of thought, quick machine response tThis work was supported by a grant from Army Research Office under grant No. DAAG29-76-G0324. ~Address all correspondence to Michael Andrews, Department of Electrical Engineering, Colorado State University, Fort Collins, CO 80523, U.S.A. 345

346

M. ANDREWSand R. Fm'~

to operator compounds is essential. Commonly, then, single-user systems/Ill employ efficient software while multi-user systems [6-10] must rely, not only on efficient software, but additional terminal intelligence. Here, the large scale central computer is supported by several distributed minicomputer or microcomputer terminals. In this paper, we describe the Colorado State University All Digital Video Imagery System for Atmospheric Research (ADVISAR) designed and developed by the author and by Fitch as part of his M.S. thesis [16] primarily as a research tool for analysis of atmospheric satellite data. The objective was to achieve, in a single yet modestly sized system, most capabilities separately found in previous imaging systems as well as provide a test bed for eventual employment of very large scale integrated circuits (VLSI)[171, including microprocessor~ and advanced memory devices (e.g. CCD's and bubble memory). In the paper, we describe the command processor, memory system, video and cursor units, followed by remarks on the current configuration. At present, the system is interfaced to an HP 2100 with disk, magtape, CRT and TTY, and discussion to follow reflects interface considerations for typical duplex port operation with the HP 2100. The paper concludes with a discussion of continuing research into alternative digital image system architectures addressing control and arithmetic alternatives~ 2. DESIGN CONSII)ERAI1ONS Our motivation for developing such a research tool is brought about by potential advances/18] both in hardware and software for meteorological data processing. To this extent we desired to capitalize on these advances with a versatile configuration upon which to test emerging technologies (microprocessors, CCD's, bubble memories, advanced digital multipliers and digital filters) as well as reconfigurable architectures for both expanded and reduced versions of ADVISAR. Expanding both memory planes and depths (essentially, picture framing and resolution) and input/output ports is desirable for applications as small as temporal studies of mesoscale weather systems and as large as synoptic weather analysis. Reducing both memory and I10 ports is desirable for limited applications as might occur in remote weather advisory stations such as FAA flight service stations (FSS) to aid general air navigation where quasi-static frames of local weather scenes may be viewed by pilots during pre-ffight weather briefings. To satisfy these conflicting requirements of versatility and compaction, both a microprogrammable control structure and a modular memory and l/O port scheme was devised for ADVISAR, The ability to microprogram nearly ati of the system tasks led to a ROMcentered design which facilitates an alterable memory and port architecture, and potentially, reduces redesign during later development stages/19]. More importantly, the microprogrammable nature of the digital image system permits investigation of novel signal processing software for pixel management (interpolation, zoom, rotate). The digital format of satellite data and the reliability of digital hardware naturally leads to all digital data processors. Such configurations, now found, incorporate low cost solid state memory replacing inferior analog memories such as video disks, tapes, and vidicon storage tubes; and employ microprocessors/20, 21] with hardwired floating-point arithmetic instead of analog processors for pre- and post- filtering, averaging, noise cancellation, and signal enhancement. Moreover, digital data processors can easily support sophisticated numerical algorithms as well as contribute to the desirable interactive capabilities required of weather analysis tasks. Quantitative measures to evaluate performance for digital imaging systems include: (aj algorithmic complexity, (b) graphics capacity, (c) image sequencing, and (d) psuedo-color enhancement. Algorithmic complexity involves the ability to perform both pointwise and spatial pixel processing operations (FFT, WARP, de-WARP, spatial convolution, etc.) for spectral transformations as well as data compression with Hadamard, Karhunen-Loueve, Slant, Fourier algorithms. Graphics capacity measures the ability to draw line segments for topological registration or to artifically contour highlighted spatial or temporal features. For example, banded dewpoint contours illustrate convective properties of moist air and employing operator driven trackball, cursor, and/or function keys can facilitate such analytical techniques. Image sequencing relates to the power of the system to classically portray temporal properties of an atmospheric study via rapid refresh of video images. Although rapid sequencing is readily solved by simple memory and clocking schemes; pseudo-color enhancement, our newest and

A digitalimagesystemfor atmosphericresearch

347

least understood measure of system performance [22] requires on-line computing power similar to, but not exactly, that required for algorithmic complexity. Whereas, algorithms can be effectively executed by novel architectures (e.g. STARAN[23] which utilizes several parallel processors embedded within permutation networks) in an off-line mode, pseudo-color enhancement relies upon on-line processing power to manipulate pseudo-color operations. Arithmetic capability, then, is most effective when embedded in the video loop similar to the scheme of [11] wherein +, - , *, [, are hardwired operations conveniently provided just prior to video output. In the final analysis, however, the four quantitative measures only partially assess the capabilities of any digital imaging system. As noted already [24-26], tolerable errors for any given algorithm or technique rest ultimately upon visual analysis of the enhanced image. 3. SYSTEM DESCRIPTION The general features of the current system (see Fig. 1) include: (a) computer control of both internal data routing and user interaction, (b) computer command interface with manual diagnostic test panel, (c) data input multiplexing for data stream merge and/or separation, (d) eight solid state memory planes, eight bits deep for 512 x 512 pixel storage reconfigurable in any combination from 64 one-bit planes to 8 eight-bit planes and randomly accessible at varying access cycles in read or write mode, (e) memory output multiplexing to route the data stream to any of five look-up tables (LUT's), (f) five video rate and format output ports via the LUT's with high speed digital-analog converters (DAC's), (g) a video tape recording output port with automatic red-green-blue (RGB) to NTSC [27] conversion for recording format, (h) a microprogrammable timing and control unit for variable recording, transmission, playback, video display and direct-memory-access (DMA) by computer, and, (i) a high speed output port for self-diagnostic tests. 4. COMMAND PROCESSOR The primary control link between the computer and ADVISAR is the Command Processor (see block 2 of Fig. l) which can perform both in automatic or manual mode. In automatic mode, the computer issues necessary commands via programs generated for various weather map manipulations. At present, 17 primary commands or instructions are available and are listed in Appendix A. Note that each command have several sub-commands or select codes. For example, the Set Output Multiplexors (SOM) command can issue the seven select codes for the output muxes which route appropriate bits to various look-up tables (LUT's). Remaining primary commands behave similarly and, when combined with their respective subcommands, can generate up to 58 instructions to ADVISAR. These instructions are then decoded by the Command Processor (much like the instruction register of a conventional computer) to actuate several signal and control lines. In the current configuration, commands essentially control input multiplexors, output multiplexors, memory addressing and cursor routing which are primarily hardwired functions. More importantly, all instructions for the command processor unit can also be generated by a manual test panel which conveniently serves as a diagnostic module for ADVISAR. Static and dynamic diagnostics can be exercised via this manual panel off-line from the computer if desired. 5. MEMORY SYSTEM ADVISAR is conceptually configured as a memory array much like STARAN is configured as an array processor. Bits or words can be vectorized in both instances; however, unlike STARAN which has a multitude of processing elements for data stream computation, ADVISAR uses memory planes reconfigurable on demand in either bits or words in any vector pattern. This reconfiguration occurs dynamically (between vertical or horizontal retrace of video) by Command Processor instructions which latch various input and output multiplexors as shown in blocks 3 and 5 of Fig. 1. The same principle of "single-instruction-stream multiple-data-stream" for STARAN processors applies to the ADVISAR memory blocks. The 16 bit data words from the computer interface can be streamed out to several planes of the memory simultaneously via the input multiplexors, one of which is shown in Fig. 2. Upon exiting memory planes, the data stream (of any width up to 8 bits) can be repacked in any fashion by the output multiplexors.

M. ANDREWSand R. FITCH

348

o

I

l f

i

~

I

o,~

m~

®

f

i

8=

m

~® I

I I

i

I g_

I---

T

r ....

I

I

_.,@

,

~

i~ , I~

t

I

, ® ~I ®

IL

I

,~,,

I

,®

t ~

~\~-.<

t TI

.<

-? -3' tl

-e-_~e-:

il it ~-----t i- ~

"J:

I

:E

,

.....

. . . . .

T

o

i

I I

i I I 1

®

® i I

. . . . . . . . . . . . . . . . . . .

l -,t

A digital image system for atmospheric research

[ 1:64 (~)

BIT

~]_ @1:64

BIT I

BIT2

FROM

/

COMPOTE./"" DATA

3

349

~

,J 1:64 [

i

~

'2" IS

64

64

; 6

64

!

,

TO MAINMEMORY RANDOMACCESS PORTS

/o,,, J,-,, I ': Fig. 2. Input multiplexor.

Two useful operations, which the input/output multiplexors and memory planes rapidly perform, are the mirror and shift operations. Mirror essentially transposes right and left images, while shift operates on individual pixels to scale contrast up or down. The objective of this memory loading scheme is to reduce execution time required for manipulating bits and words in the computer itself and provide for instantaneous picture replication on several output channels, while allowing for future incorporation of arithmetic capability in the separate output channels. The primary mass storage medium consists of NMOS random access memory/28]. Supporting the memory are the address/multiplexor unit and a timing/control module. Each block (see Fig. 3) currently contains two data input/output ports, a half-duplex random access read/write port (for computer interface) and a sequential or video read port. In sequential access, .16 consecutive one bit words are retrieved, loaded into a 16 bit high-speed shift register and clocked out as a single bit/plane at video data transfer rates, while parallel access is commonly obtained via the half-duplex port. Maximum transfer rates for the video and RAM ports exceed requirements for conventional 30 frames/sec refresh, and DMA for the HP 2100, respectively. One major engineering task involved the design of a flexible addressing scheme, described next, to provide for refresh of the dynamic RAM's regardless of the state of the ADVISAR without inhibiting video presentation (introducing flicker, lines, streaks). As shown in Fig. 3, three multiplexed addresses, MADDR, VADDR, and RADDR, make up the memory address. The random access address (MADDR) is 18 bits wide, externally loaded and automatically incremented after each access. The sequential mode or video address (VADDR) is 14 bits wide

350

M. ANDREWS and

R.

FITCH

Q W

~

t-- 11:

~,,

O.

0

i G0

w

.,.¢ I,i Z

t~ pi Z I-ee~

z O

F-- . . . . .

~,)

......

0 .-I a,-,

/

N v

>.n,0 :E w =E /

=

0

I I II

-&

// m

$ I

I--

E

E r • "

--~-I

q

I

F --

_m_f_'

W

'I I I I

p_

L-z~~ ...........

_l

J . . . . .

-.I

,*-.,i I--! (.91

F

~u LE

A digitalimagesystemfor atmosphericresearch

351

(bits 0--3 ore zero) and is incremented by (VAINC) and cleared by (VACLR) through external video timing. The upper 9 bits of the address can be viewed as a Y-address and the lower 9 bits as an X-address. On displaying the memory on the video monitor, Y is the vertical direction and X the horizontal. To read through the memory sequentially using MADDR simply requires incrementing the address with Y the most significant and X the least significant bits. In video or sequential access from memory to the monitor, the even Y-addresses are read first and then the odd (interlace). For this reason the Y-address in VADDR is rotated so bit 8 is in bit 0. In incrementing VADDR, bit 8 of the X-address carried into bit 1 of the Y-address and bit 8 of the Y-address carries into bit 0 of the Y-address. Bit 0 of the Y-address is actually a field indicator for interlace, so the field indicator FRMA (from the video sync generator) is used as bit 0. The third address register is the refresh address RADDR. This address has only 6 active bits (0-3 and 10-27 unused) since a plane can be completely refreshed by incrementing address bits 4-9 and by enabling the refresh option. In contrast to the main memory blocks which, when considered with the input/output multiplexers, dominate bit/pixel management, the five look-up tables (LUT's) provide the real-time transformations on the data stream. Among these functions for the high speed 256 x 8 RAM's/29] are scaling and histogram generation useful to radiometric conversions, pseudocolor enhancement of multi-band imagery, space variant radiometric correction for removal of vignetting and/or camera shading, and dimensionality reduction or bandwidth compression (when coupled with the arithmetic processor). The LUT's primarily perform in two states, either as mass storage for digital computer access/retrieval or for high speed imagery functions, with control provided via VSRMD (see Fig. 4). In the computer state, LUT's are loaded with the desired output functions (e.g., exponential or anti-log, similarity, contrast shading, etc.); while in the video state the LUT's serve strictly as table look-up memory. Bit/pixel shuffling is primarily performed via the input and output multiplexors. Data words from the computer are stored in the memory blocks via eight Input Multiplexor units (see Fig. 2) which route one-bit of eight to any of 64 memory planes. Selection is made by a six-bit latch loaded from the Command Processor unit. Data retrieval from the eight memory-blocks is routed via eight Output Multiplexor units (see Fig. 5) to the LUT's. Similar to the Input Multiplexors, the Command Processor unit generates a five-bit code to select various paths via the output multiplexors. Each of the five-line output busses on the multiplexor is wired-ORed bit-by-bit with seven other multiplexor outputs. 6. VIDEO SYSTEM In the current configuration of ADVISAR, the video system consists of two raster scan video monitors (black/white and color), video synchronize generator, timing unit and interface

TO COMPUTER =

r.

8 ''

]_

2 / FROM

8

COMPUTER FROM OUTPUT MULTIPLEXORS

/

8

DATA

TO rA~R.

_

-I ADDRESS REGISTER/I q

OTHER LUT UNITS

/

8

EXTERNALLY LOADED

,

I

• e

COUNTER

IN-50

MEMORY

I

TAND IMING

CONTROL

VSRMD~ Fig. 4. LUT unit.

!

DATA OUT

r

DIGITA L/ANALOG CONVERTER UNIT

352

M. ANDREWSand R FITCH 5 FROM OTHER 7 OUTPUT MUX UNITS

TO BIT ~ OF EACH LUT ADDRESS MUX

FROM ONE MAIN MEMORY BLOCK SEQUENTIAL OUTPUT PORT

BITI

5

f

BIT ~ - I--

1:5

2-3-4-5-6--

1:5 1:5 I;5 1:5

7--

I p- BIT 2

1:5

1:5 1:5

-4-4•

5

5

L

/

5

I~- BIT 7

LATCH

Y5 FROM COMPUTER COMMAND BUS

Fig. 5. Output multiplexor.

circuitry. It was principally designed for high pixel resolution, video bandwidth for 30 frames/sec refresh, and system versatility. Typically, the first two objectives can be achieved by employing high quality monitors, on-line convergence algorithms (to correct for beam spreading in 3-gun color displays), and conventional clocking circuitry. However, system versatility, for example, the ability to change from NTSC (American[27]) to PAL (European[30]) television sync format with .minimal hardware modification requires unconventional procedures. In ADVISAR, a novel microprogrammable control unit was imbedded in the video sync generator to provide both the power and flexibility of the video system. Furthermore, this microprogrammable feature permits more stable timing circuitry when driven with a phase-lock loop either externally (60 Hz) or internally (crystal oscillator). The primary role of the video sync generator (see Fig. 6) is to provide necessary synchronization for horizontal and vertical retrace; however, most systems also rely on the sync generator to clock data into the video monitors. For stable images (jitter-, flicker-free), synchronization to either the horizontal retrace frequency, fH, or the vertical retrace frequency, f~, is sufficient. In our case, the data clock was synchronized at constant phase difference with an integer multiple of fH, as shown in Fig. 7. To do so, we specify s o m e f~ck as a multiple of twice fH and combine f,,, fH and fsck with a phase-lock loop on either of two base frequency sources, fr~f (60 Hz power line frequency) and fco (a crystal oscillator), fv, f . and fs~k then provide the primary drive signals to the pulse forming circuits of drive (YVD, YHD), blank (YVB, YHB), equalization (YEQV, YEQH), and serration (YSRV, YSRH) where (YXXV) and (YXXH) correspond to vertical and horizontal synchronization, respectively. The relationship of the vertical drive signals with fv and f , is shown in Fig. 8. For NTSC format, equalize and serrate require three horizontal sync (YHSYNC) periods and, for our monitors, the blanking intervals for vertical and horizontal blank are 1 msec (tvB) and 10/~sec (t,vs), respectively. Since, for color displays, the composite sync (TCSYNC) predominantly governs framing operations, drive, serrate and equalize must be coupled with the blanking intervals. However, although fields A and B (assuming interlaced scanning) modify the composite sync to account for half lines at picture top and bottom, the vertical sync signals can be obtained by dividing twice the horizontal frequency by an odd number and, with a 512 complete visible line format,

A digital image system for atmospheric research

I

--

+ +++ ~o

•

,,

,,~

--l,

++

t

_

+I+++ ~,.,,

lq

N

~

353

~

++ii

I

~

II

i

/

+I i

,r

C

N

_ /

.=13 '.~ UJ f I X ,,,>

/I

i

',

I

Jl

,, ~

,

~,

~ T ~ ,

I® i

'

. I

•

t',

~

'-! ....

+

L-

®

r

',®

~.~|r-~',/

l/

li"

+,

,