IMAGE PROCESSING USING A CMOS ANALOG PARALLEL ARCHITECTURE Ion Vornicu1, Liviu Goraş1,2
1
Technical University “Gheorghe Asachi”, Faculty of Electronics, Telecommunications and Information Technology, Iasi, Romania E-mail:
[email protected],
[email protected] 2 Institute of Computer Science, Romanian Academy, Iasi Branch
Abstract–The possibilities of image processing using the spatio-temporal dynamics of a Cellular Neural Network (CNN) type CMOS analog parallel architecture are investigated. It is shown that the network, consisting of an array of identically coupled identical cells, can be programmed to perform various spatial filtering operations and segmentation in a very short time and with low power consumption. Comparison with previously reported segmentation techniques and simulation results are discussed. Keywords: Cellular Neural Networks, spatio-temporal dynamics, CMOS log-domain implementations, edge detection, image smoothing, image segmentation.
1. INTRODUCTION Analog parallel architectures have found applications in high speed processing of a large amount of input data. Basic analog image processing can be performed before A/D conversion in order to minimize the datasets to be further processed by the digital part. In other words, on-chip preprocessing extracts the relevant information from the input image, so that the digital processor will need reduced overall time and power consumption. Still analog to digital conversion is the weaker link in the processing chain represented by the imager - A/D converters in terms of time consumption, a significant challenge being to design high speed, low-power pipeline A/D converters [1]. Many implementations of image segmentation can be found in the literature. Among them, interesting performances have been obtained with a non-linear resistive grid. The nonlinear resistor used in such networks, so called “resistive fuse”, can be implemented in different ways. Several circuit solution are based on Chua’s negative resistor [2], on transmission gate controlled by a combinational digital circuit [3], on pulse-modulation techniques [4], or on switched capacitor [5]. In this paper results obtained in the segmentation domain using the proposed spatial-temporal filter [6, 7] are compared with the ones from the literature. Moreover, a modified architecture that includes the selective filtering techniques in order to preserve edges is proposed. Finally several results obtained with the log-domain, transistor level analog parallel architecture, whose dynamic has proved to be useful for spatial-temporal image filtering, will be shown, when the time evolution of this network is frozen after cell saturation has been reached. The filters are designed such that the initial images are loaded from an external memory or an embedded CMOS imager and the output data can be read out line by line after the temporal dynamic of the network is “frozen”. The
following basic image processing operations are considered: edge detection, smoothing, noise cancellation, contrast enhancement and segmentation. The implementation of the first four operations for a bidimensional image, using the proposed analog parallel architecture [6, 7] will be discussed.
2. THE ARCHITECTURE OF A 2D LOG-DOMAIN ACTIVE PIXEL In the following, a 2D array of MxM identically coupled identical cells network having the architecture presented in Fig. 1 will be analyzed. Each cell is connected with its neighboring cells belonging to a maximum second order neighborhood. The boundary conditions are ring, but can be zero-flux as well as it will be shown for segmentation using selective filtering technique.
Fig. 1. 2D array architecture.
The system level structure of a cell is presented in Fig. 2. The voltage controlled current sources are implemented with OTA’s (operational transconductor amplifier) using an adaptive biasing technique in order to decrease the DC power consumption.
Fig. 2. System level 2D pixel structure.
The linear differential equation valid for the (i,j) node, taking into account ring boundary conditions for homogeneous networks (Alf1=Arg1=Aup1=Adw1=A1; has the following Alf2=Arg2=Aup2=Adw2=A2), expression:
978-1-4244-5781-6/10/$26.00 © 2010 IEEE 461
blurring on the other side can be performed using the xi , j −1 network configured to have a high-pass or low= A2 ( xi − 2, j + xi + 2, j + xi , j − 2 + xi , j + 2 ) + A1 ( xi −1, j + xi +1, j + same dt pass spatial frequency characteristics respectively. xi , j +1 ) − A0 xi , j , ∀i,j = 0..M-1 (2) In order to implement a high-pass filter, the change of sign in equation (4) is equivalent to complement the where “xi,j(t)” is the (i,j) node voltage. The above linear differential equations can be structure from Fig. 3. Fig. 4 presents the obtained translated into the log-domain [6], by making a results for a chessboard type input image. convenient change of variable. Applying the following change of variable: x (i, j ) = I S eα vx ( i , j ) - I S (3) and considering now “xi,j(t)” as a current, and denoting xi,j(t) = x(i,j) and vx,ij(t) = vx(i,j), the linear Imput image a) a’) differential equation (2) becomes: C
dxi , j (t )
•
CI Sα v x (i, j )eα vx (i , j ) = A2 I S (eα vx (i −2, j ) + eα vx (i + 2, j ) + eα vx (i , j −2) + eα vx (i , j + 2) ) + A1I S (eα vx (i −1, j ) + eα vx (i +1, j ) + eα vx (i , j −1) + eα vx (i , j +1) ) − A0 I S eα vx (i , j ) − I S (4 A1 + 4 A2 − A0 )
(4) α vx ( i , j )
e
Dividing the nodal state equation (4) by using the following notations:
b)
and
b’)
Fig. 4. a,a’) edge extraction obtained with a linear (a) and log-domain (a’) filter; b,b’) smoothed image obtained with a linear (b) and log-domain (b’) filter – freezing the network after 435ns.
C x = CI S α
x offset = I S ( 4 A1 + 4 A1 − A0 ) = I X 0 A1 I S = I A1
Other results obtained for gray-scale images are presented in Fig. 5.
A 2 I S = I A2 A 0 I S = I A0
the new (i,j) state equation takes the form : •
Cx v x (i, j ) = coupling − I X 0 e−α vx (i , j ) α ( vx ( i − 2, j ) −vx ( i , j ))
coupling = I A2 (e α ( vx ( i , j − 2) −vx (i , j ))
e
+e
α ( vx ( i , j + 2) − vx ( i , j )
+e
(5)
α ( vx ( i + 2, j ) − vx ( i , j ))
+
α ( vx ( i −1, j ) − vx ( i , j ))
) + I A1 (e
Input image
Input image
+
eα ( vx (i +1, j )−vx (i , j )) + eα ( vx (i , j −1)−vx (i , j )) + eα (vx (i , j +1)−vx (i , j )) ) − I A0
The above equation can describe a low-pass/ stopband filter for positive coupling coefficients and a high-pass/ band-pass one for negative coefficients, providing a band of unstable modes as it was previous presented in [6, 7]. Based on the above equations, the log-domain implementation of a 2D pixel is given in Fig. 3.
1a
2a
1a’
1b’
1b
Input image
3a
Fig. 3. Log-domain implementation of a 2D pixel structure.
3. EDGE DETECTION AND SMOOTHING Edge detection and contrast enhancement on one side and smoothing, i.e., noise cancellation or details
2b’
2b
Input image
3a’
4a
4a’
Fig. 5. Filter used for high-pass/ low-pass filtering for different input images; 1a) high-pass filtering after 55us; 1a’) high-pass filtered version with 300mV threshold; 1b) low-pass filtering after 105us; 1b’) low-pass filtering after 78us, performed by log-domain implementation; 2a) highpass filtering after 71us; 2b) low-pass filtering after 160us, with the linear network; 2b’) low-pass filtering after 71us, using a log-domain filter; 3a,a’) high-pass filter with 300mV threshold for edge detection, after 28us; 4a) stopband filtering using asymmetric spatial frequency characteristic with higher LP amplification; 4a’) stop-band filtering using asymmetric spatial frequency characteristic with higher HP amplification.
462
4. SEGMENTATION Image segmentation aims at objects recognition; borders positions estimation for a moving object and image compression. It is known that linear resistive grids can be used only for smoothing operations. However, if the so called “resistive fuses” are used instead of linear resistors, the basic network is fragmented into zones that have the same spatial contrast that do not surpass a given threshold. Our analog architecture is able to perform this kind of operation without fragmentations techniques. This behavior is obtained by programming the network in a low-pass configuration. Since the initial differences between similar contrast level are also amplified by the low-pass filter, it somehow compensate the unlike edges filtering, finally resulting that the need for fragmentation is not a must in order to obtain a segmentation effect. It is hard to appreciate the behavior of the segmented network compared to the counterpart compact filter in what concerns the temporal evolution of the spectral components. Yet, as it can be observed from Figure 7.3a, b, c, the fragmented network has a different dynamic versus the compact one (Figure 7.1a, 2a, 2c). In order to catch the differences between both implementation, before the filter reaches nonlinearities, it has to be setup with a large selectivity (close to the instability limit) to slow down the unstable behavior. Another remark regarding these two types of implementation refers to the processing speed per frame: since a larger network has a higher inertia of instability than that of a smaller one, the fragmented filter performs the same results like the compact one, but in a shorter time. Anyway, each sub-network could be analyzed by the known means of the compact architecture. All subnetwork features are kept only if all coefficients of each active pixel remain unchanged. The interconnectivity map between network cells can be set from the beginning and kept during the filtering process, continuously updated or only at a given moments. The advantage of this implementation becomes significant when different region from an image has to be filtered in different ways. This is possible only because each sub-network exhibit an independent dynamic compared to the others. Also, the selective decoupling technique is useful for a nonlinear processing by disconnecting the saturated region from the rest of the network, thus the nonlinear part of the filter does not affect the linear one. The pixel structure is slightly changed comparing with the basic scheme from Fig. 2. In addition, it’s worth mentioning the presence of the circuit that calculates the module of the difference between voltages of two consecutive pixels. This value will be compared with a given threshold and stored, in order to control the gain of the voltage controlled current sources. In the following several results obtained using the fragmented low-pass filter will be compared with those of the compact one, for different noise levels.
Input image no. 1; 100mV ripple
Low-pass filtering (fragmented) after 60us, with Vthreshold = 1.65V
Low-pass filtering (compact) after 336us
Input image no. 2; 400mV ripple
Low-pass filtering(fragmented) after 53us, with Vthreshold = 1.65V
Low-pass filtering (compact)after 288us
Fig. 6. Comparison between compact low-pass filter and selective network.
Input image no.1
Input image no. 2
1a
1b
2a
2b
3a
3b
3c
4a
4b
4c
Fig. 7. Image segmentation using different techniques compared with the results obtained with the non-linear network implemented with pulse-modulation techniques; 1a, b – low-pass filtering after 71, 96 respectively; 2a, b – low-pass filtering with 1.65 threshold after 160us; 3a – low-pass filter using the segmented filter, after 300us and 240us (3b) respectively; 3c – low-pass filter using the segmented filter, with the reloading of the network interconnectivity configuration after 140us; 4a, b, c – segmented versions obtained with a nonlinear resistive network implemented using pulsemodulation technique.
463
In the above figure, the 1D filter was loaded with initial conditions having maximum amplitude around 1.65V, considering a 3.3V supply voltage. So, at least for these simple 1D examples, both networks exhibit similar behavior. Next, several results for image segmentation, obtained using a 2D network, implemented at transistor level in the linear or log-domain will be presented. For this purpose, a comparison between the fragmented filter, log-domain filter frozen after reaches saturation and the nonlinear resistive grid implemented by pulse-modulation technique [4] will be made. Figures 8 show several relevant snapshots taken from the log-domain filter at different times, after reaching saturation.
a)
5. CONCLUSIONS In this paper the feasibility of basic operations like edge detection, smoothing, image segmentation using an analog parallel architecture, continuous in time and discrete in space has been briefly analyzed. Besides, certain advantages of the fragmented network in applications like filtering on different parts of the same image were shown. The boundary conditions for each sub-network are zero-flux. Since any cell that reaches saturation affects the linear behavior of the rest of the network, the decoupling technique of some parts of an imager can be useful in nonlinear processing if the saturated part is cutoff from the filter, and the linear parts have independent evolution. The time constant is another significant difference between compact filter and the fragmented one. It is easy to see that the fragmented network is faster than the other one with the same nodal capacitance so that the technique is useful for increasing the processing speed. Acknowledgments–The authors gratefully acknowledge finabcial support ftom brain European Doctoral Project and the higher Education Scientific Research National Council under Grant PN2-ID_310 Respectively.
b)
Fig. 8. Image segmentation using log-domain filter after reaching saturation; a), b)– snapshot on the log-domain filter frozen after 78, 95 us.
It is apparent that the nonlinear log-domain filter can be used for image segmentation. Fig. 9 confirms the usefulness of linear/nonlinear low-pass filtering for image segmentation as seen from the comparison of the simulations performed with the compact linear filter, nonlinear log-domain filter and nonlinear restive grid implemented using another circuit solution [3].
References [1]
[2]
[3]
[4]
Input image
1a
1b
[5]
[6] Noisy input image
2a
2b
Fig. 9. Image segmentation using linear and log-domain filter after reaching saturation, comparing with the nonlinear resistive grid [3], 1a – segmentation obtained with the nonlinear resistive grid [3], 1b – low-pass filtering with the linear filter after 192us, 2a, 2b – segmentation using the logdomain filter frozen after 98, 108, 116us respectively.
[7]
464
Jesus Ruiz Amaya, “Metodologia y sintesis de convertidores A/D CMOS tipo pipeline para telecomunicaciones”, doctoral dissertation, National Institute of Microelectronics from Seville, 2010. Paul C. Yu, Steven J. Decker, Hae-Seung Lee, Charles G. Sodini, John L. Wyatt - “CMOS resistive fuse for image smoothing and segmentation”, IEEE Journal of Solid-State Circuits, 27(4), April 1992. Storace, Giacomo Pruzzo, Mauro Parodi, “CMOS implementation of a cellular nonlinear network for image segmentation”, CNNA 2004. Hiroshi Ando, Takashi Morie, Makoto Miyake, Makoto Nagata, Atsushi Iwata – “Image segmentation/extraction using nonlinear cellular networks and their VLSI implementation using pulsemodulation techniques”, IEICE Trans. Fundamentals, E85-A(2), February 2002. Johannes Schemmel, Karlheinz Meier and Markus Loose – “A scalable switched capacitor realization of the resistive fuse network”, Analog integrated circuits and signal processing, 32, pp. 135–148, 2002. L. Goraş, Ion Vornicu, “Log-domain CMOS implementation of a class of analog parallel architectures”, 2, CAS, pp. 499–502, Sinaia, Romania, 2009. L. Goras, I. Alecsandrescu, I. Vornicu, “Spatial filtering using linear analog parallel architesctures”, 2, International Symposiunm on Signals, Circuits and Systems ISSCS, pp. 409–412, Iasi, Romania, 2009.
