Pattern Recognition Letters 30 (2009) 1144–1150
Contents lists available at ScienceDirect
Pattern Recognition Letters journal homepage: www.elsevier.com/locate/patrec
An adaptive blink detector to initialize and update a view-based remote eye gaze tracking system in a natural scenario Diego Torricelli a,b, Michela Goffredo a,c, Silvia Conforto a, Maurizio Schmid a,d,* a
Department of Applied Electronics, University Roma TRE, Roma, Italy Instituto de Automática Industrial, Consejo Superior de Investigaciones Cientificas, Arganda del Rey, Madrid, España c School of Electronics and Computer Science, University of Southampton, Southampton, United Kingdom d Department of Physical Medicine and Rehabilitation, Harvard Medical School, Cambridge, MA, USA b
a r t i c l e
i n f o
Article history: Available online 2 June 2009 Keywords: Blink detection Eye gaze tracking Human–computer interface
a b s t r a c t A method for blink detection from video sequences gathered with a commercial camera is presented. This is used as a view-based remote eye gaze tracker (REGT) component performing two relevant functions, i.e. initialization and automatic updating in case of tracking failures. The method is based on frame differencing and eyes anthropometric properties. It has been tested on a publicly available database and results have been compared with algorithms found in literature. The obtained average true prediction rate is higher than 95%. The robustness of the automatic tracking failure detection has been tested on a set of experimental trials in different conditions, and yielded detection rates around 98%. The computational cost of the processing allows the blink detection algorithm to work in real time at 30 fps. The obtained results are in favour of combining blink detection with gaze mapping for the development of a robust view-based remote eye-gaze tracker to be introduced in different HCI contexts, specifically in the assistive technology framework. Ó 2009 Elsevier B.V. All rights reserved.
1. Introduction Remote eye gaze tracking (REGT) is the core component of a remote system able to non-invasively predict gaze of people (Morimoto and Mimica, 2005). By using one or more video capturing devices, placed at a convenient distance from the user, the aim of a REGT system is to reveal where the subject is looking at during the interaction with a system (e.g. a computer). The advantages of remote eye gaze trackers over head-mounted gaze trackers (Hong et al., 2005) are obvious in terms of setup simplicity and intrusiveness of the system. Moreover, REGT systems allow one to extend the application fields to a wide range of scenarios: assistive technologies based on multi-modal human–computer interfaces (HCI) (Adjouadi et al., 2004); analysis of driver behaviour and attention, behavioural studies for deception detection; diagnosis of neurophysiologic disorders; ethologic research activities (Duchowski, 2002). Optical eye gaze trackers found in literature can be grouped into two distinct approaches: IR-based and view-based REGT systems (Morimoto and Mimica, 2005). The first one uses external infrared
* Corresponding author. Address: Dipartimento di Elettronica Applicata, Università degli Studi Roma Tre, Via della Vasca Navale 84, I-00146 Roma, Italy. Tel.: +39 06 5733 7309; fax: +39 06 5733 7026. E-mail address:
[email protected] (M. Schmid). 0167-8655/$ - see front matter Ó 2009 Elsevier B.V. All rights reserved. doi:10.1016/j.patrec.2009.05.014
sources, whose reflections on the eyes allow to estimate the subject’s gaze (Hutchinson et al., 1989). The second one is view-based because it uses commercially available video cameras working in visible light spectrum. Computer vision techniques extract the gaze information from the captured images (Baluja and Pomerleau, 1994; Tan et al., 2002; Xu et al., 1998; Zhu and Yang, 2002). If IRbased systems present better performance in extreme conditions (e.g. low light, twilight or nocturnal recordings), the advantage of having no external source of light or ad hoc devices specifically designed to gather information, makes view-based REGT systems desirable in multimedia HCI applications and in scenarios where the gaze analysis needs to be either remote or covert. In most view-based REGT systems, the process for gaze estimation can be split into a number of successive phases: initialization/ calibration, eye features tracking and gaze mapping (Fig. 1). The initialization phase allows to robustly locate the eyes position in the image reference system and to extract specific templates. Once the initialization phase is concluded, the tracking phase locates the position of the features in the video sequence. Ultimately the gaze mapping block determines the relationship between the extracted features and the gaze direction. In order to succeed in gaze mapping, the tracking process usually analyses the head and eyes movements separately. As a matter of fact, gaze estimation depends on both the head pose in the world reference system and the eyes pose in the head reference system. Due to
1145
D. Torricelli et al. / Pattern Recognition Letters 30 (2009) 1144–1150
Fig. 1. Different phases for remote gaze estimation systems.
the complexity of the problem, most of the REGT systems are based on rather strict assumptions for the head movement (Zhu and Ji, 2007), e.g. asking to the user to maintain the head as still as possible, or repeating the initialization phase. A critical point of both initialization and tracking phases regards the ability to detect the eye position in the image reference system. Even if some REGT systems accomplish this objective either by using shape tracking (Khosravi and Safabakhsh, 2005; Lam and Yan, 1996), or by looking for contours in intensity or colour images (Schwerdt and Crowley, 2000), or even by exploiting image gradients (Kothari and Mitchell, 1996), projections (Zhou and Geng, 2004), and templates (D’Orazio et al., 2004; Kawaguchi et al., 2000), the use of blink detection as an estimator of eye localization is prevailing in computer vision. By detecting and locating the eyelid movements, blink-based methods are effectively able to initiate and update the eye tracking process. The following section will provide details on this latter approach 2. Literature survey on blink detection Black et al. (1998) proposed an optical flow-based method and reported a percentage of success in blink recognition of 65%. By combining the optical flow with frame differencing, Bhaskar et al., 2003 distinguished between blinking movements and other eye movements reporting an accuracy of 97%. Gorodnichy (2003) compensates for head motion by introducing the so-called second-order change detection: the image difference over three consecutive frames is combined to filter out the linear changes of intensity due to the head movement. The system works in real time with 160 120 pixel images but no accuracy is reported. The technique called ‘‘Blink Link” has been introduced by Grauman et al. (2001), where the Mahalanobis distance of the frame difference is computed to locate the regions that more likely represent the eye locations: after extracting an ‘‘open-eye” template, blinks are then detected via normalized cross-correlation functions. This technique has been used to determine the voluntary/involuntary nature of blink and a rate success of 95.6% has been achieved. The main drawback of this method derives from the necessity of an off-line training phase for different user-tocamera distances. A similar approach is presented in the work by Chau and Betke (2005), where the open-eye template window is shaped through a filtering process based on six geometrical parameters. If the user-to-camera distance changes significantly or rapid head movements occur, the system is automatically reinitialized. The system works with low-cost USB cameras in real time at 30 fps. Experimental tests on eight subjects yielded an overall detection accuracy of 95.3%. Morris et al. (2002) propose a blink-based method for automatic initialization of eye tracking applications. Successful eyeblink detection rate is 95% with a good robustness to environment changes with images of 320 240 pixels. Nevertheless, the method presents a limitation on anthropometric constraints hypothesis, i.e. the existence of a straight line connecting the four corners of the eyes. Moreover, the presence of fixed parameters would restrict the use to conditions where the user-to-camera distance is almost unchanged. To overcome these limitations, the authors of this manuscript (Torricelli et al., 2008) introduced an EGT system, where a blink estimation based on a basic frame differencing is
used to detect eye features in order to build up an automatic initialization process for eye gaze tracking. The obtained results confirmed the good performance of blink-based approaches for view-based EGT system initialization. A relevant number of methods found in literature proposed blink detection a robust approach to initialize and update a gaze tracking system. Furthermore, blinking allows the user to control and interact with the machine. To this extent, this letter presents a novel blink-based method to both initialise and update an REGT system based on visible light spectrum. The manuscript is organized as follows: the next section details the proposed blink detection technique and its function in the view-based REGT system; Section 4 describes the ensemble of experimental tests that have been devised to test the accuracy and robustness of the proposed system. The fifth section will detail and discuss the obtained experimental results and conclusions will be drawn.
3. The proposed system This section details the proposed blink detection algorithm and describes its functions in the view-based REGT system. 3.1. The blink detection algorithm The proposed method allows to detect blinking without any constraints on head pose or subject position with respect to the camera. The approach is based on two assumptions: the user generally does not present large head movement during blinking; natural blinking involves movement of both eyes. Let I(x,y,ti) and I(x,y,ti+1) be two consecutive gray-level frames (of size R C pixels) from a video sequence gathered at time T = [t1, t2, . . ., ti, ti+1, . . ., tF]. The binary image representing the inter-frame changes is defined by
Eðx; y; t i Þ ¼
1; if Dðx; y; t i Þ P bT
ð1Þ
0; otherwise
where
Dðx; y; t i Þ ¼ jIðx; y; tiþ1 Þ Iðx; y; t i Þj
ð2Þ
and bT is a threshold which has been empirically determined as explained in the experimental section. E(x, y, ti) is denoised by applying an M M filter, yielding the image F(x, y, ti) as in the following
2 8 ðM1Þ xnz þ > > < 1; if 1 4 P 2 M2 Fðxnz ; ynz ; t i Þ ¼ ðM1Þ i¼xnz 2 > > : 0; otherwise (
ynz þ
ðM1Þ
P2
ðM1Þ j¼ynz 2
3
Eði; j; t i Þ5 P 12
ð3Þ
< xnz < R ðM1Þ 2 , < ynz < C ðM1Þ 2 The non-zero pixels are clustered with a coarse to fine procedure. At first, the 1D signal to every non-zero pixel ðxnz ; ynz Þ j
hðx; ti Þ ¼
R X
Fðx; j; t i Þ
j¼1
is considered.
ðM1Þ 2 ðM1Þ 2
ð4Þ
1146
D. Torricelli et al. / Pattern Recognition Letters 30 (2009) 1144–1150
At first, the vector z = (z1,z2, . . . ,zK) is composed of all the indexes corresponding to non-zero values of h(x, ti). For each element of the vector z, if
jzk zk1 j 6
max½sgnðhðx; t i ÞÞ x min½sgnðhðx; ti ÞÞ x ; Tc
ð5Þ
then zk 2 k (the sample is accumulated in the class k), else k = k + 1 (a new class is generated) with zk 2 k. The threshold Tc is defined empirically by the same experimental procedure used for the definition of the bT parameter. Successively, the same procedure is applied to the series of 1D signals
vðkÞ ðy; t i Þ ¼
C X
LðkÞ ði; y; ti Þ;
k ¼ ½1; 2; ; k; ; C
ð6Þ
i¼1
where L(k)(x, y, ti) is the RxC image containing only the pixels belonging to the cluster k. The result of the whole clustering process is a set of k0 = [1, 2, . . ., k0 , . . ., C0 ] regions of interest (ROI) whose pixels have the same label. Fig. 2 shows the whole clustering process. A classification procedure is then applied in order to detect if the extracted ROI belong to two blinking eyes. The classification is performed by considering that: a. The ROI’s shape respects the eye’s proportion according to anthropometric studies (Farkas and Munro, 1987)
16
wk0 63 hk0
ð7Þ
where wk0 and hk0 are the width and height of the bounding box of the ROI. b. Natural blinking involves both eyes, and the size of the two clusters should not be too different. Thus, ROI corresponding to the eyes satisfy the following:
9 k 2 1; . . . ; Cj0:3 6
wk0 hk0 63 wk0 hk0
ð8Þ
c. The inter-pupillary distance d and the eyes’ mean size follow the
16
d qffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi qffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi 6 7 2 2 2 mean wk0 þ hk0 ; w2k0 þ hk0
ð9Þ
as reported by studies on face anatomy (Farkas and Munro, 1987). d. The maximum head rotation in a plane parallel to the image plane is 45°. Therefore, the angle between the line connecting the centroids of a couple of clusters and the horizontal line is lower than 45°. If two ROI satisfy the classification procedure described by (a–d), a closure-aperture eyelid movements is detected in frames I(x, y, ti) and I(x, y, ti+1). Therefore, the application of the proposed method to the F frames of the video sequence allows to estimate if and how many times the subject blinks in front of the camera. 3.2. Blink detection in REGT The blink detection technique described in the previous section completes and refines the REGT system proposed by Torricelli et al. (2008) by tasking different phases: (1) Initialization The initialization phase detects a sequence of three consecutive blinks by using the algorithm described above. In this way this phase reveals the user’s will to start a new session with the REGT system. Moreover it allows to locate the eyes so that the calibration phase will be able to define the set of features, relative to the specific user, to be tracked over time. (2) System update In a view-based REGT system, the eyes features tracking algorithm can fail in different situations, such as wide head rotation, eyes occlusion, light changes, scale changes. In this circumstances, the blink detection algorithm allows to relocate the regions of interest, re-starting the eye tracking system. The ‘‘system update” is composed of two consecutive steps: failure detection and re-initialization. The system
Fig. 2. Overview of the labeling process based on frame differencing: (a) frame acquisition; (b) frame differencing ( the contrast of the image has been artificially enhanced for the sake of clarity); (c) thresholding and binarization; (d) denoising and labeling on x direction; (e) labeling on y direction; (f) clustering result.
1147
D. Torricelli et al. / Pattern Recognition Letters 30 (2009) 1144–1150
should quickly detect any potential tracking failures and inform the user about the malfunctioning. The tracking procedure is interrupted whether one of the following
jDpl ðt i Þ Dpl ðt i Þj > W; jDpl ðt i Þj > 2 W; jDpr ðt i Þj > 2 W
jpl ðt i1 Þ pr ðti1 Þj
r
;
Subject #
ð10Þ
occurs, where subscript l and r correspond to the left and right eye, respectively,pl(ti) is the iris centre position in the image reference system at time ti, Dp(ti) = p(ti) p(ti1), andW is defined as
W¼
Table 1 Excerpt of images from ZJU eyeblink database (http://www.cs.zju.edu.cn/~gpan).
20
4 Videos per subject Clip #
View
Glasses
1
Frontal
None
1
Frontal
Thin rim
1
Frontal
Black frame
1
Upward
None
Example (subject 7)
Total blinks #
255
ð11Þ
where r represents the human ratio between the inter pupil distance and diameter of the irises, set to 6 according to human anthropometrics (Farkas and Munro, 1987). If (10) is verified, the re-initialization process occurs and the system asks the user to perform a multiple blink as soon as possible. The application of the proposed blink detection method allows to re-locate the eyes so that the system will be able to re-define the set of features to be tracked. (3) User system interaction The user system interaction phase detects a specific sequence of consecutive blinks by using the proposed blink detection algorithm. In this way, the user is allowed to control and interact with system applications. In the following, focus will be on the application of the proposed blink detection method for the ‘‘initialization” and ‘‘system update” phases of REGT. The following section will therefore describe the experimental tests that have been conducted in order to evaluate the performance of the proposed approach for ‘‘initialization” and ‘‘system update”. 4. Testing and analysis The system has been tested in two different scenarios: a publicly available database (from now on, ZJU database) of different people spontaneously blinking in front of a webcam (http:// www.cs.zju.edu.cn/~gpan) has been used to evaluate the blink detection performance; a large number of experimental sessions have been conducted to test the real time functioning of the REGT system in a set of different situations, such as wide range and kinds of head movements. 4.1. Testing on initialization The performance of the proposed blink detection module has been tested extracted by using the eyeblink database cited above, where different user-to-camera positions and glass carrying conditions are present. The ZJU Eyeblink Database, introduced by Pan et al. (2007), is composed of 80 indoor video sequences gathered with a Logitech Pro5000 webcam at 30 fps and 320 240 pixels, without any light control. 20 subjects blink spontaneously at different positions with respect to the camera and by wearing different kinds of glasses. Table 1 summarizes information associated with the ZJU database. The proposed method for blink detection was applied to the video sequences and the estimated number of blinks was extracted. Fig. 3 shows samples from the database when blinking is detected. A two-step procedure was followed to test the algorithm. The first step aimed at choosing the best set of threshold parameters TC and bT defined in Section 3.1, to determine the optimal parameter configuration. The second step regards the testing of the algorithm implemented with the optimal parameter set. For each couple of TC and bT values, in the range of [8 16] and [8 32], respectively, false prediction rate and sensitivity have
been calculated, and the Pollack and Norman parameter for each point of the ROC curve has been estimated area under the ROC curve has been estimated (Pollack and Norman, 1964). These measures allows to quantitatively estimate the influence of the thresholds values in the algorithm. The optimal solution has been found as the position of the maximum of the Pollack and Norman matrix (Fig. 4b) that corresponds to the point closer to (0; 1) in the ROC curves plot (Fig. 4a). These values are TC = 12 and bT = 12. Therefore, fixing these values of TC and bT, the algorithm has been tested again and the detection rate, defined as the ratio between the number of correctly detected blink activities and the total number of blink activities, has been calculated. Results have been compared with the ones obtained by Pan et al. (2007) in Table 2 where detection rates obtained with the application of Cas-Adaboost (Viola and Jones, 2001) and HMM (Rabiner, 1989) on the same data are also shown (as reported in Pan et al., 2007). Figures show that the proposed method is comparable with the other approaches, showing a mean detection rate of 95.7%. In particular, the system appears to be more robust to the reflection of the glass surface: it shows better results as compared to the other methods, maintaining the eye detection rate higher than 92%. Moreover our algorithm is not particularly affected by the presence of a dynamic background. In regard to the specificity, it is here outlined that the false prediction rate resulted on average around 7%, which is acceptable for HCI applications. Furthermore, Table 2 reports the computational cost of our method obtained with 2.4 GHz CPU and 2 GB RAM. The average
1148
D. Torricelli et al. / Pattern Recognition Letters 30 (2009) 1144–1150
Fig. 3. Samples of the results by applying the proposed method on the ZJU eyeblink database: (a) no glasses; (b) thin rim glasses; (c) black frame glasses; (d) upward view.
processing time per frame is three times lower than the method presented by Pan et al., allowing the system to run real time at the maximum possible frame rate with respect of the ZJU Eyeblink Data, i.e. 30 fps. The low computational cost underlines the feasibility of the system to run as an ‘‘always present agent” within an eye tracking system. 4.2. Testing on system update The system was also tested in a realistic scenario, i.e. the user interacting in real time with the eye tracking interface. The subject
was requested to move in front of the camera in five different conditions: three head movements, eye occlusions and light variations, while the system was tracking the eyes with according to Torricelli et al. (2008). In case the eye tracked failed, the automatic failure detection system automatically asked the subject to stop and reinitialize the system through three consecutive voluntary blinks. The experiments aimed at testing two parameters, i.e. the automatic detection of tracking failures (TF) and the automatic re-initialization through blinking. Four different experimental sessions were performed. Each session was composed of three trials, corresponding to three different distances of the user from the camera. For each trial the user was asked to perform 10 complete cycles of movements, thus resulting in a total of 120 cycles for each movement condition. The exception occurred for the towards/away movement, since it covered the three distances in the same trial, so that just one set was performed. The dataset is summarized in Table 3. The experimental setup was composed of a Logitech Pro9000 webcam at 30 fps and 320 240 pixels and an Acer Extensa 5200 laptop with a 1.86 GHz Cpu and 1 GB RAM. The overall performance of the system was evaluated in terms of tracking failure detection rate and re-initialization rate. The failure detection rate is defined as the ratio between the number of correctly detected tracking failures and the overall number of
Table 2 Comparative results on ZJU database. Data
CasAdaboost
HMM
Pan et al. (2007)
Our method
80.4% 60.6% 55.2%
98.2% 93.9% 91.0%
95.1% 98.4% 92.3%
59.1% – – 63.4%
95.5% – – 95.7%
97.1% 95.7% 95.7% 95.7%
–
–
25 msa
8 msb
–
–
20a
30b
Two-eye detection rate (sensitivity) Frontal w/o glasses 98.2% Frontal w/ thin rim glasses 80.0% Frontal w/ black frame 71.9% glasses Upward w/o glasses 62.3% Static background – Dynamic background – Average 78.1%
Fig. 4. Analysis of the influence of the thresholds values in the algorithm, in terms of specificity and false prediction rate. (a) ROC curves. (b) Pollack and Norman matrix: each element of the matrix represents the Pollack and Norman parameter for each point of the ROC curve.
Computational cost Average processing time per frame fps (real time) a b
Cpu 2.0 GHz, 1GB RAM. Cpu 2.4 GHz, 2GB RAM.
D. Torricelli et al. / Pattern Recognition Letters 30 (2009) 1144–1150 Table 3 Automatic updating trials. 4 trials
Example
Events #
Clip #
Condition
3
In-plane head rotation (±45°)
120
3
Vertical head rotation (±25°)
120
3
Eye occlusion
120
3
Light changes
120
1
tw/aw head movement
40
tracking failures. The re-initialization rate is defined as the percentage of successful re-initialization following a detected failure. Under natural head movements the tracking system rarely fails, so that the conditions of use had to be brought to extreme situations in order to have a consistent number of failures (which summed at 144 in 560 movements). Results in terms of rate of undetected tracking failure, show that the method detects tracking failures with a high rate under all conditions (Table 4). While auto-
Table 4 Undetected tracking failure and re-initialization rate obtained with the proposed REGT system in different scenarios. Type of movement
Distance from the camera 20 cm
35 cm
50 cm
Rate of undetected tracking failures In-plane head rotation Vertical head rotation Eye occlusion Light changes tw/aw head movement
– – – – –
– – 7.5% – –
30% – 5% – –
Average
–
1.5%
7%
Re-initialization rate = 100% in all the conditions.
1149
matic re-initialization has a 100% correct detection rate, the failure detection module presents the lowest performance in case of inplane rotations and high user-to-camera distances.
5. Discussion and conclusions The tracking system proposed in this paper is a feasible HCI solution via blink and eye movements. A novel blink-based method to both initialise and update the system has been introduced. Experimental tests on a large database demonstrate the reliability of the proposed approach for the REGT initialization phase. Moreover, the blink detector can be successfully used in the automatic failure detection module providing robust and consistent results with respect to user head movements. The overall results show how the approach detects blink not only from the front-view, but also from an upward view and in the case of users wearing glasses. The two-eyes detection rates, with a mean value of 95.7%, are higher than the ones found in the literature and are particularly encouraging for future applications. With specific reference to Morris et al. (2002), no direct comparison can be made since Morris et al. did not provide any database of videos and tested their method on a small number of subjects. Apart from that, the system here proposed gives comparable results in terms of true prediction rate, with the difference that Morris et al. need specific constraints on the relative pose between the user and the camera. Moreover, in contrast to other blink detection routines (see e.g. Pan et al., 2007), the proposed approach does not need any face detection, thus substantially reducing the computation time. The authors are convinced that the sub-optimal performance in terms of false prediction rate (especially in dynamic background conditions) can be caused by the absence of hypotheses on the temporal features of the blink motion (which is instead needed when, e.g. Hidden Markov Model is used). As a matter of fact, since no specific hardware is needed, and the algorithms perform well in different light conditions and support real time implementation within the range of typical HCI frame rates, the implementation of this kind of system in a clinical context is within reach. Even if in-plane rotation may represent a challenge, it is envisioned that the implementation of such system as a reliable human machine interface in an assistive technology context can effectively represent a valid alternative among vision techniques for intelligent human–computer interaction (HCI). Acknowledgement The work was partially supported by the Italian Ministry of Education, University and Research (MIUR). References Adjouadi, M., Sesin, A., Ayala, M., Cabrerizo, M., 2004. Remote eye gaze tracking system as a computer interface for persons with severe motor disability. In: Proc. 9th Internat. Conf. Computers Helping People with Special Needs 3118, pp. 761–769. Baluja, S., Pomerleau, D., 1994. Non-intrusive gaze tracking using artificial neural networks. In: Cowan, J.D., Tesauro, G., Alspector, J. (Eds.), Advances in Neural Information Processing Systems (NIPS), vol. 6. Morgan Kaufmann Publishers, San Francisco, CA, pp. 753–760. Bhaskar, T.N., Keat, F.T., Ranganath, S., Venkatesh, Y.V., 2003. Blink detection and eye tracking for eye localization. In: Proc. Conf. Convergent Technologies for Asia-Pacific Region, pp. 821–824. Black, M.J., Fleet, D.J., Yacoob, Y., 1998. A framework for modeling appearance change in image sequences. In: Proc. Sixth Internat. Conf. on Computer Vision (ICCV’98), pp. 660–667. Chau, M., Betke, M., 2005. Real Time Eye Tracking and Blink Detection with USB Cameras. Tech. Rep. 2005-12 Boston University Computer Science. D’Orazio, T., Leo, M., Cicirelli, G., Distante, A., 2004. An algorithm for real time eye detection in face images. In: Proc. 17th Internat. Conf. on Pattern Recognition (ICPR’04), vol. 3, pp. 278–281.
1150
D. Torricelli et al. / Pattern Recognition Letters 30 (2009) 1144–1150
Duchowski, A.T., 2002. A breadth-first survey of eye tracking applications. Behav. Res. Methods Instrum. Comput. 34, 455–470. Farkas, L.G., Munro, I.R., 1987. Anthropometric Facial Proportions in Medicine. Charles C. Thomas, Springfield, USA. Gorodnichy, D.O., 2003. Second order change detection, and its application to blinkcontrolled perceptual interfaces. In: Proc. IASTED Conf. on Visualization, Imaging and Image Processing, pp. 140–145. Grauman, K., Betke, M., Gips, J., Bradski, G., 2001. Communication via eye blinks – detection and duration analysis in real time. In: Proc. IEEE Comp. Soc. Conf. on Computer Vision and Pattern Recognition, pp. 1010–1017. Hong, L.E., Avila, M.T., Wonodi, I., McMahon, R.P., Thaker, G.K., 2005. Reliability of a portable head-mounted eye tracking instrument for schizophrenia research. Behav. Res. Methods 37, 133–138. Hutchinson, T., White, K.J., Reichert, K., Frey, L., 1989. Human–computer interaction using eye-gaze input. IEEE Trans. Syst. Man Cybern. 19, 1527–1533. Kawaguchi, T., Hidaka, D., Rizon, M., 2000. Detection of eyes from human faces by hough transform and separability filter. In: Proc. IEEE Internat Conf. Imag. Proc., vol. 1, pp. 49–52. Khosravi, M.H., Safabakhsh, R., 2005. Human eye inner boundary detection using a modified time adaptive self-organizing map. In: Proc. IEEE Internat. Conf. Imag. Proc., vol. 2, pp. 802–805. Kothari, R., Mitchell, J.L., 1996. Detection of eye locations in unconstrained visual images. In: Proc. IEEE Internat. Conf. Imag. Proc. 3, pp. 519–522. Lam, K.M., Yan, H., 1996. An improved method for locating and extracting the eye in human face images. In: Proc. 13th Internat. Conf. on Pattern Recognition, vol. 3, pp. 411–412. Morimoto, C.H., Mimica, M.R.M., 2005. Eye gaze tracking techniques for interactive applications. Computer Vis. Image Underst. 98, 4–24.
Morris, T., Blenkhorn, P., Zaidi, F., 2002. Blink detection for real-time eye tracking. J. Networks Comput. Appl. 25, 129–143. Pan, G., Sun, L., Wu, Z., Lao, S., 2007. Eyeblink-based anti-spoofing in face recognition from a generic webcamera. In: 11th IEEE Internat. Conf. on Computer Vision. Pollack, I., Norman, D.A., 1964. A non parametric analysis of recognition experiments. Psychon. Sci. 1, 125–126. Rabiner, L.R., 1989. A tutorial on hidden markov models and selected applications in speech recognition. Proc. IEEE 77 (2), 257–286. Schwerdt, K., Crowley, J.L., 2000. Robust face tracking using color. In: Proc. 4th IEEE Internat. Conf. Automatic Face and Gesture Recognition, pp. 90–95. Tan, K.H., Kriegman, D.J., Ahuja, N., 2002. Appearance-based eye gaze estimation. In: Proc. Sixth IEEE Workshop on Applications of Computer Vision (WACV), pp. 191–195. Torricelli, D., Conforto, S., Schmid, M., D’Alessio, T., 2008. A neural-based remote eye gaze tracker under natural head motion. Comput. Meth. Prog. Bio. 92, 66–78. Viola, P., Jones, M.J., 2001. Rapid object detection using a boosted, cascade of simple features. In: Proc. IEEE Comp. Soc. Conf. on Computer Vision and Pattern Recognition,.vol. 1, pp. 511–518. Xu, L.Q., Machin, D., Sheppard, P., 1998. A novel approach to real-time non-intrusive gaze finding. In: Proc. Ninth British Comp. Vis. Conf. (BMVC), pp. 428–437. Zhou, Z.H., Geng, X., 2004. Projection functions for eye detection. Pattern Recognition 37 (5), 1049–1056. Zhu, J., Yang, J., 2002. Subpixel eye gaze tracking. In: Proc. 5th IEEE Internat. Conf. Automatic Face and Gesture Recognition, pp. 124–129. Zhu, Z., Ji, Q., 2007. Novel eye gaze tracking techniques under natural head movement. IEEE Trans. Biomed. Eng. 99, 1–12.
