Face-specific responses from the human inferior occipito-temporal cortex

Pergamon

PII:

Neuroscience Vol. 77, No. 1, pp. 49–55, 1997 Copyright ? 1997 IBRO. Published by Elsevier Science Ltd Printed in Great Britain 0306–4522/97 $17.00+0.00 S0306-4522(96)00419-8

FACE-SPECIFIC RESPONSES FROM THE HUMAN INFERIOR OCCIPITO-TEMPORAL CORTEX M. SAMS,*† J. K. HIETANEN,‡§ R. HARI,* R. J. ILMONIEMI*¶ and O. V. LOUNASMAA* *Low Temperature Laboratory, Helsinki University of Technology, 02150 Espoo, Finland ‡Institute of Biomedicine, Department of Physiology, P.O. Box 9, 00014 University of Helsinki, Helsinki, Finland Abstract––Whole-head neuromagnetic responses were recorded from seven subjects to pictures of faces and to various control stimuli. Four subjects displayed signals specific to faces. The combination of functional information from magnetoencephalography and anatomical data from magnetic resonance images suggests that the face-specific activity was generated in the inferior occipitotemporal cortex. All four subjects showed the face-specific response in the right hemisphere, one of them also in the left. Our results, together with recent positron emission tomography and lesion studies, suggest a right-hemisphere preponderance of face processing in the inferior occipitotemporal cortex. Copyright ? 1997 IBRO. Published by Elsevier Science Ltd. Key words: magnetoencephalography, face, visual cortex, evoked responses, perception, human.

Processing of facial images has been studied extensively during the past few years. Neuropsychological models have emphasized the unique nature and sequence of different stages in this process (e.g., Ref. 9), while neurophysiological recordings have provided evidence for the existence of localized brain areas related to facial perception and recognition.17,21,39 This multidisciplinary approach is hoped to be successful in constructing anatomo-functional models of cerebral face processing. Many different experimental methods have been employed: measurements of performance and reaction time,8,13,42,43 tachistoscopic presentation of face stimuli to divided visual fields,20,25,36 investigations of symptoms in patients having (prosopagnostic) difficulties in perceiving and recognizing faces after local brain injuries,11,12,28 measurements of visual evoked potentials6,21,22,23 and magnetic fields27 to face stimuli, and measurements of regional cerebral blood flow with positron emission tomography (PET) during facial image processing.39 In addition, single-unit recordings on monkeys have revealed cell populations in the temporal cortex responding selectively to faces.14,19,32,33

Magnetoencephalography (MEG), a method for studying the neural basis of cognitive brain functions non-invasively, is based on recordings of the weak magnetic fields produced by electric currents flowing in active neurons (for a comprehensive review of the method, see Ref. 16). MEG differs from electroencephalography (EEG) in its selectivity to superficial, tangentially oriented neural currents. This means that MEG ‘‘sees’’ mainly the fissural cortex and, therefore, less neuronal activity than EEG. MEG offers a temporal resolution in the millisecond range which makes it useful in studying the timecourse of stimulus-evoked brain responses. In addition, because the intervening cerebral and extracerebral tissues are practically transparent to magnetic fields, whereas electric recordings are often smeared by inhomogeneities in them, it is often easier to calculate the locus of the activated brain area on the basis of MEG rather than EEG recordings.16 In a previous study from this laboratory,27 MEG was applied to identify face-responsive brain areas. Three sites outside the occipital visual cortex were observed to become active, two of them exclusively by viewing faces: an area near the occipitotemporal junction and another in the mid-temporal lobe. In this early study, MEG signals were recorded with a 24-channel magnetometer that covered an area 12.5 cm in diameter. We decided to extend and confirm the previous results with our 122-channel neuromagnetometer. This device covers the whole scalp and thus offers the possibility of studying interand intrahemispheric differences in the magnetic fields during a single recording session.

†To whom correspondence should be addressed. §Present address: Department of Psychology, University of Tampere, Tampere, Finland. ¶Present address: BioMag Laboratory, Medical Engineering Centre, Helsinki University Central Hospital, 00290 Helsinki, Finland. Abbreviations: ECD, equivalent current dipole; EEG, electroencephalography; MEG, magnetoencephalography; MRI, magnetic resonance imaging; PET, positron emission tomography. 49

50

M. Sams et al.

Fig. 1. Examples of pictures used in the experiment. From left to right: a face, the same face pointillized, an everyday object and a sphere.

Fig. 2. Whole-head 122-channel neuromagnetic responses to faces (continuous lines) and pointillized faces from subject S3. The head is viewed form the top (the nose is pointing up), and in each response pair the upper trace illustrates the field derivative along the latitude and the lower trace along the longitude. The face-specific response is best seen in the channel pair over the right occipital cortex (framed). The thin vertical line is drawn at the peak of the face-specific response. (Insert) The channel pair showing the face-specific response is depicted on an expanded scale in the center. The occipital dipolar magnetic field pattern at the peak of the face-specific response is illustrated in the upper part; the center of the white arrow indicates the approximate location of the source for this response; the head is viewed from the right and slightly behind. The isocontours are separated by 30 fT. The lower part shows the field pattern at the same latency for pointillized faces. In this case there is very weak indication of activity in the right visual cortex. EXPERIMENTAL PROCEDURES

Seven subjects with normal vision were studied (one left-handed, three females, four males, 22–35 years old). During the recordings, the subject sat in a magnetically shielded room with the head supported against the helmetshaped magnetometer.

Four different stimulus categories were presented (see Fig. 1): (i) 23 faces of persons working in the laboratory; (ii) ‘‘pointillized’’ (Adobe Photoshop=) versions of these 23 faces; the altered figures did not look like faces at all, but their mean luminance was preserved; (iii) 23 images of everyday objects such as a watering can, a watch, scissors etc.; and (iv) a white sphere on a grey background. All

Face-specific response stimulus categories were equiprobable (P=0.25) and they were presented in random order within one sequence. The subjects were instructed to fixate on a cross in the center of the image, avoid eye movements during stimuli and concentrate on looking at the pictures without any other specific task. The pictures, 3.1) in height and 2.6) in width, were presented against grey background for 500 ms, once per second, on a computer screen located outside the darkened magnetically shielded room. The subjects viewed them through a hole via a mirror system. The magnetic brain signals were recorded with the Neuromag-122 system,1 which has 122 planar first-order SQUID (Superconducting QUantum Interference Device) gradiometers covering the whole head. Each sensor block contains a pair of independent units that measure two orthogonal tangential derivatives of the magnetic field component Bz, normal to the helmet surface at the sensor location. With this instrument, brain activity produces the largest signal just above the source, where the field gradient has its maximum. The two derivatives measured at each site also give the direction of the source current. The signals were analogically bandpass-filtered (0.03– 100 Hz), digitized at 0.4 kHz and averaged on-line. The 900-ms analysis period included a 100-ms prestimulus baseline. A minimum of 120 responses were averaged for each stimulus category. To check the replicability of the data, the experiment was performed twice for each subject. A vertical electro-oculogram was used to reject data contaminated by eye movements or blinks. Three marker coils were attached to the scalp, and their positions in the head coordinate frame were measured with a three-dimensional digitizer. The coordinate system was specified by the two periauricular points and the nasion. The head position with respect to the sensor array was then determined by feeding current to the marker coils when the subject was seated under the magnetometer. To identify neural sources underlying the evoked magnetic responses, equivalent current dipoles (ECDs), each best describing the measured field pattern at a given time instant, were extracted by a least-squares search. After digital low-pass filtering at 30 Hz, single ECDs were found using a subset of channels around the one showing the largest signal. The goodness-of-fit (g) of the source model was also calculated.24 Only ECDs explaining more than 80% of the field variance during the face-specific response were accepted. In one subject (S3), the ECD locations were related to brain anatomy obtained from magnetic resonance images. The magnetic resonance imaging (MRI) coordinate system was aligned with the help of oil capsules attached to the nasion and to the two preauricular points before MRI acquisition. These marker locations were picked from the images. The volume conductor sphere was fitted on the basis of the magnetic resonance images to the local curvature of the surface of the brain over the source area.

RESULTS

Figure 2 shows the neuromagnetic responses of subject S3 to faces (continuous lines) and pointillized faces (dashed lines). The largest signals to both stimuli, as also to everyday objects and spheres, were recorded over the occipital cortex; the frontal and frontotemporal areas showed only negligible responses. Responses to faces and pointillized faces differed systematically over the right occipital cortex (framed channel pair and inset) from 120 ms onwards; the difference was largest around 160 ms. The occipital magnetic field pattern at the peak of the face

51

Fig. 3. Neuromagnetic signals of four subjects over the posterior part of the left and right hemispheres. Traces from two consecutive recordings are superimposed. The responses to face stimuli are presented with thick continuous lines. The thinner continuous curves indicate responses to pointillized faces, the thinnest short dashed lines responses to everyday objects and longer dashed lines responses to spheres.

response is shown in the upper part of the inset. The arrow depicts the surface projection of the ECD. The field pattern drawn at the same latency for the pointillized faces (lower part of the inset) suggested very weak activity over the occipital cortex. Responses to the two stimulus categories also differed at the posterior channels (Fig. 2). It is suggested that this difference reflects the differential processing of simple visual features. Response differences were also found between faces and other control stimuli. These differences were not, however, systematic but were different for various control–stimulus categories. Of the seven subjects studied, four showed clear and replicable face-specific responses. The responses from these subjects are depicted in Fig. 3. Subject S1, who is right handed, had a face-specific response over both hemispheres but the signal was clearly larger over the left hemisphere. The other three subjects produced a face-specific signal only over the right hemisphere; the responses peaked at 150–170 ms. The other stimulus categories elicited only small signals at about the same latency. Note the good replicability of the face-specific responses. Figure 4 (upper part) shows the source location for the face-specific response of subject S3, projected on her MRI surface rendering. This ECD, pointing towards the occipitoparietal area, accounted for 84% of the field variance. MRI slices cut through the source are illustrated in the bottom row. The Talairach41 coordinates of the source are 9-c-G, agreeing with the location of the fusiform gyrus.

52

M. Sams et al.

Fig. 4. (Top, from left to right) Right, back and top views of MRI surface renderings of subject S3. The source area (white dot) of the face-specific response is projected on to the surface of the brain along the viewing direction. (Bottom) MRI slices at the source level of the face-specific response. The line in the sagittal section shows the direction of the ECD.

Figure 5 illustrates the calculated source locations and directions for the face-specific responses for subjects S1–S4. The corresponding goodness-of-fits were 89, 87, 84 and 83%. The right-hemisphere response of subject S1 (Fig. 3) was not adequately explained by a single ECD. In all subjects, ECDs point towards the centroparietal or occipitoparietal scalp. The sources were in the occipital regions, but clearly outside the primary projection areas. We did not find face-specific responses in more anterior parts of the brain. Faces, pointillized faces (Fig. 2) and the other picture categories elicited many other MEG deflections at 80–400 ms. These responses were predominantly recorded over the back of the head, evidently arising from occipitoparietal areas. Because the fixation point was at the center of the images, stimuli activated the visual cortex in a very complex way. Therefore, it was not possible to determine the location of the ECDs for these different signals, which certainly reflect the mixed activation of several cytoarchitectonic areas.

DISCUSSION

The present study reports replicable face-specific neuromagnetic responses in four of seven normal human subjects. The combination of functional information from MEG and anatomical data from MRI suggests that the face-specific activity is generated in the inferior occipitotemporal cortex. Very similar responses occurring at approximately the same latency have been reported in intracranial recordings,3 evoked-potential recordings21 and in a separate MEG experiment conducted in our laboratory.15 The source location of the response agrees with that found in other recent studies.3,10,15,18,34,39 Face-specific responses peaked at 150–170 ms, in accordance with evoked potential data21 and subdural recordings.3 Single units in the temporal lobe of the macaque monkey (STPa) reach their maximum firing rate on average 180 ms after face stimuli,30 whereas the responses from V1 have a mean latency of about 50 ms.37 The first human MEG responses, probably originating from the striatal and extra-

Face-specific response

Fig. 5. Horizontal (xy), sagittal (yz) and coronal (xz) views illustrating the sources of face-specific responses in four subjects. The schematic head (radius=7 cm) is shown to give an impression of the approximate source locations in the brain.

striatal areas, peak at 80–100 ms.2 The rather long peak latency and the source location suggest that the face-specific signal reflects activation of brain mechanisms involved in complex visuoperceptual processing of the facial image. The source locations of the face-specific responses varied across the subjects. The common feature was that all ECDs point towards the centroparietal or occipitoparietal scalp and that the sources were in the posterior brain regions, outside the primary projection areas. In S3, whose magnetic resonance images were available, the ECD was located in the area of the fusiform gyrus, in the occipitotemporal sulcus. However, as suggested by previous studies, rather extensive areas in the gyral and sulcal cortices are activated by face stimuli.34 When this activity is

53

modeled by a single dipole, its location denotes a center of gravity of the activated area. In the other subjects, the right-hemisphere sources were more superior and located rather anteriorly (S4) or posteriorly (S2). This variability might be due to individual differences in the locations of the facespecific cortical areas.3 Additional variability might be caused by simultaneous activity of more than one adjacent face-specific area, whose relative contributions might vary (cf. Ref. 39). Meaningful control stimuli, comparable in their complexity to faces, typically elicit smaller and later electrical evoked responses than faces.21–23 Similarly, our control stimuli weakly activated the right occipital cortex at 150–170 ms (Fig. 2, Fig. 3). These findings suggest that even non-face stimuli can activate the face neurons, although less intensively. However, recent subdural recordings3 imply that the inferior occipital cortex contains areas which are very specific to faces and that other stimulus types may elicit responses at the same latency, but at different cortical locations. One possible explanation for the absence of facespecific responses in three of our seven subjects is source orientation. If the source is close to radial, the signal cannot be recorded by MEG. In addition, face-specific neurons may be located rather deeply,3 resulting in signals too weak to be picked up. Jeffreys21 identified an evoked potential to faces in eight of nine subjects, but because the responses were small, the conclusions were based on recordings on only three subjects. Jeffreys and Tukmachi23 screened each new volunteer in order to obtain subjects with distinct ‘‘vertex positive peaks’’, which they have demonstrated to be sensitive to faces. A previous face study from this laboratory27 suggested two face-specific areas, one near the occipitotemporal junction and another in the middle temporal lobe. The response in the occipitotemporal junction peaked at 95–160 ms and was on average 38 mm beneath the skull. The signal is probably analogous to the face-specific response of the present study, which actually suggests an origin in the inferior occipitotemporal cortex. The present study did not reveal face-specific responses in the midtemporal areas. Neuronal activity in the middle temporal gyrus may be modulated during identification of facial emotions.29 It is possible that the subjects in our previous study paid attention to the emotionality of the faces. The polarity of the face-specific response in subdural recordings3 suggests source currents oriented towards the parietal lobe and vertex. A similar direction was obtained for the present ECDs. This kind of current source appears as a scalp positivity close to the vertex, as was found in EEG recordings.21–23 Lateralization of functions involved in face processing is still a controversial issue.4,5 In subdural recordings, Allison et al.3 found face-specific signals from the fusiform and inferior temporal gyri of both

54

M. Sams et al.

hemispheres. Similarly, cortical volumes activated by face stimuli in a recent functional MRI study were not different in the two hemispheres.34 On the other hand, prosopagnostic patients studied recently with MRI and PET showed no involvement of the left hemisphere.40 Sergent et al.39 suggested that three areas of the right hemisphere play a crucial role in face processing: (i) the ventral part of the right occipital cortex is involved in perceptual operations, (ii) the more anterior ventral regions of the right hemisphere relate perceptual information to stored biographical memories, and (iii) the anterior temporal cortex of both hemispheres acts as a depository of biographical memories. The authors emphasized the role of the right hemisphere in face recognition and considered the role of the left hemisphere as complementary. In the present study, three subjects demonstrating the face-specific response showed it unilaterally in the

right hemisphere and in only one was the signal bilateral. Our subjects were looking at the faces with no particular task. In studies reporting bilateral activation during face recognition, the subjects have performed some type of stimulus discrimination.3,17,39 We suggest that the present type of ‘‘passive’’ viewing task favors the global, configurational processing of facial features, suggested to be a function of the right hemisphere.7,26,31,35,38 Our conclusion is that the present results, together with lesion data and PET studies, suggest a dominance of the right inferior temporal cortex in global visuoperceptual face processing.

Acknowledgements—This study was supported by the Academy of Finland. We thank Veikko Jousma¨ki and Satu Tissari for help in the analysis of MRI data.

REFERENCES

1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. 15. 16.

17.

18. 19. 20. 21. 22. 23.

Ahonen A. I., Ha¨ma¨la¨inen M. S., Kajola M. J., Knuutila J. E. T., Laine P. P., Lounasmaa O. V., Parkkonen L. T., Simola J. T. and Tesche C. D. (1993) 122-channel SQUID instrument for investigating the magnetic signals from the human brain. Phys. Scripta T49, 198–205. Aine C., Bodis-Wollner I. and George J. (1990) Generators of visually evoked neuromagnetic responses. Spatialfrequency segregation and evidence for multiple sources. In Advances in Neurology (ed. Sato S.), pp. 141–155. Raven, New York. Allison T., Ginter H., McCarthy G., Norbe A., Puce A., Luby M. and Spencer D. (1994) Face recognition in human extrastriate cortex. J. Neurophysiol. 71, 821–825. Benton A. L. (1985) Facial recognition 1984. In Studies in Neuropsychology: Selected Papers of Arthur Benton (eds Costa L. and Spreen O.), pp. 224–235. Oxford University Press, New York. Benton A. L. (1990) Facial recognition 1990. Cortex 26, 491–499. Bo¨tzel K. and Gru¨sser O.-J. (1987) Potentials evoked by face and nonface stimuli in the human electroencephalogram. Perception 16, A21. Bradshaw J. L. and Sherlock D. (1982) Bugs and faces in the two visual fields: the analytic/holistic processing dichotomy and task sequencing. Cortex 18, 211–226. Bruce V., Doyle A., Dench N. and Burton M. (1991) Remembering facial configurations. Cognition 38, 109–144. Bruce V. and Young A. (1986) Understanding face recognition. Br. J. Psychol. 77, 305–327. Clark V., Parasuraman R., Keil K., Maisog J., Ungerleider L. and Haxby J. (1995) FMRI studies of attention to color and face identity. Human Brain Mapping Suppl. 1, 32. De Haan E. H. F., Young A. W. and Newcombe F. (1992) Neuropsychological impairment of face recognition units. Q. Jl exp. Psychol. 44, 141–175. De Renzi E., Faglioni P., Grossi D. and Nichelli P. (1991) Apperceptive and associative forms of prosopagnosia. Cortex 27, 213–222. Ellis A. W., Young A. W. and Flude B. M. (1990) Repetition priming and face processing: priming occurs within the system that responds to the identity of a face. Q. Jl exp. Psychol. 39A, 193–210. Gross C. G. (1992) Representation of visual stimuli in inferior temporal cortex. Phil. Trans. R. Soc. Lond. B 335, 3–10. Halgren E., Raij T., Marinkovic K., Jousma¨ki V. and Hari R. (1995) Magnetic fields evoked by faces in the human brain: 1. Topography and source locations. Soc. Neurosci. Abstr. 211, 267. Ha¨ma¨la¨inen M., Hari R., Ilmoniemi R. J., Knuutila J. and Lounasmaa O. V. (1993) Magnetoencephalography— theory, instrumentation, and applications to noninvasive studies of the working human brain. Rev. mod. Phys. 65, 413–497. Haxby J. V., Grady C. L., Horowitz B., Salerno J., Ungerleider L. G., Mishkin M. and Schapiro M. B. (1993) Dissociation of object and spatial visual processing pathways in human extrastriate cortex. In Functional Organization of the Human Visual Cortex (eds Gulyas B., Ottoson D. and Roland P. E.), pp. 329–340. Pergamon, Oxford. Haxby J., Ungleider L., Horwitz B., Rapoport S. and Grady C. (1995) Hemispheric differences in neural systems for face working memory: a PET–rCBF study. Human Brain Mapping 3, 68–82. Hietanen J. K., Perrett D. I., Oram M. W., Benson P. J. and Dittrich W. H. (1992) The effects of lighting conditions on responses of cells selective for face views in the macaque temporal cortex. Expl Brain Res. 89, 157–171. Hilliard R. D. (1973) Hemispheric laterality effects on a facial recognition task in normal subjects. Cortex 9, 246–258. Jeffreys D. A. (1989) A face-responsive potential recorded from the human scalp. Expl Brain Res. 78, 193–202. Jeffreys D. A. (1993) The influence of stimulus orientation on the vertex positive scalp potential evoked by faces. Expl Brain Res. 96, 163–172. Jeffreys D. A. and Tukmachi E. S. A. (1992) The vertex-positive scalp potential evoked by faces and by objects. Expl Brain Res. 91, 340–350.

Face-specific response 24. 25. 26. 27. 28. 29. 30. 31. 32. 33. 34. 35. 36. 37. 38. 39. 40. 41. 42. 43.

55

Kaukoranta E., Hari R., Ha¨ma¨la¨inen M. and Huttunen J. (1986) Cerebral magnetic fields evoked by peroneal nerve stimulation. Somatosensory Res. 3, 309–321. Klein D., Moscovitch M. and Vigna C. (1976) Attentional mechanisms and perceptual asymmetries in tachistoscopic recognition of words and faces. Neuropsychology 14, 55–66. Levy J., Trevarthen C. and Sperry R. W. (1972) Perception of bilateral chimeric figures following hemispheric deconnexion. Brain 95, 61–78. Lu S. T., Ha¨ma¨la¨inen M., Hari R., Ilmoniemi R., Lounasmaa O. V., Sams M. and Vilkman V. (1991) Seeing faces activates three separate areas outside the occipital visual cortex in man. Neuroscience 43, 287–290. Newcombe F., Young A. W. and de Haan E. H. F. (1989) Prosopagnosia, and object agnosia without covert recognition. Neuropsychology 27, 179–191. Ojemann J. G., Ojemann G. A. and Lettich E. (1992) Neuronal activity related to faces and matching in human right nondominant temporal cortex. Brain 115, 1–13. Oram M. W. and Perrett D. I. (1992) Time course of neural responses discriminating different views of the face and head. J. Neurophysiol. 68, 70–84. Parkin A. J. and Williamson P. (1987) Cerebral lateralization at different stages of facial processing. Cortex 23, 99–110. Perrett D. I., Hietanen J. K., Oram M. W. and Benson P. J. (1992) Organization and function of cells responsive to faces in the temporal cortex. Phil. Trans. R. Soc. Lond. B 335, 23–30. Perrett D. I., Oram M. W., Harries M. H., Bevan R., Hietanen J. K., Benson P. J. and Thomas S. (1991) Viewer-centred and object-centred encoding of heads by cells in the superior temporal sulcus of the rhesus monkey. Expl Brain Res. 86, 159–173. Puce A., Allison T., Gore J. C. and McGarthy G. (1995) Face-sensitive regions in human extrastriate cortex studied by functional MRI. J. Neurophysiol. 74, 1192–1199. Rapcsak S. Z., Polster M. R., Comer J. F. and Rubens A. B. (1994) False recognition and misidentification of faces following right hemisphere damage. Cortex 30, 565–583. Rizzolatti G., Umilta` C. and Berlucchi G. (1971) Opposite superiorities of the right and left cerebral hemispheres in discriminative reaction time to physiognomical and alphabetical material. Brain 94, 431–442. Robinson D. L. and Rugg M. D. (1988) Latencies of visually responsive neurons in various regions of the rhesus monkey brain and their relation to human visual responses. Biol. Psychol. 26, 111–116. Sergent J. (1984) An investigation into component and configural processes underlying face perception. Br. J. Psychol. 75, 221–242. Sergent J., Ohta S. and MacDonald B. (1992) Functional neuroanatomy of face and object processing. Brain 115, 15–36. Sergent J. and Signoret J. L. (1991) Outstanding issues in the study of prosopagnosia. J. clin. exp. Neuropsychol. 13, 34. Talairach J. and Tournoux P. (1988) Co-Planar Stereotaxic Atlas of the Human Brain. George Thieme, Stuttgart. Valentine T. (1991) A unified account of the effects of distinctiveness, inversion and race in face recognition. Q. J. exp. Psychol. 43A, 161–204. Young A. W., Ellis A. W. and Flude B. M. (1988) Accessing stored information about familiar people. Psychol. Res. 50, 111–115. (Accepted 16 July 1996)