02e7e52b89f5082d12000000

See discussions, stats, and author profiles for this publication at: http://www.researchgate.net/publication/222679720

Prior knowledge, reading skill, and text cohesion in the comprehension of science texts. Learning and Instruction, 19(3), 228-242 ARTICLE in LEARNING AND INSTRUCTION · JUNE 2009 Impact Factor: 3.73 · DOI: 10.1016/j.learninstruc.2008.04.003

CITATIONS

READS

69

271

3 AUTHORS, INCLUDING: Yasuhiro Ozuru University of Alaska Anchorage 22 PUBLICATIONS 357 CITATIONS SEE PROFILE

Available from: Yasuhiro Ozuru Retrieved on: 20 October 2015

This article appeared in a journal published by Elsevier. The attached copy is furnished to the author for internal non-commercial research and education use, including for instruction at the authors institution and sharing with colleagues. Other uses, including reproduction and distribution, or selling or licensing copies, or posting to personal, institutional or third party websites are prohibited. In most cases authors are permitted to post their version of the article (e.g. in Word or Tex form) to their personal website or institutional repository. Authors requiring further information regarding Elsevier’s archiving and manuscript policies are encouraged to visit: http://www.elsevier.com/copyright

Author's personal copy

Learning and Instruction 19 (2009) 228e242 www.elsevier.com/locate/learninstruc

Prior knowledge, reading skill, and text cohesion in the comprehension of science texts Yasuhiro Ozuru*, Kyle Dempsey, Danielle S. McNamara University of Memphis, Psychology Building, Memphis, TN 38152-3230, USA Received 1 May 2007; revised 11 December 2007; accepted 30 April 2008

Abstract This study examined how text features (i.e., cohesion) and individual differences (i.e., reading skill and prior knowledge) contribute to biology text comprehension. College students with low and high levels of biology knowledge read two biology texts, one of which was high in cohesion and the other low in cohesion. The two groups were similar in reading skill. Participants’ text comprehension was assessed with openended comprehension questions that measure different levels of comprehension (i.e., text-based, local-bridging, global-bridging). Results indicated: (a) reading a high-cohesion text improved text-based comprehension; (b) overall comprehension was positively correlated with participants’ prior knowledge, and (c) the degree to which participants benefited from reading a high-cohesion text depended on participants’ reading skill, such that skilled participants gained more from high-cohesion text. Ó 2008 Elsevier Ltd. All rights reserved. Keywords: Text comprehension; Text cohesion; Reading skill; Science learning

1. Introduction Comprehension of expository materials is a complex process that depends on a number of factors. For example, past research on expository text comprehension has established that how well an individual comprehends and learns from expository texts is a function of a complex interaction between individual differences and text features (Linderholm, Everson, van den Broek, Mischinski, Crittenden et al., 2001; McNamara, Kintsch, Songer, & Kintsch, 1996; O’Reilly & McNamara, 2007; Voss & Silfies, 1996). However, these studies have not been in complete agreement on how individual differences (e.g., reading skill, prior knowledge) interact with text features (e.g., text difficulty, text cohesion) in comprehension processes. The goal of this article is to discern the specific nature of the contributions of two types of individual difference factors and text features to science text comprehension. The two individual difference factors examined are topic-relevant prior knowledge and reading comprehension skill. The specific text feature we focus on is text cohesion, which refers to the extent to which ideas conveyed in the text are made explicit (Graesser, McNamara, Louwerse, & Cai, 2004). Topic-relevant prior knowledge refers to readers’ pre-existing knowledge related to the text content and is often measured with open-ended and/or multiple choice questions on vocabulary and relevant factual information (see Shapiro, 2004). Readers’ topicrelevant knowledge is expected to have a large influence on text comprehension because information explicitly stated in the text is often insufficient for the construction of a coherent mental representation of the situation depicted by the text, requiring the contribution of reader knowledge (Kintsch, 1988, 1998). In support of this argument, empirical evidence has shown that a reader’s

* Corresponding author. Tel.: þ1 901 233 7348. E-mail addresses: [email protected], [email protected] (Y. Ozuru). 0959-4752/$ - see front matter Ó 2008 Elsevier Ltd. All rights reserved. doi:10.1016/j.learninstruc.2008.04.003

Author's personal copy

Y. Ozuru et al. / Learning and Instruction 19 (2009) 228e242

229

prior knowledge facilitates and enhances text comprehension, in particular, of expository materials (Afflerbach, 1986; Chi, Feltovich, & Glaser, 1981). On the other hand, reading skill generally refers to cognitive skills associated with the reading process in general (Gernsbacher, Varner, & Faust, 1990; Hannon & Daneman, 2001; Walker, 1987), and may include a variety of abilities such as word decoding (Perfetti, 1985), syntactic knowledge, and high-level inferential skills (Oakhill & Yuill, 1996). Of all these abilities related to reading skill, one of the most important elements is the ability and propensity to connect various concepts or ideas contained in the text in a coherent manner (Hannon & Daneman, 2001; Oakhill & Yuill, 1996; for a review also see Zwaan & Singer, 2003). Currently, research remains inconclusive concerning precisely what kind of cognitive factors underlie readers’ ability and propensity to integrate textual information to maintain a high level of coherence. Ability to suppress irrelevant information (Gernsbacher 1997; cf. McNamara & McDaniel, 2004), working memory capacity (Daneman & Hannon, 2001), metacognition (Hacker, 1998), reading strategies (McNamara, 2007), and motivation (Guthrie & Wigfield, 1999) are some of the major candidate factors underlying reading comprehension skill. Although reading comprehension skill and prior knowledge may not be completely separable, they are assumed to contribute to reading comprehension processes in somewhat different ways (Hannon & Daneman, 2001; Walker, 1987). Prior knowledge helps readers compensate for gaps in text-based information by affording quick and relatively effortless access to relevant information in long-term memory based on incomplete text-based information as cues. In contrast, reading comprehension skill helps readers relate multiple ideas and concepts appearing in different parts of a text through effortful inferential processes (Daneman & Hannon, 2001). This process, of relating multiple ideas, helps readers build more integrated understanding of text content even when readers do not have high levels of prior knowledge to facilitate knowledge-driven recognition of the text content. Given the different roles of prior knowledge and reading skill in reading comprehension, the relative contribution of prior knowledge and reading skill should change depending on the type of text and the level of comprehension involved (i.e., the types of questions used to assess comprehension).

1.1. Prior knowledge and reading skill in relation to text cohesion Even within a specific genre and topic (i.e., an expository biology text), texts can vary in a number of different ways. Cohesiveness of a text is one of the important dimensions along which text varies. Cohesiveness of a text, an objective feature of texts, is an important factor to determine text coherence, which is a subjective psychological state of a reader (Graesser, McNamara, & Louwerse, 2003; Halliday & Hasan, 1976). When comprehending a text, readers must establish and maintain coherence between sentences (Oakhill, Cain, & Bryant, 2003; van den Broek, 1994). When reading a highly cohesive text, the majority of information necessary to maintain text coherence is provided by the text itself. On the other hand, when reading a less cohesive text, readers need to rely more heavily on relevant knowledge to maintain coherence. Text cohesion changes by the way in which adjacent sentences are connected, such as the degree of conceptual overlap (e.g., argument overlap) between sentences and by presence of specific cues (e.g., connectives) that help readers connect ideas across sentences. Text cohesion also varies as a function of the overall organization which can be expressed by the temporal and causal sequence of the events in the text (Linderholm et al., 2001) and the presence of headers and topic sentences (McNamara et al., 1996). These features contribute to the maintenance of text coherence by reducing the need for the reader to rely on knowledge. The cohesiveness of text differs from text readability or difficulty which usually refers to sentence length and individual word difficulty, e.g., Flesch Reading Ease (Flesch, 1948). Given that text cohesion influences readers’ maintenance of text coherence, readers’ prior knowledge and reading skill should interact with text cohesion in different ways in influencing comprehension. With respect to readers’ prior knowledge level, the benefit of text cohesion should be more pronounced for readers with less knowledge. That is, whereas maintenance of text coherence in a less cohesive text demands contribution of specific knowledge, a highly cohesive text is more self-contained, hence, requires less contribution of topic-specific knowledge for maintenance of text coherence. This notion is supported by the finding that low-knowledge readers benefit from reading high-cohesion texts, whereas high-knowledge readers’ comprehension often suffers when reading high-cohesion texts (McNamara et al., 1996; McNamara & Kintsch, 1996). On the other hand, reading skill is expected to interact with text cohesion differently. Readers with poor reading comprehension skill may not benefit as much as skilled readers from reading a high-cohesion text because increasing text cohesion often involves adding more information (Beck, McKeown, Sinatra, & Loxterman, 1991), resulting in increased text length, density, and complexity. As a consequence, comprehension of a highly cohesive text may require higher level of reading skill because reading a highly cohesive text involves processing larger amounts of text-based information. This proposal is not entirely new. For example, the Cognitive Load Theory (CLT) postulates that many types of cognitive tasks, including reading comprehension, can be understood as a process by which task performers negotiate with the task demands using two resources: a limited cognitive resource determined by working memory capacity and the ability to access large amount of relevant knowledge (e.g., schema activation) with relatively little cognitive resources (Sweller, 1999). According to this view, reading comprehension performance is likely to be affected not only by the extensiveness of readers’ knowledge but also by

Author's personal copy

230

Y. Ozuru et al. / Learning and Instruction 19 (2009) 228e242

information-processing demands determined by the text features (e.g., amount of information that readers need to process using limited cognitive resources). A study conducted by Voss and Silfies (1996) attempted to address this issue. They reported that the ability to comprehend a less cohesive text is more closely correlated with a reader’s prior knowledge level whereas the ability to comprehend an expanded and more cohesive text is more closely related with reading skill. However, the Voss and Silfies (1996) study is limited for several reasons. First, their study does not provide much information about the benefit (or detrimental effect) of cohesion. Therefore, who benefits more from reading a high as opposed to a low-cohesion text remains unanswered. Another limitation is the text genre. Their study, similar to many other studies of text revision (Beck et al., 1991; Linderholm et al., 2001), was conducted with history texts (though the content was fictitious). Natural science texts (e.g., biology texts) differ from social studies or history texts in that natural science texts tend to present a number of new and abstract concepts (e.g., osmosis, mitosis, etc.) and their relations (e.g., relations between endotherms and warm-blooded animals). These scientific concepts are often difficult to ground in everyday experiences (Graesser, Leon, & Otero, 2002). In contrast, history texts often present relatively familiar information (e.g., conflict between groups, desire to gain power, independence) in a novel context (e.g., Russian revolution). Use of fictitious history texts cannot eliminate this effect of topic familiarity because people may have general schemata on typical social issues such as conflict, politics, and financial problems. Thus, the way in which cohesion manipulations influence the comprehension of social science and natural science texts (e.g., biology texts) may differ. To explore this issue with science text, O’Reilly and McNamara (2007) examined the interaction between text cohesion and two types of individual differences (prior knowledge and reading skill) using a biology text. They found that low-knowledge readers benefited from reading a high-cohesion text, whereas high-knowledge readers benefited from reading a high-cohesion text only when they had a relatively high level of reading skill. In contrast, unskilled, high-knowledge readers’ comprehension was worse for a high-cohesion text. One limitation of the O’Reilly and McNamara (2007) study is that text cohesion was manipulated as a between-subjects variable. This design of the study limits the option of statistical analyses available to examine the interaction between text cohesion and individual differences, that is, how the relative impact of prior knowledge and reading skill on comprehension depends on text cohesion. As will be described in greater detail later, the present study attempted to overcome this limitation by employing a within-subject manipulation of text cohesion with a new set of biology texts. 1.2. Effects of prior knowledge and reading skill on comprehension The other issue that the present study explored is the relative contribution of reading skill and prior knowledge to comprehension irrespective of text cohesion. As discussed earlier, there is ample evidence showing that prior knowledge has a large influence on expository text comprehension (Afflerbach, 1986; Chi et al., 1981). The present study attempted to extend these findings by exploring whether, and how, the relative contribution of prior knowledge changes depending on the level of comprehension. By level of comprehension, we refer to distinctions made by ‘‘text base’’ and ‘‘situation model’’ level comprehension (Kintsch, 1998). In this paper, we adopted a rather loose distinction between text base and situation model by assuming some continuity between them; that is, we assumed that some types of comprehension involve less integration of information (closer-to-text base), and other types of comprehension involve more extensive integration of information (closer-to-situation model), and these different levels of comprehension can be assessed by different types of comprehension questions. According to this notion, closer-to-text base comprehension can be operationally defined as performance on comprehension questions that require minimal information integration (i.e., information explicitly stated within a sentence). On the other hand, closer-to-situation model level comprehension can be operationally defined by performance on comprehension questions that require more extensive information integration (i.e., bridging that involves integration of information across two or more sentences). We explored changes in the relative contribution of prior knowledge to text comprehension as a function of different types of comprehension questions. 1.3. Research questionsdhypotheses Hence, this study explored two research questions: First, how does the relative contribution of prior knowledge and reading skill to comprehension change as a function of type of comprehension questions? Second, what is the relative contribution of prior knowledge and reading skill to the benefit and/or detrimental effect of text cohesion on comprehension? To explore these research questions, participants read both low- and high-cohesion versions of biology texts, and answered three types of comprehension questionsdthat is, text-based questions, local-bridging questions, and global-bridging questionsdbased on memory of the text content. As regards the first research question, we predicted that prior knowledge has a greater contribution to comprehension of science text than reading skill (Hypothesis 1). In addition, we also hypothesized that the contribution of prior knowledge to performance on comprehension questions would be larger for the types of questions that require more extensive information

Author's personal copy

Y. Ozuru et al. / Learning and Instruction 19 (2009) 228e242

231

integrationdi.e., local- and global-bridging questions as opposed to text-based questions (Hypothesis 2). This prediction is based on the assumption that readers may not generate inferences to attain global level comprehension unless such an inference can be drawn or readily retrieved from pre-existing knowledge (Kintsch, 1993) and with relatively little expenditure of cognitive resources (McKoon & Ratcliff, 1992). The above assumption is supported by the finding that readers often do not generate inferences (e.g., causal backward inferences) involving multiple related concepts (i.e., synthesizing the multiple text-based ideas) online when reading unfamiliar science materials (Noordman, Vonk, & Kempff, 1992). That is, when readers are able to answer local- and global-bridging questions based on science materials, their answer is likely to be largely based on retrieval of pre-existing knowledge as opposed to the product of resource consuming reasoning processes that involve linking multiple ideas in the text while reading the text. With respect to the second research question, we expected both prior knowledge and reading skill to interact with text cohesion. Specifically, we expected a benefit of increased cohesion on comprehension especially for low-knowledge participants because cohesive text fills in conceptual gaps for low-knowledge readers that cannot be resolved with prior knowledge (Hypothesis 3). We also expected the benefit of reading high-cohesion text to be generally larger for more-skilled readers because reading skill is necessary for taking advantage of added cohesion. In particular, this interaction between reading skill and text cohesion was expected to be pronounced for high-knowledge readers; that is, we expected high-knowledge participants’ comprehension to remain the same across low- and high-cohesion texts or even decline when reading high-cohesion texts (McNamara et al., 1996) unless the participant has a high level of reading comprehension skill (O’Reilly & McNamara, 2007) (Hypothesis 4). The formulation of Hypothesis 4 was based on the notion that a high-cohesion text may contain information that is more familiar to high-knowledge readers. As a consequence, they may more shallowly process the high-cohesion text because of a false sense of understanding (McNamara et al., 1996). This will occur unless they have a higher level of reading skill, which is typically associated with tendency to carefully and systematically process textual information and to generate inferences that relate multiple concepts in the text (O’Reilly & McNamara, 2007). Thus, reading skill induces a high-knowledge reader to actively process the text regardless of its cohesion. The above explanation is on the surface somewhat similar to the ‘‘expertise reversal effect’’ identified in the context of CLT (Kalyuga, Ayres, Chandler, & Sweller, 2003). According to the expertise reversal effect, providing high-knowledge participants with highly cohesive texts with ‘‘unnecessary details’’ can produce adverse effects due to interference because these highknowledge readers can efficiently maintain coherence based on their own knowledge alone. The expertise reversal effect also implies that interference from added cohesion is likely to be more pronounced among less-skilled readers because less-skilled readers are typically less efficient in using limited cognitive resources when processing a large amount of information (Daneman & Hannon, 2001). These two explanations, one based on less-skilled high-knowledge readers’ tendency to shallowly process a high-cohesion text and the other based on the expertise reversal effect, are not the same (Kalyuga et al., 2003). But within the design of the current study, both explanations generate similar predictions. Further, related to the second question, we also explored the level of comprehension at which the benefit of text cohesion is observed. We expected that the benefit of reading a high-cohesion text, in particular for low-knowledge readers, will be limited to relatively lower levels of comprehensionde.g., answering performance on text-based questions (Hypothesis 5). This proposal is based on the notion that readers are unlikely to draw inferences (i.e., local- and global-bridging inferences to attain higher level comprehension) unless they are already familiar with information related to the text content (Noordman et al., 1992). 2. Method 2.1. Participants Participants were recruited from two distinct sources in order to manipulate the level of biology knowledge relevant to the reading comprehension materials. One group of participants was 108 undergraduate students enrolled in an introductory Psychology course at the University of Memphis of which 93 were female and 15 were male. Mean age of this group was 21.1 years (SD ¼ 3.6) with range of 18 to 37 years. The other group of participants was 62 undergraduate students enrolled in an introductory Biology course at Old Dominion University of which 53 were female and 9 were male. Mean age of this group of participants was 23.3 years (SD ¼ 2.3) with range of 21 to 37 years. The participants from Old Dominion University were recruited because it was possible to specifically recruit students enrolled in Biology courses, and this was not possible at the University of Memphis. The two universities are considered comparable according to the college rankings reported in the U.S. News, 2007. Testing of the two groups of participants took place within the same year. 2.2. Designdmaterials The 2  2  3 experimental design included text cohesion (low and high) and type of question (text-based, local-bridging, and global-bridging) as within-subjects variables. Knowledge level of participants (2 levels) was included as a between-subjects

Author's personal copy

232

Y. Ozuru et al. / Learning and Instruction 19 (2009) 228e242

variable (Biology class and Psychology class participants) upon confirming the distinct knowledge level of the two groups of participants based on prior knowledge measures. In addition, participants’ reading skill was assessed and included in the analysis as control variable using a median split technique. Effects of the two individual differences factors were also analyzed with regression analyses. 2.2.1. Texts and cohesion manipulation The two texts used for the reading comprehension task were taken from high-school biology textbooks and were modified to produce low- and high-cohesion texts. One text described a plant’s response to an external stimulus (Plant text), and the other described internal distributions of heat in animals (Heat text). Manipulations to increase cohesion included: (1) replacing ambiguous pronouns with nouns; (2) adding descriptive elaborations that link unfamiliar concepts with familiar concepts; (3) adding connectives to specify the relationships between sentences or ideas; (4) replacing or inserting words to increase the conceptual overlap between adjacent sentences; (5) adding topic headers; (6) adding thematic sentences that serve to link each paragraph to the rest of the text and overall topic; and (7) changing sentence structures to incorporate the additions and modifications. Appendix A contains an example of the low- and high-cohesion versions of one of the texts (i.e., Heat text) in which the specific changes are marked. Table 1 provides key text features related to text cohesion and text difficulty. As indicated in Table 1, the text revisions increased the text length by approximately 50%. However, this level of increase in the text length is common to past text revision studies (Beck et al., 1991; Voss & Silfies, 1996). The levels of cohesion and text difficulty of the two texts were monitored based on text features that are known to be indicative of text cohesion and text difficulty. The features that are indicative of text cohesion included argument overlap and Latent Semantic Analysis (LSA) cosine between sentences. The features, which were indicative of more conventional text difficulty and readability, included word frequency and Flesch-Kincaid grade level (Flesch, 1948). Argument overlap between sentences represents the proportion of sentence pairs (adjacent or all) in the text that share an argument. Hence, adjacent and all sentence-argument overlap measures represent local and global aspects of cohesion, respectively. The LSA cosine is a proxy measure of conceptual similarity between linguistic units (Landauer & Dumais, 1997). The LSA approximates conceptual similarity using a mathematical technique similar to a factor analysis. Thus, LSA cosine represents text cohesion based on conceptual similarity, not solely by overlap of a particular word (i.e., argument overlap). Text difficulty varies due to sentence complexity and vocabulary difficulty. Text difficulty is controlled by monitoring word frequency and Flesch-Kincaid grade level. Word frequency is represented by the average word frequency of the lowest word frequency word in each sentence. Hence, texts with lower word frequency tend to have rare, less common, content. FleschKincaid grade level is computed based on word length and sentence length and thus represents difficulty in terms of both sentence length and word difficulty. Monitoring of these text features was achieved using the Coh-Metrix tool (Graesser et al., 2004), a computer-based tool that calculates over 200 measures of text features. As indicated in Table 1, in both the Heat and Plant text, there were relatively large increases in Argument Overlap Adjacent, and Argument Overlap All Sentences, as well as in the LSA Adjacent, and All Sentences, from the low- to high-cohesion version of the texts. These increases indicate that the cohesion manipulation increased text cohesion both locally and globally. On the other hand, the word frequency measure was relatively similar across the low- and high-cohesion versions of the texts, which suggests similarity of overall content before and after the manipulation. Finally, Flesch-Kincaid grade level increased as the result of cohesion manipulation. This was expected given that cohesion manipulation tends to increase sentence length as the result of adding connectives and other cohesive elements. 2.2.2. Reading comprehension questions There were 12 comprehension questions for each text, of which four were text-based questions, four were near- or localbridging questions, and four were far- or global-bridging questions (see Appendix B). A question was classified as text-based when the question could be answered based on information explicitly stated within a sentence. A question was classified as a nearTable 1 Features of high- and low-cohesion Heat and Plant text. Text cohesion

Argument overlap adjacent Argument overlap all sentences LSA adjacent LSA all sentences Word frequency Flesch-Kincaid grade level Number of words LSA, Latent Semantic Analysis.

Heat text

Plant text

Low

High

Low

High

0.58 0.38 0.35 0.31 0.73 9.94 639

0.68 0.50 0.47 0.33 0.64 11.28 999

0.64 0.36 0.34 0.35 0.90 9.05 607

0.85 0.49 0.44 0.39 0.95 10.23 968

Author's personal copy

Y. Ozuru et al. / Learning and Instruction 19 (2009) 228e242

233

or local-bridging question when the answer to the question required an integration of information located within five clauses across multiple sentences (generally adjacent sentences). Far- or global-bridging questions were similar to local-bridging questions but involved the integration of information located across larger distances, more than five clauses apart, and more than two sentences apart. In scoring the response to these questions, participants’ response to each question was compared to answer keys which had been constructed prior to the collection of the data. Whereas all the text-based questions were scored in a binary manner (i.e., incorrect or correct), bridging questions were scored using continuous scale involving half or quarter a point depending on the number of ideas involved in the ideal answer for a specific question. For example, an ideal answer for Example 3 of globalbridging question (‘‘According to the text, how would an endotherm respond to an ambient temperature of 30 degrees Farenheit?’’) should include these two ideas: (a) An endotherm would increase voluntary/involuntary (e.g., shivering) muscle movement to generate heat; (b) An endotherm would decrease the blood flow to extremity to reduce the heat loss to cold surroundings. Participants were awarded 0.5 point for each of these ideas. Participants’ responses to the open-ended questions were scored independently by two raters, and then compared. Interrater reliability was greater than 95%. Discrepancies were resolved by discussion. 2.2.3. Individual difference measures Three types of individual difference measures were collected: reading skill, biology knowledge, and topic-specific knowledge on the topic of the text. Reading skill was measured using the Nelson-Denny (Brown, Fishco, & Hanna, 1993) reading comprehension ability test. The Nelson-Denny reading comprehension ability test is a standardized reading comprehension test for college level students. Cronbach’s alpha of the 38 questions based on 170 participants in this study was 0.90. Biology knowledge was assessed with 21 multiple choice questions on anatomy, reproduction, and genetics, Cronbach’s alpha of these 21 questions was 0.61. Topic-specific knowledge questions on plants and the distribution of heat was measured with a total of 16 open-ended questions on the knowledge of plant biology (eight questions) and animal circulatory systems (eight questions). Cronbach’s alpha of these 16 questions was 0.73. The questions in the topic-specific knowledge measure involved information relevant to understanding the texts, but not provided in the texts. 2.3. Procedure Participants were first administered the biology knowledge test, followed by the Nelson-Denny reading skills assessment. Each test (i.e., prior knowledge and Nelson-Denny) was restricted to 15 min. The participants then read the texts and answered the questions, which were presented in a booklet. Participants read two texts, one low-cohesion text and one high-cohesion text, on two different topics (i.e., plant or heat) and then answered comprehension questions for both texts, in the order of text presentation. Pairing of the topic and cohesion was counter-balanced such that half of the participants read the Heat text in high-cohesion condition (and the Plant text in low cohesion) and the other half read the Plant text in high-cohesion condition (and the Heat text in low cohesion). Order of the presentation was also counter-balanced such that half of the participants read the high-cohesion text first and the other half read the low-cohesion text first. Hence, participants were randomly assigned to four counter-balancing conditions. After the participants finished reading, the experimenter took the text away from the participants so that they could not refer to the text to answer the questions. The decision to use memory-based comprehension questions was based on two factors. First, several studies upon which the present study builds used this technique (Linderholm et al., 2001; O’Reilly & McNamara, 2007; Voss & Silfies, 1996). Hence, use of this technique facilitated a more direct comparison with these studies. Second, in classrooms, students read science texts to prepare for exams and access to the textbook was generally restricted. Therefore, answering questions based on memory for what they had read better simulates a classroom situation. The participants were not allowed to return to the previous section of questions after they had moved to the next set of questions. Topic-specific prior knowledge questions were then presented. We presented the topic-specific knowledge questions after the reading comprehension task because being exposed to questions closely related to the text topic might influence the participants’ reading behavior. Care was taken to ensure that the texts did not contain answers to the topic-specific knowledge questions. 3. Results First, we report the individual difference data for the two groups of participants (i.e., participants from the Psychology class and participants from the Biology class). Table 2 provides the means and standard deviations of the two types of prior knowledge measures (biology knowledge and topic-specific knowledge) and the reading comprehension ability measure (i.e., Nelson-Denny test) for the two groups of participants. We conducted a series of one-way analyses of variance (ANOVA) to examine whether and how the two groups of participants (i.e., psychology and biology participants) differed. ANOVAs indicated that the two groups of participants differed in terms of

Author's personal copy

Y. Ozuru et al. / Learning and Instruction 19 (2009) 228e242

234

Table 2 Mean and standard deviations of participants’ performance on prior knowledge and reading skill measures as a function of knowledge level.

Biology knowledge Topic-specific knowledge Reading skill

Psychology students (n ¼ 108) (Low-knowledge level)

Biology students (n ¼ 62) (High-knowledge level)

M

SD

Min/Max

M

SD

Min/Max

0.44 0.25 0.58

0.14 0.16 0.18

0.10/0.90 0.00/0.72 0.18/0.95

0.51 0.43 0.55

0.17 0.18 0.17

0.15/0.90 0.06/0.88 0.13/0.95

both biology knowledge, F(1, 168) ¼ 9.67, p < 0.01, Cohen’s d ¼ 0.47, and topic-specific knowledge, F(1, 168) ¼ 48.72, p < 0.001, Cohen’s d ¼ 0.95, but they did not differ in terms of the level of reading skill, F(1, 168) ¼ 1.52, p > 0.2, Cohen’s d ¼ 0.17. The results of this analysis confirmed that whereas the two groups of participants exhibited similar levels of general reading skill, their knowledge levels related to biology and the topic of the texts used in reading comprehension task were significantly different. 3.1. Overall analysis As a preliminary analysis, we examined whether text-cohesion pairing (e.g., whether Plant or Heat text in high-cohesion condition) affected performance. The analyses indicated that text-cohesion pairing did not affect overall performance, F < 1.0; also, text-cohesion pairing did not influence the effect of text cohesion (i.e., the benefit or detrimental effect of reading a high-cohesion text as opposed to low-cohesion text), F < 1.0. First, we conducted a mixed model ANOVA including all the relevant variables, that is, 2 (text cohesion)  2 (level of knowledge)  2 (reading skill)  3 (types of comprehension questions) to obtain an overall picture of the effect of text cohesion and individual differences on performance on different types of comprehension questions. The variable representing the two levels of reading comprehension skill was created by median split on all participants (N ¼ 170) based on the Nelson-Denny reading skill test performance. The median split technique may have shortcomings such as artificially creating an ad hoc experimental variable. However, the sample had a large variability in reading skill as measured with Nelson-Denny reading ability assessment (see Table 2), and hence, the median split produced two sufficiently distinct groups of participants that differed in terms of reading skill (less-skilled readers: M ¼ 0.42, SD ¼ 0.09 min/max ¼ 0.13/0.58; more-skilled readers: M ¼ 0.71, SD ¼ 0.11, min/max ¼ 0.58/0.95). Furthermore, in order to overcome the possible limitation of the median split we also used multiple regression analysis (see below). The variable representing the two levels of knowledge was created by using the two groups of participants: participants from the Psychology course constituted the low-knowledge participants (n ¼ 108) and participants from the Biology course constituted the high-knowledge participants (n ¼ 68) because earlier analysis indicated that these two groups of participants significantly differed in their topic-specific knowledge and biology knowledge. This assignment created four cells representing (a) low readingskill and low-knowledge participants (n ¼ 46), (b) low reading-skill and high-knowledge participants (n ¼ 39), (c) high readingskill and low-knowledge participants (n ¼ 62), and (d) high reading-skill and high-knowledge participants (n ¼ 23). Table 3 presents participants’ performance on comprehension questions as a function of text cohesion, knowledge level, reading comprehension skill, and types of questions. The ANOVA indicated significant main effect of knowledge level, F(1, 162) ¼ 54.13, p < 0.001, partial h2 ¼ 0.25, reading skill, F(1, 162) ¼ 22.97, p < 0.001, partial h2 ¼ 0.12, and type of question, F(2, 324) ¼ 71.07, p < 0.001, partial h2 ¼ 0.31. There Table 3 Performance on comprehension questions as a function of question type, cohesion, prior knowledge, and reading skill. Low knowledge

High knowledge

Low cohesion

High cohesion

Low cohesion

High cohesion

M

SD

M

SD

M

SD

M

SD

Text-based questions Less-skilled More-skilled

0.18 0.35

0.22 0.27

0.21 0.41

0.23 0.29

0.54 0.55

0.29 0.19

0.44 0.68

0.22 0.25

Local-bridging questions Less-skilled More-skilled

0.14 0.34

0.19 0.28

0.10 0.33

0.20 0.30

0.32 0.39

0.28 0.28

0.32 0.43

0.29 0.25

Global-bridging questions Less-skilled More-skilled

0.08 0.18

0.10 0.20

0.08 0.19

0.11 0.16

0.31 0.41

0.22 0.27

0.30 0.36

0.25 0.25

Author's personal copy

Y. Ozuru et al. / Learning and Instruction 19 (2009) 228e242

235

was also two-way interactions between reading skill and text cohesion, F(1, 162) ¼ 4.17, p < 0.05, partial h2 ¼ 0.03, between type of question and knowledge level, F(2, 324) ¼ 7.52, p < 0.01, partial h2 ¼ 0.04, and a three-way interaction between text cohesion, reading skill, and type of question, F(2, 324) ¼ 3.35, p < 0.05, partial h2 ¼ 0.02. The results indicated that, first and foremost, comprehension performance was affected by the type of question and individual difference factors, namely knowledge level and reading skill. Text cohesion had no main effect, but interacted with reading skill and type of question. To better understand these interactions, follow-up analyses were conducted focusing on two aspects of the findings specifically related to our research questions: (a) the relative contribution of prior knowledge, which differed in the two groups of participants representing the two knowledge levels, and of reading skill to performance on the different types of questions irrespectively of text cohesion; (b) the effect of text cohesion and its interaction with individual difference factors. 3.2. Effects of prior knowledge and reading skill The contribution of both prior knowledge and reading skill to performance on comprehension questions is not surprising because reading skill correlated with both biology knowledge (r ¼ 0.48) and topic-specific knowledge (r ¼ 0.32). We investigated which of the two individual difference factors (biology/topic-specific knowledge or reading skill) primarily contributed to performance on the different types of comprehension questions using hierarchical regression analysis. We performed three separate hierarchical regression analyses using performance on text-based questions, local-bridging questions, and global-bridging questions across the two texts (i.e., low- and high-cohesion texts) as dependent variables. That is, the dependent variables were performance on three types of comprehension questions collapsed across low and high-cohesion conditions. In this way, the regression analysis examined the main effect of individual differences on performance for the different types of comprehension questions. Reading skill and the scores on the two types of prior knowledge measures (i.e., biology knowledge and topic-specific knowledge) were entered as predictor variables. In each of the three analyses, we examined two models. The first model examined the contribution of reading skill to performance on comprehension question by including only reading skill as a predictor variable. In the second model, we entered biology knowledge and topic-specific knowledge in the second block using the enter method. We were interested in the R2 change associated with the entry of the second group of predictor variables (i.e., biology knowledge and topic-specific knowledge) because the significance level of R2 change associated with the second group of predictor variable represents the extent to which prior knowledge contributes to the performance on comprehension question above and beyond the contribution of reading skill. We are also interested in beta weight of the three predictor variables in the second model because these beta weights reveal the relative contribution of the three predictor variables. Table 4 presents the results of the regression analyses. First, the regression analyses indicated that the two types of prior knowledge predicted performance on comprehension questions above and beyond reading skill across the three types of questions. More specifically, biology knowledge and topicspecific knowledge together accounted for about 20% of unique variance for performance on text-based questions (DR2 ¼ 0.20) Table 4 Regression analysis of the contribution of individual differences factors on performance on different types of comprehension questions. Variables

Performance on text-based questions Reading skill Biology knowledge Topic-specific knowledge R2 DR2 F for DR2 Performance on local-bridging questions Reading skill Biology knowledge Topic-specific knowledge R2 DR2 F for DR2 Performance on global-bridging questions Reading skill Biology knowledge Topic-specific knowledge R2 DR2 F for DR2

Model 1

Model 2

beta

t

sigt

beta

0.37

5.18

0.00

0.15 2.09 0.27 3.58 0.31 4.36 0.34 0.20 (2, 166) ¼ 26.30, p < 0.001

0.04 0.00 0.00

0.00

0.22 3.26 0.29 3.96 0.30 4.37 0.39 0.19 (2, 166) ¼ 28.80, p < 0.001

0.00 0.00 0.00

0.00

0.07 1.03 0.23 3.04 0.44 6.25 0.38 0.28 (2, 166) ¼ 37.70, p < 0.001

0.31 0.00 0.00

0.14 e (1, 168) ¼ 26.80, p < 0.001 0.44

6.44

0.20 e (1, 168) ¼ 41.40, p < 0.001 0.31

4.25

0.10 e (1, 168) ¼ 18.10, p < 0.001

t

sigt

Author's personal copy

236

Y. Ozuru et al. / Learning and Instruction 19 (2009) 228e242

and local-bridging questions (DR2 ¼ 0.19) and 28% (DR2 ¼ 0.28) of unique variance for global-bridging questions above and beyond reading skill. Relatively larger contribution of prior knowledge compared to reading skill was also apparent in the beta weights. Across the three types of questions, beta weights of biology knowledge and topic-specific knowledge were larger than the beta weight of reading skill. Second, the beta weights also indicated that the relative contribution of reading skill and prior knowledge, in particular topicspecific knowledge, changed depending on the type of question. First, the beta weight of reading skill for text-based and localbridging questions was 0.15 and 0.22, respectively, and they were both statistically significant. For global-bridging questions, however, the beta weight for reading skill was much smaller, at 0.07 and nonsignificant. Turning to the contribution of topic-specific knowledge, on the other hand, the beta weight of topic-specific knowledge for the global-bridging question was notably larger, 0.44, compared to the beta weight of topic-specific knowledge for text-based questions, and local-bridging questions, 0.31 and 0.30, respectively. These results indicated an increase in the contribution of prior knowledge and a decrease in the contribution of reading skill when questions demanded more extensive information integration. Together these findings support both Hypothesis 1 and Hypothesis 2, though the evidence in support of Hypothesis 2 is tentative because the contribution of prior knowledge was somewhat similar between text-based and local-bridging questions. 3.3. Effect of text cohesion and its interaction with individual differences This section describes a series of analyses performed to explore the second research question: how reading skill and prior knowledge influences the benefit from text cohesion, and what is the level of comprehension at which the effect of text cohesion is observed. The overall analysis based on the ANOVA indicated that there was no main effect of text cohesion; instead, there was a three-way interaction between text cohesion, reading skill, and type of question, indicating that the interactive effect of text cohesion and reading skill was limited to performance on a specific type of the questions. It is apparent from Table 3 that the effect of text cohesion (i.e., interaction between text cohesion, reading skill, knowledge level) was specific to text-based questions; no clear difference in performance between low- and high-cohesion text was present for local-bridging, F(1, 166) < 1.0, p ¼ 0.9, and global-bridging questions, F(1, 166) < 1.0, p ¼ 0.5. To discern more precisely the nature of the interaction between text cohesion, reading skill, and knowledge level on performance on text-based questions, we performed a 2 (text cohesion)  2 (reading skill)  2 (knowledge level) ANOVA on performance on text-based questions. The analysis showed, in addition to a main effect of knowledge level and reading skill, which has been reported earlier, a significant two-way interaction between reading skill and text cohesion, F(1, 166) ¼ 8.82, p < 0.01, partial h2 ¼ 0.05, and three-way interaction between text cohesion, reading skill, and knowledge level, F(1, 166) ¼ 4.90, p < 0.05, partial h2 ¼ 0.03. The two-way interaction between knowledge level and text cohesion was nonsignificant, hence, failing to confirm Hypothesis 3. A follow-up of these interactions indicated that among the less-skilled readers, text cohesion had no effect for low-knowledge readers (low-cohesion text, M ¼ 0.18, SD ¼ 0.22; high-cohesion text, M ¼ 0.21, SD ¼ 0.23), t(45) ¼ 0.79, p > 0.05, and produced a negative effect for high-knowledge readers (low-cohesion text, M ¼ 0.50, SD ¼ 0.31; high-cohesion text, M ¼ 0.41, SD ¼ 0.23), t(38) ¼ 2.44, p < 0.05, Cohen’s d ¼ 0.33. On the other hand, more-skilled readers tended to benefit from reading a high-cohesion text whether they had a high level of knowledge (low-cohesion text, M ¼ 0.55, SD ¼ 0.19; high-cohesion text, M ¼ 0.68, SD ¼ 0.25), t(22) ¼ 2.461, p < 0.05, Cohen’s d ¼ 0.59, or a low level of knowledge (low-cohesion text, M ¼ 0.35, SD ¼ 0.27; high-cohesion text, M ¼ 0.42, SD ¼ 0.29), although the effect of text cohesion was not significant for low-knowledge moreskilled readers, t(61) ¼ 1.72, p ¼ 0.09. Thus, it appears that reading skill is critical for benefiting from increased text cohesion in general but especially among high-knowledge readers. This finding replicates what O’Reilly and McNamara (2007) found and confirms Hypothesis 4. To complement the above analysis, we also performed hierarchical regression analysis using prior knowledge and reading skill as predictors. In these regression analyses, the criterion variable representing the effect of text cohesion was created by calculating the difference score of performance on the text-based question between low- and high-cohesion texts. In this way, the regressions examined whether topic-specific knowledge, biology knowledge, and/or reading skill predicted the extent to which participants benefited from reading high-cohesion text. Two models were examined. In Model 1, only prior knowledge (i.e., biology knowledge and topic-specific knowledge) was entered. In Model 2, reading skill was entered in the model in the second block using the enter method. Comparison of the two models (i.e., R2 change) and examination of the beta weights of the three predictor variables in Model 2 allow us to infer the relative contribution of reading skill and prior knowledge to the benefits of text cohesion. Three sets of hierarchical regressions were performed: (a) for all participants, (b) low-knowledge participants (i.e., participants from the Psychology class), and (c) high-knowledge participants (i.e., participants from the Biology class). Table 5 presents the results of these analyses. The analysis with all participants indicated, consistent with the ANOVA, that reading skill was the primary factor driving the benefit of text cohesion as indicated by a small but significant R2 change associated with reading skill among overall participants (DR2 ¼ 0.02), even when reading skill was entered after prior knowledge. However, the subsequent analyses regarding low- and

Author's personal copy

Y. Ozuru et al. / Learning and Instruction 19 (2009) 228e242

237

Table 5 Regression analysis of the contribution of individual differences factors on the effect of text cohesion on performance on text-based questions. Variables

All participants Biology knowledge Topic-specific knowledge Reading skill R2 DR2 F for DR2 Low-knowledge participants Biology knowledge Topic-specific knowledge Reading skill R2 DR2 F for DR2 High-knowledge participants Biology knowledge Topic-specific knowledge Reading skill R2 DR2 F for DR2

Model 1

Model 2

beta

t

sigt

beta

0.11 0.05

1.22 0.63

0.22 0.53

0.04 0.46 0.08 0.88 0.17 1.96 0.03 0.02 (1, 166) ¼ 3.83, p ¼ 0.05

0.64 0.39 0.05

0.93 0.74

0.352 0.460

0.08 0.09 0.04 0.01 0.00 (1, 104) ¼ 0.10, ns

0.462 0.426 0.751

0.45 1.11

0.66 0.27

0.10 0.64 0.09 0.61 0.40 2.74 0.16 0.11 (1, 58) ¼ 7.48, p < 0.01

0.01 e (2, 167) ¼ 0.74, ns

0.10 0.08 0.01 e (2, 105) ¼ 0.54, ns

0.07 0.17 0.05 e (2, 59) ¼ 1.45, ns

t

0.74 0.80 0.32

sigt

0.53 0.55 0.01

high-knowledge participants separately indicated that the contribution of reading skill to the benefit of text cohesion primarily occurred for high-knowledge participants. Specifically, among the high-knowledge readers, the benefit of text cohesion was quite strongly influenced by reading skill as indicated by a significant R2 change, DR2 ¼ 0.11, associated with reading skill above and beyond the contribution of the two types of prior knowledge. This means that over 10% of the variance associated with the benefit of text cohesion was uniquely explained by reading skill among high-knowledge readers. On the other hand, reading skill explained almost no unique variance of the benefit of text cohesion (DR2 < 0.01) for low-knowledge participants. Hence, the analyses, both ANOVAs and multiple regressions, collectively indicated that (a) the benefit of text cohesion was limited to performance on text-based questions; (b) the benefit of text cohesion depends on reading skill, such that more-skilled readers tend to have larger benefit of text cohesion. However, this two-way interaction between text cohesion and reading skill primarily occurs for high-knowledge readers in that high-knowledge less-skilled readers performed more poorly when reading a high-cohesion text than when reading a low-cohesion text. These findings are generally in line with our predictions on the second research question (i.e., Hypotheses 3, 4, and 5) with one exception. That is, we did not observe a two-way interaction between level of knowledge and text cohesion such that benefit of text cohesion is larger for low-knowledge readers. Instead, the effect of text cohesion mainly depended on reading skill. 4. Discussion In what follows, we discuss the findings by evaluating the results in terms of the two research questions we set out to explore. 4.1. Reading skill and prior knowledge effects The results indicated that science-text comprehension, as measured by performance on comprehension questions, was affected by both reading skill and prior knowledge. However, the regression analyses indicated that prior knowledge is a more significant predictor of text comprehension than reading skill. Prior knowledge explained a significant amount of variance of performance on comprehension questions above and beyond reading skill, and the beta weight of reading skill was notably smaller than the beta weights of biology knowledge and topic-specific knowledge. In addition, the effect of prior knowledge tends to be larger on global-bridging questions that require more extensive integration of the information. In contrast, the contribution of reading skill, though still small compared to prior knowledge, was larger for text-based and local-bridging questions. Together these results indicate that answering questions that involve integration of multiple sentence meanings is primarily determined by the knowledge the participants possessed prior to reading the text. This

Author's personal copy

238

Y. Ozuru et al. / Learning and Instruction 19 (2009) 228e242

finding is generally consistent with Hypotheses 1 and 2, which predicted that readers’ comprehension involving the relations between multiple related concepts in science text is primarily determined by the pre-existing knowledge level. One limitation of the finding is that we did not find a clear linear increase in the contribution of prior knowledge to performance on comprehension questions from text-based, to local-bridging, and to global-bridging question. This may be due to the still under-defined nature of the variable ‘‘type of question’’, as well as to the small number of questions in our sample. Future studies should replicate this finding using a different and larger set of questions. There is also one unresolved theoretical issue about this finding; that is, there are two alternative interpretations to this finding. On the one hand, performance on bridging inference questions is theoretically expected to be influenced by prior knowledge (Kintsch, 1998, 1999) because having knowledge helps readers make inferences. However, large effects of prior knowledge may also imply that participants may not have learned a great deal of new information by reading the passage. Which of these two accounts better explains the more pronounced effect of prior knowledge on performance on bridging inference questions cannot be answered here. Another important issue to be noted is that the present study required participants to answer questions from memory. This technique contrasts with typical reading skill assessment (i.e., the Nelson-Denny test) in which participants are allowed to inspect the text in the presence of the questions. Thus, whereas the Nelson-Denny measures one’s ability to locate information and reason with it, comprehension questions in this study required participants to remember what they read as well as reason with it. As a consequence, the memory-based comprehension questions in this study may have influenced the results by increasing the contribution of prior knowledge (Ozuru, Best, Bell, Witherspoon, & McNamara, 2007). 4.2. The interaction of prior knowledge and reading skill with text cohesion The other key issue this study attempted to address is precisely how two individual difference factors (i.e., reading skill and prior knowledge) moderate the effect of text cohesion in science-text comprehension. The overall findings indeed indicate that the effect of text cohesion depends on both reading skill and prior knowledge. First, the effect of text cohesion on science text comprehension depends on the participants’ reading skill. The presence of a two-way interaction between reading skill and text cohesion without the presence of a two-way interaction between knowledge level and text cohesion in the analysis based on overall data indicates that the effect of text cohesion is moderated first by reading skill even though the effect size was very small, partial h2 ¼ 0.03. This contribution of reading skill on the effect of text cohesion was also confirmed in the hierarchical regression analyses. However, the way in which reading skill moderates the effect of text cohesion also depends on the readers’ knowledge level (i.e., a three-way interaction). That is, the two-way interaction between reading skill and text cohesion on text-based questions was primarily due to the phenomenon that poor reading skill combined with high knowledge can produce a reversed cohesion effect among this group of readers (less-skilled and high-knowledge readers). Specifically, among less-skilled and high-knowledge readers, performance on the text-based questions was worse when reading a high-cohesion text as opposed to a low-cohesion text. This finding replicates the O’Reilly and McNamara (2007) results, and suggests that a high level of knowledge combined with a high-cohesion text tends to lead readers to process the text more shallowly when the readers do not have a certain level of reading skill. Also, the finding is generally in line with the expertise reversal effect based on CLT framework (Kalyuga et al., 2003). With respect to the issue of whether reading skill influences the benefit of cohesion irrespectively of the level of knowledge (i.e., as found by Voss & Silfies, 1996), the present study provided only partial support. According to the analysis using median split based on reading skill, among low-knowledge readers, a marginal benefit of text cohesion was observed only for more-skilled readers. Specifically, the benefit of text cohesion for text-based questions was marginally significant for more-skilled readers (lowcohesion text, M ¼ 0.35; high-cohesion text, M ¼ 0.42), while there was no benefit of text cohesion among less-skilled readers (low-cohesion text, M ¼ 0.18; high-cohesion text, M ¼ 0.21). However, regression analyses examining the effect of text cohesion on performance on text-based questions among low-knowledge readers (see Table 3) failed to show interactive effects of reading skill and text cohesion. Thus, the question on whether reading skill contributes to the benefit of text cohesion even among lowknowledge readers is somewhat inconclusive. With respect to the question regarding potential interactions of question type and text cohesion, the results indicated that both beneficial and detrimental effects of text cohesion were limited to performance on text-based questions assessing readers’ memory for information explicitly stated in individual sentences. This finding, in particular the positive effect of text cohesion among lowknowledge readers for text-based questions, supports our predictions. Nonetheless, we evaluate this finding in more detail because it is important to understand precisely how reading a high-cohesion text leads to better answering performance on text-based questions but not on bridging questions. Strictly speaking, this finding may not completely rule out the possibility that increased text cohesion might have some influence, albeit undetected influence, on global level comprehension. The process of answering text-based questions without the text is likely to be affected by higher levels of comprehension (i.e., the situation model) of the text. This is because recall of textbased information within individual sentences is likely to be facilitated by more global comprehension of texts (Bransford & Johnson, 1972).

Author's personal copy

Y. Ozuru et al. / Learning and Instruction 19 (2009) 228e242

239

However, if high-cohesion texts had facilitated inferences based on textual information (i.e., bridging inferences) and enhanced global comprehension, we would likely observe a larger contribution of reading skill on inference-based questions in the high as opposed to the low-cohesion text condition. This follows from the assumption that the ability to make text-based inferences is related to reading skill (Magliano & Millis, 2003). However, there was no sign of this effect in the current findings; the correlation between reading skill and performance on local-bridging questions slightly decreased for the high-cohesion (r ¼ 0.34) as compared to the low-cohesion (r ¼ 0.44) text, and the correlation between reading skill and performance on global-bridging questions did not change between the high-cohesion (r ¼ 0.27) and low-cohesion (r ¼ 0.25) conditions. Hence, we believe that the observed benefit of increased cohesion on text-based questions was not caused by facilitation of inferential processes from increased cohesion. Further, a close examination of the text revisions reveals that the information necessary to answer the text-based questions is mentioned more frequently in the high-cohesion texts than in the low-cohesion texts. Whereas the target information to answer text-based questions appears an average of 1.4 times in the low-cohesion texts, the same or similar information appears an average of two times in the high-cohesion texts. Thus, a likely explanation of the improved text-based question answering performance in the high-cohesion condition is the more frequent exposure to the critical information in multiple sentence contexts, which likely aids readers’ retention of the text-based information in the high-cohesion condition. This finding that the benefit of text cohesion was limited to text-based question answering (see also, McNamara, 2001; O’Reilly & McNamara, 2007) is in contrast with the Gilabert, Martı´nez, and Vidal-Abarca (2005) work that reported the positive effect of a causal cohesion manipulation on inference-based question answering. Two differences in our experiments may be responsible for this difference. First, they used a history text. As discussed earlier, readers of a history text, as opposed to a biology text, are more likely to engage in inferential processes that lead to deeper level of comprehension because knowledge of everyday experiences (e.g., conflict, desire to power) may facilitate comprehension of history texts. Second, because participants answered comprehension question by referring to the source text in their study, their results may reflect facilitation of inferential processing not at the time of originally reading the text, but at the time of answering questions after being exposed to the comprehension questions. 4.3. Conclusion We would like to articulate the three main contributions of the current study on research of text comprehension and learning. First, the present study with science (biology) texts suggests that the benefit of increasing cohesion is limited to comprehension and learning of text-based information. There was no evidence in the present study that reading high-cohesion texts facilitates more inferences based on textual information which later help readers answer bridging questions. Perhaps, the failure to observe the benefit of increased cohesion on bridging questions is related to the specific nature of science texts. When reading science texts, readers often do not have a sufficiently developed mental model that represents the overall conceptual relations between the relevant concepts (e.g., relations between hormones and tropism). This contrasts with the case of social studies or history texts for which the reader is assumed to have some general schema about the event structure. For example, most readers know that stories about war involve multiple countries, a cause of the conflict, and a specific location where the war is fought. This type of general event schema may help readers integrate specific attributes of the event described in the story, and hence, facilitate inferences (Anderson, 1978). Thus, the challenge in reading a social studies or history text is to understand the relations between unfamiliar attributes (specific location, person) of a historic event (e.g., Russian revolution) using general event knowledge (e.g., about revolution). In contrast, when reading a science text, most readers may not have such readily available knowledge about general events related to the topic (e.g., heat distribution in animals) unless the reader is a domain expert (e.g., biology teachers). In addition, readers often have difficulty understanding the meaning of the individual scientific concepts (e.g., endotherm, tropism, etc.) because they are very abstract and difficult to ground based on everyday experiences (Graesser et al., 2002). Thus, readers’ efforts are likely to be focused on comprehension of unfamiliar concepts at a very local level (i.e., individual sentences) even with added scaffolding provided by increasing text cohesion. The second contribution is on the role of reading skill. Reading skill is necessary to effectively take advantage of scaffolding provided by increased cohesion. This finding has an important educational implication. In many educational settings, textbooks are read for the purpose of learning new information. Thus, in these conditions, many readers are faced with reading texts on unfamiliar topics (e.g., chemistry, biology). The results of this study indicate that students’ difficulty in learning new concepts can be alleviated to some extent by making text more cohesive which makes readers less dependent on pre-existing knowledge. Yet, this study shows that readers are not able to take advantage of increased cohesion unless they have sufficient reading skill. In this sense, it is crucial to work on improving students’ comprehension and learning from text from two fronts. One is improving text quality (Beck et al., 1991; Graesser et al., 2003) and the other is improving students’ reading skill. In this sense, this study emphasizes not only the importance of text features on comprehension of science text, but also the importance of providing adequate training on reading strategies (McNamara, O’Reilly, Best, & Ozuru, 2006) because the ability to use reading strategies is likely to be a key element of reading skill (Magliano & Millis, 2003).

Author's personal copy

Y. Ozuru et al. / Learning and Instruction 19 (2009) 228e242

240

The third and final contribution of the present study is toward our theoretical understanding of reading comprehension for science texts. The present study indicated that comprehension and learning from science texts involves both rapid activation of pre-existing topic-related knowledge and the deliberate use of certain skills with which readers try to relate various textual information using limited cognitive resources. As such, the study highlights the distinct role played by two types of individual differences in science text comprehension. Acknowledgements This research was supported by the Institute for Education Sciences (IES R3056020018-02).

Appendix A. Example texts A1. Excerpt of the text ‘‘Heat Distribution in Animals’’ (low cohesion) The circulatory system is responsible for the distribution of heat throughout the body. This is true for both warm-blooded and cold-blooded animals. The term ‘‘warm blooded’’ is applied to birds and mammals in recognition that they can, and usually do, keep their body temperature higher than that of their surroundings. But this is not always the case; some of them allow their temperature to drop close to the ambient temperature, when they hibernate, for example. And some of them, mammals in the tropical savannah, for exampledhave to keep their body temperature below the scorching temperatures of the surroundings. However, there are two features that set birds and mammals apart from most of the rest of the animal kingdom: They maintain their body temperature within narrow limits no matter what the ambient temperature. For this reason, they are often described as being homeothermic. They are endothermic; the heat with which they maintain their body temperature is generated within the body. Some coldblooded animals, e.g., lizards basking in the sun, develop body temperatures as high as that of birds, but they are ectothermic; they secure the heat for doing so externally. A2. Excerpt of the text ‘‘Heat Distribution in Animals’’ (high cohesion) The circulatory system distributes heat through the blood vessels of an animal’s body. This system is responsible for the transport of heat for both warm-blooded animals and cold-blooded animals. Warm-blooded animals include birds and mammals, whereas cold-blooded animals include reptiles, amphibians, and fish. The term ‘‘warm blooded’’ is applied to birds and mammals because they can, and usually do, keep their body temperature higher than that of their surroundings. But this is not always the case because some warm-blooded animals allow their body temperature to drop close to the temperature of the air around them, for example, when they hibernate through the winter. Mammals who live in the heat of the tropical savannah are another example of warm-blooded animals that do not always keep their body temperature higher than the surrounding temperature. These animals often have to keep their body temperature below the scorching temperatures of their surroundings. Nonetheless, there are two features that set warm-blooded animals apart from most of the rest of the animal kingdom: 1. Warm-blooded animals are homeothermic. That is, unlike other animals, birds and mammals maintain their body temperature within narrow limits no matter what the surrounding (or ambient) temperature. 2. Warm-blooded animals are endothermic; that is, they maintain their body temperature with heat generated within their own body. Endothermic animals contrast with cold-blooded animals whose body temperature is maintained by heat from external sources. As such, even though some cold-blooded animals, such as lizards who bask in the sun, develop body temperatures as high as that of birds, these creatures secure their body heat externally. These kinds of animals are called ectothermic. Note: Underlined font represents sections added to increase local cohesion. Font in italics represents changes in sentence structure made to increase local cohesion. Bold font represents sections added to increase global cohesion.

Appendix B. Example comprehension questions (comprehension questions based on ‘‘Heat Distribution in Animals’’ text) B1. Text-based questions 1. What mechanism helps fish (e.g., tuna) keep their active muscles warmer than the water temperature? 2. How do ectothermic animals secure heat? 3. What is the major source of heat for resting endothermic animals?

Author's personal copy

Y. Ozuru et al. / Learning and Instruction 19 (2009) 228e242

241

4. What is one way an endothermic animal can voluntarily generate more heat to maintain their body temperature when their surroundings become colder?

B2. Local-bridging questions 1. 2. 3. 4.

What is a reason why a warm-blooded animal’s body temperature might drop below that of their surroundings? What does homeothermic mean in relation to animal’s body temperature? How might a rattle snake have a body temperature equivalent to that of a dog? Why do people lose heat from their hand and feet first?

B3. Global-bridging questions 1. Describe how the circulatory system regulates the distribution of heat in animals. 2. Why is efficient distribution of heat across the body important not only for warm-blooded animals but also for cold-blooded animals? 3. According to the text, how would an endotherm respond to an ambient temperature of 30 degrees Farenheit? 4. Describe how a countercurrent heat exchanger helps a mammal maintain its body temperature in cold surroundings? References Afflerbach, P. (1986). The influence of prior knowledge on expert readers’ importance assignment process. In J. A. Niles, & R. V. Lalik (Eds.), National reading conference yearbook. Solving problems in literacy: Learners, teachers and researchers, Vol. 35 (pp. 30e40). Rochester, New York: National Reading Conference. Anderson, R. C. (1978). Schema-directed processes in language comprehension. In A. Lesgold, J. Pellegrino, S. Forkkema, & R. Glaser (Eds.), Cognitive psychology and instruction (pp. 67e82). New York: Plenum. Beck, I. L., McKeown, M. G., Sinatra, G. M., & Loxterman, J. A. (1991). Revising social studies text from a text-processing perspective: evidence of improved comprehensibility. Reading Research Quarterly, 26, 251e276. Bransford, J. D., & Johnson, M. K. (1972). Contextual prerequisites for understanding: some investigations of comprehension and recall. Journal of Verbal Learning and Verbal Behavior, 11, 717e726. van den Broek, P. (1994). Comprehension and memory of narrative texts: inferences and coherence. In M. A. Gernsbacher (Ed.), Handbook of psycholinguistics (pp. 539e588). London: Academic. Brown, J., Fishco, V., & Hanna, G. (1993). Nelson-Denny reading test: Manual for scoring and interpretation, Forms G&H. Chicago: Riverside Press. Chi, M., Feltovich, P., & Glaser, R. (1981). Categorization and representation of physics problems by experts and novices. Cognitive Science, 5, 121e152. Daneman, M., & Hannon, B. (2001). Using working memory theory to investigate the construct validity of multiple choice reading comprehension tests such as the SAT. Journal of Experimental Psychology: General, 130, 208e223. Flesch, R. (1948). A new readability yardstick. Journal of Applied Psychology, 32, 221e233. Gernsbacher, M. A. (1997). Group differences in suppression skill. Aging, Neuropsychology, and Cognition, 4, 175e184. Gernsbacher, M. A., Varner, K. R., & Faust, M. (1990). Investigating differences in general comprehension skill. Journal of Experimental Psychology: Learning, Memory, and Cognition, 16, 430e445. Gilabert, R., Martı´nez, G., & Vidal-Abarca, E. (2005). Some good texts are always better: text revision to foster inferences of readers with high and low prior background knowledge. Learning and Instruction, 15, 45e68. Graesser, A., McNamara, D., Louwerse, M., & Cai, Z. (2004). Coh-Metrix: analysis of text on cohesion and language. Behavioral Research Methods, Instruments, and Computers, 36, 193e202. Graesser, A. C., Leon, J. A., & Otero, J. (2002). Introduction to the psychology of science text comprehension. In J. Otero, J. A. Leon, & A. C. Graesser (Eds.), The psychology of science text comprehension (pp. 1e15). Mahwah, NJ: Erlbaum. Graesser, A. C., McNamara, D. S., & Louwerse, M. M. (2003). What do readers need to learn in order to process coherence relations in narrative and expository text. In A. P. Sweet, & C. E. Snow (Eds.), Rethinking reading comprehension (pp. 82e98). New York: Guilford. Guthrie, J., & Wigfield, A. (1999). How motivation fits into a science of reading. Scientific Studies of Reading, 3, 199e205. Hacker, D. J. (1998). Self regulated comprehension during normal reading. In J. D. Hacker, J. Dunlosky, & A. C. Graesser (Eds.), Metacognition in educational theory and practice (pp. 165e191). Mahwah, NJ: Erlbaum. Halliday, M. A. K., & Hasan, R. (1976). Cohesion in English. London: Longman. Hannon, B., & Daneman, M. (2001). A new tool for measuring and understanding individual differences in the component processes of reading comprehension. Journal of Educational Psychology, 93, 103e128. Kalyuga, S., Ayres, P., Chandler, P., & Sweller, J. (2003). The expertise reversal effect. Educational Psychologist, 38, 23e31. Kintsch, W. (1988). The use of knowledge in discourse processing: a construction-integration model. Psychological Review, 95, 163e182. Kintsch, W. (1993). Information accretion and reduction in text processing: inferences. Discourse Processes, 16, 193e202. Kintsch, W. (1998). Comprehension: A paradigm for cognition. Cambridge, MA: Cambridge University Press. Landauer, T. K., & Dumais, S. T. (1997). A solution to Plato’s problem: the latent semantic analysis theory of the acquisition, induction, and representation of knowledge. Psychological Review, 104, 211e240. Linderholm, T., Everson, M. G., van den Broek, P., Mischinski, M., Crittenden, A., & Samuels, J. (2001). Effects of causal text revision on more or less-skilled readers’ comprehension of easy and difficult texts. Cognition and Instruction, 18, 525e556.

Author's personal copy

242

Y. Ozuru et al. / Learning and Instruction 19 (2009) 228e242

McKoon, G., & Ratcliff, R. (1992). Inference during reading. Psychological Review, 99, 440e466. McNamara, D. S. (2001). Reading both high-coherence and low-coherence texts: effects of text sequence and prior knowledge. Canadian Journal of Experimental Psychology, 55, 51e62. McNamara, D. S. (Ed.). (2007). Reading comprehension strategies: Theories, interventions, and technologies. Mahwah, NJ: Erlbaum. McNamara, D., Kintsch, E., Songer, N., & Kintsch, W. (1996). Are good texts always better? Interactions of text coherence, background knowledge, and levels of understanding in learning from text. Cognition and Instruction, 14, 1e43. McNamara, D. S., & Kintsch, W. (1996). Learning from text: effects of prior knowledge and text coherence. Discourse Processes, 22, 247e287. McNamara, D. S., & McDaniel, M. (2004). Suppressing irrelevant information: knowledge activation or inhibition? Journal of Experimental Psychology: Learning, Memory, & Cognition, 30, 465e482. McNamara, D. S., O’Reilly, T. P., Best, R. M., & Ozuru, Y. (2006). Improving adolescent students’ reading comprehension with iSTART. Journal of Educational Computing Research, 34, 147e171. Magliano, J. P., & Millis, K. K. (2003). Assessing reading skill with a think-aloud procedure. Cognition and Instruction, 21, 251e283. Noordman, L. G., Vonk, W., & Kempff, H. J. (1992). Causal inferences during the reading of expository texts. Journal of Memory and Language, 31, 573e590. Oakhill, J., & Yuill, N. (1996). Higher order factors in comprehension disability: processes and remediation. In C. Cornaldi, & J. Oakhill (Eds.), Reading comprehension difficulties: Processes and intervention (pp. 69e92). Mahwah, NJ: Erlbaum. Oakhill, J. V., Cain, K., & Bryant, P. E. (2003). Dissociation of single-word reading and text comprehension skills. Language and Cognitive Processes, 18(4), 443e468. O’Reilly, T., & McNamara, D. S. (2007). Reversing the reverse cohesion effect: good texts can be better for strategic, high-knowledge readers. Discourse Processes, 43, 121e152. Ozuru, Y., Best, R., Bell, C., Witherspoon, A., & McNamara, D. S. (2007). Influence of question format and text availability on assessment of expository texts comprehension. Cognition & Instruction, 25, 399e438. Perfetti, C. A. (1985). Reading ability. New York: Oxford University Press. Shapiro, A. (2004). How including prior knowledge as a subject variable may change outcomes of learning research. American Educational Research Journal, 41, 159e189. Sweller, J. (1999). Instructional design in technical areas. Camberwell, Victoria, Australia: Australian Council for Educational Research. Voss, J., & Silfies, L. (1996). Learning from history text: the interaction of knowledge and comprehension skill with text structure. Cognition and Instruction, 14, 45e68. Walker, C. H. (1987). Relative importance of domain knowledge and overall aptitude on acquisition of domain related information. Cognition and Instruction, 4, 25e42. Zwaan, R. A., & Singer, M. (2003). Text comprehension. In A. C. Graesser, M. A. Gernsbacher, & S. R. Goldman (Eds.), Handbook of discourse processes (pp. 83e121). Mahwah, NJ: Erlbaum.