Session F3G
A Comparative Study of Students’ Conceptual Database Frameworks across Universities Nicole Anderson Winona State University,
[email protected] Kirby McMaster St. Mary’s College of Maryland,
[email protected] Samuel Sambasivam Azusa Pacific University,
[email protected] Abstract - The purpose of this research is to examine the mental frameworks used by computer science students after their first semester of study of database systems. We explore the commonalities and differences in these frameworks across different universities and instructors. In addition, our research analyzes whether the students are able to consistently group important concepts within their frameworks. The questions this research aimed to answer include the following: As perceived by the students, what are the most important and least important concepts in the introductory DB course? How much do these perceptions vary among students, instructors, and institutions? Can we create a profile of important concepts for each course? And finally, how do students organize database concepts into a unified framework? We discovered that some database topics were universally considered important, while perceptions varied considerably for other concepts. We believe that other instructors may be able to use this method to identify and evaluate the mental frameworks they are presenting to their students. Index Terms – computer science education, database framework, gestalt, knowledge framework. INTRODUCTION
The questions this research aims to answer include the following: As perceived by students, what are the most important and least important concepts in the introductory DB course? How much do these perceptions vary among students, instructors, and institutions? Can we create a profile of important concepts for each course? And finally, how do students organize database concepts into a unified framework? Our prior research included the development of the DGestalt scale to analyze the mental framework, or gestalt, used by database textbook authors. We also performed an initial study measuring the mental frameworks of database students at a single university. Many questions were posed in response to this research, including whether this framework would be common on a broader scale, and if students would be able to meaningfully classify or explain those concepts they deemed important. Thus, this research builds upon the prior research to further analyze the frameworks used by computer science students at the conclusion of their first semester of study of database concepts. Related research includes the development of knowledge frameworks for other areas such as objectoriented development [1]. Also related is the effort to develop tools to support the knowledge acquisition process for database systems and related subjects [2][3][4]. In addition, work has been done to measure student mental frameworks for math and other computer science areas [5,6]. Student knowledge assessment is yet another related area of research [7-9]. Here, rather than focusing on whether the student has learned the material in the way the instructor feels is most appropriate, we attempt to assess what the students believe to be the most important concepts. We further wish to determine whether this is consistent with different variations of the materials as presented by unique instructors at diverse institutions. The remainder of this paper is organized as follows. First, the methodology used to gather data from the students on their conceptual frameworks and conceptual classifications is explained. Then, the analysis of the data
In computer science courses, the construction of a mental framework to integrate course concepts is critical to each student's success in understanding course materials. Understanding both what is important as well as the relationships between key concepts is essential to building upon current knowledge with future learning. The purpose of this research is to examine the mental frameworks used by computer science students after their first semester of study of database systems, and exploring the commonalities and differences in these frameworks across different universities and instructors. In addition, this research analyzes whether the students are able to competently group important concepts within their framework. 978-1-4244-6262-9/10/$26.00 ©2010 IEEE October 27 - 30, 2010, Washington, DC 40th ASEE/IEEE Frontiers in Education Conference F3G-1
Session F3G
This section reviews the methodology for our study. At three different regional universities, a survey was given to students at the end of their first semester of study of database systems. The students in each course had a different instructor, each used a different textbook, and no coordination was performed between instructors on course materials. The data collection consisted of two parts. First, students were asked to rate the importance of 36 database concepts on a ten-point scale, from 1 being “least important” to 10 being “most important.” Many of the concepts came from our prior research, including the development of a database concepts (DGestalt) scale [6] and the development of a prior questionnaire [10] used to compare student conceptual frameworks to those used by textbook authors. The prior questionnaire was developed using concepts from that scale as well as topics identified as important by database instructors. The current questionnaire was augmented and modified slightly after receiving feedback from additional introductory database course instructors and from the research community. The resulting list was randomized so there was no level of importance implied by the order of the terms. In addition, the questionnaire asks students to categorize the 36 database concepts into 5 groups, which represent levels of a computer system. They stated whether each concept was related to an application program, DBMS software, a database, an operating system, or computer hardware. From the sample data, we calculated the average perceived importance of each concept for the group of students making up a particular course, as well as the combined averages based on all responses of all students in the three course sections. The results were compared to see how the students’ frameworks vary and what all students consistently deem as important. We also examined if the importance of each concept was related to how well students were able to classify the concept in their framework. RESULTS
DGestalt Scale Word/Group table/relation data database query/SQL relationship/entity object/new key/primary/foreign attribute/column system/subsystem record/row/tuple user/client/customer model value/variable type/data type TOTAL
Avg Books Std Freq 25 453.8 25 442.3 25 343.7 24 273.9 17 260.9 23 241.2 20 237.2 23 228.6 24 199.7 22 194.0 23 194.0 21 189.2 25 172.6 25 150.6
Weight 16.2 15.7 11.2 8.0 7.4 6.5 6.3 5.9 4.6 4.3 4.3 4.1 3.3 2.3 100.0
Not all of the textbook concepts appear as top student concepts. Here, we present this primarily to show the jumping-off point for creating our list of potentially important student concepts. Next, we take a look at the students’ conceptual frameworks in more detail. You can see some of the differences between course sections by comparing this word list with the student results presented in Figure 1, and in more detail in Table II. All three sections were fairly consistent for high and low rated concepts, but Section A rated the mid-range concepts lower than the other two sections. 10
9
Mean Rating
METHODOLOGY
TABLE I WORD RATING FOR DGESTALT SCALE
8
7
6 SQ L qu er y d a ke ta y ba s D e BM re S la tio n ta bl e re la ent i tio t y ns h no a tt ip rm r i b a l ut e iza tio n d sc at a he m re a da co ta rd in de int e p e g ri nd ty en ce
we collected is presented. Finally, we summarize our results, comparing commonalities and differences across universities both in terms of identified important concepts and conceptual classifications. Last, we take a look at potential future research in this and related areas.
Database Concepts First, we examined the concepts as classified by the Section A Section B Section C Average DGestalt scale, a scale developed in prior research to analyze the mental framework of textbook authors. We see in Table I the most frequently mentioned concepts in FIGURE 1 RATINGS OF TOP 16 COURSE CONCEPTS BY 3 DATABASE SECTIONS common database textbooks. Concepts with low scores are not shown in this chart. Many of the concepts in our survey Figure 1 and Table II compare the average perceived came from this list. In our previous research we presented a importance of each concept for the three course sections. As more detailed analysis on the difference between the is clear from this figure and table, students collectively conceptual frameworks constructed by students compared to agreed on the most and least important concepts. Concepts that of the textbooks. 978-1-4244-6262-9/10/$26.00 ©2010 IEEE October 27 - 30, 2010, Washington, DC 40th ASEE/IEEE Frontiers in Education Conference F3G-2
Session F3G consistently identified with scores over 9 by all three sections include SQL, query, and database. The three groups also rated key, DBMS, relation, and table at 8 or above. At the other end of the scale, we see file, stored procedure, cursor, and trigger as consistently receiving low scores. These are concepts that are usually not covered in depth in an introductory database course.
instructors. This is evident when viewing the correlation matrix presented in Table III. Each of the three sections had a correlation over 0.800 with the overall average. The lowest correlation between pairs of sections is 0.587. TABLE III CORRELATIONS BETWEEN COURSE SECTIONS
Correlation Matrix Section B Section A 0.697 Section B Section C
TABLE II CONCEPT RATINGS BY COURSE SECTION
database relation table relationship attribute
DBMS
60-70% 50-60%
A few other concepts were not consistently rated. Some dramatic examples of discrepancy between sections include the perceived importance of normalization, schema, and integrity. This is clearly shown in Figure 1 and Table II. On a concept-by-concept basis, there is a noticeable difference in perceived importance. However, when we examine the data as a whole, we see that there is a high degree of correlation between course sections, even at different universities with unique
Average 0.835 0.962 0.917
Next, we look at the data gathered from the second portion of the survey. For this portion, students were asked to classify the concepts into five possible categories representing levels in a computer system.
70 – 80%
Avg-F09 9.79 9.59 9.48 9.47 9.19 8.98 8.76 8.65 8.59 8.58 8.47 8.44 8.10 7.92 7.90 7.71 7.46 7.25 7.22 7.14 7.14 7.12 7.10 6.77 6.74 6.73 6.68 6.66 6.56 6.46 6.02 5.63 5.27 4.83 3.70 3.17
80 +
Topic/Concept SQL query key database DBMS relation table entity relationship attribute normalization data schema record integrity data independence data type view metadata model object information variable security transaction system relational algebra concurrency user domain commit index file stored procedure cursor trigger
N=8 N=8 N = 16 Section A Section B Section C Trim Mean Trim Mean Trim Mean 9.50 10.00 9.86 9.50 9.83 9.43 8.83 9.67 9.93 9.00 9.83 9.57 8.50 10.00 9.07 8.17 9.50 9.29 8.33 9.17 8.79 7.83 9.83 8.29 7.33 9.50 8.93 7.17 9.50 9.07 6.83 9.00 9.57 8.33 8.83 8.14 6.50 9.00 8.79 7.83 8.50 7.43 6.50 7.50 9.71 7.67 7.83 7.64 7.33 7.33 7.71 8.33 6.00 7.43 8.00 7.67 6.00 7.00 7.00 7.43 7.00 7.50 6.93 6.50 7.50 7.36 7.33 7.17 6.79 5.33 7.33 7.64 3.67 8.33 8.21 6.33 7.00 6.86 7.00 6.33 6.71 4.83 7.00 8.14 6.50 6.17 7.00 6.67 6.00 6.71 3.50 6.50 8.07 5.00 5.83 6.07 5.67 5.00 5.14 4.17 5.17 5.14 4.00 3.17 3.93 4.33 1.67 3.50
Section C 0.587 0.901
SQL
entity normalization key data schema model
user
query view
record integrity data independence data type object information domain
C1
C2
C3
system
C4
TABLE IV CONCEPT CLASSIFICATIONS
Tables IV and V show the results of the classification. In Table IV above, C1, C2, C3, and C4 represent categories for Application, DBMS Software, Database, and Operating System, respectively. The percentage on the left represents the percentage of students that classified that concept in the given group. Only concepts for which the majority of students selected the same category are included in this table. Note that C5 representing the Hardware category is not included, as none of the concepts were placed in this category by a majority of the students.
978-1-4244-6262-9/10/$26.00 ©2010 IEEE October 27 - 30, 2010, Washington, DC 40th ASEE/IEEE Frontiers in Education Conference F3G-3
Session F3G Table V below shows the full percentage summary for student classification of the concepts. TABLE V CONCEPT CLASSIFICATIONS CATEGORIES: 1=APPLICATION, 2=DBMS SOFTWARE, 4=OPERATING SYSTEM, 5=HARDWARE
3=DATABASE,
better classify concepts they deem as important. We believe this is because they have integrated these concepts into their mental framework. A scatter diagram of Mean Rating vs. Max Percent is shown below as Figure 2. This diagram displays the positive relationship between these two measures for the 36 concepts. The correlation coefficient is 0.737. This demonstrates a strong relationship between level of importance and consistent classification for concepts.
FIGURE 2 MEAN RATING OF CONCEPTS VS. CLASSIFICATION MAX PERCENT
SUMMARY AND CONCLUSIONS
It would certainly be possible to analyze as instructors how well we believe the students classified these concepts, but what we focus on for the purposes of this research is how consistently students classify concepts. Let’s re-examine those concepts that students universally identified as important. For the seven concepts which students in all three sections rated as 8 or above in importance, a majority (in several cases over 90%) of the students classified that concept in the same category. For the concepts deemed least important by all, scoring under a 6 by all three sections, students could not agree on how to classify the concepts. For two of three ranked lowest, less than 30% of the students classified the concepts in the same category. For the third only 46% put them into the same category. The result seems to indicate that students can
The goal of this work was to examine whether the conceptual framework of students at the end of their first semester of database study was consistent across multiple database course sections with different instructors at different universities. In addition, we wondered how well and how consistently students would group concepts in their mental frameworks. What we discovered was that some database topics were universally considered important, while perceptions varied considerably for other concepts. Also, we found a correlation between how important students felt a concept was and whether the students would classify it consistently. This leads us as instructors to reflect upon whether it is normal for the importance and classification of these concepts to vary among students. Do we need to do a better job emphasizing what we think are the important concepts, or should we teach students how to build their conceptual frameworks? Do we need to emphasize the logical connection between concepts in order to help them classify related concepts together? We believe that other instructors may be able to use the method presented here to identify and evaluate the mental frameworks they are presenting to their students and to reflect on their own teaching.
978-1-4244-6262-9/10/$26.00 ©2010 IEEE October 27 - 30, 2010, Washington, DC 40th ASEE/IEEE Frontiers in Education Conference F3G-4
Session F3G FUTURE WORK
In this study, we compared the database frameworks of students at the end of their first semester of database study and found a lot of commonalities. One current trend is toward the development of online and hybrid coursework. It is not yet fully understood whether a similar learning experience can be had in this medium as in a classroom. Huge efforts are being placed into trying to make the online setting an effective environment for learning. For example, tutoring systems are being developed to aid student learning through online feedback, guidance, and assessment [4]. It would be useful to give this survey to several online courses and see how the results compare, both in terms of important concepts and the ability of the students to correctly classify these concepts. ACKNOWLEDGMENT Our thanks to the Computer Science students of Azusa Pacific University, St. Mary’s College of Maryland, and Winona State University for participating in our surveys. REFERENCES [1]
[2]
1-Forde, B. W. R.; Russell, A. D.; Stiemer, S. F. (1989) ObjectOriented Knowledge Frameworks. Engineering with Computers, 5, 79-89. 2- Rook, F., and J. Croghan, “The Knowledge Acquisition Activity Matrix: A Systems Engineering Conceptual Framework,” IEEE Transactions on System, Man, and Cybernetics, 19, 3, (May/June 1989), pp. 586-597.
[3]
3- Boose, J. H. 1989. A survey of knowledge acquisition techniques and tools. Knowl. Acquis. 1, 1 (Jun. 1989), 3-37. DOI= http://dx.doi.org/10.1016/S1042-8143(89)80003-2
[4]
4-Pahl, C., Barrett, R., and Kenny, C. Supporting Active Database Learning and Training through Interactive Multimedia. In Proceedings of 9th annual SIGCSE conference on Innovation and Technology in Computer Science Education, ITiCSE ’04, ACM, 2731.
[5]
5- McMaster, K., B. Rague, T. McMaster, and A. Blake. “Two Gestalts for Mathematics: Logical vs. Computational.” In The Proceedings of the Information Systems Education Conference 2007, v 24 (Pittsburgh): §2342. ISSN: 1542-7382. (A later version appears in Information Systems Education Journal 6(20). ISSN: 1545-679X.)
[6]
6- McMaster, K., B. Rague, S. Hadfield, and N. Anderson. "Three Software Development Gestalts." In The Proceedings of the Information Systems Education Conference 2008, v 25 (Phoenix): §2333. ISSN: 1542-7382.
[7]
7- Elenbogen, B. S., Maxim, B. R., and McDonald, C. 2000. Yet, more Web exercises for learning C++. SIGCSE Bull. 32, 1 (Mar. 2000), 290-294. DOI= http://doi.acm.org/10.1145/331795.331872
[8]
8- Perrenet, J., Groote, J. F., and Kaasenbrood, E. 2005. Exploring students' understanding of the concept of algorithm: levels of abstraction. SIGCSE Bull. 37, 3 (Sep. 2005), 64-68. DOI= http://doi.acm.org/10.1145/1151954.1067467
[9]
9- VanDeGrift, T. 2004. Coupling pair programming and writing: learning about students' perceptions and processes. SIGCSE Bull. 36, 1 (Mar. 2004), 2-6. DOI= http://doi.acm.org/10.1145/1028174.971306
[10] 10- Anderson, N., and K. McMaster. "Database Frameworks: Textbooks vs. Student Perceptions." In The Proceedings of the 39th ASEE/IEEE Frontiers in Education Conference, San Antonio, TX, Oct. 2009.
AUTHOR INFORMATION Nicole Anderson, Assistant Professor of Computer Science, Winona State University,
[email protected] Kirby McMaster, Visiting Professor of Computer Science, St. Mary’s College of Maryland,
[email protected] Samuel Sambasivam, Professor of Computer Science, Azusa Pacific University,
[email protected]
978-1-4244-6262-9/10/$26.00 ©2010 IEEE October 27 - 30, 2010, Washington, DC 40th ASEE/IEEE Frontiers in Education Conference F3G-5
