AAH GRAPHICS, INC. / (540) 933-6210 / FAX 933-6523 / 11-26-2002 / 14:40
Computer Support for Curriculum Developers: CASCADE Susan McKenney Nienke Nieveen Jan van den Akker
This paper examines research on a family of computer-based tools, CASCADE (Computer ASsisted Curriculum Analysis, Design and Evaluation), designed to assist in the complex task of curriculum development. It begins with discussion of curriculum developers and their activities, followed by examination of how the computer can offer support for their tasks. The main elements of four related systems for computer supported curriculum development are discussed and approaches to designing tools of this nature are considered. Following an overview of main findings, this article concludes with thoughts on fruitful directions for research on computer supported curriculum development, emphasizing the need for increased attention to implementation and impact studies.
Curriculum development is a deliberate activity directed at (re)designing, developing and implementing plans for learning (in school or corporate contexts), and it is characteristically complex. Decisions must be made around topics such as: What should learners learn and why? Which activities will lead to the learning process? What sorts of learning resources are appropriate? How much time should be spent? Those decisions must often be made at different, interrelated levels: macro (or system) level, meso (or institution) level, and micro (or classroom) level. According to several authors (cf., Gustafson & Branch, 1997; Visscher-Voerman, Gustafson, & Plomp, 1999), high quality curriculum development usually entails a mixture of activities, including:
• Analysis: performing problem analysis, task analysis, context analysis, content analysis;
• Design: deciding on substantive parts or com-
ponents of the curriculum, such as: aims and objectives, subject matter, learning and instructional strategies, learner tests, timing, location;
• Construction:
revising
• Evaluation:
of
creating and prototypes of the curriculum; testing the quality prototypes or final deliverable;
• Implementation:
the
applying the curriculum in
practice.
While careful execution of these kinds of activities refers mainly to the “technical-professional” perspective of curriculum development (Goodlad, 1994), the development processes also involve “socio-political” aspects through the inETR&D, Vol. 50, No. 4, 2002, pp. 25–35 ISSN 1042–1629
25
AAH GRAPHICS, INC. / (540) 933-6210 / FAX 933-6523 / 11-26-2002 / 14:40
26 dividual and collective views and interests of those involved, especially when making “substantive” decisions. The scope and choice of development activities is usually influenced by a combination of designer preferences and organizational factors, such as: directives of the developers’ organization, available resources (e.g., time, personnel, and financial support), other team members and disciplines involved, and communication within the team and with other stakeholders. In addition, characteristics of the curriculum itself influence the activities developers carry out. For instance, at the macro level, development of the curriculum is often much more a socio-political process with many stakeholders than the development of lesson plans at micro level. Also, developing a highly innovative curriculum generally necessitates activities different from (or, at least more extensive than) those required for developing a more conventional curriculum (cf. Wedman & Tessmer, 1993). These and other factors play an interrelated role during the development process and make the actual course of action difficult to describe. Walker (1990) illustrated this interplay as follows: To portray it [the development process] as an art fails to give due consideration to the facts and relationships it must respect and to the need for its results to be objectively defensible. To portray it as a technology narrows and trivializes the many subtle and important considerations crucial to important stakeholders in the educative process. (p. 499)
Over the years, curriculum developers have gained understanding on the complex processes associated with their tasks. In addition, several studies were carried out to make explicit and extend this body of knowledge (cf., Kessels, 1993; Rowland & Adams, 1999; Tessmer & Wedman, 1993; van den Akker, Boersma, & Nies, 1990; Visscher-Voerman, 1999). That knowledge base, when made available in a usable manner, can assist novice developers in honing their own skills. Traditionally, support for curriculum developers has been made accessible through job aids (such as handbooks on curriculum development, design manuals, models and checklists),
ETR&D, Vol. 50, No. 4
interaction with colleague developers (mentoring, modeling), and more formalized learning facilities (lectures, workshops). However, the advance of computer-based support, permeating almost every professional domain, has prompted the question of whether the computer could also play a supportive role in the complex domain of curriculum development.
DEVELOPMENT OF ELECTRONIC PERFORMANCE SUPPORT SYSTEMS
Electronic performance support systems (EPSSs) are computer programs that assist in the execution of (usually complex) tasks. They are composed of varying elements, including job aids, communication aids, and learning facilities. Job aids can take the form of online help systems and reference systems; communication aids include e-mail, news groups, text-based, audiobased, or video-based conferencing tools and shared work spaces; and learning facilities can be provided through computer-based training such as drills, tutorials and simulations. A computer-based environment that integrates these types of performance support is an EPSS. Advocates of the concept of EPSS presume several advantages of providing computer support: improved task performance, organizational learning, and increased task-related knowledge (Gery, 1991; Stevens & Stevens, 1995). Since the early 1990s, the concept of an EPSS has been applied to the field of curriculum development. Here, it is the performance of the curriculum developer that is supported by the computer. For an overview of those tools currently available in the United States and abroad (especially Australia and The Netherlands), please refer to Nieveen and Gustafson (1999). Earlier work on EPSS demonstrated a clear orientation toward “proof of concept” thinking, as evidenced in the literature that populated journals in the 1980s (for an overview of EPSSrelated literature from 1989 to 1995, please refer to Hudzina, Rowley, & Wagner, 1996). Emphasis was given to defining the field (cf., Gery, 1989, 1991) and discussing ways of exploring it (Pirolli & Russell, 1990; Stevens & Stevens, 1995). As the field of EPSS has grown, an increas-
AAH GRAPHICS, INC. / (540) 933-6210 / FAX 933-6523 / 11-26-2002 / 14:40
27
COMPUTER SUPPORT FOR CURRICULUM DEVELOPERS
ingly analytic and research-oriented perspective has begun to take shape. From experience, contemporary thinking on the topic has shifted. Whereas earlier efforts seemed more enamored with the idea of exploring what electronic systems could offer, a trend rapidly emerged in which user performance became central, with the supporting systems on the periphery (cf., Winslow & Bramer, 1994); hence the field of performance-centered design (PCD) was born. Researchers and developers began to articulate fundamental forms of support (Gery, 1995), attributes and behaviors of performance-centered systems (Gery, 1997) and even methodologies for conducting PCD (Raybould, 2000). Synthesizing these and additional trends in the related fields of knowledge management systems (KMS) and professional networks has contributed to furthering the EPSS–PCD dialogue. (For additional information regarding the link between PCD and KMS, please refer to Laffey, 1995.) The understanding of how to maximize the potential benefits of such systems has definitely grown, although very limited research-based information is yet available on their implementation and impact in real-life settings. The following section describes research activities that explored the use of EPSSs in the domain of curriculum development.
tive evaluation (Nieveen, 1997; Nieveen & van den Akker, 1999). In 1996, two follow-up studies were initiated which used the CASCADE project as a springboard for further exploration into computer supported curriculum development in very different contexts: CASCADE-SEA (Science Education in Africa) and CASCADEMUCH (MUltimedia curriculum design in CHina). The CASCADE-SEA study investigated support of science and mathematics resource teachers creating exemplary lesson materials for classroom use in sub-Saharan Africa (McKenney, 2001). The CASCADE-MUCH study examined support of teachers designing multimedia scenarios (blueprints for electronic lesson materials) in China (Wang, 2001). In 1999, a third CASCADE study was launched: CASCADE-IMEI (Innovation in Mathematics Education in Indonesia). This study focuses on the development and implementation of a learning environment for student teachers in Indonesia to learn to apply a realistic mathematics approach in their lessons (Zulkardi & Nieveen, 2001). To illustrate the main similarities and differences among computer supported curriculum development research in general (and the CASCADE family in particular), the following sections address four basic questions related to this field of study: 1. For whom might computer supported curriculum development be intended, and why?
CASCADE AS A CONTINUING LINE OF INQUIRY
2. What elements of curriculum development may be supported? 3. How can valuable support be offered?
About the CASCADE Initiatives
At the University of Twente’s Faculty of Educational Science and Technology, exploring the computer’s potential supportive role in curriculum development has been on the research agenda since 1993, when a developmental research study was initiated entitled CASCADE (Computer ASsisted Curriculum Analysis, Design and Evaluation). The CASCADE project aimed to learn more about how EPSSs could contribute to curriculum development. In particular, it focused on supporting Dutch professional curriculum developers (working at the National Institute for Curriculum Development) through the often-neglected process of forma-
4. What research approaches are the most useful, and how can they be carried out?
Users and their Needs
Most tools for computer supported curriculum development (and definitely those in the CASCADE family) have been conceived to address some kind of a problem or challenge a designer encounters when involved in curriculum development tasks. Tools are then created with the hope of being able to solve or speak to their particular needs. In so doing, careful attention is given to the context in which a tool will be used.
AAH GRAPHICS, INC. / (540) 933-6210 / FAX 933-6523 / 11-26-2002 / 14:40
28
ETR&D, Vol. 50, No. 4
Table 1
Users and goals of CASCADE systems CASCADE
CASCADE-SEA
CASCADE-MUCH
CASCADE-IMEI
Users
Dutch professional curriculum designers
Resource teachers in southern Africa
Preservice teachers in Indonesia
Goals
Planning and performing formative evaluation of professionally-made lesson materials
Understanding about and creation of exemplary lesson materials
Teachers in multimedia projects in Shanghai Understanding the story boarding process and producing multimedia scenarios
Understanding and teaching realistic mathematics
NOTE: CASCADE = Computer ASsisted Curriculum Analysis, Design and Evaluation; IMEI = Innovation in Mathematics Education in Indonesia; MUCH = MUltimedia curriculum design in CHina; and SEA = Science Education in Africa
Table 2
Elements of curriculum development supported by CASCADE systems
Curriculum level Tailoring Elements of systematic approach
CASCADE
CASCADE-SEA
CASCADE-MUCH
CASCADE-IMEI
Micro
Meso
Micro
Micro
Generic
Site-specific and generic Analysis, design, development, evaluation and implementation
Site-specific and generic Analysis and design
Site-specific and generic Design, development and implementation
Formative evaluation
This includes the profile and scope of the intended user group (professional designers or teacher designers) as well as definition of areas related to their perceived needs. In general, the CASCADE family of tools has been created to support curriculum designers in various phases of educational curriculum materials development. Table 1 summarizes the specific user groups and the goals that are addressed by each tool in the CASCADE family.
products have been tailored for a particular setting. Further, different tools may support different elements (analysis, design, construction, evaluation and implementation) of a focused approach to development of education and training. Table 2 provides an overview of the variation in these areas within the CASCADE family of programs.
Support Characteristics Elements of Curriculum Development
Support tools may target different levels of curriculum development. Tools are generally classified based on the kinds of results they help to achieve. Here, one examines the nature of the product being generated (intended for use at micro [classroom], meso [institution] or macro [system] level), and the degree to which the
Computer-based support systems may offer assistance in various forms, such as job aids, learning opportunities and communication aids. Because the very nature of performance centered design (PCD) implies a series of integrated interventions in a performance support continuum, distinctions between elements of support often become blurred—especially from the user’s perspective. However, toward study-
AAH GRAPHICS, INC. / (540) 933-6210 / FAX 933-6523 / 11-26-2002 / 14:40
COMPUTER SUPPORT FOR CURRICULUM DEVELOPERS
ing the discipline in general and the CASCADE program family in particular, efforts have been made to articulate the forms of support offered within tools. CASCADE job aids include implicit and explicit advice through decision-making support based on heuristics and reminders that provide warning of the consequences or recommendations for future actions based on choices made by the user. Further, they provide templates or checklists that prepare (sub)tasks for execution and may create draft products based on user input; additionally, external programs may be linked to the EPSS. The learning opportunities in these systems consist of visual cues or metaphors that suggest a method for doing (sub)tasks and may provide procedural or conceptual answers to specific questions (who, what, when, where, how, why). Finally, communication aids may include shared workspaces that facilitate internal communication with individual colleagues or other parts of the organization and examples that foster reflection. Each CASCADE program has ultimately aimed toward the creation of an integrated whole, in which these and many other features are blended together.
Development Research and CASCADE
Generally speaking, design teams in the field of computer-supported curriculum development aim at creating a computer support system to assist curriculum developers in optimizing the effectiveness of their efforts. The CASCADE studies have illustrated that the design of a computer support system for curriculum development is a complex and innovative task for which only few validated principles are available to structure the design and development activities. A development research approach (cf., van den Akker, 1999) has provided multifaceted opportunities to increase understanding of how to structure computer support for curriculum developers. Within such an approach, a systematic preliminary investigation of tasks, problems, and context is made, including a search for accurate and explicit links of that analysis with state-of-the-art knowledge and
29 relevant or inspiring examples. Moreover, an iterative process of evolutionary prototyping and extensive interaction with practitioners is needed to gradually clarify both the problem and the characteristics of its potential solution. Each study within the CASCADE line of inquiry has seen similar research phases including: approximately one year of needs and context analysis, two to three years of cyclic design, development, (small scale) implementation and evaluation, and approximately nine months of (more) summative evaluation. Each of the CASCADE studies relied heavily on an evolutionary prototyping approach. Compared to the provision of abstract specifications, the use of a series of concrete prototypes can often provide a better foundation for identifying the requirements of the support system in interaction with members of the target group, experts, and other interested parties (cf., Nieveen, 1999). According to Smith (1991, p. 42), a prototype is a “preliminary version or a model of all or a part of a system before full commitment is made to develop it.” In this definition, the term develop refers to the construction of the final product. Prototypes are all products that are designed before the final product will be constructed and fully implemented in practice. The aim of evolutionary prototyping is to come to successive approximations of an ideal tool that increasingly meets the users’ needs. In the design of the CASCADE systems, both paper-based and computer-based prototypes were utilized. The paper-based prototypes (or scenarios) consisted of a narrative description of typical and critical situations that prospective users experience, accompanied with a pile of papers representing all screens that may appear during the use of the system. These were used to make the tentative design specifications more concrete and to make it easier to communicate with the target group about the potential of the system. The computer-based prototypes were used at varying stages of development, thus containing varying degrees of functionality. They were particularly useful in communicating otherwise abstract ideas, for observation of usercomputer interaction, and as discussion tools with user-designers. During the last phase of the studies (following an average of four proto-
AAH GRAPHICS, INC. / (540) 933-6210 / FAX 933-6523 / 11-26-2002 / 14:40
30
ETR&D, Vol. 50, No. 4
types), final deliverables were created. Research on the different CASCADE versions and prototypes was carried out based on a shared view of quality. Namely, to develop an effective support system, it was presumed to be equally important to develop a program that is both valid and practical, because these requirements may be seen as prerequisites for effectiveness. Thus, a computer support system should meet the following requirements to be categorized as being a high-quality support system (Nieveen, 1997; van den Akker, 1999):
• Validity: The system should include state-ofthe-art knowledge and should be internally consistent.
• Practicality:
The system should meet the needs, wishes, and contextual constraints of the members of the target group.
• Effectiveness:
The system should positively impact the curriculum development efforts of the target group.
To make the software development process more transparent, the systems were decomposed into several key components that required major attention (de Hoog, de Jong, & de Vries, 1994). In the CASCADE products, the first component is the content of the system, referring to the conceptualization of the curriculum development task(s) that are supported by the tool. The support included in the system to assist the curriculum developers in performing their tasks is viewed as the second main component. The third key component is the user interface and related technical aspects that should assure that the support is readily available for users. In the CASCADE-MUCH study, a fourth component was added referring to the quality of the program’s output. Each of these components may contribute to or obstruct the overall effectiveness of the support system, and was studied within the framework of the system as an integrated whole. Formative evaluation of the prototypes played a crucial role within the evolutionary prototyping processes because it gave participants in the evaluation as well as the developers of the prototypes insight into the potential of the support system and the desired characteristics. During the formative evaluation
of the prototypes two main types of respondents participated: users and experts. The insights of these groups were elicited through a variety of activities, including walk-throughs, cooperative evaluations, expert appraisals, workshops and try-outs. During a walk-through, respondents walked through (usually paper-based) screens and their reactions were captured via workaloud protocols, debriefings and checklists. The cooperative evaluation approach featured more focused use of the system based on specific tasks and higher degrees of interaction between the respondents and the evaluator (asking questions, explaining frustrations, etc.). Expert appraisals were similar to cooperative evaluations (which were conducted with user groups) and allowed information to be gathered verbally, through checklists and questionnaires and through observation. These tended to focus on one component at a time (content, support or interface). Further, as prototypes evolved, workshops were organized with (following a brief introduction) hands-on experiences usually followed by discussion, (alternative) idea generation and written data collection activities (checklists, questionnaires, etc.). Finally, try-outs were held in which the members of the target group used the tool for its intended purpose in realistic settings. Here, the evaluator rarely intervened and data were collected via observation, questionnaires, logbooks, computer logs (of user actions) and individual as well as group interviews. Results of the formative evaluations led to revision of each prototype and adaptation of the specifications that underlie the support system. In this way, each prototyping cycle represents an evolution of intentions of the system. Based on several such cycles, the computer systems evolved toward high-quality final deliverables.
FINDINGS AND CONCLUSIONS
The CASCADE studies have yielded results in terms of both the products generated and lessons learned regarding the process through which these programs were created. Figures 1–4 briefly present the CASCADE systems through screen shots that illustrate various forms of support.
AAH GRAPHICS, INC. / (540) 933-6210 / FAX 933-6523 / 11-26-2002 / 14:40
COMPUTER SUPPORT FOR CURRICULUM DEVELOPERS
Figure 1
31
CASCADE instruments page.
The screen displayed in Figure 1 comes from the original CASCADE program and illustrates a job aid giving advice through reminders. Here, on-screen arrows indicate that the user needs to so something based on a decision made earlier in the program. Additionally, sample (customizable) instrument templates are available for use, and explanations may be requested by clicking on any underlined text. The screen in Figure 2 illustrates additional job aids that are more tailored to the user’s individual situation. Here, in the CASCADE-SEA program, a draft lesson plan has been automatically created based on user input. At the same time, further customization is also recommended and a checklist is offered to help in this process. As with the previous example, the draft may be exported and edited in a word processor and additional explanation may be accessed. Figure 3 shows the interface of the CASCADE-MUCH program, and how learning may be implicitly stimulated through the visual ap-
pearance which suggests a method for doing (sub) tasks. The manual and subscreens consistently illustrate this process. At the same time, decision-making support is offered by providing a list of options pertaining to possible goals and ways to use the program’s outputs. The screen shown in Figure 4 comes from CASCADE-IMEI. This is an example of a communication aid in which pupil work is examined and posted for analysis and review. While most systems rely on asynchronous communication, the chatting facilities on some of the CASCADE Websites also aim to stimulate real-time communication. As illustrated throughout this section, each of the CASCADE studies has been tailored to support different elements of the curriculum development process, in a variety of contexts. While the detailed findings are applicable to the individual systems, broader conclusions may be drawn with reference to the family of programs. The studies have shown that the computer does
AAH GRAPHICS, INC. / (540) 933-6210 / FAX 933-6523 / 11-26-2002 / 14:40
32 Figure 2
ETR&D, Vol. 50, No. 4
CASCADE-SEA lesson plan generation page.
have the potential to positively impact the performance of those using CASCADE tools. By offering a systematic approach and clear structure, the CASCADE system outputs tend to be more elaborate and internally consistent than those developed without the aid of the computer. Also, in many cases, CASCADE tools save the users time and help them to provide justifications for decisions made throughout the curriculum development process. This, in turn, can lead to increased confidence as well as professional development on behalf of the users. Together, the CASCADE studies have offered valuable insights regarding the potentials and limitations of computer supported curriculum development tools. Two illustrations are given here. In terms of potential, the CASCADE studies have illustrated that support can be created to assist at various levels of curriculum development. Logical extensions of the CASCADE family might even include:
• Macro: a tool that facilitates comparison of na-
tional syllabi, curriculum frameworks, standards or attainment targets, used worldwide;
• Meso: a tool designed for stimulating the discussion on alternative curriculum trajectories at school level;
• Micro: a tool that aids teachers in selecting and adapting instructional materials for their own classroom context.
Further, these studies have illustrated certain limitations. Namely, they show that EPSSs are not likely to be able to provide support for all curriculum perspectives. Earlier in this article, the substantive, technical-professional and socio-political perspectives of curriculum development were briefly mentioned. The CASCADE findings illustrated that EPSSs are primarily well-suited to supporting processes. Given that the socio-political and substantive perspectives are related to but are not, themselves, processes, the conclusion may be drawn that these cannot be directly supported through
AAH GRAPHICS, INC. / (540) 933-6210 / FAX 933-6523 / 11-26-2002 / 14:40
COMPUTER SUPPORT FOR CURRICULUM DEVELOPERS
Figure 3
CASCADE-MUCH goals and usage page.
Figure 4
CASCADE-IMEI student production page.
33
AAH GRAPHICS, INC. / (540) 933-6210 / FAX 933-6523 / 11-26-2002 / 14:40
34 an EPSS. For example, decision-making regarding curriculum content has both socio-political and substantive aspects. But the components involved (information gathering, opinion-forming, planning, creating documents, etc.) are clearly procedural in nature: thus belonging to the technical-professional perspective. Understanding this type of limitation may help researchers to concentrate efforts on those that are most promising, that is, support for specific processes. While research1 on CASCADE, CASCADESEA and CASCADE-MUCH has shown that these products meet the criteria of validity, practicality, and effectiveness, the results also indicate that such qualities tend to be more gradational than simply present or absent. That is, the degree to which the systems are valid, practical and effective seems much more difficult to pinpoint. In part, this is due to the nature of the quality aspects themselves, which tend to be inherently intersubjective characteristics. However, it is the opinion of these authors that the relatively small scale or brief evaluation events (ranging from one half day to one week each, repeated over a period of six to nine months) only allow a limited degree of insight into these areas. Such constraints appear particularly acute in terms of studying system effectiveness. This comes as no great surprise given the aforementioned tendency in this field (and its current stage of evolution) to focus on design studies. However, if the domain is to progress, more detailed and useful information should be gathered from long-term implementation studies that build on design research and further contribute to understanding not only program effectiveness but also implementationfurthering strategies. Follow-up studies with long-term evaluation in real-life settings, and more robust indications of effectiveness, may serve well to articulate and strengthen the knowledge base about the quality of these tools. If well coordinated, parallel implementation studies could allow for comparison and contrast of data across and within
1 As the CASCADE-IMEI study is still underway, end evaluation data are not yet available.
ETR&D, Vol. 50, No. 4
contexts. Further, implementation studies offer the opportunity to explore those scale and scalability issues that are less related to system design and more focused on optimizing implementation processes. Finally, the CASCADE studies highlighted the participation of the target group members to be of paramount importance during the development process.
Susan McKenney [
[email protected]], Nienke Nieveen and Jan van den Akker are at the University of Twente. For additional information regarding the CASCADE line of inquiry, please refer to: http://projects.edte.utwente.nl/cascade/. Correspondence concerning this article should be addressed to Susan McKenney, University of Twente, Department of Curriculum Technology, P.O. Box 217, 7500 AE Enschede, The Netherlands.
REFERENCES de Hoog, R., de Jong, T., & de Vries, F. (1994). Constraint-driven software design: An escape from the waterfall model. Performance Improvement Quarterly, 7(3), 48–63. Gery, G. (1989). Training vs. performance support: Inadequate training is now insufficient. Performance Improvement Quarterly, 2(3), 51–71. Gery, G. (1991). Electronic performance support systems: How and why to remake the workplace through the strategic application of technology. Boston, MA: Weingarten. Gery, G. (1995). Attributes and behaviors of performance-centered systems. Performance Improvement Quarterly, 8(1), 47–93. Gery, G. (1997). Granting three wishes through performance-centered design. Communications of the ACM, 40(7), 54–59. Goodlad, J. (1994). Curriculum as a field of study. In T. Husén & T.N. Postlethwaite (Eds.), The international encyclopedia of education (pp. 1262–1267). Oxford: Pergamon. Gustafson, K., & Branch, R. (1997). Survey of instructional development models (3rd ed.). Syracuse: Eric Clearinghouse on Information and Technology. Hudzina, M., Rowley, K., & Wagner, W. (1996). Electronic performance support technology: Defining the domain. Performance Improvement Quarterly, 9(1), 36–48. Kessels, J. (1993). Towards design standards for curriculum consistency in corporate education. Doctoral dissertation. Enschede, The Netherlands: University of Twente. Laffey, J. (1995). Dynamism in electronic performance
AAH GRAPHICS, INC. / (540) 933-6210 / FAX 933-6523 / 11-26-2002 / 14:40
COMPUTER SUPPORT FOR CURRICULUM DEVELOPERS
support systems. Performance Improvement Quarterly, 8(1), 31–46. McKenney, S.E. (2001). Computer-based support for science education materials developers in Africa: Exploring potentials. Doctoral dissertation. Enschede, The Netherlands: University of Twente. Nieveen, N.M. (1997). Computer support for curriculum developers: A study on the potential of computer support in the domain of formative curriculum evaluation. Doctoral dissertation. Enschede, The Netherlands: University of Twente. Nieveen, N.M. (1999). Prototyping to reach product quality. In J. van den Akker, R. Branch, K. Gustafson, N. Nieveen & T. Plomp (Eds.) Design approaches and tools in education and training (pp. 125–136). Dordrecht, The Netherlands: Kluwer Academic Publishers. Nieveen, N.M., & Gustafson, K. (1999). Characteristics of computer-based tools for education and training development: An introduction. In J. van den Akker, R. Branch, K. Gustafson, N. Nieveen & T. Plomp (Eds.) Design approaches and tools in education and training (pp. 155–174). Dordrecht, The Netherlands: Kluwer Academic Publishers. Nieveen, N.M., & van den Akker, J.J.H. (1999). Exploring the potential of a computer tool for instructional developers. Educational Technology Research and Development, 47(3), 77–98. Pirolli, P., & Russell, D.M. (1990). The instructional design environment: Technology to support design problem solving. Instructional Science, 19, 121–144. Raybould, B. (2000). Building performance-centered Web-based systems, information systems, and knowledge management systems in the 21st century. Performance Improvement, 39(6), 32–39. Rowland, G., & Adams, A.M. (1999). Systems thinking in instructional design. In J. van den Akker, R.M. Branch, K. Gustafson, N. Nieveen & T. Plomp (Eds.), Design approaches and tools in education and training (pp. 29–44). Dordrecht, The Netherlands: Kluwer Academic Publishers. Smith, M.F. (1991). Software prototyping: Adoption, practice and management. London: McGraw-Hill. Stevens, G., & Stevens, E. (1995). Designing electronic performance support tools: Improving workplace performance with hypertext, hypermedia and multimedia.
35 Englewood Cliffs, NJ: Educational Technology Publications. Tessmer, M., & Wedman, J. (1993). Instructional designers’ decisions and priorities: A survey of design practice. Performance Improvement Quarterly, 6(2), 43–57. Van den Akker, J. (1999). Principles and methods of development research. In J. van den Akker, R. Branch, K. Gustafson, N. Nieveen, & T. Plomp (Eds.), Design approaches and tools in education and training (pp. 1–14). Dordrecht, The Netherlands: Kluwer Academic Publishers. Van den Akker, J., Boersma, K., & Nies, A. (1990). Ontwikkelstrategieën in SLO-projecten [Development strategies in SLO projects]. Enschede, The Netherlands: SLO (National Institute of Curriculum Development). Visscher-Voerman, I., Gustafson, K., & Plomp, T. (1999). Educational Design and Development: An Overview of Paradigms. In J. van den Akker, R. Branch, K. Gustafson, N. Nieveen, & T. Plomp (Eds.), Design approaches and tools in education and training (pp. 15–28). Dordrecht, The Netherlands: Kluwer Academic Publishers. Visscher-Voerman, I. (1999). Design approaches in education and training. Doctoral Dissertation. Enschede, The Netherlands: University of Twente. Walker, D. (1990). Fundamentals of curriculum. San Diego, CA: Harcourt, Brace Jovanovich. Wang, Q. (2001). Computer support for multimedia curriculum design. Doctoral dissertation. Enschede, The Netherlands: University of Twente. Wedman, J., & Tessmer, M. (1993). Instructional designers’ decisions and priorities: A survey of design practice. Performance Improvement Quarterly, 6(2), 43–57. Winslow, C.D., & Bramer, W.L. (1994). Future work: Putting knowledge to work in the economy. New York, NY: The Free Press. Zulkardi & Nieveen, N.M. (2001). CASCADE-IMEI: Web site support for student teachers learning Realistic Mathematics Education (RME) in Indonesia. Paper presented at the International Conference about Teaching Mathematics using Technology, August 5– 9, Klagenfurt, Austria.
