A Discussion Paper for VITAL ’98

by

Michael Spector

and

Barabara Wasson

University of Bergen, Norway

and

Berner Lindström

Göteborg University, Sweden

Technology-based learning environments are growing in number and prevalence at an unprecedented rate in spite of continuing debates in the academic literature with regard to their learning-effectiveness, impacts on organizations, and overall utility to society (see, for example, Carter, 1997; Clark, 1994; Kozma, 1994; Lengel, 1997). As a consequence, there are many innovative investigations into how people might learn using new technologies, and these studies are, in turn, causing much discussion with regard to foundational issues in learning theory and instructional design. The times are certainly changing, and one challenge we face is to make sense of some of these changes so that we can make effective use of new technologies to support learning and instruction. It is our hope in this paper to move this discussion forward by providing a design framework for technology-mediated learning environments. We intend this framework to be well-founded in learning theory and to have strong implications for designing learning environments for what we regard as a challenging domain: learning in and about complex systems.

We shall proceed by briefly identifying and reviewing relevant learning theories. We shall then establish a unifying perspective in the context of these theories which has clear design and evaluation implications. This unifying perspective will then be used to describe our framework. We conclude with an illustration of this framework and indicate other instances close in kind and spirit to ours.

How do people come to acquire complex skills and knowledge? We ask this apparently simple question in order to identify relevant assumptions and highlight its complexity. First, much learning research proceeds on the following assumptions:

1. Learning is a natural, human activity.
2. The unit of analysis for learning effectiveness is an individual human learner.
3. Learners are rational.

An additional assumption prevalent in the educational research community is that instructional design is primarily a prescriptive enterprise forming a bridge between descriptive learning research and practical development of learning environments (Reigeluth, 1983).

There is clearly practical value in adopting this research perspective. By varying the instructional methods used in certain conditions, one can measure outcomes and study the effects of those methods on learning outcomes. If enough data is collected, one then hopes to be able to establish a strong argument for the desirability of a particular method given certain learning conditions, thus prescribing how one ought to design instruction to achieve desired outcomes. In this perspective, learning is a natural process, and it is theoretically possible to identify how various types of learners engage in this process and take those differences into account in the design of instruction. Learners are rational in the sense that they are goal-driven, purposeful agents with the ability to identify and select reasonably efficient means to achieve goals. Typically, conditions are held constant and various interventions (instructional methods instantiated in particular learning environments) are investigated. Learning outcomes or effects are then measured on individuals.

Such a research paradigm has in fact produced many useful findings, so it should not be discounted. For example, by emphasizing the rationality of learners, designers are able to facilitate the identification of learning goals and support activities likely to satisfy the achievement of those goals. This line of research emphasizes intentional learning, and generally ignores incidental learning, to which it is admittedly much more difficult to apply the conditions-methods-outcomes model. Since it is assumed that learners will want to achieve goals efficiently, designers can then specify how and when to provide learners with informative feedback on progress towards those goals.

We shall refer to this as the atomistic perspective because it is characterized by an atomistic view of learning, both in terms of units of learning (very specific and discrete conditions, methods, and outcomes) and in terms of learners (typically focusing evaluation on individual learners, even when the setting involves cooperative learning). The atomistic perspective can be contrasted with what we shall call the integrated perspective (Spector, 1994, 1995). The integrated perspective begins with a view of a person as a member of a society or language community. The overall goal of a society or language community typically involves a strong survival element, although this is quite often not made explicit. Living consists of working and learning, which are viewed as essentially collaborative efforts to achieve commonly held goals. From this perspective, individuals might manage to acquire extremely high levels of performance at particular tasks while the larger social group consistently falters. This would not count as effective learning from an integrated perspective.

There are definite comparisons to a team perspective when thinking of learning from an integrated perspective. A sports team may have several outstanding players, leading the league in certain categories, while the team is in last place. This is not a satisfactory situation, especially from the team's point of view. One can further imagine a team locked into poor overall performance by paying one star player an extremely large salary. This might prevent the team from paying higher salaries to others, and it may also foster jealousy and resentment within the team. While the star player may draw in large crowds and the team's owners may prosper, the team's poor performance may further decline, even when a second star player is added.

One should be careful not to carry the sports analogy too far, however. Learning organizations and societies are not necessarily competitive with other such organizations, as is the case in the world of sports. Moreover, membership criteria and reward mechanisms are entirely different. Our point here is to suggest that with many situations it is appropriate and useful to take a more integrated, holistic view of learning. We believe this is especially important with regard to learning in and about complex domains.

We are especially interested in complex domains for a number of reasons. From a research perspective, these domains present the most significant challenges, both for designing effective learning environments and for determining factors which contribute to learning. From a social perspective, these domains present the most significant challenges for the future well-being of our species on this planet. We have serious problems to confront if we are to survive, including worsening global environmental problems, persisting regional and ethnic conflicts, and wildly fluctuating economic conditions. Can we become better prepared to meet these challenges? How?

Complex systems can be depicted as a collection of inter-related items (e.g., stocks and flows in system dynamics), and they are characteristics by internal feedback mechanisms, nonlinearities, delays, and uncertainties (Sterman, 1988, 1994). These systems typically exhibit dynamic behavior, especially in the sense that how they behave has an effect on internal relationships (the structure of the system), perhaps strengthening one of the feedback mechanisms (e.g., the owners' proceeds due to acquiring a star player reinforces that mode of response to declining profits, as opposed, for example to improving overall performance.). This change in internal structure in turn has consequences for how the system will behave in the future (e.g., the team may perform even more poorly as existing players resent the special treatment and salary given the star).

Complex systems can be found in abundance at many different levels. The human body can be viewed as a complex system. Economics, ecology, epidemiology, project management, and training all typically involve complex, dynamic systems. As a species, we have well-documented difficulty in dealing effectively (understanding and making robust policies concerning) with such systems (see, for example, Dörner, 1996). There are clearly individual exceptions, persons who someone acquire a deep understanding of such systems. Typically, such deep understanding, characterized by effective decision making across a wide variety of changing conditions, takes years to acquire, and appears not to be easily acquired in spite of concentrated education and training efforts (Dreyfus & Dreyfus, 1986). Why have we failed to improve our thinking skills in complex domains in spite of such persistent and serious efforts?

In part, we have not fully understood relevant psychological and sociological factors. In part, we have not fully integrated relevant principles about human learning into design praxis. There are probably other factors, as well, but we shall hereafter focus on these two, especially the second.

We should have learned that humans have difficulty in estimating the effects of accumulation over time, in predicting the effects of delays, and in calculating nonlinear outcomes (Sterman, 1994). We should have learned that even well-intentioned persons tend to focus on local problems as opposed to whole systems (even when told that a holistic understanding is essential for solving particular problems), that people may become cynical and overlook possible solutions when their first attempts fail, that people do not communicate effectively in crisis situations, and that inferring underlying system structures from externally viewed system behaviors is not an easy task (Dörner, 1996). Other overlooked lessons can and should be identified, but these provide some clear clues as to how we can return to theory and draw together relevant principles to guide the construction of learning environments for complex domains.

At this point, we might simply say that we, as instructional scientists, have not fully understood the socially-situated learning perspective and its implications for human learning in and about complex systems. There is a great deal of discussion about situated, problem-based, and collaborative learning, but we are missing critical pieces of a design framework. Put differently, we believe that we lack a well-articulated design framework with sufficient detail to take us from a socially-situated, problem-based, collaborative learning perspective to the design of a particular learning environment for a particular subject domain. The closest such approach we find is cognitive apprenticeship (Collins et al., 1989; Collins, 1991). We regard our work as an extension to that approach.

The theoretical foundations for this effort come primarily from a socially-situated learning perspective, drawing heavily on the views of Bruner (1985), Lave (1988), Piaget (1970), and Vygotsky (1978). Within this perspective, learning is viewed as an active process of knowledge construction in which learners are typically involved with other learners in authentic, problem-solving situations. The need to learn created by a realistic problem provides motivation, and interaction with other similarly immersed learners provides facilitation. We are favorably inclined to an attitude similar Sfard (1998) that emphasizes the need to take into account both an acquisition (static knowledge objects with learners acquiring expertise) and a participation metaphor (dynamic knoweldge objects with learners as active apprentices). Much higher order learning relies on knowledge and associated learning activities that might best be supported within the acquisition view. However, to progress beyond competent performance and become a proficient expert (Dreyfus & Dreyfus, 1986), we believe that the participation metaphor with its emphasis on active learner participation in socially-situated and problem-oriented settings is crucial.

In the following sections we shall briefly review highlights of these theoretical foundations, emphasizing those aspects which we find particularly relevant to the design of collaborative learning environments for complex domains. It is worth repeating that we are not speaking in general about online learning, as many such environments do not require special care or emphasis on active participation and collaboration. We are especially concerned with the design of distance learning to support learning in complex domains. Because we draw so heavily on certain theoretical foundations, and especially Vygotsky, we begin with Vygotsky's cultural-historical perspective and proceed on to connections with activity theory and situated learning.

For Vygotsky (1978), human mental functions appear first as inter-individual and later as intra-individual. This process involves the use of socially developed tools. For Vygotsky, the unit of analysis for human activity and for human learning was the mediated action of an individual. This broadens the unit of analysis identified earlier from just the individual to include an artifact with which an individual interacts.

Leontiev (1975) expanded Vygotsky's cultural-historical theory to an activity theory approach to human interaction. From this perspective, reality consists of mediated, social activities, among other aspects. For Leontiev, the unit of analysis was extended to include the notion of a collective activity, something done by a community with a purpose (which need not be consciously recognized). This motive or purpose is composed of individual actions were directed toward a common goal. An individual's mediated actions could still be analysed, but there was now a necessary social dimension (being part of a collective activity) which was used to understand individual activity.

Davydov (1988) applied Leontiev's activity-theoretical approach to the learning process. According to Rubtsov (1993), Davydov developed a psychological theory of learning activity which focused on the "goal-oriented, joint activity of adult and child, within the social context of development" (p. 4). From this perspective, the aim of a learning activity is to teach study skills that enable learners to think on their own. Margolis (1993) suggests that within this perspective the computer can have one of two roles: as a tool for the acquisition of knowledge and empirical facts, or as tools for the development of children's thinking (i.e., as tools for reflection or metacognition). We think both roles are possible, even within the same learning environment, but we do agree with Sfard (1998), Dörner (1996), and others that when the computer is used to support higher order learning in complex domains that supporting active participation and reflection are especially important.

Collaborative learning is a collection of perspectives based on principles of interpersonal interaction (Sorensen, 1997). Three perspectives of collaboration that place emphasis on different goals include (Fjuk, 1998):

1. Joint construction of knowledge (e.g., joint problem-solving by mutual refinement). For example, see the work on "Learning through Collaborative Visualization" at http://www.covis.nwu.edu/ .
2. Joint negotiation of alternatives through argumentation, debate and other means For example, see the work on "Advanced Cognitive Tools" and the Belvedere software at http://advlearn.lrdc.pitt.edu/belvedere/index.html .
3. Student reliance on students and teachers as a resource to support their own learning and for external feedback. For example, see the work on "Collaborative Learning Environment (CSILE)" at http://www.ed.gov/pubs/EdReformStudies/EdTech/csile.html .

Collaborative learning is based on the notions of "socially shared cognition" (Resnick, Levine & Teasley, 1991), of "distributed cognition" (Salomon, 1993), and of "jointly accomplished performance" (Pea, 1993). In the latter, cognitive development is viewed as occurring though interactions between students, as well as between students and knowledgeable environments. Collaborative telelearning emphasizes collaborative interactions among students and between students and supporting actors in a distance learning environment (Wasson & Bourdeau, 1997). Introducing collaboration into a telelearning situation raises new challenges related to the logistics and to the logistical support of collaboration. The majority of research into collaborative learning has focused on collaboration between physically present actors and has generally focused on whether and under what circumstances collaborative learning was more effective than individual learning (Dillenbourg, Baker, Blaye & O’Malley, 1996). More recent efforts, however, have been directed towards a more process-oriented account of collaboration where the focus is on the role that variables such as group size, group composition, communication media, etc., play in mediating interaction. Again, this is a clear indication that the unit of analysis has been appropriately enlarged far beyond the individual learner.

In sum, we have now collected several design principles which form key features of a design framework, and we have shown their linkage to learning theory. These principles might be summarized as follows:

1. Provide support for the joint construction of knowledge objects and for the joint construction of problem solutions. This can be accomplished by providing learner-extensible databases, by providing learner-modifiable simulation models, by supporting learner-learner and learner-tutor collaboration in making changes and extensions to existing knowledge objects, and by supporting learner-learner and learner-tutor reflection on outcomes of those modifications.
2. Provide tools to support joint negotiation of alternatives. Commenting tools are a beginning. Allowing changes to be tested when appropriate is also important. It is especially important to allow learners to try out alternative simulation models, to observe results, and to support learner-learner and learner-tutor discussion and analysis of those results.
3. Provide both public and private feedback support mechanisms for learners while emphasizing to those intelligent agents providing feedback that it is important for learners to gradually improve their own ability to monitor and assess progress toward desired goals.

In short, this framework aims to support both individual work and collaborative activities, and it especially aims to support the development of collaborative learning communities.

The important implications for CSCL that emerge from these theoretical foundations include a wide range of designs for computer-based learning environments, as well as the view that the computer is a mediating tool that needs to be seen in the context of the entire learning environment within which it will be used. That context includes the instructional setting, the presence or absence of a teacher, the role of teachers and tutors, the role of the learner and other learners, the curriculum, the organizational setting, etc.). Much contemporary research that falls under the umbrella term socio-cultural theory (e.g., Collins et al., 1989; Collins, 1991) has been inspired by Vygotsky and his followers.

Contemporary theories for the instructional use of computers need to address our understanding of not only the role of the teacher in the classroom, but also the role of the computer, the design and choice of instructional software, and interactions between the teacher, student and computer. Researchers building on Davydov's notion of learning activity are concerned with such issues. For example, when considering the computer as a means of mediating the process, mechanisms and structure of learning activity, Rubtsov (1993) is concerned with two issues: (1) the content of the microworld; and, (2) the organization of the learning activity, that is, the student-computer, student-computer- teacher, and teacher-computer-students interaction modes. For Rubtsov, the microworld represents the content to be learned (i.e., a practical skill or theoretical concept) which is a concrete school subject to be mastered. Others, such as Dörner (1996) and Sterman (1994) have a broader view of the microworld, in which it is viewed as a facilitating means to help learn about a broader subject matter. The interaction modes needing organization and control by designers include both controlling student interactions with the computer (e.g., automatically regulating the rate of action performance given the stage of learning activity), and interactions between the teacher(s)-computer(s)-student(s).

CSCL requires thoughtful and careful design of learning scenarios and learning support. CSCL also requires technological design encompassing the telelearning environment configuration as well as the tools and services available.

The term telelearning is used to designate new forms of distance or of computer mediated learning, where the distance is not only distance in space or time, as in traditional distance learning, but distance in terms of culture and community, as well as distance in the mediation of learning activities and knowledge objects (e.g., multimedia shared workspaces, multimedia communication, chat boxes, MOOs, etc.) (Bourdeau & Wasson, 1997). Distance learning has evolved from an initial need to ensure equal access to education for all students (Bourdeau & Bates, 1997). The most obvious feature of a distance learning setting is that students and professors do not all meet at the same place at the same time. Individual learning, individual tutoring and asynchronous communication are typical features of a distance learning situation, requiring extensive macro- and micro-instructional design, and a strong student support system. These features, however, do not necessarily dominate in the design of an effective telelearning environment. Many variations of telelearning presence needed to be taken into consideration, including the sense of telepresence in a virtual meeting, the sense of telepresence in interactions with rich multimedia environments, and the sense of telepresence in extensive human collaborations with online knowledge objects and virtual worlds (e.g., online microworlds).

Socio-cultural theories of learning and teaching, as inspired by Vygotsky, have addressed the instructional use of computers. Some of these include situated learning (Lave, 1988), cognitive apprenticeship (Collins et al., 1989), learning by expanding (Engeström, 1987), scaffolding, cognitive artifacts, distributed cognition,and so on. In short, the theoretical heritage for collaborative online learning and telelearning is quite rich.

Cognitive apprenticeship (Collins et al., 1989) has been the foundation for a number of very successful computer-based learning environments. These include Sherlock (Lajoie & Lesgold, 1989; Lesgold, Lajoie, Bunzo & Eggan, 1992) which made use of the concept of scaffolding where learners are provided with just enough assistance to help them construct their own answer to a problem, building on Vygotsky's notion of the zone of proximal development.

In her research on computer-supported collaborative distance learning (CSCdistanceL), Fujk (1995) has carried out a number of studies of two different learning situations in which the computer system has crucial but different roles. For example, one relatively traditional distance learning situation (with highly individual and independent learning) used the computer as a medium for correspondence. A second learning situation was based on problem-oriented, project-based learning. In this situation the computer is used not only to distribute text, but also to articulate individual contribution and to mediate interaction between collaborating students (Fjuk, 1995). Focusing on the interconnection between human actions directed towards the collaborative learning process and human actions directed towards the computer application, Fjuk has developed an analytical framework (Figure 1) for both understanding CSCdistanceL and for designing computer applications to mediate human actions of collaborative learning.

The results of her research show that CSCdistanceL as a learning phenomenon implies a dialectical contradiction between these two aspects which results in a requirement that computer mediation needs to mediate human actions directed towards both individual work and cooperation. Furthermore, she feels that the relationship between these two aspects distinguishes and characterises CSCdistanceL from other forms of learning. As a consequence of this dialectical contradiction, Fjuk concludes that CSCdistanceL application needs to have both a mediating role between the individual learner and peer-students and also between the individual learner and associated learning tasks. Thus, collaborative telelearning can be understood as a medium for inter-human interactions and articulation of individual work. Similar conclusions are made by Rubtsov (1993).

Previous computer supported collaborative learning (CSCL) gives an insight into what collaborative telelearning can provide. Salomon’s work on CSCL (Salomon, 1992, 1993) provides the most complete approach to the study of CSCL in that it is built upon learning theories, relies on observations, raises strong design issues and gives methodological tools for educational research. Salomon’s focus is on the mediation in CSCL, which is a key issue in collaborative telelearning. In his view, collaboration means interdependencies, sharing, responsibility, and involvement. Genuine interdependence is characterised by (Salomon 1992) as follows:

• the necessity to share information, meanings, conceptions and conclusions;
• a division of labour where roles of team members complement one another in a joint endeavour and the end product requires this pooling of different roles; and,
• the need for joint thinking in explicit terms that can be examined, changed, and elaborated upon by peers.

Salomon’s emphasis on genuine interdependence between team members raises the following question: How can such interdependencies be specified and supported in a collaborative telelearning situation?

(Malone & Crowston, 1994) describe coordination theory as an emerging research area focused on the interdisciplinary study of how coordination can occur in diverse kinds of systems. Coordination theory provides a means for specifying (inter)dependencies between, and among, actors, tasks, and resources by identifying a dependency type (e.g., shared resource) and a coordination process (e.g., group decision making) for managing the dependency. In their work, coordination is defined as managing dependencies between activities (Malone & Crowston, 1994), hence they have focused on dependence between activities. Drawing on ideas about activity coordination in complex systems from disciplines as varied as computer science, linguistics, psychology, economics, operations research and organisation theory, they present an analysis that characterises the basic processes involved in coordination. Table 1 shows examples of dependencies between activities and possible coordination processes for managing those dependencies.

Table 1: Dependencies between Activities (from Malone & Crowston, 1994)

The notion that we find especially worth pursuing is that of collaborative learning communities. The idea here builds upon our earlier discussion comparing and contrasting the learning situation to that of a sports team. There are commonly held goals, some explicit and some implicit. One goal involves the survival of the community. Progress is assessed in terms of the entire community, rather than in terms of individuals. Progress without collaboration is extremely unlike.

Designing environments to support collaborative learning is a new enterprise, and we, as a community of professional practitioners, have yet to develop widely help competencies and frameworks for this praxis. Fjuk (1998) argues that:

Now we must indicate more specifically how such theoretical foundations and perspectives can inform a design framework for collaborative learning environments. What we can offer is the beginning of such a framework, a request for collaborative continuation, some indications how learning research might proceed within such a context.

A relevant paradigm for the design of these learning environments can be found in the computer supported collaborative learning (CSCL) literature (Koschmann, 1996). This paradigm focuses on the use of information and communications technology as a mediating tool within a collaborative framework (e.g., peer learning and tutoring, reciprocal teaching, project- or problem-based learning, simulations, games) of learning. This particular approach emphasizes an understanding of language, culture and other aspects of the social setting (Scott, Cole & Engel, 1992) and can be traced in part to a socially-situated perspective of learning (Lave, 1988; Piaget, 1929; Vygotsky, 1978), as we have already indicated. While some suggest that these theories are not compatible, we suggest that they in fact provide an excellent theoretical framework for the design of online collaborative learning environments, especially for complex domains. The common and unifying notion of situated or shared cognition emphasizes the larger environment within which learning takes place; learning is viewed in part as entering into a "community of practice" with a shared language and understanding. The problem of establishing such a community of practice in distributed and online settings is a critical design aspect that has been poorly understood and not especially well implemented in practice.

An example of an approach which aims in this direction can be found in the VIRUE Program, a collaboration of three universities in the domain of marine biology and environmental education (see the following URL for additional information: http://www.umbi.umd.edu/virtue/index.html ). One interesting line of research being followed out in association with this effort involves extending the project to include public school children from the Baltimore area. Children are brought in to the University of Maryland research facilities in the summer and offered an opportunity to help construct a database for bio-diversity in Cheseapeake Bay. The students learn a lot about specific aspects of bio-diversity, how to collect data, how to interpret data, and how the data is inserted into the database and used by professional environmental engineers. These students become members of the community of practice by collecting and interpreting data and then by helping build the database. They become contributors to and partial owners of an important environmental database.

The design methodology that we advocate is based on cognitive apprenticeship (Collins, 1991), providing more learner support and facilitation for less experienced learners and gradually fading support and facilitation for more experienced learners. The learning setting is telelearning, as previously described. The overall learning approach is collaborative in nature, consistent with the theoretical foundations cited earlier. The specific view of collaboration is that tasks and activities should be realistically arranged and mediated in ways that naturally involve and recognize particular interests and skills of learners (Salomon, 1993). That is to say that the collaborations should not be artificially enforced and that the sense of collaboration from the learner's perspective should be such that collaboration is viewed as necessary in order to complete the desired goals. In the marine bio-diversity example, students work in small groups of two and three and rely heavily on the assistance of graduate assistants and teachers to check their work. Nevertheless, they understand the significance of the activities in which they are engaged and recognize their contributions as valued and significant.

Salomon (1992, 1993) argues that successful CSCL depends on the mindful involvement of individual learners as well as on interdependencies between learners. Of primary importance is interdependence, which involves:

• the sharing of information, interpretations, and goals;
• a negotiated division of labour; and
• on-going, joint thinking.

Online collaborative learning environments can and should be designed so as to support these interdependencies. The fluid mediation of such collaborative learning activities is a major challenge for online learning environments. Mechanisms to support synchronisation, exchange of information and documents, and access to tools and services, all need to be as transparent as possible so as to minimise the cognitive overload associated with new tools and technologies.

Viewing collaborative telelearning from a coordination theory (Malone & Crowston, 1994) perspective offers a means of understanding the inter-relationships between actors and entities and how these relationships can and should be supported. We argue that such a model should inform:

• the instructional design of learning scenarios;
• the specification of the design of the online learning environment; and, when possible and appropriate; and,
• the design of intelligent agents to mediate and co-ordinate learning activities between and among learners and tutors.

We conclude with examples from existing systems to illustrate each aspect of this theoretical framework. As we have already said, we do not believe that any large-scale, commercial learning environment has been designed with a systemic view of learning according to a theoretical perspective such as that presented here. We welcome additional examples of those which approximate this perspective, and especially any evaluation research and learning outcomes that have been reported. We also realize that we may have overlooked obvious examples in our review of existing environments and learning research, and we apologize for any such oversights.

We have thus far identified the following elements of an online learning environment for complex domains:

• Provide support for the collaborative construction of knowledge objects, for the collaboration construction and analysis of problem solutions;
• Provide tools to support negotiation of alternatives;
• Provide both public and private feedback support mechanisms;
• Provide mechanisms to share and exchange information, objects, views, etc.;
• Facilitate a meaningful division of labor;
• Support joint, online thinking, commentary, etc.;
• Include meaningful learning scenarios;
• Design authentic problems and legitimate cases as the basis for learning activities;
• Take into account the entire learning environment;
• Support mediation among all the participants; and,
• Foster a sense of a collaborative learning community.

Figure 2 depcits the architecture for the design of a learning environment which satisifies all of the above requirements. This view does not depict the entire learning environment, as it omits reference to the curriculum and the community of learners, but these aspect of the learning environment have been taken into account. This framework has been used to guide the construction of online learning environment for the complex domains of instructional development, environmental management, and regional planning (socio-economic).

Figure 3 shows a screen from one version of a learning environment for resource management in large-scale instructional development projects. This screen shows that learners have multiple views of a complex domain. Consistent with the framework depicted in Figure 2, this environment supports both learner-learner and learner-tutor interactions, as well as learner manipulation and modification of the underlying simulation, after learners have passed beyond the level of advanced beginner in this domain.

One hypothesis which can be explored (and is currently under investigation) within this framework is this: providing learner-centered extensions to web-based databases and learner-manipulatable simulation models will promote learner engagement and result in improved learning effectiveness. If this hypothesis is confirmed there will be wide and pervasive implications for how web-based learning environments and websims are designed and implemented. Currently, most web-based databases used for learning purposes are static (viewable and browsable); they typically cannot be directly altered, extended, or otherwise manipulated online by individuals or groups of learners. Similarly, most web-based simulations do not allow learners to re-construct or alter parts of the underlying model, but doing so is essential in order to test various hypotheses concerning the interaction between structure and behavior in a complex system (Spector & Davidsen, 1997). Providing such capabilities is not a serious technical challenge. Associated legal and ethical implications may need to be clarified (e.g., when patient data is involved), but these are not a serious obstable. The major obstacle is to demonstrate to those who design, implement, fund, and maintain these rapidly proliferating information sources that there is significant learning value in providing support for collaborative extensions and manipulations to web-based databases and web-based simulations. The key is to see that such artifacts can be dynamic learning objects (Laurillard, Lindström, Marton, & Ottosson, 1991).

The evaluation methodology we advocate involves a combination of quantitative and qualitataive methods. However, our methods are primarily qualitative in nature, drawing on distributed cognition (Salomon, 1993) and the general approach found in activity theory (Nardi, 1996) from a learner-centered, phenomenographic perspective (Entwhistle & Ramsden, 1983). Specifically, learners and teachers are interviewed prior to exposure to a learning experience with a participatory, web-based learning environment. That learning environment constitutes the primary mediating object, and a particular feature to be evaluated might be the provision for learner construction of and contributions to a web-based database or websim. Questions are designed to establish current patterns of interactions with other learners and teachers, to determine attitudes about specific technology-based learning capabilities, and to record perceived learning value of those capabilities. During the learning experience and subsequent to the experience with the participatory learning environment, learners and teachers are again interviewed and asked similar questions. Changes are analyzed so as to determine whether and how patterns of interaction and attitudes evolve as a result of participation in the continuing construction of a digital database or simulation on the web.

Thus far, websims for instructional systems project management, regional planning, and environmental management as well as parts of a web-based database for postgraduate orthodontics education have been constructed and evaluated according to the framework and perspective articulated herein. The orthodontics learning environment exists as part of a case- and problem-based curriculum (see the following URL for additional information: http://www.eist.uib.no/orthodl/index.htm ). The curriculum is case-based in two senses. All students are required to become familiar with selected standard cases. Familiarity in this instance means being to identify a particular problem, discuss the treatment rationale, and explain the outcomes of the treatment. Students also treat patients during their course of study, so they are constructing new case histories. These cases will eventually become part of the database. Preliminary results indicate postive effects, although there is not yet enough data to determine the significance of these effects.

The websim for regional planning has been implemented by a University of Bergen graduate student who has collected the data in Egypt. In that case, an attempt has been made to determine to what extent providing learner access to the underlying simulation model facilitates understanding of second-order delays and non-linear feedback mechanisms. Groups of five persons were each allowed to interact with two versions of the websim, one in which access to the underlying model was provided and one in which it was withheld. Spector & Davidsen (1997) argue that such glass-box learning environments are more likely to facilitate learning, and this has also been suggested by Dörner (1996) and others.

Of particular interest in the glass-box approach to system dynamics based learning environments is the notion of double-loop learning (Sterman, 1994). In the first loop of learning in and about complex systems, learners begin to interpret a system dynamics model of a complex domain. This activity is itself challenging. One might characterize such an activity as using an external model to facilitate the creation of internal mental models. The external model is shared and provides a group of learners or decision makers with a common representation which is required for meaningful discussion. Much collaboration can be built into understanding such models, and such collaboration has generally proven useful.

A second step, however, is necessary to advance learners from familiarity with a complex model to genuine understanding of that model. That step invites learners into the model. In most simulation-based learning environments, learners play various roles. They interact with the environment. They typically make changes in settings various parameters and then observe results. In learner-extendable system dynamics enviornments, learners may even reconstruct the model. Learners then become part of the environment. This step advances the learning in an important way. One way of saying this is to say that when learners only make changes in settings in a simulation environment, they are placed in the mode of inferring structure from observed behavior, using the inferred underlying structure to explain and predict behavior. When learners are asked to make changes in the underlying structure, the realization that structure creates behavior is forced upon them. When learners see themselves as part of the underlying structure, they are more likely to search for a wider range of solutions to problem behavior and less likely to reach the cynical conclusions reported by Dörner (1996) for learners who did not have such access and insight.

Such interaction helps learners to keep a tight connection between the structure and behavior of complex systems. They can change the structure to test hypotheses about likely outcomes, and they can run the simulations over time to see that some behaviors cause the system structure to evolve (e.g., some feedback loops may fade in signficance over time, or certain relationships may become more or less linear over time, etc.).

It is premature to argue that a framework such as the one presented here will produce significant, long-lasting, and positive learning effects in and about complex systems. Based on existing data collected on our learning environments and from data reported on some of those which adopt similar approaches, we believe that there is great promise in designing collaborative telelearning environments from a socially-situated learning perspective with heavy emphasis on collaborative learner participation in the creation and modification of knowledge objects and artifacts. We look forward to learning about the results and findings of others in this area.