Exam #3
Social Considerations in the Development, Deployment and Adoption ofWeb-Based Organizational Memories
Andrew GormanCenter for LifeLong Learning and Design
Department of Computer Science andInstitute of Cognitive Science
University of ColoradoBoulder, CO 80309-0430
Abstract
This paper begins with the premise that people want toparticipate in designing their future. While an individualunaided human mind is powerful, real power is derived fromhumans working in conjunction with tools and other hu-mans. Developing computational support for collabora-tion among groups of individuals is a difficult task, but itspotential benefit is tremendous. This paper will describethree broad models of collaborative systems along withexamples of systems that typify each model. The socio-technical aspects of collaboration and participation will bediscussed as they relate to the development, deployment,and adoption of an organizational memory system intendedto support the collaborative design of a new building.
IntroductionThe Discovery Learning Center (DLC) is a new building
being constructed on the campus of the University of Colo-rado at Boulder. This process gathers groups of people whohave varying backgrounds and interests. The goal that iscommon to all is the construction of a new building. How-ever, each has his or her own unique agenda. In order to sup-port such a process, it is desirable to create an informationspace that can be useful to all stakeholders as they partici-pate in the design and construction of the new building. TheDLC information space began as a static repository of de-sign documents and background information. It has sincegrown into a more dynamic system for disseminating designalternatives and gathering feedback from stakeholders. Theadoption of such a system by stakeholders who have littleprior exposure to one another can be a very complicatedprocess, which is affected by both technical and social fac-tors.
Information Access is Necessary but notSufficient
In today's workplace, people need to know how to accessinformation. However, simply knowing how to access in-formation falls short of what is truly needed in today’sworkplace. For example, in the President’s InformationTechnology Advisory Committee report (PITAC, 1999),there is a call for ubiquitous information access. This call
for accessibility needs to be extended to include the under-standing that the key to the future lies not only in greateraccess to information, but in greater support for knowledgeconstruction (Scardamalia & Bereiter, 1994). Although thereis value in such an access model, its focus is incomplete. Itis based on an impoverished view that relevant knowledgealready exists, waiting to be accessed. In order to truly gainthe benefits of information technology, what is needed is notsimply greater access to information, but a greater ability foraverage people to construct and distribute new knowledge(Arias, 1999 (in press)).
People Want to Participate
How can more than 261 million individual Americans defineand reconcile their needs and aspirations with community val-ues and the needs of the future? Our most important finding isthe potential power of and growing desire for decision proc-esses that promote direct and meaningful interaction involvingpeople in decisions that affect them. Americans want to takecontrol of their lives (PCSD, 1996, p.7).
This finding of the President's Council on SustainableDevelopment (PCSD) supports the claim that somethingmore is needed than access alone. Therefore, an importantchallenge for future information technology is to enablestakeholders of problems to become involved informed par-ticipants (Brown, Duguid, & Haviland, 1994).
To make informed participation a reality, we need supportfor new forms of knowledge creation, integration, and dis-semination. People seldom explore large repositories of in-formation in the abstract (Fischer, Lemke, McCall, &Morch, 1996; Moran & Carroll, 1996). Instead, informationis typically sought in response to breakdowns encounteredduring meaningful, real-life activities (Fischer, 1994; Pop-per, 1965). By overcoming such breakdowns, new knowl-edge is created, which then must be integrated with anyknowledge that may have been generated during prior break-downs. This cycle leads to the creation and evolution of richinformation spaces that can empower interested communitymembers as they “take control of their lives.”
Exam #3
2
M1-M3 Models of ParticipationA typical model of information sharing (e.g., (Ackerman
& Malone, 1990; Ackerman & McDonald, 1996)) focuseson experts sharing information with non-experts. This is
represented by the M1 model (seen in figure 1) in which aclass of experts controls the production of information andindividuals act as consumers whose only need is that of ac-cess.
In the M2 model (figure 2), all information is funneled
through a single person or small committee. This is typicalof information generated in an open source model of soft-ware development (Raymond, 1999). Here there are manycontributors, but only a few (or one) that integrates feedbackfrom the community back into a coherent structure. Therehas been much success using this model in open sourcemovements (Fielding, 1999; Torvalds, 1999) and while thisis an advantage over the M1 model, in terms of collaborativeconstruction, it can lead to problems of scalability. Fur-thermore, there needs to be a highly dedicated person that hasthe full-time responsibility of analyzing and structuring allof the feedback.
The M3 model (figure 3) can theoretically support distrib-
uted collaboration in a more direct way. In this model, thereis no “gatekeeper.” All contributors have the ability to addcontent directly. Developing systems that support this typeof collaboration can be extremely difficult. One way of pro-viding structure to collaboratively constructed information isby codifying the knowledge and expertise of the gatekeeperdescribed in the M2 model. Another approach is to distrib-ute this responsibility among the community members byestablishing policies (Edwards, 1996) to govern the con-struction, organization and use of information.
Human Cognition and TasksThe memory of an individual can be roughly divided into
two categories: short-term memory (STM) (also described as
Figure 1 - The M1 Model of Collaboration
Figure 3 - The M3 Model of Collaboration
Exam #3
3
working memory) and long-term memory (LTM).1 STM isrelatively small, typically thought to have a capacity of 7 ±2 chunks of information (Miller, 1956). In contrast, LTM isvirtually unlimited (Matlin, 1998). In an information proc-essing theory of human cognition (Pinker, 1997), dataneeded for a given task is typically activated and retrievedfrom LTM and then held in STM while it is actively used.In this process, information is constantly being swapped inand out of STM as new information is constructed and en-coded into LTM. This model of human memory is analo-gous to register and disk storage used in modern-day com-puters.
Expert behavior is often based on a well-developed tech-nique, or pneumonic, for encoding and retrieving informa-tion. Intelligent behavior, therefore, is often attributed inlarge part to being able to effectively transfer informationinto LTM so that it can later be activated for future use inSTM. Because of the limitations of STM and human atten-tion, cognition can be viewed as a limited scarce resourcethat needs to be allocated as properly during task perform-ance. This view is analogous to operating systems thatallocate computer resources during the execution of a processor sub-process.
Given that human cognition is limited, it makes sensethat the nature of a task (i.e., the cognitive resources it de-mands) affects our ability to perform the task. Over thecourse of human history, cultures have invented tasks thatnot only push, but also transcend the limits of human cog-nitive capabilities. Because of this, humans have needed todevelop cognitive artifacts (Norman, 1991) and systems toaid in their artificial tasks. For example, long division is anartificial system that produces such a cognitive load that it isdifficult to solve even a moderately advanced problem with-out the aid of cognitive artifacts such as memory aids likepaper and pencil. With the advent of computers, some ofthese artificial tasks have been codified so that they maynow be performed with computational devices such as hand-held calculators.
Distributed CognitionDistributed cognition (Brown, et al., 1993; Fischer, 1995;
Hewitt & Scardamalia, 1996; Hutchins, 1993; Norman,1993; Salomon, 1993) emphasizes that the heart of intelli-gent human performance is not the individual human mindin isolation but the interaction of the mind with tools andartifacts as well as groups of minds in interaction with eachother. It is important to understand the fundamental differ-ence between these two forms of distributed cognition.When distributed cognition is at work between the individualhuman mind and cognitive artifacts, it often functions wellbecause the knowledge an individual needs is distributed be-tween her/his head and the world (e.g., calculators, addressbooks, e-mail messages, filing cabinets). On the other hand,when cognition is distributed among groups of minds, agroup has no head, no single mind to store the informationabout this distribution of knowledge, which is available to
1 Sensory memory is also described in the cognitive psy-
chology literature, but this is outside the scope of this paper.For more information on sensory memory see (Matlin, 1998)
all members of the group. In this case, externalizations arecritically more important. Externalizations (1) create a recordof our mental efforts, one that is “outside us” rather thanvaguely in memory and (2) represent artifacts that can talkback to us (Schön, 1992) and form the basis for critique andnegotiation. These can be thought of as cognitive artifactsfor groups.
Although creative individuals are often thought of asworking in isolation, the role of interaction and collabora-tion with other individuals is critical (Engelbart, 1995).Creative activity grows out of the relationship between anindividual and the world of his or her work, and out of theties between an individual and other human beings. The pre-dominant activity in complex problem solving is that par-ticipants teach and instruct each other (Greenbaum & Kyng,1991). Because complex problems require more knowledgethan any single person possesses, it is necessary that allinvolved stakeholders participate, communicate, and collabo-rate with each other. For example, during the design andconstruction of a new building on a college campus, thereare building architects, tenants, collage administrators, andgovernment legislators, all of whom have different agendasand background knowledge. Furthermore, communicationbreakdowns are often experienced because stakeholders be-longing to different cultures use different norms, symbols,and representations (Snow, 1993). Such a setting is gov-erned by a symmetry of ignorance (Rittel, 1984) in whichall stakeholders are aware that even though they each possessrelevant knowledge, none of them has all the relevantknowledge. Each participant must act as a reflective practi-tioner rather than as an all-knowing expert (Schön, 1983).
Organizational Memories for SupportingLong and Short Term Collaboration
The term “organizational memory” has no clear or agreedupon definition within the computer science literature(Ackerman & Halverson, 1998). In this paper, the follow-ing operational definition will be used:
Organizational memory provides a shared information spacethat supports a group of people (an organization) to do work.The information space should be "living" in the sense that it isan evolving product of the work done by the members of theorganization as opposed to simply being a static storage of in-formation (Fischer, 1998a)
In this definition, an organizational memory can beviewed as a cognitive artifact that provides an externalizationfor groups of minds interacting with each other. One couldargue that an organization or group’s memory exists inde-pendent of any explicit externalization. Every group, organi-zation, or culture has tacit knowledge that helps govern ordefine acceptable behavior, standard operating procedures, andsocial norms. It can be very beneficial to identify and makeexplicit a groups tacit knowledge, but this not the concernof this paper. Instead, this paper will consider the use oforganizational memory systems supporting the collaborationof a heterogeneous group of stakeholders during the designof an artifact.
Exam #3
4
Media and Systems Supporting OrganizationalMemory and Collaboration
A challenge for supporting collaboration is in providing amechanism that allows various participants to integrate theirperspectives (Stahl, 1993) in a meaningful way. Supportinginformed participation requires processes that integrate theindividual and the group knowledge through collaborativeconstructions. Information spaces need to be constructedcollaboratively (Scardamalia & Bereiter, 1994) and integratedinto the work and social practices of the community (Lave,1988). These collaborative constructions result in workproducts that are enriched by the multiple perspectivesemerging through community discourse.
Ackerman (Ackerman, 1998) describes the development ofAnswer Garden, a system for capturing and managing anorganizations knowledge and expertise. One of the primarymotivating forces behind such a system is an organizationwishing to share expertise and avoid reinventing the prover-bial wheel. The most common scenario for using a systemlike Answer Garden is when a worker is faced with a prob-lem to which they have no answer. Answer Garden supportsthis dilemma in two possible ways: it makes a previouslyrecorded solution available for retrieval or, in the absence ofsuch a solution, it provides access to the person who islikely to have the appropriate expertise. This approach fo-cuses on recording and structuring knowledge so that it maybe available for later use. However, this model makes astrict distinction between those who possess knowledge andthose who do not. There are two separate roles in this sce-nario and there is no intersection seen between these twogroups. This resembles the M1 model of collaborationwhere a class of experts provides access to information for aclass of non-experts. While systems such as Answer Gardenmay be useful in many situations, it falls short of support-ing groups of collaborators where each person is simultane-ously an expert and a non-expert (e.g., an expert buildingarchitect, but a non-expert campus planner). What is neededin this case is a system that is open to all users.
The DynaSites system (Ostwald, 1997) is another type ofsystem for the creation of dynamic and user-extensible web-based information spaces. This system is designed to sup-port M3 types of collaboration where all users can directlycontribute to the construction of new knowledge. One of thefundamental problems of sustaining a useful open informa-tion space is that of maintaining and organizing the informa-tion. One approach to this problem is the use of a gate-keeper (Raymond, 1999), but as mentioned earlier, this hasproblems of scalability. Another approach is to view aninformation space as a seed (Fischer, 1998b). In this ap-proach, an information space begins as a seed which evolvesover time through use. At some point, the informationspace grows to be so unwieldy that it is no longer useful(i.e., it is too difficult to find relevant information). At sucha time, the information needs to be restructured, or re-seeded.However, in practice, this is difficult to accomplish. Ques-tions of what information is pruned from the space can bedifficult to answer. Usage data can help answer questionslike these, but still, any major restructuring of critical in-formation source can potentially be very disruptive.
Participation RevisitedAll of the discussion thus far has been based on the prem-
ise that people want to participate. After all, according to thePresident's Council on Sustainable Development (PCSD,1996, p.7), ”Americans want to take control of their lives.”However, just because people want to take control, does thatmean that they actually do? What evidence exists to supportthis conclusion? According to the Institute for Democracyand Electoral Assistance, voter turnout in US presidentialelections shows a downward trend in participation (IDEA,1999). In fact, 1996 has had the lowest percentage of voterturnout (47.2%) in over 50 years2. However, is voter turn-out in US presidential elections an indicator of people’s de-sire and willingness to participate in designing their future ata local, more intimate level? Not necessarily, but it doesraise some questions.
One factor affecting the use of group information spacesis that of perceived utility. The utility of a system can beviewed as a function of perceived benefit and required effort.Two fundamental question that must be asked (and answered)are, “Who does the work?” and “Who receives the benefit?”(Grudin, 1994). A classic example is found in the practice ofsoftware documentation. The software developer who is re-quired to document his or her code may perceive no directbenefit in doing so. For this reason, software documentationoften goes undone.
One way of increasing utility is to decrease the required ef-fort. Another is to increase the perceived benefit. Traveen, etal. (Terveen, Hill, Amento, McDonald, & Creter, 1997)have sought to do both. They have developed PHOAKS(People Helping One Another Know Stuff), which is a rec-ommended system, or social filter. This system was able toautomatically search Usenet groups and find positive refer-ences, or “recommendations” of Web sources. In this case,user effort is negligible: no effort was required to input ororganize information. What about it’s perceived benefit,though? As it turns out, users perceived considerable benefit,but the programmatic identification and categorization ofsources was imperfect. At times, PHOAKS’ recommenda-tions were seen to be misleading or the sources that wererecommended became outdated (dead links). The mechanismfor correcting these problems was a manual process. In anM2-like fashion, users would email their feedback to thePHOAKS administrators, who would then evaluate and re-spond as they saw fit. While this method was somewhatsuccessful, the issue of scalability and sustainability becameobvious: as the use of the system grew, could they keep upwith user demand? By analyzing usage patterns and types ofemail requests, the PHOAKS developers were able to createmechanisms that allowed users to directly modify the con-tent of their system (Hill & Terveen, 1997). One interestingpoint that was made by these researchers was that there wasa marked distinction in consumer / producer roles for usersof their system. Specifically, they found that only a smallminority of users expended the effort of contributing modifi-cation.
2 Statistics were only available dating back to 1948.
Exam #3
5
The Design and Construction of the Dis-covery Learning Center
The Discovery Learning Center (DLC) is a building beingconstructed on the campus of the University of Colorado,Boulder. It is being built to support a broad-reaching visionfor the College of Engineering and Applied Science knownas the Discovery Learning Initiative. (DLI) In an effort toembody the spirit of the initiative, a web-based informationspace was created (Gorman, 1999) in May of 1999 to sup-port the informed participation of all stakeholders involvedin the project. Stakeholders in the project include the Deanand associate Deans of the college, perspective tenants, theuniversity’s facility management group, state legislators,local industry and private donors, and the building architectsalong with their consultants.
This information space was originally maintained in anM2-like fashion as a collection of static letters, memos,design documents, space requests from prospective tenants,meeting minutes, and other resources related to the designand construction of the building. It was designed as an orga-nizational memory for the project that would support theinformed participation of interested stakeholders. Using theM2 model, content was contributed by stakeholders and thenorganized and made statically available by a gatekeeper, orchief designer (Raymond, 1999).
Several problems exist in trying to design such an infor-mation space:1. There was only a weak or ephemeral coalescing of work
groups. As Landauer (Landauer, 1995) points out, thetrouble with so many computer systems stems from alack of user-, or in this case, group-centered design.
2. With such diverse groups of target users, it is difficultto organize the information so that each group of userscan find the information they need. Norman (Norman,1993) suggests fitting the artifact to the person. What isto be done about designing an artifact for an extremelyheterogeneous work group?
3. There were no well-established processes in the project– processes were being defined on the fly. Without es-tablished processes, it is impossible to perform taskanalysis (Lewis & Rieman, 1993) so that informationcould be structured a priori.
Through reflection on these problems, it seems clear thatstatic nature of this information space made it difficult towork with the unknowns that were present.
Participation in the Discovery Learning CenterIn September of 1999, a more dynamic component was
added to the predominantly static DLC site. During thistime, the architects and their consultants were designing thelab spaces for the initial tenant groups. This was to be aniterative process where alternate design were created eachtenant group’s lab. These designs were then to be reviewedby the tenant groups who were then responsible for provid-ing feedback to the architects.
In order to provide organizational memory support for thisprocess, a mechanism disseminating design alternatives andcollecting feedback was added to the site. A typical designfeedback forum consisted of one or more proposed designartifacts attached to a discussion forum. This was intended
to encourage discussion and collect feedback regarding a par-ticular design artifact (see Appendix A). A user could selectone of the design artifacts by clicking on its thumbnail im-age. This would generate an enlargement of the artifact in aseparate window, which could then be inspected and com-pared with other competing designs. (see Appendix C). Usercomments were collected and structured as a threaded discus-sion. Each entry would show the subject, time and the nameof the contributing user. In order to provide some awarenessof who was contributing what, users could add profile in-formation describing their stake in the project along withcontact information such as email address and home pageaddress (see Appendix B).
A cursory examination of the site’s design feedback fo-rums revealed that they generated very little discussionamong the stakeholders. At the time of writing this paper, itis unclear exactly why this tool was not used -- results froma survey (see appendix D) have just begun to return.
Speculation on Reasons for Lack of ParticipationWhile it is still a little early to identify a conclusive rea-
son for the system’s lack of use, some patterns and warningsigns have begun to emerge that point to possible causes.Grudin (Grudin, 1994) in an evaluation of the social dynam-ics of groupware systems, identified factors affecting theadoption of group-oriented systems. Some of these factorsmay very well have been an issue for the adoption of thissystem by its targeted users. There may have been little per-ceived benefit for using the feedback system. Because ofmedia competition (Whittaker, 1996) and ill-defined proce-dures, many tenants may have simply found it easier to justuse email, phone messages, or impromptu face-to-face meet-ing to deliver their feedback. The system may have felt tooformal so the added work of entering a well-developed cri-tique of their design options may have been too great(Erickson, et al., 1999).
Critical mass may also have been an issue affecting theadoption of this system. Most group systems develop a userbase through a grass roots effort (Palen, 1999) and/or a sub-stantial ramp up period. Given the fast-paced nature of thisproject and ephemeral quality of collaborative groups, theremay not time to establish enough critical mass to crate asustainable user group.
The use of this system may have created too much disrup-tion in the normal social processes that existed in the col-lege. Tenant faculty members may have been use to com-municating more directly with the Dean’s office and the useof a information system that would result in a persistentrecord of their communications may have been too formaland threatening.
Finally, groupware systems require a more careful intro-duction into the workplace than do typical software systems.Unless there is enough time to allow a grassroots adoptionto take place, any deployment of such a system will requirea significant top-down approach. In other words, with a fast-paced project there is not enough time for a more naturaladoption of group memory system. In such a case, the roleof the system needs to be more explicitly built into theprocess.
Exam #3
6
ConclusionThere is a variety of models that can be useful in the con-
struction of organizational memory systems. The M1 modelcan be useful for recording and disseminating an organiza-tion’s knowledge, but it is not very useful in supportingcollaboration. An M2 model can help provide structure andopenness to a system, but scalability issues arise, as thefeedback generated becomes too great. The M3 model pro-vides the greatest openness and support for direct collabora-tion. However, it can be very challenging to develop sys-tems with the ability to govern the creation and modificationof information as well as the structuring and retrieval of thatinformation.
These models can be viewed as points along a continuumrather than as discrete choices. As seen with PHOAKS andthe DLC, systems can be incrementally enhanced to supportgreater levels of collaboration. Usage data can be very help-ful in identifying opportunities for developing greater levelsof collaborative functionality in a system.
One of the most important considerations in the use of anorganizational memory system is the social environment inwhich it will be used. These types of systems generally taketime to develop roots and gain critical mass. Therefore, iftime for deployment is scarce, other methods must be en-forced. Having a clear work process defined can help in thedeployment phase, but whether or not a system gets usedmay ultimately depend on other factors, such as media com-petition, perceived benefit, and minimizing extra effort byproviding smooth integration into existing work practices.
ReferencesAckerman, M.S. (1998). Augmenting Organizational
Memory: A Field Study of Answer garden. ACM Transac-tions on Information Systems, 16(3), 203 - 224.
Ackerman, M.S., & Halverson, C. (1998). Consideringan organization's memory, Proceedings of the ACM 1998conference on Computer supported cooperative work (pp. 39- 48).
Ackerman, M.S., & Malone, T.W. (1990). Answer Gar-den: a tool for growing organizational memory, Proceedingsof the conference on Office information systems (pp. 31-39),Cambridge, MA USA.
Ackerman, M.S., & McDonald, D.W. (1996). AnswerGarden 2: Merging Organizational Memory with Collabora-tive Help. In M.S. Ackerman (Ed.), Proceedings of ACMCSCW'96 Conference on Computer-Supported CooperativeWork (pp. 97-105). New York: ACM Press.
Arias, E., Eden H., Fischer, G., Gorman, A., andScharff, E. (1999 (in press)). Beyond Access: Informed Par-ticipation and Empowerment, Proceedings of the conferenceon Computer Supported Collaborative Learning (CSCL'99).
Brown, A.L., Ash, D., Rutherford, M., Nakagawa, K.,Gordon, A., & Campione, J.C. (1993). Distributed Exper-tise in the Classroom. In G. Salomon (Ed.), DistributedCognitions: Psychological and Educational Considerations.Cambridge, England: Cambridge University Press.
Brown, J.S., Duguid, P., & Haviland, S. (1994). To-ward Informed Participation: Six Scenarios in Search ofDemocracy in the Information Age. The Aspen InstituteQuarterly, 6(4), 49-73.
Edwards, K.W. (1996). Policies and roles in collabora-tive applications, Proceedings of the ACM 1996 conferenceon on Computer supported cooperative work (pp. 11 - 20).
Engelbart, D.C. (1995). Toward Augmenting the HumanIntellect and Boosting our Collective IQ, Communicationsof the ACM, Vol. 38 (pp. 30-33).
Erickson, T., Smith, D.N., Kellogg, W.A., Laff, M.,Richards, J.T., & Bradner, E. (1999). Socially translucentsystems: social proxies, persistent conversation, and thedesign of "babble", Proceeding of the CHI 99 conference onHuman factors in computing systems: the CHI is the limit(pp. 72 - 79).
Fielding, R.T. (1999). Shared leadership in the Apacheproject. Communications of the ACM, 42(4), 42 - 43.
Fischer, G. (1994). Turning Breakdowns into Opportuni-ties for Creativity. Knowledge-Based Systems, Special Issueon Creativity and Cognition, 7(4), 221-232.
Fischer, G. (1995). Distributed Cognition, LearningWebs, and Domain-Oriented Design Environments. In J.L.Schnase & E.L. Cunnius (Eds.), Proceedings of CSCL'95:The First International Conference on Computer Support forCollaborative Learning (pp. 125-129). Mahwah, NJ: Law-rence Erlbaum Associates, Inc.
Fischer, G. (1998a). Organizational Memory. Web Pageathttp://Seed.cs.colorado.edu/Dynagloss.makeGlossPage.fcgi$Organizational_Memory
Fischer, G. (1998b). Seeding, Evolutionary Growth andReseeding: Constructing, Capturing and Evolving Knowl-edge in Domain-Oriented Design Environments. AutomatedSoftware Engineering, 5(4), 447-464.
Fischer, G., Lemke, A.C., McCall, R., & Morch, A.(1996). Making Argumentation Serve Design. In T. Moran& J. Carrol (Eds.), Design Rationale: Concepts, Techniques,and Use (pp. 267-293). Mahwah, NJ: Lawrence Erlbaum andAssociates.
Gorman, A. (1999). The Discovery Learning Center.Web Page at http://DiscoveryLearning.Coloardo.Edu/
Greenbaum, J., & Kyng, M. (1991). Design at Work:Cooperative Design of Computer Systems. Hillsdale, NJ:Lawrence Erlbaum Associates, Inc.
Grudin, J. (1994). Groupware and social dynamics: Eightchallenges for developers. Communications of the ACM,37(1), 92-105.
Hewitt, J., & Scardamalia, M. (1996). Design Principlesfor the Support of Distributed Processes. Paper Presented ina Symposium Distributed Cognition: Theoretical and Practi-cal Contributions, at the Annual Meeting of the American
Exam #3
7
Educational Research Association, New York City. athttp://csile.oise.on.ca/abstracts/distributed/
Hill, W.C., & Terveen, L.G. (1997). Involving remoteusers in continuous design of web content, Proceedings ofthe conference on Designing interactive systems: processes,practices, methods, and techniques (pp. 137 - 145).
Hutchins, E.L. (1993). Learning to Navigate. In S.Chaiklin & J. Lave (Eds.), Understanding Practice (pp. 35-63). Cambridge, UK: Cambridge University Press.
IDEA (1999). Voter Turnout from 1945 to 1997: AGlobal Report on Political Participation. athttp://www.idea.int/Voter_turnout/index.html
Landauer, T.K. (1995). The Trouble with Computers.Cambridge, MA: MIT Press.
Lave, J. (1988). Cognition in Practice. Cambridge, UK:Cambridge University Press.
Lewis, C., & Rieman, J. (1993). Task-Centered User In-terface Design: A Practical Introduction. Boulder, Colorado:University of Colorado, Boulder.
Matlin, M.W. (1998). Cognition (4th ed.). New York:Harcourt Brace College Publishers.
Miller, G. (1956). The Magic Number Seven, Plus orMinus Two" Some Limits on Our Capicity for ProcessingInformation. Psychology Review, 63, 81-97.
Moran, T.P., & Carroll, J.M. (1996). Design Rationale:Concepts, Techniques, and Use. Hillsdale, NJ: LawrenceErlbaum Associates, Inc.
Norman, D. (1991). Cognitive Artifacts. In J.M. Carroll(Ed.), Designing Interaction (pp. 17-38). Cambridge: Cam-bridge University Press.
Norman, D.A. (1993). Things That Make Us Smart.Reading, MA: Addison-Wesley Publishing Company.
Ostwald, J. (1997). Dynasites. athttp://Seed.cs.colorado.edu/dynasites.documentation.fcgi$node=dynasites.doc.home
Palen, L. (1999). Social, individual and technological is-sues for groupware calendar systems, Proceeding of the CHI99 conference on Human factors in computing systems: theCHI is the limit (pp. 17 - 24).
PCSD (1996). Sustainable America: A New Consensusfor Prosperity, Oppertunity and a Healthy Environment forthe Future. Washington, D.C.: President's Council on Sus-tainable Development.
Pinker, S. (1997). How the Mind Works. New York: W.W. Norton & Company.
Popper, K.R. (1965). Conjectures and Refutations. NewYork, Hagerstown, San Francisco, London: Harper & Row.
Raymond, E.S. (1999). The Cathedral & The Bazaar:Musings on Linux and Open Source by an Accidental Revo-lutionary. Sebastopol, CA: O'Reilly.
Rittel, H. (1984). Second-Generation Design Methods.In N. Cross (Ed.), Developments in Design Methodology(pp. 317-327). New York: John Wiley & Sons.
Salomon, G. (1993). Distributed Cognitions: Psycho-logical and Educational Considerations. Cambridge, UnitedKingdom: Cambridge University Press.
Scardamalia, M., & Bereiter, C. (1994). Computer Sup-port for Knowledge-Building Communities. Journal of theLearning Sciences, 3(3), 265-283.
Schön, D. (1992). Designing as reflective conversationwith the materials of a design situation. Knowledge-BasedSystems Journal, Special Issue on AI in Design, 5(1), 3-14.
Schön, D.A. (1983). The Reflective Practitioner: HowProfessionals Think in Action. New York: Basic Books.
Snow, C.P. (1993). The Two Cultures. Cambridge, UK:Cambridge University Press.
Stahl, G. (1993). Interpretation in Design: The Problemof Tacit and Explicit Understanding in Computer Support ofCooperative Design. Ph.D. Dissertation., University ofColorado at Boulder.
Terveen, L., Hill, W., Amento, B., McDonald, D., &Creter, J. (1997). PHOAKS: A System for Sharing Rec-ommendation. Communications of the ACM, 40(3), 59-62.
Torvalds, L. (1999). The linux edge. Communicationsof the ACM, 42(4), 38 - 39.
Whittaker, S. (1996). Talking to Strangers: An Evalua-tion of the Factors Affecting Electronic Collaboration, Pro-ceedings of the ACM 1996 conference on on Computer sup-ported cooperative work (pp. 409 - 418).
Exam #3
Appendix A
DLC Design Feedback Forum Centered Around Design Artifacts
Exam #3
Appendix BDLC User Profile
Exam #3
Appendix CDLC Design Artifact
Exam #3
Appendix DDLC Survey
For the following questions rate your answers on a scale from 1 to 5, where1 is very little and 5 is very much.
1) How often did you use the following modes of communication throughoutyour involvement in this project.
a) Face-to-face formal/scheduled meetingsb) Face-to-face informal/impromptu meetingsc) Telephone (including voice mail)d) Inter-office maile) Emailf) Discussion forum on the Discovery Learning Web Site
2) How valuable were the following modes of communication throughout yourinvolvement in this project.
a) Face-to-face formal/scheduled meetingsb) Face-to-face informal/impromptu meetingsc) Telephone (including voice mail)d) Inter-office maile) Emailf) Discussion forum on the Discovery Learning Web Site
3a) During your involvement in this project, have you used the DLC Web site(y or n)?
If you answered "yes" to #3a
3b) how many times did you visit the site?
a) 1 - 5 timesb) 6 - 10 timesc) 11 - 20 timesd) > 20 times
3c) what did you fine most useful at the site?
3d) what did you fine least useful at the site?
3e) what what would you say is the main purpose of the site.?
4) if you answered "no" to questions #3, were you aware of the DLC WebSite's existence?
5) On a scale of 1 to 5 please rate your level of involvement during thedesign of YOUR lab space.
5b) How often did you use the following modes of communication during thedesign of YOUR lab space.
a) Face-to-face formal/scheduled meetingsb) Face-to-face informal/impromptu meetingsc) Telephone (including voice mail)d) Inter-office maile) Emailf) Discussion forum on the Discovery Learning Web Site
6) Were you aware that proposed floor plans for YOUR lab were available onthe DLC web site (y or n)?
7) Please describe in your own words your involvement in the design of YOURlab space.
Exam #3
8) Please add any additional comments that you may have about the project.
