UCI

Knowledge Management for Software Design

David F. Redmiles

David M. Hilbert, Jason Robbins, Michael Kantor, Shilpa Shukla

Information and Computer ScienceUniversity of California, IrvineIrvine, California 92697-3425

redmiles@ics.uci.eduhttp://www.ics.uci.edu/~redmiles/ or

http://www.ics.uci.edu/eden/

UCI

Overview• Motivation and Problem Statement

• Approach — Human-Centered Software Development

• Usage Scenario

• Systems’ Theory, Implementation, Current Directions– Design: Argo Software Architecture Design

– Usage Monitoring: Expectation-Driven Event Monitoring

– Review: Knowledge Depot Project Awareness Tool

• Conclusions

UCI

Motivation and Problem Statement

UCI

Motivation• Software engineering has become harder because

software is being applied to increasingly complex problem situations.

• Complexity is measured in terms of people and workplace settings.

UCI

The Challenge of Increased Complexity in Software

• Changed Nature– Software applications have moved from predictable, static

applications to unpredictable, interactive tools.

• Expanded Scope– Software applications are no longer intended for a single user or

small group of highly trained users.

• New Problem Domains– Software applications are no longer restricted to “simple” problems

—mathematical models with a few input variables.

• Increased User Sophistication– Wide availability of personal computers has increased users’

awareness of computer potential.

UCI

Changed Nature• Software applications have moved from static to

interactive tools.

• Human behavior is “unpredictable”—people do not always use tools as anticipated by the developers.– Incomplete understandings of tools (system model).

– Incomplete understanding of problem situations (situation model).

– Other cognitive limitations (e.g., attention).

UCI

Expanded Scope• Software applications are no longer intended for a

single user or small group of highly trained users.– General-purpose tool sets are applied in a variety of situations in

mass markets.

– Special-purpose tools cannot assume long training periods because of high personnel turnover and frequent tool updates.

– In any tool, the number and variety of features implies that many are infrequently accessed.

• Moreover, new, cooperative work tools coordinate the activities of several people.

UCI

New Problem Domains• Software applications are no longer restricted to

“simple” problems—mathematical models with a few input variables.

• Technological advances make it possible to put software on every desk.– Tools must be able to coordinate with many dimensions of a

workplace: users’ relying on information from other artifacts in the work setting, other computer tools, and other people.

– New tasks challenge existing computer representations and algorithms.

– Models of tasks must be more rapidly developed.

UCI

Increased User Sophistication• Wide availability of personal computers has increased

users’ awareness of computer potential.

• Attractive interfaces in games and other personal software have raised users’ expectations.

• Competitive markets for many applications enable users to be “discretionary”—more demanding.

UCI

New Demands on Design• Interactive tools require a human-centered, not

artifact-centered, focus.It is more difficult for designers to understand how people will use tools in their workplace than it is for them to understand the domain the tools will support.

• User evaluation is essential to informing tool development.

Fluctuating requirements and other failures should be expected, good “fit” must evolve, new methods for collecting usage data and feedback can make evaluation more tenable and cost-effective.

UCI

New Demands on Design (cont.)• Design is an evolutionary process and continues after

delivery.In new problem domains, complexity unfolds incrementally. Moreover, after tools are introduced into a community, the users develop new approaches to their problems, hence new requirements for tools.

• Users and other stakeholders must share ownership of systems.

Acceptance of systems often fails for social and organizational reasons. Stakeholders must feel their intentions have been fairly negotiated.

UCI

New Demands on Design (cont.)• Developers need information about good design in

new domains.– Developers require knowledge both about problem domains and

potential solutions. Possible solutions grow daily with new software architecture techniques.

UCI

Approach• In a human-centered software development process,

feedback comes from all stakeholders.

• Stakeholders reflect on an artifact that evolves from sketchy designs to partial, and eventually, complete prototypes.

• Feedback from human stakeholders is facilitated by tools; tools can provide data about actual usage and knowledge evolved from previous development experiences.

UCI

Human-Centered Software Development

Design

UsageReview

ARGOEDEM

EDEMKNOWLEDGE DEPOT

incorporatereviews into

deploy prototype withexpectation agents

deposit observations

prototype and expectation agents

prototype with expectation agentsexpectation agents’ reports andother information

perform direct observations,interviews, other manualusability methods

perform other design reviews,including email discussions

design

UCI

Communication and Knowledge in the Lifecycle

• The human-centered lifecycle interprets software development as a process of communication (of knowledge):– Knowledge about what constitutes a good design is provided to

developers.

– Knowledge about actual system usage and user commentary is reported.

– Knowledge about system usage and other related design knowledge is discussed among stakeholders.

• Three software systems support these activities: Argo, EDEM, and Knowledge Depot.

UCI

Usage Scenario

UCI

Developing a Travel Voucher System

• A software developer architects a new application, a travel voucher system.

• Software developers author expectations about the usage of the system; they are deployed with the system.

• The agents monitor system usage and may solicit user comments.

• Usage information, comments, and other design information is collected and reviewed.

• Design knowledge provides feedback for a redesign of the system.

UCI

Design• A software developer architects a new application, a

travel voucher system.

UCI

Design and Usage• Software developers author expectations about the usage of

the system; they are deployed with the system.• This agent fires when a user edits the City or State fields

while the Zip field is empty

Trigger

Guards

Actions

}}}

AuthoredAgents {

UCI

Design and Usage• Selection of events

simplifies agent definition.

• Agents are deployed independently of applications.

UCI

Usage• The agents monitor system usage.

UCI

UsageAgent Notification (Optional)

• Agents may post messages.

UCI

UsageUser Response (Optional)

• Users may provide feedback.

UCI

Review• Usage information,

comments, and other design information is collected and reviewed.

UCI

Redesign• Design knowledge provides feedback for a redesign of

the system.

UCI

Summary• The changing nature of software places new demands on

software engineering.

• An empirically-driven, evolutionary and human-centered model is necessary.

• The human-centered lifecycle interprets software development as a process of communication (of knowledge):– Knowledge about what constitutes a good design is provided to

developers.– Knowledge about actual system usage and user commentary is reported.– Knowledge about system usage and other related design knowledge is

discussed among stakeholders.

UCI

Argo

Software Design Environments

UCI

Underlying Theories of Designers’ Cognitive Needs

• We have tried to identify what makes design challenging for people– Design feedback: Reflection-in-Action [Schoen]

– Design process support: Opportunistic Design [Guindon]

– Design perspectives: Comprehension and Problem Solving [Kintsch]

• We have specialized and applied these theories to software architecture

UCI

Design Feedback — Reflection-in-Action

• Observation: Designers work through the design– Designers rarely produce a design fully formed

– Incrementally frame a partial design; evaluate it; revise

• Suggested requirements on design environments– Support the designer in framing a partial solution

– Support the designer in evaluating it

– Integrate manipulation and analysis

• Support in Argo: Editors, Critics, To Do List

UCI

Construction with Critical Feedback

UCI

Design Process Support - Opportunistic Design

• Observation: Designers deviate from prespecified processes– Emerging issues change the design process

– Cognitive costs explain some task reorderings

• Suggested requirements on design environments– Provide process models as resources

– Use process models to keep design feedback timely

• Support in Argo: First-class decision model, To Do List

UCI

Process Navigation

UCI

Design Perspectives — Comprehension and Problem Solving

• Observation: Designers use multiple mental models– Work through successive levels of refinement

– Restructure their thinking to address different issues

• Suggested requirements on design environments– Present multiple perspectives to match mental models

– Help maintain mappings between mental models

• Support in Argo: Multiple coordinated design perspectives, customizable visualizations

UCI

Multiple Perspectives

UCI

Summary• Tools for designers should support their cognitive

needs.

• We have examined three theories of cognitive needs and how to support them.– Gives us a checklist of needed design environment features.

Critics

First ClassProcess Model

ManagedTo Do List

MultiplePerspectives

DirectManipulation

CustomGraphics

ProcessCritics

PresentationCritics

Re½ection in Action

Comprehension and Problem

Solving

Opportunistic Design

UCI

Argo Future• Apply Argo’s infrastructure to two new domains

– UML: a new object-oriented design notation

– CoRE: a formal state-based representation of behavioral requirements for embedded systems

• Evaluation of impact of cognitive support on design product and process– Feedback from Collins Avionics on CoRE tool

– Planning to use Argo/UML in ugrad class in Spring

• Evaluation of usefulness of Argo infrastructure– Metrics on effort needed to build Argo/CoRE and Argo/UML

– Feedback from other developers using Argo

UCI

BackgroundUML: The Unified Modeling Language

• UML unifies – Popular OO design notations: Booch, OMT, OOSE– Many views of the system: object model, state models, use cases,

deployment model, etc.– Notations used for analysis, design, and implementation

• UML is well defined– UML Meta-model defines what is in a UML model– Object Constraint Language specifies system constraints– Meta-model has about 100 classes and 180 constraints (that might make

good design critics)

• There is broad industrial interest in using UML

UCI

UML Example

(self.robot->size) / (self.worker->size) < 0.1

Company

Worker

Robot «Person»Employee

Trainee

Course

0..*

0..*

1..4

1..*1..*

UCI

CoRE• CoRE is a formal state-based notation for behavioral

requirements of embedded systems, based on SCR– Embedded systems monitor some variables and control others.

The values of controlled variables depends on the current state. State transitions occur in reaction to changes in monitored variables.

• Examples– Traffic lights at a simple 3-way intersection

• Monitors sensing pads, walk buttons, and internal timers• Controls color of lights and walk signs• About 9 states and 12 transitions

– Auto-pilot on a Boeing 777• Monitors buttons and switches in cockpit• Controls flight modes and some cockpit displays• Thousands of states and transitions

UCI

Intersection Overhead View

Crosswalk button

Crosswalk button

1 2

3

4

5

6

7

Cornell Drive

Cam

pus

Dri

ve

UCI

Behavioral SpecificationDiagram

Flow

Warn

Stop

Normal

Flow

Warn

Stop

Allow2

Flow

Warn

Stop

Walk

1

2 3

4

5

6

7

89

10

11

UCI

CoRE Specifications with Feedback

UCI

EDEM

Usage Data Collection Through

Expectation-Driven Event Monitoring

UCI

Expectations in Development• Developers have usage expectations that importantly affect

design decisions.

• Developers' usage expectations are based on their:– knowledge of requirements

– knowledge of the application domain

– knowledge of specific user tasks, practices, and work environments

– past experience developing systems

– past experience using applications themselves

• Developers' usage expectations impact the appropriateness and usability of their designs:– accurate expectations => good designs

– inaccurate expectations => poor designs

UCI

Characteristics of Expectations• Some usage expectations are represented explicitly.

– e.g. those specified as requirements or in "use cases"

• Most usage expectations are implicit.– e.g. those encoded in window layout, toolbar and menu design, key

assignments, and user interface libraries.

• Example usage expectations:– users complete forms from left to right and top to bottom.– frequently used features are easy to access, recognize, and use.

• Because most usage expectations are not represented explicitly, they often:– fail to be tested adequately– fail to be explicitly recognized by developers

UCI

Resolving Mismatches• Detecting and resolving mismatches between developers'

expectations and actual usage can help improve:– design, automation, on-line help, training, and use.

• Once mismatches are detected, they may be resolved in one of two ways:– Developers may modify their expectations to better match actual use,

thus refining system requirements and eventually improving the design.

• e.g., features expected to be used rarely but used often in practice can be made easier to access.

– Users may learn about developers' expectations, thus learning how to use the existing system more effectively.

• e.g., learning they are not expected to type full URL's in Netscape's Communicator can lead users to omit characters such as "http://".

UCI

Expectation-Driven Event Monitoring (EDEM)

UCI

Agent Representation• Agents are instances of a simple Java class w/ the

following members:– Trigger: patterns of user interface (or agent) events

– Guard: boolean expression involving user interface (or agent) state

– Actions: pre-supplied actions or arbitrary code

• Triggers are continually checked as users interact w/ the application.

• Guards are checked if an agent trigger has been activated.

• Actions are performed if the guard evaluates to true.

UCI

Triggers• Triggers specified in terms of the following patterns:

(1) "A or B or . . . "

(2) "A and B and . . . "

(3) "A then B then . . . "

(4) "(A and B) with no interleaving C"

(5) "(A then B) with no interleaving C"

• Where variables A,B,C are filled in by specifying:(1) a Component from the UI plus an Awt or EDEM event on that

component (e.g. "TextField1:LOST_EDIT" which occurs when TextField1 is edited and then input focus shifts and editing begins in another component)

(2) another Agent (e.g. "AddressDone" which occurs when another agent detects that the address section has been completed)

UCI

Guards• Guards specified in terms of the following patterns:

(1) "A or B or . . . "

(2) "A and B and . . . "

• Where variables A,B are filled in by specifying:(1) a Component from the UI and some expression involving its

properties (e.g. "TextField1:value='Married'" or "Button1:count>100")

(2) another Agent and some expression involving its properties (e.g. "AddressStarted:enabled=true" or "AddressDone:count>1")

UCI

Actions• Actions may include arbitrary code, but usually

involve pre-supplied actions such as:– generating higher level events for further hierarchical event

processing

– interacting with users to provide suggestions and/or collect feedback, and

– reporting data back to developers

UCI

Integrating w/ EDEM• void initialize(boolean obtrusive)

– load agents, post EDEM message window if obtrusive

• void addMonitors(Object obj)– recursively add monitors to this component and all subcomponents

• void setName(Object obj, String name)– name components to be monitored that don’t have a unique label

• void processEvent(Event evt)– pass events to EDEM for processing

• void finalize()– remove monitors and send log & summary

UCI

Summary• Usage expectations:

– focus data collection

– raise awareness of implications of design decisions

• Event monitoring architecture:– distributed event filtering and abstraction

– reduction of data to be reported and analyzed

– independent evolution of instrumentation/application

• Extensible event model:– data collected/analyzed at multiple levels of abstraction

– events of interest unavailable in a given event model are added by defining them in terms of available events

UCI

Global Transportation Network (GTN) Application

UCI

Action Sequence Dependency

UCI

Knowledge Depot

Information Review and Project Awareness

UCI

Communication and Group Memory

• The Knowledge Depot, an enhanced version of GIMMe (Zimmermann, Lindstaedt, Girgensohn, 1997), is a group memory that captures email conversations and documents submitted over the web.

• Users may browse through the information to learn or to rediscover why different decisions were made (or are being made), allowing the user to gain some of the context in which those decisions were made..

UCI

UCI Knowledge Depot

UCI

Characterizing Individual and Group Awareness

UCI

Project Awareness• Project Awareness is an approach to providing

communications links between individuals and all of the different aspects of the project that affect those individuals.

• The strength of those communication links should depend on the relevance to each individual.

UCI

Types of Information• Changes to component’s design or interface

• Changes to completion dates

• Changes to higher level design

• Changes to management

• Changes to customer requirements

• The knowledge that the above changes are being discussed

• The concerns that lead to the discussions

UCI

Types of Links

UCI

From Group Memory to Project Awareness

• Group Memories can enhance Project Awareness by allowing users to request periodic summaries of new information in topics of interest. This gives a group memory three key attributes for creating an awareness system:– Information Capture: Information captured or added to the group

memory

– Information Distribution: Emailed to interested users

– Information Display: A list of documents added to the group memory, accompanied by brief descriptions

UCI

Future Work• How well does Knowledge Depot enhance project

awareness relative to existing approaches?

• How accurate are our predictions on the benefits of Project Awareness?

• How can Knowledge Depot more effectively support project awareness?

• How effective are categorization techniques on personal/frequently used mailboxes as well as the less frequently used archives.

UCI

Java Depot: Integration between Project and Individual Depots

UCI

Conclusions

UCI

Research Goals• To develop a model of software development as a

process of on-going communication;

• To support this model through active mechanisms in software tools; and

• To improve the accessibility and acceptance of usability methods by software practitioners.

UCI

Research Approach• To focus on a human-centered software development

process, in which feedback comes from all stakeholders.

• Stakeholders reflect on an artifact that evolves from sketchy designs to partial, and eventually, complete prototypes.

• Feedback is not limited to human stakeholders: tools can provide data about actual usage and knowledge evolved from previous development experiences.