software engineering lecture 1 introduction to software engineering

Software Engineering

Lecture 1

Introduction to Software Engineering

Code of Conduct

Software Engineering is a collaborative activity. You are encouraged to work together, but ...

Some tasks may require individual work.

Always give credit to your sources and collaborators.

Good professional practice: To make use of the expertise of others and to build on previous work, with proper attribution.

Unethical and academic plagiarism: To use the efforts of others without attribution.

Projects

Project teams, about 3 to 5 peoples.

Select your own project, any branch of software engineering

Real project for real client who intends to use the software in production.

Feasibility study and plan: during semester

Presentations:

• requirements

• design

• final

Project Selection

Some suggested projects

Recitation section to suggest projects

Contact potential clients:

Gain idea of their expectations

Estimate scope and complexity of the project

Discuss business decisions

Assemble project team

Advertise on the web site

Previous Experience

Your background

Biggest program that you have written?

Biggest program that you have worked on?

Biggest project team that you have been part of?

Longest project that you have worked on?

Most people who have used your work?

Longest that your project has been in production?

My background

Course Themes

1. Leadership of large software projects

Software as a product

Clients and their needs Quality

Requirements and specification

Usability Evolution

Project management

Personnel management Economic, legal, and social factors

Course Themes

2. Large and very large systems

Software design

Software architecture Object-oriented design

Dependable systems

Reliability Verification

Legacy systems

Characteristics of Software Products

General characteristics

Usability Maintainability Dependability Efficiency

Good software products require good programming,

but ...

Programming quality is the means to the end, not the end itself.

Software as a Product

Software is expensive!!

Every software project has a trade-off between:

Functionality Resources (cost) Timeliness

Example: Accounting Management System

Client (a.k.a Customer)

The client provides resources and expects some product in return.

Client satisfaction is the primary measurement of success.

Question: Who is the client for Microsoft Excel?

Variety of Software Products

Examples?

-Operation System

-Database Management System

-Embedded System

-Games

-Application Software

-…

Categories of Product

Categories of client and software product:

Generic (e.g., Microsoft Excel)

Bespoke (customized) (e.g., IRS internal system)

Many systems are customized versions of generic packages (e.g., Cornell's payroll system)

Variety of Software Products

Software products are very varied

--> Client requirements are very different

--> There is no standard process for software engineering

--> There is no best language, operating system, platform, database system, development environment, etc.

A skilled software developer knows about a wide variety of approaches, methods, tools. The craft of software engineering is to select appropriate methods for each project and apply them effectively.

Professional Responsibility

Organizations put trust in software developers:

Competence: Software that does not work effectively can destroy an organization.

Confidentiality: Software developers and systems administrators may have access to highly confidential information (e.g., trade secrets, personal data).

Legal environment: Software exists in a complex legal environment (e.g., intellectual property, obscenity).

Acceptable use and misuse: Computer abuse can paralyze an organization (e.g., the Internet worm).


Lecture 2

The Software Process

Books

Frederick P. Brooks, Jr. The Mythical Man Month. Addison-Wesley, 1972.

Ian Sommerville, Software Engineering, 6th edition. Addison-Wesley, 2000.

Grady Booch, James Rumbach, Ivar Jacobson, The Unified Modeling Language. Addison-Wesley 1999.

Software Process

Fundamental Assumption:

Good processes lead to good software

Good processes reduce risk

Risk Management

What can go wrong in a software project?

How can the risk be reduced?

The Software Process (Simplified)

Requirements

Operation andMaintenanceImplementation

Design

Feasibility andPlanning

The Waterfall Model

RequirementsDefinition

System andSoftware design

Programmingand Unit Testing

Integration andSystem Testing

Operation andMaintenance

Requirements Analysis and Definition

The system's services, constraints and goals are established by consultation with system users. They are then defined in a manner that is understandable by both users and development staff.

This phase can be divided into:

Feasibility study (often carried out separately)

Requirements analysis

Requirements definition

Requirements specification

System and Software Design

System design: Partition the requirements to hardware or software systems. Establishes an overall system architecture

Software design: Represent the software system functions in a form that can be transformed into one or more executable programs

Unified Modeling Language (UML)

Programming and Unit Testing

The software design is realized as a set of programs or program units. (Written specifically, acquired from elsewhere, or modified.)

Individual components are tested against specifications.

Integration and System Testing

The individual program units are:

integrated and tested as a complete system

tested against the requirements as specified

delivered to the client

Operation and Maintenance

Operation: The system is put into practical use.

Maintenance: Errors and problems are identified and fixed.

Evolution: The system evolves over time as requirements change, to add new functions or adapt the technical environment.

Phase out: The system is withdrawn from service.

Discussion of the Waterfall Model

Advantages:

Process visibility Dependence on individuals Quality control Cost control

Disadvantages:

Each stage in the process reveals new understanding of the previous stages, that requires the earlier stages to be revised.

Feedback in the Waterfall Model






Iterative Refinement(Evolutionary Development)

Concept: Initial implementation for user comment, followed by refinement until system is complete.

Vaporware: user interface mock-up

Throw-away software components

Dummy modules

Rapid prototyping

Successive refinement

Iterative Refinement

Requirements

DesignImplementation

(prototype)

Evaluation


OutlineDescription

ConcurrentActivities

Requirements

Design

Implementation

InitialVersion

IntermediateVersions

FinalVersion

Iterative Refinement & Software Process

OutlineDescription

ConcurrentActivities

Requirements

Design

Implementation FinalVersion


When is iterative refinement appropriate?

Iterative Refinement + Waterfall Model:

Graphics for Basic

Outline Description: Add vector graphics to Dartmouth Basic.

Phase 1: Extend current language with a preprocessor and run-time support package. (1976/77)

Phase 2: Write new compiler and run-time system incorporating graphics elements. (1978/80)

Iterative Refinement + Waterfall Model: Graphics for Basic

Design Issues:

Pictorial subprograms: coordinate systems, window/viewport

User specification of perspective

Design Strategy: (Iterative Refinement)

Write a series of prototypes with various proposed semantics

Evaluate with a set of programming tasks

Iterative Refinement + Waterfall Model: Graphics for Basic

Phase 1: Implementation (Waterfall)

When the final specification was agreed, the entire preprocessor and run-time support were recoded.

The system was almost entirely bug-free.

Phase 2: New compiler (Waterfall)

Phase 1 was used as the requirements definition for the final version.

Observations about Software Processes

Completed projects should look like the Waterfall Modelbut ... the development process is always partly evolutionary.

Risk is lowered by:

Prototyping key components

Dividing into phases

Following a visible software process

Making use of reusable components


Lecture 3

(a) Feasibility Study

(b) Requirements Definition

Feasibility Study

Before beginning a project, a short, low-cost study to identify

• Client

• Scope

• Potential benefits

• Resources needed:

staff, time, equipment, etc.

• Potential obstacles

Where are the risks? How can they be minimized?

Feasibility Study

A feasibility study leads to a decision:

go aheaddo not go aheadthink again

In production projects, the feasibility study often leads to a budget request.

In research, a feasibility study is often in the form of a proposal.

CS 501: Client

In CS 501, you have two clients:

• The client for the project

• The professor for the course

Can you satisfy them both?

Scope

What are the boundaries of the project?

CS 501 Examples:

• Static web pages with open access on the Web [Web Profiler]

• Used by the general public [Digital Collections]

• Varying data formats [Legal Information]

• Thousands of sensors [Data mining]

• Support for Windows, Mac, Unix [SALSA]

Potential Benefits

Why are you doing this project?

Examples

• Create a marketable product

• Improve the efficiency of an organization

• Control a system that is too complex to control manually

• New or improved service

• Safety or security

• Get a good grade on CS 501

Resources

Examples: CS 501

Staff: 5 to 7 students, with some help. How many hours per week? What skills do people have?

Time: Must be completed by end of semester, including operational system, documentation, presentation

Equipment and software: What special needs are there?

Client: Will the client be sufficiently available and helpful?

Obstacles

CS 501 projects

Start-up time. Creating a team, scheduling meetings, acquiring software, learning new systems, ...

Business considerations. Licenses, trade-secrets, ...

Too ambitious. Nothing to show at the end of the semester.

Changing circumstances. Client leaves the university, ...

What else?

How to Minimize Risk?

CS 501 Projects

• Several target levels of functionality: required, desirable, optional

• Visible software process: intermediate deliverables

• Good communication within team and with Teaching Assistant


Good processes reduce risk

Feasibility Report

A written document

• For a general audience: client, financial management, technical management, etc.

• Short enough that everybody reads it

• Long enough that no important topics are skipped

In CS 501, I am looking for a well written, well presented document.

Requirements Definition and Analysis






Example: Library of Congress(A Partial Failure)

Outline Description

The Library of Congress requires a repository system to store and make accessible very large amounts of highly varied material over long periods of time.

Chronology

1993-94 CNRI carries out research on architectures for digital libraries

1995-97 CNRI implements prototype repository for Library of Congress

1998 CNRI and Library of Congress carry out requirements definition

Repository

Identification System Search System

Users

The Repository

Storage and Representation of Complex Objects

Data

Several representations: thumbnail image reference image archival image

Metadata

Each representation may have its own metadata

Repository: Research Achievements

1. CORBA implementation of repository access protocol.

2. Integration of persistent naming through handle system.

3. Use of structural metadata to describe complex objects, elementary typology.

4. Access management framework and implementation.

5. Applet-based middleware for user interfaces.

6. Information visualization program to view the structure of large collections.

Good Discoveries During Prototype

• Structuring complex information in digital libraries

• Data driven digital library interfaces

• Comparison of object-oriented, relational, and file based storage systems

• Naming and identification of library objects

• Boundaries of required repository system

Bad Discoveries During Prototype

• Resistance to change within Library of Congress

• Technical weakness of Library of Congress

• Gaps in CNRI architecture

Mistakes

• Confusion of objectives (research and implementation)

• Failure to involve all stakeholders

• Over-ambitious (no proper feasibility study)

The Requirements Process

FeasibilityStudy

RequirementsAnalysis


RequirementsSpecification

FeasibilityReport System

Models Definition ofRequirements

Specification ofRequirements

RequirementsDocument

Requirements Definition

High-level abstract description of requirements:

• Specifies external system behavior

• Comprehensible by customer, management and users

Should reflect accurately what the customer wants:

• Services that the system will provide

• Constraints under which it will operate

Library of Congress Requirements Study

Team (all experienced): Librarian, Software Engineer (CNRI), Computing Project Leader (Library of Congress), + 2 others

Advisors: Mailing list of about 20 knowledgeable stakeholders.

Timetable: Preliminary report (2 months). Final report (1 month).

Functional Requirements

Example: Library of Congress repository

• Support for complex digital objects

• Access management

• Identification

• Information hiding

• Open protocols and formats

• Integration with other systems (scope)

Current Storage Structure (in Unix files, by aggregate)

Index Generation(including pre-processing)

American Memory User Interface(retrieval, navigation, & display)

Object Administration System

Repository

NDLP Workflow Tracking Support

Handle-server

NDLP collections already released

NDLP collections in conversion

Coolidge collection(for repository test)

Future NDLP collections

NOW FUTURE

ILS OPAC InterfaceOther User Interfaces (e.g. RLG, OCLC, DLF partners)

Other applicationsand materials

ILS

Handle assignment & registration Handle resolution

Supporting infrastructure

DRAFT OVERVIEW OF ITS SUPPORT FOR NDLP PRODUCTION AND DELIVERY OF AMERICAN MEMORY

AM user interface plus access management

for objects/collections

Non-functional Requirements

Environment:

• Estimates of sizes, numbers of users, etc.• Reliability and performance measures and targets

Preferred:

Example: Library of Congress repository

• Hardware and software systems (e.g., IBM/Unix)• Database systems (e.g., Oracle)• Programming languages (e.g., C and C++)

Evolution of Requirements

• If the requirements definition is wrong, the system will be a failure.

• With complex systems, understanding of requirements always continues to improve.

Therefore...

• The requirements definition must evolve.

• Its documentation must be kept current (but clearly identify versions).


Lecture 4

Management I: Project Management

The Aim of Project Management

To complete a project:

• On time

• On budget

• With required functionality

• To the satisfaction of the client

• Without exhausting the team

The Project Manager

• Create and maintain the schedule• Should track progress against schedule• Keep some slack in the schedule• Be continually making adjustments:

Start activities before previous activity completeSub-contract activitiesRenegotiate deliverables

• Keep senior management informed

Project Planning Methods

The Critical Path Method, Gantt charts, Activity bar charts, etc. are roughly equivalent.

These methods are best when:

• Model is updated regularly (e.g., monthly)

• The structure of the project is well understood

• The time estimates are reliable

• Activities do not share resources

[Critical Path Method is excellent for large construction projects.]

Example: An Open University Course

Deliverables:

16 Written texts (bound in pairs)

8 Television programs

8 Radio programs

4 Computer programs

1 Home experimental kit (scientific calculator)

4 Assignments and sample solutions

Flexibility

Schedule: Dates for broadcasting TV and radio programs are fixed. Printing and mailings can be accelerated if overtime is used.

Functionality: The course team can decide what goes into the components of the course.

Resources: The size of the course team can be increased slightly.

Scheduling: Critical Path Method

An activity

A dummy activity

An event

A milestone

Critical Path Method

Edit Unit 3

PrintUnit 3

Revise Unit 3

MailUnit 3

otheractivities

START END


Edit Unit 3

TypesetUnit 3

Revise Unit 3

MailUnits 3/4

otheractivities

Edit Unit 4

PrintUnits 3/4

Revise Unit 4

otheractivities

TypesetUnit 4

START


START

Edit Unit 3

ScriptTV 2

MakeTV 2

Edit Unit 4

Prototype Computer 1

ProgramComputer 1

DocumentComputer 1

MailDelivery

Time Estimates for Activities (Weeks)

64

2

2

3

3

13

3

82

1 1

4

12

12

1

4

Earliest Start Dates

64

2

2

3

3

13

3

82

1 1

4

12

12

1

4

0

1

12

12

4

15 17

17

17

19

22

26

23 25

Latest Start Dates

6 4

2

2

3

3

1

3

3

82

1 1

4

12

12

1

0

11

12

14

13

15 17

17

17

20

23

26

24 25

4

Critical Path

0/0

1/11

12/12

12/14

4/13

15/15 17/17

17/17

17/17

19/20

22/23

26/26

23/24 25/25

Slack

0/0

1/11

17/17

23/24

10

5

0

12/12

12/14

4/13

15/15 17/17

17/17 19/20

22/23

26/26

25/25

10

00

00 0

0

22

9

9

1

1

3

1 1

Key Personnel

In computing, not all people are equal:

• The best are at least 5 times more productive• Some tasks are too difficult for everybody

Adding more people adds communications complexity

• Some activities need a single mind• Sometimes, the elapsed time for an activity can not be shortened.

What happens to the project if a key person is sick or quits?

Key Personnel: Schedule for Editor

Earliest Start Date Activity

Weeks 15-16 Edit Unit 3Weeks 17-18 Edit Unit 4Weeks 19-20 Edit Unit 5Weeks 21-22 Edit Unit 6

Week 15 Review draft of Unit 7Week 17 Review draft of Unit 8Week 19 Check proofs of Unit 3Week 21 Check proofs of Unit 4

Weeks 18-19 VacationWeek 22 Out sick

Start-up Time

On a big project, the start-up time is typically three to six months:

• Personnel have to complete previous projects (fatigue) or recruited.

• Hardware and software has to be acquired and installed.

• Staff have to learn new domain areas and software (slow while learning)

• Clients may not be ready.

Experience with Critical Path Method

Administrative computing department at Dartmouth used the Critical Path Method for implementation phase of major projects.

Experience: Elapsed time to complete projects was consistently 25% to 40% longer than predicted by model.

Analysis:

• Some tasks not anticipated (incomplete understanding)• Some tasks had to be redone (change of requirements, technical changes)• Key personnel on many activities (schedule conflicts)• System ZZZ (non-billable hours)

CS 501: Software Engineering

Lecture 5

(a) Documentation

(b) Requirements Analysis

Assignments

September 13 Feasibility and plan Group

October 4 Requirements Group/individual

October 16 Midterm exam Individual

November 8 Design Group/individual

Nov 29 - Dec 1 Project presentations Group

Exam week Final examination Individual

Details are subject to change.

Assignment 1

Wednesday, September 13: Project plan due -- report.

Title of project Client/customer Team members Outline description Current status (e.g., previous work) Plan (e.g., major stages, assignment to tasks, technical environment, schedule, etc.) Any other relevant information

Documentation

• Reasons for documentation:visibility (e.g., project plan, interim report)

user support (e.g., user manual) team communication (e.g., interface specifications)

maintenance and evolution (e.g., requirements) • Characteristics of documentation:

accurate and kept currentappropriate for audiencemaintained online (usually)simple but professional in style and appearance

Documentation is expensive --> Quality not volume

Form of Documentation

External

• Printed

• Web site

Internal

• Program documentation

• Program context (e.g., copyright notices)

Requirements Definition and Analysis







FeasibilityStudy








Requirements Analysis

1. Understand the requirements in depth:

• Domain understanding

Examples: science research, application…

• Stakeholders

Example: companies, ministries, Danang City…

Viewpoint Analysis

Example: University Admissions System

• Applicants

• University administrationAdmissions officeFinancial aid officeSpecial offices (e.g., athletics, development)

• Computing staffOperationsSoftware development and maintenance

• Academic departments

Interviews with Clients

Clients may have only a vague concept of requirements.

• Prepare before you meet with them

• Keep full notes

• If you don't understand, delve further

• Small group meetings are often most effective

Clients often confuse the current system with the underlying requirement.


2. Organize the requirements:

• Classification into coherent clusters

(e.g., legal requirements)

• Recognize and resolve conflicts

(e.g., functionality v. cost v. timeliness)

Example: Dartmouth general ledger system


3. Model the requirements:

• Informal

Prose

• Systematic

Procedural models

Data-centric models

Object models

• Formal models

Procedural Models: Flowchart

Operation

Decision

Manual operation

Report

Flowchart: University Admissions

Form received New?

Database record

T

Notify student

F Update database

Complete?

Notify student

T

FEvaluate

Procedural Models: Pseudo-code

Example: Check project project plan

check_plan (report)

if report (date_time) > due_date_time then error (too_late) if report (client) = none then error (no_client) if report (team) < min_team or > max_team then error (bad_team) if error() = none then comments = read_report (report) return (comments (text), comments (grade)) else return error()

Data-Flow Models

External entities

Processing steps

Data stores or sources

Data flows

Example: University Admissions

Applicant

Applicationform Receive

application

Completedapplication

Evaluate

Rejection

Offer

Example: University AdmissionsAssemble Application Stage

Applicant

Applicationform

Receive

Completedapplication

Supportinginformation

Pendingdatabase

Acknowledgment

Initiateevaluation

Applicantdatabase

Evaluationrequest

AND

AND

Acknowledgment

Example: University AdmissionsProcess Completed Application Stage

Rejection

Evaluation

Applicantdatabase

Evaluationrequest Acceptance Financial

aid

Offer

Specialrequest

Requirements Analysis v. System Design

Dilemma.

• Requirements analysis should make minimal assumptions about the system design.

• But the requirements definition must be consistent with computing technology and the resources available.

In practice, analysis and design are interwoven. However, do not to allow the analysis tools to prejudge the system design.


Lecture 6

(a) Requirements Analysis (continued)

(b) Requirements Specification


FeasibilityStudy









Methods for data modeling and design

• Data flow diagrams

• Entity-relation diagrams

• Data dictionaries

• Object models

Many of these methods blur the distinction between analysis and design.

Entity-Relation Model

A Design Methodology for Relational Databases

• A database of entities and relations

• Tools for displaying and manipulating entity-relation diagrams

• Tools for manipulating the database (e.g., as input to database design)

Warning: There is much confusion about definitions and notation

Entity-Relation Diagram

An entity

A relation between entities

An entity or relation attribute

An inheritance relation

Example: CS 501 Project

Student

CS501 Student

Major

Project

5 to 7

1

Member of

Person

Client1

Tech contact

0:n0:n

0:n

MARC Format for Monographs (Books)

001 89-16879 r93245 Campus strategies for libraries and electronic information260 {Bedford, Mass.} : Digital Press, c1990.650 Academic libraries--United States--Automation.650 Libraries and electronic publishing--United States.700 Arms, Caroline R. (Caroline Ruth)

Entity-Relation Diagram for MARC

Book

Short title

Catalog record

Describes

Control numb

Subject heading

Is about

CreatorEditor of

Author of

1:n

1

0:n

0:n

0:n

0:n

0:n

0:n

Data Dictionaries

A data dictionary is a list of names used by the system

• Brief definition (e.g., what is "date")

• What is it (e.g., number, relation)

• Where is it used (e.g., source, used by, etc.)

• May be combined with a glossary

As the system is implemented, the data dictionary in the requirements is input to the system data dictionary, which is a formal part of the system specification.

A Note on Object Models

This course teaches object models as a tool for design.

Some people recommend object models for requirements analysis, but it is difficult to use them without constraining the system design.

Non-Functional Requirements

Product requirements

performance, reliability, portability, etc...

Organizational requirements

delivery, training, standards, etc...

External requirements

legal, interoperability, etc...

Examples of Non-Functional Requirements

Privacy (Mercury digital library)

Functional requirement: Usage data for management of system

Non-functional requirement: Usage data must not identify individuals

Minimizing records (NeXT)

Functional requirement: Retain all required records

Non-functional requirement: Discard all other records

Unspoken Requirements

Example:

Resistance to change at XXX

Requirements Specification

What is the purpose of the Requirements Specification?

Requirements Specification: Purpose

1. It describes the requirements to the stakeholders

• Expressed in the terms that the stakeholders understand

• Comprehensible from many viewpoints

• Reviewed by stakeholders so that they understand implications

• Must be clear about assumptions (things left out)


2. It describes the requirements to the implementers

• As precise and specific as possible

• Expressed in terms that they understand

• Comprehensible to new team members


3. It records the requirements for the future

• An essential part of system evolution

4. If may be a contractual document

• See you in court!

Requirements Specification: Approaches

• Natural language

• Structured natural language

• Design description language

• Requirements specification language

• Graphical notation

• Formal specification

See Sommerville, Chapter 7.


Lecture 7Management II

Business and Legal Aspects of Software Engineering

Legal Environment

Software is developed in a complex legal and economic framework. Changes in laws follow changes in technical world.

Jurisdictions:

• Vietnamese laws• International treaties• Federal and state statues• Precedents• Supreme Court• Cost of establishing precedent

Legal Topics

• International

• Intellectual property (copyright, patent, contract)

• Tort (e.g., liability of Internet service provider)

• Privacy

• Free speech and its limitations (government secrets, obscenity, blasphemy, hate)

Legal Information Institute: http://www.law.cornell.edu/

Copyright

A copyright gives the owner the exclusive right to:

• reproduce

• distribute

• perform

• display

• license

Gradually extended to cover text, music, photographs, designs, software, ...

Copyright

Copyright at creation

• Works for hire

• Contracts and licenses

• First sale

• Fair use

• Infringement (contamination)

International differences

• Moral rights

• Copyright registration

Software Patents

• Should be: non-obvious, novel, useful

• 17 years from award (20 years from application)

• Poor quality of examining can lead to broad patents for routine computing concepts

• International differences

Copyright applies to the expression of ideas, patents to the ideas themselves.

Contracts and Licences

Contracts allow intellectual property to be sold or licensed

• Promise in exchange for adequate consideration

• Written document with signature

• Permanent or temporary, whole or part

• Exclusive or non-exclusive

• Termination, problems and difficulties

• Terms and conditions as agreed

• Enforceable by courts

Derivative Works

When software is derived from other software:

• New code is owned by new developer

• Conditions that apply to old code apply to derived work

If you write S, which is derived from A, B, C and D, you can not distribute or licenses S unless you have right to distribute each of A, B, C and D.

To create a software product, you must have documented rights to use every component.

Privacy

Invasions of privacy:

• intrusion

• appropriation of name or likeness

• unreasonable publicity

• false light

Be very careful about collecting personal data without the knowledge of the individual

Software Business Questions

• You are employed for company X writing software. When you leave, who owns your work? What use can you make of the work?

• You work free-lance for company X. When you finish, who owns your work? What use can you make of the work?

• Read the contract!

Your Next Job ...

• Employment contract may restrict your next job (not working for competitors, etc.)

• Trade-secret information (non-disclosure agreement)

Ask when you are interviewed!

Trade Secrets and Non-Disclosure Agreements

Trade Secret

"... information, including a formula, pattern, compilation, program, device, method, technique, or process that derives independent economic value from not being generally known and not being readily ascertainable and is subject to reasonable efforts to maintain secrecy."

Uniform Trade Secrets Act

Non-Disclosure Agreement

Legal agreement not to disclose trade secrets.

Some Business Models

• Software developed in-house

• Package licensed to customer, binary only (Microsoft model)

• Package licensed to customer, source code for customer's modifications

• Bespoke software for customer (may be owned by supplier or customer)

• Software bundled with hardware product (PalmPilot)

Free-Lance Software Development

You and a few friends create a company to develop software.

How much should you charge per hour?

You plan to work 40 hours a week for 50 weeks of the year and want to earn $50,000.

Hourly rate = $50,000 / (40 x 50) = $25

But ...

Free-Lance Software Development

Salary $50,000Taxes and benefits $15,000Rent, equipment, etc. $10,000Fees, services, etc. $15,000Travel and misc. $10,000 TOTAL EXPENSE $100,000

Hours worked 2,000less administration 400less marketing 350 BILLABLE HOURS 1,250

Hourly rate = $100,000 /1,250 = $80

Fixed and Variable Cost: Packaged Software

Example:

• The initial development cost of a software product is $10 million.

• The cost of packaging and distribution of each copy is $5.

• Technical support costs average $15 per copy.

• The package sells for $200 per copy.

Fixed cost = $10 million

Variable cost = $20

Fixed and Variable Costs: Profit or Loss

$15M

$10M

$5M

Unit sales2,500 5,000 7,500

Community Development

• Shareware

• Open source (e.g., Linux, Apache, Perl, etc.)

-> Shared development

-> Market penetration

Example: TCP/IP for Vax/VMS

Software may be open source, but packaging and services can be profitable businesses

Open Source

• Free redistribution

• Source code

• Derived works

• Integrity of the author's source code

• No discrimination against persons or groups

Open Source

• No discrimination against fields of endeavor

• Distribution of license

• License must not be specific to a product

• License must not contaminate other software

http://www.opensource.org/osd.html

Practical Advice

Be aware of the law, but do not pretend to be a lawyer. Use a professional for:

• Contracts and licenses

• Troubles (complaints, injunctions, subpoenas, etc.)

• Personnel issues

• When in doubt, ask help!

Source Code Management

Or

Configuration Management: How I learned to Stop Worrying and Hate My

Co-workers Less

Source Code Management

Also known as Configuration Management Source Code Managers are tools that:

– Archive your development files

– Serve as a single point of entry/exit when adding or updating development files

Why You Want A Source Control System

Supports concurrent development Manage diverging source code bases Records file/release versions Easy access to all previous revisions Can record why a revision was made Optimal disk space usage You’ll end up doing something equivalent anyway so it

may as well be automated

Source Code Management Tools Are Not

A substitute for project management A replacement for developer communication

How They Work

Central database of source code, documentation, build tools

Each file stored only once - all other versions are diffs of that one copy

To Make a Change

– Check out the latest version of a file

– Make the changes

– Update the database

What should be in the database

Source Code Documentation Build Tools

– Often need old versions of the tools to build old versions of the software

– Ensures software is rebuilt exactly as the customer received it

Test Suites Anything else you might want later

Version Control

Companies ship several products from the same source base (i.e. Win NT and Windows 2000 versions of MS Office)

When tracking down bugs you want to examine the code as it was when the product shipped

Code Sharing

Multiple people can work on the same source base without colliding

– (1) Locks individual files so only one person at a time can modify it *OR*

– (2) Allows multiple people to modify a source file and the system will automatically merge the changes (usually)

Locking

Only one person can work on a file at once Works fairly well if developers work on different areas of

the project and don’t conflict often Problem 1: People forget to unlock files when they are

done Problem 2: People work around locking by editing a

private copy and checking in when the file is finally unlocked - easy to goof and lose changes

Merging

Several people can work on a file at once Before committing changes, each user merges their copy

with the latest copy in the database

– This is normally done automatically by the system and usually works, but you should not blindly accept the result of the merge

Labeling

Label all the files in the source base that make up a product at each milestone

Just before and just after a major change (e.g.. changing several interfaces)

When a new version ships

Version Trees

Each file in the database has a version tree Can branch off the version tree to allow separate

development paths Typically a main path (trunk) for the next major version

and branches off of shipped versions for maintenance

Branching

When a new version ships, typically create a branch in the version tree for maintenance

Double update: fix a defect in the latest version and then merge the changes (often by hand) into the maintenance version

Also create personal versions so you can make a change against a stable source base and then merge in the latest version later

Examples

RCS

– Solaris: man rcsintro CVS

– Solaris: man cvs

– www.cyclic.com/cvs/info.html Visual SourceSafe

– msdn.microsoft.com/SSAFE ClearCase

– www.rational.com

RCS

File management only Transaction model

– check out and lock

– edit

– check in and unlock Little support for binaries

CVS

Built on top of RCS

– Therefore little support for binaries Database can be remote No locking: merge before commit Fast Integrates with emacs

SourceSafe

Microsoft’s entry into the field Project-based Checkout-edit-checkin model Built-in web site creation tools Integrates with MSDEV

Clearcase

Clearcase is configuration management on steroids You create a view of the database with a config spec,

which describes how to select files from the database. When you set a view, Clearcase creates a virtual filesystem

containing only those versions of the files selected by the config spec

Clearcase Features

Distributed System

– Several groups at different locations can work on the same database

Can install triggers

– Example: e-mail the author of a file when some one makes a change to it

Uses merging model like CVS, but can also lock files

More Clearcase Features

Integrates with MSDEV Build Management

– Knows to rebuild out-of-date files even if your makefile doesn’t

Slow and a bit buggy

Helpful Rules for Version Control Bliss

Archived Files Should Always Compile Code Review Files Before Check-in Compile and run latest archived files *as a set* before

Check-in No Cheating (even “simple bug fixes” need to undergo this

process)


Lecture 10

Formal Specification


Why?

• Precise standard to define and validate software

Why not?

• May be time consuming

• Methods not suitable for all applications


Ben Potter, Jane Sinclair, David Till,

An Introduction to Formal Specification and Z

(Prentice Hall) 1991

Jonathan Jacky

The Way of Z

(Cambridge University Press) 1997

Mathematical Specification

Example of specification

B1, B2, ... Bk is a sequence of m x m matrices

1, 2, ... k is a sequence of m x m elementary matrices

B1-1 = 1

B2-1 = 21

Bk-1 = k ... 21

The numerical accuracy must be such that, for all k,

BkBk-1 - I <

Specification of Programming Languages

<unsigned number> ::= <unsigned integer> | <unsigned real>

<unsigned integer> ::= <digit> {<digit>}

<unsigned real> ::= <unsigned integer> . <digit> {<digit>} | <unsigned integer> . <digit> {<digit>} E <scale factor> | <unsigned integer> E <scale factor>

<scale factor> ::= <unsigned integer> | <sign> <unsigned integer>

<sign> ::= + | -

Pascal number syntax

Formal Specification Using Diagrams

digitunsigned integer

digit. E

+

-

unsigned integerunsigned integer

unsigned number

Two Rules

• Formal specification does not guarantee correctness

• Formal specification does not prescribe the implementation

Informal: The function intrt(a) returns the largest integer whose square is less than or equal to a.

Formal (Z):

intrt: N N

a : N •

intrt(a) * intrt(a) < a < (intrt(a) + 1) * (intrt(a) + 1)

Example: Z Specification Language

Example: Algorithm

1 + 3 + 5 + ... (2n - 1) = n2

Example: Program

int intrt (int a)/* Calculate integer square root */{ int i, term, sum; term = 1; sum = 1; for (i = 0; sum <= a; i++) { term = term + 2; sum = sum + term; } return i;}

Finite State Machine

A broadly used method of formal specification:

• Event driven systems (e.g., games)

• User interfaces

• Protocol specification

etc., etc., ...

Finite State Machine

Example:

Therapy control console

[informal description]

State Transition Diagram

Patients Fields Setup ReadyBeam

on

Enter Enter Start

Stop

Select field

Select patient(interlock)

(ok)

State Transition Table

SelectPatient

SelectField

Enter ok Start Stop interlock

Patients

Fields

Setup

Ready

Beamon

Fields

Fields

Fields

Patients

Patients

Patients

Setup

Setup

Setup

Ready

Beamon

Ready

Z Specification (continued)

control = no_change transitions

no_change = { s : STATE; e : EVENT • (s, e) s }

transitions = { (patients, enter) fields,

(fields, select_patient) patients, (fields, enter) setup,

(setup, select_patient) patients, (setup, select_field) fields, (setup, ok) ready,

(ready, select_patient) patients, (ready, select_field) fields, (ready, start) beam_on, (ready, interlock) setup,

(beam_on, stop) ready, (beam_on, interlock) setup }

Schemas

Schema:

• The basic unit of formal specification.

• Describes admissible states and operations of a system.

LibSys: An Example of Z

Library system:

• Stock of books

• Registered users.

• Each copy of a book has a unique identifier.

• Some books on loan; other books on shelves available for loan.

• Maximum number of books that any user may have on loan.

LibSys: Operations

• Issue a copy of a book to a reader.

• Reader return a book.

• Add a copy to the stock.

• Remove a copy from the stock.

• Inquire which books are on loan to a reader.

• Inquire which readers has a particular copy of a book.

• Register a new reader.

• Cancel a reader's registration.

LibSys

Level of Detail:

Assume given sets:

Copy, Book, Reader

Global constant:

maxloans

Schemas Describing Operations

Naming conventions for objects:

Before: plain variables, e.g., r

After: with appended dash, e.g., r'

Input: with appended ?, e.g., r?

Output: with appended !, e.g., r!

Operation: Issue a Book

• Inputs: copy c?, reader r?

• Copy must be shelved initially: c? shelved

• Reader must be registered: r? readers

• Reader must have less than maximum number of books on loan: #(issued {r?}) < maxloans

• Copy must be recorded as issued to the reader: issued' = issued {c? r?}

• The stock and the set of registered readers are unchanged: stock' = stock; readers' = readers

Domain and Range

dom mX Yx

ran my

m : X Y

dom m = { x X : y Y x y}

ran m = { y Y : x X x y}

Operation: Issue a Book

stock, stock' : Copy Book

issued, issued' : Copy Reader

shelved, shelved': F Copy

readers, readers' : F Reader

c?: Copy; r? :Reader

[See next slide]

Issue

Operation: Issue a Book (continued)

[See previous slide]

Issue

shelved dom issued = dom stockshelved' dom issued' = dom stock'shelved dom issued = Ø; shelved' dom issued' = Øran issued readers; ran issued' readers'r : readers #(issued {r}) maxloansr : readers' #(issued' {r}) maxloansc? shelved; r? readers; #(issued {r?}) < maxloansissued' = issued {c? r?}stock' = stock; readers' = readers

<<

LibSys: Schema for Abstract States

Library

stock : Copy Bookissued : Copy Readershelved : F Copyreaders: F Reader

shelved dom issued = dom stockshelved dom issued = Øran issued readersr : readers • #(issued {r}) maxloans<

Schema Inclusion

LibDB

stock : Copy Bookreaders: F Reader

LibLoansissued : Copy Readershelved : F Copy

r : Reader • #(issued {r}) maxloansshelved dom issued = Ø

<

Schema Inclusion (continued)

Library

LibDBLibLoans

dom stock = shelved dom issuedran issued readers

Schema Decoration

Issue

LibraryLibrary'c? : Copy; r? : Reader

c? shelved; r? readers#(issued {r?}) < maxloansissued' = issued {c? r?}stock' = stock; readers' = readers

Schema Decoration

Issue

Libraryc? : Copy; r? : Reader

c? shelved; r? readers#(issued {r?}) < maxloansissued' = issued {c? r?}stock' = stock; readers' = readers

The Schema Calculus

Schema inclusion

Schema decoration

Schema disjunction:

AddCopy AddKnownTitle AddNewTitle

Schema conjunction:

AddCopy EnterNewCopy AddCopyAdmin

Schema negation

Schema composition

=̂

=̂


Lecture 11Object-Oriented Design I

What is in a Requirements Document?

Example (Web Butler and Web Site Profiler)

• Run web data collection in real time or batch mode How are jobs started?

• Job parameters How are the parameters set up (interactive, edit file, ...)? What are the parameters (specify)? Can job parameters be stored and used again? If so, how?

• Job monitoring What feedback is given while job is running? Can the user pause or break a job? If so, are the results retained?

What is in a Requirements Document?

Remember

• The requirements document specifies the functionality that you plan to deliver to the client

• It must be comprehensive and detailed. Everything must be written out -- no hand waving!

The requirements document is likely to be several times as long as Assignment 1.

Assignment 2 -- Individual Parts

One approach:

With your document, include a list of who contributed what part to the Requirements study, e.g.,

Person A

Requirements analysis for database design (member of team of 3), wrote Section 3.1 of document, worked with client to identify software needs.

Person B

Prepared visual aids for presentation, edited entire document, specified the security needs and wrote Section 4.2.

The Waterfall Model


Implementationand Unit Testing




Useful Texts

Grady Booch, James Rumbaugh, Ivar Jacobson, The Unified Modeling Language. Addison-Wesley 1999.

Grady Booch, Object-Oriented Analysis and Design with Applications, second edition. Benjamin/Cummings 1994.

Rob Pooley, Perdita Stevens, Using UML Software Engineering with Objects and Components. Addison-Wesley 1999.

The Importance of Modeling

• A model is a simplification of reality.

• We build models so that we can better understand the system we are developing.

• We build models of complex system because we cannot comprehend such a system in its entirety.

Models can be informal or formal. The more complex the project the more valuable a formal model becomes.

BRJ

Principles of Modeling

• The choice of what models to create has a profound influence on how a problem is attacked and how a solution is shaped.

• Every model can be expressed at different levels of precision.

• The best models are connected to reality.

• No single model is sufficient. Every nontrivial system is best approached through a small set of nearly independent models.

BRJ

The Unified Modeling Language

UML is a standard language for modeling software systems.

• Serves as a bridge between the requirements specification and the implementation.

• Provides a means to specify and document the design of a software system.

• Is process and programming language independent.

• Is particularly suited to object-oriented program development.

Notation: Classes

Window

originsize

open()close()move()display()

name

attributes

operations

A class is a description of a set of objects that share the same attributes, operations, relationships and semantics.

Notation: Interface

An interface is a collection of operations that specify a service of a class or component, i.e., the externally visible behavior of that element.

ISpelling

Notation: Collaboration & Use Case

Place order

A use case is a description of a set of sequence of actions that a system performs that yields an observable result.

Chain of responsibility

A collaboration defines an interaction, i.e., a society of roles and other elements that work together to provide some cooperative behavior.

Notation: Active Class

EventManager

eventlist

suspend()flush()

An active class is a class whose objects own one or more processes or threads and therefore can initiate control activity.

Notation: Component & Node

orderform.java

A component is a physical and replaceable part of a system that conforms to and provides the realization of a set of interfaces.

Server

A node is a physical element that exists at run time and represents a computational resource.

Notation: Behavioral Things:Messages & States

display

An interaction is a behavior that comprises a set of messages exchanged among a set of objects within a particular context to accomplish a specific purpose.

Waiting

A state machine is a behavior that specifies the sequence of states an object or an interaction goes through during its lifetime in response to events.

Notation: Grouping and Annotation

A package is a general-purpose mechanism for organizing elements into groups.

Business rules

return copy of self

A note is a symbol for rendering constraints and comments attached to an element or a collection of elements.

Notation: Relationships

A dependency is a semantic relationship between two things in which a change to one may effect the semantics of the other.

0..1 *

employer employee

An association is a structural relationship that describes a set of links, a link being a connection among objects.

Notation: Relationships (continued)

A generalization is a specialization/generalization relationship is which objects of the specialized element (child) are substitutable for objects of the generalized element (parent).

child parent

A realization is a semantic relationship between classifiers, wherein one classifier specifies a contract that another classifier guarantees to carry out.

Diagrams in UML

A diagram is the graphical representation of a set of elements, usually rendered as a connected graph of vertices (things) and arcs (relationships).

• Class diagram shows a set of classes, interfaces, and collaborations with their relationships.

• Object diagram shows a set of objects and their relationships.

• Use case diagram shows a set of use cases and actors (a special kind of class) and their relationships.

Diagrams in UML (continued)

• Interaction diagram shows an interaction, consisting of a set of objects and the relationships, including the messages that may be dispatched among them.

=> A sequence diagram emphasizes the time ordering.

=> A collaboration diagram emphasizes the structural organization of the objects that send and receive messages.

Diagrams in UML (continued)

• Statechart diagram shows a state machine consisting of states, transitions, events, and activities.

• Activity diagram is a statechart diagram that shows the flow from activity to activity within a system.

• Component diagram shows the organization and dependencies among a set of components.

• Deployment diagram shows the configuration of processing nodes and the components that live on them.

The HelloWorld Example

HelloWorld

paint()

class

name

operations

Abstraction for HelloWorld

HelloWorld

paint() g.drawString ("HelloWorld", 0, 10)"

class

name

operations

annotation

The "Hello, World" Example

import java.awt.Graphics;class HelloWorld extends java.applet.Applet { public void paint (Graphics g) { g.drawString ("Hello, World!", 10, 10); }}

Example from: BJR

Class Diagram

Applet

HelloWorld

paint() Graphics

generalization

dependency

Note that the Applet and Graphics classes are shown elided.

Class Inheritance Diagram

Object

Component

Container

Panel

Applet

HelloWorld

ImageObserver

interface

Packaging Classes

applet

awt

lang

HelloWorld

java

Graphics

package

Notation for Classes and Objects

Classes Objects

AnyClass

attribute1attribute2

operation1()operation2()

AnyClass

or

anObject:AnyClass

:AnyClass

anObject

The names of objects are underlined.

or

or


Lecture 12Object-Oriented Design II

Requirements: the Long Term

Believe that your software will be in use 5 years from now.

• What happens at end of semester?

Packaging and hand-overClient's technical preferences (C++, Java)

• Some system decisions based on short-term considerations

• Which formats, protocols, etc. do you think will last? (IIOP, RMI, SNMP, ...)

Requirements, Design and Implementation

Remember the definitions.

Example: Consistency between two players of a board game

• The requirement is .....

• The design is .....

What is a requirements specification?

Modeling Classes

Given a real-life system, how do you decide what classes to use?

• What terms do the users and implementers use to describe the system? They are candidates for classes.

• Is each candidate class crisply defined?

• For each class, what is its set of responsibilities? Are the responsibilities evenly balanced among the classes?

• What attributes and operations does each class need to carry out its responsibilities?

Noun Identification: A Library Example

The library contains books and journals. It may have

several copies of a given book. Some of the books are

reserved for short-term loans only. All others may be

borrowed by any library member for three weeks. Members

of the library can normally borrow up to six items at a time,

but members of staff may borrow up to 12 items at one time.

Only members of staff may borrow journals.

The system must keep track of when books and journals are

borrowed and returned and enforce the rules.

Candidate Classes

Library the name of the systemBookJournalCopyShortTermLoan eventLibraryMemberWeek measureMemberOfLibrary repeatItem book or journalTime abstract termMemberOfStaffSystem general termRule general term

Relations between Classes

Book is an ItemJournal is an ItemCopy is a copy of a BookLibraryMemberItemMemberOfStaff is a LibraryMember

Is Item needed?

Operations

LibraryMember borrows Copy

LibraryMember returns Copy

MemberOfStaff borrows Journal

MemberOfStaff returns Journal

Item not needed yet.

Class Diagram

MemberOfStaff

BookCopyJournal is a copy of

1..* 1

LibraryMember

1

0..*0..12

1

on loanon loan

Rough Sketch: Wholesale System

A wholesale merchant supplies retail stores from stocks of goods in a warehouse.

What classes would you use to model this business?


RetailStore

Warehouse

Order

Invoice

Product

Shipment

Merchant


Warehouse

Order

Invoice

Product

MerchantRetailStore

nameaddresscontactInfofinancialInfo

Shipment

Responsibilities-track status of shipped products

Reversals

damaged()return()wrongItem()

responsibility (text field)

Expanding a Class: Modeling Financial Information

RetailStore

Transaction1 *association

Invoice

PaymentWhich class is responsible for the financial records for a store?

Modeling Invoice

Shipment

Invoice

invoiceNumber

+goodsShipped()-sendInvoice()

goodsShipped

PartsListadornments+ public- private

RetailStore???

invoiceRecord

Lessons Learned

Design is empirical. There is no single correct design. During the design process:

• Eliding: Elements are hidden to simplify the diagram

• Incomplete: Elements may be missing.

• Inconsistency: The model may not be consistent

The diagram is not the whole design. Diagrams must be backed up with specifications.

Levels of Abstraction

The complexity of a model depends on its level of abstraction:

• High-levels of abstraction show the overall system.

• Low-levels of abstraction are needed for implementation.

Two approaches:

• Model entire system at same level of abstraction, but present diagrams with different levels of detail.

• Model parts of system at different levels of abstraction.

Component Diagram

HelloWorld.class

hello.java

hello.hml

hello.jpg

executable component

Actor and Use Case Diagram

• An actor is a user of a system in a particular role.

An actor can be human or an external system.

• A use case is a a task that an actor needs to perform with the help of the system.

Borrow book

BookBorrower

Use Cases and Actors

• A scenario is an instance of a use case

• Actor is role, not an individual(e.g., librarian can have many roles)

• Actor must be a "beneficiary" of the use case(e.g., not librarian who processes book when borrowed)

In UML, the system boundary is the set of use cases.

Use Cases for Borrowing Books

Borrow copy of book

BookBorrowerReturn copy

of book

Reserve book

Extend loan

Relationships Between Use Cases: <<uses>>

BookBorrower

Check for reservation

Extend loan

<<uses>>

<<uses>>

Borrow copy of book

Relationships Between Use Cases: <<extends>>

Borrow copy of bookBookBorrower

Refuse loan

<<extends>>

Use Cases in the Development Cycle

• Use cases are a tool in requirements analysis

• Intuitive -- easy to discuss with clients

• Use cases are often hard to translate into class models

• Scenarios are useful to validate design


Lecture 13Object-Oriented Design III

Comments on Presentations

Presentation

• Standard of graphics has been high• Some text too small (diagrams, screen dumps)

Content

• Level of detail• Requirements v. design

The client defines the requirements

Well done, but time is short. What is your critical path?

Modeling Dynamic Aspects of Systems

Interaction diagrams: set of objects and their relationships including messages that may be dispatched among them

• Sequence diagrams: time ordering of messages

• Collaboration diagrams: structural organization of objects that send and receive messages

Activity diagram: flow chart showing flow of control from activity to activity

Statechart diagram: models a state machine

Bouncing Ball Diagrams

Example: http://www.cs.cornell.edu/

Client Servers

domain name

TCP connection

HTTP get

Actions on Objects

call

return

send

create

destroy

returnCopy(c)

okToBorrow() local

status

notifyReturn(b) asynchronous signal

<<create>>

<<destroy>>stereotypes

Links

CopyLibraryMember

+borrowCopy()+returnCopy()

1 0..*on loan

libMem:LibraryMember c:CopyborrowCopy(c)

association

message

link

class

object

Sequence Diagram: Change in Cornell Program

Cornellian

:MEngStudent

1 : getName()

sequence numbers added to messages

:PhDStudent

1.1 : name

2: new PhDStudent(name)

3: <<destroy>>

Sequence Diagram: Borrow copy of a Book

BookBorrower

libMem: LibraryMember

theCopy:Copy

theBook:Book

borrow(theCopy)okToBorrow

borrow borrow

Class Inheritance Diagram

Object

Component

Container

Panel

Applet

HelloWorld

ImageObserver

interface

Sequence Diagram:Painting Mechanism

:Thread :Toolkit :ComponentPeer target:HelloWorld

runrun callbackLoop

handleExpose

paint

Activity Diagram: Process Modeling

Release work order

Assign tasks

Reschedule[materials not ready]

[materials ready]guard expression

branch

Activity Diagram: Parallel Activities

Decompress

Stream audioStream video

fork

join

start state

stop state

State Diagram

State diagram for class Book

not borrowable

returned()

returned()

borrowable

borrowed()[not last copy]

borrowed()[last copy]

guard expression

Implementation Modeling

Subsystem

A grouping of elements that specifies what a part of a system should do.

Component (UML definition)

"A distributable piece of implementation of a system, including software code (source, binary, or executable) but also including business documents, etc., in a human system."

A component can be thought of as an implementation of a subsystem.

Component Diagram

HelloWorld.class

hello.java

hello.hml

hello.jpg

executable component

Components and Classes

agent.dll

AgentAction

Policy

PatternSearch


agent.dll

RealizesAgentActionPatternSearchPolicy

extended component


Classes represent logical abstractions. Components represent physical things.

Components may live on nodes.

Classes have attributes and operations directly. Components have operations that are reachable only through interfaces.

Interfaces

simulation.exe render.java

IRender

dependency

interface

realization

Application Programming Interface (API)

API is an interface that is realized by one or more components.

simulation.exe

IModels ILighting

IRender

Components and Replaceability

Components allow system to be assembled from binary replaceable elements.

• A component is physical -- bits not concepts

• A component can be replaced by any other component(s) that conforms to the interfaces.

• A component is part of a system.

• A component provides the realization of a set of interfaces.


Lecture 14

System Architecture I

Data Intensive Systems

System Architecture

The overall design of a system:

• Computers and networks (e.g., monolithic, distributed)

• Interfaces and protocols (e.g., http, CORBA)

• Databases (e.g., relational, distributed)

• Security (e.g., smart card authentication, SSL)

• Operations (e.g., backup, archiving, audit trails)

• Software environments (e.g., languages, source control tools)

Data Intensive Systems

Examples

• Electricity utility customer billing

• Telephone company call recording and billing

• Car rental reservations (e.g., Hertz)

• Stock market brokerage (e.g., Charles Schwab)

• Web sales (e.g., Amazon.com)

Example 1: Electricity Utility Billing

First attempt:

Data input Master fileTransaction Bill

Each transaction handled as it arrives.

Criticisms of First Attempt

Where is this first attempt weak?

The requirements have not been specified!!!

Transaction Types

• Create account / close account

• Meter reading

• Payment received

• Other credits / debits

• Check cleared / check bounced

• Account query

• Correction of error

• etc., etc., etc.,

Typical Requirements

• All payments to be credited on day received

• Customers must be able to query account by telephone

• Cutting off service for non-payment requires management authorization

• Data input staff should process n transactions per day per person

• Error rate must be below 0.01%

• System available 99.9% of business hours

Batch Processing: Validation

Data input

Master file

Edit & validation

read only

errors

Validated transactions

Incoming transactions

Batch Processing: Master File Update

Master fileupdate

Bills

Validated transactionsin batches

Sort by account

errors Reports

Instructions

Benefits of Batch Updating

• All transactions for an account are processed together

• Backup and recovery have fixed checkpoints

• Better management control of operations

• Efficient use of staff and hardware

Online Inquiry

Data input Master fileTransactions Bills

read only

Customer service

Example 2: A Small-town Stockbroker

• Transactions

Received by mail or over telephone

For immediate or later action

• Complex customer inquiries

• Highly competitive market

A Database Architecture

Database(s):

• Customer and account database

• Financial products (e.g., account types, pension plans, savings schemes)

• External databases (e.g., stock markets, mutual funds, insurance companies)

Database Architecture

Customer & account database

Products & services database

External services

Real-time Transaction



External services

Real-time transactions

Real-time Transactions & Batch Processing



External services

Real-time transactions

Batch processing

Data input

Architectural considerations

• Real-time service during scheduled hours + batch processing overnight

• Combine information from several databases

• Database consistency after any type of failure

two-phase commitreload from checkpoint + logdetailed audit trail

• How will transaction errors be avoided?

• How will transaction errors be corrected?

Example: Merger of Two Banks

Each bank has a database with its customer accounts. The databases are used by staff at many branches and for back-office processing.

The requirement is to integrate the two banks so that they appear to the customers to be a single organization and to provide integrated service from all branches.

Merger of Two Banks: Options

???

???

AA BB

Merger of Two Banks: Architectural Options

I. Convert everything to System A.

convert databasesretrain staffenhance System A (software and hardware)discard System B

II. Build an interface between the databases in System A and System B.

III. Extend client software so that it can interact with either System A or System B database.

Distributed Computing: General Problem

An application that is running on one computer wishes to use data or services provided by another:

• Network connectionprivate, public, or virtual private networklocation of firewalls

• Protocolspoint-to-point, multicast, broadcastmessage passing, RPC, distributed objectsstateful or stateless

• Quality of service

Network Choices

Public Internet:

Ubiquitous -- worldwideLow cost

Private network:

SecurityPredictable performanceChoice of protocols (e.g., IBM's SNA)

Quality of Network Services

Performance

Maximum throughputVariations in throughputReal-time media (e.g., audio)

Business

SuppliersTrouble shooting and maintenanceUpgrades

Firewall

Public network

Private network

Firewall

A firewall is a computer at the junction of two network segments that:

• Inspects every packet that attempts to cross the boundary

• Rejects any packet that does not satisfy certain criteria, e.g.,

an incoming request to open a TCP connectionan unknown packet type


Lecture 15

System Architecture II

Distributed and Real Time Systems

Comments on Requirements Report

Audience

• Client and design team• Will be updated over time

Content

• Level of detail -- will be used to validate the implementation• Requirements, not design• Precise, but not legalistic

Sequence Diagram: Notation

BookBorrower


theCopy:Copy

theBook:Book


borrowborrow

dotted line shows object

lifetime

rectangle shows focus

of control

Sequence Diagram: Branching

BookBorrower


theCopy:Copy

theBook:Book

1:borrow(theCopy)2:okToBorrow

[ok]3:borrow4:borrow

branch

[not ok]3:noborrow

Example: Distributed Database

two copies of the same data

Distributed Data and Replication

Distributed Data

Data is held on several computer systems. A transaction may need to assemble data from several sources.

Replication

Several copies of the data are held in different locations.

Mirror: Complete data set is replicated

Cache: Dynamic set of data is replicated (e.g., most recently used)

With replicated data, the biggest problem is consistency.

Example: Broadcast Search

User interfaceserverUser

Databases

Example: UseNet

Stateless Protocol v. Stateful

Stateless protocol

Example: http

Open connectionSend message Return replyClose connection

State in http must be sent with every message (e.g., as parameter string or in a cookie)


Stateful (session) protocol

Example: Z39.50Open connectionBegin sessionInteractive sessionEnd sessionClose connection

Server remembers the results of previous transactions (e.g., authentication, partial results) until session is closed.

Firewall

Public network

Private network

Firewall





The Domain Name System

.edu server

cornell.edu server

cs.cornell.edu server

First attempt to resolve www.cs.cornell.edu

1

2

3

Discussion of the First Attempt

Problems?


.edu server

cornell.edu server


Better method

3

1

almaden.ibm.comcornell.eduece.cmu.eduibm.comacm.org.edu

2

Localcache

local DNS server

Real Time System

A real time system is a software system whose correct functioning depends upon the results produced and the time at which they are produced.

• A soft real time system is degraded if the results are not produced within required time constraints

• A hard real time system fails if the results are not produced within required time constraints

Example: Web Server

http messagedaemon

spawned processesTCP port 80

The daemon listens at port 80.

When a message arrives it:spawns a processes to handle the messagereturns to listening at port 80

Embedded Systems

Software and hardware are combined to provide an integrated unit, usually dedicated to a specific task:

• Digital telephone

• Automobile engine control

• GPS

• Scientific instruments

The software may be embedded in the device in a manner that can not be altered after manufacture.

Example: Autonomous Land Vehicle

Sensors

GPS

Sonar

Laser

Signal processing

Model Control signals

Steer

Throttle

Controls

Other Applications

Response critical

• Network router

• Telephone switch

• Seat bag controller

Shared systems

• Multi-user data processing

• Time sharing

Techniques

• Special purpose hardware

• Multi-threading and multi-tasking

• Parallel processing

=> digital signal processing

• Interrupts

=> levels and priorities

Multi-Threading

Several similar threads operating concurrently:

• Re-entrant code -- separation of pure code from data for each thread

• Testing -- single thread and multi thread

May be real time (e.g., telephone switch) or non-time critical

Real Time Executive

Schedules and dispatches tasks in a real time system

• Real time clock

• Interrupt handler

• Scheduler

• Resource manager

• Dispatcher

Must be extremely reliable

Timing

Timing mechanisms

• Synchronous (clocked) -- periodic stimuli

• Asynchronous -- wait for next signal

Example: Communications protocols may be synchronous or asynchronous

Hardware v. Software

Design of embedded systems requires close understanding of hardware characteristics

• Special purpose hardware requires special tools and expertise.

• Some functions may be implemented in either hardware of software (e.g., floating point unit)

• Design requires separation of functions

Distinction between hardware and software may be blurred.

Example: Dartmouth Time Shared System

Communications processor


Centralprocessor

Centralprocessor

Centralprocessor

I/OMulitplexor

master processor

Software Considerations

Resource considerations may dictate software design and implementation:

• Low level language (e.g., C) where programmer has close link to machine

• Inter-process communication may be too slow (e.g., C fork).

• May implement special buffering, etc., to control timings

Example: CD Controller

Input block Output

block

12

345

67

Circular buffer

Continuous Operation

Many systems must operate continuously

• Software update while operating

• Hardware monitoring and repair

• Alternative power supplies, networks, etc.

• Remote operation

These functions must be designed into the fundamental architecture.

Routers and Other Network Computing

• Interoperation with third party devices

• Support for several versions of protocols

• Restart after total failure

• Defensive programming -- must survive

=> erroneous or malicious messages

=> extreme loads

• Time outs, dropped packets, etc.

• Evolution of network systems