"The Profession of IT, and Software Engineering"

Professor Doug GrantDean of Information Technology

Second Argentine Symposium on Software Engineering, Buenos Aires, 10 – 12 Sept 2001

2

Doug Grant : a Personal Note

EdinburghMelbourne

Second Argentine Symposium on Software Engineering, Buenos Aires, 10 – 12 Sept 2001

3

A Look Back to 1978!

We're on the march with Ally's ArmyWe're going to the ArgentineAnd we'll really shake them up When we WIN the World CupCos Scotland are the Greatest Football team...... EASY!

Second Argentine Symposium on Software Engineering, Buenos Aires, 10 – 12 Sept 2001

4

What Happened?

PERU 4

SCOTLAND 1

Second Argentine Symposium on Software Engineering, Buenos Aires, 10 – 12 Sept 2001

5

What Happened?

Second Argentine Symposium on Software Engineering, Buenos Aires, 10 – 12 Sept 2001

6

Why is This Relevant?

We’ll see!

Second Argentine Symposium on Software Engineering, Buenos Aires, 10 – 12 Sept 2001

7

For the Future Progress of SE, What are the Issues?

• I’ll consider current ‘state-of-practice’ in SE

• I will look briefly at issues on ‘professionalism’ in SE

• I have surveyed academics and practitioners in Australia– Interested in research and education as it impacts

‘mainstream’ practice

• I chaired the recent Australian Software Engineering Conference, and report on the chief agendas of the keynote speakers

Second Argentine Symposium on Software Engineering, Buenos Aires, 10 – 12 Sept 2001

8

DDG Questionnaire

• Should SE be considered as an ‘engineering’ profession?

• How well does current practice conform?• What are the main current issues facing real

software developers (ie practitioners)?• Should the researchers help? • How can the researchers help?• How can the educators help?

NB : NOT fully validated research!

Second Argentine Symposium on Software Engineering, Buenos Aires, 10 – 12 Sept 2001

9

The Profession of IT and Software Engineering

• Peter Denning, in a recent CACM column, suggests we view IT as a profession, not a discipline

• He suggests there are several IT-related disciplines– IT-Specific Disciplines (inc Computer Science, Software Engineering)– IT-Intensive Disciplines (inc Information Systems)– IT-Supportive Occupations

• Denning’s call is for us to focus on the IT profession, for the disciplines to be mutually respective, and to turn away from our previous focus on what makes the different disciplines distinctive

• This could be interpreted, in the SE community, as a call to focus away from our determination to see SE as a profession in itself.

Peter J Denning, “The Profession of IT”, CACM 44 (2), Feb 2001, 15 - 19

Second Argentine Symposium on Software Engineering, Buenos Aires, 10 – 12 Sept 2001

10

The Software Engineering Discipline

• We have sought to validate SE as an ENGINEERING discipline– By identification of ‘outlook’ with the old, traditional

engineering disciplines

• We have sought, in particular, distinction from Computer Science– Emphasising the distinction between Science, as oriented

towards discovery of truth, and Engineering, as application of discovery

• In some respects, validation as ‘Engineering’ has been more important than acceptance as a ‘discipline’

Second Argentine Symposium on Software Engineering, Buenos Aires, 10 – 12 Sept 2001

11

Definition of Engineering

"Engineering is the profession in which a knowledge of the mathematical andnatural sciences, gained by study, experience, and practice, is applied withjudgment to develop ways to utilize, economically, the materials and forcesof nature for the benefit of mankind."

ABET

Second Argentine Symposium on Software Engineering, Buenos Aires, 10 – 12 Sept 2001

12

Definition of Software Engineering

“The application of a systematic, disciplined, quantifiable approach to the development, operation and maintenance of software; that is, the application of engineering to software”

IEEE – CS

Second Argentine Symposium on Software Engineering, Buenos Aires, 10 – 12 Sept 2001

13

Question 1 : Response from Academics

A driving force in the establishment of SE as an engineering profession has been the desire to make the ABET engineering definition true for SE.

N = 28 Academics

(i) To what extent do you believe that (as a whole) current software development practice fulfills this definition :

Second Argentine Symposium on Software Engineering, Buenos Aires, 10 – 12 Sept 2001

14

Comments

• There was a strong bias towards viewing software development practice as not (yet) conforming to the ‘standard’ definition of engineering (as might be interpreted for software)

Second Argentine Symposium on Software Engineering, Buenos Aires, 10 – 12 Sept 2001

15

Question 2 : Response from Academics

To what extent do you believe it is essential for software development practice to fulfill the ABET engineering definition :

N = 28 Academics

Second Argentine Symposium on Software Engineering, Buenos Aires, 10 – 12 Sept 2001

16

Comments

• There was a strong bias towards the importance of software development practice conforming to the ‘standard’ definition of engineering (as might be interpreted for software)

• Hence, by comparison with the results for Q1, there was a quite definite view that current software development practice falls short of the ideal

Second Argentine Symposium on Software Engineering, Buenos Aires, 10 – 12 Sept 2001

17

Requirements for a Profession

• Durable domain of human concerns• Codified body of principles (conceptual knowledge)• Codifies body of practices (embodying knowledge including

competence)• Standards for competence, ethics and practice

Denning suggests that the IT disciplines cannot address these issues on their own. Yet they are a hallmark of the established engineering disciplines. So, perhaps if we retain the desire for SE to be recognised as ‘engineering’, then we cannot escape from the SE thrust towards recognition as a profession in its own right.

Peter J. Denning, “The Profession of IT”, CACM 44(2), Feb 2001, 15 – 19.

Second Argentine Symposium on Software Engineering, Buenos Aires, 10 – 12 Sept 2001

18

Professionalism : How is SE Progressing (1 of 2)?

• Certainly software is a durable area of human concerns

• SWEBOK project is well on the way towards codifying body of principles and practices

• SE Code of Ethics has recently been developed by IEEE-CS/ACM Joint Task Force on Software Engineering Ethics and Professional Practices

• Standards for competence and practice are an area of significant debate (ACM withdrawal from SWEBOK, opposition to licensing, withdrawal from the Software Engineering Coordinating Committee (SWECC))

Second Argentine Symposium on Software Engineering, Buenos Aires, 10 – 12 Sept 2001

19

Professionalism : How is SE Progressing (2 of 2)?

Prof Barrie Thompson, Univ of Sunderland (UK) (at recent ASWEC)•SE is ‘lurching’ towards professionalism•Too many initiatives are excessively US-centric•ACM / IEEE-CS disagreements on SWEBOK and licensing have been a set-back•Potential restrictions on accreditation of SE degrees in US also a problem (esp for the rest of the world)

Second Argentine Symposium on Software Engineering, Buenos Aires, 10 – 12 Sept 2001

20

The Starting Point : A Discipline in Failure, or one Maturing with Success?

• Two views of SE are possible– A discipline in failure, with the average

project failing to meet cost, time and quality constraints?

– A discipline that is clearly maturing, with many great achievements on projects of enormous complexity?

• Which should be our starting point?

Second Argentine Symposium on Software Engineering, Buenos Aires, 10 – 12 Sept 2001

21

Software Engineering : A Discipline in Failure?

• We are all too familiar with the catalog of software ‘disasters’

• We still regale our students with statistics about project failure

Second Argentine Symposium on Software Engineering, Buenos Aires, 10 – 12 Sept 2001

22

Software Engineering : Unrealistic Expectations?

But - have our expectations been unrealistic (like Scotland in the 1978 World Cup)?

• On individual projects– We do well when we adopt best practice

• As a discipline in development– We are young, and have made very fast

progress by comparison with old disciplines

Second Argentine Symposium on Software Engineering, Buenos Aires, 10 – 12 Sept 2001

23

Why do we have ‘Failures’ in software development?

Failure : to meet the scope / cost / time / quality constraints

• Failure to manage requirements? (Thompson)• Poor communication between developers and clients?• Poor estimation techniques? (Robert Glass)• Telling lies, after our good estimations are perceived

to be unacceptable to clients?• Failure to implement known best-practice?

Second Argentine Symposium on Software Engineering, Buenos Aires, 10 – 12 Sept 2001

24

We ARE Getting Better, and we DO know How to Build Software

• We do build many large very complex systems that work!

• Significant improvement in assessed community’s CMM ratings http://www.sei.cmu.edu/sema/profile.html

• Empirical evidence that moving up the CMM levels has positive business impact

• Empirical evidence that the PSP has positive impact on individual developer productivity and competence

• New ‘agile’ processes for small systems, appropriate for some developer cultures

Second Argentine Symposium on Software Engineering, Buenos Aires, 10 – 12 Sept 2001

25

Our Starting Point

I prefer to take as our starting point the FACT that SE is a discipline that IS maturing rapidly, achieving very considerable success, and with much to contribute towards the IT profession

Second Argentine Symposium on Software Engineering, Buenos Aires, 10 – 12 Sept 2001

26

Question 3 : Response from Academics

In your opinion, what are the 3 most pressing issues facing mainstream software developers today?

Second Argentine Symposium on Software Engineering, Buenos Aires, 10 – 12 Sept 2001

27

Question 4 : Response from Academics

To what extent should the immediate needs of mainstream software developers drive the SE research agenda?

N = 28 Academics

Second Argentine Symposium on Software Engineering, Buenos Aires, 10 – 12 Sept 2001

28

Comments

• Opinion neutral, but slight bias toward the need for research to be driven by the immediate concerns of industrial developers

Second Argentine Symposium on Software Engineering, Buenos Aires, 10 – 12 Sept 2001

29

Question 5 : Response from Academics

How well do you think current SE research is addressing the real issues faced by mainstream software development practice?

N = 28 Academics

Second Argentine Symposium on Software Engineering, Buenos Aires, 10 – 12 Sept 2001

30

Comments

• Opinion was neutral, with a slight bias towards research ‘not meeting’ industrial needs

• No one surveyed believed that research was helping practitioners ‘very well’!

• Although researchers have a tendency to believe that meeting industry needs is important (Q4), they don’t believe that it currently does (Q5)

• Yet the researchers clearly don’t want their research agenda to be dominated by immediate practical needs!

Second Argentine Symposium on Software Engineering, Buenos Aires, 10 – 12 Sept 2001

31

Question 6 : Response from Academics

In your opinion, what are the 3 most important research questions in SE today?

Second Argentine Symposium on Software Engineering, Buenos Aires, 10 – 12 Sept 2001

32

Question 7 : Response from Academics

In your opinion, what are the most important facets of SE that you believe an undergraduate degree in SE should ensure in its graduates?

Second Argentine Symposium on Software Engineering, Buenos Aires, 10 – 12 Sept 2001

33

ASWEC keynotes

• The Australian SE Conference (ASWEC 2001) has just taken place

• Four excellent keynote addresses, looking at major current issues in SE– Nazim Mahdavji (Univ of Otago)– Dieter Rombach (Fraunhofer Inst,

Kaiserslautern)– Barrie Thompson (Univ of Sunderland) (already

mentioned)

– Paul Bailes (Univ of Queensland)

Second Argentine Symposium on Software Engineering, Buenos Aires, 10 – 12 Sept 2001

34

Madhavji

• We need ‘grounded theories’

• In process improvement, WHAT works, and WHY?

• We need to open up the ‘black box’ to observe the relationships between activities and outcomes

• EG, in PSP, WHY do certain threshold values for ratios imply better performance

Second Argentine Symposium on Software Engineering, Buenos Aires, 10 – 12 Sept 2001

35

Rombach

• We need empirical evidence to back our advocacy

• Process improvement is no good without product improvement

• We want to build process models where we can predict their efficacy

Second Argentine Symposium on Software Engineering, Buenos Aires, 10 – 12 Sept 2001

36

Bailes

• ‘new’ research agenda based on a serious re-consideration of functional programming– Programming = language extension

• Against the tide– But how else do we make serious

progress

• (As a comment : we MUST escape the constraints of the big guys, no matter how forceful and clever they are, because their motives for progress are value-laden! So it is good to reflect once again on approaches to computation that are away from the mainstream)

Second Argentine Symposium on Software Engineering, Buenos Aires, 10 – 12 Sept 2001

37

Conclusions

• Two presentations that emphasised empirical research, and understanding of underlying theories that explain the observations : process-oriented, towards the development of predictable processes

• One presentation that focussed on invention of theoretically sound, very different techniques, that, although ‘old’ in concept, challenge today’s accepted ‘wisdom’

• In my view : a great snapshot of where our research has to head!