master thesis - masarykova univerzita

46
MASARYKOVA UNIVERZITA FAKULTA INFORMATIKY Topic packages as an e-learning service in IS MU MASTER THESIS Bc. Peter Bálint Brno, autumn 2011

Upload: others

Post on 13-Mar-2022

7 views

Category:

Documents


0 download

TRANSCRIPT

MASARYKOVA UNIVERZITA

FAKULTA INFORMATIKY

}w���������� ������������� !"#$%&'()+,-./012345<yA|Topic packages as an e-learning

service in IS MU

MASTER THESIS

Bc. Peter Bálint

Brno, autumn 2011

Declaration

Hereby I declare, that this paper is my original authorial work, which I have worked outby my own. All sources, references and literature used or excerpted during elaborationof this work are properly cited and listed in complete reference to the due source.

Advisor: doc. Ing. Michal Brandejs, CSc.

ii

Acknowledgement

I wish to thank my advisor doc. Ing. Michal Brandejs, CSc. and IS development teamfor their help and advice during the analysis and implementation of described services.I would also like to thank E-tech group for their patience during the testing of the newfunctions and applications. Last but not least I would like to thank my parents for theirendless support.

iii

Keywords

e-learning, IS MU, information system, service, topic, package

iv

Abstract

The objective of this thesis is to describe the current implementation of Topic packagesin IS MU, analyze this implementation and, in cooperation with E-tech user supportgroup, design, develop and implement new features requested by academic public ofthe university.

In first chapter I would like to briefly explain what exactly is E-learning, point outseveral key application of IS and describe the current principles behind the Topic pack-ages in IS MU, their positive and negative aspects and also to point out the main draw-backs that prevent this application to by widely used. In this chapter I will also conductthe comparison of Topic packages and other application used for the same purposeoutside the Information System (IS).

In the second chapter I will analyze the requests posted on Suggestions discussiongroups by teachers, students and members of user support.

In third chapter I will describe the process of analysis in terms of service science,techniques I used to manage and control this extensive project and also methods thatwere necessary to achieve the project goals in the time frame I was given.

In fourth chapter I will describe methods I used to increase inter-connectivity ofTopic packages agenda with other E-learning agendas, such as Teacher‘s notebook,ROPOT (Revision, Opinion Poll and Testing) application etc. In this chapter I wouldalso like to describe the service background behind the cooperation processes with theE-tech user support group.

In the last chapter I will summarize the main analysis and implementation problemsas well as complications that came up during the design stage of the project.

The solutions and implementation methods described in this thesis have alreadybeen tested (or are currently being tested) and implemented and are being used byover thirty thousand users of Information system of Masaryk University.

v

Contents

1 Topic packages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31.1 E-learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

1.1.1 E-learning applications . . . . . . . . . . . . . . . . . . . . . . . . 3Teacher‘s notebook . . . . . . . . . . . . . . . . . . . . . . . . . . . 4ROPOT (Revision, Opinion Poll and Testing) . . . . . . . . . . . 4Interactive syllabi . . . . . . . . . . . . . . . . . . . . . . . . . . . 4Homework vaults . . . . . . . . . . . . . . . . . . . . . . . . . . . 4Study materials . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

1.2 Current implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51.2.1 Package . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51.2.2 Topic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

1.3 Positives and negatives of current implementation . . . . . . . . . . . . . 61.3.1 Positives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61.3.2 Negatives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61.3.3 Main drawbacks . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

1.4 Comparison with applications outside IS MU . . . . . . . . . . . . . . . . 72 User and support request analysis . . . . . . . . . . . . . . . . . . . . . . . . . 8

2.1 Main sources of request and suggestions . . . . . . . . . . . . . . . . . . . 82.1.1 Suggestions discussion groups . . . . . . . . . . . . . . . . . . . . 82.1.2 User support group E-tech . . . . . . . . . . . . . . . . . . . . . . 82.1.3 Feature selection . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

3 Project life-cycle . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103.1 Project management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

3.1.1 Definition of project management . . . . . . . . . . . . . . . . . . 103.1.2 Project . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103.1.3 Project product . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113.1.4 Triple Constraint („trojimperativ”) . . . . . . . . . . . . . . . . . . 113.1.5 ToC (Theory of Constraints) . . . . . . . . . . . . . . . . . . . . . . 123.1.6 Project life cycle . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

Initiation stage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13Planning stage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14Execution stage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14Control and monitoring . . . . . . . . . . . . . . . . . . . . . . . . 14Closure stage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

3.2 Project planning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15Plan WHAT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15Plan HOW . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16Plan WHO . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16Plan WHEN . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

1

Plan FOR HOW MUCH . . . . . . . . . . . . . . . . . . . . . . . . 163.2.1 Work Breakdown Structure . . . . . . . . . . . . . . . . . . . . . . 163.2.2 Gannt chart . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

Milestones . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 193.2.3 PERT (Project Evaluation and Review Technique) . . . . . . . . . 193.2.4 Network diagram . . . . . . . . . . . . . . . . . . . . . . . . . . . . 203.2.5 CPM (Critical Path Method) . . . . . . . . . . . . . . . . . . . . . . 213.2.6 Project plan . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21

3.3 Risk management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 223.3.1 Risk identification methods . . . . . . . . . . . . . . . . . . . . . . 23

Brainstorming . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24Talking with experts . . . . . . . . . . . . . . . . . . . . . . . . . . 24SWOT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24Black swan theory . . . . . . . . . . . . . . . . . . . . . . . . . . . 24

3.3.2 Risk analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25Qualitative risk analysis . . . . . . . . . . . . . . . . . . . . . . . . 25Quantitative risk analysis . . . . . . . . . . . . . . . . . . . . . . . 25

3.3.3 Risk elimination . . . . . . . . . . . . . . . . . . . . . . . . . . . . 263.3.4 Monitoring and risk assessment . . . . . . . . . . . . . . . . . . . 26

3.4 Program & Portfolio . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 273.4.1 Program . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 273.4.2 Portfolio . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 273.4.3 The comparison . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27

4 Project definition and implementation, testing . . . . . . . . . . . . . . . . . 294.1 Basic analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 294.2 Increase of inter-connectivity with e-learning agendas . . . . . . . . . . . 30

4.2.1 Connection between Topic packages and Classes . . . . . . . . . . 304.2.2 Sharing Topic packages between multiple courses . . . . . . . . . 314.2.3 Assigning Teacher‘s notebooks to topics . . . . . . . . . . . . . . . 324.2.4 File vaults assigned to packages & topics . . . . . . . . . . . . . . 334.2.5 AJAX & JavaScript . . . . . . . . . . . . . . . . . . . . . . . . . . . 344.2.6 Packages accessible via simple URL . . . . . . . . . . . . . . . . . 35

4.3 Testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 355 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36A Attachments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37

A.1 List of suggested features, upgrades, etc... (in Czech) . . . . . . . . . . . . 37

2

Chapter 1

Topic packages

1.1 E-learning

The term E-learning (also Elearning, eLearning) stand for electronically supported teach-ing and learning. In this case computer systems are the main media to implement thelearning process. The term usually references to education through the means of tech-nology. E-learning is basically skill and knowledge transfer. E-learning consists of appli-cations such as Web-based learning or computer-based learning. The learning contentis transferred over the Internet. It can be self-led or instructor-led (depending on thenature of requirements) and includes all kinds of media, e.g. text, image, animation,streamed video and/or audio. So far E-learning went through 3 development genera-tions.

IS MU uses the third generation, E-learning 2.0, which brings several improvementin comparison to older standards, such as:

• shorter development time (in comparison to E-learning 1.0)

• delivery system (in E-learning 1.0 information could be delivered only at onetime, E-learning 2.0 provides information whenever it is needed)

• Learning management system (LMS, in 1.0) vs. Search engines and RSS feeds (2.0)

Main components of E-learning 2.0 are:

• Wikis - web based content created by users

• Social networking and bookmarking tools

• Blogs

• Add-ins

• Mash-ups

1.1.1 E-learning applications

IS MU uses this concept of E-learning in several key applications which are used ondaily basis by hundreds of teacher across the entire university. There are other appli-cation solutions for E-learning agendas, such as Moodle, however due to the highlysensitive data stored in IS and the extend of required options, the IS development teamdevised their own solution.

3

1.1. E-LEARNING

Teacher‘s notebook

Teacher‘s notebook provides teachers with all the means necessary to effectively con-duct a collage course. It allows them to evaluate their semestral work via notebooks.These notebooks may or may not be available to students. This depends on the wayteacher wishes to use them. Either as a way of communication with students, wherehe can point out their successes or shortcomings. Or it can be just a place for teacher‘snotes to student‘s work. According to these notebooks teacher can grade students atthe end of semester.

ROPOT (Revision, Opinion Poll and Testing)

Without a doubt one of the most complex agendas in IS. Includes applications for cre-ating and editing question sets, creating ROPOT descriptions, over-viewing student‘sanswers, correcting incorrect questions and distributing test results into student‘s note-books. These applications proved to be invaluable in large courses (500+ students).With such a large number of students, standard testing would take several days, oftenup to 2 weeks. However ROPOT provides “scanned tests”, that are evaluated automat-ically. It is now common practice (mainly on FI, but also on several other faculties), that200+ students from one examination date have their final grades within three or fourhours after the exam itself.

Of course ROPOTs have dozens of combinations of settings to accommodate allrequirements (simple testing, examination at the computer with teacher‘s supervision,etc.)

Interactive syllabi

Interactive syllabi help teacher to divide the semester into several blocks, each concen-trated on a specific topic. Each topic can have a set of subtopics, that are made availableto students each week as the semester progresses (or all at once, depending on teacher‘ssettings). Teacher can include URLs relevant to the course, pictures, audio files or ani-mations as well. Syllabus can also be linked to existing ROPOT, where students can tryout their acquired knowledge.

Homework vaults

Special folders for collecting homework electronically. Teacher can set a deadline uponwhich students must submit their papers. After this time system automatically locksthe folder. Teacher can than start over-viewing the papers and immediately grade themvia student‘s notebooks and provide the results to students.

Study materials

Special folder in IS File manager created for each individual course (except specialcases, when one Study material folder is shared among more courses, typically becausethey are conducted by one teacher or have very similar syllabus). In this folder, all e-learning materials are stored:

• ROPOT question definitions, ROPOT description files

4

1.2. CURRENT IMPLEMENTATION

• interactive syllabi files

• teacher‘s information regarding the course organization

• Homework vault folder

• study materials uploaded by teacher (slides for lectures, old versions of test, etc.)

1.2 Current implementation

The base model behind current Topic packages implementation consists of packagesand topics. In general package is a larger unit that contains a number of topics. Studentsenroll in particular topics, the number of which is set within package settings. Eachpackage and topic contain information about dates, pre-requisites and other vital dataneeded to manage them effectively.

1.2.1 Package

Package is an organized unit that contains closely related topics, e.g. package of mastertheses abstracts, seminar works etc. Each package has several attributes.

• name of the package - designation by which students can identify the package

• note - further information about the package or any information supervisor wishesto provide

• date range - a period of time during which students can choose topics from thispackage or create topics in this package (if supervisor allows this)

• maximum number of topics - describes the number of topics of this package onestudent can enroll in

• enrollment pre-requisites - list of courses students must successfully complete be-fore he/she is allowed to enroll in this package

• voluntary list of information that will be stored with the package (e.g. approvalof the head of the department, defence date, additional note, etc.)

• information whether students can submit their topics into this package

1.2.2 Topic

Topic is the smallest unit within the package. It describes one particular problem andprovides further information about it. Topic, as well as packages, stores a number ofinformation:

• topic is available to students - information whether this topic is already available tostudent to enroll in

• list of persons responsible for topic (e.g. supervisor, consultant, opponent, contactperson, etc.)

5

1.3. POSITIVES AND NEGATIVES OF CURRENT IMPLEMENTATION

• name of the topic - topic‘s designation (topic also allows to store information aboutthe name in english and the name‘s transcription in TEX syntax)

• preliminary and final topic description - supervisor can specify these description sep-arately as they may change in time and supervisor wants to store both variants

• literature - list of publications supervisor suggests to be read or supervisor consid-ers to be helpful

• related website

• pre-requisites - list of courses the student must complete successfully to be allowedto enroll in this topic

• note - information not necessarily related to the topic itself, but that may proveuseful

• notifications

◦ supervisor will be notified if student enrolls in this topic

◦ supervisor‘s approval is required for students to enroll in this topic

1.3 Positives and negatives of current implementation

1.3.1 Positives

Even though Topic packages (TP) agenda was designed when modern web standardsdid not exist, it‘s complexity still surpasses modern applications that are used for sim-ilar purposes. TP integration into IS e-learning services makes it a very useful tool.In combination with the massive amounts of data stored in IS, teacher can create andmaintain sets of topics (or packages) that can address a wide audience.

Using Topic packages makes it easier to organize various scenarios that are usedin courses as well as across entire faculties. By organizing different topics into sepa-rate packages it‘s a lot easier to browse through them, especially for students that arelooking for one particular topic they would like to process.

1.3.2 Negatives

Information system (IS) was firstly presented in early 1999. Topics packages is one ofthe oldest applications, which means that during it‘s implementation, requirements foruser interface were very different. This is the main cause why the application strikesusers as “unfriendly”. Set-up screens for package or topic look fairly complicated anddiscourage teachers from using them. Also the number of available operations withselected package is quite extensive, however since they are all displayed on one set-uppage, the overall look seems incoherent and without internal logic.

1.3.3 Main drawbacks

The main drawback is most likely the user interface, which does not follow modernUI standards (this is caused by the age of the application). Some parts are is still quite

6

1.4. COMPARISON WITH APPLICATIONS OUTSIDE IS MU

sufficient, however other e-learning agenda in IS have developed over the years andTopic packages have been neglected. This is the main cause why it is quite separatedfrom other important E-learning applications, mainly Teacher‘s notebook and ROPOTagenda. This inter-connectivity will be improved by adjustments and new applicationsdeveloped during the writing of this thesis. Also extensive changes were made in thebehavior of the applications, especially during the manipulation with multiple pack-ages and topics at once.

1.4 Comparison with applications outside IS MU

There are several open-source or payed solutions for E-learning agendas on the market.However during my research I have encountered none so complex as the one integratedin IS. Most of the are programmed in PHP (or PHP frameworks) in combination withJavaScript and AJAX (none of them sophisticated enough is programmed in Perl suchas IS). Among the most interesting were:

• Moodle - probably most commonly used, even on some MU faculties (FF)

• eFront - has it‘s own agenda similar to ROPOT

• Dokeos- provides Flash video video-conferencing

• Olat - has it‘s own integrated instant messaging system

Even though I tried to find something similar to Topic packages in IS I was not ableto find a complete match. Closest to the concept was eFront with it‘s agenda Project.However project is a final unit, it can not be divided into smaller units (as oppositeto the concept Package -> Topic). Project can have it‘s own deadlines, participants andlist of objectives. Also each project has it‘s own web space (this feature is among newlyimplemented features of Topic packages). Also eFront Project allows the administrationof user groups with various rights within project (also this feature is implemented inTopic packages, although in slightly different form).

In conclusion (even in it‘s current state, if we omit UI) the IS implementation pro-vides a lot more complex solution. Teachers can set up and store vital information forevery package and topic and since IS provides data about courses, that students havesuccessfully accomplished, teachers can manipulate topics in such way, so studentswith specific knowledge base would find the right topic for their area of interest.

7

Chapter 2

User and support request analysis

2.1 Main sources of request and suggestions

2.1.1 Suggestions discussion groups

IS MU has three dedicated discussion groups:

• student‘s suggestions

• teacher‘s suggestions

• clerk‘s suggestions

Each discussion group provides valuable input for development team on how to im-prove information system in the way users wish to use it. Each discussion group con-tains multiple discussion threads, each dedicated to one specific agenda. Over the yearsusers have posted dozens of suggestions for functions they would expect in respectiveagendas. Suggestions for Topics packages could be divided into two groups:

• suggestions for packages - how to improve the process of creating, editing andpublishing new packages, new features, more attributes that could be stored foreach package, improvement of UI etc.

• suggestions for topics - few suggestions were similar to those posted for packages,e.g. improvements of UI, several new attributes that could be stored within topicetc. However there were several suggestions that are unique for topics (these willbe explained in detail later on)

2.1.2 User support group E-tech

The user support group (E-tech) is a group of CVT employees who specialize in e-learning agendas, their usage in education process and helping teachers to use it. Weconducted 2 major meetings, where request from teachers and the E-tech group werediscussed. Since E-tech has most experience with dealing with user‘s requests (andcomplaints) these meetings allowed us to anticipate problems in design and implemen-tation long before they would occur. Via this approach I was able to follow the projecttime line with great precision during the entire project. Requests submitted by E-techoriginated from their experience with users over the years, which means many of theirsuggestions were already considered to be implemented. However a large number oftheir suggestions were designed to help them use Topic packages simultaneously inmany courses at once. So their main aim were features for manipulation with packagesand topics in bulks, import and export according to a number of different criteria etc.

8

2.1. MAIN SOURCES OF REQUEST AND SUGGESTIONS

2.1.3 Feature selection

Many suggestions, that were proposed, increased inter-connectivity with other e-learningagendas (Homework vaults, Notebooks, sharing one package across several courses,etc.). All these suggestions were over-viewed and analyzed. About 23% were dismissedfor various reasons, such as:

• function requirements were not specific enough - neither users nor E-tech were entirelysure what would the expect from the feature (application). Such applications oftenprove a waste of resources since nobody know what to use them for

• implementation was too difficult (time consuming) - it would be (nearly) impossibleto implement such a feature (application) in the given time frame

Suggestions and requests that we chose to implement (among others) were:

• assign a specific file vault to each package and/or topic

• assign a specific Notebook to each package and/or topic

• add a deadline setting for each individual topic, that would supersede the pack-age setting

• bulk manipulation with topics

◦ create, edit and delete topics

◦ edit certain attributes of topics

◦ assign persons to topics

• bulk manipulation with packages

◦ editing attributes

◦ adding supervisor, opponents, consultants, ...

• assigning virtual classes to particular topics (in case more than one student isenrolled in the topic)

• dedicated web space

◦ dedicated discussion group, noticeboard, e-mail notifications

◦ accessible only for the students enrolled in the topic

9

Chapter 3

Project life-cycle

3.1 Project management

Managing the project was largely my responsibility, from planning the meetings withE-tech, IS development team analyst and my advisor to risk assessment, time manage-ment etc. Especially time management proved to be very challenging since my otherresponsibilities at work often took precedence. Other projects (especially from othercollages that we outsource IS to) often required my immediate attention, which obvi-ously resulted in unexpected gaps in time plan. I was able to almost eliminate thesegaps by moving the project development to a less critical time slots, usually duringnights and weekends, when I was able to maintain concentration on a single task forlonger periods of time. Without the necessity of context shifting [5] my productivitywas significantly increased and I was able to follow the time plan almost without delay.

3.1.1 Definition of project management

According to Project management institute (PMI), project management is the appli-cation of knowledge, skills, tools and techniques to project activities to meet projectrequirements.

In general we can describe project management as a set of knowledge, techniquesand tools that allow us to accomplish project goals in designated budget, quality 1 andtime.

3.1.2 Project

Project has several slightly different definitions according to different sources

• ISO 10 006 definition - project is a unique process comprised of a series of coor-dinated and controlled activities with given start and completion dates that iscarried out in order to accomplish a pre-defined goal that meets specified require-ments including time, cost and resource constraints.

• IMPA definition - project is a time and cost constrained operation to realize a setof defined deliverables (the scope to fulfill the project’s objectives) up to qualitystandards and requirements.

• PMI definition - a project is a temporary endeavor undertaken to create a uniqueproduct, service, or result.[3]

1. Quality - project product, that fulfills product specification

10

3.1. PROJECT MANAGEMENT

• Project definition according to Zdenko Stanícek - project is a sequence of activitieswith a start and an end, and allocated resources, directed to the creation of certainproducts or services. [8]

The described definitions suggest the following project attributes:

• uniqueness - each project is conducted just once

• temporariness - project time frame, start and finish dates

• resources - projects are maintained via human, material or other sources

• goal - project has defined product (expected result of the project)

3.1.3 Project product

The goal of a project is to fulfill specified expectations and to create a unique product- service, object or other quantifiable result. Project product is goal or result or otherproject deliverable, that should be created during the project [2]. An important projectattribute is it‘s uniqueness. The aspect of similarity should however be considered.Computer System Unit at Faculty of Informatics is currently outsourcing Informationsystem to eight other collages. And even though many aspect of the deployment aresimilar, some are very different, i.e. terms, risks, budget, etc.

3.1.4 Triple Constraint („trojimperativ”)

The main aim of project management is to fulfill project goals. However several aspectsof project are mutually contradictory and usually can not be satisfied to the higheststandard at the same time. Three of these aspects, that are contradictory most often,are:

• quality

• time

• resources

In 1969 Martin Barnes was the first one to depict these three project aspects in a trian-gle. The original aspect of „quality” was later transformed into „performance”, sincethe perception of quality differs between customer and contractor. From the contrac-tor‘s point of view quality product is a product that meets the project specifications.However from the customer‘s point of view a quality product is the one that bringsprofit.

The „triple constraint (trojimperativ)” term was created by project management ex-pert Dr. Zdenko Stanícek. He describes the triangle as composition of WHAT should bedone, WHEN should it be done and FOR HOW MUCH should it be done.

11

3.1. PROJECT MANAGEMENT

Picture 1 - Triple constraint triangle

The balance between following constraints must be found in order to deliver projectin time with approved budget and of the highest possible quality:

• quick and of high standard, but not cheap

• quick and cheap, but not of high quality

• high quality and cheap, but takes a long time.

3.1.5 ToC (Theory of Constraints)

The basic claim of ToC is that every project is limited by constraints. The fundamentalinsight of the Theory of Constraints is: „the output of any system is determined by onebottleneck.” A chain is as strong as its weakest link and if we want to make the chainstronger, we need to work on the weakest link. A constraint is anything that preventsthe system from achieving most of it‘s potential. To remove these constraints ToC usesfive focusing steps:

1. Identify constraints - it is possible to find more than one constraint of the project,however there are never many

2. Find a way to exploit the constraints - take the limitations and make the most out ofthem in any way possible

3. Non-constraints should be subordinated to the decisions made in steps 1 and 2 - it isnot critical for the non-constraints in the system to be the best. However it iscritical to operate in such way that the constraints are performing as decided bymanagement „exploit” decisions.

4. Elevate the constraints - After performing steps 1, 2 and 3 you should be able toachieve as much performance from the system as possible. If this is not the caseor if you want higher levels of performance there must be other factors that arelimiting you.

5. If a new constraint is created due to the effects caused by steps 1-4, repeat theprocess from step one.

12

3.1. PROJECT MANAGEMENT

3.1.6 Project life cycle

Each project, while we consider it to be a process, can be divided into several separateparts - stages. These stages describe the project life cycle and are conducted sequen-tially. Separation allows us to better evaluate each stage, define deliverables and also tosimplify project management and control. As every project is different and unique inmany ways, so are the stages of each project. Stages can differ in each project or orga-nization and these differences can be caused by a number of various factors, i.e. teamhabits, organization environment, etc.

The transition between stages is a good opportunity for control of the deliverablesfrom previous stages of the project. Also it enables the participants to be aware of allproject aspects and it increases the project‘s chance to succeed.

Life cycle of each stage is defined by:

• What should be done during this stage (definition of activities and deliverables)

• Who is participating on activities during this stage

As I have previously mentioned, each project can be divided into stages. And eventhough these stages can differ project to project, there is some basic resemblance amongthem.

Picture 2 - Project life cycle

Initiation stage

This stage is basically about formally starting the project. Usually the common activitiesare:

• identification of the project - to formally describe what is the project going to beabout, create a business case

• definition of project deliverables

• establish the project charter

13

3.1. PROJECT MANAGEMENT

• appoint the project team

Activities performed in the initiation stage should reflect and answer questions, suchas what is the point of the project, what are the project benefits, what are we trying toachieve, what will be the project environment, what are the project risks etc. Productsof initiation stage are documents, that describe the project:

• list of members of project team

• project definition document describing the project

• defined scope of the project

• approved business case 2

Planning stage

This stage follows the initiation stage. In order to create a valid project plan we mustestablish 5 plans: WHAT, HOW, WHO, WHEN and FOR HOW MUCH it should bedone. Deliverables from the previous stage are crucial in this project, since they provideinsight on project goals, time frame and available resources.

Execution stage

Activities conducted in this stage generate deliverables. Both activities and deliverablesmust be described in project plan created in initiation stage.

It is essential to coordinate all kinds of resources during this stage in order to achieveproject goals in requested quality and time frame. It is also important to ensure thefollowing:

• project deliverables quality

• risk management

• project documentation

• conduct activities according to time plan

Control and monitoring

This stage is usually conducted simultaneously with the execution stage. This way itis possible to detect problems in the execution process and to take necessary actions toprevent time delay or unnecessary waste of resources. Since it is impossible to coverall basis during the preliminary analysis, the process of control and monitoring is oftenthe only way to prevent costly delays. Process of control and monitoring can occur„naturally”, however the efficiency of this process can be greatly increased if these rulesare followed:

• risk monitoring

2. Business case - describes the reasons why the project is being conducted

14

3.2. PROJECT PLANNING

• project efficiency monitoring

• time frame control - if the delay is too long we should adjust assigned resources

• continuous control of deliverables

The deliverables of this stage are project changes that are necessary in order to preventfurther delay, recommended pre-emptive actions, project progress reports, updates ofproject plan, etc.

Closure stage

This is the final stage in project life cycle. Usually it involves handing the deliverablesover to the customer, signing contracts and/or evaluating the project itself. It is goodpractice, during this stage, to summarize the experience of the project, list the problemsand their solutions in order to prevent these complications from happening in futureprojects, thus increasing our ability to deal with unexpected complications.

Typical deliverables of this stage are:

• final project products

• project final report

• „EXPerience document” - this document describes internal experience, problemsand their solutions and suggestions for future projects

3.2 Project planning

To successfully plan a project means to find answers to following questions:

• WHAT should be done?

• HOW should it be done?

• WHO should it be done with?

• WHEN should it be done?

• FOR HOW MUCH should it be done?

Plan WHAT

The WHAT plan contains precise products specification that should be created duringthe project solving (scope). Time issue is irrelevant in this plan. The aim of this planis to define criteria of product quality. These criteria allow us to evaluate whether wefulfilled the project specifications or not. The WHAT plan is usually described by WBS(Work Breakdown Structure).

15

3.2. PROJECT PLANNING

Plan HOW

The main objective of HOW plan is to establish particular deliverables of WHAT plan.Time issue is irrelevant, as well as resources issue. Usually 3-level hierarchy is used:

• stages - sequentially organized, at the end of each stage there is a “gate”. Projectcan be stopped at any gate if it obvious that project has no chance of success

• steps - are performed during each stage. They can be performed sequentially or inparallel to each other. Steps are planned (as well as stages) for the entire project.The deliverable of each step is a partial product.

• tasks - is the smallest unit of project planning. Tasks are assigned to individualsor small teams (2-3 members). Tasks are usually planned only for the current onnext stage of project, since the project change constantly and partial tasks wouldhave to be re-planned (waste of time and resources).

Plan HOW may have a form of project charter. The co called stage-gate principle allowsbetter monitoring and project control. In time between stages it is possible to evaluategoals, that were achieved during the finished stage according to plan and, if neces-sary, to adapt the next stage accordingly to the new development, thus more effectivelyshape the project. The stage-gate principle is described by Zdenko Stanícek, Ph.D. inarticle [1]

Plan WHO

Describes all interested parties in a project and their roles. The deliverable of this plan isthe organizational structure of the entire project, responsibilities of respective membersof the team for product quality in plan WHAT and for processes in plan HOW.

Plan WHEN

Time planning. This is very important part of project planning. The WHAT plan de-scribes the order in which particular stages of project will be executed. It also assignsresources that are responsible for the fulfillment of partial products in respective stagesof the project.

Plan FOR HOW MUCH

The final part of project planning is to describe financial background of all project ele-ments. This means detailed estimate of resources needed to deliver the project in timeand do so with allocated resources. This particular stage of planning also involves theestimate of the duration of tasks and the total cost of the project.

3.2.1 Work Breakdown Structure

One of the basic tools of project management is work breakdown structure. It describesdetailed list of work assignments that need to be done. The following guidelines shouldbe followed during creation of WBS:

16

3.2. PROJECT PLANNING

• top level represents the final deliverable or project

• sub-deliverables contain work packages that are assigned to a team of a unit

• work packages should be independent of other work packages in the WBS

• work packages are unique and should not be duplicated across the WBS

• all elements of the WBS don’t have to be defined to the same level

To create WBS several methods can be used:

• brainstorming - for the entirely new projects, where there are no previously suc-cessful project, team brainstorming might help to find essential products andwork assignments for the project

• inspiration from similar project - if there is WBS of a similar project available thisis a good and quick way to create a new one by adjusting the original WBS toour needs and specifications. However it is important to take the problems ofprevious project into account and apply the experience from the previous projectin the current one.

• choose products from the list - the list is usually part of service oriented organiza-tion‘s methodology

• expert help - in some cases (especially if the team has no experience in particulararea or don‘t have the necessary know-how) it is common to hire an expert for thegiven area, that will help to establish WBS for the project.

The following diagram show approximate flow of work as we proceeded in coopera-tion with E-tech team. Each 2nd level bubble (blue) represents one stage in develop-ment process. Green bubble describes specific activities performed during that particu-lar stage.

Testing stage was somewhat specific as it required iterative approach in order to fixas many potential bugs as possible. In first testing stage E-tech tested improvements im-plemented into existing applications and function in order to ensure compatibility withpackages and topics created before the changes were implemented. In second stage theytested entirely new applications and they interoperability with other e-learning agen-das, such as ROPOT and Teacher‘s notebook. In third and final testing stage they testedTopic packages as a whole, it‘s behavior in unusual settings and whether the data aredistributed to other parts of IS correctly.

17

3.2. PROJECT PLANNING

Picture 3 - Work Breakdown Structure

3.2.2 Gannt chart

There are several ways to describe project time plan. The most important are (accordingto [2]):

• milestones and important dates

• WBS transformed into sequence of steps and tasks

• approximate length of work stages

• relations between work stages (this is very helpful to keep the internal logic ofwork-flow coherent)

• information to keep the schedule in alignment with project monitoring and con-trol during the project life cycle

Gannt chart is a very useful way of showing activities (tasks or events) displayedagainst time. Each activity is represented by a bar. The position and length of the barreflects the start date, duration and end date of the activity. Each activity has it‘s ownassigned resources and time frame in which it should be completed. It provides a graph-ical illustration of a schedule that helps to plan, coordinate, and track specific tasks in aproject. The main advantage of Gannt chart is that important time information can beeasily over-viewed, such as:

• what activities are in the project

• when each activity begins and ends

18

3.2. PROJECT PLANNING

• schedule of each activity (how long should it last)

• where activities overlap with other activities and by how much they overlap

• the start and end dates of the whole project

Picture 4 - Gannt chart

Milestones

Part of Gannt diagram are milestones. Usually they describe significant points duringthe project development that have to be approved of before the project is allowed toprogress any further.

Milestone Date

Project start 1. 7. 2011

First preliminary meeting with E-tech 15. 7. 2011

1st. project stage (improvement of existing functionality 30. 7. 2011

2nd. project stage (new functions implementation) 18. 10. 2012

3rd. project stage (testing) 15. 12. 2012

Project finish 21. 1. 2012

Table 1 - Project milestones

3.2.3 PERT (Project Evaluation and Review Technique)

PERT method was originally developed by U.S. Navy in the 1950‘s to manage the Po-laris submarine missile program. A similar technique, CPM, was developed at aboutthe same time in private sector.

A PERT chart is a graphic illustration of a project as a network diagram. This dia-gram consists of numbered nodes representing the events, or milestones in the project,linked by labeled arrows (directional lines) representing tasks. The direction of the ar-row on the line indicates the sequence in which the tasks must be performed.

Tasks that are performed at the same time are called parallel (or concurrent) tasks.These tasks do not depend on completion of one another and can be performed atthe same time regardless of the results of other concurrent tasks. Tasks, that must becompleted in a sequence, are called dependent (or serial) tasks. One task can only beginif the previous task is already finished. Tasks, that must be completed in a sequence, but

19

3.2. PROJECT PLANNING

that do not require any resources or don‘t have completion time, are considered to haveevent dependency (they are also called dummy activities).

The process of creating a PERT chart consists of the following steps:

1. Identify the specific activities and milestones

(a) activities are tasks required to complete the project

(b) milestones mark the beginning and the end of one or more tasks

2. Determine the correct sequence of activity

3. Create a network diagram

(a) diagram do show serial and parallel activities during the project

4. Estimate the time for each activity

(a) in PERT it is possible to use 3 time estimates:

i. optimistic time - the shortest time in which the activity can be completedii. pessimistic time - the longest time the activity might requireiii. most likely time - time the activity will most likely require

5. Find the critical path

(a) critical path is determined by adding times to each activity in each sequenceand finding the longest path

(b) change of time requirements of activities outside the critical path do notchange the timing of the entire project

6. Change the PERT chart according to project progress

(a) changing estimated times to actual times the activities needed

(b) if there are delays additional resources might be needed

3.2.4 Network diagram

Network diagram is used to describe the contextual dependencies between activitiesduring the project. It consists of bubbles (rectangles) and arrows. Each arrow representsan activity, each bubble represents a milestone. Between correlated activities there arerestrictive rules, such as that one activity can not start before the related activity is over(serial activities). There are three main types of network diagram:

• AON - Activity On Node - nodes represent activities, arrows between nodes rep-resent relations between activities (serial vs. parallel)

• AOA - Activity On Arrow - nodes represent milestones, arrows represent activi-ties

• EON - Event On Node - events are represented by nodes (events - beginning andend of activities)

20

3.2. PROJECT PLANNING

One of the main conditions for a valid network diagram is that it doesn’t contain anycircular references. Network diagrams are easily adjustable to changes in project devel-opment. In case the activity duration change it is not necessary to recreate the entirediagram, but only to change the dates in respective nodes.

3.2.5 CPM (Critical Path Method)

The critical path method is technique for process planning that defines critical and non-critical tasks. The goal is to prevent time frame problems and process bottlenecks. Inorder to create a valid critical path these steps should be followed:

• identify the critical and non-critical paths between activities

• determine the expected completion or execution time for each activity

• locate or create alternatives (backup plans) for the most critical paths

For each activity CPM utilizes four theoretically possible values:

• ES - early start

• EF - early finish

• LS - late start

• LF - late finish

Baseline for time planning according to CPM are activities and their time durations.Project start date (ES) and project finish date (LF). Latest allowed dates are extrapolatedfrom these dates. Time reserves and critical path can be determined from the results.

Forward Pass method - is used when the Early Start date is set. This method adds theduration of each activity from left to right to ES date. This way we calculate ES and EFfor each activity.

Graph iteration with highest time duration is the critical path. In critical path weuse EF also as LF.

Backward Pass method - in this method we proceed from right to left and calculate LSand LF. Time reserve of the activity is determined by the difference between it‘s EF andLF.

3.2.6 Project plan

Using previously described concepts we can now create a project plan. There are 5 basicsteps towards it:

1. Build network diagram - This step is based on WBS. In first step we create product-oriented WBS with time duration of the tasks and relations to previous tasks. Wealso try to determine whether the tasks can be performed in parallel or if the tasksneed deliverables from previous tasks. In the first version of network diagram weconsider only “finish to start” tasks (one task can start only after the previous taskis finished).

21

3.3. RISK MANAGEMENT

2. Workload to duration transformation - In this step we assign duration to particulartasks which we approximated in the previous step of process planning. Now wemust decide how to transform estimated workload to time duration. For example8 hour workload does not necessary mean 1 work day. This transformation shouldbe done in the context of company habits and the habits of the development team.

3. Create time plan - First draft of time plan. Tasks are transferred int Gannt chart,where each bar corresponds with one task. The length of a bar represents the timeduration of a task. Also it is useful to draw critical path into the Gannt chart (3.2.2)

4. Resource allocation - During this step we assign resources to particular tasks. In theprevious step we assumed that each task is performed only by one person. Byassigning more persons to tasks in critical path we can shorten it.

5. Time plan optimization - Last step is to optimize the time plan. Fine tuning of re-source allocation and elimination of unnecessary multitasking (which decreaseswork efficiency) are the most effective ways to do it. It is also possible to achieveshorter duration of critical tasks via re-evaluation of task relations. More extensivetask parallelization can also help to improve the work efficiency. 3

3.3 Risk management

The process of identification, analysis and acceptance or mitigation of uncertainty. Ba-sically, risk management occurs anytime we try to analyze and attempt to quantify thepotential for time loss and then take the appropriate action (or inaction). Inadequaterisk management can result in severe consequences for the project development. Riskmanagement is a systematic process of making a realistic evaluation of the true level ofrisks to our project and subsequent reactions to these risks. Before risks can be properlymanaged, they need to be identified. The following questions are a good way to start:

• What can go wrong?

• What can I do to prevent it?

• What do I do if it happens?

To properly describe risk management we shall use the following terms:

• risk factor - potential occurrence of, for the project negative, event.

• risk event - risk factor manifestation

• risk scenario - adverse chain of events caused by risk event with negative impacton the project.

• probability - probability of the occurrence of risk event with risk scenario.

• impact - enumeration of adverse event for the project.

3. Multitasking (definition according to IPMA) - Work planning and performing where one resource dur-ing one time unit works on multiple assignments. It is preferable to start working on an assignment assoon as possible.

22

3.3. RISK MANAGEMENT

• risk - probability we suffer certain damage. It is a quantified pair of numbers de-scribing both the probability ratio and the damage to the project. 4

According to PMI PMBoK there are six basic risk management processes:

1. Risk management planning

2. Qualitative risk analysis

3. Quantitative risk analysis

4. Risk monitoring and control

5. Risk response planning

6. Risk identification

Countermeasure is a process, procedure, plan or any technical mean that was created asa reaction to identified risk, alleviate or completely remove risk event and it‘s impacton project development.

Picture 5 - Circle of risk management

3.3.1 Risk identification methods

There are several methods for risk identification, each has it‘s own merits. However Iwill mention only those I used myself during the development of Topic packages.

4. Risk (definition according to PMI PMBoK) - Risk is an uncertain event which, in case it happens, caninfluence the achievement of the project goals in both positive and negative way.[3]

23

3.3. RISK MANAGEMENT

Brainstorming

Probably most commonly used risk identification technique is team brainstorming,where all members of the team are trying to identify potential risks together. All ideasare written on board or flip-chart. They are analyzed, categorized and described. Themain disadvantage of this method is that it can be very time demanding. Our brain-storms usually lasted over an hour, however the benefits were worth it. Also it is im-portant that the facilitator of the brainstorm meeting should be able to direct the con-versation in such a direction that the influential individuals within the group are notallowed to sway weaker members (weaker in a sense of not having such a strong opin-ion about a risk) entirely to see just their point of view, but to be able to maintain theirown perspective as well.

Talking with experts

Person responsible for risk management finds experts with experience relevant to theproject. Experts are then familiarized with the project and asses risks according to theinformation about the project and their expertise.

SWOT

SWOT stands for Strengths, Weaknesses, Opportunities, Threats. These factors canbe utilized to measure project‘s strengths and weaknesses, external opportunities andthreats to the evaluated object. The object does not have to be the project itself. To findstrengths, weaknesses and correctly identifying threats means that we are aware ofthem and that we are taking them into account. It is essential to precisely choose theobject of the analysis, so that other potential aspects do not interfere with the analysisitself (i.e. strength of the project vs. strengths of the development team). By asking someadditional questions we can ensure the object is well specified, i.e. “What are the threatsto the project?”. It is also good practice to enlist the author(s) of the analysis as well asto mention the method via which the analysis was obtained (i.e. brainstorm).

Black swan theory

The most problematic issue in risk management is dealing with never-seen-before events,that are very hard or even impossible to predict. The Black swan theory [4] is aboutevents or phenomena that have great impact on the project but can not be extrapolatedfrom the previous experience. However in retrospective these events appear to be log-ical and can be easily explained. People involved in the project or in the event itselfusually use remarks such as “Well, of course it happened, because ...”. The problemwith black swan experience is that instead of preparing ourselves for other possibleblack swans, we fixate on this one in particular, thus leaving the project vulnerable toother black swans. According to [4] the main reason for this is that we do not perceivewider timeless context of the human society development. This is caused by educa-tional methods, that are based on learning the facts instead of principles and correla-tions. A lot of black swans are normal, common events. However due to our limitedway of thinking we are unable to see them. Uncertainty should be considered a factand crisis is to be perceived not as tragedy, but as an opportunity. The role of project

24

3.3. RISK MANAGEMENT

manager is not to create projects impervious to black swans. They should create projectsthat is able to utilize the “good” black swans and to be resilient to the “negative” ones.

3.3.2 Risk analysis

Risk analysis is a process of defining threats, their probability and determining theirimpact on the project. Possible methods for risk analysis are quantitative, qualitative orthe combination of both (semi-quantitative).

Qualitative risk analysis

This analysis is sometimes performed at the same time as risk identification via theset of methods. It is a process of evaluating the impact and probability of identifiedrisks. This process prioritize risks according to their possible influence on project goals.Qualitative analysis evaluate risks either by probability (0,1), on a numbered scale (1-10)or word description (low, medium, high). Project risks can be categorized. The analysisprovides update list of risks, their classification and severity.

The main advantage of this analysis is that it is relatively quick and simple. How-ever there is no financial estimate of the risks involved and the analysis can be quitesubjective.

Quantitative risk analysis

This analysis involves mathematical calculation of the risk from the probability of therisk occurrence, total value threatened by the risk and expected impact on the project.Impact is usually enumerated by cost. Quantitative analysis can be performed by oneof the following methods:

• sensitivity analysis - analysis based on changing process parameters and measur-ing the changes in the outcome of the process. [6]

• Monte Carlo - the basic principle behind this method is to generate a large numberof random scenarios and their statistical evaluation. Point of origin in Monte Carlomethod is static project time plan. A probability distribution is set for each task,the length of the task, and source task relations. The calculation itself is conductedin loop phases. In each phase a sequence of random numbers is generated. Arespective length of each task is set according to these numbers. Based on thelength of tasks a simulation run through the project plan is executed. In the endof each simulation the simulation is evaluated and compared to the static projecttime plan. After thousands or tens of thousands of loops a statistical analysis isperformed, summarizing the risks in project time plan. [7]

• decision tree - graphical representation of alternatives with quantification of sce-nario impact

The difference between qualitative and quantitative analysis is described in followingtable:

25

3.3. RISK MANAGEMENT

Qualitative Quantitative

easier to perform more precise

do not provide financial estimate highly formalized output

easily readable output more time demanding

Table 2 - Difference between qualitative and quantitative risk analysis

3.3.3 Risk elimination

The main aim of this process is to choose the best reaction to risk and to develop coun-termeasures to reduce the impact of the risk on the project. The choice of countermea-sures depends on circumstances, resources and financial aspects of the risk. Some riskscan not be eliminated or their elimination would be more expensive the the risk impactitself.

Following methods (or their combination) are used to eliminate (or decrease) therisk:

• accept - accepting the risk and taking it into account. It is usually good practice toset aside a part of budget to deal with expected risks.

• avoid - completely cancel the risk activity.

• mitigate - to develop such countermeasures that will lower the probability of therisk or the cost of the risk impact on the project. However it is important to estab-lish whether the cost to mitigate the risk does not exceed the potential benefit.

• transfer - transferring the risk elsewhere (i.e. insurance).

3.3.4 Monitoring and risk assessment

This process incorporates all processes mentioned above. The entire knowledge-baseis used during monitoring and assessment. According to IPMA [6] all risks should beconstantly monitored since the eventualities can radically change during the project.Events that can change the risk probability are i.e.:

• countermeasures are activated - situation occurred, that required the activation ofprepared countermeasures

• countermeasures are no longer effective - must be replaced or modified

• threat is no longer relevant - risk can be removed from monitoring

• new significant threat - risk must be evaluated and countermeasures must be de-veloped

• change in conditions of a threat - change in probability or in cost of risk impact on theproject. In this case we must re-evaluate the risk and re-design countermeasures

26

3.4. PROGRAM & PORTFOLIO

3.4 Program & Portfolio

Each organization, that is solving more projects at the same time, faces a problem ofmanaging these project effectively. In order to fulfill each project‘s goals, it is necessaryto use efficient project management, as well as coordination of projects to fulfill orga-nization‘s strategic goals. Similar or related project are merged into programs, whichare controlled by program management. Control of programs is the task for portfoliomanagement.

3.4.1 Program

Program is a set of related or similar projects, created by organization unit in order toachieve strategic goals of the organization. Means required to achieve these strategicgoals (products and benefits) are deliverables of mutually interconnected project pro-grams.

Processes to control business goals and business benefits are also defined within theprogram. The main managing person is program manager, who controls projects viaproject managers. Program manager is responsible for program benefits, he is howevernot responsible for projects implementations.

Program definitions:

• according to IPMA - program of projects is put together to realize a strategic goal,set out by the organization. The program also defines business benefits, manage-ment process as well as tracking the business benefits.[10]

• according to PMBoK - a program is a group of related projects, managed in a co-ordinated way, to obtain benefits and control which is not available by managingthem individually. Program management is the centralized coordinated manage-ment of a program to achieve the program’s strategic objectives and benefits.[10]

3.4.2 Portfolio

Portfolio is a term for a set of programs and projects united by their common strategy.Definition according to Národní standard kompetencí projektového rízení:

Portfolio is a set of programs and projects, not necessarily related, that were merged for thepurpose of easier management, coordination and optimization of the portfolio as a whole.[11]

Portfolio manager is responsible for the portfolio achieving organization‘s strate-gic goals. He assigns new projects or programs to the portfolio or cancels ineffectiveprojects.

The aim of portfolio management is to ensure coordination of projects and programsof organization, to optimize their performance, portfolio risk management and organi-zation strategic management.

3.4.3 The comparison

The following table shows the comparison between project, program and portfolio incontext of goal, vision & strategy, business benefits, organization change and time &resources [11] :

27

3.4. PROGRAM & PORTFOLIO

Project Program PortfolioGoal to create

benefitsto achievestrategic change

coordination,optimization &unification withstrategy

Vision &strategy

are related viabusiness caseof the project

are realized byprogram

are unifiedwithin portfolioand aremonitored

Businessbenefits

are mostly notpart of theproject

are mostly partof the program

are mostly notpart of theportfolio

Organizationchange

is often notpart of theproject

is usually part ofthe project

is not part of theportfolio

Time &resources

are defined inbusiness caseand can bemanager viaproject

are basicallydefined inprogram strategyand divided intorespectiveprojects

are based onpriorities andstrategic goals ofportfolio

Table 2 - Portfolio/program/project comparison

28

Chapter 4

Project definition and implementation, testing

In this chapter I will describe the usage of method and techniques described previouslyin the thesis. Each part of the project has it‘s own specifics. Since the main aim of theproject was to upgrade a number of outdated features as well as to create entirely newparts of Topic packages agenda, the usage of some of the described techniques had tobe slightly modified in order to achieve maximum efficiency.

4.1 Basic analysis

In order to properly analyze the project, it is necessary to define the main goals, thatshould be achieved via the implementation process.

The main goal is to increase the user friendliness of the agenda itself. Since the Topicpackages is one of the oldest agendas in Information system, it does not follow currentmethods in user interface design. Furthermore it does not use any of the modern webtechnologies, such as AJAX (Asynchronous Java And XML) or JavaScript. The first partof the project analysis was to engage in brainstorm with members of user support andto create new user interface design, that would be up-to-date and reflect current stan-dards.

In the next phase it was necessary to create a list of new features, that were requestedby teachers and by members of user support. To make the list as current as possible E-tech group gathered user input from teachers for a time period of several months. Thefirst draft of this list was revised several times in order to choose the most requestednew features. Several suggestions were refused due to either time management issues(the implementation process of that particular feature would be too time consumingand would cause the whole project to be delayed) or the lack of proper definition of thenew feature (teacher did not know exactly what he/she would expect from the featureand user support was unable to determine the exact definition). Suggestions, that weredismissed due to time management issues, are still kept on the list and are consideredto be implemented later, in the next iteration.

After the creation of list of new features, me and the user support engaged in an-other brainstorm session, to which we invited a couple of experts with experience withthis sort of project. The aim of this session was to define and describe the changes,that should be done in the current implementations of Topic packages. The idea be-hind this step is to improve the capabilities of the agenda. This decision to invite theexperts proved to be very appropriate, since the expertise provided by them showed usa number of serious complications, that would arise during the implementation phase.This would cause the project to be delayed and also the budget would probably beoverstepped.

After this last brainstorm session I commenced the implementation phase. This

29

4.2. INCREASE OF INTER-CONNECTIVITY WITH E-LEARNING AGENDAS

phase took several months, since the amount of code I had to write, as well as theamount the code I had to understand and change, was quite extensive. In some casesthe time it took to understand and improve the code proved longer than the time itwould took to create entirely new code altogether. However this was taken into ac-count during the creation of time plan, so it did not delay the project for much morethan it was expected.

4.2 Increase of inter-connectivity with e-learning agendas

Information system of Masaryk University is developed in Perl programming languageand operates with Oracle database. Topic packages agenda is using Perl functions frommodule RozpisStudentu. I will describe several new features and updates of pre-existingfeatures, that proved to be most challenging or interesting from the implementationpoint of view.

4.2.1 Connection between Topic packages and Classes

Classes is agenda designed for alumni to ease the communication between each other.However due to it‘s nature it can be very well used to create web-space for current stu-dents for their work. Team work is a very current method of work and is often requiredby potential employers. If students gain experience with this kind of collaborative effortduring their studies it can give them necessary edge on the job market.

Web-space created via classes agenda provides:

• discussion groups - opened only to those participating on the project

• lighter version of notice board

• list of the class members - with their email address (and/or other information theychoose to show to other class members)

• web depository - with full support for IS access rights management system. Thisallows only person involved in the project to work with (possibly sensitive) filescreated during the elaboration

• list of recent activity of class members

To create the connection between these two agendas it was necessary to change thedata table that stores information about topics. During the analysis it was decided thata class will be assigned to exactly one topic (1:1 relation). This way it is easy to ex-actly determine who belongs to each class, since the most current list is assigned to thetopic itself. The list of students enrolled in particular topic is stored it separate tablereferenced via primary key determined by topic unique identificator.

Each member of the class is assigned with full privileges within the class. Thismeans that all members are equal and are not limited in their work during the project.To ensure liability each action of each member is visible to all other members on classnoticeboard.

To ensure effective communication within the class, the notice board can serve thepurpose, as the name suggests, to pass notes to all members. Should the members

30

4.2. INCREASE OF INTER-CONNECTIVITY WITH E-LEARNING AGENDAS

choose, they can be informed about new posts via e-mail sent to their university e-mail (which can be forwarded to a reliable e-mail address of their choosing). Posts canbe posted either as plain text or rich text format with support of HTML tags. This waystudents can post images, tables, URL links, etc. All these project related elements canbe stored in the associated web depository for easier manipulation and higher security,since they are accessible only to the members of the class authenticated in IS. Teachercan decide if a topic shall have it‘s own class in topic settings, however only topics des-ignated for more than 1 student can have classes. Class is only created after first studentenrolls in the topic to ensure classes would be created only for topics students are in-terested in. If a class is created and later on all students leave the topic and thus alsoleaving the class, such classes are scanned for and deleted automatically once a week.

Picture 6 - Class title page

4.2.2 Sharing Topic packages between multiple courses

Often the university courses have similar or even the same syllabus, however they aredesignated for different students - present, combined or other forms of studies. There-fore IS has the capability to share study materials of these courses, so the teacher has tomaintain only one repository. This solution proved to be very effective and teachers arewidely using it across the university.

Similar suggestion were submitted for Topic packages. If a number of courses haveshared study materials, teacher still had to maintain Topic packages for each separately.Especially for large courses with dozens or even hundreds of students this task discour-aged teachers from using Topic packages altogether. During the analysis, it was decidedthat this feature will be implemented in order to increase the usability of the agenda andto motivate teachers to start using it more often.

This task proved to be more difficult during the implementation that previouslyexpected, since some unexpected dependencies were discovered, i.e. database table,that stores topics, has an attribute that defines the code of related course. Thereforeone topic could be related only to one course. In order to maintain compatibility withtopics created in previous (older) version of the agenda, this table structure had to bemaintained. The solution of this problem also had to take into account that not only

31

4.2. INCREASE OF INTER-CONNECTIVITY WITH E-LEARNING AGENDAS

the entire package can be assigned to multiple courses but a topic might be as well.A teacher can share a package or a topic only across courses where he/she is listed aslector or instructor.

From number of solution I chose the creation of another table that would allowme to bind one topic with multiple courses while keeping the old structure intact. Theprinciple behind the binding table is in principle somewhat similar to the access rightmanagement system used in IS, however I devised a lighter version of access rightssuited to the needs in this agenda. Typical use case of package or topic is that if studenthas access to it, he should be able to enroll in it. To determine, whether student hasaccess to it or not, each package and topic has a list of prerequisites that has to befulfilled before the student gains access to it.

4.2.3 Assigning Teacher‘s notebooks to topics

Among teacher‘s requests was also the possibility to assign a designated notebook toeach topic. Assigned notebook would contain all the students enrolled in particulartopic and would be dynamically updated if students would enter or leave the topic.This task was quite complicated due to a number of factors that needed to be taken intoaccount:

• the implementation required changes in a very complex and widely used agendaof Teacher‘s notebook (even slight changes require quite extensive testing to en-sure no flaws were introduced into the system)

• data table containing information about notebooks is not designed to store at-tribute that would describe the type of a notebook - I had to create a system ofunique acronyms to ensure that each topic always has exactly one assigned note-book

• the list of students had to be updated in real time - every time a student enrollsor leaves a topic the list of students in notebook needs to be updated. This isnecessary to ensure teacher will not grade a student that no longer participates inthe topic

All these aspects had to be considered. Due to the nature of notebook I could not makeany changes to data tables containing information about them to ensure compatibilitywith older versions.

I created a solution to connect each notebook to particular topic via notebook abbre-viation and course identificator. Each is unique for the entire system which is ensuredvia constraints in the Oracle database. The usage of course identificator had to be useddue to the possibility of the course being shared, i.e. having shared study materials ofthe topic package being shared through multiple such courses. Even shared coursesmust have unique identificators, so this solution proved to be optimal for the problem.Notebook assigned to a topic is only created after first student enrolls in it. If a studentenrolls and later decides do leave, the notebook remains. Once a week IS automaticallyscans all the notebooks and those without any entry are deleted.

Update function for the students included in the notebook was implemented intothe application students use to enroll in topics. Should the teacher chooses to, he/shecan be informed about student enrolling in the topic. Since the notebook is updated

32

4.2. INCREASE OF INTER-CONNECTIVITY WITH E-LEARNING AGENDAS

instantaneously, teacher can immediately put information into it or send students ane-mail.

4.2.4 File vaults assigned to packages & topics

Topics, that have their own classes, also have their private web space accessible onlyto those participating on the project. However some topics are designated only for onestudent (these topics can not have their own classes) or teacher chooses not to create aclass for a topic. But even in this case it might be necessary to collect student‘s notes,materials related to project or reports of progress via IS. This is standard use case of thisfeature.

Each Study materials assigned to a course have a special folder called „Odevzdávárny”or File vaults. If the teacher chooses package and it‘s topics can have their designatedfolders. In order to maintain easy access to student‘s posted work I decided to use atree file structure that exactly replicate the structure of package and topics in it.

Picture 7 - Folders tree structure of Study materials

Each package has it‘s own folder divided into folders dedicated to each topic. Accessright are set as follows:

• package

◦ read - all students enrolled in package

◦ write - none

◦ administer - none

33

4.2. INCREASE OF INTER-CONNECTIVITY WITH E-LEARNING AGENDAS

• topic

◦ read - all students enrolled in associated topic

◦ write - all students enrolled in associated topic

◦ administer - all students enrolled in associated topic

These access rights can be changed by teacher at any time through IS File manager, de-scribed setting are only recommended and implicitly set. The package folder is createdimmediately after the package, however folders for topics are created after the first stu-dent enrolls in the topic. If all students leave the topic and no files are present in theassigned file vault, such folder is deleted automatically once a week.

The issue of a course having shared study materials with other courses is settledthrough study materials root node, since this is also shared for all the assigned courses.This ensures uniqueness of each File vault.

Teacher also see an URL link directly to the topic folder assigned to the notebook,along with number of files updated and the date of last update.

4.2.5 AJAX & JavaScript

Due to the age of the Topic packages agenda, it does not use any modern and widelyused technologies, such as asynchronous request processing or JavaScript modules(such as jQuery). To make the agenda more user friendly, these technologies were in-troduced into the system on several places. Since the number of operations that can beperformed with packages as a whole, and with it‘s topics, is quite extensive, the menuhad to be revised and significantly simplified.

Also the set of options for each package and topic were divided into two (for pack-ages) and three (for topics) groups. Options were divided into these groups based onthe frequency of their use or their importance. For example the name and abbreviationare mandatory data, so obviously they fall into the first group. However topic namewritten in TEX syntax is seldom used, so it was decided to put it into the last group. Thesecond (in topics also third) group is implicitly contracted and is only displayed afterteachers clicks on the provided link. This is achieved via jQuery library.

After teacher creates a new topic, he/she can immediately enroll students in it.However the list of all students in a course can be very long (dozens, possibly hun-dreds of names). To create such a list every time the page is reloaded would be veryslow, thus frustrating for the user. The new implementation provides just a title of sucha list, and only after teacher decides to enroll students and use that list, it is loaded intothe web page (on demand).

A lot of information is stored about each topic or package, but they are not alwaysdisplayed in the right place or in the right way for the user. Also due to the age of theagenda, new CSS designs of IS are not used for most titles, tables and other visuallyimportant elements of the page. This was achieved through code revision. The amountof information displayed on pages was reduced or divided into groups, similar to thesystem used in the application for creating topics and packages. Some information isimplicitly displayed, the rest is showed on demand. The division of information wasbased on suggestions from E-tech user support group during the 2. project stage meet-ing.

34

4.3. TESTING

4.2.6 Packages accessible via simple URL

IS development team is trying to create a system that would be complex but still simpleenough to use. A lot of effort is put into achieving this goal one of which are simpleURL links to certain agendas. These are used in:

• discussion groups - i.e. https://is.muni.cz/auth/df/podnetovna_is/

• file manager - i.e. https://is.muni.cz/auth/do/

• noticeboard - i.e. https://is.muni.cz/auth/bb/MU/dalsi/29966190/

• blogs - i.e. https://is.muni.cz/auth/blog/is_info/

To make packages more accessible I implemented a feature that would allow teachers(and students) to access package via such a simple URL link in format:

• https://is.muni.cz/auth/balik/xxx

where ’xxx’ is substituted by package abbreviation. This would redirect teacher to a webpage where he could edit package preferences, add or edit topics etc. Student would beredirected to a web page that would allow him to enroll or leave a topic or even createa new topic in a package, where teacher allowed it.

Such a URL can be pasted into various study materials, interactive syllabi, sent viae-mail, etc.

4.3 Testing

The testing stage took several months to finish due to the amount of changes madein the code. Since the applications are in the active use, and the compatibility withpackages and topics created before the changes were implemented had to maintained,the testing process proved to be very complex. In cooperation with members of E-techsupport group we proceeded in three iterations of testing.

1. First iteration involved testing of the updated functions - added options, their dis-tribution through IS, inter-connectivity with other agendas. We were especiallylooking for possible data inconsistencies that could occur, if the manipulationwith data already stored in database was incorrect or incoherent.

2. Second iteration of testing was focused on entirely new features, that were im-plemented. Adjusting minor and major details of application‘s behavior to matchthe requests submitted by teachers was a goal we tried to achieved. Great deal ofsuggestions was submitted to user interface, since E-tech group need some specialfunctions in order to help teachers effectively.

3. Last stage of testing was focused on functioning of Topic packages as a wholecomplex agenda with the environment of IS. Coherrency of data and their prop-agation, especially to Teacher‘s notebook, was tested particularry hard, since theagenda works with very sensitive data, such as grades.

Of course, not all the flaws and errors can be fixed during closed testing. However themost critical ones were found and dealt with. The solutions devised during analysisstage proved to be functional, most mistakes were in missing conditions or because ofquite obscure seetings, which however unlikely, can occur in practical use.

35

Chapter 5

Conclusion

The aim of the thesis was to describe the current implementation of Topic packages anddescribe it‘s current pros and cons which is described in the first chapter. The analysisof user requests and improvement suggestions are dealt with int the second chapter.Service science methods and techniques used during the project, meeting and devel-opment are described in the third chapter. The implementation itself along with briefdescription of selected features is described in fourth chapter. The description of se-lected applications and improvements does not include all the changes made in theagenda, however represent a portion of problems that I had to deal with during the im-plementation process. Due to their nature I consider them interesting for the the reader.The main acquisition of this thesis could be the description of problems that can ariseduring such a project, the way to avoid them or, in case they occur, to deal with themeffectively.

The design and functions described in this thesis are either already implemented orare in the final stage of testing. IS MU has over 30 thousand active users a day and thefact that the implementation of applications and functions that are already in active useis a prove that it works.

During the processes of this project I had to deal with, and solve, a number ofcomplicated issues and problems. However the process of discovering the solutions,overcoming the problems and implementing them into such a vast and complicatedenvironment provided me with a great deal of experience that I‘m sure will prove in-valuable in the future.

36

Appendix A

Attachments

A.1 List of suggested features, upgrades, etc... (in Czech)

The original report from a meeting with E-tech support group during the second phaseof features selection. The text has been slightly revisited and divided into sections forbetter understanding.

Správa rozpisu témat

1. Zalozit nový rozpis témat

2. Prehled existujících rozpisu témat

(a) Operace s rozpisem studentu (Zakliknu si rozpis, u kterého chci dále necoprovádet a kliknu na "operace s rozpisem studentu" a zobrazí se mi prehledmozných operací, které muzu provádet. Nebo prímo u kazdého rozpisu mít"operace s rozpisem studentu".)

3. Nastavení zasílání zmen Výber 1 a více balíku, u kterých chce být vyucující infor-mován, pokud doslo k prihlásení/odhlásení studentu.

4. Sdílení rozpisu témat Výber balíku/u, které se mají sdílet s jiným predmetem.

• Ad 1) Zalozit nový rozpis témat

◦ Název rozpisu

∗ Zkratka pro URL

◦ Nastavení spolecných parametru k celému rozpisu (Nabídka by se rozbalilana kliknutí.)

∗ [1] Kdo má právo pristupovat k rozpisum témat

· Tohle právo znamená, kdo smí videt témata= muze se prihlásit. Jde oto, aby se zjednodusil systém zadávání prístupových práv, a zárovenjestli vzniknout balíky napr. pro seminární skupiny, kterých muzebýt i 20 v jednom predmetu, tak pak by se studentum nenabízeloneuveritelné mnozství jim neurcených rozpisu témat. Tzn., ze nazáklade zadaného práva se bude rozpis nabízet jen studentum, kterýmje urceným.

∗ [2] Kdy se smí studenti k tématum prihlásit [od–do]∗ [3] Smí si studenti vlozit téma [od–do]

37

A.1. LIST OF SUGGESTED FEATURES, UPGRADES, ETC... (IN CZECH)

∗ [4] Zverejnit témata studentum ANO [ihned X az od] x NE∗ [5] Kolik témat si muze student prihlásit∗ [6] K jednomu tématu se smí prihlásit studentu Pokud bude vyplneno,

nemelo by jít menit u témat. Pokud mají témata mít rozdílný pocet stu-dentu, kterí se smí k tématu prihlásit, pak by mel vyucující nechat nevy-plnené a doplnit po vlození témat.

∗ [7] Vytvorit k rozpisu odevzdávárnu nebo pripojit jiz existující∗ [8] Vytvorit k rozpisu PB nebo pripojit jiz existující PB∗ [9] Umoznit studentovi vepsat poznámku k prihlásení∗ [10] Vlozit spolecnou poznámku k tématum∗ [11] Kontaktní osoba pro vsechny vypsaná témata Vlození osoby z ISu

a dále umoznit vepsat jméno a email osoby, která není z MU napr. pro-jekty, které vedou lidé z firem viz podnet https://is.muni.cz/auth/df/podnetovna_int/6033192/6050198) Jsou-li kontaktní osoby ruzné,vyucují, nechá nevyplnené a doplní az po vlození témat.

◦ Vlození témat

∗ [1] Vlozit témata formulárem∗ [2] Importovat témata ze souboru Na základe výberu se bud’ nabídne

formulár pro témata (základ formulár pro vlození 20 témat s mozností’pridat dalsí pole’) nebo se umozní import souboru v .csv.

∗ [3] Importovat témata z jiného rozpisu Vybrat rozpis témat, a to bud’jiného predmetu, nebo i ze stejného predmetu. Po výberu rozpisu proimport umoznit importovat vsechny témata x provést výber témat proimport a importovat. Po vlození témat pridat volbu ’Editovat práve vytvorenátémata’, ’Prejít na prehled rozpisu témat’

◦ Poznámky k rozpisum obecne

∗ [1] Hromadný výber témat a zmena spolecných údaju napr. doplnení kvybraným odpovednou osobu za daná témata, zmena poctu prihlásenýchk vybraným tématum atd.

∗ [2] Výber 1 a více témat > nakliknu si témata, u kterých chci neco zmenita budu menit postupne u kazdého tématu neco jiného. Neco jako ‘Pridatdo výberu’ (obdobne funguje v Publikacích).

∗ [3] Vlození vlastního tématu studentem = automatické prihlásení k té-matu. Nyní je tak, ze si student téma nejdrív vlozí a pak se k tématujeste musí navíc prihlásit. Tohle je pro studenty k nepochopení, ze musíucinit dva kroky.

∗ [4] Nejcastejsí pozadavek vyucujících je na sdílení rozpisu na podobnémprincipu jako sdílení studijních materiálu.

∗ [5] Evidence událostí: nikdo nikdy nechtel, ale LuckaP. si myslí, ze bytam nekde mely být skryté, i kdyz události typu císlo komise obhajoby,termín obhajoby atp. nemají v predmetových balících vubec význam.

• Ad 2) Prehled existujících rozpisu témat

◦ Operace s rozpisem studentu

38

A.1. LIST OF SUGGESTED FEATURES, UPGRADES, ETC... (IN CZECH)

∗ [1] Zmena spolecných parametru rozpisu∗ [2] Správa témat

· Zobrazit: vsechna témata – kde je nekdo prihlásený – kde není nikdoprihlásený – dle odpovedné osoby za téma – dle vlastního výberu té-mat U vypsaných témat zobrazovat, zda je soubor jiz v odevzdávárne.

· Tisk/Export údaju o tématu a seznamu prihlásených studentu k té-matum Výber 1 a více témat seznam vybraných témat a k nim prih-lásených studentu pro tisk nebo export do csv. Je-li rozpis propojenýs PB a obsahuje-li PB nejaké hodnocení, tak exportovat i s tímto hod-nocením.

· Pridat dalsí témata do rozpisu· Priradit témata ke Kruhu lidí (nebo tak nejak)· Editace/úprava témat Hromadný výber 1 a více témat > následná

editace výberu· Rusení témat Hromadný výber 1 a více témat > následné zrusení

vybraných

∗ [3] Rozvrh témat

· [] Vytvorit rozvrh Automatické vytvorení: Na základe poctu vyp-saných témat a v závislosti na rozsahu výuky se vytvorí adekvátnípocet slotu. Propojení s rozvrhem. Vlastní nastavení: Vyucující sivytvorí vlastní rozlození (tj. kdy a kolik témat). Editace rozvrhu:Presouvání jiz vytvorených slotu v rozvrhu. Datum prihlasování dorozvrhu nedávat, vyucující pozadují jednotný datum. Pokud si vyucu-jící nezvolí vytvorení rozvrhu, tak se rozpis témat chová pouze jakovýber tématu bez návaznosti na nejaký termín výuky, napr. jde-li ovýber tématu práce, která bude soucástí ústní zkousky nebo rozpravyu kolokvia.

· [] Priradit témata vyucujícím Student se hlásí k urcitému tématu,které má jiz prirazené datum z rozvrhu.

· [] Student muze téma priradit do rozvrhu Student se hlásí k tématua muze si jej zavést do rozvrhu. [] Student musí téma priradit dorozvrhu Student se hlásí k tématu a musí si jej zároven prihlásit dorozvrhu.

∗ [4] Zrusení celého rozpisu∗ [5] Manipulace se seznamem prihlásených studentu k tématu

· [] Prihlásit/prehlásit studenta k tématu· [] Odhlásit studenta k tématu

∗ [6] Poslat email

· [] vsem studentu prihláseným v tomto rozpise· [] studentum pouze u vybraných témat > Po zabliknutí této volby

výber témat, u kterých je nekdo prihlásen a umoznit hromadný výbertémat.

· [] neprihláseným k zádnému tématu v rozpise

39

A.1. LIST OF SUGGESTED FEATURES, UPGRADES, ETC... (IN CZECH)

· [] vsem co se mohou prihlásit I kdyz se dá provést na základe omezeníseznamu studentu, vyucující casto hledají prímo v rozpisech.

∗ [7] Prejít do odevzdávárny vázané na rozpis (Pocet odevzdaných souboru:XYZ) Tzn., zobrazovat, kolik je v odevzdávárne celkový vlozených souboru.

∗ [8] Hodnotit do poznámkového bloku Kliknu, ze chci hodnotit. Zobrazíse výber témat, vyberu ta, která chci ohodnotit a následne zadám spolecnéhodnocení, které se rozkopíruje lidem prihláseným u témat do PB.

∗ [9] URL rozpisu témat Vyucující pozadují pekné url, které by mohli vlozitdo IO, mailu, DF.

40

Bibliography

[1] Z. Stanícek - Rízení projektu [online, cit. 2011-11-24]. URL: http://www.systemonline.cz/clanky/rizeni-projektu.htm

[2] A. Svozilová - Projektový management, Expert (Grada). Grada, 2006

[3] Project Management Institute. A guide to the project management body of knowl-edge: PMBoK guide. PMBoK Guides. Project Manage- ment Institute, Inc., 2004

[4] N. Taleb. The Black Swan: The Impact of the Highly Improbable. Random House,2007

[5] Jitka Danková. Metodika rízení a koordinace prací pri vývoji software. Diplomovápráce, Fakulta informatiky Masarykovy univerzity, Brno, a a 2011. URL: http://is.muni.cz/auth/th/173357/fi_m/diplomova_prace.pdf

[6] J. Dolezal, B. Lacko, P. Máchal a kolektiv. Projektový management podle IPMA.Grada Publishing a.s.

[7] James F. Wright. Monte Carlo Risk Analysis and Due Diligence of New BusinessVentures. American Management Association, 2002.

[8] Z. Stanícek. Pojmy projektového rízení [online, cit. 2011-12-05]. URL:http://www.fi.muni.cz/~stanicek/P055/rok%202009/Projects_and_IT_support/Pojmy_projektoveho_rizeni_uvodni_vyklad.pdf

[9] E.M. Goldratt. What is this thing called theory of constraints and how should it beimplemented? North River Press, 1990

[10] Dr. Thomas Grisham PMI & IPMA: Differences & Synergies. [online, cit.2011-12-11]. URL: http://www.allpm.com/index.php?name=News&file=article&sid=2486

[11] Národní standard kompetencí projektového rízení. Spolecnost pro projektovérízení, o.s., 2008

41