istqb advanced test manager certificate in software testing · istqb advanced test manager...

ATM Sample Exam Paper A Version 1.3 October 2011 © UK Testing Board, 114-116 Curtain Road, London. EC2A 3AH UK Testing Board is the trading name of UKTB Ltd, Company Registration No. 6505485

of 28

ISTQB Advanced Test Manager Certificate in Software Testing

With Learning Objectives, Answers and Rationales

Sample Paper A - 3 Hour Examination This is not a complete exam paper Record your surname/last/family name and initials on the Answer Sheet, writing in block capitals at the top and marking the relevant letter in each column.

Attempt all 65 multiple-choice questions. There are no trick questions. Pass Mark is 65/100. Your answers should be indicated inside the box representing your chosen answer to the right of the question number. Do NOT turn over this page until instructed by the invigilator Copying of this paper is expressly forbidden without the direct approval of UKTB.

Candidate Name: ………………………………..…………………

ATM Sample Exam Paper A Version 1.3 October 2011 © UK Testing Board, 114-116 Curtain Road, London. EC2A 3AH

of 28

SCENARIO 1

The following scenario applies for Questions 1 to 11 inclusive.

Avionics system enhancement.

Aerotech plc is an established avionic systems company that has developed and delivered a series of successful integrated cockpit management systems for advanced passenger aircraft. The Aerotech CM1 family of cockpit management systems integrates flight control, navigation, communications and warning systems into a single display and associated control panel. Aerotech is now developing a new family of cockpit management systems, CM2, which offers improved performance, new display technology and simpler system controls to further reduce crew workload. The display technology has been used successfully in other cockpit applications for high performance military systems. The system design will utilise existing components of the CM1 family, develop new components, and integrate these via a new architecture that has been developed and tested in the company’s R&D department. The development team will be made up of experienced developers and testers from the CM1 project and the architecture project and will use an Agile approach that has been documented after being trialled in the architecture development project. Development and testing tools are being carried over from the CM1 development. Testing personnel are located in separate premises from the developers. The CM2 family will have 20% more code than the CM1 family and will have a more complex architecture, but about 80% of the CM2 code will be common with CM1. An analysis of the development and test data for the CM1 development has determined that testing took approximately 120 days of effort, broken down into 20 days for test planning and control, 45 days for test analysis and design, days hours for test execution and days hours for test evaluation and reporting. The Project Manager has requested that an FMEA workshop be staged to assess the key risks identified for the project. The risks are:

Risk Number

Risk Description

1 There may be serious interface issues between the new modules and the original CM1 modules.

2 The CM2 system may not be able to meet the system response times currently achieved by CM1.

3 The new display technology may not exactly meet user expectations. 4 The new architecture may not be able to support all of the required

functionality for CM2 with the new display technology.


of 28

1 Referring to Scenario 1 – In the CM2 project, which one of the following is MOST likely to be a factor in causing inaccuracy in any estimates of time, cost or duration of testing?

A The level of detail in specifications. B Assimilation or development of new tools. C Training and orientation of team members. D The complexity of the development and test processes.

K2 (1mark)

Learning Objective - 3.4 Understand and give examples to the factors listed in the syllabus which may lead to inaccuracies in estimates. Answer: A. Rationale: The 4 options are all factors listed in the syllabus. Scenario 1 explicitly eliminates options B, C and D (respectively - tools are being carried over from the CM1 project, experienced test/devpt resource from CM1 is being deployed and an Agile process has already been trialled and documented). Option A is not specifically identified as an issue but is a known limitation of estimating agile development as an approach, especially in relation to testing. Option A is therefore the most likely factor.


of 28

2 Referring to Scenario 1 – Which three of the following will need to be given highest

priority in the test strategy and planning for the CM2 product family? A Timebox planning to achieve early delivery. B Detailed system integration testing. C Use of development and testing tools. D Detailed reviews of unit design. E Preparation of a software maintenance plan. F Establishing redundancy in the architecture. G Regression testing in each timebox.

K4 (3 marks) Learning Objective - 1.2 Analyse software life-cycle models and outline the most appropriate tasks/test activities to be executed (distinguish between test and development activities). Answer: B, F, G. Rationale: This is an example of a safety critical system and the answer hinges on recognising that this is the main driver of test strategy. Items F and G are test activities; F specifically important for safety critical systems and G important for agile developments. For F the scenario identifies that the architecture has been tested in R&D but redundancy is still to be established in the context of the CM2 family. Similarly item B is clearly a testing priority because there is a high degree of integration of old and new technology with a new architecture. Answer A is not a testing specific activity and cannot be allowed to dominate (though it might if not overridden by other factors), answer C is relevant but less significant, E is not a testing specific activity and clearly not a priority, and D is not likely to be an important element in an Agile development.


of 28

3 Referring to Scenario 1 – Four separate estimates have been provided for the initial

CM2 development testing effort. Based upon the information provided in the scenario, which estimate is the MOST realistic?

A 144 days. B 100 days. C 300 days. D 170 days.

K3 (2 marks) Learning Objective - 3.4 Estimate the testing effort for a small sample system using a metrics-based and an experience-based approach considering the factors that influence cost effort and duration. Answer: D. Rationale: We will have to test 20% extra code. The extent of commonality offers us little advantage in testing effort because all of the functionality will need to be tested in the new architecture. The added complexity suggests an increase in testing. The estimate must take account of all aspects of testing (i.e. planning right through to reporting). So if we take 120 hours as a baseline and add 20% we will get 144 hours. The added complexity of the architecture and a switch to an Agile approach suggests a number higher than this but doubling it would be excessive, so 170 hours (an allowance of around 20% for complexity) is the best answer.


of 28

4 Referring to Scenario 1 – In preparation for the FMEA workshop you need to

consider the likelihood, impact and testing activities that would BEST mitigate each risk. The table below records four possible impact and likelihood combinations (W to Z):

Impact (severity) Likelihood W Low High X High Low Y High High Z Low Low

The following lists four possible mitigating testing activities: a. System integration testing. b. Performance testing. c. Usability testing. d. Static testing.

Which option below provides the correct pairing of impact and likelihood combinations (W to Z) and testing activities (a to d) for risk number 3?

A Y, b. B W, c. C Z, d. D X, c.

K3 (1 mark) Learning Objective - 3.9.2 Execute a risk analysis for product from a testers perspective, following the specific approach FMEA. Answer: B Rationale: The likelihood of the display technology not exactly meeting user expectations, implying look and feel issues, must be relatively high in spite of it having been used in a different environment, but the impact is likely to be relatively low (disappointment because of the look and feel rather than a potential safety issue). The most relevant testing option would be usability. Option A assesses likelihood and impact as high but links this with performance testing, which is unlikely to address user satisfaction concerns. Option C assumes both impact and likelihood are low and associates this with static testing. This should be simple to eliminate but is needed for completeness of the options. Option D has the correct mitigation but has the Impact and Likelihood values the wrong way around.


of 28

5 Referring to Scenario 1 – Following the FMEA workshop the test manager initiated

test planning so that mitigation of identified risks COULD be incorporated into the plan. Which of the following is a further benefit of the decision to begin test planning at this stage? Select one option.

A The test manager has resources allocated for test execution that can be employed

earlier. B The test plan can be used to get an increase in testing resources and budget to give

the test manager some contingency. C Critical components that need to be tested early can be identified and, if necessary,

their delivery accelerated. D The scope of testing can be broadened to include project risks identified in the

project plan. K2 (1 mark)

Learning Objective - 3.5 Explain the benefits of early and iterative test planning. Support your explanation by examples. Answer: C Rationale: Option A would be tempting if test planning could be done by the same resources as those who do test execution, but this is not normally the case. Option B distorts a genuine opportunity to recognise the need to increase staffing and/or budget into the realms of making sure testing is not under pressure. Option D also distorts a genuine opportunity to recognise new project risks (that are outside the scope of testing) as a result of performing early test planning, but testing primarily mitigates product risks, not project risks. Option C is from the syllabus without the distortion.


of 28

6 Referring to Scenario 1 – The final draft test summary report for CM2 includes the

following details: Testing was completed in 110 days. A total of 473 defects were raised, of which 452 were fixed and retested. Three outstanding defects are at priority 2 and one at priority 1. All requirements have been tested and 100% decision coverage achieved.

Which two of the following list of additional information would need to be included to complete the test summary report?

A Product risks covered by the testing. B Effort expended on preparation of test environments. C Test tools used in system testing. D Project risks not covered. E Definition of the testing carried out.

K4 (2 marks) Learning Objective - 3.6 Analyse test results and determine the test progress, documented into a monitoring report and a final test summary report covering all 4 reporting dimensions. Answer: A, E Rationale: Options B and C are of interest and could be included in a final report but are not essential for a test summary report. Option D is a red herring, since testing targets product risk rather than project risk. Options A (Product Risks) and E (Tests) are explicitly identified as required in a test summary report, together with a summary of Defects and Coverage, which were already included. N.B. One might say that B could relate to the dimension of Tests as the syllabus does include planned and actual testing effort as an example of Tests metrics. However the more fundamental Tests metrics of number of tests planned, run, passed and failed has not supplied yet so this dimension cannot be regarded as complete.


of 28

7 Referring to Scenario 1 – The Test Manager for CM2 has consulted widely on risk

to try to generate a cost-effective test strategy because the cost of development has been well above budget. The co-opted aircrew who contributed to evaluating early versions are very happy with the functionality. The marketing department is concerned that the new version of the system should be at least as intuitive as the previous one, and the development team are slightly concerned that response times appear to be marginally slower than on CM1.

Which one of the following would be the MOST appropriate content of a test strategy for CM2?

A Decision testing to achieve 100% coverage of all code; performance testing with

scenarios that test up to and beyond expected loads; usability testing conducted by co-opted aircrew; full functional testing of every requirement.

B Decision testing to achieve 100% coverage of the new code; usability testing conducted by co-opted aircrew, to include response time measurements; functional testing of the highest risk requirements only.

C Decision testing to achieve 100% coverage of all code; performance testing of the

new functionality only; usability testing conducted by the testers from the development team; full functional testing of every safety-related requirement.

D Statement testing to achieve 100% coverage of all code; usability testing conducted

by co-opted aircrew, to include response time measurements; full functional testing of every requirement.

K4 (2 marks) Learning Objective - 3.9.2 Summarise the results from the various perspectives on risk typically held by key project stakeholders, and use their collective judgement in order to outline test activities to mitigate risks. Answer: A Rationale: A incorporates the safety-critical mandatory testing and adds testing to address documented risks. B reduces the mandatory testing in 2 areas (structural coverage and requirements coverage and also reduces performance testing to a less rigorous approach. C reduces the performance scope, the requirements coverage and the usability independence. D reduces structural coverage and performance testing.


of 28

8 Referring to Scenario 1 – The initial testing of CM2 has been completed in 110 days. In total, 473 defects were raised and 452 of these have been fixed and retested. Defects are categorised as priority 1 (needing immediate attention), priority 2 (temporary work around available while investigation proceeds), or priority 3 (acceptable work around available). All requirements have been tested and 100% decision coverage has been achieved. The outstanding defects are mostly at priority 3, but three are at priority 2 and one is at priority 1. The priority 2 defects all relate to response times, which are 8% slower than specified and therefore only 5% faster than those for CM1. The priority 1 defect is related to the new display technology, in that the display intermittently fails to switch from one mode to another on demand. The test manager is drafting a test report to guide the project manager on a release decision.

Which one of the options below would be the MOST practical guidance to offer the project manager?

A Release the product but encourage aircrew to report any intermittent switching

problems so that the problem can be investigated. B Release the product with a warning about intermittent switching problems and slow

response. C Delay release to investigate the intermittent switching problem and draft a warning

about the response shortfall. D Hold release until the intermittent switching problem and the response time

problems are resolved. K4 (2 marks)

Learning Objective - 10.2 Use a given questionnaire in order to determine strengths and weaknesses of team members related to use of software systems, domain and business knowledge, areas of systems development, software testing and interpersonal skills. Answer: C, D, G Rationale: The key is recognising that this is a very immature testing organisation (CMMI 1) which effectively has to be built from scratch so we are looking for fundamental testing experience/skills not high-end specialism. Also that the stage of the project lifecycle indicates that the priority should be on recruiting test analysts for system testing, as development/component testing is already underway. A Tempting but not essential, as this is not mission/safety critical or a particularly

niche business. Not a top three choice, but close. B Could be useful but at CMMI level 1, test tooling is not the first priority. C Yes – important for effective test analysis. D Yes - It is already late to be recruiting test personnel so schedule pressures are

likely to necessitate effective test prioritisation. E Important for component testing but as this is already underway the focus now has

to be on tests analysis for system testing. Current component testing should still be looked at carefully though, so this skill may be required in the near future

F Important for a test management role but less so for test analysts. G Yes - important in any testing role, especially during test analysis where

communication with business analysts and developers will be frequent. H Definitely not needed at this stage (see B).


of 28

9 Referring to Scenario 1 – A risk-based test strategy was created for the CM2

development project in which the key risks identified were:

• the risk of problems developing in the new components.

• integration of the architecture and new components.

• performance of the entire CM2 system.

To address these risks, which three of the following elements are the MOST essential inclusions in the test strategy?

A Regression testing of the architecture with each new component. B Performance testing of the new components. C Functional testing of the new architecture. D Performance testing of the completed system. E Functional testing of the reused components. F Decision testing of the new components. G Usability testing of the architecture.

K3 (3 marks) Learning Objective - 3.9.3 Translate a given risk based test strategy to test activities and monitor its effects during the testing. Answer: A, D, F Rationale: This question requires the candidate to recognise the requirements for testing safety critical systems, the risks identified in the system, and the testing already done in the R&D department. A is needed to ensure that new components integrate correctly and do not cause new problems; D is needed because the system will be operating in a real-time environment; F is needed for a safety critical system. B is unnecessary because the system as a whole will be performance tested; C is unnecessary because that testing was done in R&D; E is unnecessary because their functionality has been tested before; G is not required for safety critical certification and is not signalled by any of the key risks.


of 28

10 Referring to Scenario 1 – The Agile approach documentation is currently in draft

format only and you must agree upon a review plan for the document prior to its use on the CM2 project. Select the two BEST options.

A Conduct a management review to obtain stakeholder buy-in of the process. B Do not hold a review, the process has already been trialled successfully. C Conduct a contractual review to ensure that the process does not risk the

attainment of contractual milestones. D Ensure that the client is involved in the document’s review. E Ask the author to conduct a walkthrough with representatives from each project

team. K4 (2 marks)

Learning Objective – 6.4 Outline a review plan as part of a quality/test plan for a project considering review techniques considering defects to be found, available skills of staff, and aligned with appropriate dynamic testing approaches. Answer: D, E Rationale: A No - Sounds tempting at first but bear in mind that the agile process has already

been trialled so one must assume that it has already been accepted as the way forward

B No - The fact that the trial worked OK doesn’t mean it has been properly documented. There must be a review of the document by all interested parties before deployment.

C No - Sounds tempting as this is a safety critical system, for which contractual reviews are recommended, but such a review is normally aimed at specific milestones and whether delivery is on track to meet them rather than on processes/procedures. However management would need to consider whether the agile process conflicts with any contractual agreement (e.g. production of formal documents). Not the best two options in this list.

D Yes - The client must buy-in to an agile process. E Yes - This would be a good way of educating key staff on the process before it is

actually deployed and would follow the inspection outlines in option B.


of 28

11 Referring to Scenario 1 – You have been asked by the project manager to

recommend the MOST cost-effective testing organisation for the CM2 release. As test manager, which one option would you choose?

A Transfer test implementation and execution activities to the company’s off-shore

test organisation, which offers much cheaper resource rates. B Outsource all testing activities to a well established quality assurance consultancy

firm who offer cheaper resource rates when work is performed at their test lab, but you retain full test management control.

C Keep the current CM1 testing personnel but co-locate them with the developers. D Give the developers greater testing responsibility, given experience gained during

CM1, to reduce the need for so many independent testers. K2 (1 mark)

Learning Objective – 10.4 characterise the various organisational options and compare them with in-/out source and in-/off-shoring. Answer: C Rationale: The key to this is that CM2 is adopting an agile approach. Also bear in mind that it is a safety critical system. So: A Off-shoring any part of the testing is the least attractive option as close working and

effective communication between development and test personnel is essential for an agile project. May also be hard to sell given it is a safety-critical system.

B Not much better than A as testing an agile project is difficult when the developers and testers are physically separated.

C The best option and one that is often recommended for agile projects. It is likely to cost more if they teams remain separate, as the testers have to learn fast and communicate well in a more rapid development environment

D This assumes developers are cheaper than testers which is unlikely to be the case. Furthermore this risks quality – independence is still needed in an agile project.

--End of Scenario 1--


of 28

12 Which one of the following is likely to have the greatest impact upon the effectiveness of an inspection of a complex requirements specification?

A The review meeting overran the planned two hour time box slot by 30 minutes. B The document author was unable to attend the review meeting. C The document was made available to the moderator and review team the day

before the review meeting. D The time spent by the review team was incorrectly captured in the project database.

K2 (1 mark) Learning Objective – 6.5 Explain the risks behind not considering the technical, organisational factors and people issues for performing reviews. Answer: C Rationale: A An overrun, at worst, could result in a drop in effectiveness (in defect detection) as

time went on due to tiredness, especially for a complex document. Inspections are best time boxed for this reason. However not the greatest impact of the four

B Not desirable, but not a major issue for an inspection (would be for a walkthrough/technical review). There is little time for discussing issues – it is all about capturing as many potential defects in the time allocated so the author’s absence is not a showstopper

C The inspection should not proceed if the moderator has not been able to vet the quality of the document first (which is unlikely with only one day to go) and for a complex document the reviewers must have time to read the document prior to attending the meeting. This would have the greatest impact on inspection effectiveness (in terms of defect detection during the meeting)

D Could cause issues for future reviews in terms of budgeting/scheduling but not for the current inspection, which is what the question asks


of 28

13 Which one of the following is characteristic of the life cycle for systems of systems? A Informal transfer of information between developers and testers. B Single level of integration and version control. C Requirement for regression tests when moving from each level of integration to a

higher level. D No requirement for maintenance testing due to replacement of components.

K2 (1 mark) Learning Objective – 1.3 Explain by giving examples the specifics of testing systems of systems. Answer: C Rationale: C is specifically stated in the syllabus, though in a longer form. A, B and D are the inverse of statements in the syllabus, slightly reworded and focussed on testing. Both the correct and incorrect answers can all be inferred from a sound understanding of the nature of systems of systems. The question therefore goes beyond recall but does not require application of the concepts.


of 28

This page is intentionally blank.


of 28

SCENARIO 2

The following scenario applies for Question 14.

Introduction of a formal test process.

WebSpace is a small business that provides mobile text messaging using Short Message Services (SMS), and associated services. The company’s customers range from small businesses, using text messaging for alerts or marketing purposes, to large enterprises using text messaging for customer relationship management. One challenge of WebSpace’s business model is support of the large number of customers running similar but not identical services, and another challenge is the relatively high rate of defects reported through their support web site. On average, the construction of a new application generates 9 defects. Defect detection relies principally on reviews, which typically identify 5 of the 9 defects per application review and cost 10 person-hours per review to implement. Informal testing is done by the developers and the cost is included in the development activity. A metrics programme has determined that informal testing takes, on average, 5 hours per application and every undetected defect that generates a failure in informal testing costs 10 person-hours to correct. Typically 1 defect per application reaches the market place. External failure costs vary but the management of WebSpace has decided that the potential impact on their image in the market place is equivalent to 100 person-hours of effort per defect. Due to the high external failure costs a more formal test process is to be implemented, with formal testing replacing informal testing. The resulting cost impact is as follows:

• The review process and its defect detection rate will remain unchanged at 10 hours per application review and 5 of the 9 application defects being detected during the review process

• Debugging costs for defects found during formal testing remain the same

• Formal testing costs per application will be three times as much as the informal testing rate, due to the use of formal test specification templates and techniques.

It is anticipated that there will be an 80% reduction in defects reaching the market place per application.


of 28

14 Referring to Scenario 2 – What is the current cost of quality in this scenario and

what benefit, if any, would be achieved by implementing formal testing? Select one option.

A Current cost of quality = 145 person-hours/application; benefit from formal testing =

62 person-hours/application.

B Current cost of quality = 55 person-hours/application; increase from formal testing = 10 person-hours/application.

C Current cost of quality = 195 person-hours/application; benefit from formal testing = 62 person-hours/application.

D Current cost of quality = 145 person-hours/application; benefit from formal testing = 80 person-hours/application.

K3 (2 marks) Learning Objective – 3.7 For a given context list the quantitative and/or qualitative values that apply. Answer: A Rationale: Current cost of quality per application:

• 10 hrs per review (which picks up 5 of the 9 defects per application)

• 5 hrs informal testing costs

• 30 hrs for 3 defects found during informal testing

• 100 hrs for 1 external defect. Total COQ = 145 person-hours. With formal testing:

• 10 hrs for review

• 15 hrs for formal testing costs (300% increase)

• 38 hrs for 3.8 defects found during formal testing

• 20 hrs for 0.2 external defects (80% reduction). Total COQ = 83 person-hours, a saving of 62. This is answer A. Option B assumes no external defects current or new (i.e. 4 residual defects found during testing), option C forgets that reviews pick up 5 defects before testing starts, and option D has the correct current cost of quality but ignores the increase of costs due to formal testing.



of 28

15 Which two items in the following list will always be preconditions for execution of a given test created using formal test design techniques?

a. Test environment ready. b. Code reviewed. c. Functional specification reviewed. d. Expected results defined. e. Entry criteria specified for next test level.

A a and b. B b and d. C a and d. D c and e.

K2 (1 mark) Learning Objective – 2.5 Explain the pre-conditions for test execution. Answer: C Rationale: Straightforward theory. Only test environment and availability of expected results are essential for any test at any level. The other options are all good practice but none is essential as a pre-condition of executing a test created using formal test design techniques.


of 28

16 Which two of the following are typical metrics used to measure how testing is

progressing towards completion?

a. Number of approved changes not implemented for this release. b. Number of testers deployed against planned team size. c. Testing costs compared with those planned and budgeted for. d. Total test time planned against effective test time carried out. e. Planned use of test tools against actual test tool usage.

A a and c. B b and d. C b and e. D c and d.

K2 (1 mark)

Learning Objective – 2.6 Summarize the information necessary to collect during the test process to support accurate reporting and evaluation against exit criteria. Answer: D Rationale: Straightforward theory. The correct answer is contained in a list of metrics given in the syllabus. The other options look similar and may be interesting metrics, but none are related to completion. Changes not implemented are not tested and therefore cannot affect test completion. Number of testers deployed and tools deployed may be factors in determining optimum test resources but they do not directly affect test completion.


of 28

This page is intentionally blank.


of 28

SCENARIO 3

The following scenario applies for Questions 17 and 18.

Introduction of a formal test process.

You are the test manager for an IT services company which has just delivered the first release of a new web-site to its client, FRESHCO, a small supermarket chain. The release included the following features:

• Customer maintenance (create, amend and delete account)

• Shopping cart facility (create order, add, amend and remove items from order)

• Checkout order

• Pay for order

• Product search The release was delivered late, primarily due to issues with the failure to correctly interface with external card processing systems when attempting to pay for groceries in the shopping cart and also because the performance of the product search facility failed to meet the contractual response times. Although the interface issues were resolved before implementation, the performance issues were not, resulting in your company incurring financial penalties which will continue to accrue until a satisfactory solution is delivered. The second release for the web-site will include the following:

• Allow customer to create and use a ‘Favourites’ list

• Include the facility to maintain multiple delivery addresses in the amend customer account function

• Performance fixes for the product search module

• Other fixes to various residual defects from the first release


of 28

17 Referring to Scenario 3 – You are to create a defect taxonomy to aid test design for

the second release. The taxonomy is to initially cover known problem areas only. The following Release 1 defect report has been produced to assist you in this task.

Taking into consideration the issues encountered in the first release, the contents of the second release and the defect report, which three of the following system functions would be the MOST important entries in the defect taxonomy? Note that ‘closed ‘defects in the report have been sub-categorised as either ‘Fixed’ (i.e. fixed and successfully re-tested) or ‘Not a Problem’. A Create Account. B Amend Account. C Delete Account. D Cart – Create order. E Cart – Maintain order. F Checkout order. G Pay for order. H Product search. I Maintain Favourites list.

K4 (3 marks) Learning Objective – 7 Analyse the defect reports created over time and update the defect taxonomy. Answer: B, E, H


of 28

Rationale:

• H (Product search) must be given highest priority for Release 2 as the Product Search function is costing the supplier financial Penalties until it is fixed. This is irrespective of the number of residual defects.

• E (Maintain order) has the highest number of residual defects from Release 1, by far indicating attention during testing at Release 2

• B (Amend account) is roughly equal to A and D for the number of residual defects (5, 5 and 6 respectively). However as Amend customer account is being enhanced in Release 2, and the others are not, it should be give higher priority

G (Pay for Order) may have been problematic in release 1 but there are no residual defects and no changes are planned. I (Favourites list) is new and of course should be tested but it is not a known problem area, which we are told the report should be confined to. C and F are not changing and have little or no residual defects.


of 28

18 Referring to Scenario 3 – You have instigated a lessons learned review following

implementation of the first release and attempt to analyse the incident management database from that release. However, it becomes clear to you that insufficient data was being captured during an incident’s lifecycle and the process itself seems to be missing important steps, making it hard to establish where improvements could be made. You have therefore decided to seek external guidance to improve the project’s incident management process.

Which one of the following sources of standards would provide the BEST guidance for this task?

A International standards. B FRESHCO’s live incident reporting standards. C National standards. D Domain specific standards.

K2 (1 mark) Learning Objective 8 Summarize sources of software standards and explain its usefulness for software testing. Answer: A Rationale: A - International standards include IEEE, specifically in this case IEEE Std. 1044-1993 (Standard Classification for Software Anomalies), IEEE Std. 1044.1 1995 (Guide to Classification of Software Anomalies) and more generally and IEEE829 (Standard for Software Test Documentation) which covers the contents of an Incident report. Note that IEEE 1044 is also addressed in some detail Chapter 7, Incident Management, so candidates should be familiar with this standard. The syllabus in Chapter 8 makes it clear that IEEE is an international standard. Option B (not mentioned in the syllabus) would not be appropriate as Live Incident reporting often requires a different process and data from incidents arising during the project development lifecycle.



of 28

19 You are testing a new missile guidance system for a commercial aeronautics company. This is the first phase in a multi-phase project, which develops a basic prototype of the new system; subsequent phases add more functionality until the final product has been built, tested and ready for implementation. You already have test and defect management tools from a single vendor, used by the team on previous projects and are keen to invest in a test automation tool for the prototype phase.

You have identified a number of commercial and open source tools which make the following claims:

A Tool ABC, from a second vendor, is a stand-alone automation tool that provides

basic capture/replay features, is simple to use, quick to learn and much cheaper than its rivals.

B Tool XYZ, provided by your current vendor, is feature-rich and interfaces to other tools in the suite but is the MOST expensive option.

C Open source tool OP1 is free and provides developers the ability to enhance the tool for future use.

D Tool PQR, from a third vendor, uses a proprietary scripting language for which free training is provided, can be configured to work on multiple platforms and is mid-priced.

Which one of the four tools above would be MOST suitable for your project?

K2 (1 mark) Learning Objective - 9.2 Test Tool Concepts. Answer: B Rationale: A Don’t be tempted by short term savings – this is only the prototype of a multi-phase

project so consider long term use. Unlikely to be able to integrate with existing tools which will cost time and effort longer term.

B The best choice. May be expensive but likely to be a better long-term investment on a multi-phase project and will integrate with other tools

C Open source tools should not be used on mission/safety critical systems (ref 9.2.7). D We don’t know whether multiple platforms are an issue. Understanding and

maintaining a proprietary language could prove costly in the long term and is unlikely to make the tool available for the prototype phase.


of 28

20 You have recently been appointed test manager for an organisation that was

certified at CMMI level 1 maturity following delivery of its last project, which overran schedule and budget. It is already underway with the delivery of its next project for the same customer, a DIY/Home Improvement retailer. Requirement specifications have been agreed and baselined, with development and component testing already underway.

The test team for the last project was staffed by business analysts and developers, and the customer has insisted that this is rectified for the latest project. System test analysis is due to start in two weeks time, so you need to urgently recruit new test personnel. You have received a large number of CV’s from various recruitment agencies and have an interview schedule in place.

You have found a sample questionnaire covering the recruitment of test personnel and want to refine it for use during the interviews. Which of the following interview questionnaire items would be MOST appropriate for the type of test personnel needed? Choose the three BEST options.

A Does the candidate possess a good working knowledge of the client’s business?

B Does the candidate have experience in using test management tools? C Does the candidate possess a good working knowledge of specification based test

design techniques? D Does the candidate understand a risk-based testing approach? E Does the candidate possess a good working knowledge of structure based test

design techniques? F Does the candidate demonstrate good negotiating skills? G Does the candidate demonstrate good communication skills? H Does the candidate have experience in using capture replay tools?

K3 (3 marks) Learning Objective - 10.2 Use a given questionnaire in order to determine strengths and weaknesses of team members related to use of software systems, domain and business knowledge, areas of systems development, software testing and interpersonal skills. . Answer: C, D, G Rationale: The key is recognising that this is a very immature testing organisation (CMMI 1) which effectively has to be built from scratch so we are looking for fundamental testing experience/skills not high-end specialism. Also that the stage of the project lifecycle indicates that the priority should be on recruiting test analysts for system testing, as development/component testing is already underway.


of 28

A Tempting but not essential, as this is not mission/safety critical or a particularly niche business. Not a top three choice, but close.

B Could be useful but at CMMI level 1, test tooling is not the first priority C Yes – important for effective test analysis D Yes - It is already late to be recruiting test personnel so schedule pressures are

likely to necessitate effective test prioritisation E Important for component testing but as this is already underway the focus now has

to be on tests analysis for system testing. Current component testing should still be looked at carefully though, so this skill may be required in the near future

F Important for a test management role but less so for test analysts G Yes - important in any testing role, especially during test analysis where

communication with business analysts and developers will be frequent H Definitely not needed at this stage (see B)

istqb advanced test manager certificate in software testing · istqb advanced test manager...

Documents