taking testing to cloud

Your Monthly Magazineon

Software Testing

www.TestingCircus.com

with

Smita Mishra

Testing CircusVolume 5 - Edition 3 - March 2014

Article submission guidelines –● Subject of article can be based on any area of Software Testing. If you want to publish your article on

theme based subject please read our announcement of monthly theme published in our site. Articlecan be submitted without any theme based subject.

● There is no minimum and maximum length of article. If you feel the article is lengthy, please dividethe article into logically separated parts so that we can print them in a monthly series.

● Give a meaningful title to the article. If you want a sub-title as well , then add that in a different line.● Add images/pictures if necessary. If you are using any image/picture which is not yours own work,

please include the source. Take care of copyrighted materials. Images need to sent separately with thearticle.

● Send us the article in MS word (doc/docx) format only. Pdf files are not accepted.● Write a short write up on the author(s). Usually 7/8 liners in 3rd person descriptive language.● Include photograph of author(s). Preferred in high resolution .jpeg/.png format. Ideal size would be

50mmX 50mm.● You can send your article any time. Since we publish every month, your article can be included in any

month. There is no such thing called as a dead line.● Send in your article to [email protected] with a subject line “Article for Testing Circus –

Author Name – Title of the article”● If you think you can write a column in Testing Circus for at least 6 months, please submit 3 articles in

advance. We are open to any idea that may improve the user experience of Testing Circus.● We will publish the articles in our website a week after the pdf magazine is published.

ArticleSubmissionGuidelines

Do you have something to share with thetesting world? We can make your voice

heard to testers.

http://www.testingcircus.com/article-submission-guidelines

www.TestingCircus.com March 2014 - 02 -

Volume 5 - Edition 3 - March 2014Testing Circus


Topic Author Page #

Interview with Smita Mishra Jay Philips 5

Examples of Testable Requirements Ulrika Park 10

Taking Testing to the Cloud Vipin Jain & Anubha Jain 14

A Common Pitfall of Test Cases Raj Subramanian 22

Mobile Testing on the Cloud Gagneet Singh 23

How Quicker can mean Slower Peter Morgan 25

7 Types of Testers - What is your identity Arslan Ali 27

Book Review - “Don’t Hire the Best” WoBo 31

A Fake Tester’s Diary, Part - 39 Fake Software Tester 33

Testers to Follow Editorial Team 35

Test Environment for Security Testing Santhosh Tuppad 37

Running Webdriver Script on a VM Mihai Sarlea 39

Test Automation: New Venues, New Challenges ToolsJournal.com 48

TestComplete Support For Automated Mobile Testing ToolsJournal.com 50

Squish Coco Supports Code Coverage Of C# & Tcl Code ToolsJournal.com 52

Testing Events Around the World TestEvents.com 53

Testing Circus TeamFounder & Editor – Ajoy Kumar SinghaTeam -� Srinivas Kadiyala� Jaijeet Pandey� Pankaj Sharma� Bharati Singha� Chanderkant Saini� Dwarika Dhish Mishra

Editorial Enquiries: [email protected] and Promotions: [email protected]

Testing Circus IndiaChaturbhuj Niwas, 1st Floor,Sector 17C, Shukrali,Gurgaon - 122001India.

© Copyright 2010-2014. ALL RIGHTS RESERVED. Anyunauthorized reprint or use of articles from this magazineis prohibited. No part of this magazine may be reproducedor transmitted in any form or by any means, electronic ormechanical, including photocopying, recording, or by anyinformation storage and retrieval system without expresswritten permission from the author / publisher.

Edition Number : 42 (since September 2010)*On the Cover Page - Smita Mishra

What i

s wher

e?

Write to us at [email protected]


Testing Circus is three and half years old. This magazine has been running onvolunteer effort, assisted by few passionate testers from time to time. It has been anamazing journey so far. But, like all other good things in life, it is coming to an end.We are announcing discontinuation of Testing Circus after this edition.

It is a sad moment for all of us. Running a regular magazine purely based onvolunteer effort is hard task, specially when all volunteers including me have fulltime jobs to look after. We were always supported by many testers by contributingsome awesome articles but let me also admit that it is always a challenge to gathergood articles when you don’t pay the writers and busy testers do not reply to youremails and direct messages. There was too much effort for our volunteers.

Testing Circus site will remain as it is. We will continue to publish our old articlesin the site. We will continue to promote good testing stuffs in Facebook/Twitter. Butour regular monthly magazine will stop.

I would like to thank all you of you who supported us in this wonderful journey. Wewill cross the roads again.

Till then, stay hungry, stay foolish and test passionately.

- Ajoy Kumar Singha@TestingCircus // @AjoySingha

From the Keyboard of Editor

https://twitter.com/TestingCircus

https://twitter.com/ajoysingha

SMITA MISHRA

Smita Mishra is the founder and chief consultant at QAzoneInfosystems, which is a pure-play software testing organization. She isa first generation Entrepreneur and is a Test professional who hasspent over 12 years practicing testing and leading test efforts ofvarying sizes, cutting across all key domains and technologies. In thepast, she has worked with multiple organizations, likes of - HCLTechnologies Ltd, Fidelity Investments, Nucleus Software Exports Ltd,Churchill Insurance (Now RBS).In her current role, she is involved in creating test teams, managingtesting for software companies, leading the overall test strategy forthem. She is also engaging constantly with different forums to assistgrowth for women in her field and women in general too.

Organization: QAZone Infosystems Pvt LtdCurrent Role/Designation: Chief Test Consultant & CEOLocation: New Delhi, India

Interview with Testers


1. Tell us about your journey to becoming a softwaretester. How did it start and how this has been so far?Was it planned or by accident?My journey to become a good tester still continues.However, it began in 2001, with my first job atNucleus Software Exports Limited, where I wascampus placed after my engineering. A batch ofabout 20 engineers were picked from across thenation and put together as the founding(independent) test team. Honestly, I had nointroduction to “Testing” until then other than thefew chapters I read on SDLC, Verification andvalidation in a book of software engineering by“Roger Pressman”. It was planned by my employersnot me but wasn’t really an accident. Once in testing,I quickly began to learn it and found it interesting.I will agree that though I really enjoyed my work,the initial years were nothing exciting. But over theyears, I realized that I enjoy testing more thananything else. There have been tremendous learningthrough the 13 yrs spent in the field. The more Ilearn the more I see I have yet to learn.Hence, I can easily say that the journey so far hasbeen terrific. 2014 seems to be extraordinarily goodyear for me and I am loving every moment of it, as atest professional.

2. When did you realize your passion was softwaretesting?I would be wrong if I said that it was love at firstsight. It was not. But as they say – good music growson you even if you don’t like it much in the first go.It was similar in my case. I realized I enjoyed testingearly in my work life. But it wasn’t until 2006 that Ibegan to love it. I attended the first conference in mylife – it was a QAI conference where I was presentinga practice paper on estimations. Met many testersthere and I loved the feeling of belonging to acommunity too. Then in 2007 I went to work in GEHealthcare account under the most dynamictechnology leader and sharp as razor – GazanfarHakeem and an excellent Test Manager Smita Sethi(who also happens to be a very sharp professional). Igot exposed to very technical testing in DW-BI (ETLTesting) and performance testing of these ETLs andimplementing lean methodology. I would term thatas the turning point in my career on how I began tolook at my testing career. I realized I was cut out todo this.

3. Do you regret being associated with softwaretesting today? Given a chance would you move fromtesting to any other field in IT?

* Interviewed by Jay Philips


No, clearly no. Quietly no. Loudly no. Simply no.I am proud to be a part of an ever evolving field andthat has a huge legacy of achievers. I can easily say Inever had second thoughts about my work. Also, Iwould find it very difficult to work on a singletechnology or domain all my life – and to lookconversely at it - which other field can give me somuch exposure in understanding “what reallymatters” in IT field, while keeping me hands on withall latest technology.

4. What does QAZone do? How are its servicesdifferent from the services offered by other similarorganizations?QAZone is a pure play software testing organizationthat offers Testing solutions and services, Testconsulting, Test training and test support services(Test data management, test environment setup). Weoffer Business as usual testing as well as specializedtesting. We have built a client base of over 20 clientsin last 3+ years.Our business focus is our key differentiator. Ourtesting solutions are designed to work for yourbusiness. We look at technology as a platform tomake business happen. All our testing has acomponent of domain focus which becomes a veryserious need of our clients when we are dealing withregulated environments like Aerospace andHealthcare.Our next differentiator is that against the trend - webelieve in hiring locally, even if it means trainingpotential resources at our expense. For example –anytime we need a resource in US, our first approachis to find a resource locally. This has so far proven tobe a very fruitful approach.

5. Last year you organized a conference calledThinkTest in India. Was that the first year of theconference? What were some key sessions andfeedback?

We had planned for a conference ThinkTest 2013,which would have been the first of our initiative inthis direction. We were very keen to bring JamesBach to this part of India (and we still want this tohappen). Unfortunately, due to unforeseencircumstances at my personal end and not havingsuitable alternatives, we couldn’t take this forward.We will keep the test community updated of futureplans related to this.

6. In February 2014 you held the first meetup for TestPractitioner’s Club. What made you want to createthis meetup?It always feels easier to work with people whom youhave met and can relate to them as a face and as thevibes you share with them. All leading testconferences are still physical conferences notwebinars. That’s because learning happens in manyforms today and one of them is networking postconferences. With the thought of building a networkof co-learners, we created a LinkedIn group – TestPractitioner’s Club and planned for its meet-ups thatwould help local testers to learn more from eachother and get introduced to global platforms too. Ifwe could – we would have such test meetups allover the country. We are planning to do one meetevery month at the very least. We will keep postingthe updates from our meets at my blog, for all toread.

7. I noticed that you are certified in ISTQB and QAI(CSTE). What made you decide to go for bothcertifications?Yes, that’s true – I have indeed done both thecertifications. I went for ISTQB (in 2005) becauseuntil then I had no certifications. And under ourtraining and development program, my organizationpaid for this certification. However, later on I wentfor QAI (CSTE), as it was not widely done so far dueto its cost and students failing in it and this made it

SMITA MISHRA

http://www.associationforsoftwaretesting.org/programs/ast-grant-program/


appear more respected and challenging. I paid forthis one myself.

8. Do you recommend that all testers get bothcertifications? Are there any other certifications youwould recommend to other testers?I would let testers decide this for themselves. But Ican say it with reasonable confidence that thesecertifications do not prepare testers to handle serioustesting problems. I enjoy being part of miagi-do anddiscussing actual test issues with real testers thereand how to resolve them. I have heard very goodthings about BBST and most of the leaders I followare associated with it. I would suggest all testers toatleast go through the content and format and decidefor themselves.If learning is the objective and certification is notreally the goal, I would suggest testers to go throughJames Bach’s site for RST classes and RTI (online)classes.

9. According to you, what is lacking in today’scommercialized training industry, especially intesting?With all due respect to the 2 certifications I did, Ibelieve an ideal test certification would be one whichwould have more practical problems to deal with, astests for their students and would involve real lifesituations of testing than focusing on glossary oftesting and terminologies. Also, I would invest indomain led testing certifications for my team. So far,there are no such certifications available whichwould certify testers for regulated environments likethose of aerospace and healthcare. I have heard of anew certification that Cem Kaner has come up with. Ineed to look into that one too, if it has domain basedtesting certifications.

10. What qualities will you look for in a candidatewhen you want to recruit someone for softwaretesting job?I like to work with enthusiastic people who enjoylearning new things and working on new areas. Thisis true not just for software testers but front officeexecutive, accountant, HR and Admin folks - all ofthem.Many today do not feel the need to look at the pastexperience and hire only for attitude. However,when I am hiring testers particularly – I like to seethe kind of work they have done before and like tohear their stories of projects on exactly what was thatthey were testing and what were the key defectsfound. What was their approach and the keychallenges faced and how did they bring up thechallenges to the notice of relevant stakeholders andhow did they do the contingency or mitigation asapplicable.Formally speaking, we have 2 set of patterns – onefor freshers and other for lateral hiring.For laterals particularly - We have a process thathelps us map the required technical skills with theavailable resumes. So, anytime we have a need welook into our database of testers who have appliedwith us in past and from the matching resumes, weshortlist based on availability / joining time andexpected compensation. Post this, we share certainproject links with these potential candidates andexpect them to submit us their Bug Reports in agiven time frame. Finally, after a candidate is foundtechnically suitable, we setup face to faceinteractions to evaluate more HR aspects and generalcommunication.For freshers, it’s really about what they have to offerbeyond their formal education. No firm expectations,as long as they are able to convince us of them beingfast learners and having aspirational attitude, theymight.

SMITA MISHRA


11. What will you suggest to people who want to joinIT industry as software testers?I am very happy to be part of the software testingcommunity. And I can only suggest the new folksthat learn what is testing and then get into it. I findpeople to be doing testing for all the wrong reasonslike – its easier that Development, it doesn’t needcoding, timelines are easier and hence the workpressure, etc. None of this is true. Testing isbecoming as challenging as any other part is with theever evolving IT andWhen I look back, I think the biggest gap that I seeduring my initial work tenure was how closed andwe were in our approach to testing. working in silosas a closed group and not being exposed to thewhole world full of knowledge and awesometeachers and trainers willing to invest time in you.

12. Name few people you would like to thank,people who helped you directly or indirectly in yourcareer as a software testing professional.I would like to thank my family for always standingby me – ALWAYS. But a special mention of my sonhere for being the world’s most loving and caringson and being very understanding when his mommyneeds to work.

At work front there are too many names and I wouldlike to thank each good and bad interaction, becauseit has helped me grow in one way or other.Everyone I know – Thanks for being part of my life.

13. One last question – Do you read Testing CircusMagazine? If yes, what is your feedback to improvethis magazine?Yes, I do read Testing Circus Magazine. I first heardabout it in 2010 I guess when my manager Akashhad been invited to be interviewed in it. But I neverfollowed it then. I began to read Testing Circus sincesometime in 2013 onwards.I liked the original format of the magazine. But thenew format is truly amazing. I really enjoyed theeditions coming out in 2014. And I think they aredoing all the right things to get noticed and to put inright content. My only words will be – continue thegood work.And maybe you can cover small meetups and grouptest events happening across the globe as a regularcolumn. This could help other meetups to learn whatmore can be done and how to be effective.

Blog/Site: http://smitamishrablog.wordpress.comTwitter ID: @smita_qazone

SMITA MISHRA

http://www.testmile.com

The simple thing is this - write your requirement as atest. With defined inputs and outputs. Expected resultsand expected (and unexpected) data.

If you do this, your life will become bright, shiny, andyou will live happily ever after. Or at least your soft-ware will become much much, much more reliable.And you’ll probably find out a lot of things about yourideas before you’ve invested in building unnecessaryfeatures and details.

You can do this with high level requirements, such asbusiness goals and overall objectives, as well as withlow level isolated features, and everything in between.

I think that the “everything in between” part is wherewe (software industry people) lacks the most care andinsight about the importance of concrete, testable re-quirements.

When it comes to high level requirements, we may havebusiness people who do follow up business cases &objectives, i.e. test the results of the investment, at leastI have seen it done once or twice.

When it comes to very low level requirements, or mi-cro-requirement as my friend @spindelmanne callthem, TDD do take care of it to some extent. Such as“When renaming item x the list will keep the same sortorder”. It’s hard to separate micro-requirements fromreal business requirements sometimes. “What is reallya valid input string here?” “How should we present the

date format” etc but good developers generally canmake some good micro-requirements decisions.

For the “everything in between” requirements, we havea lot of work to do to make them testable. From what Iknow it seems as there are mainly 3 ways of communi-cating requirements today.

Either you’re “agile” and have a loosely defined prod-uct backlog, filled with short user stories and then notso much more information.

Or you have a heavy regulated requirements process,with hundreds of pages of use cases or “shall”-require-ments. Often with abstract statements such as (fromreal example): “Purchase has generated a receipt”

Or the ad-hoc requirements: “Let’s send an email to thedeveloper telling what I need to have”. Example: “Weneed to update the purchasing order receipt page. Rightnow it doesn’t show the total. The total need to be there.When can this be done?”.

As a requirements analysts / project manager I haveseen and practiced a way out of these three abstract,ambiguous, non-informative ways of communicatingrequirements. Much thanks to developers who serious-ly cared about taking TDD to the next level, and byhaving the chance to work with testers close by whotaught me how to express what I want as test scenarios.

I’ll share some examples from a previous project.


- Ulrika Park

Requirements People NeedYour Help!

Examples of Testable Requirements

https://twitter.com/spindelmanne

A testable business requirementI was asked by the business owner to implement a fea-ture: “Cardholders should be able to edit the rights for awhole household to use the money on their bonus card”Since money and banking was involved, it was a bitcomplicated to implement.

My first question to the business owner was: “why?” andhow will you know it works?”

After quite a lengthy conversation, he said that what hereally cared about was that the money on the bonusaccount was spent. He didn’t want the money to stay ontheir bonus cards.

“How can we verify that this target is achieved?” I asked.“Well.. he said. If the money is spent, then the featureworks.”“So.. when in time is realistic that we can check this..?”“Well.. within 6 months we should have a better rate ofspending the bonus money than now” he said.“Ok. So what do you mean by ‘better’?”“Hm…” he said. “I’d be content for now if 50% of thetotal money paid out to customers bonus accountswould be spent”.“Thanks for clarifying! Our feature could help out withachieving that goal. But to achieve this, other things areinvolved. Customers need to know about how to sharebonus money between people in their family. How willthey know? Marketing, customer service.. a lot of factorsmight affect if this feature is used by the customer.”

“Yes, of course. But this is what I really care about. Andwhen you have a feature households can use, we shoulddo an effort to inform customers”.

Good. Now we had a high level business goal, a testablebusiness requirement. Even though our featurewouldn’t be the sole solution to make the businessachieve this goal, knowing the target for sure helped usa lot in developing the feature.

What to do when you don’t have access to the businessowner? When maybe you just get a bunch of use casesfrom somewhere to implement?

Well, in these cases I try to define my own hypothesisabout the main goal and result. “This is how I / we have

interpreted the target since we don’t know” and thenshow for those stakeholders I do have access to. Often Ido get some feedback on my hypothetical business goalstatement. Even “You’re totally wrong in your assump-tion!!” is good to know before developing anything. Andyou have a reason to ask for answers.

Before testing or developing any feature, we have toknow or make a clear defined assumption about theexpected result for business.

A testable middle level, user requirementSo now we knew the business goal of the feature. Thefeature could be implemented in many ways, with op-tions from everything from printing and scanning paperforms to digital authorization functionality.

As a requirements analysts, turning into a tests-before-development tester, I defined some user stories. (We dida lot of other things too to understand what solutionmight fit, but that’s another story).

The main “middle level” user story:As main cardholder I want to authorize other card-holders inmy family in order for anyone to use the money on the bonusaccount.

Before communicating this to the development team, Istart to think about.. how to test this? What would I test?I brought in a tester for a chat. The tester was busy withother assignments, but he did have a few minutes to helpme out. And I asked him “How would you test thisstory?”“Identify scenarios” he told me. And with some coach-ing I made up some scenarios. (here is just a snapshot tokeep the article short)

Scenario 1: Give authorization to other cardholder in ahousehold with only 2 cardholders.Scenario 2: Give authorization to other cardholder in ahousehold with several cardholders.Scenario 3: Authorization process is actively canceledby cardholderScenario 4: Authorization process is canceled by un-planned interruptionetc.

Then, exemplify these scenarios with Gherkin inspiredsyntax:


Scenario 1: Give authorization to other cardholder in ahousehold with only 2 cardholders.Given that:Household has 2 and only 2 cardholdersThe 2nd cardholder doesn’t currently have the right touse bonus moneyMain cardholder has actively selected the 2nd cardholderThe 2nd cardholder is >= 12 years oldExpected result:Information is shown: “You have now given authoriza-tion to <2nd cardholders full name> with SSN: <2ndcardholders SSN>.The 2nd cardholder now has authority to use bonus.

Scenario 3: Authorization process is actively canceled bycardholderGiven that:Household has 2 and only 2 cardholdersThe 2nd cardholder doesn’t currently have the right touse bonus moneyMain cardholder has actively selected the 2nd cardholderandhas entered external digital identification application toauthorizeandshuts down the digital identification applicationExpected results:The 2nd cardholder doesn’t have authority to use bonusMessage to main cardholder: The change has been can-celed.

These were just two examples. Doing this I had now:1. A specification that can be used for testing2. Got forced to find out the exact business rules regard-ing who actually could be authorized. The age limit, forinstance, was discovered by defining the tests. By defin-ing test scenarios, I got a foundation to ask the rightquestions to domain experts. “Can anyone be autho-rized?”3. A way to at any time go back and show for who itmight concern (developers, stakeholders, customer ser-vice) what exact requirements and rules we built for.

These are just small examples from a big domain, so Iexpect you have a lot of critique “Where are scenario x??And those examples could for sure cover much more!And aren’t that statement a bit vague? What does ‘ac-tively’ mean really..?”

The point here is not to give the full picture. For that I’dneed to write a book :-) The point is to show that bydefining tests while working with the requirement, therequirement got much more explicit. We saved time forour tester, who could focus on exploratory testing whentime came. We saved a lot of time for developers whoactually got the information ahead development on whatwould be tested, and what rules should be applied, sothey saved a lot of rework.

Also when having conversation with developers aboutthe examples we modified them a bit, and removedunnecessary ones, or added missing. Some of the scenar-ios could even be quite easily automated while develop-ing, which also saved us a lot of time.

So, what to do as a tester then? If you’re just not involvedin creating the requirement definitions? Maybe the re-quirements are just handed off to you in one way oranother, and when you get them they’re not testable atall?

One thing you can do as a tester, is to make an effort tobe included when other people are working with re-quirements. Offer your help!Only once I have been approached by a tester with thisoffer!

It’s always me who’ve approached testers to help mewith making testable requirements.

Have you ever as a tester tried to offer your help – andseriously tried? To the guys working with requirements? And there are much more efficient ways than to say “Ifyou need me you know where I am”.

A tester could say something like this: “I know I’mgoing to work on testing for this project a couple ofmonths from now. Is it possible for me to see some of therequirements already now? It will help me get rampedup quicker when I’m in the project. I’m in another projectright now, but still I should be able to take one or twohours to look at what you have. And yeah, incompleteuse cases or draft user stories will work fine too!”

When you do get hold of some kind of requirementsearly in the process, take a quick look. Try to define somesimple test cases or scenarios, and maybe you will get anopportunity to discuss these with the requirement ana-


lysts (or whatever role who works with the require-ments).. Then why not inviting her for a lunch or avirtual coffee break (or other social excuse) over Skype incase of distributed teams?

Requirements people need your help! They just don’tknow it yet. Who, if not you, will seriously invite tomake their work testable?

Ulrika Park is arequirements geek witha passion for testing,methods, learning & thedevelopment of prod-ucts & services withinorganizations andteams.With 15 years ofexperience in software

development, management & businessshe now works at SmartBear. She believes in thesynergy of people, software and quality thinkingto change the world.


Look around in various IT companies, flip pages of some IT magazine, read blogs, see forums and you will besure to find the today’s buzz word, the Cloud. You cannot find a word that has created so much of ripples thanthe Cloud. Very dramatically, the thinking process on providing computing resources has been shifted to Cloudcomputing. With this shift of focus, a lot of confusion has accumulated along the way in people’s mind.Companies are taking this concept very seriously. Many have moved to cloud, and those who are still on the“ground” are planning to move to cloud in next couple of years.

People are still confused. They still are not comfortable with the Cloud thought. You ask anyone what a cloudservice is and you still won’t get a very confident answer. Even if you get an answer, ask what benefits they offerand what are the disadvantages at stake, and you can see the person will be uncomfortable answering this. Makehim skip the discussion by asking how secure they are, and it will be end of discussion. When it comes to testingapplications, we are not fully sure whether to adopt the cloud approach of testing or not. What are the benefitswe get and what are the hurdles on the way?

Let’s take a closer look at the Cloud and how it can benefit out day today testing activities.

So what is a Cloud?Cloud computing is a business and economic model. This model has been successfully deployed and executedfor various material commodities since its inception, but in the recent years it has been formalized for IT productsand services’.

Let me try and explain the basic difference between a Cloud service and a Non-Cloud service.

Consider that you have to move from one place to another. You can use your car OR use the services of a Taxi.Both of these vehicles have several similarities

· Both are automobiles, having very similar structure and machinery.

· Both provide basic functionality of transferring people / goods from one place to other.

So where lies the difference?The difference is in the business model for the service provided by them.

· When the car is owners, the owner has to pay for the fuel, regular maintenance and even possibly agarage. In turn, the car provides the service solely to the owner - you.


- Vipin Jain- Anubha Jain

Taking Testing toThe “Cloud”

· When the car is a Taxi, the service provided by the taxi cab can be described as ‘Travel as a service’. Whoowns the car and who pays the maintenance is not your concern. As a customer, you just have to pay totravel from your place to another desired place. No maintenance fees and no parking fees need to be paidby you. This responsibility lies with the cab driver.

The Cloud is synonymous withthe phrase ‘On Demand’. Youpay only on demand (when yourequire it.)

Cloud gained momentum whenIT industry got associated withit. Today, we find that there is ahuge range of products and serv-ices available on demand. All are“As a service” e.g. ‘Games as aservice’, ‘Java as a service’, ‘Stor-age as a service’ and the list isendless.

Fig 1: TaaS or ‘Travel as a service’ – A Metro Cab service

How Testing and Cloud work together?Cloud-based testing offers a remarkable combination of low costs, pay-per use model and elimination of initialcapital expenditures. The benefits, however, are more than just cost effectiveness. The non-cost factors includeon-demand flexibility, a respite from holding various infrastructure assets, enhanced collaboration, higher levels ofefficiency and, most importantly, reduced time to-market for key business applications. Economically the vendorand end user gets huge benefits from the Cloud. This comes directly from reduced subscription prices of anyproduct or service.

Other benefits are:· Cloud Apps are scalable: The elastic business model, as it is popularly known as, can be customized based

on the requirement

· Auto-Provisioning: Depending on needs of end users, various Cloud vendors provide and withhold theofferings in a manner which is in an automatic and self-serving format.

· Unlimited resources: At least the end user thinks and feels this. The services / products are available as andwhen they are demanded and in the required quantity

The Cloud computing model should be coined as ‘Green Model’ as it maximizes usage of resources and minimizeswastage making it environment friendly.

Current state of testing in cloudMany companies are still taking a cautious approach with cloud computing. This is not the case with testinghowever. They are largely ready for testing in cloud and following reasons will account for that readiness:

· As we all know, testing is not a one time, but a periodic exercise and each project will require a newenvironment to be set up. If a company creates a Test lab, it typically sit unused for longer periods, resultingin a waste of cost, electricity and space. Lots of published reports indicate that more than 50% of the


technical infrastructure for testing remains underutilized.

· Since testing still being considered as a must but non-critical activity for business, taking testing in cloudpremises is pretty safe as it doesn’t include important company data and has minimum impact on theorganizational business activities.

· Today’s applications are increasingly getting dynamic, more complex, distributed across continents andmore component-based. Testing them is getting more challenging. For instance, with mobile and Webapplications testing, testing needs to be done for multiple operating systems and regular critical updates,various browsers and their versions, variety of hardware and a large pool of concurrent users to understandreal time performance. It is pretty difficult to follow the age-old approach of creating so many in-housetesting environments. These will become very complex and will need huge capital and resources.

Fujitsu in a 2010 research suggested that testing ranked second (57%) as the most likely workload to be put into thecloud after Web sites (61%). The on-demand provisioning by Cloud addresses all the above explained issues withone click. On top of it, the effort and resources saved by using cloud can be redeployed for core business functions.

The Cost FactorEconomic benefits are the main factor influencing companies to take testing to cloud. Another 2010 survey by IDChinted the same. As the global economy recesses, companies continue to find ways to regulate costs and improveROIs. Cloud testing reduces the unit cost of computing.


Small and medium-size businesses that cannot afford high costs find cloud-based testing as a new lifeline. Thecompanies no longer need to invest in infrastructure and software licenses. They also do not need to worry aboutvarious configuration issues and maintenance of test environments, and pay only for what they use.

Beyond cost Benefits?Cloud is not all about cost saving. There are lot more benefits companies can extract using it.

- A standard infrastructure and pre-configured software is available, reducing efforts in getting servers andlicenses.

- On-demand provisioning helps companies to think forward instead of spending time to set up test labs. Alltesting resources required for testing exist within the system and can be called upon instantaneously.

- Better analysis and control are offered to test teams to build and execute their tests and identify thebottlenecks. This helps in identifying possible runtime bugs a lot before they are actually found.

- It’s a great concept in motion between geographical distributed teams. Once a tester logs in and runs a test,the results are available over the cloud. The developer can then assess it and fix over the cloud itself. Thiseliminates back-and-forth communication between teams.

Limitations- Lack of standards: Absence of universal/ standard solutions to real world problems is a big issue. Each

cloud provider can have its own hardware, operating models and prices and may or may not offer anyinteroperability. This poses a huge challenge for companies when they plan to switch to a new vendors

- Security issues: Security in the public cloud is still a huge concern because the data may be stored in alocation which is outside a company’s legal and functioning jurisdiction.

- Usage: The everyday usage costs increases very rapidly if testing is done without a proper usage ofcloud-based test. Though pay-as-you-go clouds are used, it can be expensive if the testing is out of sync withrequirements.

www.TestingCircus.com March 2014 - 17-

- Performance: It is always an issue since publicclouds are used by many users in parallel. Situa-tions got created a company had to wait for therequired bandwidth to execute their tests.

ConclusionThis paper tries to explain the benefits and limitationsassociated with taking testing to Cloud. We have tried toexplain that why companies should start small and gainconfidence slowly to capture maximum benefits ofcloud-based testing. Once they believe that it has ledthem in speeding time to market, lowering of costs andensuring standards compliance, they can go big. Usingof pay-as-you-go or on-demand services intelligentlyand efficiently, companies can reduce cost of operationand ownership. Companies should pilot cloud-basedtesting as early as they feel comfortable before going tomainstream testing.

Bibliography

- “Determine if a Cloud is Usable,” blog post,Bloomberg Businessweek, Jan. 31, 2011.

- “Solving the Challenges of Enterprise Mobile Appli-cation Development With Cloud-Based Testing,”blog post,

- CIO, Feb. 17, 2011.- Rajagopal Sattaluri, “Testing Considerations for Ap-

plication Migration to Cloud Computing,”- Cloud Computing Journal, Feb. 8, 2011.- “Cloud Computing: The Good, Bad and the Ugly,”

blog post, Dynamic Data Inc., Feb. 1, 2011.- “Cloud Testing, A Growing Trend,” blog post, So-

nata Software, April 4, 2010.- Nivedan Prakash, “Cloud Testing: Attracting De-

mand,” Express Computer, Feb. 1, 2010.


Vipin has dedicated last 10 years of his professional career to the softwarequality and testing. Currently working with Metacube Software, India, hedeveloped his key skills in developing automation frameworks and automat-ing applications. With a proven record of implementing and refining testprocesses for various clients across the globe. he is the author of several articlesand seven well sold books in India. He is pretty active within the softwaretesting community by speaking at International and national conferences,writing articles and contributing to various blogs and forums.

Anubha is working as Associate Professor & Head, IT department, Internation-al College for Girls (ICG), located in Jaipur, India. An academician for last 11years, she has been involved in teaching and mentoring several students in thefield of Computer Science. Knowing that teaching is the best form of givingknowledge back to society, she worked as a lecturer in Subodh college, Jaipurbefore settling in her current role at ICG. She is currently pursuing her PhD, infield of Information architecture. Anubha is the author of 9 books and a regularcontributor in various forums.

To Subscribe

Click Here

Still relying onreading

Testing Circusfrom tweets& facebook

updates?Subscribe

just with youremail id and

get themagazine

delivered toyour email

every month,free!Testing Circus

www.testingcircus.com


http://www.testingcircus.com/subscribe-to-testing-circus-magazine





I recently gave a talk at a conference. I learnt manythings from my interactions with the attendees at mysession. One specific thing that stood out for me washow – testers no matter how intelligent, smart, criticalthinking they are, still fall into the common pitfallof using test cases for communicating testing progress andtest coverage.

If anyone reading this article, is still under this mindset,I would ask the following questions-

· Do the test cases cover each and every scenario, eachand every part of the system under test?

· Can you give accurate information of testing effort interms of test cases? Does it really make sense?

· Are we trying to hide from reality and give some sortof number to make us and our project managers feelgood when we report the number of test cases coveredto communicate testing progress?

· What do you actually mean by test coverage? Canany other information about the system influencethis?

Here is the common pitfall of completely relying on testcases.Firstly, 100% test coverage of the system is impossible.“Exhaustive Testing” is a misnomer. This being said, ifanyone says “I have 10000 test cases, I executed all ofthem and thus all my testing is completed”, it is flawedbecause it is impossible to cover every scenario of asystem by just executing all the test cases

· What if there are some hidden scenarios that thetester still hasn’t uncovered?

· What if another system influences the system undertest in a different way than expected?

· What if the system behaves differently under differ-ent situations and our test cases did not cover this?

….and so on. There are so many “What if” questions toask ourselves. It’s an endless list.

So the point is, NO; we cannot possibility write everypossible test case to cover each and every scenario of asystem. This is where as testers, we think about testingapproaches complementary to just executing test caseslike exploratory testing, automated testing, combinato-rial testing and other approaches based on the contextand scope of what we intend to cover. This may help touncover other weird scenarios or trigger unexpectedoutcomes. With this being the case, it may not be validto say, “I have 100% test coverage because I executed allthe test cases”.

Secondly, you may then ask, how should I give up-dates on testing progress to my stakeholders? In myexperience, I usually do not have a problem in explain-ing testing progress to my stakeholders even if they areold school and believe “test cases are the solution to allthe problems”. I would of course have some test casesthat I execute, but also do lot of complementary testingto test case execution. Finally, I give the approximatepercentage of modules covered/tested in the system asmy testing progress.

So for example- If there is a system A, I generally splitit into different modules M1, M2…….Mn (test areas). Iuse different testing approaches to test these modules


- Raj Subramanian

A Common PitfallOf

Test Cases

and give an update in terms of the percentage of mod-ules I have covered in the system. (This is a generalapproximation as there is always a possibility that Ihaven’t thought about some other module in the systemwhich may cause problems. To err is human.)

So, say if I had covered M1, M2 out of the 5 modules Ihave identified i.e M1, M2……M5, then I may have cov-ered about 40% of the system ([2 modules/5 modules] *100, in case you were wondering how I came up withthis percentage ) . This is how I give my updates ontesting progress. This by itself helps to give the stake-holders some level of visibility into testing progress andhelps to make decisions accordingly.

To summarize, I think as testers we know it is ethicallywrong to make false claims. That being said, just usingtest cases to estimate the amount of testing progress andtest coverage falls under this ethical problem as we aregiving wrong impression to our stakeholders who trustus. Doing this tarnishes tester credibility which is one ofthe most important factors that influence a tester’s workand helps to establish trust among his peers.

Raj Subramanian is a pas-sionate tester, who hasprevious experience work-ing as a developer andmoved to testing to focuson his passion. He gradu-ated from Rochester Insti-tute of Technology with aMasters in Software Engi-

neering, worked as a developer for a payrollprocessing company and currently works as a testengineer for a major insurance company withspecific focus on mobile testing. He has been pret-ty active in terms of learning and contributing tothe testing society by speaking at conferences,writing articles, blogging and being directly in-volved in various testing related activities. Hecurrently lives in Cleveland, Ohio.Raj blogs at http://www.rajsubra.com/blog/And tweets with @epsilon11


moolya sucks

we test fast and don’t know to make more money from our customers.

we are like this only

[email protected]

You have been asked to test the latest Mobile basedapplication, created painstakingly by your organizationover the last few months. All you have with you is yourexclusive Smartphone, which you saved for, andbought, over the last few months / years. The choicesare now with you, either beg/borrow/steal from col-leagues or do the right thing. Fire up that web browseron your Smartphone, open up ‘Google’ and have a goat searching for ‘Mobile Test Partners on the Cloud’. Iwould generally as a rule go for the second option andnot put ‘my precious’ phone to the test on un-testedapplications.

What you achieve when you go through the search, is awide variety of web sites and commercial vendors of-fering this facility to you, at a fraction of the cost thatyou would spend in setting up your own facilities andtest lab. You need to now make an informed decision byresearching and figuring out which one of these wouldserve your purpose the best and also give you value forthe money spent.

Mobile testing has come a long way. From the initialfragmented scenario of having to check on each kind ofscreen resolution and phone type and screen size, addto that the variety of mobile browsers being offered bythe various vendors, to the current situation of havingapps created using HTML5 versus native apps. Addingto the general confusion is the non-app area, wherecompanies wish to check the overall ‘responsiveness’ oftheir “web pages” across the same wide variety ofdevices, which includes [and not restricted to] iOS(iPhones and iPads), Android (Tablets and Phones –low end and high end), and recently the Windows 8

(WP8 devices and Windows RT devices and desktop’swith Windows 7 & 8 included) and Chrome OS (mainlylow end laptops & devices). This vast array of devicesalso brings up with them an equally confusing array ofbrowsers along with them (at the last count it wassomething like 9 browsers and growing).

Faced with a similar dilemma, we take refuge in thewell-trodden path of checking on the search engines fora Web Accessible Mobile Testing Tool (applications andbrowser based systems), that would be able to serve ourpurpose and not cause the management to jump fromtheir warm seats when they finally receive the bill forthe services used to test their precious new mobileapplication or website.

To get through all this, I listed down a few of the up andcoming offerings which provide a good cross-section ofthe devices and are reasonable in costs. Although thesecloud platforms do provide a service which is extreme-ly useful, keep in mind, you might need to make use ofphysically handling the device to run certain tests,which cannot be checked with automation (but thereare ways and means to handle this, so don’t be disheart-ened). A hybrid cloud installation in these situationscan be one of the simpler solutions which come handy.A hybrid cloud provides a small subset of devices atyour physical location and the wider variety at a remotelocation. Of the Could Mobile Test providers, the mostimportant thing to look into is if they provide you withstorage for your tests and a way to run the automatedtests that you have so painstakingly created for yourapplication from within the cloud infrastructure. This isalong with the use of making sure that these tests run


- Gagneet Singh

Mobile Application TestingUsing the Cloud Infrastructure

over a wide variety of network speeds simulated toprovide the 2G, 3G, 4G and Wireless speeds prevalent inthe World Wide Networks around the world.

Practically a Cloud Based system should be able to caterfor most of the requirements outlined above; and thegood news is that most of them do exactly that. A few ofthe prominent ones which I have used recently and theones which come to my mind are: pCloudy.com,appurify.com, perfectomobile.com, HP mobile testinglab, to name just a few of the upcoming entrants in thisfield, where by Gartner estimates, there are 5.6 billionhandsets present. pCloudy.com has stood out as an ex-cellent upcoming product, being run by engineers whohave previously worked with Nokia and other top mo-bile hardware development companies. They have builtthe framework for testing on mobile devices from theground up and have got some really cool features whichgo with it. (Disclaimer: I know one of the Co-Founders asa colleague from my previous companies). I loved theway they have handled the storage of tests and results onthe site itself, along with providing features such asmock location maps, allowing the users to experience thedifferent states of the application in separate geographi-cal locations. They have recently launched a feature ofhaving multiple browsers and simplified their launchingfrom within the cloud interface, making things easier forusers again. And have launched the latest Android‘KitKat’ version with the Nexus5. All in all, an excellentpackage to go for, with reasonable rates.

All said, the main purpose of having a cloud basedmobile test experience is important for any companywanting to launch its web presence these days. With theadvent of HTML5 and other technologies like Founda-tion (more on this later), the web has become a placewhere people love a responsive site (or applicationacross iOS, Chrome OS and Windows [Phone] 8) thatcaters for whatever device they are working on, and doesnot have the staid look and feel when they change froma Desktop -> Laptop -> Handheld Tablet -> Smartphone.They want to get the feel that the web site developer /organization has done their homework and providedthem with a site where they do not have to pinch-in andout, just to read content. With the expansion of Smart-phone markets in the developing nations and organiza-tions wanting to tap into the ‘billions’ of people there toadvance their products, it has become imperative forthese organizations to go through the process of Mobile

Testing their web sites and mobile based applicationsacross various low end and high end devices to be useful.

Sites like pCloudy.com, appurify.com, etc. are making iteasier and faster to send out mobile products into themarket by providing the required platform at a fractionof the cost of actually acquiring a complete mobile testlab. That said, you would still need to work out a smallsubset of your tests at a physical location, but I am surethat this would also become possible over the cloud.With the advances these start-ups are making with tech-nology and improvements on their own feature sets. Icertainly would be looking forward to features whichprovide a friendly interface and let me run across multi-ple devices; and for that the Mobile Test on Cloud pro-viders are definitely a better option than trying to keepup with the influx of the hardware been thrown into themarket by all the top end and low end hardware manu-facturers around the world.

Gagneet Singh has beenworking in the QualityAssurance/Test field forthe past 8 years (with anadditional 4 years in Sys-tem Tools development)and has been involvedwith companies such asToshiba, Adobe (Macro-

media) , McAfee, Oracle, Yahoo! and recently Mi-crosoft.He likes to blog and to write about the experienceshe faced in the various organizations and situa-tions. His work has mostly been with AutomationTesting, along with Performance QA. Also, Secu-rity testing over the much hyped “Cloud Comput-ing” (using Hadoop and Azure) has figured in hiswork area.Currently working out of this place they call the“Down Under“, where he lives in Sydney, NewSouth Wales!


I want to define what I mean by ‘late’ and ‘early’ andthen go on to give some context from a firm I onceworked for – let’s give them the fictitious name “Lin-man Manufacturing” for convenience. If I don’t definemy terms, we will all get lost. I will use ‘early’ and ‘late’in an almost intuitive way, although perhaps not aseveryone uses them. They are not of the HumptyDumpty variety (Alice Through the Looking Glass byLewis Carroll: 'When I use a word,’ Humpty Dumptysaid in rather a scornful tone, ‘it means just what Ichoose it to mean — neither more nor less.’).

So, for me ‘early’ is before we were ready, and ‘late’ isafter the planned date. These are pragmatic terms – let’snot get argue about them now. My hope is that you willsee the usefulness for yourself. To start with, I need tosay something about project deadlines. First of all, it isnot testers (or test managers) who determine when awork product is implemented (into PROD, obviously).Test resources can speak into, or provide information,that helps others make that kind of decision, but thefinal decision lies outside “testing”. Secondly, we per-haps all recognize that if the decision were left to tes-ters, work products would not get implemented. Wetesters are a pessimistic breed, and tend to focus onthings that don’t work. It is not always the 390 require-ments that work that get our attention, but the 1 thatdoes not. We concentrate on that.

Lastly on deadlines, some deadlines are more impor-tant than others. Take the day I am writing this article.‘Tomorrow’ I am preaching at a local church, and some-time before standing up, I need to have prepared whatI am going to say. That is an immoveable deadline.Similarly, the deadline to ‘post’ this article is midnighttomorrow. If the editor gets it by then, it will be consid-ered for the next edition. Otherwise, it will be a candi-

date for the edition that follows. This is anotherdeadline that has significant consequences if it ismissed. Some software projects have hard-wired dead-lines that cannot be missed. Perhaps there are legisla-tion changes that need to be complied with. Theintroduction of the Euro is a good example. However,even when there is a hard (i.e. cannot be moved) dead-line, there is sometimes wriggle-room around the edg-es. Some parts of company accounting in Euro-zonecountries needed immediate changes when the Eurowas introduced. Other parts did not – activities aroundthe end of the company financial year may not havebeen needed to be correct on Euro day-1 – only at theend of the company financial year. So even where thereis a hard deadline, some parts may not need to be readyon the first day, whilst other parts must be available,and be seen to be working.

Right. The pre-amble is over. I want to tell you aboutmy time at “Linman Manufacturing”. In my manymonths working for them, there were 8 major imple-mentations, where significant new functionality wasintroduced. Although each implementation was sepa-rate, there were both implicit and explicit points ofinteraction; data was loaded into and extracted fromthe same database, with some common database tablesused. As time progressed, later functionality relied up-on the data that had been introduced in earlier phases,and was loaded month-on-month. I want to concentrateon the last 4 major deliverables, giving the targeteddelivery date into PROD and the actual date.


- Peter Morgan

Better to Implement LateThan Early

How Quicker can mean Slower

I mentioned that there 8 major implementations. Ofcourse, there were other times that software was promot-ed into the PROD environment. User-requested changes,software platform upgrades, and system tweaking oc-curred throughout this time frame. But if the projectteam as a whole had been asked before each phase tabledabove, for phases F and G the answer would have been‘yes’, and for phases E and H ‘no’. In the time that wasgiven, the best testing, based upon risk, had been carriedout. But there were huge swathes where the answer atbest was “we don’t know”, and some parts were clearlynot working. The architecture was basically to take mul-tiple-format files on a regular basis (daily / weekly /monthly) and to load this into a standard star schemadata warehouse, and then extract the summary informa-tion. For phase H, data had been successfully loadedmany times, but the reporting layer only developed inoutline. Now it is not until data is output that it is possi-ble to tell whether it was loaded successfully (i.e. it is fitfor purpose). And so it proved to be.

For phase E, there were immediately amendments re-quired, both to data loading routines and to the report-ing layer. Seven weeks later (and after sevenimplementations), the solution was stabilizing. Let metell you, those seven weeks were interesting. Some of theinput processes could not be run for part of that time,meaning that reported information was only partiallycorrect, and it was not until nine weeks after the date thatusers could gain any confidence in the output. January2009 was the first time that month-end reconciliation (forDecember 2008) could be attempted. That is quite a longtime after the actual implementation date.

If anything, the situation was worse with phase H, andinterestingly enough from the table above, this phasewas both ‘late’ (after the targeted delivery date) and‘early’ (implemented before it was ready). The firstweeks were spent loading a backlog of data, with 10months of data to load immediately. The reporting layerquickly highlighted ‘problems’ with 7 of the 16 datafeeds – problems that could only be rectified by remov-ing any data loaded to date, and either re-engineeringthe data creation process (before it was made available toload), or changing the data load procedures. There weretimes that it seemed data was being backed out fasterthan it was loaded – even though once a data feed wasidentified as ‘wrong’, nothing else was loaded for thatfeed until the problem was resolved. A big score-boardshowed what data had been loaded for which feeds on a

month-by-month basis. March 2010 (when data for Feb-ruary 2010 was loaded) was the first month-end withstability.

There are business reasons for ensuring that “somethingis delivered”, and that progress is seen to be made.However, for “Linman Manufacturing” the informationloaded into the Data Warehouse was for long-term stra-tegic planning, not for the day-to-day business opera-tion. There were, in one sense, no unmovable projectdeadlines.

If you have a choice, never implement a software solu-tion ‘early’. Doing so makes for a frenetic period ofactivity just after the implementation date, and some-times aborted product launches. If you take longer, it canmean that the software is available earlier. When thathappens, staff can be truly released to go onto the nextproject without constant drag-back from activities thatshould have been put to bed!

To do otherwise means that quicker is slower.


Peter Morgan is a testingprofessional who has beeninvolved in the ICT indus-try for more than 30 years,and worked in the free-lance marketplace formuch of that time. His time

has sometimes moved from testing to ‘develop-ment’, but he would add “always using the mind-set of a tester”. He is passionate about testing anda firm advocate of testing qualifications. An en-thusiastic speaker and author, Peter tries to basehis output on hands-on experience, attempting torelate fine sounding ideas back to how it willaffect Joe or Jane Tester in their everyday workinglives in the war of attrition that we call softwaretesting. He is a regular at EuroSTAR conferences,and is speaking at Belgium Testing Days in 2014 –the forth year in a row. At this time of life, Peteroffers experience, and can sometimes say whenoffered a tricky problem: “Been there, done that,and here are 2 or 3 options that may, just may,work in this situation”. He continues to learn,adding technical skills to his impressive range orhardware / software / business sector portfolio.

I don’t know if anyone has noticed but the world welive in now is different from the world we used to livein just 5 years ago. For me, the change that occurred isfar rapid than anticipated, because if we refer this to a20 or even 10 year past leap then most of the skepticswould say “Why not!, it should have changed!” but theshortness and the speed of this period leaves no roomfor being skeptic or naïve – We simply need tounderstand and adapt!

Mobility has broken free from just being a device tocommunicate into an instinct.

We are using our hand held devices in so many differentdimensions other than it was meant to be that it ischanging the complete concept of computing andnetworking. Hand held devices are now our nodes tothe clouds; we are now the new form of terminals; “TheHuman Terminals”; the value to the word “Touch” hasenhanced itself from Touch screen, and then within avery short time it again reboots itself to “HumanSenses”! It feels like a history in making where thesystems which are categorized as something purelymechanical and electrical combination of componentswith mix of complex logics and design are re-emergingto sapiens, like an event in space and time; As I see it,the systems are reaching to our senses, our beliefs, dailylives and simply becoming an extension to our selfbeing, like an implanted new body part. Gadgets arenow not need to be held in hands; they are wearable,and soon they will represent a mere extension to itsusers.

The human mind is now adapting these effects; see forexample our children; how they have adapted the useof touch screen, games and the use of smart phones.

How people are getting aware of the communicationand sharing bounds and exploits over social networks;Where to “say and share” what is now dependent on“where and how” you are registered as a user; Twitters,Facebook, LinkedIn, Instagram, Four Square, Google,pint it, flicker and so on. We are not structured anymore;we are living multiple lives under multiple scenarios.Somewhat, this is not based entirely on our choices, thisis how the environment around is “turning out to be”– and we have to live it!

We are now thinking! Or in other more adaptablesenses we are now becoming Context Driven!

These factors are effecting what we used to know as thesolutions and systems; these factors are now playing amajor role in provisions of the right solutions to the corerequirements and needs of the users. And… the samefactors are affecting how we “Test” these systems;In a way, we are now moving to an age where due tothe magnitude of the contextual effect on any scenario,generic approaches will simply fall apart and fail! – Inso many words, we need specialists! We needempowered human beings and their expertise toaddress the right issues at the right time; we need toidentify the right coverage to find the important bugs,and mind you – We need to identify ALL OF THEM!Under several scenarios and contextual situations anddue to certain trendy and cultural bonds we have notyet succeeded in creating our own identities as Testers.We are still based on the “Types” and “Approaches”scenario. Where, Testers when tend to define what theydo, start explaining the testing types and the approachesthey follow. They tend to list down several technicaltools and programming languages as skills, and not as“Tools” to support testing. In this stream of self-


- Arslan Ali

7 Types of TestersWhat is Your Identity?

discovery they forget the very scarlet thread they crossfrom being a “Tester” to “being” a Coder.Well don’t worry we have worked that out as well;

James Bach has already published twice about the types oftesters there are after receiving several queries regarding thelatter; in his recent blog post, he mentioned the following asthe testers’ types:

(I am quoting him here, rather putting in my own words –it is better that way)

(SEVEN KINDS OF TESTERS)· Automated? Manual? There is no such thing as

manual or automated testing. It’s all just testing.Testing is often supported by tools that attemptto simulate user interaction with the system.This is what people call “test automation” eventhough it is only automating a crudeapproximation of one aspect of testing. If youhave the ambition to be a one-man test team, itis extremely valuable to learn how to make yourown tools.

· Exploratory? Scripted? There is no such thing asan exploratory or scripted tester. All goodtesting is exploratory to some degree andscripted to some degree.

· Tester. This is a testing generalist who cancontribute to any test team. Sometimes called aQA analyst, QA engineer, or test engineer. Iprefer the simplicity of “tester.”

· Omega Tester. The omega tester (which Isometimes call a test jumper, after the analogyof a paratrooper) is one who can do anything.An omega tester is equipped to be the only testerin a project team, if necessary. Omega testers canlead testing, or work with a team of other testers.I am an omega tester. I aspire to be a good one.

· Performance Tester. The performance testerunderstands the mathematics and dynamics ofthe performance of large-scale systems. They usetools that create high loads and measure theperformance envelope of systems as they scaleup. Performance testers often don’t think ofthemselves as testers.

· Usability Tester. The usability tester is a bitmythical. I have met only two dedicated

usability testers in my entire career, but I haveseen more of them from a distance. A usabilitytester specializes in studying how users feelabout using and learning a product.

· Security Tester. Security testers also often don’tthink of themselves as testers. Security is anexciting, specialized form of testing that requiresthe mastery of a great many facts about a greatmany technologies.

· Testing Toolsmith. A testing Toolsmith is aprogrammer dedicated to writing andmaintaining tools that help testers. This is whata lot of people would call an “automated tester”but you better not use that term around me.

· “SDET” Software Development Engineer inTest. This means a full on programmer who doestesting using his programming skills.

On the other hand there are several characteristics as atester which can stood up and create an exclusive testeridentity; For example, boundary Testing heuristics; nowI know that several of us have used this term in our CVsand have somehow read or taken part in forums to talkabout this type testing; but how many of us haveactually bifurcated this into “Galumphing”, or“Steeplechase” or even “Leap and Creep” - I don’t thinkany of us!

OR

While discussing about product coverage we have usedthe term “San Francisco” depot instead of discussingthe missing out requirements, and blaming thedevelopers of not providing the right specs or cursingthe Implementation team of not discussing the rightrequirements; that is what common professional does!Why not stand out and use your own language?I have seen professional testers asking questions andworrying about “Testing without Requirements” or“Finding the right bugs” within a short period of time,but I have not seen any of them discussing about usingor creating their own Heuristics? Why not?

There is also no denial in discussing Quality and itscriteria, but have we ever tried to use and educateourselves with the use of the “Quality Criteria Heuristiccalled “FEW-HICCUPPS”? Try and do that!


Let us understand now.

Why can’t we move beyond being a technological freakto a Tester in a meaningful term; Does the mention ofthe very word “Heuristic” is not a mention of “Tool” inyour sense. Technology is a wonderful thing, beingtechnical is in the very genes of a computer professional;but as testers we need to identify ourselves as someonewith a niche of being a tester and being technical at thesame time; Being technical comes with the package; Sowhy not mention who you really are? “A Tester”!

Empower yourself, your language, attitude andreputation as tester. Learn Un-Learn and the re-learn!Try and improve your language as testers, differentiateyourself from the peers of Developers andImplementers. Yet sewed in and work for their supportOr I can only say this then; you are in the wrong gamebaby!

Arslan Ali has more than14 years of Experiencerelated to IT, Industry andTraining Institutions withexclusive experience of 5years in teaching variousdisciplines and projects in

IT Institution. He has worked in various roles incapacity of Software Engineering, SoftwareTester, Trainer and Quality Assurance Roles. TheMajor focus of his expertise lies in Coordination,Implementation and Testing of ERPs andCustomized Applications. He is also a trainer forContext Driven Testing for various companiesand individuals.Arslan is currently working at Sidat HyderMorshed Associates (www.sidathyder.com.pk)as a “Sr. Consultant – Information Solutions; butbeside that he is also an active founding memberof TestersTestified (www.testerstestified.com)(@testtified), Outtabox! (www.outtabox.co)(@OuttaBoxPk) and OISOL – Open IntegratedSolutions (www.oisol.com) as a trainingconsultant for Software Testing and ContextDriven Testing Workshops.You can follow him on twitter @arslan0644 andon LinkedIn atpk.linkedin.com/in/thegoodchanges/


www.stepinforum.org www.qsitglobal.com

Interaction » Insights » Opportunities

1 1 t h I n t e r n a t i o n a l S o f t w a r e Te s t i n g C o n f e r e n c e

Today’s tester is a unique being, driven, not by deadlines and mundane tasks, but by a relentless curiosity and a fascination with possibilities. Awaken the tester in you. This is where you want to be; this is NOW. Walk away with exciting ideas and quality insights from other industries. Make your mark by challenging the existing paradigm.

Most Popular Conference among Start-Ups to Fortune 1000 Companies

Excited to participate as a SPEAKER / DELEGATE / SPONSOR ? write to [email protected]

or call +91 - 80 - 4081 8888

Over 900+ delegates from India and abroad attended in STeP-IN SUMMIT in 2013

Testing NOWSTeP-IN SUMMIT 2014

Asia Pacific’s Largest Software Testing Conference

Produced by: Hosted by:

JUNE 25-27, 2014 AT BANGALORE &JUNE 18 & 19, 2014 AT PUNE & HYDERABAD

CONFERENCE PROGRAM

View Past Conferences View Past Speakers’ Gallery

www.stepinsummit.stepinforum.org

Pre-conference tutorial workshops (two parallel tracks) @ Pune, Hyderabad and BangaloreTarget Audience: Test Leads, Test Architects, Test Managers, Test Analysists and Consultants

Conference @ Bangalore Target Audience: CXOs, Vice Presidents, Directors, Business Heads; Customers and end-users of IT products and services; Test Management professionals – Managers, Test Leads, Software, Test Engineers; Quality / SEPG Management Professionals – Managers, QA Leads, Software QA Engineers; Product Development Managers; Change Agents

18, 19, 25June, 2014

26 - 27 June, 2014

http://stepinforum.org/index.php/2012-03-27-11-28-34/stepin-summits

http://www.pinterest.com/stepinforum/past-speakers-gallery/

BOOKWORM’SCORNER

March is the beginning of the appraisal season; that’s why 50% of theorganization actually wonder if they should be continuing in the or-ganization or not. That’s why I decided to crawl through “Don’t Hirethe Best”.This book is written by Abhijit Bhaduri. In case you are wondering ifit is a misguide to hiring the right team, then you are partially incor-rect. This book preaches and lets you come to your own conclusions.The book tries to sell itself by lecturing about good hires and how tomake good hires, however, it tries to draw the right line between“hiring right” and “hiring the best”. Unlike many other interviewingbooks that are filled with choc-a-bloc aptitude and problem solvingquestions, this book also tries to examine the organization culture andwhy culture fitment is important for the company. The case studieswith the standard disclaimer of having been altered to protect identi-ty is very useful and causes the reader to ponder as to how he couldhave done things better. Reasons as to why it is important to create afitment between the interviewee personality and the management ex-pectations, individual traits that need measurement, why making thewrong hire call can result in financial losses are stories talked about indepth in this book.

What did it teach me? After reading the book, I ensure that I pass ona set of good and bad candidates to my interviewing team; that varia-tion causes them to be on their toes and interview without bias. If I

send only good candidates their way, a form of bias sets in over the time which would result in all of us overlookingthe most obvious reason for not hiring the candidate.

Why should you buy this book? Because it’s not free; in addition to that, you will learn valuable lessons on how tocreate a team, why the culture fitment is important for your teams and why you need to ask questions beyond therole competencies while forming teams. When is it important to look into a resume to know what he’s capable of andwhen should you look beyond the resume? You will ask yourselves these questions if you were to read this book.I hope, after reading the book, you don’t hire the best anymore; you will hire “better than the best”.

Love,WoBo


Request a free demo by sending us an email at [email protected]

http://www.sahi.co.in

Become our fan -https://twitter.com/_sahihttp://www.facebook.com/sahi.software


Part 39


http://www.testingcircus.com/category/a-fake-testers-diary/

Random MusingsMy 39th month with this magazine; that's when I real-ized that I had inadvertently become a career counsel-lor for most of my family members who were in theirfinal year of college. Is there any scope for "Java"?Should I learn ".NET"? Is the industry scope for "SQL"or "Oracle"? With so many questions throw at me fromfamily members, I've stopped visiting any family func-tions or any event where I get to meet my extendedfamily. Quoting client-meetings and a workaholicmanager asking for meetings at mid-night, I manage toescape attending most of these functions. And thenthere is that irritating cousin of mine who keeps brag-ging of his topper credentials in schools and constantlyasks me if there's more scope for "development" or"testing"? I have a good mind to tell him that "If peoplelike you join Development teams, then testing careerwill have a lot of scope", but I dare not.

Why do I write this column? --- "When a tester logs abug and engages the developer in a sadistic discussionasking him to fix the bug, the tester is also torturinghimself on some levels; the tester's life is such that heneeds such discussions to survive his job" --- This iswhat a famous manager had to say about corporateculture for testers. Though there has been someamount of hue and cry against this column, there are afew instances that agree that this column unearthswhat lies beneath. The real thought of documentingthis diary is to try and unmask not just the bad guys,but also the horrid practices that have created faketesters all over the place. I also fell prey to fake testingsometime back and believed in the concept of sanitytesting, smoke testing, BVT testing and that all of themwere different testing types; I also thought that I couldbecome the greatest tester if I cleared all those exams.And now, I stand before you, as the FAKE TESTER.

That's what I stand today. And I hope my writings willstop creating more fake testers.

So far, in most columns, I have (unsuccessfully) tried toprovide an insight as to what's wrong with today'stesters. Most of them can be categorized as

· the inability to find good and authentic testingteachers

· how there are some people sitting in manage-ment who actually decide and state what test-ing is all about and the entire company thinksthat's what testing is all about

· the low pay of the testing engineer -- how manyof them do you think have equal opportunitiesat growth and how many of them have 40 hourworking weeks?

· And the inability of the testing team in anycompany to function as an independent body,but function as a sub-body of another manage-ment function (development or support or mar-keting or sales or .....) there is no independentperson or body leading the testing vision for thecompanies.

And at the end of a year, many testers realize thedisdain that others have for him; the disdain becomesreality. When this is pointed out to the management,the management don't try to make good the wrongthings; they do some actions that makes the tester's lifefrom bad to worse.

For a long time, the sole objective of most testing com-panies seems to be a single agenda --- bill the clientagainst the number of test cases executed; to showcase

smartness, they record and replay some of the testingand quote savings in the name of automation. Theclients cannot back out due to the legal agreementssigned at the beginning. An impact of this is that thetest results for periodic testing that's being sent toclients cease to matter; and regardless of the frequencyof app usage by customers, testing companies continuethis trend. 1 look at the testing economy would tell youthat 2.5 Billion Dollars was the money given to off-shoring companies for testing alone in the 3rd quarterof 2013. With new testing companies sprouting allover, this is set double in the next few years. And thetesting companies have also started to exploit this.With substandard test cases and haphazard testing,they make hefty profits without providing any valueby testing.

An indicator of the lack of quality can be seen by themost unimportant projects being handed to theoff-shoring testing companies; this can be seen by thevery fact that in most of the offshore company cases,not much of business revenue would be lost if theoffshore projects are closed down. Secondly, accordingto the reports of our special correspondent, in the last 5months, there has been ZERO new product launches innon-Asian markets due to testing done from here.

I am unsure if others realize this, but looks like India'stesting industry has started to face its worst crisis; mypersonal thought is that the rest of this year will begone in people realizing their mistakes and the indus-try auto-correcting itself. What do you think will hap-pen? As I mentioned last month, only time will tell.



Anna RoyzmanContext-Driven Scholar

https://twitter.com/QA_nna

Sharath ByregowdaA passionate software test professional, Principal Consultant atSQS, London in their agile group.

https://twitter.com/sharathb

Mike LylesInternational speaker, writer & Sr QA Manager in PerformanceTesting, Test Automation & Service Virtualization (20+ yrs IT).

https://twitter.com/mikelyles

Joris MeertsPublishing the latest news from the software testing weblogs.Keeper of lists. Testing historian, critic, thinker, member of#DEWT. #Capgemini

https://twitter.com/testingref

http://Twitter.com/TestingCircus

#Testers2Follow


#FollowUsAtTwitter

http://Twitter.com/TestingCircus

Testing Circus

Testing Circus is a world’s leading English language magazine for software testers

and test enthusiasts. Monthly editions since September 2010. #testing

Follow us at https://twitter.com/TestingCircus

https://twitter.com/signup?follow=TestingCircus&context=profile

Part 15

- Santhosh Tuppad

Santhosh Tuppad specializes in exploratory testing approach and his coreinterests are security, usability and accessibility amidst other quality criteria.Santhosh loves writing and he has a blog http://tuppad.com/blog. He has alsoauthored several articles and crash courses in the past. He attends conferencesand confers with testers he meets. Santhosh is known for his skills in testingand has won the uTest Top Tester of the Year 2010 apart from winning severaltesting competitions from uTest and Zappers.


Security Testing Tips

One of the challenges in security testing is, setting upthe test environment. If you are a small scale organiza-tion and there are no lots of processes, then it could beeasy for you to go ahead and setup a test environmentlike you want, but in the bigger organizations when ithas lots of processes it could be hard. There could benetwork related blockers that you may want to clear(Example: You want to download a software whichyou want to use for security testing purpose, and thatsource of download is blocked by the network of theorganization). If your test environment is a blocker foryou, then I would better recommend to not performingsecurity testing and thereby, you can at least save costs.

Isolated network of computersIt is important to have a separate network dedicated forsecurity testing. This is because, you do not want toaffect the other computers on the network if you down-load some software and it is infected with malware /adware or any malicious thing.

No website blockersThe network shouldn’t be blocking any website thatyou want to browse for learning about some hacks ordownloading any software to aid your hacking activi-ty. Let your network policy doesn’t end up blockingyour learning.

Administrator rights for computers & other devicesIn my experience, I have faced lot of blockers when thecomputer that was given to me did not provide admin-istrative rights. For changing some of the settings inorder test for security, I had to e-mail the infrastructureteam to change the settings and that consumed time.Before starting, it is important to ask for a computerwith all the rights to change / modify any setting.

Installations of software before commencementMake sure that you are ready with installations of allrequired software before you commence security test-ing activity. This is because, it will save you time ifsomething doesn’t get installed or doesn’t work prop-erly while you are testing for security. So, installing therequired software like proxy, burp suite, Wireshark,backtrack, kali linux, mantra browser, nmap and lot ofmany other tools happens without any hassles.

No code changes to be doneWhen testers are testing for security, no code changesshould be done. It is important that you have separateenvironment and not the same which developer usescheck in his / her code. And also, security testing needsto be done once functional testing is done and all thebugs reported are fixed.

DIY: Test Environment for Security Testing

http://www.facebook.com/TestingCircus


Visit our facebookpage to share.

Someone else toocould enjoy whatyou're reading

right now.

Since this is my first article for Testing Circus I’d like to thank all the wonderful contributors for sharing info,because there’s no other way to evolve in the IT industry!

Why would we need to run tests on a Remote VM?You don’t actually need to do this, but from my own experience I’ve isolated a couple of scenarios where this canprove vital:

a. Having a test-script (or suite) running on a JavaScript-rich page that needs to have complete focus all thetime. So, no touching the mouse or keyboard while this is running. If this takes 1hr to run, you’re going toget bored and/or sleepy and let’s not forget you have to justify for that particular hour to your management.

b. The VM can run on your local machine (making it not that “remote”), it can run on another machinelodged on a cloud-hosting server (e.g. RackSpace) or in your blade-center sitting neatly in your compa-ny’s basement. Unless the VM is running from your computer’s VBox you’re hogging way less resources byrunning this remotely.

c. You might be running across situations where your script runs smooth on your local MacOS / Windowsenvironment, but, for some weird reason you get weird failures on the CI machine running (in most cases) someLinux distribution. How will you debug this?

Where to start? VBox setup and installI’m going to run through the steps needed to build your local Ubuntu VM assuming you have a working HostMachine (I’m currently running Mac OS 10.8.5, but this is 99% valid for Windows as well)

a. Download and install VBox from Oracle’s websitehttps://www.virtualbox.org/wiki/Downloads

b. Download Ubuntu. I went for 13.04 64bit but you can choose whatever suits youhttp://www.ubuntu.com/download/desktop

c. Start VirtualBoxd. Click the NEW pictogram in the lefte. Populate the fields according to your need


- Mihai Sarlea

Running Remote Selenium 2WebDriver Scripts On a VM

www.TestingCircus.com March 2014 - 42-

f. From here on it’s a standard Ubuntu installation the likes of which you can find athttp://www.ubuntu.com/download/desktop/install-desktop-latest

Configure UbuntuI’m just going to run through the basics of Ubuntu configuration so you can get your VM up and running ASAP.

a. Install OpenJDK-7 and MavenSince I’m using Java to run my Selenium tests, I’ll need to install the Java Development Kit and Maven (inmy case) and I’m willing to share this nice trick useful also when setting up a fresh CI based on Ubuntu:This is the sequence that will get you a clean usable OpenJDK on a Deb machine:

apt-get remove openjdk*apt-get remove java*apt-get install openjdk-7-jdkapt-get install openjdk-7-*apt-get install maven

b. Test the install

java –versionmvn -v

c. Check the IP address and the “visibility” of your host computer

ping your_host_computer_ip


Here’s how I’ve setup the VM in VBox and Ubuntu:

Now you should be able to ping to and from the VBox installed Ubuntu VM.


Setting up SELENIUM 2 on the VMa. Open up a Terminal window and download the

Selenium Server Standalone jar

wgethttp://selenium.googlecode.com/files/selenium-server-standalone-2.35.0.jar

b. What we’re doing is creating a small Selenium 2GRID network and that’s why we need to startup a hub here:

java -jar selenium-server-standalone-2.35.0.jar-role hub

c. Open up a 2nd Termnal window and setup a(single) basic node with default config on thesame machine:

java -jar selenium-server-standalone-2.35.0.jar-role node -hubhttp://localhost:4444/grid/register

d. Find your VM’s IP address – open up anotherTerminal window:

Ifconfig -a

That’s it, all setup!

Adding the VM to your Selenium WebDriver JavaFramework

This is as easy as adding the lines bellow:switch (useThisDriver) { case FIREFOX: aDriver = newFirefoxDriver();//profile); break; caseFIREFOX_LOCAL_VM_UBUNTU: DesiredCapabilities capability =DesiredCapabilities.firefox();

aDriver = newRemoteWebDriver(newURL("http://192.168.10.26:4444/wd/hub"), capa-bility); break;DesiredCapabilities capability =DesiredCapabilities.firefox();

driver = new RemoteWebDriver(newURL("http://192.168.1.138:4444/wd/hub"),capability);

driver.findElement(…) // whatever you want to do

If you’re not running the test as part of a framework, justgo ahead and add the lines like this when starting thetest:

ConclusionI’ve tried to go through every step of setting up a niftylightweight GRID 2 on a local (or remote) Ubuntu VM.The knowledge is applicable, in much the same way toa Windows VM or any other OS you might need, for thematter. I hope it helps you get up and running withyour local *nix VM.

Selenium GRID 2 is a very cool feature that can save alot of time, but we’ll dive more into this in a later arti-cle. Till that time, feel free to check out the documenta-tion Google put up at:https://code.google.com/p/selenium/wiki/Grid2

More on the framework “Driver” class I used at point 6:http://blog.cloudtroopers.com/content/%E2%80%9Cdriver%E2%80%9D-class-selenium-2-webdriver

Mihai Șarlea is an SQA Auto‑mation Engineer forCloudTroopers.com andsound engineering enthusi-ast.Telecom Engineer by forma-tion, Mihai started flyingabove radar while working

as Simlock Security EU region Specialist for Nokiaand, gradually, shifted gears towards QA-ing forEndava Romania on financial projects. For the lastyear and a half, he's been focusing on automatedwebsite testing, continuous integration, light-weight performance scripts and all adjacent tasksat CloudTroopers.com.Outside office hours, Mihai occasionally handlesFront of House Engineering for various clubs inCluj-Napoca, Romania and volunteers to help theless fortunate artists record decent, career kick-starter, materials.Feel free to check out his LinkedIn profile:http://ro.linkedin.com/pub/mihai-sarlea/40/924/b92


Thinking what to do with your career?

We can HELP

www.TalentPlusPlus.com


http://www.talentplusplus.com

Tools Journal Testing Corner

Testing Circus in exclusivepartnership with Tools Journal(http://toolsjournal.com) presentsthe Tools Journal Testing Corner.

1. Test Automation: New Venues,New Challenges

2. SmartBear TestComplete 10Brings In Support ForAutomated Mobile Testing

3. Squish Coco Brings In SupportFor Code Coverage Of C# & TclCode

"Thank You" to all subscribers who have joined usand have been giving us some good feedback. Mostof all Thank you "Testing Circus" for providing usa good platform to shout our views .


About Us:A start up journal and aspiring socialcommunity with an aim to gain anddistribute knowledge on softwaretools and concepts in Testing, Agile,Cloud, Mobile and EnterpriseIntegration.http://www.toolsjournal.com

With over 500 products listed withquality articles, product ownerinterviews, we are moving swiftly tolaunch product editorial/userreviews, community module in next2 months.

Connect With Us

@toolsjournal

www.toolsjournal.com

www.facebook.com/toolsjournal

Thank You

Testing Circus & Tools Journal

While most of the test automation engineers are awareof the many pitfalls of test automation; there are someassociated to the automation planning & designconsiderations. There are some widely used automationframeworks, & hybrid framework tops them all.

While it’s a good practice to use hybrid framework withbenefits taken from all the conventional frameworkslike keyword, data driven etc…; the real trick in comingup with a good design is to cater to all the project needswhile keeping the maintenance cost low, and thelearning curve short.

While there is no single way to implement this solutionbut again, it’s no rocket science either. One may have toconsider various factors that automation framework hasto handle – technical & non-technical. On technical sideone may have to consider the level of complexity,abstraction-levels, so that the front-end whether it’s aGUI or an excel sheet needs to be easy to use & intuitiveat the same time.

While one should use excel file for easy readability,overusing it may lead to slow execution – contradictingthe very reason automation is used e.g. keyword-drivenframework using excel file has its benefits, it makesexecution bit slower, each read and write to-from excelfile may add up to huge delays by the time automationsuite is completed. Extensive use of excel or any otherfile format may lead to bulky automation leading toslow execution & having its own weight to pull it down.While it is good idea to give maximum control (if notall) to the executor to manipulate suite content , itrequires good judgmental call on the skill set of executor, reusability & degree of flexibility that needs to beprovided so that its benefits are maximized.

Though there have been good amount of focus givenlately on application performance, automationperformance is another area which is unexplored butsaid that, doesn’t mean we can overlook the automationexecution speeds.

· Terminology like Library-architecture etc. issomething that is intrinsic to any softwareproject whether it is automation ordevelopment, so it becomes really necessary,that we form meaningful libraryfunctions/methods, keep them light, reusableand ensure proper error handling.

With the advent of various automation tools it becomesreally important to keep a track of its advantages &disadvantages; before implementing the same.

· With loads of new browsers and devicesmushrooming in - Cross-browser & cross-devicetesting is something which can’t be avoided. Butagain each browser/device has its ownengine/mechanism to handle web elements/objects. Taking an eye off such mechanisms canlead to unexpected results, and automationperformance issues.

Many times failures in automation needn’t be becauseof scripting or application alone, browsers also havetheir own drawbacks which lead to automation scriptfailures. While various studies depict various statistics,one major thing that comes up is IE seems to have manyissues when then comes to test automation, suchstatistics need to be looked at before commencingautomation.

· Object identification/locator mechanisms alsoplay key role. Those who are well versed with


Test Automation: NewVenues, New Challenges

automation testing, don’t have to be told that,each mechanism has its own benefits, like onemay be robust mechanism, another may be easyto use e.g. name, id as locators; thereby selectinga mechanism which fits well in all the areas ispreferred.

· With client side technologies like Silverlight,HTML5, JavaScript etc. dominating the web-pages, it becomes really critical to take them intoconsideration before committing to the client.Though the pages may appear normal, the tool,browser may have issues to work with. Therebyit is always advisable to do a thorough POC,before dwelling deeper into full-fledgedautomation.

· Like in selenium, it is quite obvious to startautomation using Firefox and committing toclient on providing complete solution, whatfurther is required is the foresight into client’sfuture requirements like cross-browser testing,mobile device testing.

· It is always recommended to think of thebreadth during planning & design phase &during implementation focus on individualautomation functionality, creating a self-contained unit, preferably reusable. Anincremental approach is highly advised - eachautomation component or test script needs to bewell tested on various browsers & devices beforemoving to next level. This approach wouldavoid surprises that come with big-bangapproach. (It is advisable to avoid big-bangapproach, wherein various automationcomponents are clubbed together to fulfill therequirements. )

· With extensive usage of open sourcetechnologies & tools (like selenium), it becomesreally important to create a proper test

environment, wherein various versions of theeach tool, process or platform is welldocumented. Any change in one may lead toautomation failure.

· Integration with various other tools &/orapplication/platform compatibility also iscritical, as slightest change in a version mayimpact automation framework e.g. whichversion of testlink (test management tool)integrates with which version of selenium orwhich version of UFT supports which browser& version. Various configurations & versions –inclusive of auto-updates are to be thoughtabout & it is really important that each step istaken carefully to avoid last-minute surprises.

· Requirement to automate huge number of testcases, and to get proper ROI, Distributedtesting, is fast catching up. Various machineswith various configuration, pose a majorchallenge to distributed automation testing; asingle script to be executed for - cross-browser ,cross-platform, cross-device each having theirown intricate setup (including java version, OSversion – service packs installed etc..) – eachsystem depending on the system below it,creating a delicate balance for things to work :eventually all this can add-up to unimaginablepermutation & combinations making itmandatory to document the details.

Automation is no more limited to a single machine or asingle tool, but ranges from various tool integration toits implementation on various machines & browsers,posing new technological challenges to be faced withadvent of new versions, compatibility issues,configuration issues making it mandatory to foresee allthe major unforeseen pitfalls to be addressed & risks tobe averted or mitigated, ensuring successful automationsolution delivery.


SmartBear TestComplete has introduced support for automated testingof native applications for mobile devices and mobile web applications. So now, every single TestComplete feature for Desktop and Web Appstesting, including object recognition, scripting, automatic test execution,object spy, visualizer and reporting is applicable to automated testing ofmobile applications.

TestComplete Mobile is available now with Android support.TestComplete Mobile with iOS support is currently available in beta; thefull support will come later in Q1.

TestComplete lets you create and run automated tests on real Android phones, Android tablets as well asemulators and virtual machines. You can create a mobile test once and run it on multiple Android devices tomake sure your app works correctly on all screen sizes and orientations, without the need to root devices. Youcan also include multi-touch gestures and geo-specific information to make your tests more realistic.

TestComplete 10 also talks about an open, extensible platform that allows third party testing tools to be tied intothe TestComplete automation framework.


SmartBear TestComplete 10Brings In Support For

Automated Mobile Testing

Curious to know what is it that SmartBear offers aheadof other test automation solutions which alreadysupport Mobile devices, and the benefits TestCompletePlatform brings to the end users, I got in touch withNiclas Reimertz, VP of Product Strategy at SmartBear.

So here is more information on SmartBear TestComplete10 directly coming from Niclas Reimertz.

On what sets the solution apart from the rest, Niclassaid,

Most test automation vendor solutions fall into twocategories. Some vendors offer point solutions, whichonly support desktop, web OR mobile applicationtesting. Vendors such as Rational, appear to have nomobile application testing capability. In the case ofmobile-only vendors, they often lack reporting orfeatures such as deep parameterized data driven testing,making it necessary to integrate with the testautomation suites.

Other vendors depend on loosely-coupled integrationto 3rd party mobile testing solution from other vendorswith whom they partner. HP, for example, partnerswith Experitest to offer their customers with capabilityfor mobile application testing. In this second case,customers do not have an integrated testing experienceneeded for comprehensive testing. The TestComplete10 offering now includes testing capability for desktop,web, mobile and mobile-web applications.

SmartBear offers a single-environment solution fortesting of enterprise/desktop, web, mobile and mobile-web applications, in contrast to other vendors whoeither provide point solutions or rely on 3rd partypartnerships and loosely-coupled integrations.

Talking about the TestComplete Platform, Niclas said,

Support for integration of third party tools was therebefore also, but we’ve modularized it and made it easier

and streamlined to integrate through the SDK and thelow-level APIs. Repackaging the TestCompletecapabilities has made them more accessible and takesadvantage of the extensibility of the TestCompletePlatform. As a result users can more easily consolidatemultiple testing methods and tools into one unifiedtesting framework. For example, customers cancombine manual tests, unit tests and Selenium tests intoa single test case.

Additionally, when companies extend or customizeapplication objects, such as commercial control sets,they lose the ability to automate testing of theirapplications. The TestComplete Platform, with its SDK,makes it easier for partners and customers to developextensions that support unique testing requirements.As a result, customers can increase development agility,speed application delivery, time-to-market and lowertesting costs. Integration upstream with solutions likeSpiraTest still remain, as the TestComplete Platform isnot for managing requirements and test cases, butrather, it’s a dynamic Test Execution platform forrunning your tests.

In terms of benchmarking with HP and IBM theTestComplete 10 is ahead of times, but I came acrosssome other tools which provide seamless TestAutomation capabilities across desktop, mobile andweb apps like TestComplete now does. Froglogic SquishGUI Tester, Telerik Test Studio and Ranorex are few ofthem. Of course each comes with its own set of pros andcons and one tool cannot fit everybody’s requirements.

I am sure customers hooked on to TestComplete willreally appreciate these add-on features. WithTestComplete Framework it would now be much easyfor developers and testers to consolidate multipletesting methods and tools into one unified testingframework, flexibility which would definitely surfacein the form of higher productivity gains.


The latest release of Squish Coco, the complete, cross-platform, cross-compiler code coverage tool chain, brings in support for C# & Tclprogramming languages. With the support of four programminglanguages in one tool – C, C++, C# & Tcl, Squish Coco emerges as a singletool which glues the big gap that exists in the quality assurance effortsof heterogeneous software teams.

Squish Coco not only supports these programming languagesindividually but also includes support for hybrid applications which mixlanguages, for example making C++ calls from C# or embedding Tcl into

a C application. In addition to EMMA-XML, HTML and plain text reporting format, version 3.0 brings in supportfor Cobertura format. On top of the Cobertura format additional results can be fed into SobarQube, an openplatform to manage code quality.

Version 3.0 also sports an improved Coverage Browser and Scanner. The Code Scanner now supports clangcompiler on Mac OS X and introduces robust handling of parse errors.

Squish Coco tool suite that can run on Windows, Linux, Mac OS X, and UNIX is capable of reporting

· Tested code.

· Untested code.

· Unreachable code.

· Redundant tests.

· Optimal test execution order so as to maximize the overall test coverage for each run.

Squish Coco can be used at every stage of testing and in conjunction with every kind of testing method like unittests, automated tests & manual tests. Squish Coco has the capability to collect, compare and merge differenttests' execution reports to support advanced analysis.

Squish Coco’s licensing is user node-locked. Each Squish Coco node license includes a One Year Support &Updates Subscription which provides:

· Unlimited e-mail support.

· Access to all bugfix, minor, and major releases of Squish Coco.

Licenses can be purchased in batches of 5 user nodes starting at €3,000 for 5 User Nodes.


Squish Coco Brings InSupport For Code Coverage

Of C# & Tcl Code

taking testing to cloud

Technology

testing circus author

testing circus site

testing world

testing events

area of software testing

good testing stuffs

subject line article

automated mobile testing