soasta 51.11 (cloudtest/touchtest...

SOASTA 51.11 (TouchTest 7040.105) Apr 17, 2014

Table of Contents

SOASTA 51.11 (CloudTest/TouchTest 7040.105) ................................................1

Bugs Fixed ........................................................................................................1 CloudTest ......................................................................................................1 TouchTest .....................................................................................................1

SOASTA 51.10 (TouchTest 7040.102) .................................................................2 Bugs Fixed ........................................................................................................2

TouchTest .....................................................................................................2 SOASTA 51.09 (TouchtestTest 7040.84) .............................................................3

Bugs Fixed ........................................................................................................3 CloudTest ......................................................................................................3 TouchTest .....................................................................................................3

SOASTA 51.08 (CloudTest/TouchTest 7040.75) ..................................................4 Bugs Fixed ........................................................................................................4

CloudTest ......................................................................................................4 TouchTest .....................................................................................................5

SOASTA 51.07 (CloudTest/TouchTest 7040.58) ..................................................6 Features ............................................................................................................6

TouchTest .....................................................................................................6 Dynamic Instrumentation for Android.........................................................6

Making an Android APK TouchTestable .............................................6 Enhancements ..................................................................................................8

TouchTest .....................................................................................................8 Better MATT Out of Memory error message..............................................8

Bugs Fixed ........................................................................................................9 SOASTA 51.06 (CloudTest/TouchTest 7040.50) ................................................10

Enhancements ................................................................................................10

Copyright 2014. CloudTest is a registered trademark of SOASTA, Inc. and/or its affiliates. Other names may be trademarks of their respective owners.

TouchTest................................................................................................10 MATT appfile parameter renamed appbundle ......................................10

Bugs Fixed ......................................................................................................11 CloudTest ................................................................................................11 TouchTest................................................................................................12

SOASTA 51.05 (CloudTest/TouchTest 7040.31) ................................................13 CloudTest ................................................................................................13 TouchTest................................................................................................13

SOASTA 51.04 (CloudTest/TouchTest 7040.27) ................................................14 Bugs Fixed ......................................................................................................14

CloudTest ................................................................................................14 TouchTest................................................................................................14

SOASTA 51.03 (CloudTest/TouchTest 7040.24) ................................................15 Enhancements ................................................................................................15

TouchTest................................................................................................15 New Capture Mode option in outputScreenshot and verifyScreenshot (71485) .................................................................................................15


SOASTA 51.02 (CloudTest/TouchTest 7040.20) ................................................19 Enhancements ................................................................................................19

CloudTest ................................................................................................19 Switch to Load Mode popup opt out (73108) ........................................19


SOASTA 51.01 (CloudTest 7040.13)..................................................................22 Bugs Fixed ......................................................................................................22


CloudTest ................................................................................................22 TouchTest................................................................................................22

SOASTA 51 (CloudTest/TouchTest 7040.11).....................................................23 New Features..................................................................................................23

CloudTest ................................................................................................23 JSON extraction using JSONPath ........................................................23

JSONPath Extraction and Clip Custom Properties ...........................24 JSONPath Extraction and Validation ................................................26 JSONPath in Scripts .........................................................................27 JSONPath and Custom Property Sets ..............................................28 JSONPath Extraction in the Session Template Package Wizard......31 JSONPath in Result Details ..............................................................32

Cloud Provider Account Support for Google Compute Engine .............34 Browser Prerequisites .......................................................................34 Google Developer Prerequisites .......................................................34 Setup Google Compute Engine as a Cloud Provider ........................40

Launching Test Servers using Google Compute Engine ......................45 Creating a Grid using Google Compute Engine ................................45

Proceed with Allocated Servers............................................................48 Scripting Custom Module Java API Improvements...............................51 JavaScript API Improvements - Simulated Actions and Simulated Transactions .........................................................................................52 Dashboard Improvements ....................................................................53

Widget-on-Widget Layout..................................................................53 Drag and Drop Improvements...........................................................54 New Dark Dashboard Theme............................................................57 Using Dashboard Display Properties ................................................57

TouchTest................................................................................................59 Dynamic Instrumentation for iOS..........................................................59


Making an iOS IPA or APP TouchTestable.......................................59 Optional parameters for IPA/APP: ....................................................61 Dynamic instrumenting and Jenkins CI .............................................61 Dynamic instrumenting and TouchTestIDs .......................................61

Updated MakeAppTouchTestable Help ................................................61 Enhancements ................................................................................................63

TouchTest................................................................................................63 New TouchTest Web scheme for Xcode 4 users (75811) ....................63

Bugs Fixed ..................................................................................................64 CloudTest ................................................................................................64 TouchTest................................................................................................66

SOASTA 51.11 (CloudTest/TouchTest 7040.105)

Bugs Fixed

CloudTest 78730: Unable to pull metric data from AppDynamics Metrics from an External Data source were not retrieved as expected.

TouchTest 77255: Android application is not recording anything after being made TouchTestable and deployed [requires new TouchTestDriver] The TouchTestable app unexpectedly recorded nothing after being deployed.


1

http://cloudlink.soasta.com/t5/Knowledge-Base/Case-Requires-New-TouchTest-Driver/ba-p/15121

SOASTA 51.10 (TouchTest 7040.102)

Bugs Fixed

TouchTest 78627: Updating MATT help message The MakeAppTouchTestable -help now indicates that underscores and spaces in the launch URL are not supported by Apple, and therefore, not by MATT. 76664: Unable to input the text correctly on iOS 7 devices [requires new TouchTestDriver] This fix augments the earlier fix reported in SOASTA 51.08. 73839: Implement storeAllElementPropertyValue and storeAllElementText in Android TTW [requires new TouchTest Web] For Android TTW the storeAllElementPropertyValue and storeAllElementText have now been implemented


2



http://cloudlink.soasta.com/t5/Knowledge-Base/Case-or-Release-Requires-New-TouchTest-Web/ba-p/17571

SOASTA 51.09 (TouchtestTest 7040.84)

Bugs Fixed

CloudTest 78441: Unable to get property 'createRange' of undefined or null reference This error occurred in Internet Explorer 11 while using the Property Chooser from the Clip Editor’s lower panel Message Editor. 78385: Volume initialization fails in SOASTA 51 Initializing a volume while running SOASTA 51 would unexpectedly fail. 77624: Null pointer exception when CloudTest update is cancelled Null pointer exception when CloudTest update is cancelled. 72600: Uncaught TypeError: Cannot read property 'Global' of null; JS line 11 A null DOM check error would occur while working with a Chain container in a test clip.

TouchTest 73817: Carriage return in property set is getting stripped [requires new TouchTestDriver] A carriage return found in a property set was unexpectedly stripped.


3




Bugs Fixed

CloudTest 77602: WidgetLogics are undefined in getCompleteFilters This widget error occurred in the Composition Editor, Play tab. 76766: CloudTest cannot parse Set:Cookie The date format of the following type was not handled as expected: Thu,01-Jan-1970 00:00:00 GMT; 77576: Various date parsing errors This fix established a useful, valid default for date parsing. 50499: Error duplicating seed data object Once a seed data column was marked null, it could no longer be duplicated.


4

TouchTest 77259: After enabling services on Titanium app, a duplicate Android app is deployed to the device [requires new TouchTestDriver] This fix removes the Android node in the sample app, which enables resumed recording. 76664: Unable to input the text correctly on iOS 7 devices [requires new TouchTestDriver] Input text inside a textfield was not entered on playback, resulting in playback failure. 76188: Null Pointer Exception when Tiapp.xml doesn’t contain Android node Fixes NPE when Tiapp.xml file does not contain an <android> node 74324: outputCookies does not work with TouchTest Web [requires new TouchTest Web] This fix adds the functionality necessary to make outputCookies work as expected. 61121: storeAllElementPropertyValue behavior with one or multiple results [requires new TouchTestDriver] | [requires new TouchTest Web] This option is provided to always create an array on a Property Set, even if only one value is returned.


5









Features

TouchTest

Dynamic Instrumentation for Android This release introduces the same Dynamic Instrumentation support for Android APK files as was introduced for iOS versions 6 and 7 in SOASTA 51 (7040.11). Note: TouchTest does not support Android apps that use Custom Class Loading

Making an Android APK TouchTestable For example in prior releases, Android users would specify the full path to the Xcode project file using the project parameter. Now, MATT users can specify the APK instead:

• -apk – The full path to the compiled APK file To instrument an APK using the MakeAppTouchTestable utility, run: java -jar MakeAppTouchTestable.jar -apk <APK file path> -androidsdk <Android SDK Path> -url <CloudTest URL> -username <CloudTest user name> -password <CloudTest password>

where:

• <compiled APK> is the full path to the compiled Android APK file Note: If an Out Of Memory Exception is encountered while running MATT against

an APK, then add the following Java heap memory parameter (shown in italics below): java -Xmx2g -jar MakeAppTouchTestable.jar

APK required parameters: Choose either of the following arguments: -project or -apk.

-apk <apkfilepath> Path of the Android APK file (e.g. ~/Documents/MyApp.apk)

-androidsdk <androidsdkpath> Path of the directory where Android SDK is located.


6

http://android-developers.blogspot.com/2011/07/custom-class-loading-in-dalvik.html

APK optional parameters: The following parameters are optional for APK instrumenting (but may be required in some cases):

-keystore <keystorepath> Path of the keystore to be used to sign the APK file.

-storepass <keystorepassword>

Password of the keystore to be used to sign the APK file.

-keypass <privatekeypassword> Password of the private key (if different than the keystore password) to be used to sign the APK file.

-alias <aliasname> Alias for the key to be used to sign the APK key. Only the first 8 characters of the alias are used.

-donotconfiguremanifest Skip the Android manifest configuration step. Before using this option, please make sure that you have configured the AndroidManifest.xml file to insert changes required by TouchTest before building the APK file.

APK Example: java -jar MakeAppTouchTestable.jar -apk ~/Documents/MyApp.apk –androidsdk ~/Documents/android-sdk -url http://ctserver/concerto -username bob -password pass


7

http://ctserver/concerto

Enhancements

TouchTest

Better MATT Out of Memory error message The MakeAppTouchTestable utility will now output a more directive error message when Heap memory errors are found: JVM ran out of memory. Please increase the heap space and try again.

Example: java -Xmx1g -jar MakeAppTouchTestable.jar


8

Bugs Fixed CloudTest 77004: Uncaught TypeError: Cannot read property 'multiComboBox' of null; JS line 962

The DOM wasn't being constructed in certain circumstances.


9


Enhancements

TouchTest

MATT appfile parameter renamed appbundle The MakeAppTouchTestable utility’s -appfile parameter, which is used to dynamically instrument a compiled app file in iOS 6 and 7 versions, has been renamed to -appbundle.


10

Bugs Fixed

CloudTest

77724: Cannot select widgets that are bound to invalid result IDs Widgets with invalid result ID's couldn’t be selected. This also fixed a NPE when checking if a result ID is invalid. 77691: NPE Error is returned when attempting to use a Track's Seed Data property in a ForEach repeat on the clip This null pointer exception would occur while setting Seed Data: Property on a track that also had a repeats using ForEach. 77622: Collection Analysis does not show Base Page Avg Response Time as type “time” The Collection Analysis widget was displaying the Base Page Average Response time as a count rather than as type "time" 77595: TTLB is incorrect (too small) When processing a large HTTP response with a "Content-Length" header (as opposed to chunked transfer encoding), the time-to-last-byte metric was reported incorrectly, measuring only the time to the last byte of the first chunk (as opposed to the last byte of the entire response). 77558: Clip filters should work w/ the Collection Analysis Grouped widget The Collection Analysis (Grouped) did not recognize a Clip filter. Clip filters now work and will filter to the nearest parent clip of a collection. 77543: NullPointerException in FunctionActionEditor.isTypePassword Additional null checking has been added to detect further occurrences of this issue. 77330: Opening Waterfall Chart causes a DWR stack trace When a Waterfall chart was closed, based on timing, it was possible for us to still be referencing a variable that was null. 76246: Hide Database Password in System Properties for Direct to DB Testing System Properties unexpectedly showed the RDBMS password. Now, it is not shown.


11

74231: Monitors never had a "Today" time filter option before so it was giving an exception. In the prior release, setting a time range filter on a monitor would unexpectedly produce this error because monitors never had a "Today" time filter option. This is now handled.

TouchTest

77665: TouchTest Web Targets had # and anything after it stripped from Starting URL Choosing Record Mobile Web, and entering a Starting URL that contained a # would result in a auto-created target that didn’t have the # or any string after it. 76928: iOS TouchTest launch enter in infinite loop and crash the app [requires new TouchTestDriver] The given mobile app would enter an infinite loop on recording launch, inevitably crashing the app.


12



CloudTest 76641: CloudTest record HTTP Conductor list does not scroll in Safari browser The Recording Options, Conductors list unexpectedly failed to scroll in Safari while HTTP recording was being set up.

TouchTest 77184: TouchTest Web recording does not record correct locators and fails to playback [requires new TouchTest Web] This regression unexpectedly cause incorrect locators and subsequent playback failure in TouchTest Wed for iPod/iPhone.


13



Bugs Fixed

CloudTest 77235: Error updating CloudTest Lite to 51.03 This error and upgrade cancellation ensued after choosing Upgrade in the About box. 76280: Generated Value set as Random doesn't include the To number While creating a new clip custom property as Generated Value using Save value for analytics checked, selecting Number -> Random and From 1 To 5, the result unexpectedly failed to include the number 5. 71041: Should silently fail if we can't get disk usage due to server being unreachable Disk usage will not fail silently as expected.

TouchTest 77222: iOS App Installer support for 7.1 [requires new iOSAppInstaller] This release includes iOSAppInstaller utility support for iOS 7.1. 75403: Mobile Monitored Device data keeps moving off to the left and eventually disappears Mobile monitor charts were showing absolute Last 5 minutes instead of the result time range as expected.


14

http://cloudlink.soasta.com/t5/Knowledge-Base/Case-Requires-New-iOSAppInstaller/ba-p/26154


Enhancements TouchTest

New Capture Mode option in outputScreenshot and verifyScreenshot (71485) verifyScreenshot now has an additional Capture Mode option, which is used to capture not just the selected view and its subviews, but also all views visible within the frame of the view selected.

The Capture Mode includes the following options: View and subviews –"View and subviews” will capture screenshots of the view selected by the locator and all related subviews of that view. Use view as a frame – “Use view as a frame” will capture screenshots of the view selected by the locator and all views visible within the frame of the view selected.


15


16

Bugs Fixed

CloudTest 77036: Save path location not displayed correctly when saving new external data source Changing the location of an external data source unexpectedly failed.


17

TouchTest 76909: iOS app deledate discovery - class get initialized and should not [requires new TouchTestDriver] This fix removes all messages sent to Class deledate during load sequence. 76890: TTWebview listener enter in infinite loop [requires new TouchTestDriver] TouchTest will now process the delegate class if superclasses do not have the delegate method already. 76885: TTScroll listener enter in infinite loop [requires new TouchTestDriver] TouchTest will now process the delegate class if superclasses do not have the delegate method already. 76554: MATT unable to insert service to Android manifest with no prior service [requires new TouchTestDriver] MATT utility was unable to insert TouchTest service to AndroidManifest files that did not already contain a service. 74477: Select works for UIPickerView when UILabel is used to define the text [requires new TouchTestDriver] Since the introduction of multi-touch, TouchTest no longer uses scrollToVisible for UIPickers. Instead, users should now use "select" instead. For example, while recording, a select and a pan may both be recorded. Delete the pan and retain the select.


18








Enhancements

CloudTest

Switch to Load Mode popup opt out (73108) The Result Prompt(s) dialog box that appears when a user plays a composition in General mode that has more than the recommended number of test server instances will now include an additional opt-out field, "Remember my decisions for this composition."

Check this box to skip this prompt box on subsequent composition plays.


19

Bugs Fixed

CloudTest 74115: Handle ‘Server Hangup’ This error occurred in the Composition Editor – Play tab. 76959: SOASTA Conductor runs in an infinite loop on install Conductor install would loop endlessly without ever completing the install routine. 76947: Error while creating new folder in Grid Manager This error occurred while creating a new folder and then selecting it. 76931: Volumes snapshots are not sorted correctly in the Delete Snapshot dialog The Delete Snapshot didn’t sort by name as expected. 76604: Attempting to open a clip owned by another user in a folder that you set permissions on, returns an error This now works as expected. 76551: Null exception while replaying a clip This Results Service fault occurred while replaying a clip 76407: Show the appropriate message if the dashboard has been removed. This error occurred in the Dashboard while invoking Edit mode if the dashboard had been removed (e.g. by another user). 76321: If a default dashboard has been deleted, show the dashboard creation wizard. In the event a default dashboard has been deleted, CloudTest will now display the Dashboard Creation Wizard. 72404: a.dimensionValues is null; JS line 9711 Additional null checking of return bean values is now in place. 70483: a is undefined; JS line 573 Additional null checking has been added to detect further occurrences of this error.


20

67258: Cannot read property ‘filter’ of undefined This fix adds an undefined array entry check. 66981: Cannot call method ‘getData’ of null Additional null checking has been added to detect further occurrences of this error. 66807: Handle tenants that cannot create dashboards error This fix improves handling for tenants that cannot create dashboards (e.g. while attempting to create a dashboard from the comp play tab). 55435: Cannot call method ‘getMonitorIDs’ of undefined Additional null checking has been added to detect further occurrences of this error.

TouchTest 77029: MATT issues in SOASTA 51 The TTD Static and Dynamic libraries are now compiled in separate directories.


21

SOASTA 51.01 (CloudTest 7040.13)

Bugs Fixed

CloudTest 76939: Right-clicking a server in Results Service Status does not select it Selecting a Results Service in the Central, Results Service list other than the default one and then right-clicking selects on the default instead of the manually selected one. 76894: Google Compute Engine Cloud Provider UI needs to tell user about pop-up blocker requirements As of this release, the user will be alerted if the Popup blocker prevented the OAuth session started after choosing the JSON document from starting. 76765: Format error converting number while changing DateTime filter This "format error converting number" occurred when trying to change filters and aggregation levels in "View Analytics" (dynamic) dashboards.

TouchTest 76909: iOS app deledate discovery - class get initialized and should not [requires new TouchTestDriver] This fix removes all messages sent to Class deledate during load sequence.


22


SOASTA 51 (CloudTest/TouchTest 7040.11)

New Features

CloudTest

JSON extraction using JSONPath This release introduces support for extracting values from JSON document responses using JSONPath. This change addresses some drawbacks of the previously available modes of extraction for JSON. New JSONPath UI options are found in the Message Editor, in the Session Template Package Wizards's Clip Scanner, or while using the Session Template Wizard. Additionally, scripts can use two new JSONPath options in the getResponse method. This additional functionality resolves many limitations imposed by XML that are not limitations of JSON itself including use of whitespace, special characters, and numbers. For example, in prior releases JSON responses were converted to XML, and then XPath was used. So that a bit of JSON such as the following: {

“customerID”: 1234,

“productID”: 5678,

“quantity”: 2

}

would become (in XML): <JSON>

<customerID>1234</customerID>

<productID>5678</productID>

<quantity>2</quantity>

</JSON>

Then, while extracting using XPath: /JSON/customerID -> returns ”1234”

/JSON/productID -> returns “5678”

//quantity -> returns “2”


23

Since JSONPath is designed for JSON, and not XML, CloudTest is capable of producing much better extractions using it and now avoids the limitations found in prior releases. TIP: Users new to JSONPath may wish to refer to the following resources.

https://code.google.com/p/jsonpath/wiki/Javascript (good examples) http://jsonpath.curiousconcept.com (a handy tester) https://github.com/jayway/JsonPath (additional examples).

Examples using JSONPath: $.my key.42 // returns "a very fine number"

$..converted? // returns "true”

$.store.book[5].title

$.store.book[?(@.price < 10)].title

New JSONPath UI options are found in the Message Editor, in the Session Template Package Wizards's Clip Scanner, or while using the Session Template Wizard. Additionally, scripts can use two new JSONPath options in the getResponse method. Note: The attributes JSONPath and XPath are mutually exclusive.

JSONPath Extraction and Clip Custom Properties In a test clip based on the same sample JSON document as above (which is found here), we can manually set up a simple property set using JSONPath and the constant value $.store.book[*].author.

1. Add a Property Sets form as you would for any message where you'll refer to an existing clip custom property or where you'll create a new one.

2. Specify JSONPath as the retrieval method. When you do so, the Validations form changes to add a JSONPath field.


24

https://code.google.com/p/jsonpath/wiki/Javascript

http://jsonpath.curiousconcept.com/

https://github.com/jayway/JsonPath

http://code.google.com/p/json-path/

In this case, the JSONPath is supplied from an existing clip custom property named in the Property Path field, which has a constant value of $.store.book[*].author.


25

Note: If we were to eliminate the clip custom property and provide its value (e.g. as $.store.book[*].author) we would get a match on the message played response.

In this case, having the clip custom property allows us to manipulate the data in other ways.

JSONPath Extraction and Validation In a test clip based on the same sample JSON document as above (which is found here), we can easily set up a validation on our document using JSONPath and the Message Editor's Validations form.

3. Add the Validations form as you would for any message-based validation. 4. Specify JSON as the response type. When you do so, the Validations form

changes to add a JSONPath field. TIP: Note that not all JSON documents have a response type of JSON. Use the

format suitable for the content you are testing.

5. Specify the JSONPath to the response text to extract. For example, $.store.book[*].author.


26


6. Specify the Match Type. In the shown example, we are using Matches

exactly and then specifying the response text to extract (e.g. Nigel Rees) as a Constant expression.

JSONPath in Scripts JSONPath extraction can be performed by script using either or both of the following new response properties with JSONPath.

• RESPONSE_HTTP_BODY_WITH_JSONPATH

This constant can be passed as the "indicator" parameter in calls to the "getResponse" method.


27

For example, this script above uses a test clip based on the sample JSON document found here. Using previousItem, it gets a portion the response with JSONPath (e.g. books with a price greater than 8): var msg1 = $context.currentItem.previousItem;

// try RESPONSE_HTTP_BODY_WITH_JSONPATH

var matches = msg1.getResponse(msg1.RESPONSE_HTTP_BODY_WITH_JSONPATH, "$.store.book[?(@.price > 8)]");

• RESPONSE_TEXT_WITH_JSONPATH

This constant can be passed as the "indicator" parameter in calls to the "getResponse" method.

The same result can be achieved using the second property, which gets the same value (e.g. $.store.book[?(@.price > 8) response text using JSONPath.

For example, in a test clip using the same sample JSON document here and retrieving the same // try RESPONSE_TEXT_WITH_JSONPATH

matches = msg1.getResponse(msg1.RESPONSE_TEXT_WITH_JSONPATH, "$.store.book[?(@.price > 8)]");

JSONPath and Custom Property Sets It is possible to set up much more complex Property Sets using JSONPath extraction and a script that retrieves elements from an array created from the response using property sets. In such a test clip, a script retrieves values from an array created using the property sets (with values from the sample JSON response) shown below.


28



In propertySet0, shown below, ClipProp1 will retrieve the value of the first book's title, Sayings of the Century (e.g. $.store.book[0].title.


29

The result for ClipProp1 is shown below in the Result Details, Custom Properties section.

Similarly, in propertySet-1, Test on Played Response shows the outcome, Nigel Rees (e.g. the author of Sayings of the Century). In this custom property, the Match Options uses "all value(s)" in tandem with a slightly different JSONPath than we've used to get this author in several prior examples (e.g. in this case rather than $.store.book[*].author in tandem with Matches exactly we are using $.store.book[0].author—the author for the first book in the given array).


30

In propertySet-2, we get the same value via the JSONPath $.store.book[*].author using yet another method—via the specific index of the item in the array as our Match Option (e.g. index 2 as shown below).

JSONPath Extraction in the Session Template Package Wizard The Session Template Package Wizard now supports JSONPath as a Value Retrieval Method and will designate that method, where appropriate. This value retrieval method can also be manually selected where JSONPath is a relevant option.


31

Where JSONPath is the assigned Value Retrieval Method, click the Edit icon to access the JSONPath Parser.

This dialog box shows the precise JSONPath string with the suggested extraction highlighted in orange. Similarly, JSONPath can be specified as the Retrieval Method while using the Session Template Wizard on selected text.

JSONPath in Result Details JSON Response Text is now presented within the given XML Response Text tag in JSONPath format, whereas in prior releases an XML representation of the same output was presented in the Result Details, Output section.


32

JSON events in the Result Details widget are also now updated.


33

Cloud Provider Account Support for Google Compute Engine As of this release, CloudTest Pro users can create Cloud Provider Accounts for the Google Compute Engine cloud provider. The Google Compute Engine provides data centers in the U.S. and Europe and uses Google's OAuth 2.0 standard for authentication.

Browser Prerequisites Enable browser popups for the following URLs using the recommended steps for your browser:

• CloudTest URL (the CloudTest URL is the same URL you use to login to CloudTest) *.google.com

Google Developer Prerequisites To get started with Google Compute Engine in CloudTest, you'll first need to take care of some prerequisites in your Google Developer Account. Note: The following instructions presume an existing Google Developer ID with

billing enabled so that network services such as Compute Engine are enabled.

TIP: Your Google account isn't charged unless you create a server instance (the following steps do not include creating a server instance).

• First, you'll need a project. If you don't have one, create it now. 1. Login to the Google Developer Console and then click Create Project.


34

https://console.developers.google.com/

The New Project box appears. 2. In the New Project box, set the (editable) Project ID and note it for later

use. Check the Terms of Service box to continue and click Create to continue.

3. Click to open your project (e.g. My Project) and locate the Networks node.

TIP: Billing services must be enabled for the Compute Engine node to appear in

this project.

4. Click Compute Engine, and then in the expanded node, click Networks.


35

5. Click to open the 'default' network page.

Note: CloudTest supports only the Google Compute Engine default network at this

time.

6. On the 'default' network page, locate the Firewalls section, and click the Create new link text.

Note: CloudTest requires inbound ports 80 and 443 in order to deploy CloudTest

grids.


36

7. In the Create a new firewall box, give the new firewall a name and enter the following text into the Protocols & ports box (shown below): tcp:80;tcp:443

8. Click Create to complete firewall configuration for the 'default' network.

The new firewall appears in the list of firewalls (above). This completes the network requirements for Cloud Provider Account configuration using Google Compute Engine.


37

• Next, we will register a new Client ID and then download its corresponding JSON file to use in CloudTest. 1. Click APIs & auth, and then click Credentials (once again, in the Google

Developer Console).

2. Click the Create New Client ID button.


38

In the Create Client ID box; select Application type 'Installed application'.

3. Click Create Client ID to continue.

4. In the new Client ID for native application section, click Download JSON,

choose the download folder, and note its location. You'll be asked to provide this file in later steps.

Note: Previously-configured credentials can be used as long as they are of the type “native application”


39

Setup Google Compute Engine as a Cloud Provider Use the following steps to enter your valid Google Compute Engine credentials as a CloudTest Pro Cloud Provider Account.

1. Select Central > Cloud Provider Accounts and then click New to launch the Cloud Provider Account dialog box.

2. Enter a name for the new cloud provider account. This name is used in locations and also appears in the Cloud Provider Account drop-down in the Grid Manager.

3. Change the “Type” drop-down to “Google Compute Engine”.

4. Enter the Google Compute Engine project ID.

Note: All Google Compute Engine resources belong to a project. Specify that project ID noted earlier (i.e. ren-stimpy-941).


40

https://developers.google.com/compute/docs/projects

5. Choose the Google Compute Engine JSON file that corresponds to this Client ID (e.g. the one downloaded from the Google Compute Engine API above).

After doing so, the Cloud Provider Account box refreshes and the Authentication Code field appears.


41

At the same time, the Google Compute Engine OAuth "Request for Permission" popup window appears. 5. If you're not signed into Google, sign in now. The page below appears.

Complete the sign-in by clicking Accept.

TIP: As noted earlier, browser popups must be enabled for this to work.

After clicking Accept, the landing page above appears. This page contains the Authentication Code you'll need to enter into the Cloud Provider Account box, Authentication Code field. 6. Copy the Authentication code and paste it into the Cloud Provider Account

box.


42

7. Optionally, configure an access control list in the Cloud Provider Account

box in CloudTest. Refer to Cloud Provider Accounts for additional Access Control List steps.

8. Click OK to complete configuration of this Cloud Provider Account. The completed item appears in the Cloud Provider Accounts list in Central.


43

http://cloudlink.soasta.com/t5/Knowledge-Base/Cloud-Provider-Accounts-CloudTest-Pro/ba-p/468

When the Cloud Provider Account is saved, a new set of locations for Google Compute Engine will be automatically created, if they do not already exist.

One or more locations will be specified during Grid Manager configuration of grids. For more information about Locations, refer to Using Locations.


44

http://soasta.com/findouthow/admin/locations.html

Launching Test Servers using Google Compute Engine Once a Google Compute Engine Cloud Provider Account has been created for use with CloudTest Pro, you are ready to create a grid that will provision servers via that cloud vendor.

Creating a Grid using Google Compute Engine 1. Select Central > Grids and then click New. The Grid Manager appears with

Step 1 General displayed. 2. Enter a Grid Name, and optionally, enter a description and a repository

location for the new grid, and then click the right arrow. The Step 2 Server Instances page appears.

3. Change the “Location” drop-down to one of defined Google Compute

Engine locations. For example, Google Compute Engine US Central. US Central and Europe West are the only two Google Compute Engine locations offered at this time.

4. Specify the number of Test Server instances, and then optionally, define Results Server Instances. For most cases, the default “One for every 50 Test Server” instances” is acceptable for result servers.

5. By default, Google Compute Engine servers will be launched using Large instances. Extra-Large instances can also be selected via the Server Class dropdown.

• Large” is translated to the “n1-standard-2” machine type.

• “Extra-Large” is translated to “n1-standard-4”. 6. If you wish to combine the Google Compute Engine servers with servers

from other cloud providers, click the Add a Location box. Otherwise, click the right arrow icon to proceed.


45

The Step 3 Summary and Deploy Tear Down page appears. Click the “Deploy Instances” button to begin launching servers.

Note: CloudTest Pro users may see an additional Grid Monitor Database field for

those configurations where more than one Results Database is available. For more about using and launching grids, refer to Managing Grids.

Note: At the time of the first grid deployment using your Google Compute Engine

account, CloudTest creates an image that will appear in your in the Google


46

http://soasta.com/findouthow/admin/managing_grids.html

Developer Console's list of Images. No additional costs are associated with this image.


47

Proceed with Allocated Servers

CloudTest Grid Manager users can now proceed with the servers already allocated in Grid Manager's Summary and Deploy / Tear down page any time after the Deploy Instances button is clicked. When the Grid Manager is in the deployment state, the Proceed with X of X servers button is active (shown below).

This can be done in any cases where all of the requested servers have not yet been allocated (e.g. during the middle of the allocation process), or most especially, in cases where allocation is hung or delayed for any reason.


48

While the Grid Manager is in this state, click Proceed with X of X while it's active displays these subsequent options.

• Terminate Pending – Choose this option to terminate the remainder of the

server allocation tasks. When you do so, the Proceed window disappears and the grid is ready to be used with the number of servers indicated by the Proceed button (i.e. 4 of 11, etc.)

• Keep Allocating Pending – Choose this option to resume the server allocation for the remaining servers.

If Proceed was clicked, and then Terminate Pending, the Proceed window will go away. Then the following behavior will occur:

1. Any already-allocated servers will “proceed” through the usual grid process, including server-health checking.

2. Any servers that were still pending will be terminated.


49

The Grid Ready icon will display only once (1) and (2) are complete.

For example, in the screenshot above 50 of 100 Test Server instances and 2 Result Service instances were allocated prior to clicking Proceed, resulting in a total of 52 server instances and Grid Ready status.


50

Scripting Custom Module Java API Improvements This release improves upon the earlier iteration of Custom Java Modules through a Java API that makes it easier for CloudTest users to do asynchronous communication, such as found in full-duplex protocols such as the WebSocket protocol, in which both the server and client can push data simultaneously. The Scripting Custom Module Java API allows the Java code inside the Custom Module to interact directly with the playing Composition’s environment without having to return to the calling JavaScript. The latest JavaDoc for the Scripting Custom Module Java API can be found here. Note: CloudTest now provides both a JavaScript API, which is used to write

CloudTest scripts, and the entirely different Custom Module Java API. These two APIs should not be confused, even though they may be used simultaneously and have similar capabilities. Unlike CloudTest scripts, which are written in JavaScript, custom modules are written in Java. The provided Custom Module Java API is not a generic API, and cannot be used to interact with the CloudTest product. The Custom Module Java API only works when it’s executed inside of a Script running in a Composition.

By deploying Java code in CloudTest Custom Java Modules using the provided API, and then calling that module(s) from within a Composition, CloudTest users can accomplish tasks that cannot currently be done in a "standard" Composition. An example duplex session is shown below.


51

http://cdn.soasta.com/productresource/download/JavaDoc/latest/doc/index.html

JavaScript API Improvements - Simulated Actions and Simulated Transactions This release also includes improvements to the JavaScript API used by CloudTest Scripts, which is described here. In prior releases, CloudTest script authors had the capability to record a “simulated Action” into the Result by using the recordActionCompletion method. In this release, an additional ability to create “simulated Transactions" in the Result has been added. Users can then place simulated actions into simulated Transactions. In addition, CloudTest now allows the parent container to be specified for both simulated actions and simulated Transactions (in prior releases actions always went into the container of the Script). This release includes two new methods in support of this new feature: recordTransactionStart(transaction)

Records the start of a simulated Transaction into the Result. It will be recorded into the Result similar to a standard Transaction object that would appear inside of a Clip. The Transaction will be empty unless simulated actions are recorded inside of it using the recordActionCompletion method, and/or nested Transactions are created inside of it using this method. Code that is adding simulated actions to the Result can also create Transactions and place the actions inside of those Transactions, to control how things appear and how statistics are computed in the Result. This function has one required parameter. It can be any type of JavaScript object, although usually it would just be a plain Object. The object is expected to have certain properties with certain pre-defined names. The object is expected to have property values that describe the Transaction to be started. recordTransactionCompletion(transaction) Records the completion of a simulated Transaction that was started previously via the recordTransactionCompletion method. This function has one required parameter. It can be any type of JavaScript object, although usually it would just be a plain Object. The object is expected to have certain properties with certain pre-defined names. The object is expected to have property values that describe the Transaction to be completed.


52

http://cdn.soasta.com/productresource/download/jsDoc/latest/index.html

http://cdn.soasta.com/productresource/download/jsDoc/latest/symbols/Result.html#recordActionCompletion

Dashboard Improvements This release introduces numerous Dashboard user interface improvements, including a Dark theme, improved visual feedback in drag and drop actions on the Dashboard, as well as Widget-on-Widget layout, which provides the ability to float widgets on top of other widgets.

Widget-on-Widget Layout Transparent Widget-on-Widget widgets are now implemented. The process of adding a widget to float atop another widget is similar to that of the Combine/Correlate process in earlier releases. When the widget to float is dropped onto the host widget, the layout box shown below appears. Click Place on Top to accept the widget-on-widget layout.

If the two widgets also have a potential Combine/Correlate relationship, those options are shown as well.

• Choose Place on Top to use Widget-on-Widget layout.

Floated widgets have the following characteristics:

• The Title-Bars of ALL Widget-on-Widget widgets are transparent, and fade in on-hover.


53

• The Backgrounds of ALL Widget-on-Widget widget containers are 100% transparent.

• All Widget containers have their borders removed. • Borders are separately applied to the Widget title, and Canvas (where

widget is drawn).

• The default Background color for charts is set to 60% opaque white. • The default setting for Tear-Offs is 100% transparent background. • On Hover of Tear-Off Widget, the Widget background is set to dark

transparent blue, and has visible borders.

Drag and Drop Improvements This release includes a number of improvements to visual feedback during user drag-and-drop actions in Dashboards. Among the drag and drop improvements in this release are:


54

Landing Pad Heights - The Landing Pad elements are now dynamically sized and shown at the bottom of each column on the drag of a new Widget.

Insert Between Widgets – In prior releases, when you inserted between Widgets, the full drop area was not shown, and background color offset was not helpful. Now, that drop area is a mid-grey to match that of the Landing Pad elements, with the same corner-radius.


55

Insertion Lines – Insertion lines are now animated to clearly show the area that a widget will occupy if it is dropped on the given target.

Insert Above Top Widget: The insertion line above the top widget also has improved visual feedback. When the user is in the zone to insert a widget the insertion line is visible. This line disappears if the user is too far up on the Dashboard title bar, which is not a valid drop insertion point.

Dark Theme: In dark theme, the insertion line is blue.


56

New Dark Dashboard Theme Custom dashboards now include a platform-wide 'Dark' Dashboard Theme, which can be found in the Dashboard Edit Panel, Properties tab. This new theme controls the widget color for all dashboard widgets.

• Once in Dashboard Edit Mode, select the Dark Dashboard Theme as

shown above. Click Apply to finalize your selection to all widgets in the display.

Note: As always, entering dashboard edit mode from the default CloudTest Dashboard will prompt you to create a new, custom dashboard.

Using Dashboard Display Properties The Dashboard, Properties tab also includes additional dashboard display properties that can be used with either the Light or Dark Theme.

• Background Color - Background Color can be separately set to offset the widget color. Choose from among the presets or enter a new color using hex-code style (i.e. #FFFFFF for white, etcetera).

• Background Image – A Default Dashboard Texture image is provided. Use the Import button to utilize a custom image of your choosing. The background image is Tiled (by default) or can be Stretched). Click Import if that image is not also already in the repository.

• Widget Padding - increase internal and external space around the widget border

• Link Chart Timelines - Chart timelines are linked across the dashboard by default, unchecking this will unlink them

• Enable widget animations - Turn chart animations associated with real-time deltas on or off.

• Hide widgets when not playing – This setting applies to CloudTest and TouchTest only.

• Column widths - Set a column width in pixels


57

In the screenshot below, the Default Dashboard Texture image is applied with 50% texture opacity selected.

• Import – Use the Import button to launch the select a new background

image to import. When you do so, the New Image dialog box appears for you to choose the image file to import. Dashboard background images should be 72 pixels wide by 27 pixels tall. The Maximum file size is 5,120 bytes.

• Provide an Image Name and use Choose File to select the image to

upload. The Image Name will appear in the Background Image dropdown.


58

TouchTest

Dynamic Instrumentation for iOS In prior releases, making an iOS app TouchTestable was done using static app instrumentation techniques via the MakeAppTouchTestable utility –project argument. In this release, additional dynamic instrumentation techniques provide support for making IPA and APP files TouchTestable. With static app instrumentation in prior releases, iOS and Android projects could be made TouchTestable. That is, TouchTest users could use the MATT utility only on iOS project files and Android project folders. Now, compiled apps in iOS versions 6 and 7—can be made TouchTestable using these new MakeAppTouchTestable (MATT) dynamic instrumentation techniques.

Making an iOS IPA or APP TouchTestable In prior releases, iOS developers would specify a command including the full path to the Xcode project file using the –project argument. Now, MATT users can dynamically instrument the file by specifying either the project file, or one of the two following flags:

• -ipa – The full path to the compiled IPA file

• -appfile – The full path to the compiled APP file To instrument an IPA using the MakeAppTouchTestable utility, run: java -jar MakeAppTouchTestable.jar -ipa <compiled IPA> -url <CloudTest URL> -username <CloudTest user name> -password <CloudTest password>

where:

• <compiled IPA> is the full path to the compiled iOS IPA file Note: One and only one of the project file, IPA, or app file can be specified when

using the MATT utility.

To dynamically instrument an APP using the MakeAppTouchTestable utility, run: java -jar MakeAppTouchTestable.jar -appfile <compiled APP > -url <CloudTest URL> -username <CloudTest user name> -password <CloudTest password>

where:

• <compiled APP > is the full path to the compiled iOS .app file


59

Note: One and only one of the project file, IPA, or app file can be specified when using the MATT utility.


60

Optional parameters for IPA/APP: The following optional parameters are also available for dynamic instrumentation of IPA files (in some cases their use may be necessary to succeed):

-signingidentity <signingidentityname> The name of the signing identity to be used for codesigning the application (e.g. "iOS Distribution: Developer Name").

-provisioningprofile <profilepath>

Path of the Provisioning profile to be used for building IPA file.

-entitlementsfile <entitlementsfilepath>

Path of the entitlements file to be used for codesigning the application.

Dynamic instrumenting and Jenkins CI The workflow necessary to incorporate a dynamically instrumented mobile app into continuous integration environments including Jenkins and Bamboo will be different than the workflow for static instrumentation using the -project method. In prior releases, and in environments where the static instrumentation method is used, the CI workflow requires that the MATT utility be applied prior to building the app. Now, the CI workflow for dynamically instrumented apps naturally requires that the app be built prior to when the MATT utility is applied to it.

Dynamic instrumenting and TouchTestIDs In the prior release, users were advised to add the #ifdef TOUCHTESTDRIVER statement to the Xcode project as a means of enabling TouchTest IDs (in tandem with writing the individual TouchTest identifiers into project code) using static instrumentation. TouchTest users who intend to use dynamic instrumentation solely will no longer need the #ifdef TOUCHTESTDRIVER and it should be removed from existing projects that will use only dynamic instrumentation (or, if both methods are used, it may be necessary to maintain different projects for each type of instrumentation).

Updated MakeAppTouchTestable Help This release features greatly improved command line help for the MakeAppTouchTestable utility. The complete reference now includes all of the following additional content for IPA and APP dynamic instrumenting. Usage: java -jar MakeAppTouchTestable.jar -project PATH [OPTIONS]

Required parameters:

project <projectpath> Path of the Xcode project directory (e.g. ~/Documents/MyApp/MyApp.xcodeproj)


61

target <name> the name of the Xcode target to modify

url <url> the CloudTest URL (e.g. http://ctserver/concerto)

username <username> The CloudTest user name

password <password> the CloudTest password

Example: java -jar MakeAppTouchTestable.jar -project ~/Documents/MyApp/MyApp.xcodeproj -target MyApp -url http://ctserver/concerto -username bob -password pass

IPA required parameters: Choose only one of the following arguments: -project,-ipa, or -appfile. Note: IPA and APP instrumenting is unavailable for iOS 5 apps.

ipa <ipafilepath> Path of the iOS IPA file (e.g. ~/Documents/MyApp.ipa)

appfile <appfilepath> Path of the iOS app file (e.g. ~/Documents/MyApp.app).

IPA optional parameters: The following parameters are optional for IPA instrumenting (but may be required in some cases):

-signingidentity <signingidentityname>

The name of the signing identity to be used for codesigning the application (e.g. "iOS Distribution: Developer Name").

-provisioningprofile <profilepath>

Path of the Provisioning profile to be used for building IPA file.

-entitlementsfile <entitlementsfilepath>

Path of the entitlements file to be used for codesigning the application.

IPA Example: java -jar MakeAppTouchTestable.jar -ipa ~/Documents/MyApp.ipa -url http://ctserver/concerto -username bob -password pass


62




Enhancements

TouchTest

New TouchTest Web scheme for Xcode 4 users (75811) A new Xcode scheme, TouchTestWeb-Xcode4, had been added in this release to accommodate Xcode users still developing in this earlier Xcode version.

Xcode 4 users will need to use this scheme to build TouchTest Web projects in Xcode 4.x versions. Xcode 5 users should continue to use the ‘TouchTestWeb’ scheme.


63

Bugs Fixed

CloudTest 76670: Zoom into less than 15 minutes was not working in all cases Bringing up a chart that showed more than 15 minutes of data--displayed in absolute time—and then zooming into a section of less than 15 minutes would result in this error. 76650: Incorrect wording in confirmation dialog when deleting a Global Property List The confirmation message that appears when you delete a Global Property List in Central, there is a confirmation message now refers to "Global Property List" as expected (rather than the object's internal name). 76617: Assigning reserved IP fails when replacing a failed server When using reserved IPs, if a server fails the health check and has to be replaced, we fail to assign EIP to the replacement. Reserved IP "ec2-50-16-74-160.compute-1.amazonaws.com" could not be associated with the server "i-b4330c95": Amazon EC2 failed to process the request (Invalid value 'ec2-50-16-74-160.compute-1.amazonaws.com' for PublicIp. Not a valid IPv4 address.).

76282: Containers cut off in list mode If the user opened a container where the List view workspace was still short enough that there isn't a scrollbar required, and then a second container forces the workspace to scroll—then the first container would be cut off. 76076: jQuery being loaded twice in dashboard widgets jQuery is now included only the once in Central but appears twice in the head of at least the dashboard widgets. 75002: Exception: No Results database is set as default If the previous default RSDB was not the default, setting the current RSDB to default failed with this exception. 74955: Invalid Repository URL error Occasionally when a system is first brought up on Amazon EC2, an attempt to delete a result or view a dashboard could result in the error "Invalid Repository URL."


64

74654: A result with ID '1813434' does not exist in the Results Server database This error likely occurred due to an unavailable RSDB. 74518: Deletion of RSDB while deployed is no longer allowed

Attempting to delete a deployed Results Service database will now inform the user an in use resource cannot be deleted. 74488: Opening a dashboard from Central should connect to a playing composition In prior releases, if the users opened a number of stand-alone dashboards in Central, each Dashboard would then need to be manually connected to a playing composition, requiring a lengthy series of clicks for each dashboard. Now, on opening a dashboard from Central > Dashboards, CloudTest will check to see if there are compositions running. If there is one or more running, a prompt dialog appears to let the user choose the composition to connect. 73872: Waterfall chart error while viewing a non-running Results Database

The Waterfall chart would display an error if any attempt was made to view a result that was located on a Results Database which was not running. 72451: Substring parser viewer window doesn't auto-scroll to highlighted data

After running Scan Clip in the Session Template Package Wizard, when a user opened the SubString Parser window, that window would not scroll to the highlighted (i.e. orange) text as expected. 65955: Session Template Wizard error in CloudTest Lite This error occurred in the wizard while or after running Scan Clip. 56179: Exception: ERROR: relation "containerdimension76" does not exist When a filter was opened for a result that has since been deleted, a java exception was displayed.


65

TouchTest 76583: Better handling on iOSAppInstaller installation error In prior releases, in cases where the app install failed, the iOSAppInstaller utility wouldn't fail without alerting the user—now such failures will present the user with an additional option. This also fixes case 76491. 76416: Composition mobile test failing This failure occurred because an iOS 6 select was recorded with a text-based option locator. This issue was resolved when the original code text-based locators when iOS < 7 were restored. 76392: Recording webTypePassword is playing back the encrypted value Unencrypted password values were unexpectedly recorded in some cases. 76309: TouchTest Web select action crashes on iOS6 iPod [requires new TouchTest Web] The TouchTest Web select action wasn't working on iOS6. 75702: Hybrid test app shuts down after first action done on screen [requires new TouchTestDriver] A given hybrid app would crash on startup after being made TouchTestable. 75687: TouchTest Web Android quits in the middle of recording [requires new TouchTestDriver] TouchTest Web no longer injects JavaScript scripts if the page is not reloaded because it sometimes causes this segfault crash. 74898: Reduce frequency at which TouchTest agent/page authenticates In prior releases, the TouchTest agent (Android) and webpage (iOS), the Appcelerator dashboard was getting requests every second, which due to the large number of agents in use resulted in a significant performance hit on the dashboard server. Now, the TouchTest agent will re-use the Login session for the current test session. 66409: MakeAppTouchTestable should not inject code into app.js [requires new TouchTestDriver] In prior releases, MakeAppTouchTestable (MATT) would inject TouchTest code into the app.js for the Titanium project. Now, MATT utilizes the Titanium SDK functionality to run equivalent custom native code within the module before the application starts.


66







