soasta cloudtest release notes build 6806.68 table of...

61
SOASTA CloudTestRelease Notes Build 6806.68 Dec 12, 2013 Table of Contents Build 6806.68 ........................................................................................................ 1 Bugs Fixed ........................................................................................................ 1 CloudTest ...................................................................................................... 1 mPulse .......................................................................................................... 2 Build 6806.65 ........................................................................................................ 3 Bugs Fixed ........................................................................................................ 3 CloudTest ...................................................................................................... 3 mPulse .......................................................................................................... 3 Build 6806.63 ........................................................................................................ 4 Bugs Fixed ........................................................................................................ 4 CloudTest ...................................................................................................... 4 TouchTest ..................................................................................................... 5 Build 6806.54 ........................................................................................................ 6 Bugs Fixed ........................................................................................................ 6 CloudTest ...................................................................................................... 6 TouchTest ..................................................................................................... 8 mPulse .......................................................................................................... 8 Build 6806.50 ........................................................................................................ 9 Bugs Fixed ........................................................................................................ 9 CloudTest ...................................................................................................... 9 TouchTest ................................................................................................... 11 mPulse ........................................................................................................ 12 Build 6806.34 ...................................................................................................... 13 Bugs Fixed ...................................................................................................... 13 CloudTest .................................................................................................... 13 TouchTest ................................................................................................... 15

Upload: others

Post on 21-Mar-2021

8 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: SOASTA CloudTest Release Notes Build 6806.68 Table of ...cdn.soasta.com/productresource/download/SOASTA_CloudTest...SOASTA CloudTest Release Notes Build 6806.68 Dec 12, 2013 Table

SOASTA CloudTest™ Release Notes Build 6806.68 Dec 12, 2013

Table of Contents

Build 6806.68........................................................................................................1 

Bugs Fixed ........................................................................................................1 CloudTest ......................................................................................................1 mPulse ..........................................................................................................2 

Build 6806.65........................................................................................................3 Bugs Fixed ........................................................................................................3 

CloudTest ......................................................................................................3 mPulse ..........................................................................................................3 

Build 6806.63........................................................................................................4 Bugs Fixed ........................................................................................................4 

CloudTest ......................................................................................................4 TouchTest .....................................................................................................5 

Build 6806.54........................................................................................................6 Bugs Fixed ........................................................................................................6 

CloudTest ......................................................................................................6 TouchTest .....................................................................................................8 mPulse ..........................................................................................................8 

Build 6806.50........................................................................................................9 Bugs Fixed ........................................................................................................9 

CloudTest ......................................................................................................9 TouchTest ...................................................................................................11 mPulse ........................................................................................................12 

Build 6806.34......................................................................................................13 Bugs Fixed ......................................................................................................13 

CloudTest ....................................................................................................13 TouchTest ...................................................................................................15 

Page 2: SOASTA CloudTest Release Notes Build 6806.68 Table of ...cdn.soasta.com/productresource/download/SOASTA_CloudTest...SOASTA CloudTest Release Notes Build 6806.68 Dec 12, 2013 Table

Copyright 2012. CloudTest is a registered trademark of SOASTA, Inc. and/or its affiliates. Other names may be trademarks of their respective owners.

mPulse ........................................................................................................16 Build 6806.29......................................................................................................17 

New Features..................................................................................................17 CloudTest ....................................................................................................17 

New CloudTest Style for Popups, Dialog Boxes, and Wizards ................17 Faster load time for Central screens........................................................19 Faster load time for Clip Editor and no virtualized scrolling .....................19 Pause/Resume HTTP Recording in the Clip Editor .................................19 Add host name to Exclude list when removing an HTTP target ...............20 Support for Stored Procedures/Functions................................................21 Duration Timespan Filter .........................................................................22 Series Filtering.........................................................................................24 

mPulse ........................................................................................................26 Time Windows .........................................................................................26 

Defining a Time Window using the Filter Toolbar .................................28 Defining a Dashboard Time Window using the Edit Panel....................29 

Percentile Filter Attribute .........................................................................31 Enhancements ................................................................................................32 

CloudTest ....................................................................................................32 Stopping a Script or ISSE ........................................................................32 Clip, Track, or Band Doesn't Get Focus in Results Details Dashboard....32 Better error message when there are too many components in the Results Details......................................................................................................32 Pending timeout should be ignored for Cloud Providers that don't support early termination ......................................................................................32 Database Monitoring Improvements ........................................................33 

Setting Up Monitoring Server Groups and Monitors for Databases ......33 New Database Action XML and JSON Format Outputs for Supporting Multiple Results .......................................................................................36 Stored Procedures in the Result Details Dashboard................................37 

Page 3: SOASTA CloudTest Release Notes Build 6806.68 Table of ...cdn.soasta.com/productresource/download/SOASTA_CloudTest...SOASTA CloudTest Release Notes Build 6806.68 Dec 12, 2013 Table

Copyright 2012. CloudTest is a registered trademark of SOASTA, Inc. and/or its affiliates. Other names may be trademarks of their respective owners.

New Database Monitoring Widgets..........................................................39 Track Filtering Improvements ..................................................................42 Error Analysis for App Actions and Browser Actions ...............................43 

App Actions in the Error Analysis Widget .............................................43 Browser Actions in the Error Analysis Widget.......................................44 

mPulse ........................................................................................................45 Defining a Dashboard Percentile using the Edit Panel .........................45 

New 'All' default option in A/B Test filter ..................................................46 Allow CSV exporting of data from the Geography widget. .......................48 Other Analytic Changes in This Release .................................................48 TouchTest Web User-Agent Setting (iOS Only).......................................49 

Bugs Fixed ..................................................................................................51 CloudTest Bugs Fixed .............................................................................51 TouchTest Bugs Fixed.............................................................................55 mPulse Bugs Fixed..................................................................................57 

Page 4: SOASTA CloudTest Release Notes Build 6806.68 Table of ...cdn.soasta.com/productresource/download/SOASTA_CloudTest...SOASTA CloudTest Release Notes Build 6806.68 Dec 12, 2013 Table

Build 6806.68

Bugs Fixed

CloudTest 72295: Virtual Users Split by Location not displaying data when split by location is checked The Virtual Users - Split by Location widget was not displaying any data on initial start unless the user changed it to combine the virtual users, and then changed it back again to Split by Location. 72248: IndexOutOfBounds Exception This check-in augments the earlier fix reported for this case in SOASTA 48.04. 71228: Azure host names cannot be resolved consistently Sometimes Azure host names would resolve to 0.0.0.0 causing concomitant issues. 69986: Using EC2 internal host name does not work across accounts in VPC Internal host name conflicts emerged in an Amazon EC2 environment that utilized VPC. Since CloudTest up to this point has assumed that EC2 instances within the same region can communicate using their internal host name, no matter what (even if in different AWS accounts), changes to VPC disrupted this assumption. With VPC, the internal host name now only works within the same virtual network. Now, CloudTest's EC2 server list configurations will always use the external host name.

Copyright 2013. CloudTest is a registered trademark of SOASTA, Inc. and/or its affiliates. Other names may be trademarks of their respective owners.

1

Page 5: SOASTA CloudTest Release Notes Build 6806.68 Table of ...cdn.soasta.com/productresource/download/SOASTA_CloudTest...SOASTA CloudTest Release Notes Build 6806.68 Dec 12, 2013 Table

TouchTest 72620: Null pointer when starting a test that has no connected device This exception would occur in some cases when starting a test composition for which no connected mobile device existed. 72568: Device agent chooser UI problem A layout issue caused the Device Agent Chooser UI to be misaligned. 71095: Unable to start monitoring while playing a mobile composition Mobile monitoring unexpectedly failed to start for a given mobile test compositon.

mPulse 72565: mPulse API endpoint returning a 500 error The mPulse API endpoint returned a 500 error affecting Page Groups.

Copyright 2013. CloudTest is a registered trademark of SOASTA, Inc. and/or its affiliates. Other names may be trademarks of their respective owners.

2

Page 6: SOASTA CloudTest Release Notes Build 6806.68 Table of ...cdn.soasta.com/productresource/download/SOASTA_CloudTest...SOASTA CloudTest Release Notes Build 6806.68 Dec 12, 2013 Table

Build 6806.65

Bugs Fixed

CloudTest

72494: Intermittent deadlock while deploying a grid Under certain conditions, deploying a grid would cause the server to become unresponsive. 72467: Clicking OK on the ISSE Editor doesn't work The ISSE Editor was unresponsive after clicking OK while entering Clip Editor lower panel inputs. 67001: External Data Service fault - malformedURLException This exception occurred while using the External Data Source box to configure a data source URL. Now, the URL is validated before save.

mPulse 72538: Uncaught TypeError: Cannot set property 'innerText' of null; JS line 12378 This null error occurred while viewing the Globe dashboard. 72515 Uncaught TypeError: Cannot read property 'style' of null This null error occurred while viewing the Globe dashboard. 72483: Potential issue with User Agent parser and IE 11 The user agent string for Internet Explorer 11 was misidentified by mPulse.

Copyright 2013. CloudTest is a registered trademark of SOASTA, Inc. and/or its affiliates. Other names may be trademarks of their respective owners.

3

Page 7: SOASTA CloudTest Release Notes Build 6806.68 Table of ...cdn.soasta.com/productresource/download/SOASTA_CloudTest...SOASTA CloudTest Release Notes Build 6806.68 Dec 12, 2013 Table

Build 6806.63

Bugs Fixed

CloudTest 72474: this.m_oUeberDailog is undefined; JS line 12 This happened while trying to set up an external data source (e.g. integrating Wily Introscope or AppDynamics into mPulse). 72461: Unable to get value of the property 'parentElement': object is null or undefined; JS line 2099 This dashboard error occurred in the Composition Editor, Play tab. 72451: Substring parser viewer window doesn't auto-scroll to highlighted data - Firefox only Scrolling to a match didn’t occur as expected in the Session Template Package Wizard, Substring Parser. 72248: Widget filter with no or null criteria gets out-of-bounds or null error If a widget filter had no criteria or one of the criteria is null then an out-of-bounds or npe would occur. This is now handled.

Copyright 2013. CloudTest is a registered trademark of SOASTA, Inc. and/or its affiliates. Other names may be trademarks of their respective owners.

4

Page 8: SOASTA CloudTest Release Notes Build 6806.68 Table of ...cdn.soasta.com/productresource/download/SOASTA_CloudTest...SOASTA CloudTest Release Notes Build 6806.68 Dec 12, 2013 Table

TouchTest 72481: Approve Devices unavailable on 48.03 A user with privileges was unable to approve a mobile device as expected. 72269: MATT utility over-writes launchURL of Titanium Apps across multiple runs The MakeAppTouchTestable (MATT) utility over-writes the launchURL of Titanium Apps across multiple runs when the utility is also creating an app object. 71891: Android Automation Tests showing device timeout error while running tests [requires new TouchTestDriver] An automated test would succeed on an initial play and then unexpectedly fail on subsequent runs. 70832: Unable to validate an element but can click on it Validation of a webClick action unexpectedly failed, although the action itself was recorded and played back accurately. [requires new TouchTestDriver] 54171: Text Entered in a field is not recorded [requires new TouchTestDriver] While recording a scenario that included emailing an item from storage, the email address entered was not captured and therefore not entered on playback.

Copyright 2013. CloudTest is a registered trademark of SOASTA, Inc. and/or its affiliates. Other names may be trademarks of their respective owners.

5

Page 9: SOASTA CloudTest Release Notes Build 6806.68 Table of ...cdn.soasta.com/productresource/download/SOASTA_CloudTest...SOASTA CloudTest Release Notes Build 6806.68 Dec 12, 2013 Table

Build 6806.54

Bugs Fixed

CloudTest 72293: Pressing Play while loading a composition causes the Play button to become inactive Now, once loading has completed, the Play button will become available. It will remain inactive during composition loading. 72253: Newly created grids are not stored in the specified location While creating a new grid, specifying a folder in which to store it unexpectedly failed. 72289: Exported CSV files differ from dashboards because of Duration Timespan changes effective vs full The CSV export(s) of Transaction Average Duration and Page Average Duration were only using the standard definition of duration. Since the Export feature does not allow for filters and since these two CSVs are a column per Collection format, the fix is to use the default definition of duration-standard duration for old tests created prior to the introduction of this feature, and to use use effective duration for newer tests. 72267: Dashboard throwing an exception when changing Duration Timespan The onChangeFilter was not correct when the filter was in the widget rather than the dashboard 72189: Component Type and Name not displayed in Result Viewer for Database Actions and UI Automation Script events When displaying events for Database Actions and UI Automation Scripts, the Result Viewer doesn't list the name of the Database Action or UI Automation Script, and instead lists the Target name.

Copyright 2013. CloudTest is a registered trademark of SOASTA, Inc. and/or its affiliates. Other names may be trademarks of their respective owners.

6

Page 10: SOASTA CloudTest Release Notes Build 6806.68 Table of ...cdn.soasta.com/productresource/download/SOASTA_CloudTest...SOASTA CloudTest Release Notes Build 6806.68 Dec 12, 2013 Table

72184: ArrayIndexOutOfBounds errors when there are servers that couldn't be loaded If there were servers that couldn't be loaded but the Composition was played anyway (a "partial load"), then ArrayIndexOutOfBounds errors would appear in the status line during play. In addition, ArrayIndexOutOfBounds errors would occur if you used Dynamic Ramp. 72102: When you open a clip in a composition from central, the tab icon is a clip Right click on a clip in central and choose open in composition. If you watch the new concerto tab, it will have the clip icon and will show "Clip Editor" before the name of the new composition is loaded 44566: Convert-to-clip failure if "Content-Encoding" header is incorrect The Convert-Recording-to-Clip wizard would fail if there was an invalid Content-Encoding header.

Copyright 2013. CloudTest is a registered trademark of SOASTA, Inc. and/or its affiliates. Other names may be trademarks of their respective owners.

7

Page 11: SOASTA CloudTest Release Notes Build 6806.68 Table of ...cdn.soasta.com/productresource/download/SOASTA_CloudTest...SOASTA CloudTest Release Notes Build 6806.68 Dec 12, 2013 Table

TouchTest 72116: Unable to upload icon images to Mobile Apps The Mobile Apps upload page unexpectedly failed to upload mobile app icons specified by the user. 71526: Ability to select touchtestweb for BrowserType in WebUI clip This fix is provided to improve the capability to run a test clip on both a browser and a mobile device (TTW). To that end, this fix allows the user to select all WebUI system properties in property sets, including BrowserType and DeviceAgent. Additionally, there is now a DeviceAgent system property.

mPulse 72262: Selecting the "Between" time range causes null pointer exception This error occurred when the time filter didn’t supply a valid end date.

Copyright 2013. CloudTest is a registered trademark of SOASTA, Inc. and/or its affiliates. Other names may be trademarks of their respective owners.

8

Page 12: SOASTA CloudTest Release Notes Build 6806.68 Table of ...cdn.soasta.com/productresource/download/SOASTA_CloudTest...SOASTA CloudTest Release Notes Build 6806.68 Dec 12, 2013 Table

Build 6806.50

Bugs Fixed

CloudTest 72177: Empty dashboard shows dual drop targets, and the marker div is going below the bottom drop target This fixes the issue with dual draggers and incorrect marker. 72166: RepositoryObjectBrowser not working in most places Users couldn't select repository objects, such as in a linked script in the while defining a validation in the Clip Editor expanded view or in the lower panel. 72064: Average Duration incorrect in Clip Analysis and Composition Analysis Clip Analysis and Composition Analysis were using the wrong value for the average effective duration. Other analysis widgets were correct. 72044: Overlap of ISSE / Property Chooser tabs A layout issue occurred with the tabs in this dialog box. 71979: Error displaying warning in Composition Editor This Composition Editor error occurred while formatting a warning. 71869: Enhancements to scommand result outputs This takes care of the inconsistency in naming between sCommand and the UI and the 4 missing export data sources by introducing a new dataSource parameter. This parameter takes the UI name, in parentheses, of the data sources that are also presented in the Export dialog box when CSV or XML Spreadsheet formats are selected (e.g. dataSource="Clip Details). For more information, see the scommand Reference, Export Command. 71747: Bandwidth filtered by clip now working This fix introduces a new Bandwidth by Clip widget. This widget shows bandwidth only for top-level clips (e.g. not for nested clips or containers).

Copyright 2013. CloudTest is a registered trademark of SOASTA, Inc. and/or its affiliates. Other names may be trademarks of their respective owners.

9

Page 13: SOASTA CloudTest Release Notes Build 6806.68 Table of ...cdn.soasta.com/productresource/download/SOASTA_CloudTest...SOASTA CloudTest Release Notes Build 6806.68 Dec 12, 2013 Table

71536: Problem with Checkbox selection recording [requires new Firefox extension] A Checkbox that was selected during a browser recording session didn't play back as expected. 70962: Uncaught TypeError: Cannot read property 'criteria' of null; JS line 70

Copyright 2013. CloudTest is a registered trademark of SOASTA, Inc. and/or its affiliates. Other names may be trademarks of their respective owners.

10

Page 14: SOASTA CloudTest Release Notes Build 6806.68 Table of ...cdn.soasta.com/productresource/download/SOASTA_CloudTest...SOASTA CloudTest Release Notes Build 6806.68 Dec 12, 2013 Table

TouchTest

72118: scrollToVisible is causing stackoverflow on Samsung S4 [requires new TouchTestDriver] scrollToVisible would crash the Samsung D4 devices in some case. 71247: Cannot generate build for iPadAir [requires new TouchTestDriver] This fix backports arm64 support. 70489: Mobile Web Spinner on playback takes too long to choose the right item in the spinner The playback of a spinner on a mobile web site would fail. 56125: Chrome not playing back in WebUI This fixes the logic to find Google Chrome's location on Windows 7 machines.

Copyright 2013. CloudTest is a registered trademark of SOASTA, Inc. and/or its affiliates. Other names may be trademarks of their respective owners.

11

Page 15: SOASTA CloudTest Release Notes Build 6806.68 Table of ...cdn.soasta.com/productresource/download/SOASTA_CloudTest...SOASTA CloudTest Release Notes Build 6806.68 Dec 12, 2013 Table

mPulse

72109: Dashboards not updating with Filter Changes

The dashboard wouldn't refresh when filters at the dashboard level were updated. 72034: Cannot open mPulse Library items Dashboards that were added to the Library > My Items node in Soasta 48 cannot be opened from that list. 72021: Add Last 12 Hours option to time filter Moving time windows listed in the Date drop-down in mPulse Dashboards will now include an additional option, Last 12 Hours. 71909: ConcurrentModificationException This error occurred in the mPulse Dashboard in some circumstances. 71887: NPE when you create a new dashboard, but have no Date/Time filter

An mPulse dashboard that had no Date/Time filter would get this null error. 71719: Display issues in Globe This fix resolves some display issues with the Globe display. 71540: Cannot call method 'log' of null This null error occurred in an mPulse Dashboard in Chrome. 71197: ConstraintError; JS line 94 This widget error occurred in some cases while editing a dashboard (adding widgets, etc.). 71173: VersionError; JS line 18 An attempt was made to open a database using a lower version than the existing version. 71000: Uncaught TypeError: Cannot set property 'innerHTML' of null; JS line 1 Simple fix for a HUD UI event failing. 70489: Mobile Web Spinner takes too long on playback The tap and scroll on anything inside of a spinner would prevent playback

Copyright 2013. CloudTest is a registered trademark of SOASTA, Inc. and/or its affiliates. Other names may be trademarks of their respective owners.

12

Page 16: SOASTA CloudTest Release Notes Build 6806.68 Table of ...cdn.soasta.com/productresource/download/SOASTA_CloudTest...SOASTA CloudTest Release Notes Build 6806.68 Dec 12, 2013 Table

Build 6806.34

Bugs Fixed

CloudTest 71877: Don't allow renaming a Device Agent before it's been accepted It is no longer possible to rename a device agent once it's been submitted for administrator approval. The device agent can be renamed after acceptance. 71795: Double clicking a time-based chart that was not zoomed would lose filters The filters on a chart were dropped when double-clicking an empty. 71770: waitForPageToLoad failing when used in web functional clip The waitForPageToLoad accessor didn't work as expected in a WebUI test clip. 71760: Value is null This null error occurred in the Clip Editor lower panel. 71630: Fix string comparison The TTAutomationTaskHandler initError function to take an array of params instead of a variable argument list. 71057: Expected character not present This character error occurred in CloudTest Lite. 70608: Cannot read property This null error occurred in the Clip Editor lower panel.

Copyright 2013. CloudTest is a registered trademark of SOASTA, Inc. and/or its affiliates. Other names may be trademarks of their respective owners.

13

Page 17: SOASTA CloudTest Release Notes Build 6806.68 Table of ...cdn.soasta.com/productresource/download/SOASTA_CloudTest...SOASTA CloudTest Release Notes Build 6806.68 Dec 12, 2013 Table

69708: Grids not doing health checks Grid health checks were not as robust as expected and in some cases failed to be done. 69254: a is undefined in ClipElementName This script error occurred in the Clip Editor workspace. 67408: The object name cannot contain a slash mark, or square bracket marks An illegal name wasn't handled as expected. 63058: Component returned failure code This error occurred while using the Locator tool when switching messages so that the input was no longer in the DOM.

Copyright 2013. CloudTest is a registered trademark of SOASTA, Inc. and/or its affiliates. Other names may be trademarks of their respective owners.

14

Page 18: SOASTA CloudTest Release Notes Build 6806.68 Table of ...cdn.soasta.com/productresource/download/SOASTA_CloudTest...SOASTA CloudTest Release Notes Build 6806.68 Dec 12, 2013 Table

TouchTest 71750: Unable to use Device Clouds with Universal Mobile App objects If the device type was set to Universal, the device was not usable. 71698: TouchTest playback issue [requires new TouchTestDriver] The mobile app was terminating while playing back the recorded test, but the result in Cloud Test posted as successfully completed.

Copyright 2013. CloudTest is a registered trademark of SOASTA, Inc. and/or its affiliates. Other names may be trademarks of their respective owners.

15

Page 19: SOASTA CloudTest Release Notes Build 6806.68 Table of ...cdn.soasta.com/productresource/download/SOASTA_CloudTest...SOASTA CloudTest Release Notes Build 6806.68 Dec 12, 2013 Table

mPulse 71786: JavaScript error while closing mPulse dashboard This JavaScript error woud occur while closing a dashboard. 71775: RumNavTiming custom metrics results in format number exception A number format exception would occur in custom metrics. 71694: Filters on widgets for default dashboard; Performance Breakdown need to be removed Some Performance Breakdown filters that were added to widgets as a workaround in the previous mPulse Dashboard have now been removed. 71693: Labels on default Performance Summary Dashboard are not consistent The Performance Summary dashboard labels have been updated. 71450: Legend is not updated when axis title is changed Changing the axis series title wouldn't update in the legend as expected.

Copyright 2013. CloudTest is a registered trademark of SOASTA, Inc. and/or its affiliates. Other names may be trademarks of their respective owners.

16

Page 20: SOASTA CloudTest Release Notes Build 6806.68 Table of ...cdn.soasta.com/productresource/download/SOASTA_CloudTest...SOASTA CloudTest Release Notes Build 6806.68 Dec 12, 2013 Table

Build 6806.29

New Features This release includes the following major new features.

CloudTest

New CloudTest Style for Popups, Dialog Boxes, and Wizards This release introduces (or concludes) some subtle changes that have been going on in CloudTest for the last several releases. Users may've noted slight improvements to the look-and-feel of dialog boxes, popups, forms, buttons, tabs, and wizards—changes involving new fonts, color scheme, layout, and button/tab colors . Changes in this release are notable for completing this new look-and-feel for all SOASTA popups, modal dialogs, dialog boxes, and wizards (including the five full-page wizards—Grid Manager, Seed Data, Test Environment Editor, Results Service Editor, and Dashboard Wizard). The changes are also found in the lower panel forms such as those found in the Clip Editor lower panel tabs for message or action properties. The new style is a muted, two-tone gray with an offset blue color to highlight suggested user actions. Inactive fields are easier to distinguish from active ones.

Copyright 2013. CloudTest is a registered trademark of SOASTA, Inc. and/or its affiliates. Other names may be trademarks of their respective owners.

17

Page 21: SOASTA CloudTest Release Notes Build 6806.68 Table of ...cdn.soasta.com/productresource/download/SOASTA_CloudTest...SOASTA CloudTest Release Notes Build 6806.68 Dec 12, 2013 Table

Dialog Boxes –Dialog boxes have the new style (as shown in the updated Global Properties List Editor box below). Alert and modal popups also use it.

Wizards – Wizards also display the new style.

Copyright 2013. CloudTest is a registered trademark of SOASTA, Inc. and/or its affiliates. Other names may be trademarks of their respective owners.

18

Page 22: SOASTA CloudTest Release Notes Build 6806.68 Table of ...cdn.soasta.com/productresource/download/SOASTA_CloudTest...SOASTA CloudTest Release Notes Build 6806.68 Dec 12, 2013 Table

Faster load time for Central screens This release includes significant performance improvements in loading Central screens, including in Central lists with numerous items.

Faster load time for Clip Editor and no virtualized scrolling This release includes specific, significant improvements to Clip Editor load time and the elimination of virtualized scrolling.

Pause/Resume HTTP Recording in the Clip Editor This release introduces Pause and Resume commands to the Clip Editor, Recording menu. Clip Editor sessions can be paused and resumed according to need. While the Clip Editor is in Record mode, two new commands will appear on the Record menu.

• Pause Recording – If a recording session is in progress, the user can choose this command to suspend recording.

A Clip Editor HTTP recording session lasts until Stop Recording is chosen from the menu. When Pause Recording is chosen, the recording session will be suspended until resumed or stopped.

While recording is paused, the Clip Editor's Status Indicator (top right) confirms this status.

• Resume Recording – If recording is paused, the user can choose this

command to restart the recording session.

Copyright 2013. CloudTest is a registered trademark of SOASTA, Inc. and/or its affiliates. Other names may be trademarks of their respective owners.

19

Page 23: SOASTA CloudTest Release Notes Build 6806.68 Table of ...cdn.soasta.com/productresource/download/SOASTA_CloudTest...SOASTA CloudTest Release Notes Build 6806.68 Dec 12, 2013 Table

Add host name to Exclude list when removing an HTTP target As of the SOASTA 47 release, users have had the ability to define host name excludes in HTTP clips via the Clip Editor, Etc. (…) menu's Edit HTTP(s) Recording Excludes… command. As of this release, whenever an HTTP(S) target is removed from a test clip, a prompt will appear to confirm this removal. The Remove This Target prompt box also includes a checkbox giving the user an opportunity to add the host name to this excludes list.

1. Right-click the target in the Included Targets list, and then choose Remove

Target from the context menu.

2. In the Remove This Target? box that appears, check the box to add this

host name to the set of excludes for this clip. 3. Click OK to remove the target from the clip or Cancel to abandon target

removal. Excludes added using this method are appended to the existing clip excludes, which can be reviewed in the Edit HTTP(s) Recording Excludes.

Copyright 2013. CloudTest is a registered trademark of SOASTA, Inc. and/or its affiliates. Other names may be trademarks of their respective owners.

20

Page 24: SOASTA CloudTest Release Notes Build 6806.68 Table of ...cdn.soasta.com/productresource/download/SOASTA_CloudTest...SOASTA CloudTest Release Notes Build 6806.68 Dec 12, 2013 Table

Support for Stored Procedures/Functions This release introduces support for stored procedures/functions for both PostgreSQL and SQL Server (Oracle support will be introduced in a later release). A database action (in a clip element) may contain one or multiple SQL statements and/or stored procedure/function calls. Since only one output option can be specified per database action, the same output option will apply for all the SQL statements and/or stored procedures/functions for the same database action. Note that the output options for CSV, JSON, and XML formats apply to queries only. When one of those output options is specified for a database action that contains both queries and non-queries, the output option of Summary will be used for the non-query statement(s). For example, for a database action that contains two SQL statements: INSERT and SELECT. If the output option is set to “xml-with-summary” (which is for queries), the output format for the INSERT statement will be “summary” (and the output format “xml-with-summary” will be used for the SELECT statement). Note also that the output options for CSV formats cannot be used for multiple SQL statements and/or stored procedure/function calls because the columns in the results can be different. A stored procedure can contain one or multiple SQL statements. If it contains one SQL statement, the result format is same as current. In the following screenshot, a stored procedure for PostgreSQL is shown.

Copyright 2013. CloudTest is a registered trademark of SOASTA, Inc. and/or its affiliates. Other names may be trademarks of their respective owners.

21

Page 25: SOASTA CloudTest Release Notes Build 6806.68 Table of ...cdn.soasta.com/productresource/download/SOASTA_CloudTest...SOASTA CloudTest Release Notes Build 6806.68 Dec 12, 2013 Table

In the following screenshot, the database action shown consists of three stored procedure calls and two queries. There are five results respectively.

Duration Timespan Filter The addition of a Duration Timespan filter for dashboards and widgets allows users to track both full and effective duration for all containers The Duration Timespan filter appears on the dashboard edit panel's Filter tab, Attribute drop down for dashboards that use duration widgets and for the duration widgets themselves. This filter can be added to a dashboard or widget, but effective duration data can only be displayed for tests that were run after the introduction of this feature. When Duration Timespan is used, the underlying Maestro server will provide a number that includes only transport times for each container type. If no Duration Timespan filter exists then the default behavior is to show effective duration for tests created after the introduction of this feature and full durations for tests run prior to this.

At the widget level, the Duration Timespan filter appears only for duration widgets.

Copyright 2013. CloudTest is a registered trademark of SOASTA, Inc. and/or its affiliates. Other names may be trademarks of their respective owners.

22

Page 26: SOASTA CloudTest Release Notes Build 6806.68 Table of ...cdn.soasta.com/productresource/download/SOASTA_CloudTest...SOASTA CloudTest Release Notes Build 6806.68 Dec 12, 2013 Table

Duration widgets are found in the Widget Type Selection Panel's Widget Types, Charts, Durations category.

The possible operator values for Duration Timespan are: "Effective" and "Full."

Effective – Effective duration includes only that time it took to execute resources from point A to point B in time. This measure is the overall amount of time doing “real work” as opposed to the “overhead” created by ancillary tasks (i.e. time spent doing validations, scripts, IF statements, etc.). Users should note that parallel tasks (such as in the resources section of a page) are not counted separately. For example, if a Page had two resources, which were sent in parallel, and one took 10 seconds to send and receive whiled the other took 15 seconds, the “effective duration” of the resources section of that page is 15, and not 25.

Note: Delay times are an exception to the general rule about calculating

effective duration. They are included in this measure.

Full – The full amount of time including the overhead of ancillary tasks for the given container from beginning to end.

Copyright 2013. CloudTest is a registered trademark of SOASTA, Inc. and/or its affiliates. Other names may be trademarks of their respective owners.

23

Page 27: SOASTA CloudTest Release Notes Build 6806.68 Table of ...cdn.soasta.com/productresource/download/SOASTA_CloudTest...SOASTA CloudTest Release Notes Build 6806.68 Dec 12, 2013 Table

In the screenshot below, effective duration is applied at the dashboard level.

If the Duration Timespan, Effective Duration value is set on a dashboard or widget, and the displayed result does not have this data, then the dashboard will show "empty" (or, if it's a chart, "No Data", and, if it's a cell in an Analysis widget, then the same dash shown "-" for other non-applicable fields will be shown).

Series Filtering This release introduces a new Series filter that allows users to filter the various series in a result at both the dashboard level, and where applicable, at the widget level. The Series filter appears in all dashboards. Series filtering will only impact those charts that support it.

The Series filter also appears in all charts that present comparative information in series form.

Copyright 2013. CloudTest is a registered trademark of SOASTA, Inc. and/or its affiliates. Other names may be trademarks of their respective owners.

24

Page 28: SOASTA CloudTest Release Notes Build 6806.68 Table of ...cdn.soasta.com/productresource/download/SOASTA_CloudTest...SOASTA CloudTest Release Notes Build 6806.68 Dec 12, 2013 Table

A chart series is one category of data presented in a chart as points on a timeline.

Note: The Value field for the Series filter will display series by the order

of the number of points in that series--with the series with the

most points showing up first.

For example, in the shot below, note that the possible values to filter by are listed by the order of the number of points in the given series.

Copyright 2013. CloudTest is a registered trademark of SOASTA, Inc. and/or its affiliates. Other names may be trademarks of their respective owners.

25

Page 29: SOASTA CloudTest Release Notes Build 6806.68 Table of ...cdn.soasta.com/productresource/download/SOASTA_CloudTest...SOASTA CloudTest Release Notes Build 6806.68 Dec 12, 2013 Table

mPulse

Time Windows mPulse now includes additional Date filters that can be used to define time windows between specified dates, as well as other menu-driven time windows such as Last 24 Hours, Last Hour, This Week, and This Month. A time window is a time bound or a time range window, in this case one that is specified as between two dates in time. In the prior release, the Date filter had limited options:

• Today (the current day) and,

• On (if selected, the Calendar would appear to set the filter-by date.

As of SOASTA 48, Today is still the default Date filter and a specific date can still be set using "On" with the specified calendar date. However, all of the following new "moving time window" settings are also available:

• Last 24 Hours – Filter by the prior 24 hour time window

• Last hour – Filter by the prior hour time window

• This week – Filter by the current calendar week that is underway

• This month – Filter by the current calendar month that is underway

• On – Filter is set to the specified calendar date

• Between – Filter by the defined time window Additionally, the Calendar drop-down, Date settings can now be used to filter the window between two dates (in addition to On a specific calendar date).

Copyright 2013. CloudTest is a registered trademark of SOASTA, Inc. and/or its affiliates. Other names may be trademarks of their respective owners.

26

Page 30: SOASTA CloudTest Release Notes Build 6806.68 Table of ...cdn.soasta.com/productresource/download/SOASTA_CloudTest...SOASTA CloudTest Release Notes Build 6806.68 Dec 12, 2013 Table

Copyright 2013. CloudTest is a registered trademark of SOASTA, Inc. and/or its affiliates. Other names may be trademarks of their respective owners.

27

Page 31: SOASTA CloudTest Release Notes Build 6806.68 Table of ...cdn.soasta.com/productresource/download/SOASTA_CloudTest...SOASTA CloudTest Release Notes Build 6806.68 Dec 12, 2013 Table

Defining a Time Window using the Filter Toolbar This release offers precise time window settings to within five-minute increments (e.g. Thursday 1:05 AM to Friday 2:05 PM). Such time windows are useful for isolating issues that happen right after a rollout, or major event in an application life cycle. Use the following steps to define a time window by date, and also, to the hour.

TIP: If you are using the mPulse Dashboard, you can already define a

time window by using the Date drop-down, and selecting Between.

If you'd like to change the defaults using the Edit Panel, you can

do so in any custom dashboard. An mPulse prompt will appear if

the current dashboard needs to be cloned. In such cases, you can

make the cloned dashboard your default one via the Edit Panel,

Dashboard, General tab's Set as my Default Dashboard button.

1. Using the default Filter toolbar, select Date, and then select Between from the drop-down.

When you do so, two date/time combos appear on the filter toolbar. Use these to define the starting point and ending point of the time window.

2. Set the ending point for this time window first by clicking the second date and then using the second Calendar.

3. Optionally, also define the time (e.g. to the hour or minute using the time spinners).

The time window is between two points in time.

4. Optionally, also define an hour in the day. Use the spinners on the time to set to the minute filtering in your time window.

Copyright 2013. CloudTest is a registered trademark of SOASTA, Inc. and/or its affiliates. Other names may be trademarks of their respective owners.

28

Page 32: SOASTA CloudTest Release Notes Build 6806.68 Table of ...cdn.soasta.com/productresource/download/SOASTA_CloudTest...SOASTA CloudTest Release Notes Build 6806.68 Dec 12, 2013 Table

Note: If no hour is specified then note that the range includes the entire

day (e.g. until the next day begins).

5. Set the starting point for this time window by clicking the first date and then using the first Calendar.

6. Optionally, also define the time (e.g. to the hour or minute using the time spinners).

Defining a Dashboard Time Window using the Edit Panel If the dashboard is new and wasn't cloned from an existing system dashboard, you'll need to add the necessary filters using the following steps.

1. Open the dashboard and Toggle Dashboard Edit Mode to show the lower panel. The panel opens with the Dashboard node selected.

2. Click the Filter tab. Note that the dashboard below has one generic filter with None as its value. If your custom dashboard was based on an existing system dashboard it may have many more filters already set than are shown below.

TIP: Always check "Display filter toolbar" if you'd like to be able to

change filter values on the dashboard without opening this panel.

3. Either click the Attribute drop-down to change the default to Date, or click the green Plus (+) icon and then set the new row to Date. If you're defining a multi-attribute dashboard filter, add the other attributes in the order you'd like them to display, ensuring that Date is defined in one row.

4. Set the given row to the Attribute, Date.

Copyright 2013. CloudTest is a registered trademark of SOASTA, Inc. and/or its affiliates. Other names may be trademarks of their respective owners.

29

Page 33: SOASTA CloudTest Release Notes Build 6806.68 Table of ...cdn.soasta.com/productresource/download/SOASTA_CloudTest...SOASTA CloudTest Release Notes Build 6806.68 Dec 12, 2013 Table

5. Set the operator to Between to filter the dashboard by that value now, or, if

you intend to display the filter toolbar, set it to any value and set it at runtime.

6. Set the ending point for this time window. 7. Optionally, also define an hour in the day. 8. Set the starting point for this time window. 9. Optionally, also define an hour in the day. 10. Click Apply and then OK to exit the Edit Panel.

Copyright 2013. CloudTest is a registered trademark of SOASTA, Inc. and/or its affiliates. Other names may be trademarks of their respective owners.

30

Page 34: SOASTA CloudTest Release Notes Build 6806.68 Table of ...cdn.soasta.com/productresource/download/SOASTA_CloudTest...SOASTA CloudTest Release Notes Build 6806.68 Dec 12, 2013 Table

Percentile Filter Attribute This release introduces the filter attribute, Percentile, to mPulse dashboards, with the added ability for users to select an aggregation level from among pre-sets or by entering one. With this change, mPulse now supports viewing any percentile. Percentile filtering appears on the default filter toolbar of the mPulse Dashboard with pre-sets of 25th, 50th, 75th, 95th, and 98th percentiles.

Users can select a preset from the Percentile drop-down.

Or, enter a custom percentile using the drop-down's entry field.

Copyright 2013. CloudTest is a registered trademark of SOASTA, Inc. and/or its affiliates. Other names may be trademarks of their respective owners.

31

Page 35: SOASTA CloudTest Release Notes Build 6806.68 Table of ...cdn.soasta.com/productresource/download/SOASTA_CloudTest...SOASTA CloudTest Release Notes Build 6806.68 Dec 12, 2013 Table

Enhancements

CloudTest

Stopping a Script or ISSE This enhancement allows a long-running Script, or script within an ISSE, to be stopped prematurely when a Composition is stopped, such as via the Stop button. Note that a Script can only be stopped in-between JavaScript "instructions", so if there is a single JavaScript statement that is taking a long time (such as a complex Regex, for example), the Script may not be able to be stopped until that JavaScript statement completes.

Clip, Track, or Band Doesn't Get Focus in Results Details Dashboard As of this release, Clips, Tracks, and Bands will get focus in the right panel of the Result Details dashboard, when selected on the left navigation tree.

Better error message when there are too many components in the Results Details The Play (General mode) prompt will now indicate more precisely how many components is too many.

Pending timeout should be ignored for Cloud Providers that don't support early termination The pending timeout setting in the Grid Manager is now ignored for Cloud Providers that don't support pending servers (i.e., GoGrid, IBM SmartCloud).

Copyright 2013. CloudTest is a registered trademark of SOASTA, Inc. and/or its affiliates. Other names may be trademarks of their respective owners.

32

Page 36: SOASTA CloudTest Release Notes Build 6806.68 Table of ...cdn.soasta.com/productresource/download/SOASTA_CloudTest...SOASTA CloudTest Release Notes Build 6806.68 Dec 12, 2013 Table

Database Monitoring Improvements This release expands the ways in which CloudTest users can monitor databases—including two new database types as well as eight new resource types, and the ability to view these new resource types in new analytic charts provided for that purpose.

Setting Up Monitoring Server Groups and Monitors for Databases The Central > Monitoring Server Groups and/or Monitor wizards remain the same for this change, however, four database types are now available, as well as eight new monitoring resource types (discussed below). Two of the four resource provider (e.g. database) types for monitoring, Oracle and SQL Server, are new to this release.

1. To monitor a database, go to CloudTest and click Monitoring Server Groups under Monitoring to create a group for the database instance.

2. Then, click Monitors to create a monitor for the monitoring server group. 3. During either Monitor Server Group or Monitor configuration, select the

database type to monitor in the Database Type and Authentication page.

In the screenshot above, database type selection occurs as part of the overall definition of a new Monitor Server or Monitor using the Database(s) page. Existing Monitor Server Groups or Monitors can also be revised to include this database monitoring.

4. Provide the necessary database connectivity credentials, minimally to include user name, password, and password confirmation.

Copyright 2013. CloudTest is a registered trademark of SOASTA, Inc. and/or its affiliates. Other names may be trademarks of their respective owners.

33

Page 37: SOASTA CloudTest Release Notes Build 6806.68 Table of ...cdn.soasta.com/productresource/download/SOASTA_CloudTest...SOASTA CloudTest Release Notes Build 6806.68 Dec 12, 2013 Table

5. Click Test Connection to verify successful connectivity.

6. Next, define the Database Resource Selection for the given database type by checking the relevant resources to monitor.

7. Click Finish after performing the above steps. 8. Ensure that you Enable Server Monitoring and check the box for the

monitor(s) you wish to use in a test composition (refer to Starting a Monitor Automatically When a Test Composition Runs). You can also use the Central > Monitors, View Analytics link for any given, listed monitor (refer to Starting and Stopping a Monitor).

The eight new resource types for monitors are shown in alphabetical order alongside the previously existing four resources types in the resource selection page (above). The new resource types are:

Copyright 2013. CloudTest is a registered trademark of SOASTA, Inc. and/or its affiliates. Other names may be trademarks of their respective owners.

34

Page 38: SOASTA CloudTest Release Notes Build 6806.68 Table of ...cdn.soasta.com/productresource/download/SOASTA_CloudTest...SOASTA CloudTest Release Notes Build 6806.68 Dec 12, 2013 Table

• Buffer Cache Size – The size of the database buffer cache, in bytes

• Deletes – The number of DELETE statements issued to the database (MySQL, Oracle, SQL Server), or the number of rows deleted from the current catalog (PostgreSQL).

• Disk Sorts – The number of sorts that required creating a temporary table on the disk. This metric is not supported when monitoring PostgreSQL.

• Full Table Scans – The number of full table scans across all queries.

• Index Scans – The number of index scans across all queries.

• Inserts – The number of INSERT statements issued to the database (MySQL, Oracle, SQL Server), or the number of rows inserted into the current catalog (PostgreSQL).

• Selects – The number of SELECT statements issued to the database (MySQL, Oracle, SQL Server), or to the current catalog (PostgreSQL).

• Updates – The number of UPDATE statements issued to the database (MySQL, Oracle, SQL Server), or the number of rows updated in the current catalog (PostgreSQL).

The previously existing resource types were:

• Commits – The number of transactions committed.

• Per Catalog Connections -- The number of active connections to

the specified catalog. Copyright 2013. CloudTest is a registered trademark of SOASTA, Inc. and/or its affiliates. Other names may be trademarks of their respective owners.

35

Page 39: SOASTA CloudTest Release Notes Build 6806.68 Table of ...cdn.soasta.com/productresource/download/SOASTA_CloudTest...SOASTA CloudTest Release Notes Build 6806.68 Dec 12, 2013 Table

• Rollbacks -- The number of transactions rolled back.

• Total Connections -- The total number of active connections,

across all catalogs.

New Database Action XML and JSON Format Outputs for Supporting Multiple Results Direct to Database early adopters should note that in this release the schemas for XML and JSON formats have been changed to allow for the output to contain multiple "results".

TIP: For an introduction to this important new feature, which was

introduced in SOASTA 47, see Direct to Database Testing.

In the case of XML, a new Result tag has been added underneath the existing root Results tag, as well as a new Output tag as the root. For JSON, the top level tag is “output” for an JSON object represented by the second level tag “results”, which is in turn a JSON array for multiple results. Each result is a JSON object consisting of the “summary” and “data" objects or just the "data" object.

Note: Support for multiple results using the CSV output option is not

available at this time. This is due to the limitations of the CSV

format, Since we can't know this ahead of time, there will be a

runtime error if a given Database Action ends up returning

multiple result sets and the output format requested was CSV.

Summary of XML and JSON Output Format Changes

For examples of output formats, see Database Query Results in Result Details.

xml Retrieved rows will be formatted as XML. Applies to "SELECT FROM" statements only. The root node of the XML document is now Output, which contains the Results element that contains one or more Result elements for multiple results. Each result contains one Data element.

xml-

with-

summary

Retrieved rows will be formatted as XML with summary element. Applies to "SELECT FROM" statements only. The root node of the XML document is Output, which contains the Results element that contains one or more Result elements for multiple results. Each result contains the Summary and Data elements.

The summary element contains some information about the columns specified in the "SELECT FROM" statement, including column name,

Copyright 2013. CloudTest is a registered trademark of SOASTA, Inc. and/or its affiliates. Other names may be trademarks of their respective owners.

36

Page 40: SOASTA CloudTest Release Notes Build 6806.68 Table of ...cdn.soasta.com/productresource/download/SOASTA_CloudTest...SOASTA CloudTest Release Notes Build 6806.68 Dec 12, 2013 Table

column (data) type, and approximate column size.

json Retrieved rows will be formatted as JSON. Applies to "SELECT FROM" statements only. The top level tag is output. The tag at the next level is results, which for a JSON object contains one or more results represented by a JSON array. Each result contains one JSON object with the data tag.

json-

with-

summary

Retrieved rows will be formatted as JSON with summary. Applies to "SELECT FROM" statements only. The top level tag is output. The tag at the next level is results which for a JSON object contains one or more results represented by a JSON array. Each result contains two JSON objects with the summary and data tags. The summary value contains some information about the columns specified in the "SELECT FROM" statement, including column name, column (data) type, and approximate column size.

 

Stored Procedures in the Result Details Dashboard In the following screenshots, five results are shown individually. The first of the five (Result 1/5) results is shown below.

Note that the following three screenshots correspond to the first three results for the stored procedure calls (shown above). Three columns of data are shown for the given row.

Copyright 2013. CloudTest is a registered trademark of SOASTA, Inc. and/or its affiliates. Other names may be trademarks of their respective owners.

37

Page 41: SOASTA CloudTest Release Notes Build 6806.68 Table of ...cdn.soasta.com/productresource/download/SOASTA_CloudTest...SOASTA CloudTest Release Notes Build 6806.68 Dec 12, 2013 Table

Each of the three results on this page contains three columns of data.

The fourth query returns one column of results.

The fifth query returns two columns of results.

Copyright 2013. CloudTest is a registered trademark of SOASTA, Inc. and/or its affiliates. Other names may be trademarks of their respective owners.

38

Page 42: SOASTA CloudTest Release Notes Build 6806.68 Table of ...cdn.soasta.com/productresource/download/SOASTA_CloudTest...SOASTA CloudTest Release Notes Build 6806.68 Dec 12, 2013 Table

New Database Monitoring Widgets There are eight new monitor dashboard widgets for the eight new resource types. These are found in the Widget Type Selection Panel's Widget Type, Monitor category:

• Buffer Cache Size – The size of the database buffer cache, in bytes

• Deletes – The number of DELETE statements issued to the database

(MySQL, Oracle, SQL Server), or the number of rows deleted from the current catalog (PostgreSQL).

• Disk Sorts – The number of sorts that required creating a temporary table

on the disk. This metric is not supported when monitoring PostgreSQL. The

Copyright 2013. CloudTest is a registered trademark of SOASTA, Inc. and/or its affiliates. Other names may be trademarks of their respective owners.

39

Page 43: SOASTA CloudTest Release Notes Build 6806.68 Table of ...cdn.soasta.com/productresource/download/SOASTA_CloudTest...SOASTA CloudTest Release Notes Build 6806.68 Dec 12, 2013 Table

Disk Sorts widget does a simple count. When the count is zero, the Y-axis lists 1, 2, 3, 4.

• Full Table Scans – The number of full table scans across all queries.

• Index Scans – The number of index scans across all queries.

Copyright 2013. CloudTest is a registered trademark of SOASTA, Inc. and/or its affiliates. Other names may be trademarks of their respective owners.

40

Page 44: SOASTA CloudTest Release Notes Build 6806.68 Table of ...cdn.soasta.com/productresource/download/SOASTA_CloudTest...SOASTA CloudTest Release Notes Build 6806.68 Dec 12, 2013 Table

• Inserts – The number of INSERT statements issued to the database (MySQL, Oracle, SQL Server), or the number of rows inserted into the current catalog (PostgreSQL).

• Selects – The number of SELECT statements issued to the database

(MySQL, Oracle, SQL Server), or to the current catalog (PostgreSQL).

• Updates – The number of UPDATE statements issued to the database

(MySQL, Oracle, SQL Server), or the number of rows updated in the current catalog (PostgreSQL).

Copyright 2013. CloudTest is a registered trademark of SOASTA, Inc. and/or its affiliates. Other names may be trademarks of their respective owners.

41

Page 45: SOASTA CloudTest Release Notes Build 6806.68 Table of ...cdn.soasta.com/productresource/download/SOASTA_CloudTest...SOASTA CloudTest Release Notes Build 6806.68 Dec 12, 2013 Table

Track Filtering Improvements In prior releases, filtering by track was not supported by any widgets.

Note: The dashboard-level Track filter was added for use by the Virtual

User Ramp Controller; no widgets supported it prior to this release.

Now, widgets for Clips and for Collections can also be filtered by track.

For TouchTest users, by-track filters can also be used as an equivalent of by-device filtering when there is one device per track (e.g. by selecting a device in the Track's Location drop-down). It should be noted that if devices are set at the clip-level, then using the track filter as an equivalent of "location" will not be possible.

Copyright 2013. CloudTest is a registered trademark of SOASTA, Inc. and/or its affiliates. Other names may be trademarks of their respective owners.

42

Page 46: SOASTA CloudTest Release Notes Build 6806.68 Table of ...cdn.soasta.com/productresource/download/SOASTA_CloudTest...SOASTA CloudTest Release Notes Build 6806.68 Dec 12, 2013 Table

Error Analysis for App Actions and Browser Actions As of this release, TouchTest devices will appear in the Error Analysis widget under the Domains node, which formerly only listed HTTP messages (e.g. under the Domains node in the Error Analysis widget). Now, both browser actions (WebUI) and app actions (TouchTest) are listed.

App Actions in the Error Analysis Widget For app actions, devices appear under the Domains node.

Expand the error node under the Domains node to inspect the Device(s) and its errors. If there is more than one, the errors are organized by the given device.

Note: As noted in the Track Filtering Improvements section of these

release notes, in cases where one device is utilized per track, the

Track filter can also be used as an equivalent of device location

(e.g. via awareness of a given device in a given location or by

Copyright 2013. CloudTest is a registered trademark of SOASTA, Inc. and/or its affiliates. Other names may be trademarks of their respective owners.

43

Page 47: SOASTA CloudTest Release Notes Build 6806.68 Table of ...cdn.soasta.com/productresource/download/SOASTA_CloudTest...SOASTA CloudTest Release Notes Build 6806.68 Dec 12, 2013 Table

naming a device agent (TouchTest Agent) to correspond to a given

location).

Browser Actions in the Error Analysis Widget For browser actions, errors appear under the Domains node.

Expand the error node under the Domains node to inspect the Conductor(s) and its errors. If there is more than one Conductor, the errors are organized by the given conductor name.

Note: As noted in the Track Filtering Improvements section of these

release notes, in cases where one Conductor is utilized per track,

the Track filter can also be used as an equivalent of client location

(e.g. via awareness of a given client in a given location or by

naming a Conductor to correspond to a given location). Copyright 2013. CloudTest is a registered trademark of SOASTA, Inc. and/or its affiliates. Other names may be trademarks of their respective owners.

44

Page 48: SOASTA CloudTest Release Notes Build 6806.68 Table of ...cdn.soasta.com/productresource/download/SOASTA_CloudTest...SOASTA CloudTest Release Notes Build 6806.68 Dec 12, 2013 Table

mPulse

Defining a Dashboard Percentile using the Edit Panel Percentile can be set up for custom dashboards either by cloning a system dashboard, or, by defining Percentile as an attribute to filter by via the Dashboard Edit Panel in a new dashboard.

1. Enter Toggle Dashboard Edit Panel mode in the custom dashboard. 2. Do the following for the dashboard or widget:

• For dashboard filtering, select the Dashboard node in the list and then click Filter.

• For widget filtering, select the Widget node in the list and then click Filter

3. Revise an existing row, or add a new row, and then set the row attribute to

Percentile. 4. Select a pre-set percentile or enter a custom value.

Copyright 2013. CloudTest is a registered trademark of SOASTA, Inc. and/or its affiliates. Other names may be trademarks of their respective owners.

45

Page 49: SOASTA CloudTest Release Notes Build 6806.68 Table of ...cdn.soasta.com/productresource/download/SOASTA_CloudTest...SOASTA CloudTest Release Notes Build 6806.68 Dec 12, 2013 Table

TIP: Click Display filter toolbar if you intend for this to appear on the

dashboard filter toolbar.

5. Click Apply and then OK to exit the Edit Panel. Percentile filtering is available in all of the following widgets:

• AB Tests

• Browsers

• Geography

• Multi-Histogram

• Page Groups

• Summary Percentiles are not relevant to User Agent breakdown or Bounce Rate & Session Metrics.

New 'All' default option in A/B Test filter The A/B Test widget filter now shows a new All option, which is also the new default option for this filter. When All is selected, dashboard data will show results from all test groups. The filter is shown below as displayed on the Dashboard filter toolbar.

As with any filter, A/B Test, including its new All option, can also be added to custom dashboards and toolbars via the Dashboard Edit Panel.

Copyright 2013. CloudTest is a registered trademark of SOASTA, Inc. and/or its affiliates. Other names may be trademarks of their respective owners.

46

Page 50: SOASTA CloudTest Release Notes Build 6806.68 Table of ...cdn.soasta.com/productresource/download/SOASTA_CloudTest...SOASTA CloudTest Release Notes Build 6806.68 Dec 12, 2013 Table

Formerly, this toolbar filter setting had only None (and whichever A/B groups are defined) as an option.

Copyright 2013. CloudTest is a registered trademark of SOASTA, Inc. and/or its affiliates. Other names may be trademarks of their respective owners.

47

Page 51: SOASTA CloudTest Release Notes Build 6806.68 Table of ...cdn.soasta.com/productresource/download/SOASTA_CloudTest...SOASTA CloudTest Release Notes Build 6806.68 Dec 12, 2013 Table

Allow CSV exporting of data from the Geography widget. As of this release, users can now use the Export icon on the mPulse, Geography widget to export widget data to a CSV format spreadsheet. To do so, hover the mouse over the Geography widget title bar, and then select the Excel icon and specify the local folder to save the comma-separated value(s) (CSV) file.

The exported fields in this release are Country ISO Code, Country Name, Median Measurements, and Number of Measurements.

Other Analytic Changes in This Release The following changes to RUM widgets are also effective as of this release:

• Load Time by Minute is now known as Timer Over Time

• Timer Over Time (a.k.a. Load Time by Minute) and Median Load Over Time have been combined. The Timer Over Time widget now covers both of these cases.

• Bounce Rate Per Page Load Time is now Metric per Timer (noted elsewhere)

• Metrics by Minute and Metric over Time have been combined. The Metric over Time widget now covers both of these cases.

• The Timer Over Time, Metric over Time, and Measurements Over Time widgets all support user definable time windows now.

Sessions per Page Load Time is now Sessions Per Timer (noted elsewhere)

Copyright 2013. CloudTest is a registered trademark of SOASTA, Inc. and/or its affiliates. Other names may be trademarks of their respective owners.

48

Page 52: SOASTA CloudTest Release Notes Build 6806.68 Table of ...cdn.soasta.com/productresource/download/SOASTA_CloudTest...SOASTA CloudTest Release Notes Build 6806.68 Dec 12, 2013 Table

TouchTest

TouchTest Web User-Agent Setting (iOS Only) This feature [requires new TouchTest Web]. TouchTest Web for iOS users can now customize mobile web recording sessions to emulate a specific mobile browser by specifying the necessary user-agent string in the mobile device's Settings. Use the following steps to specify a browser (User-Agent) by string from within the mobile device's iOS Settings, TouchTest Web settings.

1. To set the User-Agent for recording or playback, open the iOS device's Settings.

2. Locate and tap TouchTest Web.

3. When TouchTest Web is selected, the Browser Setting field appears.

TIP: When the Browser Setting field is blank, the default browser for

the given device is in use.

Copyright 2013. CloudTest is a registered trademark of SOASTA, Inc. and/or its affiliates. Other names may be trademarks of their respective owners.

49

Page 53: SOASTA CloudTest Release Notes Build 6806.68 Table of ...cdn.soasta.com/productresource/download/SOASTA_CloudTest...SOASTA CloudTest Release Notes Build 6806.68 Dec 12, 2013 Table

4. To emulate a specific browser, enter either its long form or short form user-agent string.

For example, in the screenshot below the Safari short-form, Safari/9537.53, is in use.

The Safari long-form will also work (as will the short- or long-form for all major browsers: Mozilla/5.0 (iPhone; CPU iPhone OS 7_0_3 like Mac OS X) AppleWebKit/537.51.1 (KHTML, like Gecko) Version/7.0 Mobile/11B511 Safari/9537.53

−Or—

Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/31.0.1650.57 Safari/537.36

Note: As noted earlier, this setting is persistent and it's a good practice

to blank the field when you are done with a specific

recording/playback.

When in doubt, inspect the Settings, TouchTest Web, Browser

Setting on the given device, or, you can also use

http://www.whatsmyuseragent.com to see what user-agent is set).

Copyright 2013. CloudTest is a registered trademark of SOASTA, Inc. and/or its affiliates. Other names may be trademarks of their respective owners.

50

Page 54: SOASTA CloudTest Release Notes Build 6806.68 Table of ...cdn.soasta.com/productresource/download/SOASTA_CloudTest...SOASTA CloudTest Release Notes Build 6806.68 Dec 12, 2013 Table

Bugs Fixed

CloudTest Bugs Fixed 71443: (CloudTest) Dropping widgets at bottom fails sporadically Attempting to drop widgets into the bottom of a dashboard would fail unexpectedly in some cases. 71408: (CloudTest) Unable to rename a reserved IP address Renaming a Reserved IP Address from the lower panel does not seem to have any effect. 71355: (CloudTest) Validations are not preserved during duplicate clip element Duplicating a clip element would fail to carry over any of its validations. 71218: (CloudTest) Charts not showing data for test For a given result in an environment, the Average Response Time and Send Rate are not showing any values in charts. 71156: (CloudTest) Capture Screenshot size seems to be random

Sometimes the Result Details, captureScreenshot output was too small to be viewable, leading users to believe that Conductor had failed to take a screenshot. Maximizing would expand the window enough looked fine and the screenshot was viewable. 71155: (CloudTest) Error while duplicating action that had a Screenshot Validation After recording an action, and then adding a screenshot validation, this error would occur if that action was duplicated. 70815: (CloudTest) Charts not displaying after changing filter or other changes Charts were not behaving well in using remaining space after chart restart. 70813: (CloudTest) Drag and drop issues in dashboard Dragging widgets on other widgets would sometimes get stuck. Additionally, dragging widgets into the footer was hard to do if there weren't widgets there already.

Copyright 2013. CloudTest is a registered trademark of SOASTA, Inc. and/or its affiliates. Other names may be trademarks of their respective owners.

51

Page 55: SOASTA CloudTest Release Notes Build 6806.68 Table of ...cdn.soasta.com/productresource/download/SOASTA_CloudTest...SOASTA CloudTest Release Notes Build 6806.68 Dec 12, 2013 Table

70812: (CloudTest) Dashboard doesn't allow widget to be dropped The dashboard didn't show drop targets if there weren't any widgets on drag. 70650: (CloudTest) Total VU count is not refreshing after changing the 2nd track VU in Composition Editor This case was due to a webkit (chrome/safari) specific rendering bug where setting innerText doesn't immediately redraw the element all the time. This caused the VU count bug, as the number inside the element when inspected was 40, but it was still showing 20 because of the webkit redraw bug. The VU calculation was working fine, and now, CloudTest will use textContent where it formerly used innerText, resulting in performance improvements. 70282: (CloudTest) Monitoring Server Groups - Label missing "Group" The Monitoring Server Groups wizard incorrectly used the label Monitoring Server. 70075: (CloudTest) Dimension filter values queried twice when a value is selected This fix will prevent a query for filters for the lower panel while changing the surface if the lower panel isn't visible. 70024: (CloudTest) Only check for non-playing results once per tenant Tokens were unexpectedly kept alive more than the maximum five hours. 70005: (CloudTest) Add Validation didn't work if the locator contained Xpath This error would occur while adding a validation to a message whose locator was an Xpath. 69174: (CloudTest) Grid deploy gets stuck if the attached IP address fails

Deploying an environment with Reserved IP Addresses in the Repository that are actually invalid would result in this error. 69092: (CloudTest) Fundamentals Widget broke at end of test The Fundamentals widget would display an error message after test completion indicating that a result of ResultID 0 couldn't be found. 68701: (CloudTest) External Data Source UI allowing illegal URLs A user was unexpectedly able to enter an illegal URL (e.g. it had a space) in the External Data Source user interface. 67708: (CloudTest) Severs not being shown in running grid In some cases, servers didn't show up until a grid was terminated.

Copyright 2013. CloudTest is a registered trademark of SOASTA, Inc. and/or its affiliates. Other names may be trademarks of their respective owners.

52

Page 56: SOASTA CloudTest Release Notes Build 6806.68 Table of ...cdn.soasta.com/productresource/download/SOASTA_CloudTest...SOASTA CloudTest Release Notes Build 6806.68 Dec 12, 2013 Table

67699: (CloudTest) Saving a composition that was opened from a clip in Central fails to stop spinning This happened on Save when a user right-clicked a clip in Central to open in a composition. After picking a name and clicking ok, the "Saving Test Composition" status text never went away. 67414: (CloudTest) Misaligned Checkbox and label in Clip Editor, Outputs section The "Only if there is an error" checkbox was not left aligned as expected. 67321: (CloudTest) isBoolean discrepancies in the inline editor, lower panel for waits and validations In some cases an additional text input field would appear in the Waits or Validations forms in an expanded action or in the lower panel. 67103: (CloudTest) Null pointer occurred because the list of running results included a null when a result was for a different RS DB This null pointer exception occurred in a multi-Results-Database scenario. 66729: (CloudTest) Property sets in Clip Editor are misaligned Layout in the Clip Editor lower panel, Property Sets <Before/After> tabs was off.

66022: (CloudTest) report generation dialog spinner misplaced The Creating Report spinner appeared in the wrong place after exporting a report. 65876: (CloudTest) Script error in Server List editing An error occurred while editing the Server List. 65548: (CloudTest) Renaming a tenant error causes the CloudTest Error Message box to become inaccessible Renaming an existing tenant caused this error and the error box to become inaccessible. 65457: (CloudTest> The cloud provider account: cannot launch results databases The error box should've caught an underlying error. 65455: (CloudTest) Unable to use return key in error descriptions Entering a bug description with a carriage return in it and it submitted the dialog instead of inserting a carriage return.

Copyright 2013. CloudTest is a registered trademark of SOASTA, Inc. and/or its affiliates. Other names may be trademarks of their respective owners.

53

Page 57: SOASTA CloudTest Release Notes Build 6806.68 Table of ...cdn.soasta.com/productresource/download/SOASTA_CloudTest...SOASTA CloudTest Release Notes Build 6806.68 Dec 12, 2013 Table

65351: (CloudTest) Selenium will now auto-remove alerts/confirms/prompts in Firefox Cross-Domain Alerts and other modal boxes didn't work as expected if a browser test was using Firefox Cross-Domain 65274: (CloudTest) Error Analysis widget does not work for database actions Database actions in results didn't display as expected in the Error Analysis widget. 62395: (CloudTest) Switching tabs in the clip editor causes focus to be in the wrong frame This happened after saving a clip with 1 or more clip elements in it. When the Save dialog comes up, switching to another CloudTest tab like Central, then switching back to the Clip Editor tab and pressing the Delete key would cause focus to be in the wrong frame. 62136: (CloudTest) Composition failed to load A test composition that was trying to load to 1200 servers should have had enough verified servers to load but unexpectedly failed. Now, the Server List version is checked on the abort and close of a monitor.

Copyright 2013. CloudTest is a registered trademark of SOASTA, Inc. and/or its affiliates. Other names may be trademarks of their respective owners.

54

Page 58: SOASTA CloudTest Release Notes Build 6806.68 Table of ...cdn.soasta.com/productresource/download/SOASTA_CloudTest...SOASTA CloudTest Release Notes Build 6806.68 Dec 12, 2013 Table

TouchTest Bugs Fixed 71454: (TouchTest) VerifyElementPresent failing when it is present, when switching from VerifyElementText Switching a true validation from verifyElementText to verifyElementPresent, would cause the verifyElementPresent to fail, even if it is present. 71365: (TouchTest) Able to select a Device Agent in Track-level drop-down that isn't Mobile Web Enabled An underlying issue in the Track/Clip drop-down code caused devices that were not mobile-web enabled to be selectable. Now, that track/clip dropdown will correctly disable device agents that are not mobile-web enabled whenever there is a mobile-web target in use. Additional help text appears on the disabled device in the dropdown indicating why it can't be selected. 69255: (TouchTest) outputViewHierarchy doesn't use available space in expanded Result Details, Outputs section The output, outputViewHierarchy, didn't use the available space when expanded in the Outputs section. 67677: (TouchTest) Error opening CloudTest from Jenkins plugin An iframe error occurred accessing a CloudTest window from within Jenkins. 71387: (TouchTest) Android Device Timeout on TouchTest Web clips [requires new TouchTestDriver] and [requires new TouchTest Web] The TouchTest Agent unexpectedly timed out while TouchTest Web clips were in action. 71235: (TouchTest) Unable to select same Device Cloud on different track The Composition Editor was treating device clouds like device agents and unexpectedly subjecting them to the same restrictions. 71233: (TouchTest) Ability to see DeviceType for a TTW Clip. [requires new TouchTestDriver (Android only)] The Device Type in TouchTest Web can now be seen. 71138: (TouchTest) outputEval for box missing float: left causes output value box to be on the left of Value label A layout issue occurred for a TouchTest Web result in the Summary panel of Result Details.

Copyright 2013. CloudTest is a registered trademark of SOASTA, Inc. and/or its affiliates. Other names may be trademarks of their respective owners.

55

Page 59: SOASTA CloudTest Release Notes Build 6806.68 Table of ...cdn.soasta.com/productresource/download/SOASTA_CloudTest...SOASTA CloudTest Release Notes Build 6806.68 Dec 12, 2013 Table

71091: (TouchTest) storeWebEval unexpectedly failed [requires new TouchTestDriver] The storeWebEval accessor unexpectedly failed to store an extracted property set's value. 70878: (TouchTest) Pre-processor didn't define TiUISearchBar or TiUIButton, preventing successful Android app build Now, TiUISearchBar and TiUIButton are dynamically defined if the preprocessor knows about them. 70752: (TouchTest) TouchTest Agent stuck in connected mode A TouchTest Agent indicated Connected status after a composition was force aborted, but the repository status was still Testing. 70605: (TouchTest) Bug in Master Detail Layout in iOS 7; [requires new TouchTest Driver] Due to a bug in iOS 7 with the master detail layout, on some devices using TouchTest, the UIWebView (detail contents) was shifted down from where it should be. This circumstance would cause the captured touch locators to be wrong (e.g. because TTD saw the UIWebView where it should be, rather than where it was). 70315: (TouchTest) Screenshot Validation Tabs Wrapping The three tabs in the Screenshot Validation form in Result Details were unexpectedly wrapping (e.g. the Diff tab would appear on a new row). 69650: (TouchTest) Don’t modify AppDelegate when handleOpenURL or openURL are commented out If the handleOpenURL or openURL method is commented out in a project's AppDelegate file, MATT will now display an error message and should not modify the AppDelegate file. 68968: (TouchTest) Auto Stop Time fail on mobile composition Additional null checking has been added to detect further occurrences of this issue. 67030: (TouchTest) Hybrid App records but does not play back A mobile test clip recorded from a given hybrid app wouldn't play back as expected.

66224: (TouchTest) If condition evaluates to be false when it should be

true The clip will now wait for the loading page to go away before evaluating the conditional. 65813: (TouchTest) Global Navigation menu issue

Copyright 2013. CloudTest is a registered trademark of SOASTA, Inc. and/or its affiliates. Other names may be trademarks of their respective owners.

56

Page 60: SOASTA CloudTest Release Notes Build 6806.68 Table of ...cdn.soasta.com/productresource/download/SOASTA_CloudTest...SOASTA CloudTest Release Notes Build 6806.68 Dec 12, 2013 Table

A tap on a button menu would succeed to expand a button menu but wouldn't succeed to collapse it.

mPulse Bugs Fixed

65961: (mPulse) Performance Breakdown triple chart stops polling In the browser that the request was made, not all of the responses were returned. Since a combined chart will not poll until it has received all of the data the issue really was that a response was lost. 71417: (mPulse) BarCharts series should not stack up on each other This fix resolves an occasional issue in which bar chart lines would appear stacked unexpectedly. 71438: (mPulse) At tilted camera views the heatmap surface can look clipped The Filled surface on large landmasses such as the US gets clipped prematurely. 71398: (mPulse) Hybrid layout isn't computing min path height when widget has filter bar The widget filter bar would cause unexpected layout issues. 71308: (mPulse) Heatmap Pop-ups position incorrect in recent Firefox The Heatmap label would popup in an incorrect position in Firefox 25. 71288: (mPulse) Framebuffer attachments causing CPU latency The mPulse Globe dashboard's performance was less than expected due to underlying CPU latency. 71279: (mPulse) Widgets are too tall when maximized or popped out Maximized or popped out widgets were too tall. 70720 (mPulse) JS error when opening newly imported operational dashboard This error would occur in Firefox 12 on Linux. 70178: (mPulse) Other dashboards polling every second for past days Dashboards other than the main mPulse dashboard were querying every second even when the current day was in the past. 70663: (mPulse) TimeInterval code assumes a day is 24 hours The Time Interval assumed a day is 24 hours long; however, this is not true of days when DST switches over.

Copyright 2013. CloudTest is a registered trademark of SOASTA, Inc. and/or its affiliates. Other names may be trademarks of their respective owners.

57

Page 61: SOASTA CloudTest Release Notes Build 6806.68 Table of ...cdn.soasta.com/productresource/download/SOASTA_CloudTest...SOASTA CloudTest Release Notes Build 6806.68 Dec 12, 2013 Table

69397: (mPulse) Tile Layers do not match for Globe/Columbus Views - Globe view is correct This fix adds zoomRatio offsets for both 3d/2d to be able to have some finer control on when the level loads. 69396: (mPulse) Globe sizing issue Opening a globe in half screen (e.g. via a custom dashboard layout) and then switching it to Columbus view would result in a flat earth that wasn't properly scaled. 69252: (mPulse) Multiple globes causes error and crash Running multiple instances of the Globe would cause a context clash resulting in an underlying memory leak. TouchTest failed to record a shake action that is expected to be recordable. 68738: (mPulse) Changing time zone also changes date if time zone is ahead of UTC Changing times and time zones in the UI wasn't doing what it should consistently.

66187: (CloudTest) TypeError: this.selected_cell is undefined; JS line 2630 An error occurred while selecting or unselecting widgets in the dashboard. 66030: (mPulse) Tenant list on mPulse can show duplicate entries In some cases with many tenants, the Tenant list could show duplicate entries. 63365: (mPulse) DWR Exception on Metrics Analysis dashboard when running in offline mode If network connectivity was missing, the attempt to access a given web host would cause this error whenever that connection failed.

Copyright 2013. CloudTest is a registered trademark of SOASTA, Inc. and/or its affiliates. Other names may be trademarks of their respective owners.

58