Download - 20110812 CyberTAN presentation
![Page 1: 20110812 CyberTAN presentation](https://reader033.vdocuments.net/reader033/viewer/2022060110/555e8068d8b42abd468b4b46/html5/thumbnails/1.jpg)
QualiSystems Proprietary & Confidential1
TestShell FrameworkIntroduction
Wednesday, April 12, 2023
Standards of E
xcellence
Richard Hsu
![Page 2: 20110812 CyberTAN presentation](https://reader033.vdocuments.net/reader033/viewer/2022060110/555e8068d8b42abd468b4b46/html5/thumbnails/2.jpg)
9001:2000 90003:2004
Global Software CompanySanta Clara, USALondon, UKMunich, GermanyTel-Aviv, Israel APAC
• Singapore , APAC HQ • Shanghai, China • Taipei, Taiwan• Seoul, Korea
• APAC Channels, Australia, Japan , India
Founded 2004
Flag product – TestShellTM Test-Automation Framework
“Customer First” Culture
QualiSystems – Company Overview
QualiSystems Proprietary & Confidential2
![Page 3: 20110812 CyberTAN presentation](https://reader033.vdocuments.net/reader033/viewer/2022060110/555e8068d8b42abd468b4b46/html5/thumbnails/3.jpg)
What we do - TestShell Framework
QualiSystems Proprietary & Confidential3
End-to-End framework for Test Automation that supports the entire life cycle of a test for components, devices, or systems
TestShell automates and simplifies the process of testing, allows you to focus on pure testing, and thus substantially:
Improve Product Quality
Shorten Time to Market
Reduce Expenditures
![Page 4: 20110812 CyberTAN presentation](https://reader033.vdocuments.net/reader033/viewer/2022060110/555e8068d8b42abd468b4b46/html5/thumbnails/4.jpg)
Lengthy test processes that can take up to several daysThe wide array of benchmark applications/Scripts include Hardware and SoftwareThe variety of vendors and models requiring diverse specificationsThe need to control and manage many distributed test stations Time-consuming test result collation and analysisCompound aggregation of extensive amounts of test data
Testing Challenges without Automation
QualiSystems Proprietary & Confidential4
Time Lost
Quality Impacted
Expense Overhead
Resources Unexploited
![Page 5: 20110812 CyberTAN presentation](https://reader033.vdocuments.net/reader033/viewer/2022060110/555e8068d8b42abd468b4b46/html5/thumbnails/5.jpg)
Significantly reduce Time-To-Market Full regression cycle 64 hours → 24 hours at USI (環隆電氣 )Single test cycle 15 days → 1.5 day at Cameo (友勁科技 )
Significantly reduce Operational ExpendituresResources involved in test creation down by 50% at Deutsche TelekomSingle test cycle effort 2 Days → 5 hours at Telstra
Managing and Improving Quality ControlDaily Build and regression at Ceragon NetworksCustomer service centers at CrowSaving X10 on RMA at Syneron
Proven ROI within the first year of use
QualiSystems Proprietary & Confidential5
![Page 6: 20110812 CyberTAN presentation](https://reader033.vdocuments.net/reader033/viewer/2022060110/555e8068d8b42abd468b4b46/html5/thumbnails/6.jpg)
QualiSystems Proprietary & Confidential6
Our Customers – across the value chain
![Page 7: 20110812 CyberTAN presentation](https://reader033.vdocuments.net/reader033/viewer/2022060110/555e8068d8b42abd468b4b46/html5/thumbnails/7.jpg)
End-to-End Test Automation Framework
7 QualiSystems Proprietary & Confidential
Write Tests & Regressions
Create reusable functions
Control Test Assets
Coordinate Test Execution
Launch Test Stations
Run Tests & Regressions
Collect Logs & Results
Monitor Real-time
Aggregate Results
Customized Reports
Customized Dashboards
Create
ExecuteSchedule
Analyze
CentralManagement
Tests & Regressions
Assets
Equipment
Test Results
![Page 8: 20110812 CyberTAN presentation](https://reader033.vdocuments.net/reader033/viewer/2022060110/555e8068d8b42abd468b4b46/html5/thumbnails/8.jpg)
TestShell Platform Details
QualiSystems Proprietary & Confidential8
![Page 9: 20110812 CyberTAN presentation](https://reader033.vdocuments.net/reader033/viewer/2022060110/555e8068d8b42abd468b4b46/html5/thumbnails/9.jpg)
End-to-End Test Automation Framework
9 QualiSystems Proprietary & Confidential
Create
ExecuteSchedule
Analyze
CentralManagement
![Page 10: 20110812 CyberTAN presentation](https://reader033.vdocuments.net/reader033/viewer/2022060110/555e8068d8b42abd468b4b46/html5/thumbnails/10.jpg)
• Messages, Instructions• Alerts, Remarks• Watchers, Graphs• etc.
Dialogs
• Variables• Cases, Loops, Delays• Parallelization, events• etc.
Logic Modules
• Traffic generators• Network analyzers• Spectrum, scope • Generators, etc.
T&M Modules
• Symbian• Windows Mobile• iPhone, Blackberry• Android
Mobile Modules
• Mail & SMS, File Engine• GUI Automation, Attributes• Requirements Management,
QTP ,LR , Etc..
Utilities
• Telnet, SSH, TCP, UDP• SNMP (& Traps), FTP• Traffic Generators, Switches• etc.
Networking
• Web GUI• Web Services• etc.
Web Modules
• Exe, DLLs, CLI, VIs• Script-Engines• TCL, Perl, Python, VB, Java• MatLab, OCR
Scripts & Software
TestShell SystemInterfaces to Software and hardware compnents (Any many more..)
![Page 11: 20110812 CyberTAN presentation](https://reader033.vdocuments.net/reader033/viewer/2022060110/555e8068d8b42abd468b4b46/html5/thumbnails/11.jpg)
QualiSystems Proprietary & Confidential11
Testing in the All IP World - Example Scenario
Typical test flow for validating :
Allocate all testing resources
Configure the Device Under Test
Configure Traffic Generator(s)
Applying traffic to the DUT and
sniffing /getting statistics
Analyzing
Typical tests:Traffic blasting, protocol emulation, load & stress,
conformance , acceptance, interoperability, etc
Typical validations:
End-to-End: Path Recovery, Clock Recovery,
Delay, Latency, Throughput, Loss Ratio,
Information-leakage, routing/switching performance, etc
Start
Setup Topology
Run Traffic
Analyze Results
End
Configure Traffic Generator (ex. IXIA
, SPIRENT)
Configure DUT
![Page 12: 20110812 CyberTAN presentation](https://reader033.vdocuments.net/reader033/viewer/2022060110/555e8068d8b42abd468b4b46/html5/thumbnails/12.jpg)
QualiSystems Proprietary & Confidential12
Testing in the All IP World - Example Scenario
Multiple Network Elements from different vendors such as:
Ethernet SwitchOSSWireless devices (Access Points)WiMAX/4G (STB)Multi Service RouterHigh Speed RouterBBRARDSLAMGatewayModem
Start
Setup Topology
Run Traffic
Analyze Results
Store & Report Data
End
Configure Traffic
Configure DUT
![Page 13: 20110812 CyberTAN presentation](https://reader033.vdocuments.net/reader033/viewer/2022060110/555e8068d8b42abd468b4b46/html5/thumbnails/13.jpg)
QualiSystems Proprietary & Confidential13
Testing in the All IP World - Example Scenario
Various network protocols and interfaces
Start
Setup Topology
Run Traffic
Analyze Results
Store & Report Data
End
Configure Traffic
Configure DUT
TelnetSSH
SFTP
TCP
Serial
UDP
SNMP
SNMP Traps
GPIB
HTTP
VISA
SOAP
SMTP
FTP
TFTP
![Page 14: 20110812 CyberTAN presentation](https://reader033.vdocuments.net/reader033/viewer/2022060110/555e8068d8b42abd468b4b46/html5/thumbnails/14.jpg)
QualiSystems Proprietary & Confidential14
Testing in the All IP World - Example Scenario
Multiple test equipment typesfrom different vendors
Traffic GeneratorsNetwork EmulatorsNetwork SniffersNetwork Analyzers Baseband EmulatorsLogic analyzersSignal analyzersOscilloscopesSignal generators
Start
Setup Topology
Run Traffic
Analyze Results
Store & Report Data
End
Configure Traffic
Configure DUT
![Page 15: 20110812 CyberTAN presentation](https://reader033.vdocuments.net/reader033/viewer/2022060110/555e8068d8b42abd468b4b46/html5/thumbnails/15.jpg)
QualiSystems Proprietary & Confidential15
Testing in the All IP World - Example Scenario
DEMO
![Page 16: 20110812 CyberTAN presentation](https://reader033.vdocuments.net/reader033/viewer/2022060110/555e8068d8b42abd468b4b46/html5/thumbnails/16.jpg)
TestShell Common Scenario for Network
QualiSystems Proprietary & Confidential16
![Page 17: 20110812 CyberTAN presentation](https://reader033.vdocuments.net/reader033/viewer/2022060110/555e8068d8b42abd468b4b46/html5/thumbnails/17.jpg)
DEMO 1: Router Testing via Traffic Generator- Configure DUT by Telnet
Check aliveCommand line
Configure Router/Switch
TelnetRun TrafficDriver
Analyze ResultsDriver
Traffic Sender
Traffic Receiver
DataData
Rou
ter
![Page 18: 20110812 CyberTAN presentation](https://reader033.vdocuments.net/reader033/viewer/2022060110/555e8068d8b42abd468b4b46/html5/thumbnails/18.jpg)
DEMO 1: Router Testing via Traffic Generator (cont.)Code Free Test Creation as easy as Visio
Studio FlowVisio Flow
18 QualiSystems Proprietary & Confidential
![Page 19: 20110812 CyberTAN presentation](https://reader033.vdocuments.net/reader033/viewer/2022060110/555e8068d8b42abd468b4b46/html5/thumbnails/19.jpg)
DEMO 2: AP Router Testing via Traffic Generator- Configure DUT by Website(GUI)
Check aliveCommand line
Configure RouterWebsite(GUI)
Run TrafficDriver
Analyze ResultsDriver
Traffic Sender
Traffic Receiver
DataData
![Page 20: 20110812 CyberTAN presentation](https://reader033.vdocuments.net/reader033/viewer/2022060110/555e8068d8b42abd468b4b46/html5/thumbnails/20.jpg)
Special features for Wireless Device:1. Use different configuration files to complete
diverse testing, ex. different mode(802.11n,b,g), different channel(1~11) or different encryption type(WEP, TKIP).
2. Use excel file to maintain the path of configuration files and import data of excel file by “File Engine” on Testshell.
3. TestShell’s GUI Automator can provide “Inspect” and “Valuate” feature to check states of web screen.
4. TestShell can get snapshoot of testing result and export to report automatically.
DEMO 2: AP Router Testing via Traffic Generator (cont.)
- Configure DUT by Website(GUI)
QualiSystems Proprietary & Confidential20
![Page 21: 20110812 CyberTAN presentation](https://reader033.vdocuments.net/reader033/viewer/2022060110/555e8068d8b42abd468b4b46/html5/thumbnails/21.jpg)
Drag & Drop Test Creator – non programmers can join the automationReusable Building BlocksRecord and Replay ActionsIntegration with both Traffic Generators and T&MSupported interfaces:Network Protocols (Telnet, SSH, Serial, TCP, UDP, SNMP, etc)GUI – Web, Java, Win appLogic and DialogIntegration with external files (configuration and results)Resource Asset Management & DeploymentDocumentationScriptsExecutablesDlls
Tests Creation - Benefits
![Page 22: 20110812 CyberTAN presentation](https://reader033.vdocuments.net/reader033/viewer/2022060110/555e8068d8b42abd468b4b46/html5/thumbnails/22.jpg)
End-to-End Test Automation Framework
22 QualiSystems Proprietary & Confidential
Create
ExecuteSchedule
Analyze
CentralManagement
![Page 23: 20110812 CyberTAN presentation](https://reader033.vdocuments.net/reader033/viewer/2022060110/555e8068d8b42abd468b4b46/html5/thumbnails/23.jpg)
QualiSystems Proprietary & Confidential23
Scheduler Snapshot
Tests scheduling – overnight & weekend
Resource management & Equipment utilization
![Page 24: 20110812 CyberTAN presentation](https://reader033.vdocuments.net/reader033/viewer/2022060110/555e8068d8b42abd468b4b46/html5/thumbnails/24.jpg)
Automatic scheduling according to time, station and priority
Test scheduling across single or multiple stations
Advanced "Outlook-like" interface
Virtual station locking
Fully customizable execution rules
Automatic real-time execution conflict handling
Tests Scheduling – Scheduler
![Page 25: 20110812 CyberTAN presentation](https://reader033.vdocuments.net/reader033/viewer/2022060110/555e8068d8b42abd468b4b46/html5/thumbnails/25.jpg)
End-to-End Test Automation Framework
25 QualiSystems Proprietary & Confidential
Create
ExecuteSchedule
Analyze
CentralManagement
![Page 26: 20110812 CyberTAN presentation](https://reader033.vdocuments.net/reader033/viewer/2022060110/555e8068d8b42abd468b4b46/html5/thumbnails/26.jpg)
Execution planningSelect Test or RegressionSelect Unit Under Test and TopologyRecord Admin informationSerial, Version, Batch etc… (Dynamic)
RunComplete RunStep by StepAutomatic troubleshooting and action executionReal-time data tracking
Standardize data collectionData transfer guarantee
Tests Execution – Runner
![Page 27: 20110812 CyberTAN presentation](https://reader033.vdocuments.net/reader033/viewer/2022060110/555e8068d8b42abd468b4b46/html5/thumbnails/27.jpg)
End-to-End Test Automation Framework
27 QualiSystems Proprietary & Confidential
Create
ExecuteSchedule
Analyze
CentralManagement
![Page 28: 20110812 CyberTAN presentation](https://reader033.vdocuments.net/reader033/viewer/2022060110/555e8068d8b42abd468b4b46/html5/thumbnails/28.jpg)
Melbourne
3,3 V
12 mA
51 kb/s81,9101,3
MHz3E-07128 ms
21°C69%11,9 mA
Test Analysis - InSightCentral data collection
5,3Canbara3,2 V
57 kb/sSydney
21,6°C19 W102,1
MHz
Multi-Site Repository
Test Lab 1 Test Lab nTest Lab 2
Data from all test labs and stations is collected and stored in a central server
Data is then ready for comprehensive analysis, using TestShell InSight
www
![Page 29: 20110812 CyberTAN presentation](https://reader033.vdocuments.net/reader033/viewer/2022060110/555e8068d8b42abd468b4b46/html5/thumbnails/29.jpg)
Test Analysis - InSightOnline customizable display
The analayzed data is presented via online Dasboards:• Providing real time analysis• Reviewed any time anywhere
www
The dashboards are completely customizable according to user preferences, supporting multiple analysis & display options, including OLAP
![Page 30: 20110812 CyberTAN presentation](https://reader033.vdocuments.net/reader033/viewer/2022060110/555e8068d8b42abd468b4b46/html5/thumbnails/30.jpg)
End-to-End Test Automation Framework
30 QualiSystems Proprietary & Confidential
Create
ExecuteSchedule
Analyze
CentralManagement
![Page 31: 20110812 CyberTAN presentation](https://reader033.vdocuments.net/reader033/viewer/2022060110/555e8068d8b42abd468b4b46/html5/thumbnails/31.jpg)
Test Management modules - Snapshot
QualiSystems Proprietary & Confidential31 QualiSystems Proprietary & Confidential
Central repository across the organization
Tests and Regression management
DUT management
Fully scalable and open frameworkTests version control Permission and user control
![Page 32: 20110812 CyberTAN presentation](https://reader033.vdocuments.net/reader033/viewer/2022060110/555e8068d8b42abd468b4b46/html5/thumbnails/32.jpg)
TestShell Components
QualiSystems Proprietary & Confidential32
![Page 33: 20110812 CyberTAN presentation](https://reader033.vdocuments.net/reader033/viewer/2022060110/555e8068d8b42abd468b4b46/html5/thumbnails/33.jpg)
TestShell Applications
QualiSystems Proprietary & Confidential33
TestShell Studio
TestShell Runner
TestShell InSight
TestShell Server
Create
Execute
Analyze
ManageTestShell SchedulerTestShell Controller
![Page 34: 20110812 CyberTAN presentation](https://reader033.vdocuments.net/reader033/viewer/2022060110/555e8068d8b42abd468b4b46/html5/thumbnails/34.jpg)
QualiSystems Proprietary & Confidential34
Thank you
To Schedule a Live Demonstration go to: www.qualisystems.com/demo