fact checking the internet of things€¦ · fact checking the internet of things . sanil pillai,...

Fact Checking the Internet of Things Sanil Pillai, Director of Infostretch Labs, Infostretch

©2017 Infostretch. All rights reserved. 2

Agenda

• Overview of the IoT Market• Key Components of IoT• The IoT Landscape• Testing Challenges in IoT• IoT Testing Focus• IoT Testing Framework• Where are bugs in an IoT system?• Testing Approach for Internet of Things


Overview of IoT Market

Enterprise leaders that embrace IoT could disrupt other enterprises and achieve unprecedented growth

By 2020, over 20 billion connected things will be in use across a range of industries; the IoT will touch every role

across the enterprise

Source: Gartner


Key Components of IoT

Computing Network

Things

Combine devices with processors, sensors and software to collect and process data

Interaction amongst connected Things as well as external networks using ZigBee, BLE, Wi-Fi network

Leverage the data collected from sensors to offer augmented services using Analytics and Computation


Source: Forrester

It’s All About Connectivity


Testing Challenges in IoT

Support for High Volume and

Velocity of Data

Testing Scenarios involve multiple Users and

Devices

Handling of Big Data

Use Case Complexity

Security & Privacy

Securing the Private and Sensitive User Data

(Healthcare, Payments)

Rapid Response Time, Handling Diverse Data and Local processing

Performance Requirements

Consistent User Experience

Ensuring Uniform User Experience for All Devices

and Channels


MVP Development and Prototypes MVP Development and PrototypesEnd to End Application Development

Where are bugs in an IoT system?


Data Logging and Monitoring Layer TestingSensors Input to Controllers

• What needs to be tested?• Specific Sensor Stimulus

• Can it detect the change in the physical stimulus and turn that into signal that is measurable?• Power Management

• Battery drain rate and application specific usage• Digital Protocol Engines

• Simulate network communication to the sensors through different protocols

• Automation of Data Logging & Monitoring Layer Testing• Using a Humanoid to perform the actions like changing of location to test proximity sensor,

applying pressure to test the pressure sensor, move to test accelerometer, etc.


Processing and Controlling Layer TestingProcessors, Controllers and Operating System

• What needs to be tested?• Edge Processing Capability

• Check whether the sensor data can be processed by the edge devices efficiently• Power Consumption and Efficiency

• Determine Battery life as per the power consumption and check heat dissipation of the processor • Safety and Security

• Ensure that the device adheres to the safety and security compliances e.g.: health, finance, etc.

• Automation of Processing and Controlling Layer Testing• Simulators can be used for testing the processing capability and execution of time of programs

during early phases of testing. Automate the testing of power consumption for various scenarios: searching for connections, connected to the network, idle phase and data transfer phase


RF Communication Layer TestingM2M Protocols for different Ranges

• What needs to be tested?• RF Parameters

• RF conformance testing and fading scenarios• Antenna performance

• Antenna Sensitivity and Power radiation• Handovers

• Handover and Complex signaling scenarios • Interference

• Interference of signals in cases of In-device coexistence and Antenna colocation

• Automation of RF Communication Layer Testing• Testing of an IoT application by emulation of different network sizes, topology and environment

conditions


Session and Messaging Layer TestingMessaging Sessions

• What Needs to be Tested?• Load Testing

• Simulate number of virtual users running test case on multiple servers • Publish and Receive Messages

• Establish MQTT connection & publish message as specified• Subscribe MQTT topic and wait for first message

• Drop Connection• Monitor the disconnection with server – normal way, unexpectedly, or intentionally

• Automation of Session and Messaging Layer Testing• Using Protocol simulators with a variety of devices and interfaces to record and validate packet

exchanges from real servers/sensors/devices


Solution Layer TestingPresentation, Visualization and User Experience

• What Needs to be Tested?• Encryption and Decryption

• Ensure the data communication is fully encrypted and secure• Validation of Data Integrity

• Ensure the data conversion is in a valid data format• Data Compression

• Ensure that data compressed in an optimized manner to ensure maximum throughput

• Automation of Solution Layer Testing• Service virtualization for automated generation and testing of device interaction data


Application Layer TestingBusiness Cases per Vertical

• What Needs to be Tested?• Different Compliance Need

• Medical Devices (MDDS Class 1 or 2) that needs to be complied to US FDA, etc.• Data Integration

• Patient’s vital coming from different connected devices - e.g. Weighing scale, Blood Pressure, Glucose monitor, etc.

• Data Mashups• Different data converted into meaningful dashboard and appropriate triggers/alerts

• Automation of Application Layer Testing• Automating the testing via spoofing different data sources to test Data Integration & Data

Mashups

©2017 Infostretch. All rights reserved. 14©2017 Infostretch Corporation. All rights reserved. 14

Testing Approaches for Internet of Things


Camera based Tracking for Data

Validation

Automated Data Calculation through

Video Analysis

Verify Multiple Parameters for Various Sensors

Challenges

Algorithm Validation to determine Effectiveness

Determining Sensor Accuracy and Noise

Elimination

Measurement of Various Performance Parameters

Solutions

Sensor Validation


Automated Testing Action

Camera based Measurement of

actual Data Values

Cross Verification of Sensor Data

Sensor Data Recorded

Sensor Validation Using Motion Capture Cameras


Device SimulationTesting Device through Simulation on Cloud

Microcontroller for simulation of Smart Devices

Test Multiple Environments

through simulation

Simulate User Interaction through

Virtual Sensors

Challenges

Reduced focus on Product Development

Lack of Standardization: Complex and Evolving

Ecosystem

Pressure of Faster Time to Market

Solution


Smart Device Under Testing

Synchronization

Device SimulationTesting Device through Simulation on Cloud

Cloud Device Testing Server

Microcontroller for Device Simulation

Configuration

Connectivity Data Manager


Instrumented ApplicationsAutomating Library based Test Scenarios for IoT

Automate Test Scenarios for Device

Hardware

Uniform Test Flow for all Devices: Optimize Effort

Optimize Test Suite Execution Process:

Save Time

Challenges

Testing of IoT applications Costly and Time Consuming

Existing Automation Frameworks dependent on

UI based Testing

Hardware Simulation Cost Prohibitive or not at all

possible in some Scenarios

Limited Support for IoT Applications in Cradle based

Cloud Testing

Solutions


Instrumented ApplicationsAutomating Library based Test Scenarios for IoT

Continuous Integration Server

App Binary

Automation Configuration

Specify Library Modules

Modified App Binary

Device Cloud Server

Location Camera Touch

BLE Date & Time Payment

Automation Script


ConditioningRobotic Process Automation for Conditioning Devices

Challenges

Significant Time and Effort Required

Obstacles Impact on the Range

Contextual Messaging

Interface and Overlap

Solutions

Assessment of Signal Strength

Proximity Validation

Automated Defined Path based Testing

Test Device Identification Accuracy

Configuration Management: Advertising, Interval, Power


BLE Testing Using Robotic Process Automation:

Beacon 1 Detected

Beacon 3 Detected

Beacon 2 DetectedTurtle Bot

Conditioning

CONNECT WITH [email protected]+1-408-727-1100

Thank you

fact checking the internet of things€¦ · fact checking the internet of things . sanil pillai,...

Documents