these materials are © 2018 john wiley & sons, inc. any ...€¦ · any dissemination,...


Industrial Internet of Things for Developers

by Ryane Bohm


Industrial Internet of Things for DevelopersPublished by John Wiley & Sons, Inc. 111 River St. Hoboken, NJ 07030-5774 www.wiley.com

Copyright © 2018 by John Wiley & Sons, Inc., Hoboken, New Jersey

No part of this publication may be reproduced, stored in a retrieval system or transmitted in any form or by any means, electronic, mechanical, photocopying, recording, scanning or otherwise, except as permitted under Sections 107 or 108 of the 1976 United States Copyright Act, without the prior written permission of the Publisher. Requests to the Publisher for permission should be addressed to the Permissions Department, John Wiley & Sons, Inc., 111 River Street, Hoboken, NJ 07030, (201) 748-6011, fax (201) 748-6008, or online at http://www.wiley.com/go/permissions.

Trademarks: Wiley and related trade dress are trademarks or registered trademarks of John Wiley & Sons, Inc. and/or its affiliates in the United States and other countries, and may not be used without written permission. All other trade-marks are the property of their respective owners. John Wiley & Sons, Inc., is not associated with any product or vendor mentioned in this book.

LIMIT OF LIABILITY/DISCLAIMER OF WARRANTY: THE PUBLISHER AND THE AUTHOR MAKE NO REPRESENTATIONS OR WARRANTIES WITH RESPECT TO THE ACCURACY OR COMPLETENESS OF THE CONTENTS OF THIS WORK AND SPECIFICALLY DISCLAIM ALL WARRANTIES, INCLUDING WITHOUT LIMITATION WARRANTIES OF FITNESS FOR A PARTICULAR PURPOSE. NO WARRANTY MAY BE CREATED OR EXTENDED BY SALES OR PROMOTIONAL MATERIALS. THE ADVICE AND STRATEGIES CONTAINED HEREIN MAY NOT BE SUITABLE FOR EVERY SITUATION. THIS WORK IS SOLD WITH THE UNDERSTANDING THAT THE PUBLISHER IS NOT ENGAGED IN RENDERING LEGAL, ACCOUNTING, OR OTHER PROFESSIONAL SERVICES. IF PROFESSIONAL ASSISTANCE IS REQUIRED, THE SERVICES OF A COMPETENT PROFESSIONAL PERSON SHOULD BE SOUGHT. NEITHER THE PUBLISHER NOR THE AUTHOR SHALL BE LIABLE FOR DAMAGES ARISING HEREFROM. THE FACT THAT AN ORGANIZATION OR WEBSITE IS REFERRED TO IN THIS WORK AS A CITATION AND/OR A POTENTIAL SOURCE OF FURTHER INFORMATION DOES NOT MEAN THAT THE AUTHOR OR THE PUBLISHER ENDORSES THE INFORMATION THE ORGANIZATION OR WEBSITE MAY PROVIDE OR RECOMMENDATIONS IT MAY MAKE. FURTHER, READERS SHOULD BE AWARE THAT INTERNET WEBSITES LISTED IN THIS WORK MAY HAVE CHANGED OR DISAPPEARED BETWEEN WHEN THIS WORK WAS WRITTEN AND WHEN IT IS READ.

ISBN 978-1-119-45693-3 (pbk); ISBN 978-1-119-45692-6 (ebk)

Manufactured in the United States of America

10 9 8 7 6 5 4 3 2 1

Publisher’s Acknowledgments

For general information on our other products and services, or how to create a custom book for your business or organization, please contact our Business Development Department in the U.S. at 877-409-4177, contact [email protected], or visit www.wiley.com/go/custompub. Some of the people who helped bring this book to market include the following:

Project Manager and Development Editor: Chad R. Sievers

Executive Editor: Katie Mohr

Editorial Manager: Rev Mengle

Business Development Representative: Karen Hattan

Custom Publishing Project Specialist: Michael Sullivan

Production Editor: Vasanth Koilraj

http://www.wiley.com

http://www.wiley.com/go/permissions

http://www.wiley.com/go/permissions

mailto:[email protected]


http://www.wiley.com/go/custompub

http://www.wiley.com/go/custompub

1


Preface

Developers have become heroes in the popular imagination. They’re seen as having the imagination and ability to bring new ideas to life. When you think of the iconic garages in Silicon Valley, the engineers are the ones who made the magic, as they did at HP and Apple. In today’s world, developers understand the APIs, the platforms, and the way the appli-cations are packaged for customers. Many of today’s largest companies, ranked by market value, began in the minds of developers.

As the opportunities facing developers evolve, so does the nature of the heroes required. This book explains how you can get started in writing applications that tap into the massive potential of the Industrial Internet of Things (IIoT). To do that, you’ll need to reimagine your role as a developer and expand your understanding of the supporting environment.

The Opportunity, The ChallengeFew doubt the size of the IIoT opportunity. Technologist Tim O’Reilly says that the IIoT is the largest and most underesti-mated opportunity in Silicon Valley: “Obviously, Silicon Valley is all over this,” O’Reilly said, speaking of the proliferation of narrowly defined consumer-oriented IoT devices and applica-tions. “But I think they are missing the point. They are creating some gadgets, but they aren’t thinking about systems.”1

The IIoT is all about systems, the systems that manage flows of goods, chemicals, energy, and manufacturing processes that run the world. It’s about improving health and safety, connect-ing high-tech equipment in hospitals, and ensuring safety on airlines and rail lines around the world.

2 Industrial Internet of Things for Developers


The IIoT market is larger than the consumer IoT market. Estimates indicate the IIoT could be a $225 billion market by 2020, compared to a $170 billion consumer market.2 Further, in the coming years, the Industrial Internet could add an addi-tional $10 to 15 trillion to the global GDP—an amount equiva-lent to the size of today’s US economy.3

Seeing the opportunity is easy. What is less well under-stood, and more difficult to explain, is that the IIoT requires a new kind of developer willing to be immersed in a highly com-plex, challenging industrial environment, ready to be stretched in ways never dreamed of, all to create applications that will produce more of the stunning returns seen in the early days of the IIoT.

The World of IIoT AppsEvery day, exabytes of data—and lost opportunities—are left on the industrial floor, on the ground in oil fields, and in the air amid wind turbines. On the factory floor, machines have had interfaces since the 1970s. In hospitals, nearly every piece of equipment has an interface, but very few have been connected, let alone interconnected. More recently, as industrial control systems were introduced, they used machine data in limited ways for specific, repetitive tasks.

Opportunities abound for building applications to lever-age this data broadly. If you listen, machines can tell you that they’re running too hot, that they’re running more efficiently than the machine on the next line, and whether they need maintenance within a few days.

But in most industrial landscapes, the data that could transform the assets, systems, and businesses goes unana-lyzed. When developers imagine and implement applications that use this data, the transformation begins.

3Preface


The Playing Field for IIoT ApplicationsThe environment for IIoT applications is far different than that used for the sort of applications that are created for web-sites, mobile apps, or even the consumer IoT along a variety of dimensions. Here are some of the differences:

• The stakes are higher. If your phone drops a call, you get irritated. But you can’t just turn off the Hoover Dam. Similarly, if your FitBit doesn’t accurately count the number of steps you took today, you may not even notice. But if a power plant goes down, thousands lose electricity.

• IIoT applications must address operational technology requirements. Operational technology (OT) includes systems engaged in electricity generation and distribu-tion as well as manufacturing systems that must remain reliable and secure to avoid potential damage to people and equipment.

• IIoT applications require a broad range of skills. Because IIoT applications involve equipment that con-trols flows of chemicals, energy, or physical items, the

Some of the problems IIoT applications can solve

Here is just a sampling of problems that IIoT apps can solve:

✓✓ Providing visibility to remote equipment, production pro-cesses, and energy use

✓✓ Preventing unplanned downtime

✓✓ Optimizing the operation of an individual asset in the field

✓✓ Optimizing the operation of a process or plant in the field



skills to create an application include software develop-ment, domain knowledge of physics, disciplines such as chemical or mechanical engineering, and OT.

• IIoT applications are intelligent learning systems. The volume, variety, and velocity of big data in the IIoT dwarf most other realms and represent a theme park for data science. The complexity of equipment and indus-trial processes and the task of modeling and extracting signals and interpreting them may require the use of artificial intelligence. Machine learning, deep learning, and other techniques can extract meaning from data and help create and tune predictive models. IIoT apps that aren’t built to learn have trouble keeping up with changing conditions and evolving interactions.

Although many of these aspects are found in some con-sumer IoT applications and the full stack applications of the Internet, in the IIoT these requirements are present most of the time. The IIoT will clearly change the role of both applications and the development teams who create them.

The Evolution of IIoT ApplicationsThe pioneers of the IIoT have already shown how the power of IIoT applications builds up to allow larger transformations of both businesses and operating environments. Pitney Bowes, which sells and services large mail inserting machines, went through the following stages in creating IIoT applications:

• Internal productivity and efficiency: The first wave of applications enabled improved operations, better uptime, and a variety of optimizations that could never previously be performed. For Pitney Bowes, its

5Preface


machines could be run with greater efficiency and pro-vide more useful data.

• Improved customer service: After the OT environ-ment becomes an open book for a company, it’s pos-sible to share that visibility to allow customers to better understand and optimize how their equipment works for them. New types of visibility lead to new types of services. For example, Pitney Bowes created software for its customers, following the pattern that every com-pany is a software company. The Pitney Bowes Clarity application gives customers more visibility, control, and value from their Pitney Bowes equipment.

• New digital revenue streams: The final evolution is when a company takes the capabilities it has developed and turns those capabilities into products for others to use. Pitney Bowes created a service for location intelli-gence that is now available to developers through GE’s Predix platform. In this way, innovations can be mon-etized as products and services. Other developers can widely reuse those innovations, accelerating their abil-ity to build apps for the IIoT.

The App Needs a TeamEven the most advanced software development teams who can build web, mobile, and consumer IoT applications based on existing APIs don’t yet have the skills to build the IIoT appli-cations that we’ve described. But this book, and the lessons you’ll be inspired to learn and apply on your own, will help you gain the skills and understanding you need to succeed. This book is a practical guide for anyone interested in develop-ing IIoT applications.



IIoT applications are often built jointly by subject matter experts and full stack developers. Webscale apps are typically built by a team of developers and ops people for backend services, the UX layer, and scalability. For the IIoT, that team expands. The opportunity and its attendant urgency require all hands on deck. Sometimes OT subject matter experts—who can contribute as citizen developers4—may create apps themselves using low code or domain-specific languages. To accelerate progress, IIoT development environments should support both full stack developers and citizen developers with various types of expertise.

Here are some of the types of developers and experts who participate in creating IIoT applications:

• Asset modelers

• Control and process engineers

• Data architects

• Data scientists

• Domain experts

• Field engineers

• Full stack developers

• Industrial automation specialists

• IoT platform architects

• Network architects

• OT experts

• Plant managers

• Security specialists

• Software architects

• Solution architects

• UX designers

In explaining the range of skills needed, I’m not abandon-ing the idea of the heroic vision of the developer. Most applica-tions still come to life as an insight in the mind of an individual who then develops the idea with the help of others.

In the IIoT, full stack developers need to understand OT environments, equipment, data and processes, edge-to-cloud architectures, and Digital Twins. Just as a graduate student in geology may learn R to do a master’s thesis, a domain expert

7Preface


may need to learn about IIoT development platforms, whether using low-code or full stack methods. OT developers need to widen their knowledge as well.

By expanding understanding in this way, developers can play a vital role in envisioning and creating transformative applications for the IIoT.

The Team Needs a PlatformThe final element developers need is a platform to productize the integration and collaboration between all these experts. Broadly speaking, a platform is a set of interconnected ser-vices designed to enable all the parts of an application to work together.

These platforms not only provide basic services, but they also create ecosystems so that new components can be added and components can be offered for sale. Just as mobile apps became a way for developers to monetize their talents, IIoT platforms will allow developers and vendors to fill key gaps and make money by selling their wares.

The IIoT has spawned a variety of platforms, including GE’s Predix, which serves as a reference architecture and a running example in this book. One important goal of this book is to explain the nature of a modern IIoT platform and show what it will do for developers.

Bringing Imagination to LifeAfter the full set of skills required to create an IIoT applica-tion is set forth, it may seem impossible for one person to ever master them all. In a way, it is. If developers think of their expertise in terms of height and width, most are tall in one area and shorter in many other areas.



The goal of this book is to help developers gain a broader understanding of OT domains so they can use those insights to build applications for the industrial world. I’ll cover how IIoT applications are constructed and built and how to get started.

IIoT platforms can be architected in a variety of ways, but one common pattern for distributed IIoT architectures is the edge-to-cloud pattern used by Predix and described in this book. To conceptualize this pattern, think of a barbell. The edge is the left side of a barbell, anchored in the world of OT where data can be consumed from sensors, controllers, and industrial equipment. The right side of the barbell is the cloud platform. These two software stacks are in a dance, with many considerations driving what types of processing and analytics will be done at the edge and what will be done in the cloud.

Decisions about how an IIoT application will use such a distributed architecture greatly vary by the use case. For that reason, this book presents design patterns to inspire your thinking about how to build your own IIoT applications.

Chapter 1 describes the world of operational technology and key differences between the world of OT and IT. It covers application design patterns focused on the edge.

Chapter 2 describes an architecture for IIoT applications and the characteristics of an IIoT platform. It describes how IIoT platform services can accelerate development.

Chapter 3 explains the building blocks of Digital Twins, which are full digital representations of individual physical systems and all the data related to them over time. Digital Twins can be used in many ways because they capture the real-world operations of a particular component or asset.

Chapter 4 offers inspiration and design considerations for building your own IIoT application. It includes topics such as assembling your team, collaborating, doing field research,

9Preface


designing for the OT world, considering tradeoffs that inform edge-versus-cloud decisions, and more.

I’m excited about the potential of IIoT applications. All over the world, companies, governments, and communities are waking up to what is possible. They all need developers to make it happen. That’s you, so let’s get started.

Resources for This BookThroughout this book, you’ll find references to tutorials, videos, case studies, and other resources where you can dis-cover more about the IIoT and get hands-on experience. Visit ge.com/digital/iiot-for-developers for chapter-by- chapter links.

AcknowledgmentsA book is never a solo effort. I want to thank Vineet Banga, Samta Bansal, Rich Carpenter, Susheel Choudhari, Kevin Collins, Dan Harrelson, Jean Lau, Rebecca Lawson, John Magee, Steve Rokov, Marc-Thomas Schmidt, Tom Turner, and Dimitri Volkmann who generously gave of their time and tal-ents by providing interviews and sharing important resources, in some cases weighing in with multiple rounds of reviews. Sourabh Dash, Joel Markham, and Achalesh Pandey reviewed the manuscript under extremely tight deadlines, offering important feedback and guidance. Daniel Erwin and Joanne Mendel supplied a process diagram to illustrate the teamwork involved in creating an IIoT application. I especially want to thank Jayson DeLancey who helped us at every stage of creat-ing this book, from the first conversation to outlines to drafts to final review.

http://www.ge.com/digital/iiot-for-developers



Several people shared what drew them to the IIoT, includ-ing John Andrechak, Andy Cash, Jayson DeLancey, Jay Lakumb, Girish Modgil, Paul Park, Shamla Soans, and Tom Turner. Their stories appear as sidebars throughout the book.

Most of all, I want to thank the Predix community, which is building IIoT applications at a rapid pace, for sharing their experience and use cases.

Endnotes

1. Chris O’Brien, “Tim O’Reilly: Silicon Valley is massively underestimating the impact of IoT (interview),” VentureBeat, March 04, 2015, accessed July 18, 2017, https://venturebeat.com/2015/03/04/tim-oreilly-silicon-valley-is- massively-underestimating-the-impact-of-iot-interview/.

2. https://gereports.ca/new-industrial-internet-report-from-ge-finds-that- combination-of-networks-and-machines-could-add-10-to-15-trillion-to- global-gdp/

3. https://www.ge.com/digital/blog/everything-you-need-know-about-industrial-internet-things

4. “Citizen developer,” Gartner IT Glossary, http://www.gartner.com/it-glossary/ citizen-developer.

https://venturebeat.com/2015/03/04/tim-oreilly-silicon-valley-is-massively-underestimating-the-impact-of-iot-interview/

https://venturebeat.com/2015/03/04/tim-oreilly-silicon-valley-is-massively-underestimating-the-impact-of-iot-interview/

https://gereports.ca/new-industrial-internet-report-from-ge-finds-that-combination-of-networks-and-machines-could-add-10-to-15-trillion-to-global-gdp/



https://www.ge.com/digital/blog/everything-you-need-know-about-industrial-internet-things

https://www.ge.com/digital/blog/everything-you-need-know-about-industrial-internet-things

http://www.gartner.com/it-glossary/citizen-developer

http://www.gartner.com/it-glossary/citizen-developer

11


1

Ten years ago when a pump in a remote oil field failed, nearly a month might pass before anyone even knew it had hap-pened. That’s because the oil company dispatched a technician in a pickup truck to drive around and check each of thousands of pumps scattered over hundreds of square miles over the course of weeks. After discovering that a pump had failed—potentially weeks earlier—the company then dispatched another technician to repair the bad pump. Meanwhile, that pump had lost weeks of production.

Now there’s a way to find out what’s happening at each of those pumps. Developers are creating applications that get information from what’s called the Industrial Edge where smart sensors, ruggedized routers, and other connected equipment collect real-time data from all sorts of industrial machinery. These applications monitor and analyze that data to identify problems and potential issues. As soon as a pump fails, or even before it fails, a technician can be on the way, replace-ment parts in-hand. Time saved: three weeks. Money saved: millions of dollars.

Understanding the Industrial Edge



Developing applications for the Industrial Internet of Things (IIoT) requires understanding the Industrial Edge. For most app developers, it’s a cross-disciplinary experience with forays into a new world. This chapter provides you with a general orientation to that world, an orientation you can take forward as you find your particular niche, whether it’s saving the planet, preventing catastrophic equipment failure, or trans-forming factories.

Welcome to the World of OTAfter you get a vision for an IIoT application and start to think about the potential of all the data available at the edge, you just want to grab it and run with it. But how do you even know what data is there and how you can access it? That data exists in an industrial context under conditions that are vastly differ-ent from those you’ve worked with before. For example, in the OT world, you upgrade a large turbine in a specific, determin-istic way. It’s complicated.

Your application will mine data that’s incredibly unstruc-tured and complex compared to even the most intricate enter-prise data sources. You’ll have to correlate and match data from different data sources including some assets that are in motion. You’ll have to analyze historical patterns and track time series data in real time.

Perhaps the easiest way to begin to understand the IIoT environment is to take a step inside the operational technol-ogy (OT) world. Your guide is Ralph, an OT field engineer with forty years of experience.1 He’ll give you a hard hat and a breathing mask. Brace yourself for noise, bright lights, dark spaces, and lots of big machines.

13Chapter 1: Understanding the Industrial Edge


A team of managers, engineers, operators, and field engi-neers like Ralph run almost all such environments. Often the operators and technicians have the most intimate relationship with the equipment. Ralph can anticipate problems when he recognizes subtle variations in how the equipment performs and diagnose equipment like a faulty pump by palming it and feeling the vibrations. When creating applications, you’ll want to listen to the people who are closest to the machines.

Doing so is particularly critical because one day soon, Ralph will retire. If you listen, Ralph can bring his deep knowl-edge into the modern world by telling you about the machines, their control systems, and their machine diagnostics. Working with Ralph, you can begin to understand the data that’s avail-able and how it can help your company as the worlds of IT and OT converge. Ralph is a domain expert, one of a number of people you’ll collaborate with in writing IIoT apps.

To effectively mine complex data for hidden gold nuggets, you’ll need to understand Ralph’s world. However, that cul-tural understanding is challenging. The people who come at it only from the IT side don’t always understand the purpose and mechanics behind the control systems or the limitations in far-flung fields and factories. For example, your application may have to assume dial-up speeds—when you have Internet connectivity at all. Table 1-1 gives you an idea of some of the key differences between OT and IT environments.

Because of the nature of industrial systems and equipment, Ralph works in a static world where things stay the same for years. Compare that to modern software release cycles where continuous delivery is a key goal—for example, Amazon mea-sures software updates in seconds.

Ralph may have used and maintained the same equipment for years. People in the OT world come at the issue from the controls standpoint. They often don’t realize that pairing this environment with a massive computing infrastructure could



OT Environment IT Environment

Pace of change

Hardened environment designed to be intact for years or even decades.

Goal is continuous delivery; refresh cycles in seconds, months at most.

Environment of users

Unfamiliar for most develop-ers. Wide variation. Extreme heat, cold, dust particulates in the air, moisture, humid-ity, bright sunlight, low light, underground mines, loud noises, hard hats, gloves. Sanitary operating rooms. Aircraft in flight. High-speed rail.

Familiar. Offices, homes, mobile users on the go.

Standardization Many protocols, often pro-prietary, standards vary by industry.

Standardized on Internet protocols.

Use of the latest computers, analytics, cloud resources

Sometimes. Usually.

Downtime Limiting downtime is critical and an order of magnitude more expensive than IT.

Not as important as in OT (unless you’re talk-ing about online order-ing on Black Friday).

Safety One of the highest priorities. Not usually a factor.

Connectivity Intermittent, dial-up. High speed Internet.

Constraints Hard: Laws of physics, deter-ministic processes.

Soft: Business drivers, evolving priorities.

Top priority (prime directive)

Keep operations going at all costs.

Dynamic shifts may occur based on evolv-ing business strategy.

Table 1-1. Some differences between OT and IT

solve problems that appear to be impossible. Developers need to be prepared to invest time in understanding the OT environ-ment to provide workable and compelling business reasons for change.



Why I left DreamWorks Animation for the IIoT

“I spent eight-plus years making animated films. My IMDb page has sixteen feature film credits including Shrek, Madagascar, Kung Fu Panda, and How to Train Your Dragon. Moving pixels around a screen was fun, but as a manager once told me: ‘You can go home. We make cartoons. Nobody dies if a render fails until tomorrow.’ I wanted to find something new and challeng-ing to work on. I explored mobile and then cloud computing, but felt those areas were well understood. I wanted to find something closer to working with hardware and the maker communities.

“Now at the other extreme I’ve been to nuclear facilities and bio-chemistry labs to see how software is used to solve big problems in large industrial settings that can impact everybody. At one point my mission was to make people laugh. But now that I’ve dived into robotics and the Industrial Internet, I’ve found I can help developers move, cure, build, and power the world while exploring emerging technology in artificial intelligence, machine learning, and embedded systems.”

Jayson DeLancey

A Quick and Dirty Intro to the OT System LandscapeIIoT developers need a basic understanding of the types of machinery they may encounter in a plant. I mentioned that people in OT environments come at the world from the con-trols standpoint. They’re focused on operations, which is directly related to controls.

But what are industrial control systems? A control system is an arrangement of physical components that is designed to



regulate the equipment to which it’s attached. Controls fall into two basic categories: open loop and closed loop controls.

• Open loop controls operate the same way regardless of conditions. For example, a washing machine runs for a certain amount of time with particular settings. It doesn’t measure whether clothes are clean or not.

• Closed loop controls have sensors, and the control takes action based on sensor feedback, like a thermostat does. Closed loop controls have become increasingly sophisticated, but typically have one thing in common: They use information in limited and repetitive ways to control aspects of a production process.

Some systems you may encounterWith that basic understanding of industrial controls in place, here’s a quick rundown of some of the systems you may encounter in an OT landscape:

• Programmable logic controllers (PLCs): As you move into Ralph’s world, you’ll hear him talk about PLCs, which are industrial computers with specialized languages that electricians, controls engineers, or process engineers can easily use. PLCs control sequences of operations and control the time constant of a plant. Through a PLC and its input and output infrastructure, Ralph can get information that leads to a decision to increase a machine’s rotational speed by 5 rpm, for example. The PLC sends signals to actuators, which increase the gas so the machine rotates faster.

• Distributed control systems (DCSs): A factory or other industrial environment might have dozens of control loops, keeping tabs on speed, temperature, chemical reactions, and gas output. The control loops always



need to know how much to put into each process to keep that process in a steady state. The DCS synchro-nizes a hierarchy of control loops and orchestrates them to obtain the desired plant output.

• Supervisory control and data acquisition (SCADA): SCADA is often coupled with what’s called HMI, the human machine interface. The HMI SCADA system interfaces with multiple PLCs and DCSs to manage all the information flowing through a plant or industrial process to optimize production.

• Data historians: Data historians aggregate data from industrial systems, storing selected types of data over time. These systems are useful sources of data trends for creating IIoT applications. But realize that their view, of all the available data, is limited. More data could be available to you than it may appear when you look at higher-level systems such as data historians.

• Edge gateway systems: Edge gateway systems are a key element of an architecture that combines edge and cloud capabilities. Such systems translate OT protocols and data formats, help manage storage and edge ana-lytics, and facilitate secure data flows between the edge and the cloud.

Figure 1-1 shows a simple example of edge to cloud com-munications facilitated via an edge gateway. Say that you have sensors that speak Modbus, an OT protocol. The edge gateway server translates that data into a standard data format and then transmits it to a cloud platform via WebSockets.

IIoT platform has the ability to manage software assets centrally even though they’re highly distributed. The edge infrastructure can be set up to know where the assets are, what versions of software they are running, and how to download



new versions. Doing so allows the equipment to be stable to meet industry requirements and also have the software capabil-ities needed to rapidly evolve the way a modern digital indus-trial company requires. With that in mind, developers and IT architects work hand in glove with operational technology (OT) technicians and engineers in the field and on the factory floor, collecting data from each asset and then analyzing it at the edge or sending it to a cloud-based environment for analysis.

A diversity of protocolsWhat enabled the immense leap forward in progress in modern computing? You can argue that TCP/IP did with HTTP close behind. Internet protocols paved the way, with rough consen-sus and running code.2

But the world of OT is highly specialized, and standards development reflects that specialization, which means that as you find your edge environment, you’ll also discover, as in a trip to India, many official languages. This is why you need an edge gateway that converts between protocols and standardizes data streams. Table 1-2 lists a few of the protocols we’ve seen in use.

Figure 1-1. Sensor data sent to an IIoT cloud platform via an edge gateway.



Take Action at the EdgeControl loops at the edge enable machines to do a finite number of things in a deterministic way. IIoT applications expand the repertoire of machines so that they can react nimbly to changes on the factory floor.

Use Case What It Is Links

MQTT Edge gateway to cloud

Open, lightweight, reliable, simple

Uses publish/ subscribe broker framework

OASIS standard

mqtt.org

Modbus Edge gateway to devices; Modbus server stores data from Modbus clients

A serial communica-tions protocol intro-duced in 1979

De facto standard

modbus.org

WebSockets Machine to machine

Runs over HTTP; can use TLS

Can ingest time series data

tools.ietf. org/html/rfc6455

gRPC Lightweight RPC

Lightweight protocol for IoT devices

Supports multiple programming languages

grpc.io

AMQP Messaging broker

Supports publish/subscribe

ampq.org

OPC-UA Edge gateway to devices

Introduced in 2006

Designed as an open protocol follow-on to OPC

opcfoundation.org

Table 1-2. A few of the protocols used in building IIoT applications

http://mqtt.org

http://modbus.org

http://tools.ietf.org/html/rfc6455

http://tools.ietf.org/html/rfc6455

http://grpc.io

http://ampq.org

http://opcfoundation.org



There are multiple reasons why you might want a piece of equipment on the edge to respond differently to different circum-stances. Sometimes you want a quick reaction to happen right on the edge—before sending the relevant data to the cloud. You can remotely monitor and manage numerous pieces of equip-ment, enabling Ralph to remotely shut off a machine when it’s flying enough red flags to be a concern. You may want to know more. For example, what else was going on when the pump’s temperature spiked? Was the oil pressure up? In some cases, the machine flying red flags could be programmed to automatically shut off. Or, you and Ralph might want to cycle off a piece of machinery and set it to turn back on when energy is cheaper.

How do you go from a deterministic process to a more flexible one and do so safely? At a high level, think of two data-processing loops separated by a hypervisor running on the edge gateway system, as shown in Figure 1-2. The inner loop (on the left) faces the Industrial Edge and is connected directly to the sensors and other inputs and outputs. The edge gateway translates industrial protocols and data for-mats into a digestible form for your application. The outer loop (on the right) faces the cloud where historical data can be processed and optimized (mixing in other data sources, such as weather data). The outer loop can provide optimized recipes back to the inner loop for processing. The inner loop keeps people and equipment safe. The hypervisor on the edge gateway system cleanly separates the two loops, pro-viding additional security for the sensors and controllers at the edge. (For more information, see ge.com/digital/iiot- for-developers.)

http://ge.com/digital/iiot-for-developers




Design patterns for the edgeConsider a few basic design patterns that use architecture. Applications often use more than one of these design patterns, and leverage both the edge and the cloud in doing so.

See: Design patterns for monitoringYou may want to know what’s going on with this machine. Gain visibility into the status of equipment by writing an application that enables a user to answer that question from anywhere. Instead of knowing about a broken-down pump in the oil field weeks later, get that information in real time, take action, and save money.

To start creating your own IIoT applications, sign up for a free trial of Predix at predix.io. At ge.com/digital/iiot-for-developers, you can access tutorials and down-load code for a remote monitoring and diagnostics (RMD) reference application.

Think: Design patterns for diagnosticsAdd diagnostics to the application by asking yourself these questions: How hot is the turbine running? How much fuel am I using?

With that level of visibility, Ralph and his boss can get more detailed information about the pumps in the oil field.

Figure 1-2. Conceptual diagram of a new approach to closed loop controls.

http://predix.io





They can see not only which pumps are running but also deter-mine the performance of the pumps and their yield. They can decide whether to run them all or a subset and when to sched-ule maintenance.

Do: Design patterns for taking actionYou can enable programmatic action to be taken, either remotely via an app or locally via machine-to-machine com-munications. For example, if the oil temperature reaches a cer-tain threshold, shut down the machine.

Optimize: Design patterns for analyticsYou analyze the data, at the edge or in the cloud, to drive opti-mizations. For example, consider an app that optimizes elec-tricity use by scheduling equipment to run when demand is reduced and rates are lower. Some cities are saving $1.5 million a year on energy with this type of technology. By analyzing data, you can also identify faulty equipment before it breaks. For example, after analyzing ten years’ worth of wind turbine data, developers and technicians found a pattern in the data that predicted certain types of failures. They realized if they could detect that pattern and stop it every time it started, they could prevent certain failures across a windmill fleet. This type of analytics should happen in the cloud.

Data drew me to the IIoT"For me, the biggest driver to work on the Industrial Internet is to work with data from machines that serve as the foundation of modern society. I enjoy the fusion of subject matter expertise and data science to improve operations and deliver customer outcomes.”

Andy Cash



Working with edge dataExabytes of data are left on the shop floor every day. This data is by nature more voluminous, complex, and harder to decipher than data in the IT world. Multiple approaches are evolving to ingest, integrate, manage, and make use of this data. When con-sidering the use of this data, here are some things to think about.

Decide what data is worth keepingSending raw sensor data to the cloud for analytics often doesn’t make sense. Raw data may be kept locally in a historian or a data lake environment and preprocessed and summarized for cloud-based analytics. Think of time series temperature data. If sensors emit a stream of time series data once a minute that reports the current temperature, and the temperature remains constant for three hours, when that data is ingested, it can be summarized without losing any of the information. Sensors may emit data more frequently than you need it so filtering it makes sense.

Don’t lose the data you wantIntermittent connectivity puts data streams at risk. Use a store- and-forward approach to ingest all data locally, and then store it in a holding area (similar to a buffer or cache) until the data can be transferred and stored permanently, whether locally or in the cloud. After the data is transferred to permanent storage, it’s then deleted from the local store.

Find out what the data is sayingIndustrial data is noisy. It takes work to find the signal. Platforms are evolving to help find the signal in OT data sources by leveraging machine learning and AI.



Expect Evolution at the EdgeSo far this chapter has described the Industrial Edge as it exists today in many places, with equipment that may be decades old and industrial control systems that were designed for an unconnected world. Applications that use such equipment are sometimes called brownfield applications.

Connected controls were designed for a connected world. These newer controls assume connectivity; applications that leverage data from such controls are called greenfield applications.

Edge-to-cloud architecture, a topic covered in Chapter 2, provides more details about IIoT platform capabilities such as data integration. Overall, there’s a movement to do more at the edge, adding computing power and local storage and analyt-ics. This is sometimes called fog computing, because it blends the cloud with the edge.

A key reason for doing more at the edge is data gravity—the difficulty of moving large amounts of data. The voluminous data of the IIoT comes from the edge. There are arguments for storing data at the edge and moving only what is needed to the cloud.

Resources for Getting StartedAs you’re learning about the IIoT, you may want to work with Predix to get further experience. The Predix Developer Kit features Predix software running on Intel hardware. The kit enables developers to get experience with ingesting data from a device into Predix. It comes with an edge gateway prein-stalled. It’s designed to enroll itself as a secure device in Predix.



For this and other resources mentioned in this book, go to ge.com/digital/iiot-for-developers.

Find Your EdgeUnderstanding the Industrial Edge is one of the main chal-lenges as you begin to write IIoT applications. This chapter introduces you to the lay of the land and what to look for, but your edge will be specific and specialized. Is your edge in motion, jets in flight, or high-speed rail? Is your edge an oil and gas platform? Under the sea? Whatever your edge envi-ronment, you’re likely to find that data has been trapped there. Providing visibility to that data offers low-hanging fruit for application development. Figure 1-3 shows examples of some different edge environments.

Endnotes

1. Ralph is fictional, here to personalize the world of OT for you. He is a guide with a deep understanding of how everything works. You may learn from industrial domain experts and plant managers, but I suggest getting to know the people closest to the machines. OT experts like Ralph are retiring at a rapid rate, so learn now.

2. The Tao of the Internet Engineering Task Force (IETF). See ietf.org/tao.html.

Figure 1-3. A sampling of edge environments.



http://ietf.org/tao.html

2

26


When most people drive by a line of fast-spinning wind tur-bines, they muse about the clean energy being generated. Unlike hydroelectric power (think of the Hoover Dam), where falling water generates a constant stream of electricity, wind turbines generate power only when the wind blows. But energy markets don’t work that way. Like most markets, you need to know in advance that you have a product available before you can sell it.

Extra energy capacity generated by unexpected wind gusts, called operational ramp events, may go unsold unless the energy supplier can predict the wind gusts accurately in time to sell the power generated. Energy suppliers bid each day, with adjust-ments each hour, on how much power, including wind power, they’ll be able to sell on the energy spot market that day and by the hour. To profitably leverage and sell that fickle wind power, energy suppliers must know in advance that they’ll have some-thing to sell and how much capacity they’ll have available to sell when strong gusts blow through. If a prediction model is less accurate, energy suppliers have to hedge what they say they’ll sell, potentially losing money on excess generation they can’t sell. To be useful, a wind prediction model must be accurate.

A Platform Architecture for IIoT Applications

27Chapter 2: A Platform Architecture for IIoT Applications


Exelon, a large power generation company, sought to improve the responsiveness of its forecasts to consistently predict strong gusts in time to monetize the power they generated. GE and Exelon teams built a solution that, combining turbine data, histor-ical data, and weather data, improved forecasting by 50 percent and led to the sale of an additional 70 megawatts of new capacity.

The question is, how would you go about building an appli-cation like this clean energy solution? Writing applications as a cut-to-the-chase one-off to solve a pressing business problem is certainly possible. In fact, this approach is often used to address a high-value use case quickly (and truth be told, many early IoT projects have been run in that way). But in the long term, such an approach is shortsighted because it doesn’t offer a way to make building the next application faster than the first.

Developers writing IIoT applications face a variety of chal-lenges. In a 2016 survey from Evans Data, more than 1,200 industrial developers cited their top challenges in writing IoT applications (see Figure 2-1).

Figure 2-1. Top three challenges industrial developers face in building applications.



Notice that the challenges cited fall into categories of secu-rity, standards, specialized systems, and data management.

The Need for an ArchitectureIndustrial developers need a way to address all of these chal-lenges systematically in a way that improves their ability to spend their time productively on industrial applications versus on peripheral tasks. The industrial developers surveyed stated that they spend little more than a quarter of their time devel-oping. Putting an architecture in place enables you to meet challenges head-on and implement a platform that simplifies development through the use of platform services, increasing time spent on development.

Why the IIoT was my next step as an industrial developer

“The IIoT was the logical next step of my career. I worked my way ‘up the stack’ from embedded systems for robotics and automation, to SCADA systems for natural gas, to enterprise software with data historians, and now to Industrial Internet from edge to cloud. As a product manager, my OT and IT background helps me grok customer needs and then explain how we can help solve their problems and unlock value with software/data/analytics. Our winning formula is simple: We build the hardware so we know the assets, we service the assets so we know how they operate, we have the domain knowledge for industry, and now we have the technology with the platform and our digital solutions. Those are the key ingredients to win in the IIoT.”

Jay Lakumb



IIoT applications are different from other applications, particularly in that they must take into account more factors that are considered critical to quality. The Preface mentioned the many types of developers who participate in building IIoT apps. Chapter 1 explored OT and the systems at the edge, so it’s a given that numerous and diverse stakeholders are involved in building apps.

An architecture provides a common understanding that enables the participation and coordination of multiple groups of stakeholders, not only those with IT knowledge but also those with OT and domain knowledge. The Industrial Internet Consortium has already laid the groundwork for such an architecture.

An IIoT Reference ArchitectureThe Industrial Internet Consortium (IIC) is an organization with more than 250 members from industrial, technology, and software companies as well as universities, open source proj-ects, and standard-setting bodies.

The IIC has been at work for a number of years creating a reference architecture that incorporates many different per-spectives and architectural patterns.1 The Industrial Internet Reference Architecture identifies several architectural patterns for IIoT applications. Some of the patterns are edge focused, like the gateway-mediated pattern, which relies on a wide area network or WAN to gather data from the edge. Others follow a distributed architecture pattern. For example, the IIC’s Three-tier Architecture Pattern (see to Figure 2-2) identifies an edge tier, a platform tier, and an enterprise tier.



Implementing an IIoT ArchitectureAfter deciding on the type of architectural patterns you want to implement, the next step is deciding how to assemble a plat-form based on this architecture that can accelerate the creation of IIoT applications by as many people as possible.

There are many ways to build a platform. GE faced a situa-tion similar to Amazon. Amazon took its expertise in running massive data centers and created a development platform. It then began offering what it had built via Amazon Web Services (AWS), allowing others to benefit.

In the same way, GE took its wealth of Industrial Internet and development expertise and built Predix, creating a plat-form that integrates and further secures existing cloud plat-forms (such as Cloud Foundry) and incorporates open source capabilities. The platform was built to serve real-world indus-trial use cases. Because industrial environments incorporate

Figure 2-2. Three-tier IIoT System Architecture.



many types of equipment, Predix was designed to be agnostic as to equipment manufacturer and interoperates with both GE and non-GE equipment.

Predix implements the IIC’s three-tier architecture while also following what the IIC calls an edge-to-cloud architec-ture pattern (see Figure 2-3). The platform extends beyond the cloud to the third tier of apps, which the IIC refers to as the enterprise tier. Note how the platform connects minds (people using apps) with industrial machines and their data.

The wind forecasting example mentioned earlier in this chapter uses Predix to connect Exelon wind farms, which are comprised of GE and non-GE wind turbines. Wind turbine data is aggregated through a data historian and interfaces with the edge gateway. The application ingests data into the cloud; runs it through forecasting models that use current wind farm data, historical data, and weather data; and sends the results to the edge gateway, which writes it to the data historian to drive real-time forecasts.

Figure 2-3. Predix as an example of edge-to-cloud architecture.



IIoT Platform CapabilitiesAn IIoT platform must address the most critical concerns that IIoT developers have in a systematic way to provide a secure framework for developing applications. In this way, platform capabilities accelerate development of IIoT apps, enabling you to rely on platform services rather than starting from scratch each time.

Platforms also enable collaboration. With a platform in place to provide common ground, you can invite more people to the app-writing party, not just stakeholders with IT knowl-edge, but also citizen data scientists or asset reliability manag-ers with OT and domain knowledge. Ralph and many of his

The smart grid drew me to the IIoT

“My journey within the transmission and distribution (T&D) of the electricity space started with the great Northeast United States and Canada blackout in August 2003. The panel tasked with finding the root cause of the problem showed how fragile the T&D system in North America was and how desperately it needed upgrades in intelligence and reliability.

“Initiatives to modernize the electrical grid started in a few areas in North America, most notably in California, Texas, and Ontario with smart meters. Around that time, I worked on developing software to enable the smart grid. Things started to take off around 2009 when President Obama pledged to modernize the grid. With the IIoT, I saw the opportunity to lead rather than to be led. I wanted to be involved with the Industrial Internet revolution from the very start.”

Paul Park



contemporaries will be retiring within a decade, resulting in the loss of a huge body of knowledge or brain trust. A platform can be used to codify their knowledge.

The following sections describe each type of capability and in some cases refer to its implementation in Predix.

Distributed architectureThe IIoT requires capabilities both at the edge and in the cloud. Should data be stored at the edge and analyzed there? Should it be ingested into the cloud, combined with historical data, and analyzed using machine learning techniques? For any given IIoT application, these may not be mutually exclusive design choices. Some analytics may be better performed at the edge for a variety of reasons.

Think of a speeding train, with a video stream monitoring the train tracks for obstructions. The analytics on that stream that recognize possible obstacles must work on the edge—on the train—to alert the operator in real time. That same video stream may be later uploaded to the cloud and analyzed to prioritize maintenance activities such as repairing track or switches or addressing environmental elements such as trees or streams that may impede future train operations.

Having a platform that offers flexible support for a distrib-uted architecture offers developers the ability to choreograph the dance between the edge and the cloud stacks in a way that makes the most sense for their application and its unique requirements. (See Chapter 4 for more on this topic.)

Further, a platform built for industrial scale can manage the complexity of having thousands of assets at numerous locations and provide visibility and insight into those assets through an edge manager. An edge manager enables you to manage software assets centrally even though physical assets are highly distributed. The edge infrastructure knows where



the assets are, what versions of the software they’re running, and how to download new versions. This allows the equip-ment to be stable the way OT people like Ralph need it to be yet have the software capabilities rapidly evolve the way a modern digital industrial company requires.

End-to-end securityThe security of IIoT applications is a topic of critical concern to developers, and rightly so. Entire books and certificate pro-grams2 are available on the subject of industrial cybersecu-rity, and more will follow. When evaluating IIoT platforms, consider their support for end-to-end security so that your applications can take advantage of platform capabilities. The following sections highlight some aspects of end-to-end secu-rity but are by no means comprehensive.

When you focus on security for a consumer app or plat-form in the IT realm, you’re working in a world where Internet connectivity generally is a given. Your tasks run along well-defined patterns and include penetration testing, code review, and static code analysis. As you move into OT applications and architecture, the tasks and challenges you face will be new. You’re often working with legacy devices built decades earlier. These devices, controllers, and other equipment don’t meet the security standards implemented in newer devices. That’s because they were never intended to be connected to the Internet or any larger network.

Coming from the IT side, your immediate thought might be to replace these legacy devices. You’re probably used to replacing older computers with newer ones and realizing a fast ROI and increased productivity. That’s not how the indus-trial world works. In the industrial space, much of this special-ized equipment is expensive with an expected life measured in decades. Certification mandates and compliance rules also



make it difficult to quickly swap out industrial equipment. OT infrastructure is critical: think of a gas turbine generating power for the life-saving equipment in a hospital or an air-craft engine that must be built for ultra reliability to ensure the safety of passengers.

As an IIoT developer, you’ll have to derive maximum value from communication with legacy equipment, yet make sure that the equipment remains protected. Now that these legacy devices are connected to the Internet, one big challenge you’ll face is maintaining a secure, tamper-proof connection between the edge and the cloud. On the edge, each device needs a unique, verifiable identity. Then that device needs to be able to communicate its identity to the cloud so the cloud knows who the device is. That end-to-end communication in both direc-tions must be verified as secure and not interruptible nor cor-ruptible. And the authentication must go both ways: Directives to edge devices must come from authenticated sources.

You may be tempted to rely on internal experts when developing security for your IIoT application. But a platform designed for OT tasks and OT security will achieve better secu-rity than code you write yourself. There’s no need to reinvent the wheel: You’re likely to find out the hard way that your shiny new wheel is missing a few spokes.

What about air-gapping?In the OT world, the primary security mechanism in the past was physical access control. Many places still accept this pattern as secure and as an argument against the IIoT in general. The rationale was that if equipment isn’t attached to the Internet at all—referred to as an air gap—it was safe by definition.

In an age of smartphones, USB drives, and other electronic tech-nologies that bridge the fence around the plant, air-gapping isn’t sufficient. Defense in depth is required to protect industrial assets.



An IIoT platform needs to have multiple components and layers designed to establish secure communication and a standards-based, defense-in-depth approach to security.

Edge devices are doubly vulnerableDevices on the edge are doubly vulnerable. First, they’re located in the field where malicious people could tamper with them directly. Such devices might include oil pumps and smart energy meters. Second, edge devices may be on networks that aren’t as well protected as web servers behind a firewall. When those devices connect to the cloud, making sure that they can be identified and authenticated with confidence is important.

Again, the stakes are higher in the operational technology world. A security breach in the OT realm could cause equip-ment failures, with millions of dollars in losses, injuries, and deaths.

Protecting both sides of the conversationThink of a two-way conversation between the industrial edge and the cloud. Both sides of the conversation—edge and cloud—need to be protected, secure, and unmodified.

Consider how that can be achieved. The edge device, not the cloud, should initiate all communications and ser-vices requests. A cloud service verifies that the edge device is authentic and unaltered. The cloud service then channels authenticated requests to the right resources on the cloud side and issues a security token to be used for future, secured communications.

Secure cloud infrastructureAs another layer of security, IIoT platforms should be built on secure cloud infrastructure specifically designed to support the requirements of Industrial Internet applications. The plat-form should offer support for data governance, federation, and



privacy, as well as meeting stringent security requirements in areas such as perimeter security, data security, access control, and data visibility. Table 2-1 summarizes some of the layers of security required by IIoT platforms.

Table 2-1. Security layers for IIoT platforms

Cloud Multitenant and multicloud to support regional data laws. Adherence to cloud standards and certifica-tions. Mapping to the Cloud Security Alliance Cloud Controls Matrix. Compliance with national, interna-tional, and governing body regulations.

Authentication and authorization

Enrollment of edge devices via PKI; authentication of edge devices prior to data ingestion.

Authentication of connections from edge to cloud.

Fine-grained control over read/write access to data.

Encryption Data in transit encrypted with highest level of transport layer security (TLS). Secure tunnels, VPNs, from edge to cloud. White-listing to allow traffic only from certain IP addresses.

Data at rest can be encrypted and in virtual isolation from data of other tenants.

Code security Code signing, code review, security-first mind-set.

Work through tutorials related to Predix security ser-vices at ge.com/digital/iiot-for-developers.

Data integration and managementIndustrial data is the heart of IIoT applications. IIoT platforms should be evaluated based on their ability to handle industrial data at scale.

Support for the following types of data is required:

• Time series data: The most common type of data coming from edge devices.




• External data: Regardless of format (JSON, XML, CSV, text, relational) to enrich models and analytics.

• Images, audio, and video: Increasingly, rich media is being captured at the edge.

IIoT platforms must support a variety of database types, and to the degree possible, make interacting with any type of database as generic as possible to enable loose coupling between the application and the underlying persistence store.

The platform must offer flexible options for ingesting that data, whether at the edge or in the cloud, depending on the use case. The scale of data can be enormous, and the ability of the edge to affordably and quickly transmit data to the cloud may be limited—Internet connectivity may be limited or nonexistent. Data gravity, a term that refers to the difficulty in moving large amounts of data, may suggest that data should be analyzed where it is—at the edge—versus in the cloud. A distributed edge-to-cloud approach enables applications to be architected to clean, filter, ingest, and analyze data where it makes the most sense. It also enables cases where some analyt-ics are handled in the edge data center3 and some in the cloud.

A platform for the IIoT must make it easy to manage and explore industrial data at scale. Don’t underestimate the com-plexity of data integration for the Industrial Internet. Think of thousands of devices streaming real-time data. Some of those devices are at rest while others are in motion. Multiply those thousands of devices by fifteen plant locations (or more). Now imagine ingesting and correlating all those data streams, along with historical data for each device and system in the industrial landscape. A platform that can ingest and correlate industrial data sources at scale and make them searchable and easy to explore can permit business stakeholders to explore industrial data and answer relevant questions, fueling the work of appli-cation developers and citizen developers alike.



Connecting and tagging data and making it searchable are large parts of the rationale for the purchase of systems such as Informatica. Even with such specialized systems, identifying related fields can take months of work. Platforms that offer this type of capability accelerate the use of IIoT data. Often it takes machine learning to ingest and correlate industrial data sources, enabling extraction of signal from noisy data and making it possible to get results in days as opposed to weeks or months.

For tutorials related to data capture and management and information about the low code environment for Predix, see ge.com/digital/iiot-for-developers.

Data scienceThe ultimate goal of IIoT apps is analyzing data to gain insights, improve operations, and predict behavior. Use cases include providing visibility into remote equipment, production processes, and energy use; preventing unplanned downtime; and optimizing the operation of individual assets, a process, or even an entire plant. Each of these use cases requires analytics. Support for a wide variety of data science tools and techniques is a baseline requirement for IIoT platforms.

An IIoT platform needs analytics services as well as machine learning capabilities to enable the broadest use of ana-lytics capabilities by all developers. A key benefit of working with an open platform is the ability to widely leverage models and analytics built by data scientists—write once, use many. Furthermore, using a platform that captures industrial domain knowledge offers the option of building on the work of other industrial professionals. By using a platform that offers ana-lytics built for various types of industrial analysis, including anomaly detection, time series analytics, quality control, and




predictive models such as mean time to failure, developers can leverage the work of numerous data scientists.

Rather than coding analytics and machine learning algo-rithms, look for analytics services that may do some or all of the work for you.

Enable mobile app creationMobile apps help deliver relevant knowledge to the edge so that people can act immediately. They offer visibility from any-where. An IIoT platform must provide services that make it straightforward to build apps that deliver needed capabilities to the mobile workforce. Further, the platform should sup-port building apps that serve mobile users and enable them to do useful work even when their device is unable to reach a network.

Capturing information on-site is a key aspect of such use cases. Apps designed for field technicians should use device capabilities to gather as much information as possible through the camera, video, voice memos, case notes, and more. These apps should upload relevant information as soon as connec-tivity is available again. Leveraging the collaboration aspect of the connected cloud completes the loop, giving the mobile user greater scale and the capability of the whole organization.

A platform should make it easy to develop apps that secure data on the device and that provide offline functionality, a fre-quent requirement for industrial applications.

Support for developers and citizen developers alikeIn the United States, about 10,000 people turn sixty-five each day. Many of them are in the manufacturing sector. And as they retire, their knowledge goes with them. An IIoT platform must therefore empower the capture of expert knowledge



and allow people of all skill levels to participate in the devel-opment of IIoT applications, both full stack developers and citizen developers—people who create applications using low-code, often visual, environments.

In order to create great apps that will be widely adopted, you need to leverage the deep expertise of the OT world and codify workers’ knowledge. Developers can work with OT to create these apps, and, as a force multiplier, OT experts can be empowered with tools that enable them to create apps via low-code techniques and the use of domain-specific languages.

An IIoT platform should offer the best of both worlds. Developers should be able to build applications using their favorite open source tools and languages and be able to lever-age platform services that provide key functionality such as fine-grained access control and authentication.

A platform should also enable OT experts and domain experts to participate in the software development lifecycle without requiring them to learn how to code in Go, Java, or Python. This type of low-code approach offers promise for empowering a larger group of experts to work together effec-tively. Domain-specific languages or fourth generation lan-guages, with visual environments, are becoming increasingly popular. By building expertise right into the app, knowledge is codified and app adoption grows because the app is relevant to and trusted by the users.

Look at the Predix Catalog to review available services. Visit ge.com/digital/iiot-for-developers.

Impacting the real worldChapter 1 introduced the inner and outer control loops that handle processes of See, Think, Do, and Optimize in the con-text of edge-to-cloud IIoT architecture. The Optimize portion

http://www.ge.com/digital/iiot-for-developers



of the loop has the potential to take work done using advanced analytics and use it to change the physical operation of the inner loop based on the optimizations identified. This is one of the reasons that the IIoT is such an exciting area. You not only identify what could be changed, but you also can change the physical running of equipment to implement those changes in the real world.

But with great power comes great responsibility. Security is paramount for such applications. And compliance and safety may require the use of advanced automation in order to drive any change to the physical world. At one company that deals with equipment related to natural gas incidents, some 5,000 logic conditions must be checked before any change can be made to the environment. Managing change at that scale and level of detail requires advanced automation because omitting a step could quite literally result in an explosion.

Endnotes

1. The document describing the Industrial Internet Reference Architecture is 58 pages long at this writing, and is well worth reading and studying. Go to iiconsortium.org/IIRA.htm to download the latest version.

2. The International Society of Automation (ISA) offers certification and train-ing in cybersecurity for the Industrial Internet through its IEC 62443 pro-gram. Visit isa.org to learn more.

3. The term data center has a great deal of flexibility when it comes to the edge. The edge might be in motion: on a locomotive or aircraft. In such cases, all data collected is stored at the edge, at least temporarily.

https://iiconsortium.org/IIRA.htm

https://isa.org

3

43


Digital Twins are dynamic digital representations for modeling assets, systems, and processes to learn from the past, under-stand the present, and predict the future to achieve improved business outcomes. Most of the time the goal in creating a Digital Twin is to predict and optimize the performance of machines that play a crucial role in a business.

Each of those machines or “things” is designed for the average case and for potential extremes, but after a machine leaves the factory, just as with nature versus nurture for human beings, things change. It’s why car manufacturers say that your mileage may vary (YMMV). The Digital Twin gives insights into the longitudinal study of nature and how the environment and maintenance (or lack thereof) impact a specific machine or component. By creating a model of a thing as-manufactured and adding to that all the available data about the use and operating conditions of that thing over time, you can craft a Digital Twin of a given asset and use that Digital Twin to drive specific business outcomes.

Some of the earliest and most sophisticated examples of Digital Twins come from aviation where the stakes are high.

Digital Twins



Preventing loss of life from errors in aircraft maintenance has made the aviation industry the leader in preventative and predictive maintenance processes. Digital Twins are used to model wear and tear on parts in complex systems such as jet engines and landing gear. By using sensors to collect data about engine performance and the surrounding environment, models based on physics and materials science can predict the wear on parts and provide clarity about what needs fixing when. The models are based on takeoffs, climbs, cruises, and landings in dusty deserts and in snow, ice, and subzero con-ditions with various levels of pollution. These Digital Twins have increased safety, saved money, and reduced unexpected flight delays due to maintenance problems.

One of the key skills for IIoT developers is to understand how to build Digital Twins and put them to work when cre-ating applications. This is what this chapter is about, from a practical perspective. When doing so though, remember that there is no standard definition of a Digital Twin. Different com-panies define and use the concept in different ways.

In addition, a lot of research and innovation is taking place, so both the theory and practice of Digital Twins are moving forward at a rapid pace. But that’s pretty much true in every exciting area of technology.

GE Aviation uses Digital Twins to monitor more than 35,000 engines. The Digital Twins can find out what is normal for each engine and how conditions affect wear and tear. With this data GE has created an Analytics Based Maintenance pro-gram for the engines on the Boeing 777 (see Figure 3-1).

45Chapter 3: Digital Twins


The Digital Twins in this program exhibit many of most interesting aspects of Digital Twin-based applications:

• Massive amounts of data about the context for each engine and its sensors are stored for each engine.

• Analytics extract important signals and generate insights from time series data.

• It uses virtual sensors. Jet engines have a limited number of physical sensors. For example, the GE90 engine has only 14 sensors. Physics models are used to estimate various virtual sensors to get insights.

• Models for helping predict wear and tear and other problems were created based on an understanding of the physics governing wear on parts caused by sand, pollution, and extreme temperatures.

• Predictive models were developed by combining physics-based insights with machine-learning techniques.

• One Digital Twin supports a variety of applications.

Figure 3-1. A Digital Twin generates critical data.



Digital Twins play a central role in the IIoT because they provide a foundation for reuse and containment of complexity. Digital Twins can be simple or extremely intricate. The chal-lenge for developers is to understand what they do, how they are constructed, and how they are supported by platforms and ecosystems so that developers can use them as an architectural component when they imagine applications.

As the rest of this chapter will show, Digital Twins are a powerful foundation for a wide variety of scenarios for auto-mation and optimization.

The Anatomy of a Digital TwinIn one sense, Digital Twins are complex models that combine data, analytics, domain knowledge, and various software capabilities to create applications that can do amazing things. But in practice, software, especially in the manufacturing and OT realm, has used such complex models for decades.

The big deal about the Digital Twin is the awareness that this model should have a life of its own. In previous genera-tions of applications, the models, the data, and the software were tightly bound and in effect trapped inside applications. They weren’t intended for wider reuse. Michael Grieves, who coined the term, calls a Digital Twin “the idea that a digital informational construct about a physical system could be cre-ated as an entity on its own.” Think of Digital Twins as object-oriented programming for the IIoT. Instead of trapping the model and associated functionality inside an application, it’s loosely coupled: You create it explicitly as a reusable object.

One of the reasons that the models in older generations of applications stayed trapped is that there was no coherent structure for the model, the data, and the associated func-tionality. In other words, there was no formal class definition.



Digital Twins create the most value when they come to life as reusable components typically controlled by RESTful APIs that can power many applications.

Enough work has been done on Digital Twins that early patterns for their anatomy are starting to emerge. In most cases, Digital Twins capture three aspects of the devices they are paired with: the structure, context, and behavior.

StructureThe mission of a Digital Twin is to make a physical system work better. So, to do its job, the Digital Twin must reflect the structure of the device in a useful way and change as the physi-cal device does, either because of wear or maintenance.

The starting point for most Digital Twins is to create a model of the physical asset. The granularity and scope of the model of the physical world varies widely based on the way that the device works and the opportunities for optimization. In many cases, aspects of the physical systems are ignored or greatly simplified because such detail won’t help. In some cases, when important information can be harvested, a Digital Twin might be a detailed model of just one subsystem.

Here are a few examples showing the range of models:

• For a wind turbine, you might have a digital representa-tion of the blades, the gearbox, the controller, and the pin that holds the construct up.

• The models of the jet engines focused on six crucial parts out of thousands. It isn’t uncommon for a Digital Twin to focus on the single most crucial part in a device.

• It’s also possible for a Digital Twin to focus on multiple devices, such as a set of pumps working together to keep a flow of liquid going, or even a whole plant or assembly line. In this case, the system is being optimized.



Much of the modeling of Digital Twins involved gathering data from sensors that provide data about individual parts of the system or about the inputs and outputs of the system.

In addition, domain experts often play a crucial role in con-structing Digital Twins. Sensor data, for example, may mea-sure temperature, vibration, and other aspects of the system. But that data, when combined with models based on physics, can reveal much more about the operation of the system and of the state of parts that may not have sensors directly on them.

The simplest domain models measure aspects like wear and tear. A tire on a landing gear or metal parts wear out a bit each flight, sometimes at an accelerating rate based on conditions.

A domain model using physics and materials science gov-erning aspects such as wear and tear can keep track of just how worn out a part is. In the aviation example, physics and materials science models created by domain experts show that the parts wear out faster when flying through air with high levels of pollutants. More advanced models measure oxidation and spallation due to high temperatures or external pollutants.

Of course, the model of the structure also has a history of the sensor readings so that the system can be monitored over time and analytics can be performed.

ContextIt’s important to realize that almost every Digital Twin model also tracks the key conditions that affect the operation of the system. In the jet engine example, temperature and other exter-nal conditions are tracked just as sensors from the device are.

In addition, in most OT environments, a multitude of other systems in play such as ERP, MES, historians, PLC and DCS systems, and so on can provide additional, rich detail.

Of course, doing all of this can entail a massive data inte-gration problem that requires knowledge of protocols and data



formats. But that’s what IIoT platforms are for, right? Digital Twins as implemented in Predix often use a graph repository for asset structure, a relational database (RDBMS) for track-ing contextual metadata, and a time series repository for the streaming data from sensors.

Machine learning and AI have added powerful elements to the creation of context for Digital Twins. These techniques make it possible to automatically or semi-automatically inte-grate and correlate fiendishly complex industrial data, includ-ing all the streams of sensor data. Machine learning and AI also make it possible to monitor dozens, hundreds, or even more data sources to determine if any key signals are correlated with important events. Machine learning enables the creation of better models using advanced techniques for model discov-ery. Machine learning and AI have enabled multiple modes of learning for Digital Twins. These Digital Twins can learn from peers, humans, historical context, and simulations.

BehaviorA Digital Twin also has methods, that is, code serving a vari-ety of functions, which are sometimes referred to by these umbrella term behaviors:

• Reporting and analytics are the most common starting point for Digital Twin behaviors. Making sense of the data is often the first step to figuring out what else to build and how to build it.

• Predictive models are the way that many early Digital Twin projects have succeeded. But the rich collection of time series data that most Digital Twins assemble pro-vides fertile ground for predictive models of all sorts that achieve many kinds of optimizations. The predic-tions are compared to actual results so that the models can be improved on a continual basis.



• Simulation and optimization capabilities can use com-binations of multiple predictive models to run what-if analysis and generate optimal strategies to operate and maintain assets and facilities.

• Process control capabilities provide ways to control the asset. Methods on the Digital Twin can be connected in a secure way to controllers at the edge so that behavior can be controlled. Through such controls, dynamic opti-mizations can be achieved, and systems with feedback loops that optimize themselves can be created.

• APIs can allow the Digital Twin’s capabilities to be accessed by other systems or Digital Twins.

These are all of the moving parts of a Digital Twin. The next section explains how to create such a Digital Twin and who is involved in the process.

Why I left eBay for the IIoT“My last two jobs have been in ecommerce and end user cus-tomer facing; I had no industrial background.

“The primary driver for me was data and analytics. Data is growing, and there are a lot of new developments like AI and machine learning.

“The area that clicked for me was a use case about building a data lake to do predictive analytics on aviation. We collected data from aircraft sensors and by analyzing this data, we were able to identify that aircraft that travel certain routes such as to Egypt or the Middle East require maintenance more frequently. We achieved a huge win by focusing only on the aircraft flying routes that require extra maintenance.”

Shamla Soans



Building a Digital TwinThe level of interest in Digital Twins has spawned a rapidly growing collection of development tools and platforms to make it easier to build and deploy them. More than likely new experts will write books about each of these offerings. Our task is simply to describe the steps in the process of designing and creating a Digital Twin so that a developer encountering any of these tools will know what needs to be done.

Keep in mind the analogy between Digital Twins and object-oriented programming.

• A Digital Twin is like a class. A Digital Twin starts out as a generic expression of the structure of an asset: a piece of equipment, a part, or a system of many pieces of equipment. In this sense, a Digital Twin is like the platonic ideal of a device, like a class in object-oriented programming. At this point, the Digital Twin is in its as-manufactured state.

• A Digital Twin is an instance. The generic Digital Twin is brought to life to track a specific piece of equipment. In effect, an instance of the Digital Twin is created from the class, the generic description. The copy of the Digital Twin takes on a life of its own that is all about tracking and understanding what is happening with a specific device, in the case of aviation, an engine or landing gear.

The Digital Twin works this way:

• The class for a Digital Twin is created to describe a device.



• The instance for a specific Digital Twin is created from the class definition.

• The instance is bound to the device, so that data flow-ing off the device comes to a repository connected to the Digital Twin.

In addition, the Digital Twin can be enriched with data from other sources that adds context to the device’s operations, such as weather data.

Saying that a Digital Twin is a reusable object implies that it exists in one place. However, like IIoT applications themselves, Digital Twins can exist in the cloud, on the edge, or distributed across both. IIoT application teams can decide which of these options makes the most sense for the business outcomes the Digital Twin is designed to support.

Much of the application code added to a Digital Twin has to do with ingesting, storing, and managing data at scale. Usually platform services help provide needed data management and support for machine learning on data. Another large portion of the application code added to a Digital Twin has to do with processing the data to do analytics and reporting, to apply advanced techniques such as machine learning and physics modeling, and to create predictive models. Depending on the type of twin being built, some of this code may be in the twin, and some of it may be in the applications built on the twin.

Twins are built to be used by other applications. For this reason the data and functionality of the twin is available through RESTful APIs and other forms of integration such as message queues that are appropriate for the types of applica-tions that are supported. Again, much of the code to support integration is usually provided by the platform that supports Digital Twin development and operation.

So, with this high level description of how to create a Digital Twin in mind, these sections dive a little deeper into a few key areas.



Creating the asset modelThe level of detail in the asset model of a Digital Twin can vary widely depending on the mission of the Digital Twin and the nature of the physical device.

The goal of the asset model is to provide a foundation for organizing the data, the analytics, and the application func-tionality that will provide the value of the Digital Twin.

At the most detailed end of the spectrum are Digital Twins that start with the sort of hyperaccurate models imported from Product Lifecycle Management (PLM) systems used in prod-uct development. These models, which were used for debug-ging the design and creating specifications for parts before the physical device existed, are usually overkill for a variety of reasons. Some subsystems aren’t controllable, don’t provide useful information, and don’t have parts that wear out that often. These systems can be treated like a black box or ignored. Most Digital Twins don’t need a complete Bill of Materials– based model of every part but rather the most important parts. Often these models are represented in graph databases so that the relationships between the parts are clear and the structure can be adapted as the asset model develops.

Graph databases track relationships between entities. Graph structures are intuitive and easily sketched on a white-board. An asset and its parts can be drawn using a graph struc-ture (for example, this assembly includes the following parts and components). Graph databases can model data at any level of detail required. A jet engine with thousands of moving parts can be represented as a graph, as can the simplest wheel assembly with only one or two parts.

RDBMSs are made up of tables that consist of rows and columns. Although such databases are important for many applications, they aren’t very good at modeling highly con-nected data. Additionally, the schema for RDBMSs is difficult



to change without breaking associated applications. Graph data structures can be changed on an as-needed basis.

The asset model for a Digital Twin is often modeled in an as-manufactured state, at the level of detail currently required (because of the flexibility of graph data structures, detail can be added later if desired). The asset model is then inherited by or copied to the individual Digital Twin instance, where it’s enriched with data about that particular machine and its history.

Some Digital Twins have a larger scope and focus on an entire plant or an assembly line. In such cases, the data for the Digital Twin with a larger scope, say, an assembly line, might incorporate data from Digital Twins that represent each machine in that assembly line.

In addition to the structure of the asset, metadata about each part may be captured, such as information from the Bill of Materials, the date of install, links to other service records, and notes from technicians or operators relating to the part. Depending on the implementation, the metadata may be stored in the graph database or in a companion repository.

Creating the data collection modelThe data collection model brings the asset model to life, adorn-ing it with evidence that is the raw ingredient for creating value.

In most Digital Twins, sensor data that is tightly bound to the asset is the star of the show. This data is commonly a large volume of time series data that is being collected from the physical half of the twin.

The first task is to store this data as it arrives. There can be a lot of it from a variety of sensors. Most Digital Twin plat-forms provide robust services for capturing and storing data and for processing it as a stream.

But then, in a process similar to a data warehouse, the data must be massaged into meaningful units for use in analytics and in application coding. These semantic models can become



quite ornate and can be represented in time series, RDBMS, graph, or JSON formats. Platform services generally provide ways to use all of these sorts of repositories.

Here’s where things get even trickier. The volume of data is such that it often doesn’t make sense to store all of certain types of data in the cloud. It’s vital that developers can choose to process some data at the edge and only bring what is needed to the cloud. For this reason, Digital Twins rely on platform services that are aware of the distributed processing patterns needed to make Digital Twins work.

Another point to keep in mind is that the task of build-ing the data model overlaps with the job of creating analytics and predictive models. Analytic functions are often used to clean data, remove outliers, and perform statistics that make the data cleaner and more meaningful. Machine learning rou-tines, domain rules, and physics models may be used for the same purposes.

The data model also must be able to store the context data, which can be about the conditions such as weather or tempera-ture at which the device is operating, or other information such as the products being processed by equipment or other infor-mation from MES, ERP, work order and maintenance manage-ment systems, or other systems. This data is vital to creating a full picture of the operation of a device.

Creating the analyticsThe analytics in a Digital Twin range from the most basic forms of reporting to advanced machine learning and AI.

Analytics can be created from scratch in most Digital Twin environments by using languages such as R, Python, or Java. But much of the use of analytics is driven by reusable analytics services that perform all sorts of functions from data clean-ing to application of physics to execution of machine learning algorithms.



One of the more challenging aspects of analytics is bring-ing in the domain experts to enhance the core models with knowledge of the physics or mechanical engineering princi-ples of the physical twins. With this knowledge, the amount of information that can come from a sensor can be supplemented. Numerous virtual sensors can be placed in a Digital Twin based on principles of physics, enabling sensor data to be inferred and gathered from places in the twin such as a combustion chamber where real sensors couldn’t withstand the heat.

In a real sense, a Digital Twin is a learning system. Platform services that automate learning via machine learning and AI are especially helpful for analyzing Digital Twin data.

For a developer webinar, tutorials, and a starter kit on GitHub, see ge.com/digital/iiot-for-developers.

Creating the predictive modelsPredictive models help a twin optimize performance and improve maintenance and also allow twins to become learn-ing systems. Making predictions and then comparing them to actual data can improve models.

Behavior and characteristics of physical assets change with time due to aging, operational mode changes, maintenance activities, repair and hardware upgrades, and so on. Therefore, continuously adapting these predictive models, asset models, and knowledge models is essential. These models can be improved in a variety of ways, including directly by human intervention or through machine learning using knowledge gained from the fleet, from similar systems, or from simulations.

Predictive models can be created from scratch, use pre-built models and templates, or utilize machine learning to discover models. In most Digital Twin platforms, creating libraries of advanced analytics is possible. Predictive models




are stored and made reusable as part of these libraries. In this way, generic techniques for creating models or for applying models from physics, materials science, or other domains can become a starting point for development of predictive models.

Creating the application servicesApplication services must be able to knit together everything a Digital Twin has to offer and also employ, as needed, a collec-tion of platform services. Again, both the application code and the platform services must be aware of the distributed nature of the IIoT and support code that runs in the cloud or on the edge. Characterizing what might be in an application service is difficult. Often services support analytics or data management. If it turns out that all applications are creating a similar service, it might make sense to move that service to the twin.

Most of the time, the Digital Twin development envi-ronments are polyglot, allowing a variety of programming languages to be used. In addition, platform services for API management, identity, security, encryption, and other aspects of creating OT-ready applications must be available.

The IIoT will transform the way we work and live

“I have always loved working with emerging technology and using it to find creative ways to solve business problems. It was only natural to move to the Industrial Internet as it allows me to be at the center of a space that will transform the way we work and live.”

John Andrechak



Fostering reuseOne of the results of the success of the jet engine Digital Twin program was an increased appetite for creating more Digital Twins for different parts of the aircraft and for enhancing the capabilities of those that proved successful. This desire shined a spotlight on the issue of increasing developer productivity.

The Digital Twins used in the jet engine program took an average of twenty-five weeks to build and deploy. By work-ing with a platform that supports reuse, GE Aviation hopes to reduce that time to six weeks.

Accelerating development of Digital Twins requires a plat-form that performs the same sort of functions seen in other development domains. The key is to create infrastructure ser-vices and other components that support a division of labor among all of the needed participants.

Putting Digital Twins to WorkAs stated at the beginning of this chapter, the simple idea of creating a Digital Twin of a physical device, like everything else, gets complicated when you face the task of bringing it to life. The OT requirements of the IIoT, the need to manage com-munication, transfer of data, and distribution of capabilities between the cloud and the edge, only make things more chal-lenging. And finally, the large number of skills needed to make a high quality Digital Twin work in a real-world environment is yet another hurdle.

But at the end of the day, when have developers ever been turned away by a challenge?

4

59


A GE team started thinking about what it could do to make power plants more efficient. The team found a thermal camera attachment for an iPhone that could generate thermal images quicker, more efficiently, and less expensively than existing technologies. They hopped a plane to do some field research with a power plant operator in Atlanta, who told them about all the critical parts of the power plant he wanted to monitor for heat loss but couldn’t.

The plant manager showed them thousands of yards of pipes across the plant; the seams on those pipes could be leaking steam, reducing plant efficiency. The team used what they learned to create an app at a Predix hackathon (and they won). The app detects heat loss anomalies by combining machine learning on sets of images with field engineers’ expertise to classify those images. The excitement generated by creating the IIoT app so quickly garnered attention. The team pivoted and was able to create a Thermal Imaging Tool for Anomaly Notification (TITAN) in just six weeks. The first power plant operator who used TITAN identified steam leaks that, when fixed, save that facility $50,000 a year.1

Creating Your IIoT Application



Building an App: What and WhyIdeas for IIoT applications may come from anywhere, but they start with a business objective. Using outcome-based design, you begin with the end in mind. The process may or may not start with an idea for a specific app, but it definitely starts with an objective. That objective might be to improve safety, increase production output, reduce energy consumption, cut costs, provide new products and services, or drive greater operational efficiency. For TITAN, the goal was to improve energy efficiency in power plants. What kinds of outcomes are you looking to drive with your application?

Getting Everyone in the RoomAfter you zero in on the goal for your IIoT app, you’re ready to work. Not so fast. IIoT applications aren’t usually solo efforts.2 Teams build IIoT applications for other teams. To develop the app, bring the design lead, the engineering lead, and the product management lead in the same room to create a shared mindset around the app. Plus, with that many people in the room, someone may bring doughnuts.

A team that drives an IIoT app may include a diverse group:

• Executives: They have been driving efficiency for years, and they have deep knowledge of both how the busi-ness runs and (usually) of the OT world.

• Domain experts: These experts from industrial engi-neers to field engineers can tell you where they have problems to resolve. (Some of them may become citizen developers3 and start creating apps themselves.)

61Chapter 4: Creating Your IIoT Application


• Full stack developers: They can leverage the strengths of a platform and fill in gaps and business logic for your application (that may be you!).

• User experience (UX) designers: They can bring screen designs to life, make the user interface work well, and optimize the app for mobile devices where needed.

If Ralph and his people don’t like the app, they won’t use it. That’s right—you can’t forget Ralph, with his 40 years of OT experience. You could pull Ralph away from his pumps and get him in the meeting, but you’ll also need to go to him.

Going to the FieldTo design effective IIoT apps, your team needs to take a close look at the particular challenges industrial users face. Consider users who climb power poles to fix transformers, users who wear work gloves and can’t readily access touch screens, users whose connectivity is intermittent or at dial-up speeds, and users working on remote oil platforms. You’ll need to take the

Joining an IIoT data science team“I recognized that the Industrial Internet has the capacity to transform how the world does business in the era of big data, machine learning, and massively scalable cloud computing. After seventeen years in the aerospace industry, I joined a data science team with a glint in my eye, eager to do my bit toward the digital industrial revolution.”

Girish Modgil



team to see Ralph in his environment to determine what type of app he needs and what type of device that app will run on. You’ll also need to determine where the analytics need to happen: can the analytics run in the cloud or does Ralph need contextual analytics on edge data so he can make a decision immediately?

This is where field research comes in. It’s hard to imagine crawling through a power plant wiring controllers, working as a parking enforcement officer doling out parking tickets in a big city, or flying by helicopter to spend weeks on remote oil platforms in the middle of the ocean. A large part of design-ing IIoT applications requires understanding where those apps will be used and who will use them. And for your team that just might mean, as it has for some of the developers I know, getting certified to survive helicopter crashes in the middle of the ocean as part of the app design process.

Processes and WorkflowsMost successful IIoT applications support processes and workflows and have a strong collaboration component. Your development team doesn’t just take the data from a machine on the edge, flow it into the cloud, run complex analytics, and say, “Poof, here’s a 2 percent productivity gain. Enjoy.” Often there’s one more step: helping the users of the app collaborate to make a decision, which then results in that 2 percent pro-ductivity gain.

An app might provide users with intelligence about a downward trend in the performance of a piece of equipment. At this point, because teams and not single individuals own almost all industrial work, the app needs to provide access to



collaboration tools so that the users can start to cross-correlate other data sets, reach out to other subject matter experts, and ping people in the field. That collaboration drives the improvement.

Designing IIoT apps often involves more systems design than mockups of user interfaces. When you look around at the artifacts that UX designers create to support such applica-tions, you’ll find at least as many process diagrams that outline workflows and the relationships between people, machines, and systems as you will design mockups for the UI.

Designing for a Different WorldIn most industrial contexts, the role of the UX designer has been underappreciated and underemphasized. Design think-ing is even more critical for creating an effective user experi-ence for IIoT apps, whose environments and users may differ significantly.

As you develop your IIoT app, you’ll want to learn about what’s different in the OT world, compared to the world where traditional business or consumer apps function, as Table 4-1 outlines. You’re no longer in Kansas: all those things you take for granted are wrong.

IIoT Applications: Edge Versus CloudAn important consideration when architecting IIoT applica-tions is where the work of the application will be done: will your app run at the edge or in the cloud?



Traditional App IIoT App

Uniform lighting in work environment

Yes. Lighting isn’t a given. May be intensely lit (think operating room), dim, or even dark.

Temperature, noise, and pollution levels

Assumes fairly constant human friendly environment found in office buildings (AC in summer, heat in winter).

Can vary significantly in terms of temperature, humidity, par-ticulates in the air, loud noises, distractions.

Connectivity High speed Internet, always on.

Poor or intermittent connectiv-ity and near dial-up speed. Sometimes no connectivity: middle of the ocean, in the air.

Data signal to noise ratio

Typically high signal.

Voluminous data, multiple sources to correlate, low signal.

You know what you’re looking for

Yes. Not necessarily. Rare to get one clear signal saying a pump is about to fail. Instead you cor-relate and parse a lot of opaque data that, taken together, points to a future failure.

Stakes for failure Invoice goes out late.

Facebook or FitBit crashes.

Airplane crashes. Equipment moves unexpectedly, putting worker safety at risk.

Table 4-1. Traditional app versus IIoT app assumptions

With some applications, you have a choice about whether to run them on the edge or in the cloud. But often the charac-teristics of the application, the data it uses, the availability of the network, and whether it drives action at the edge become factors in architecting your application. Some of the elements to consider include the following:

• Internet connection: Is it wired, wireless, wireline, intermittent satellite, cellular, data, or none at all? If the



connection is unreliable or costly, analyzing data in the cloud won’t make sense. When a machine needs to be shut down immediately, you can’t afford to wait.

• Data gravity: If the optimization requires high fidelity data or mission critical analytics and must run with or without Internet, the app should run at the edge.

• Control of physical hardware: Any app that runs phys-ical machines should be run at the edge.

• Cloud-based data sources: Is edge data being combined with other data sources in the cloud? In such cases, ana-lyzing the data in the cloud makes sense.

• Real time versus batch: Cloud analytics are likely to be done as a batch. Real time analytics are possible on the edge.

IIoT Application Development: A Basic ChecklistYour application will have an extremely specific industrial con-text. This chapter is written to a broad audience. The checklist in Figure 4-1 outlines some areas to consider as you develop your application.

Security is a foundational topic for IIoT applications. It’s mul-tilayered (see Chapter 2) and best provided by platform services. Security isn’t a checklist item; it requires a detailed analysis.



Figure 4-1. IIoT application checklist.



Microservices and IIoT AppsDevelopers of IIoT apps can and should take advantage of as many modern software development techniques as pos-sible. Although it’s true that the lifespan of OT equipment is measured in decades, it’s equally true that you want to leverage the latest IT developments as much as possible. One consistent theme in modern software architecture is the principle of loose coupling—that each component depends on others to the least extent possible. There’s a strong argument that loose coupling is even more important in IIoT applications than in most other environments. You want to be able to use the latest analytics, DevOps, microservices, containers, and cloud and persistence stores without impacting the underlying infrastructure.

Rather than building IIoT applications as one large code-base, breaking them down into smaller, more independent pro-cesses, or microservices, offers multiple benefits:

• The services themselves become easier to update and replace.

• Services can be organized around capabilities, such as front-end user interface, maintenance scheduling, logis-tics, data management, analytics, intelligent environ-ments, geospatial, mobile, and so on.

• Services can be implemented using different program-ming languages, making them hardware and software agnostic, so you can use the language that fits best.

You don’t realize these benefits unless you have an under-lying platform that manages dependencies for you. As one example, Predix leverages the manifest/app binding mecha-nism in Cloud Foundry for this reason, which enables the use of microservices at scale. Using such a platform, microservices



can be connected using language-agnostic APIs that can be integrated, updated, and reused, decoupled from code.

Multiple approaches to evaluating and using microservices are available.

• Build and reuse. Microservices offer building block–style functionality for creating IIoT applications. If you create your application using microservices that can be repurposed, you make building your next application easier than your first. Over time, you and your team increase the number of reusable building blocks, which reduces the time for building subsequent applications.

• Buy versus build. If you’re using an IIoT platform that offers microservices, app development includes look-ing at existing services and deciding which to use and which to build yourself to fill any gaps.

• Buy and adapt versus build. In many cases, platform microservices may do most of what you need them to do. You can then make your own version of the micro-service, tailoring it to the needs of your application.

• Build and share. The microservices you build for your IIoT application may represent capabilities that others outside your organization can use as well. Some IIoT platforms enable you to add your microservices to the platform so that others can use them.

Build an App from Wing to Wing: Full-Stack Code vs. Low CodeThere are two approaches to coding IIoT applications:

• Coding using traditional development languages such as Go, Java, C/C++, Python, and so on

• Low code using visual development environments



Coding an IIoT app requires a wide range of skills in deal-ing with complex industrial data, as described in Chapter 2. Numerous tradeoffs have been identified in this chapter, but there is no substitute for experience and working with tuto-rials to develop and hone your skills. See ge.com/digital/ iiot-for-developers for numerous resources. Using a purpose-built platform accelerates IIoT app development considerably, enabling developers to take advantage of platform services for security, data management, analytics, and more.

Low code offers an approach that leverages the power of an underlying platform that handles much of the work for devel-opers and those with domain expertise who may lack coding expertise. Such an approach expands the number of partici-pants in application development, empowers citizen develop-ers, and offers improved time-to-value for IIoT applications.

Low code isn’t a new idea; the approach traces its roots to the fourth-generation languages and rapid application devel-opment environments of the 1990s and 2000s. What is new about today’s approaches is the desire to abstract the underly-ing complexity of industrial data, which is parsed using artifi-cial intelligence, machine learning, robotics, and automation.

Why I left independent software consulting for the IIoT

“I was a consultant for seventeen years running my own busi-ness. Working on IIoT software attracted me because it’s a once- in-a-lifetime chance to bring important change to all the aspects of life that we take for granted: healthcare, power generation, transportation, and more.”

Tom Turner





Low code not only refers to how applications are written but also to how industrial data sources are cleansed and inte-grated. The less work it takes to ingest massive industrial data, relate it, tag it, and make it searchable by everyone who needs access to data, the better. Some low-code approaches remove the burden of coding ETL scripts as a means of ingesting data. Sophisticated automated data ingestion, organization, and exploration capabilities in low code environments may appeal to full stack developers and low code developers alike.

What Will You Build?The last few years have seen the rise of new development paradigms, with microservices, containers, automation, and orchestration. New techniques and platforms for handling data science, machine learning, and AI on multiple types of databases and on streaming data arrive in a steady flow. But the real-world applications of these techniques and their abil-ity to impact physical systems are often lacking.

Developers are needed to write the applications that will drive the world forward. Will you build an ecommerce rec-ommendation system or write code that reduces energy con-sumption, improves healthcare outcomes, prevents natural gas explosions, and drives positive environmental impact? I hope, after reading this book, that you’ll be inspired to begin build-ing applications for the Industrial Internet.

Endnotes

1. Contact the TITAN app team (John Andrechak, Andy Cash, Girish Modgil, and Paul Park) at [email protected].

2. Most apps need a team. But as mentioned in Chapter 1, industrial data has been trapped for a long time, and there are many simple use cases that repre-sent low-hanging fruit. If you have an idea that falls in this category, go for it.

3. See gartner.com/it-glossary/citizen-developer.


http://gartner.com/it-glossary/citizen-developer


http://predix.io

http://predix.io

WILEY END USER LICENSE AGREEMENT

Go to www.wiley.com/go/eula to access Wiley’s ebook EULA.

http://www.wiley.com/go/eula

these materials are © 2018 john wiley & sons, inc. any ...€¦ · any dissemination,...

Documents