aveva pi world lab pi adapters, our new data ingress for

© 2021 AVEVA Group plc and its subsidiaries. All rights reserved. AVEVA, the AVEVA logos and AVEVA product names are trademarks or registered trademarks of AVEVA group plc or its subsidiaries in the United Kingdom and other countries. Other brands and products names are the trademarks of their respective companies. AVEVA Group plc High Cross, Madingley Road Cambridge CB3 0HB, UK Tel +44 (0)1223 556655 Fax +44 (0)1223 556666

aveva.com

AVEVA PI World Lab PI Adapters, our new data ingress for PI Server, Edge Data Store, and OSIsoft Cloud Services

Dan Lopez Jerome Lefebvre Mike Norton v1.0 / October 2021

OF 33

Table of contents

1. INTRODUCTION TO PI ADAPTERS ..................................................................................... 4

1.1. PI ADAPTER USE OF THE OSISOFT MESSAGE FORMAT ...................................................................6

1.2. PI ADAPTER BUFFERING ...............................................................................................................6

1.3. PI ADAPTER SECURITY .................................................................................................................6

1.4. PI ADAPTER ARCHITECTURE .........................................................................................................7

1.5. PI ADAPTER CONFIGURATION ......................................................................................................7

1.6. PI ADAPTER CONFIGURATION VIA THE EDGECMD UTILITY ............................................................9

1.7. PI ADAPTER CONFIGURATION VIA REST API TOOLS AND POSTMAN ..............................................9

2. PI ADAPTER FOR MODBUS TCP ...................................................................................... 10

3. PI ADAPTER FOR OPC UA ............................................................................................... 18

4. PI ADAPTER FOR MQTT (PRE-CONFIGURED) ................................................................... 20

5. PI ADAPTERS + EDGE DATA STORE ................................................................................. 25

6. VISUALIZING SDS DATA ................................................................................................. 26

OF 33

1. Introduction to PI Adapters PI Adapters are our latest real-time data collection technology. They represent the continuing evolution of our software to span from edge to cloud, enabling completely new industrial IoT data patterns. In addition to being able to natively send data to an on-premises PI Server, PI Adapters can also write data up to OSIsoft Cloud Services or to Edge Data Store. Additionally, PI Adapters are supported on both Windows and Linux operating systems, with additional support for containers. Because of this, PI Adapters can be deployed on resource-constrained, ruggedized devices to monitor remote assets. PI Adapters enable PI Systems to collect data that was previously stranded, or data that was not technically or commercially viable to measure. In summary, PI Adapters offer:

• A lightweight installation footprint that can be deployed on both Windows and Linux operating systems, with support for container-based deployments (e.g., via Docker containers)

• The ability to send data to PI Server, OSIsoft Cloud Services, and Edge Data Store

• Four supported protocols (with more coming soon): - PI Adapter for OPC UA

- PI Adapter for Modbus TCP

- PI Adapter for BACnet - PI Adapter for DNP3

Regarding Windows and Linux support, PI Adapters are specifically supported on a variety of platforms and processors. Installation kits are currently available for the following platforms:

In addition, custom Linux installers can also be built for PI Adapters by preparing your own tar.gz archives of the required PI Adapter binaries.

Operating System Platform Processor(s)

Windows 10 Enterprise Windows 10 IoT Enterprise

x64 Intel/AMD 64-bit processors

Debian 9, 10 Ubuntu 18.04, 20.04

x64 Intel/AMD 64-bit processors

Debian 9,10 Ubuntu 18.04, 20.04

ARM32 ARM 32-bit processors

Debian 9,10 Ubuntu 18.04, 20.04

ARM64 ARM 64-bit processors

OF 33

Some of you may have experience with other data collection technologies, such as PI Interfaces and PI Connectors. For comparison, before we move on to the next section, we’ll walk through a quick table that highlights similarities and differences between those three product families.

Can be set up via a GUI

Can perform data

discovery on data sources

Supports Linux, ARM Linux, and Windows

Designed to run in

containers

Allows local data access

prior to egress

PI Interfaces ✓

PI Connectors

✓ ✓

PI Adapters ✓ ✓ ✓

Edge Data Store

✓ ✓ ✓ ✓

Data egress capabilities is a specific area where we can draw distinctions. Here are how those technologies stack up regarding sending data to a particular destination:

Includes Buffering

Encrypts egressed

data

Can write to PI Server

Can write to OCS

Egresses data via

OMF

Allows local data

access prior to egress

PI Interfaces ✓ ✓ ✓

PI Connectors

✓ ✓ ✓

PI Adapters ✓ ✓ ✓ ✓ ✓

Edge Data Store

✓ ✓ ✓ ✓ ✓ ✓

In brief summary, it’s only PI Adapters and Edge Data Store that can write to OCS in addition to writing to PI Servers (and moreover, via the OSIsoft Message Format, or OMF for short). And, as its name implies, of all of the data ingress options discussed, only the Edge Data Store provides local, edge-accessible data storage for accessing data streams before they are egressed to their final destination.

OF 33

1.1. PI Adapter use of the OSIsoft Message Format PI Adapters egress data to their target endpoints (e.g. PI Server, Edge Data Store, and/or OSIsoft Cloud Services) as OSIsoft Message Format (OMF) messages. OMF defines a set of message headers and message bodies that are used to generate compliant messages for inputting data into any compliant back-end system. OSIsoft Cloud Services and Edge Data Store can both consume OMF messages with no additional components. However, PI Server requires a PI Web API component with OMF ingress enabled in order to consume OMF messages, including data from PI Adapters.

1.2. PI Adapter Buffering When it comes to the edge, buffering data can be essential. PI Adapters include built-in data buffering to minimize data loss due to network disruptions. Buffering is applied both when egressing data source data as well as health telemetry, using a first in, first out approach. Depending on the hardware or solution requirements, buffering can be configured as in-memory-only buffering or on-disk buffering.

1.3. PI Adapter Security OSIsoft Message Format (OMF) messages are encrypted using the TLS 1.2 protocol by default when HTTPS is used (which should always be the case, in production environments). Data egress over HTTP is used when egressing data locally to an Edge Data Store, but at minimized risk, since the Edge Data Store only accepts data from local, same-host data sources.

OF 33

1.4. PI Adapter Architecture PI Adapters are designed to fully operate without requiring any third-party vendor software. Below is an example architecture for a PI Adapter, in this case, the PI Adapter for Modbus TCP. In the below architecture, like all PI Adapters, the OSIsoft Message Format (OMF) is used to send data to data and health endpoints:

1.5. PI Adapter Configuration Fundamentally, PI Adapter configuration and administration is performed using REST API HTTP requests with configuration settings in JSON format. For customers who wish to use a client tool to accomplish those tasks, we provide the EdgeCmd Utility (OSIsoft’s proprietary tool for configuring PI Adapters and EDS), but other REST client tools, such as Postman, can also easily be used. The REST API interface is very powerful and flexible, allowing you to both edit any single component or facet of the system individually as well as configuring the entire PI Adapter as a whole with a single REST API HTTP request. REST configuration calls can only be made locally, from sources that are already past the host operating system’s external access layer. With that safeguard in place, currently, there is no additional authentication required when making REST calls to the Adapter. All PI Adapters can be configured following a common workflow. First, before beginning data ingress, you must define a new PI Adapter component within the system components configuration, with one new component for each unique device from which the Adapter will collect data. The connection information for each of those devices, and the information for

OF 33

specifying which particular data items to collect, is next needed for configuring each Adapter component. For data egress, you provide information that identifies the particular destination for the data—such as the address for the OMF endpoint of a PI Server, EDS, or OCS tenant—along with authorization details for the outgoing connection. When applying authorization details, any passwords or secrets that are entered into a PI Adapter are stored internally in encrypted form, with cryptographic key material stored separately in a secure on-disk location. Additional parameters can be specified to enable egressing health and diagnostics data, tailoring buffering configuration to protect against data loss, and customizing logging information for troubleshooting purposes. Regarding health and diagnostics data, once you have configured the Adapter, and once it is sending data, you can query the Adapter for information such as:

• the status of connected devices

• adapter system functions

• the number of active data streams

• the rate of data ingress

• the rate of errors

• the rate of data egress In summary, the overall workflow for configuring a typical PI Adapter (which we’ll be walking through during this lab) are:

1. Add a component of the respective PI Adapter type (this step must be completed before steps 2 and 4)

2. Add data source configuration for that component 3. Optionally edit default schedules and data filters appropriately to tailor how data will

be collected 4. Configure data selection for that data source, to select specific data items for

collection (this step must be performed after step 3 if you will be using schedules or data filters)

5. Optionally configure health and diagnostics settings 6. Configure data egress, to specify your desired target Data Endpoints (this step can be

done at any time)

After these steps are performed:

1. The Adapter will begin collecting data at the configured schedule 2. The Adapter will send filtered data to the configured Data Endpoints 3. The Adapter will send health and diagnostics data to the configured Health

Endpoints

OF 33

1.6. PI Adapter configuration via the EdgeCmd utility Since PI Adapters are configured and administered through a REST API, you’ll need to use a REST API client tool to make HTTP requests to the PI Adapter REST API. AVEVA provides the EdgeCmd utility to enable configuring and administrating a PI Adapter using user-friendly syntax. EdgeCmd is available for both Linux and Windows operating systems, and it is installed separately from the PI Adapters. Because EdgeCmd utility is a self-contained command line tool, you can make adjustments to a PI Adapter’s configuration on the fly without programming or explicit REST API calls (e.g. via curl or Python). The basic structure of an EdgeCmd command is as follows (strict parameter order isn’t enforced):

EdgeCmd <Action> <Facet> <Arguments>

1.7. PI Adapter configuration via REST API tools and Postman As you’ve learned, EdgeCmd acts as a REST API client tool, but there are numerous third-party REST API client tools that can be used instead. Some popular tools include cURL and Postman (which can be downloaded from https://www.postman.com and used on Windows, Linux and Mac). Postman is a general-purpose third-party GUI-based REST tool. Postman can be particularly useful for learning more about PI Adapter REST APIs. It is important to note that natively, PI Adapters do not explicitly require a GUI for configuration or operation, and that PI Adapters can be installed on platforms that don’t support GUI-based clients. For this lab we’ll be using both EdgeCmd and Postman to configure PI Adapters.

https://www.postman.com/

OF 33

2. PI Adapter for Modbus TCP In this section, we are going to fully configure the PI Adapter for Modbus TCP with just a single command! Below is a diagram of what our completed architecture will look like.

We are going to do this by using the EdgeCmd configuration utility to pass a JSON configuration file to a pre-installed PI Adapter for Modbus TCP. The Modbus TCP communication protocol is a commonly available industrial communication protocol used for connecting and transmitting information between industrial electronic devices. The PI Adapter for Modbus TCP can connect to any device that uses the Modbus TCP communication protocol.

Remember, the basic structure of an EdgeCmd command is as follows:

EdgeCmd <Action> <Facet> <Arguments>

1. Before we get started configuring the PI Adapter for Modbus TCP, we’ll need a Modbus TCP data source. In this case, we have a Modbus TCP data simulator available on the PISRV01 machine. Log into PISRV01, and start the Modbus TCP simulator by selecting the following icon on the taskbar:

2. Let’s go ahead and use our first command to start getting more comfortable with the EdgeCmd configuration utility. On your PISRV01 machine, open a new command prompt. In the command prompt, type the following command, then press ENTER: EdgeCmd help

You should see the help overview for the utility appear, including details about the

OF 33

command pattern syntax, available operations, arguments, targets, component IDs, and several example commands. More detailed help can be found in the EdgeCmd documentation (you’ll find a bookmark for it in your lab machine’s browser).

3. Next, let’s use a “get” command to get the current PI Adapter version. In the command

prompt, type the following command, then press ENTER: EdgeCmd get version

You should see a response showing the current PI Adapter version information.

4. Now, let’s use another “get” command to get the current PI Adapter application configuration. In the command prompt, type the following command, then press ENTER: EdgeCmd get application

You will see the default configuration for a newly installed PI Adapter for Modbus TCP. In the next few steps we are going to update this configuration, so that the fresh installation will actually begin collecting and streaming real-time data.

5. Let’s go ahead and configure the PI Adapter to start collecting Modbus TCP data and

sending it to a local PI Server through the PI Server’s PI Web API OMF endpoint (which will accept the real-time data messages and store the received data in PI Points). We have a pre-written configuration JSON file already prepared on your demo VM for this first use case. Open that file here:

"C:\Users\student01.PISCHOOL\Downloads\ModbusAdapterConfig.json"

Look through the configuration file and see what you can find. If you need additional help, you can access the PI Adapter for Modbus TCP documentation, which is bookmarked in the lab VM’s web browser).

OF 33

In this first configuration file, you’ll see the specification of a new Modbus component with the ID of “ModbusSimulator”:

For this new “ModbusSimulator” component, you’ll see a corresponding Modbus TCP data source with a source IP address of “127.0.0.1” and port “502”:

OF 33

You’ll also see a data selection configuration, specifying five different specific Modbus data streams to begin polling from the Modbus data source:

Next, you’ll see a data collection schedule specified with the ID “schedule1”, with a data collection period of 10 seconds:

OF 33

Finally, so that data from those five data items will end up in PI Points on the lab VM PI Server, you’ll see the OMF egress configuration, with a data endpoint PI Web API target, including the endpoint address and the authentication username and password.

6. Now that we looked through the configuration file, let’s go ahead and apply the details contained within this configuration file to the PI Adapter application configuration by using a “set” command. In the command prompt, type the following EdgeCmd command, then press ENTER: EdgeCmd set application -file

C:\Users\student01.PISCHOOL\Downloads\ModbusAdapterConfig.json

Well done! With just that one single command, you just configured the entire PI Adapter for Modbus TCP—to connect to the data source, monitor specific data items, and egress that data to a PI Server!

7. Let’s retrieve the current PI Adapter configuration again to verify that our prior step

succeeded in making our changes to the original, fresh installation. In the command prompt, type the following command, then press ENTER: EdgeCmd get application

OF 33

You should now see that the original PI Adapter application configuration has been replaced with the settings of the configuration JSON file that we just used.

8. Now that the PI Adapter is fully configured, let’s check on our destination PI Server to see if new PI Tags were created and if they are updating with new data from the PI Adapter. On your VM, open a web browser, and visit the PI Vision webpage (you’ll find a bookmark for it):

https://localhost/pivision

Near the top-right of the PI Vision home page, click the “New Display” button to create a new display:

https://localhost/pivision

OF 33

Using the left-hand search pane, search the VM PI System for the new Modbus tags (that were created by the PI Adapter) by searching for “Modbus*”:

You should see the newly created Modbus TCP PI Tags! Select all the Modbus PI Tags and drag them from the left search pane onto the empty display canvas on the right in order to create a trend. You should see new data updates that arrive every 10 seconds (remember, this was the schedule period that we specified when we uploaded configuration to the PI Adapter using EdgeCmd):

OF 33

9. With the “set application” EdgeCmd command that we used earlier, we were able to

change the entire configuration for the PI Adapter. Now, let’s see how we can change just a single property with the EdgeCmd configuration utility with an “edit” command. In the command prompt, type the following command, then press ENTER: EdgeCmd edit Schedules -cid Modbussimulator -id schedule1 -Period

00:00:01

This command edits the schedule for the “ModbusSimulator” PI Adapter component and “schedule1” schedule ID from the previously configured value of 10 seconds to a new value of 1 second.

After making this configuration change, you should next see that data with values every 1 second now appear in your PI Vision display:

Congratulations, you have just fully configured your instance of PI Adapter for Modbus TCP to collect and egress data to a PI Server! Don’t worry if you don’t think you fully understand all the configuration steps that we just walked through, since we admittedly configured quite a bit in that single command, and with the help of a pre-written configuration file. To get you much more comfortable with PI Adapters, we will be walking through the configuration steps more slowly, step-by-step, with a configuration of a PI Adapter for OPC UA in the next section.

OF 33

3. PI Adapter for OPC UA We will be configuring the PI OPC UA Adapter using its REST API. REST APIs are increasingly the way that communication happens between systems—and for us, they offer a flexible and a granular way to build up the configuration of any Adapter. One of the great benefits of REST APIs is that there is an enormous wealth of tools available to interact with them, as well as the fact that REST APIs are able to be called from just about any programming language. One of these more popular tools used for testing, developing, and learning about REST APIs is called Postman. We will use it primarily for learning about the REST APIs of adapters, but it can be a useful tool in production systems when configuring Adapters as well. Just as a preview, below is a diagram of what our completed architecture will look like:

To get started creating that architecture, open the Prosys application from the taskbar: The Prosys OPC UA Simulation Server is an OPC UA server useful for testing with OPC UA clients such as the PI Adapter for OPC UA. It will give us a realistic set of simulated data items that we can use to walk through the common configuration steps for the Adapter. To see the data items that are available within this OPC UA server, click on the objects tab, and expand the “Simulation” folder.

To configure the PI Adapter, now click on the Postman icon in the taskbar: On the left-hand side of Postman, make sure that the Collections tab is chosen. You will see two Postman Collections listed, with each containing a grouping of predefined requests. To see how an Adapter is configured via its REST API, expand the “Basic” collection and proceed to complete and execute the REST requests one by one, from top to bottom. The requests that you’ll be completing are:

https://www.prosysopc.com/products/opc-ua-simulation-server/

OF 33

With each of request, in either Postman Collection, you’ll find documentation (see the Documentation button, below left) that we’ve added within the tool, with each set of documentation aimed at providing details about each request. As an example, we’ve provided a screenshot (below, right) of the documentation that we prepared for the first request.

During the hands-on lab sessions your instructor will walk you through each of the requests in the Basic Collection; for those of you who are completing this lab at a later date, you may consult the embedded Postman documentation included with each request for instructions on how you can complete the exercises on your own.

OF 33

4. PI Adapter for MQTT (Pre-configured) The beauty of all Adapters sharing a common framework is that it makes it incredibly easy and straightforward to set up additional data collection to new protocols. Let’s examine just what that can look like, reviewing everything that we’ve learned about PI Adapters thus far and examining an already-configured installation that performs new data collection from a completely new device. Below is a diagram of what that prepared architecture looks like:

1. Our exploration will consist of inspecting the four main steps that we walked through earlier:

A) adding a new PI Adapter component to a freshly installed PI Adapter, B) adding a new data source, C) configuring data selection for that data source, and D) configuring data egress. Again, as a reminder, in this case, those Adapter configuration steps have been already completed for you, and we’ll be examining the finished results. To begin, open a File Explorer window to where we’ve prepared the configuration for you, located at the following path: C:\Users\student01.PISCHOOL\Downloads\PIAdapterForMQTTExampleConfig Within that folder, open the file "00 Commands.txt", and examine its contents. Here you’ll see listed those four main steps as four very simple command-line commands (with helpful highlighting to draw attention to what’s being done in each step):

REM A) adding a new Adapter component to a freshly installed Adapter curl -d "@ConfigureComponents.json" -H "Content-Type: application/json"

-X POST "http://localhost:6010/api/v1/configuration/system/components"

REM B) adding a new data source curl -d "@ConfigureDataSource.json" -H "Content-Type: application/json"

-X PUT "http://localhost:6010/api/v1/configuration/MQTT1/DataSource" REM C) configuring data selection for that data source

OF 33

curl -d "@ConfigureDataSelection.json" -H "Content-Type:

application/json" -X PUT "http://localhost:6010/api/v1/configuration/MQTT1/DataSelection" REM D configuring data egress curl -d "@ConfigureEgressEndpoints.json" -H "Content-Type:

application/json" -X PUT "http://localhost:6010/api/v1/configuration/OmfEgress/dataendpoints" REM E) to view the configured Adapter curl http://localhost:6010/api/v1/configuration

We recommend leaving this file with all of the commands in it open, since we’ll be referring to it frequently, calling each command in turn throughout the next few steps.

2. Each of those files references a .json file that contains the configuration settings required for each specific step. We’ll examine those steps in turn to review what we’ve covered regarding Adapters. Looking at the first file in step A, ConfigureComponents.json:

[ { "ComponentId": "MQTT1", "ComponentType": "MQTT" } ]

We can see a block that merely defines a new component named MQTT1; should we wish to collect data from multiple different data sources, we’d be adding more components with different names. In this example, we’ll use a single data source, so that small block of text is all that we need (after which we can run command A, from above). The important thing is that this step should look familiar! It is identical to the first step we did when setting up the PI Adapter for Modbus TCP. At this point, you have all the information you need to prepare any additional ConfigureComponents.json files for any new Adapters you’re working with.

3. For step B, we will be configuring the ConfigureDataSource.json file, which is different for each different data source you are using. In this case, we will be setting it up for the MQTT protocol:

{ "HostNameOrIpAddress": "127.0.0.1", "Port": 1883, "ClientId": "PI-Adapter-for-MQTT", "ValidateServerCertificate": false, "TLS": "none" }

Since our connection is TCP based, you’ll see something similar to what you saw earlier with the Modbus TCP exercise: specification of a target host name and IP address. Since we’re working with MQTT, we see new parameters specific to MQTT, such as the MQTT Client Id and our preferences for certificate validation and TLS—which we will turn off, since we’re working

OF 33

with a development MQTT broker. The important point here is that an individual with knowledge of the data source—of the MQTT broker—should find it fairly simple to input in the necessary connection parameter values (and then run command B, above), and then going forward, re-use this configuration file as a template for additional MQTT connections. It’s also worthwhile to dive into the importance of the MQTT protocol in particular as a new member of the Adapter family. A massive advantage of MQTT is that it simplifies our architecture. Specifically, MQTT’s pub/sub, broker-based design means that large amounts of individual MQTT sensors can be installed and all configured to publish their data to a single, central software broker (such as the free, open-source Eclipse Mosquitto broker that we’re using here). To read all of that data, our Adapter merely needs to establish a single connection to that central broker, rather than to each of the individual sensors (see below).

Thus, by setting up this connection to our MQTT broker data source, we can now read new data from additional MQTT devices very, very easily, should we wish to purchase and add in new sensors.

4. In our case, with that MQTT broker installed, the data that we’ll be retrieving will be generated by an MQTT application (a device simulator) that can be activated from a shortcut on your VM desktop. On your VM desktop, to start generating MQTT data, click the shortcut named “Start MQTT Data Source.bat”, then proceed to the next step to show how we instructed the Adapter to collect specific data items from that data source.

5. Having added our component and specified connection information, we now select which data items we’d specifically like to read from our MQTT broker. In our example, we have set up our Mosquitto MQTT broker software to receive real-time data values in this format:

{ "TempSensorNode1": [{ "Time": "09/08/202112:48:05", "Value": 68.522 }] }

The PI Adapter documentation (https://docs.osisoft.com/bundle/pi-adapter-mqtt/page/configuration/data-selection-generic.html#mqtt-data-selection-examples) describes how we can create a new configuration file for step C, ConfigureDataSelection.json,

Mosquitto MQTT Broker

MQTT Temperature Sensor (an MQTT publisher)

MQTT Vibration Sensor (an MQTTpublisher)

PI Adapter for MQTT (an MQTT

subscriber)

https://docs.osisoft.com/bundle/pi-adapter-mqtt/page/configuration/data-selection-generic.html#mqtt-data-selection-examples

https://docs.osisoft.com/bundle/pi-adapter-mqtt/page/configuration/data-selection-generic.html#mqtt-data-selection-examples

OF 33

to instruct the Adapter on how to specifically extract the data values that we need. The contents of that file merely need to be:

[ { "Topic": "MQTTSensorTopic1", "ValueField": "$.TempSensorNode1[:1].Value", "DataType": "double", "Selected": true, "StreamId": "TempSensorNode1.Value" } ]

Which instructs the Adapter to subscribe to the MQTTSensorTopic1 MQTT topic, and for every message that is published to that topic (i.e. from MQTT sensors), take each message from TempSensorNode1 and extract the value that matches the syntax:

$.TempSensorNode1[:1].Value

Here is where you can also draw parallels between this Adapter and the PI Adapter for OPC UA, in that the contents of this file are very protocol-specific, but can be easily reused with other data sources that get added. And like OPC UA, the MQTT Adapter can also perform auto-discovery of the data source, so it can take care of some of this work for you. In our case, having pre-specified this configuration, we can run command C) from above, after which the last step is to configure data egress.

6. Finally, we come full circle with the egress step, step D, which sends this data to a destination for storage. Just like in section 2.5 earlier, where you configured egress, we’ll be setting up this Adapter to write a buffered stream of data—only this time, we’ll be writing it to a local Edge Data Store, rather than to a PI Server or to OSIsoft Cloud Services. Setting up that egress is accomplished by only two lines in the file ConfigureEgressEndpoints.json (which we’ll upload to the Adapter using command D from above, in step 1 of this section):

[{ "Id": "EDS", "Endpoint":

"http://localhost:5599/api/v1/tenants/default/namespaces/default/omf" }]

The Edge Data Store exposes its own OMF ingress endpoint, as seen in the URL http://localhost:5599/api/v1/tenants/default/namespaces/default/omf; as our Adapter receives data and egresses it to the EDS, EDS will catch that data and store it in its own internal Sequential Data Store, where the data can be accessed locally before it is streamed to a final destination for permanent storage.

As a bonus to this particular step, should we have time during the live lab session, we can even leverage what we’ve previously learned to configure this Adapter to also write to the local PI Server and to OSIsoft Cloud Services!

http://localhost:5599/api/v1/tenants/default/namespaces/default/omf

OF 33

7. To wrap up this section and validate that the MQTT Adapter is working properly, we’ll launch

the SDS visualization example project. On your VM desktop, you’ll find a shortcut for starting the SDS visualization, named “Start SDS Visualization.bat"; click it, and wait until the project successfully starts (it may take a minute or two), which will look like the following:

Look for the following line; that is how you’ll know that the app is ready for use:

** Angular Live Development Server is listening on localhost:4200, open your browser on http://localhost:4200/ **

8. With the visualization started, open it in a web browser by navigating to http://localhost:4200/home, then selecting the “default” namespace and “default” tenants within the EDS, and finally, selecting to trend our MQTT data stream. In your browser, you should see live data appear:

http://localhost:4200/home

OF 33

At this point, though we’ve verified that data is being successfully collected via the MQTT Adapter, we have not yet gone into detail about what the Edge Data Store actually is, how it works, and what’s going on under the hood to support this collection and visualization (including this example visualization tool that we just opened, and where you can obtain your own copy for free). That, therefore, is exactly what we will discuss in our next section!

5. PI Adapters + Edge Data Store What exactly is the Edge Data Store, then? Edge Data Store (EDS) is a lightweight data collection and storage application designed to capture data at the edge of networks for historical storage and analysis. It runs on small, rugged devices or is embedded in existing industrial hardware, and is designed to be resilient and require minimal installation and administration. While not a replacement for a PI System or OSIsoft Cloud Services (OCS), EDS augments the PI System and OCS by collecting and storing data in situations where deploying a full system is impractical. It can collect data that is beyond the reach of automation systems, in unreliable network conditions, and in environments too rough for traditional computers. Edge Data Store can run almost anywhere you can install a sensor, such as beam pumps, mining trucks, wind turbines, etc. The following diagram shows conceptually how EDS captures data and sends to permanent storage:

EDS collects data using any of the following methods:

• Built-in OPC UA connectivity

• Built-in Modbus TCP connectivity

• Custom application using OSIsoft Message Format (OMF)

• Custom application using REST API

Once collected, the data is stored locally in configurable data storage within EDS, until it can be sent to permanent storage in a PI System or in OSIsoft Cloud Services through periodic egress. The data can also be read from local storage by custom applications using REST APIs.

OF 33

In our case, the data ingress to the Edge Data Store is not coming from either of its built-in adapters, but from our PI Adapter for MQTT that we just discussed in the previous section. Now that this data is within the Edge Data Store, it can be egressed to a PI Server or to OCS, as well as being retrieved locally through the Edge Data Store’s REST API—which is exactly the approach taken by the visualization that we opened at the end of the previous section, to verify that we were indeed collecting data. That visualization is a great example of the kinds of applications that can be built on top of the Edge Data Store—but how does it work, and what kinds of other options are there for visualizing Edge Data Store data? That’s what we’ll look at in the next section.

6. Visualizing SDS Data Yes, the title of this section (SDS, as opposed to EDS) is spelled correctly! Here we’ll be talking about how to visualize data from a Sequential Data Store, which is indeed the storage used by the Edge Data Store—but which is also used by OSIsoft Cloud Services. The benefit here is that the visualization tools that will be discussed can both work with either platform, thanks to both sharing a common storage architecture. First let’s examine the visualization that we used earlier to view MQTT data within our Edge Data Store. It was obtained from our GitHub account, at the following repository: https://github.com/osisoft/sample-sds-visualization-angular. There you can find instructions on how to download the source code and build the project to create the exact same visualization that you see in our lab machines.

WARNING: The web server used in this sample is intended for use in testing or debugging sample applications locally. It has not been reviewed for security issues.

https://github.com/osisoft/sample-sds-visualization-angular

OF 33

This project allows the developer to enter in connection settings for either connecting to a local Edge Data Store (as we’ve done) or to OSIsoft Cloud Services. Details of the steps required are available at the project web site (above); what we’d like to draw your attention to is how the project specifically uses the REST API to retrieve data. If you return to the visualization, running in your web browser, and press CTRL+SHIFT+I, you’ll open a developer tools pane that will let you examine the REST calls being made by the project. Under the Network tab, you can see a list of the REST calls that are being made:

Those calls for data follow the syntax outlined in the SDS documentation here: https://docs.osisoft.com/bundle/edge-data-store/page/sds/read-data/reading-data-api.html#get-values. For example:

The calls being made by the app, therefore, are requesting the most recent 100 data values from the EDS for the selected stream, starting at the current time (as the start index)—searching backwards, historically (hence reversed = true) instead of chronologically. How to make similar REST calls for retrieving data from OSIsoft Cloud Services can be found at the documentation here: https://docs.osisoft.com/bundle/ocs/page/api-reference/sequential-data-store/sds-read-data-api.html#list-values

https://docs.osisoft.com/bundle/edge-data-store/page/sds/read-data/reading-data-api.html#get-values

https://docs.osisoft.com/bundle/edge-data-store/page/sds/read-data/reading-data-api.html#get-values

https://docs.osisoft.com/bundle/ocs/page/api-reference/sequential-data-store/sds-read-data-api.html#list-values

https://docs.osisoft.com/bundle/ocs/page/api-reference/sequential-data-store/sds-read-data-api.html#list-values

OF 33

You can see how the structure of the queries is identical! That means that it should be possible to build other applications that can also read from a Sequential Data Store, whether it’s in an Edge Data Store or in OSIsoft Cloud Services—and yes, that’s what we’ll look at next. That exact same approach was used by our R&D team to create another GitHub example—this time, one for a plugin for the popular open-source visualization application Grafana (see https://grafana.com/). In this case, our team created an example that allows Grafana dashboards to use a Sequential Data Store as a data source, for trending real-time data from both an Edge Data Store and from OSIsoft Cloud Services. To demonstrate that to you, onto the lab VM we obtained and pre-installed Grafana from the public Grafana download page:

We have already downloaded and installed Grafana for you, and it now runs on your VM as a Windows service which you can see within services.msc (the service is named “Grafana”). You can explore that Grafana installation by opening a local web browser in your VM and navigating to http://localhost:3000/login, and signing in with the following Grafana credentials: User: admin

Password:

https://grafana.com/

http://localhost:3000/login

OF 33

dataingresslab We next obtained the Grafana Sequential Data Store data source plug-in source code from our GitHub repository, https://github.com/osisoft/sample-ocs-grafana-nodejs:

After obtaining that source code, we performed the necessary steps (as outlined in the project online documentation) to build and install the plug-in. With that done, upon opening Grafana and navigating to “datasources” (http://localhost:3000/datasources), you can see that two new data sources have been added to the stock Grafana installation, one for our local Edge Data Store data and one for our lab data in OSIsoft Cloud Services.

https://github.com/osisoft/sample-ocs-grafana-nodejs

http://localhost:3000/datasources

OF 33

Taking a look at the first data source, for the EDS, we can see that this example allows us to specify the namespace where our data resides, along with the EDS port number, which are exactly the pieces of information that we need in order to build data query URLs for retrieving data. Retrieving that data, in this case, doesn’t require specifying any further credentials or authorization information, since our Grafana installation is running on the same host where we have the Edge Data Store installed.

And now, data from that Edge Data Store can be visualized in a dashboard! From the left-hand menu in Grafana, open the dashboards page:

If you then open the dashboard named “Local EDS Data”, you can see a simple trend that we’ve created to visualize some of the data that our Edge Data Store installation has collected. For a customer, then, who not only wanted local edge data collection, storage, and egress, but who also

OF 33

wanted edge visualization directly from the EDS, we’ve now satisfied their requirement using Grafana and our R&D team’s Grafana SDS plugin!

Again, since this plugin works for Sequential Data Store data specifically, and not merely data from within an Edge Data Store, we can also use Grafana to visualize data from OSIsoft Cloud Services, even if that data originated from a data source hundreds of miles away! To see how we did this, open up the data sources list from the left-hand menu (using the gear-shaped icon), and select the OCS data source from the list. This second data source, for viewing OCS data, contains more fields than the EDS data source. That’s because in this case we now need to specify the cloud URL, OCS tenant ID and OCS namespace ID for retrieving our specific data from our cloud instance. Here we also see that we now have to enter in an OCS Client ID and Client Secret, authorization credentials that were generated for us from within the OSIsoft Cloud Services portal. Those credentials will give us specific privileges for viewing particular OCS data streams (to learn more, see https://docs.osisoft.com/bundle/ocs/page/set-up/clients/clients-concept.html#client-credentials-client).

https://docs.osisoft.com/bundle/ocs/page/set-up/clients/clients-concept.html#client-credentials-client

https://docs.osisoft.com/bundle/ocs/page/set-up/clients/clients-concept.html#client-credentials-client

OF 33

With this data source configured, if you navigate back into the dashboards list, you can open up the OCS dashboard to now see data from OSIsoft Cloud Services, now pulled down securely from our cloud tenant and ready for visualization and analysis!

OF 33

And with that, our visualization demonstrations are complete! We’ve showcased two separate currently-available, open-source, no-cost tools for visualizing data that was collected by PI Adapters—and that was sent to both an Edge Data Store and to OSIsoft Cloud Services. And there are even more visualization and analysis options out there to choose from as well! However, such discussion on the delivery of our Adapter data is a topic best suited for another lab, or at the very least, for questions that you can ask of your course instructors! If you have any comments or questions about any of this content, we highly encourage you to reach out, and we’d be happy to answer them. Thank you for accompanying us on this exploration of modern data ingress to PI System and OSIsoft Cloud Services, and we look forward to hearing from you!

aveva pi world lab pi adapters, our new data ingress for

Documents