Using & Accessing Using & Accessing Historian DataHistorian Data

Understanding the Proficy Understanding the Proficy Historian and how to use the Excel Historian and how to use the Excel

Add-in to view plant dataAdd-in to view plant data

Stephen Friedenthalsfriedenthal@evsystems.net 617.916.5101 (w) / 617.901.9193 (m)

AgendaAgenda System OverviewSystem Overview

• Quick introduction to the Proficy HistorianQuick introduction to the Proficy Historian• GreenFuel system architectureGreenFuel system architecture

Excel InterfaceExcel Interface• Installation and configurationInstallation and configuration• Basic queries (to get our feet wet)Basic queries (to get our feet wet)• A primer on data storageA primer on data storage• Data retrieval modesData retrieval modes• Advanced analysis and interactive Advanced analysis and interactive

spreadsheetsspreadsheets Wrap-up & questionsWrap-up & questions

Plus! A cool (?) bonus… a sample Excel report you can use when you’re done!

Introduction to the iHistorianIntroduction to the iHistorian

Key featuresKey features• High-speed collection & retrieval (20k/sec)High-speed collection & retrieval (20k/sec)• ““Understands” time-series dataUnderstands” time-series data

Able to provide time-based data aggregation (min, Able to provide time-based data aggregation (min, max, average, etc.)max, average, etc.)

Data compression (“dead banding”)Data compression (“dead banding”)

• Robust data collection from plant floorRobust data collection from plant floor• Event & time-driven calculations on historical Event & time-driven calculations on historical

datadata• Near real-time access to plant dataNear real-time access to plant data• Store/add comments with dataStore/add comments with data

Historian Introduction (cont’d)Historian Introduction (cont’d) Additional FeaturesAdditional Features

• Easy-to-useEasy-to-use THETHE easiest easiest to use, fastest to install and configure, and to use, fastest to install and configure, and

lowest maintenance historian available in the world today lowest maintenance historian available in the world today (really!)(really!)

Information access is easy without requiring any Information access is easy without requiring any specialized trainingspecialized training, programming or database skills, programming or database skills

• Security, auditibility & traceabilitySecurity, auditibility & traceability Secure, 21CFR11 compliant applicationSecure, 21CFR11 compliant application User & role-based security & User & role-based security & 100% auditing100% auditing 100% of the 100% of the

time time • Analysis and enterprise connectivityAnalysis and enterprise connectivity

A fast, flexible, easy-to-use Client Tools include, Excel, web A fast, flexible, easy-to-use Client Tools include, Excel, web portal, Plant Applications™, SDK, and OLE DBportal, Plant Applications™, SDK, and OLE DB

Getting StartedGetting Started

Installing the Excel toolInstalling the Excel tool

How to install…How to install…

3. Enter the server name and continue with the setup. Example below connects to the server called, “NANO”

2. Go through the usual prompts and select the Excel add-in, then press Next..

Step 1: Insert the CD or run the network install

How to Configure ExcelHow to Configure Excel

To add the iHistorian To add the iHistorian Excel “add-in””Excel “add-in””

1.1. Select “Tools”, Select “Tools”, from the menu from the menu bar, then “Add-bar, then “Add-ins” ins”

2.2. Select “Proficy Select “Proficy Historian Add-in” Historian Add-in” and press OKand press OK

Note: If you don’t see the Proficy Historian listed, search for the file, iHistorian.xla and then use the Browse button to select it.

Configuring Excel (cont’d)Configuring Excel (cont’d)

After selecting the After selecting the add-in, add-in, HistorianHistorian will be a new menu will be a new menu item on your item on your toolbar whenever toolbar whenever you start Excel.you start Excel.

Historian FunctionsHistorian Functions

An overview of some of the An overview of some of the thing you can do with the thing you can do with the

historian Excel add-inhistorian Excel add-in

A Quick Example – Search TagsA Quick Example – Search Tags

A tag search lets you quickly find and A tag search lets you quickly find and display process variables display process variables

This is usually the first step to This is usually the first step to retrieving data, since the retrieving data, since the datadata is is accessed by the accessed by the tag nametag name

Ex.:Ex.:• The tag name for a bio reactor CO2 The tag name for a bio reactor CO2

monitor is monitor is GFAZ1.200_AE_245H.F_CVGFAZ1.200_AE_245H.F_CV• This is then used to retrieve dataThis is then used to retrieve data

Performing a tag searchPerforming a tag search

6 Easy Steps

First, select Search Tags from the add-in menu

1. Enter the search criteria for the tagname (blank for all)

2. Select the tags for the report

3. Select outputs

4. Type or select the spreadsheet location

5. Press OK

Historian Excel add-in functionsHistorian Excel add-in functions

Just as with the tag search, the add-in can Just as with the tag search, the add-in can be used to:be used to:• Search TagsSearch Tags• Query Current ValuesQuery Current Values• Query Calculated ValuesQuery Calculated Values• Query Raw DataQuery Raw Data• Query Alarms & Events (NA for GreenFuel)Query Alarms & Events (NA for GreenFuel)• Query Calculated DataQuery Calculated Data• Query Filtered DataQuery Filtered Data

Tag SearchTag Search

Purpose: To retrieve tag names, Purpose: To retrieve tag names, description, units, etc. description, units, etc.

Features:Features:• Search by tag name or descriptionSearch by tag name or description• Dynamic searching by using a cell Dynamic searching by using a cell

address vs. a string (i.e., cell A2 is used address vs. a string (i.e., cell A2 is used for the tag mask)for the tag mask)

Advanced Tag SearchAdvanced Tag SearchThe text in cell $B$2 is used to

retrieve tag namesInstead of searching each time, just type a new entry in $B$2

Advanced Tag Search IIAdvanced Tag Search II

After you create a dynamic search based on tag name, you can edit the formula in Excel to add the tag description as a criteria

Notes:1. Tag name & description

are a Boolean AND2. Blank = All 3. Wild card characters:

1. * = ALL chars to right2. ? = single character

Retrieving DataRetrieving Data

But first… we need to learn a But first… we need to learn a little about data storagelittle about data storage

Time-series data storage Time-series data storage conundrum conundrum

Data is not typically Data is not typically stored at even intervals, stored at even intervals, but…but…• We may need evenly We may need evenly

spaced reports (e.g., spaced reports (e.g., hourly avg.)hourly avg.)

• We need to know what We need to know what the value is “now” even the value is “now” even though new data may not though new data may not have been stored for a have been stored for a long timelong time

We need a way to know We need a way to know “what happened” & “what happened” & when without looking at when without looking at MB (or GB!) of dataMB (or GB!) of data

0

10

20

30

40

50

60

70

80

90

100

9:00 9:05 10:00 12:30

Red marks are raw samples that were stored. (Note different intervals)

Historian data retrievalHistorian data retrieval

The historian “understands” the nature of The historian “understands” the nature of the data and:the data and:• When you ask for the “current value” it returns When you ask for the “current value” it returns

the the last good valuelast good value (even if it’s hours or days (even if it’s hours or days old)old)

• When you ask for evenly spaced samples it When you ask for evenly spaced samples it interpolates the raw data*interpolates the raw data*

• When you ask for data in between real samples When you ask for data in between real samples it gives the best interpolated result (e.g., if you it gives the best interpolated result (e.g., if you ask for the maximum it will return an ask for the maximum it will return an interpolated maximum) interpolated maximum)

* Note that Lab data and Trend Sampling do not interpolate or smooth the data

Example: Retrieving current valuesExample: Retrieving current values

Select Select Query Query Current ValuesCurrent Values from add-in menufrom add-in menu

Enter the tag Enter the tag names, or use a names, or use a cell reference from cell reference from the tag searchthe tag search

Select outputs and Select outputs and locationlocation

Retrieving current valuesRetrieving current values

Historian CalculationsHistorian Calculations

Now for the fun stuff…Now for the fun stuff…

Different ways to retrieve dataDifferent ways to retrieve data

Sampling ModesSampling Modes• InterpolatedInterpolated – Just like it sounds, – Just like it sounds,

historian interpolates between raw historian interpolates between raw samples and return results based on the samples and return results based on the time interval (e.g., every 30 min.) or time interval (e.g., every 30 min.) or number of samples number of samples

• CalculatedCalculated – Historian will perform one – Historian will perform one of several aggregate calculations (min., of several aggregate calculations (min., max., avg., etc.) on the data and return max., avg., etc.) on the data and return results (more on this in a bit)results (more on this in a bit)

More ways to retrieve dataMore ways to retrieve data

• LabLab – Strangely named, but Lab – Strangely named, but Lab sampling returns “raw” values, evenly sampling returns “raw” values, evenly spaced by a user selected interval.spaced by a user selected interval.

For each interval it returns either (1) a raw For each interval it returns either (1) a raw sample that was stored at that time, or (2) sample that was stored at that time, or (2) the last good value that was storedthe last good value that was stored

Very useful when you don’t want any Very useful when you don’t want any interpolation or smoothinginterpolation or smoothing

Primarily used to trend “raw data”, with Primarily used to trend “raw data”, with evenly spaced samples to make nice chartsevenly spaced samples to make nice charts

Lab sampling ExampleLab sampling Example

TimeTime ValueValue

9:009:00 25.025.0

9:059:05 30.030.0

9:319:31 15.515.5

TimeTime ValueValue

9:009:00 25.025.0

9:059:05 30.030.0

9:109:10 30.030.0

9:159:15 30.030.0

9:209:20 30.030.0

9:259:25 30.030.0

9:309:30 30.030.0

9:359:35 15.515.5

9:409:40 15.515.5

Actual raw data stored Lab Sampling, 5 Min. Interval

Trend SamplingTrend Sampling

Trend Sampling solves a problem Trend Sampling solves a problem that few realize is there, but can be that few realize is there, but can be critical:critical:How do know that you are seeing all How do know that you are seeing all of the extrema of a process?of the extrema of a process?

Not necessarily an issue over short Not necessarily an issue over short time periods with short retrieval time periods with short retrieval intervals, but what about longer time intervals, but what about longer time periods?periods?

Trend sampling ExampleTrend sampling Example

TimeTime ValueValue

9:009:00 25.025.0

9:069:06 125.0125.0

9:109:10 50.050.0

9:159:15 25.025.0

TimeTime ValueValue

9:009:00 25.025.0

9:059:05 25.025.0

9:109:10 50.050.0

9:159:15 25.025.0

Actual raw data stored Lab Sampling, 5 Min. Interval

TimeTime ValueValue

9:009:00 25.025.0

9:309:30 25.025.0

Lab Sampling, 30 Min. IntervalSee the problem??

• With 5 Minute sampling, we see a smaller peak• With 30 minute sampling we don’t see the peak at all!

Trend sampling (cont’d)Trend sampling (cont’d)

Trend sampling solves the dilemma by returning Trend sampling solves the dilemma by returning a High-Low pair for each time intervala High-Low pair for each time interval

Trend sample divides the interval in half and Trend sample divides the interval in half and returns the high or low value (whichever occurred returns the high or low value (whichever occurred first) for the middle time samplefirst) for the middle time sample• We never miss a maximum or minimum extrema – We never miss a maximum or minimum extrema –

regardless of the sample intervalregardless of the sample interval• Able to get a very good view of the process with only a Able to get a very good view of the process with only a

small sample of data (e.g., review a year of data with small sample of data (e.g., review a year of data with month-long intervals)month-long intervals)

• Note that the Note that the time of the min/maxtime of the min/max is NOT the actual time is NOT the actual time of the raw dataof the raw data

Historian calculation modesHistorian calculation modes

CalculatioCalculationn

DescriptionDescription

AverageAverage Returns the interpolated average of the data Returns the interpolated average of the data collected during the sample periodcollected during the sample period

MinimumMinimum Returns the interpolated minimum value during Returns the interpolated minimum value during sample periodsample period

MaximumMaximum Returns the interpolated maximum value during Returns the interpolated maximum value during the sample periodthe sample period

Standard Standard DeviationDeviation

Returns the interpolated standard deviation of the Returns the interpolated standard deviation of the data during the sample perioddata during the sample period

TotalTotal Returns the integrated total of a process value Returns the integrated total of a process value based on a 24 hour sample period (e.g., calculate based on a 24 hour sample period (e.g., calculate the total flow through a flow meter with a given the total flow through a flow meter with a given flow rate) NOTE that this is NOT the same as flow rate) NOTE that this is NOT the same as adding up the raw valuesadding up the raw values

Historian calculation modesHistorian calculation modes

CalculatioCalculationn

DescriptionDescription

CountCount Returns the number of samples stored during the Returns the number of samples stored during the sample periodsample period

Raw Raw AverageAverage

Returns the arithmetic average of the raw data Returns the arithmetic average of the raw data stored during the sample periodstored during the sample period

Raw Raw Standard Standard DeviationDeviation

Returns the actual, non-interpolated standard Returns the actual, non-interpolated standard deviation of the data during the sample perioddeviation of the data during the sample period

Raw TotalRaw Total Returns the arithmetic total of the raw data stored Returns the arithmetic total of the raw data stored during the sample periodduring the sample period

Time of Time of Minimum Minimum valuevalue

Returns the time of the minimum valueReturns the time of the minimum value

Historian calculation modesHistorian calculation modes

CalculatioCalculationn

DescriptionDescription

Time of Time of Maximum Maximum valuevalue

Returns the time of the maximum valueReturns the time of the maximum value

Time Time GoodGood

Returns the total time in milliseconds that the data Returns the total time in milliseconds that the data was of good quality during the sample period. was of good quality during the sample period. Useful for calculating pump run times and the likeUseful for calculating pump run times and the like

Example: Calculation QueryExample: Calculation Query

Raw Data QueriesRaw Data Queries

When you just want the beefWhen you just want the beef

Raw Data QueriesRaw Data Queries

Raw data queries let you access the raw, Raw data queries let you access the raw, unfiltered dataunfiltered data

Very fast, but some caveats:Very fast, but some caveats:• A typo can cause A LOT of data to be A typo can cause A LOT of data to be

requestedrequested• Excel can’t handle more than 64,000 rowsExcel can’t handle more than 64,000 rows• Because the source data is stored only on a Because the source data is stored only on a

change, raw data can be confusingchange, raw data can be confusing You may get no data back based on your start & end You may get no data back based on your start & end

datesdates

Raw Data Query OptionsRaw Data Query Options

Options1. Select start & end dates

(but you must select an output range large enough to hold the data)

2. Select a time and ask for a number of samples forward or back from that point (you only need to select the first cell of the output range in this case)

Advanced TopicsAdvanced Topics

But wait… There’s More!But wait… There’s More!

Filtered Data QueriesFiltered Data Queries

Filtered queries let you retrieve data Filtered queries let you retrieve data (lab, interpolated, calculation, or (lab, interpolated, calculation, or trend) for one or more tags, based trend) for one or more tags, based upon the value of a different tagupon the value of a different tag

For example, retrieve conductivity For example, retrieve conductivity based upon pH or temperaturebased upon pH or temperature

Filtered Data Query MenuFiltered Data Query MenuSame as Query Calculated Data menu

Enter the filter tag (e.g., algae temperature), and the condition (e.g., <= 90)

Create Tags, Import and Modify Create Tags, Import and Modify DataData

Administration functionsAdministration functions• Create, modify tag propertiesCreate, modify tag properties• View on-line archives and storage statusView on-line archives and storage status

Data storage functionsData storage functions• Export data, import new data, modify stored Export data, import new data, modify stored

data*, even add comments to stored datadata*, even add comments to stored data Audit functionsAudit functions

• Export and search audit trail, import new audit Export and search audit trail, import new audit messagesmessages

*But, don’t worry.. The original value is still retained

QuestionsQuestions