milestone analytics events 2.0; developers...

Analytics Events 2.0 Developer’s Manual

Milestone

Milestone Analytics Events 2.0; Developer’s Manual

www.milestonesys.com Target Audience for this Document

Target Audience for this Document

This document is intended for video content analysis system developers wishing to integrate with the Milestone Analytics Events feature. The content of this document is not relevant for regular surveillance

system users. Furthermore, this document is mainly intended for developers who are prevented from using the Milestone Integration Platform (MIP) Software Development Kit (SDK). The MIP SDK allows a more tight integration with Milestone XProtect products than what is described in this document. Among

other things, this includes programmatically defining new event types in a plug-in to the event server and posting these from any MIP environment, thus freeing end-users from adding these manually. The

MIP SDK however restricts your application to .NET on Windows. Please see the Analytics Integration and Access Control samples in the Plug-in Integration Application Catalog in the MIP SDK Documentation.

MAE-AnaEvent-dm-2-310111


www.milestonesys.com Contents

Contents

COPYRIGHT, TRADEMARKS AND IMPORTANT INFORMATION............................................................ 4

INTRODUCTION .......................................................... 5

XProtext Corporate Alarms & Analytic Events ....................................... 5

XProtect Analytics & MAD Compatibility ............................................... 6

ARCHITECTURE AND DATA FLOW ............................... 7

Analytics Event XML .............................................................................. 8

PROTOCOL ................................................................ 10

INTEGRATING WITH ANALYTICS EVENTS ................. 11

APPENDIX A ............................................................. 13


www.milestonesys.com Copyright, Trademarks and …

Copyright, Trademarks and Important

Information

Copyright © 2011 Milestone Systems A/S.

Trademarks XProtect is a registered trademark of Milestone Systems A/S. Microsoft and Windows are registered trademarks of Microsoft Corporation.

All other trademarks mentioned in this document are trademarks of their respective owners.

Disclaimer This document is intended for general information purposes only, and due care has been taken in

its preparation.

Any risk arising from the use of this information rests with the recipient, and nothing herein should be construed as constituting any kind of warranty. Milestone Systems A/S reserve the right to make adjustments without prior notification. All names of people and organizations used in this document's examples are fictitious. Any resemblance to any actual organization or person, living or dead, is purely coincidental and

unintended.


www.milestonesys.com Introduction

Introduction

Video content analysis—VCA—functionality span from face recognition over advanced motion detection to complex behavioral analysis, where various types of abnormal behavior—both of

humans and vehicles—can be detected. Currently, there are roughly 35 important VCA players. This number keeps growing, and the increased competition will keep improving the functionality and performance of these systems.

With the Milestone Analytics Events feature, VCA systems can integrate with and hook into XProtect Corporate and the XProtect event server. Even multiple VCA systems can co-exist in a

Milestone environment at the same time and may work together to create an optimal solution for a given customer in a given situation.

XProtext Corporate Alarms & Analytics Events

In XProtect Corporate, the Analytics Events feature is handled by the Alarms feature. Based on functionality handled in the XProtect event server, the Alarms feature provides central overview, control and scalability of alarms in any number of federated XProtect Enterprise and XProtect Corporate installations. It can be configured to generate alarms based on either:

System related events; for example, motion, server responding/not responding,

archiving problems, lack of disk space, etc. VCA-based events (a.k.a. analytic events); typically data received from an external VCA-

based application. To ease overview, delegation and handling of alarms, these will appear in the XProtect Smart

Client’s alarm list and can, if relevant, be integrated with map functionality. The Alarms feature is thus a powerful monitoring tool, providing instant overview of alarms and possible technical problems. How does the XProtect Alarms feature and the XProtect event server work?

1. Surveillance System & Event Server

2. XProtect Corporate Management Client

3. Smart Client 4. Alarm Configuration 5. Alarm Data Flow

Later in this document, the architecture and data flow of the Analytics Events feature is described. Furthermore, how to integrate VCA systems with XProtect Corporate by sending events to the

XProtect event server, triggering alarms, and managing these in the Smart Client are discussed.


www.milestonesys.com Introduction

XProtect Analytics & MAD Compatibility

The Milestone Alert Data (MAD) format was introduced with XProtect Analytics 2.0, and allowed the sending of analytics alerts directly to the analytics server over TCP/IP via the Generic VA interface.

With the Analytics Events feature it is possible to send MAD formatted alerts to the XProtect event server over TCP/IP in exactly the same way. Therefore, no changes are needed to current XProtect Analytics integrations in order for these to work with the XProtect event server. However, in the XProtect event server, all received alerts are treated as analytics events, and it is

therefore necessary to create an alarm definition in the XProtect Management Client before alarms

can be triggered and managed in the XProtect Smart Client. The analytics events format supports all the information of the MAD format, as well as some additional information. For developers who have not already implemented a solution using the MAD format it is recommended using the analytics events format instead.


www.milestonesys.com Architecture and Data Flow

Architecture and Data Flow

VCA Server Data Flow

Camera with On-board VCA Data Flow

The overall data flow of the framework is as follows: the underlying analytic systems detects a predefined behavior (which is configured by setting up rules on the third-party VCA system), and sends out an analytics event to the XProtect event server using the protocol and format described

in this document. The address of the XProtect event server should be set up on the third-party VCA. The XProtect event server then matches the event message against a predefined alarm definition, and—if it matches—an alarm is triggered. This alarm appears in the XProtect Smart Client alongside any other alarms generated by the

XProtect event server, and is managed in exactly the same way.



Screenshot from XProtect Smart Client: Effects of analytics events in the XProtect Smart Client with loaded data in the alert list

Analytics Event XML

Analytics events have the same ”status” as internal events and can generate alarms the same way. When sending analytics events to the XProtect event server, the analytics events should be formatted in XML, as defined by the XML Schema Definition (XSD) in Appendix A. The most important element in the analytics event format is the Message in the EventHeader, which

is used to match against the alarm definition created in the management client, and determines

whether or not an alarm should be triggered. Another important element in the analytics event format is the Source in the EventHeader. For VCA

systems, this will typically be a camera. If an alarm is triggered, the source camera will be displayed in the XProtect Smart Client when the alarm is selected. In addition, any overlay information specified in the ObjectList and RuleList will also be displayed on the camera, when

the XProtect Smart Client is in playback mode. Cameras may be specified using the IP address in the Name element of the source.

Finally, the EventHeader ID must be specified. It is recommended to pass a zero GUID, as shown in

the examples, in which case a new GUID will be assigned by the XProtect event server.

This is an example of an analytics event with the IP address of a source camera, and a single object with a bounding box: <?xml version="1.0" encoding="utf-8"?> <AnalyticsEvent xmlns:i="http://www.w3.org/2001/XMLSchema-instance" xmlns="urn:milestone-systems"> <EventHeader> <ID>00000000-0000-0000-0000-000000000000</ID> <Timestamp>2011-01-26T15:19:39.7342498+01:00</Timestamp> <Message>My Analytics Event</Message> <Source> <Name>10.100.50.23</Name> </Source> </EventHeader> <ObjectList>



<Object> <BoundingBox> <Top>0.2</Top> <Left>0.4</Left> <Bottom>0.9</Bottom> <Right>0.6</Right> </BoundingBox> </Object> </ObjectList> </AnalyticsEvent>

Furthermore, specific channels on video servers may be specified by appending the (0-indexed) channel number separated by a comma. This would therefore represent “Camera 4 on

10.100.50.23”: <Source> <Name>10.100.50.23,3</Name> </Source>

Alternatively, the source can be specified precisely using the “Fully Qualified ID” of the device in the Source FQID element. This is required for non-camera sources, but may also provide a small

performance gain. In this example, the device and server ID’s are used, as well as the static “kind” of a camera: <Source> <FQID> <ServerId> <Id>beef2f96-03db-4815-a33c-adcc9fcb8610</Id> </ServerId> <ObjectId>6832da9f-a09b-49c2-bbaa-d52f430b8a90</ObjectId> <Kind>5135ba21-f1dc-4321-806a-6ce2017343c0</Kind> </FQID> </Source>


www.milestonesys.com Protocol

Protocol

XML formatted analytics event may be sent to the XProtect event server using the HTTP POST method or directly over a TCP/IP connection. A small performance gain may be achieved using

HTTP. When using HTTP, the content length must be specified. A minimal sample of an analytics event including an HTTP header is as follows: POST / HTTP/1.1 Content-Type: text/xml Content-Length: 426 Connection: Keep-Alive <?xml version=”1.0” encoding=”utf-8”?> <AnalyticsEvent xmlns:i=”http://www.w3.org/2001/XMLSchema-instance” xmlns=”urn:milestone-systems”> <EventHeader> <ID>00000000-0000-0000-0000-000000000000</ID> <Timestamp>2011-01-26T15:19:39.7342498+01:00</Timestamp> <Message>My Analytics Event</Message> <Source> <Name>10.100.50.23</Name> </Source> </EventHeader> </AnalyticsEvent>

The XProtect event server will respond with an HTTP response regardless of the method used to send the analytics event. The HTTP response will report on the success of the processing of the received data. The following HTTP responses may be expected:

HTTP Status Code Content Description

200 OK May contain a warning if the analytics event is unknown by the XProtect event server.

The analytics event was successfully received.

400 Bad Request Serialization exception message.

The received data is not in a valid format.

403 Forbidden None. The sender is not on the list of

allowed addresses.

500 Internal Server Error Error message. An internal error in the

XProtect event server. See the event server log for more information.


www.milestonesys.com Integrating with Analytics Events

Integrating with Analytics Events

Configuring analytics events is done from the XProtect Corporate Management Client. The following screenshots offer a brief introduction to the configuration process. See the XProtect Corporate

Management Client documentation for more details.

Part one - enabling the service and setting up security. A list of allowed addresses can be used to control who can send event data to the system. Furthermore, in this example, the server will listen on port 9090.

Part two – creating the analytic event. It is also possible to create a description of the event and test it.


www.milestonesys.com Integrating with Analytics Events

Part three – Using the analytics event in an alarm definition. The analytics event can now be used as a source for an alarm.

www.milestonesys.com Appendix A

Appendix A

<?xml version="1.0" encoding="utf-8"?> <xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema" xmlns="urn:milestone-systems" targetNamespace="urn:milestone-systems" elementFormDefault="qualified" version="1.0">  <xs:element name="AnalyticsEvent"> <xs:complexType> <xs:sequence>  <xs:element minOccurs="1" maxOccurs="1" name="EventHeader" type="EventHeader" />  <xs:element minOccurs="0" maxOccurs="1" name="Description" type="xs:string" />  <xs:element minOccurs="0" maxOccurs="1" name="StartTime" type="xs:dateTime" />  <xs:element minOccurs="0" maxOccurs="1" name="EndTime" type="xs:dateTime" />  <xs:element minOccurs="0" maxOccurs="1" name="Location" type="xs:string" />  <xs:element minOccurs="0" maxOccurs="1" name="Count" type="xs:unsignedInt" />  <xs:element minOccurs="0" maxOccurs="1" name="RuleList" type="RuleList" />  <xs:element minOccurs="0" maxOccurs="1" name="ObjectList" type="AnalyticsObjectList" />  <xs:element minOccurs="0" maxOccurs="1" name="ReferenceList" type="ReferenceList" />  <xs:element minOccurs="0" maxOccurs="1" name="SnapshotList" type="SnapshotList" />  <xs:element minOccurs="0" maxOccurs="1" name="Vendor" type="Vendor" /> </xs:sequence> </xs:complexType> </xs:element>  <xs:complexType name="EventHeader"> <xs:sequence>  <xs:element minOccurs="1" maxOccurs="1" name="ID" type="GUID" />  <xs:element minOccurs="1" maxOccurs="1" name="Timestamp" type="xs:dateTime" />

<xs:element minOccurs="0" maxOccurs="1" name="Class" type="xs:string" />  <xs:element minOccurs="0" maxOccurs="1" name="Type" type="xs:string" />  <xs:element minOccurs="0" maxOccurs="1" name="Version" type="xs:string" />  <xs:element minOccurs="0" maxOccurs="1" name="Priority" type="xs:unsignedShort" />  <xs:element minOccurs="0" maxOccurs="1" name="Name" type="xs:string" />  <xs:element minOccurs="1" maxOccurs="1" name="Message" type="xs:string" />  <xs:element minOccurs="0" maxOccurs="1" name="CustomTag" type="xs:string" />  <xs:element minOccurs="0" maxOccurs="1" name="Source" type="EventSource" /> </xs:sequence> </xs:complexType>  <xs:simpleType name="GUID"> <xs:restriction base="xs:string"> <xs:pattern value="[a-fA-F0-9]{8}-[a-fA-F0-9]{4}-[a-fA-F0-9]{4}-[a-fA-F0-9]{4}-[a-fA-F0-9]{12}"/> </xs:restriction> </xs:simpleType>  <xs:complexType name="EventSource"> <xs:sequence>  <xs:element minOccurs="0" maxOccurs="1" name="Name" type="xs:string" />  <xs:element minOccurs="0" maxOccurs="1" name="Description" type="xs:string" />  <xs:element minOccurs="0" maxOccurs="1" name="FQID" type="FQID" /> </xs:sequence> </xs:complexType>  <xs:complexType name ="FQID"> <xs:sequence>  <xs:element name="ServerId" minOccurs="0"> <xs:complexType> <xs:sequence>  <xs:element minOccurs="0" maxOccurs="1" name="Type" type="xs:string" />  <xs:element minOccurs="0" maxOccurs="1" name="Hostname" type="xs:string" />  <xs:element minOccurs="0" maxOccurs="1" name="Port" type="xs:int" />

<xs:element minOccurs="1" maxOccurs="1" name="Id" type="GUID" /> </xs:sequence> </xs:complexType> </xs:element>  <xs:element minOccurs="0" maxOccurs="1" name="ParentId" type="GUID" />  <xs:element minOccurs="0" maxOccurs="1" name="ObjectId" type="GUID" />  <xs:element minOccurs="0" maxOccurs="1" name="ObjectIdString" type="xs:string" />  <xs:element minOccurs="1" maxOccurs="1" name="FolderType" type="xs:unsignedByte" />  <xs:element minOccurs="1" maxOccurs="1" name="Kind" type="GUID" /> </xs:sequence> </xs:complexType>  <xs:complexType name="RuleList"> <xs:sequence>  <xs:element minOccurs="0" maxOccurs="unbounded" name="Rule" type="Rule" /> </xs:sequence> </xs:complexType>  <xs:complexType name="Rule"> <xs:sequence>  <xs:element minOccurs="0" maxOccurs="1" name="ID" type="GUID" />  <xs:element minOccurs="0" maxOccurs="1" name="Name" type="xs:string" />  <xs:element minOccurs="0" maxOccurs="1" name="Type" type="xs:string" />  <xs:element minOccurs="0" maxOccurs="1" name="PolygonList" type="PolygonList" /> </xs:sequence> </xs:complexType>  <xs:complexType name="PolygonList"> <xs:sequence>  <xs:element minOccurs="0" maxOccurs="unbounded" name="Polygon" type="TPolygon" /> </xs:sequence>

</xs:complexType>  <xs:complexType name="TPolygon"> <xs:sequence>  <xs:element minOccurs="0" maxOccurs="1" name="Closed" type="xs:boolean" />  <xs:element minOccurs="0" maxOccurs="1" name="Color" type="TColor" />  <xs:element minOccurs="0" maxOccurs="1" name="FillColor" type="TColor" />  <xs:element minOccurs="0" maxOccurs="1" name="PointList" type="PointList" /> </xs:sequence> </xs:complexType>  <xs:complexType name="PointList"> <xs:sequence>  <xs:element minOccurs="0" maxOccurs="unbounded" name="Point" type="TPoint" /> </xs:sequence> </xs:complexType>  <xs:complexType name="TPoint"> <xs:sequence>  <xs:element minOccurs="1" maxOccurs="1" name="X" type="xs:double" />  <xs:element minOccurs="1" maxOccurs="1" name="Y" type="xs:double" /> </xs:sequence> </xs:complexType>  <xs:complexType name="TColor"> <xs:sequence>  <xs:element minOccurs="1" maxOccurs="1" name="A" type="xs:unsignedByte" />  <xs:element minOccurs="1" maxOccurs="1" name="R" type="xs:unsignedByte" />  <xs:element minOccurs="1" maxOccurs="1" name="G" type="xs:unsignedByte" />  <xs:element minOccurs="1" maxOccurs="1" name="B" type="xs:unsignedByte" /> </xs:sequence> </xs:complexType>  <xs:complexType name="SnapshotList"> <xs:sequence>  <xs:element minOccurs="0" maxOccurs="unbounded" name="Snapshot" type="Snapshot" /> </xs:sequence> </xs:complexType>  <xs:complexType name="Snapshot"> <xs:sequence>  <xs:element minOccurs="1" maxOccurs="1" name="TimeOffset" type="xs:int"/>

<xs:element minOccurs="0" maxOccurs="1" name="Width" type="xs:unsignedInt" />  <xs:element minOccurs="0" maxOccurs="1" name="Height" type="xs:unsignedInt" />  <xs:element minOccurs="0" maxOccurs="1" name="HasOverlay" type="xs:boolean" />  <xs:element minOccurs="0" maxOccurs="1" name="SizeInBytes" type="xs:unsignedInt" />  <xs:element minOccurs="0" maxOccurs="1" name="Image" type="xs:base64Binary" />  <xs:element minOccurs="0" maxOccurs="1" name="Path" type="xs:string" />  <xs:element minOccurs="0" maxOccurs="1" name="FQID" type="FQID" /> </xs:sequence> </xs:complexType>  <xs:complexType name="AnalyticsObjectList"> <xs:sequence>  <xs:element name="Object" type="AnalyticsObject" minOccurs="0" maxOccurs="unbounded" /> </xs:sequence> </xs:complexType>  <xs:complexType name="AnalyticsObject"> <xs:sequence>  <xs:element minOccurs="0" maxOccurs="1" name="ID" type="GUID" />  <xs:element minOccurs="0" maxOccurs="1" name="Name" type="xs:string" />  <xs:element minOccurs="0" maxOccurs="1" name="Type" type="xs:string" />  <xs:element minOccurs="0" maxOccurs="1" name="Description" type="xs:string" />  <xs:element minOccurs="0" maxOccurs="1" name="Confidence" type="xs:double" />  <xs:element minOccurs="0" maxOccurs="1" name="Value" type="xs:string" />  <xs:element minOccurs="0" maxOccurs="1" name="AlarmTrigger" type="xs:boolean" />  <xs:element minOccurs="0" maxOccurs="1" name="Removed" type="xs:boolean" />  <xs:element minOccurs="0" maxOccurs="1" name="Color" type="xs:string" />  <xs:element minOccurs="0" maxOccurs="1" name="Size" type="xs:double" />  <xs:element minOccurs="0" maxOccurs="1" name="SizeUnit" type="xs:string" />  <xs:element minOccurs="0" maxOccurs="1" name="BoundingBox" type="BoundingBox" />

<xs:element minOccurs="0" maxOccurs="1" name="Polygon" type="TPolygon" />  <xs:element minOccurs="0" maxOccurs="1" name="Motion" type="ObjectMotion" />  <xs:element minOccurs="0" maxOccurs="1" name ="Mask" type="xs:base64Binary" /> </xs:sequence> </xs:complexType>  <xs:complexType name="BoundingBox"> <xs:sequence>  <xs:element minOccurs="1" maxOccurs="1" name="Top" type="xs:double" />  <xs:element minOccurs="1" maxOccurs="1" name="Left" type="xs:double" />  <xs:element minOccurs="1" maxOccurs="1" name="Bottom" type="xs:double" />  <xs:element minOccurs="1" maxOccurs="1" name="Right" type="xs:double" />  <xs:element minOccurs="0" maxOccurs="1" name="Color" type="TColor" /> </xs:sequence> </xs:complexType>  <xs:complexType name="ObjectMotion"> <xs:sequence>  <xs:element minOccurs="0" maxOccurs="1" name="Speed" type="xs:double" />  <xs:element minOccurs="0" maxOccurs="1" name="SpeedUnit" type="xs:string" />  <xs:element minOccurs="0" maxOccurs="1" name="Path" type="TPolygon" /> </xs:sequence> </xs:complexType>  <xs:complexType name="ReferenceList"> <xs:sequence>  <xs:element minOccurs="0" maxOccurs="unbounded" name="Reference" type="Reference" /> </xs:sequence> </xs:complexType>  <xs:complexType name="Reference"> <xs:sequence>  <xs:element minOccurs="0" maxOccurs="1" name="FQID" type="FQID" /> </xs:sequence> </xs:complexType>  <xs:complexType name="Vendor"> <xs:sequence>  <xs:element minOccurs="0" maxOccurs="1" name="Name" type="xs:string" />

<xs:element minOccurs="0" maxOccurs="1" name="CustomData" type="xs:string" /> </xs:sequence> </xs:complexType> </xs:schema>

Milestone Systems offices are located across the world. For details about office addresses, phone and fax numbers, visit

www.milestonesys.com.

milestone analytics events 2.0; developers...

Documents