perfsonar: schema, topology and discovery martin swany
TRANSCRIPT
perfSONAR:Schema, Topology and
Discovery
Martin Swany
OGF IPR Policies Apply
“I acknowledge that participation in this meeting is subject to the OGF Intellectual Property Policy.”Intellectual Property Notices Note Well: All statements related to the activities of the OGF and addressed to the OGF are subject to all provisions of Appendix B of GFD-C.1, which grants to the OGF and its participants certain licenses and rights in such statements. Such statements include verbal statements in OGF meetings, as well as written and electronic communications made at any time or place, which are addressed to:
the OGF plenary session, any OGF working group or portion thereof, the OGF Board of Directors, the GFSG, or any member thereof on behalf of the OGF, the ADCOM, or any member thereof on behalf of the ADCOM, any OGF mailing list, including any group list, or any other list functioning under OGF auspices, the OGF Editor or the document authoring and review process
Statements made outside of a OGF meeting, mailing list or other function, that are clearly not intended to be input to an OGF activity, group or function, are not subject to these provisions.Excerpt from Appendix B of GFD-C.1: ”Where the OGF knows of rights, or claimed rights, the OGF secretariat shall attempt to obtain from the claimant of such rights, a written assurance that upon approval by the GFSG of the relevant OGF document(s), any party will be able to obtain the right to implement, use and distribute the technology or works when implementing, using or distributing technology based upon the specific specification(s) under openly specified, reasonable, non-discriminatory terms. The working group or research group proposing the use of the technology with respect to which the proprietary rights are claimed may assist the OGF secretariat in this effort. The results of this procedure shall not affect advancement of document, except that the GFSG may defer approval where a delay may facilitate the obtaining of such assurances. The results will, however, be recorded by the OGF Secretariat, and made available. The GFSG may also direct that a summary of the results be included in any GFD published containing the specification.”
OGF Intellectual Property Policies are adapted from the IETF Intellectual Property Policies that support the Internet Standards Process.
SchemaKey Goals: Extensibility, Normalization, ReadabilityBreak representation of performance measurements down into basic elementsData and MetadataMeasurement Data
A set of of measurement events that have some value or values at a particular time
Measurement MetadataThe details about the set of measurement data
Schema Normalization
Can simply the database representation for many types of measurement data
While optimizations are certainly possible, many measurement types can be viewed as one value over time
Assists Combination/Concatenation of metrics
Creating derived metrics
Normalization helps with inferring relationships between types of metrics
Schema Basic Elements - Metadata
SubjectThe measured/tested entity
Characteristic/Event Type (Verb)What type of measurement, value, or event occurred
Parameters (Adjectives and Adverbs)How, or under what conditions, did this event occur?
Schema Basic Elements - Data
Some sort of value - DatumExistence of an event might point to the case where there no additional value
As in “Link up/down” or threshold events
TimeMust be extensible since even agreement about the right structure is not easy
E.g. UNIX timestamp vs NTP time
A MessageMessageMessage
Metadata
Data
An Object StoreStore
Metadata
Data
A Data is Linked to a MetadataMetadata
<id>someId</id>
Data
<metadataIdRef> someId</metadataIdRef>
A Metadata may be linked to another
Metadata
<id>someId</id>
Metadata<id>someOtherId</id>
<metadataIdRef> someId</metadataIdRef>
Schema Namespaces
All measurements have some sort of Data and Time
All measurements can be described by the Metadata identifying who, what and how
The specific structures of the Data and Metadata elements depend on the measurement
Approach: Consistently use Data and Metadata elements and vary the namespaces of the specific elements
Schema Namespaces - 2
Why encode the measurement/event type in the namespace?
The encoding is essentially redundant
Some components of the system can pass Data and Metadata elements through without understanding their specific structure
Allows and implementation to decide whether it supports a particular type of data or not
Allows validation based on extended (namespace-specific) schemata
Schema Namespaces and Extensibility
One key to extensibility is the use of hierarchy with delegation
Similar to OIDs in the IETF management world
The Global Grid Forum’s NM-WG has a hierarchy of characteristics
Good starting point
However, not all tools are cleanly mapped onto the Characteristic space
Often a matter of some debate
Schema Namespaces and Extensibility - 2
Organization-rooted tools namespace addresses this
Some top-level toolsping, traceroute
Easy to add new tools in organization-specific namespaces
Performance Event RepositoryAdd a schema and get a URI
Add Java classes
Versioning Example<nmwg:message id="snmpmsg5" type="MetadataKeyRequest" xmlns:nmwg="http://ggf.org/ns/nmwg/base/2.0/" xmlns:snmp="http://ggf.org/ns/nmwg/tools/snmp/2.0/" xmlns:nmwgt="http://ggf.org/ns/nmwg/topology/2.0/">
<nmwg:metadata id="meta2"> <snmp:subject id="subject2"> <nmwgt:interface> <nmwgt:hostName>scotch.pc.cis.udel.edu</nmwgt:hostName> <nmwgt:ipAddress type="ipv4">128.4.133.141</nmwgt:ipAddress> </nmwgt:interface> </snmp:subject> <nmwg:eventType>SNMP.Get.Request</nmwg:eventType> </nmwg:metadata>
Versioning
Separate revisions of base schema and eventType schema
Open Issues:Query for supported versions must be defined
This will enable a migration strategy
Topology Schema
Topology SchemaTopology schema
Reusable Subject elements for common casesAlso reduces redundancy
Relationships between Subjects
Same basic structure at all layersNetworks are graphs
Define:NodesInterfacesLinksNetworks
Topology
Topology - Recursive Links
Topology SchemaStructured by layers and the same elements recurring thereVaried again by namespaces
Reuse visualization logic, etc.
4 Layers: Base (both abstract and L1), L2, L3, L4
<?xml version="1.0" encoding="UTF-8"?>
<nmwg:store xmlns:nmwg="http://ggf.org/ns/nmwg/base/2.0/" xmlns:nmwgt="http://ggf.org/ns/nmwg/topology/3.0/" xmlns:nmwgtl3="http://ggf.org/ns/nmwg/topology/l3/3.0/" xmlns:nmwgtl2="http://ggf.org/ns/nmwg/topology/l2/3.0/">
Relationships between Subjects To completely capture the relationships, we need to do a few more thingsRecursive definition of links
Logical links consist of physical links
Ordered lists of linksLike above, but we need to introduce an Index attribute
NetworksPhysically consist of links but that is not always the most convenient logical viewSpecial element to which Interfaces or Links belong
Relationships between Subjects
Elements at the same layer have relationshipsA link “contains” two interfaces
At Layer2 or Layer3
Elements of the same sort have relationships between themselves at different layers
A Layer 1 Interface (physical NIC) can have one or more Layer 2 Interfaces, which can each have one or more Layer 3 Interfaces
Node is special Since a Node doesn’t really have any higher-layer characteristic independent of its Interfaces
Schema Status
Three documents in preparation for OGFBase Schema
Topology Schema
Extension guide With examples of utilization, traceroute, ping
Drafts to NMWG by January
OGF meeting in February
Schema Status
Three schema componentsBase
Defines Message, Store, Data, Metadata, Parameters
TopologyDefines Subjects that can appear in the Metadata and Version 3 represents their relationships as well
EventTypesEach data type extends the Base and can define what Parameters are acceptable and what subjects are required
Outstanding Issues
Namespaces for Message typesCurrently a simple text field, but namespaces might make Message syntax/semantics easier to track (and version)
Uniform EventTypes that match the EventType namespace
XML Factoring to further reduce redundant information
Some work at UD on “views”