Skip to content

mapping.md

Latest commit

 

History

History
102 lines (77 loc) · 5.03 KB

mapping.md

File metadata and controls

102 lines (77 loc) · 5.03 KB

UDMI / Docs / Specs / Mapping

Mapping

The overall "mapping" flow consists of a number of separate subflows stitched together for a complete end-to-end process to take an "unknown" device and ensure that it's properly integrated with backend services.

At a high-level, the process involves different message subgroups that handle slightly different scopes of device data:

  • (Native): Device communication using some non-UDMI native protocol (e.g. BACnet, Modbus, etc...)
  • Discovery: Messages relating to the discovery (and provisioning) of devices (e.g. messy BACnet info)
  • Mapping: Messages relating to a 'resolved' device type and ID (e.g. the device is an AHU called AHU-1)

Sequence Diagram

The overall mapping sequence involves multiple components that work together to provide the overall flow:

  • Devices: The target things that need to be discovered, configured, and ultimately communicate point data.
  • Provisioning Engine: Cloud-based agent/Provisioning Engine responsible for managing the overall discovery and mapping process (how often, what color, etc...).
  • UDMIS: The Universal Device Management Interface Services, or UDMIS, is a cloud-based infrastructure that serves as the foundation for various tools, including the Registrar and Validator.
  • Mapping Service: Mapping service that uses heuristics, ML, or a UI to convert discovery information into a concrete device/pipeline mapping.
  • Source Repo: Ultimate source of truth for the particular site, having all the devices and Gateways part of the site in the Cloud Source Repository.
  • Modeling Phase: Managing and maintaining the site model data, manually or by automated change-detection process.
sequenceDiagram
  %%{wrap}%%
  participant Devices as Devices<br/>(w/ Spotter)
  participant Provisioning Engine
  participant UDMIS
  participant Mapping Service
  participant Source Repo 
  participant Modeling Phase
  Devices->>Provisioning Engine: Incremental Results<br/>(Discovery Events)
  Provisioning Engine->>UDMIS: Incremental Results
  Devices->>Mapping Service: Discovery Complete Event
  Source Repo->>Mapping Service: Base Model Import
  UDMIS->>Mapping Service: All Results
  Note over Mapping Service: Map Results
  Mapping Service->>Source Repo: 'discovery' branch
  Source Repo-->>Modeling Phase: Notify branch updated
Loading
  • Discovery Events wraps the device info from the discovery into a UDMI-normalized format, e.g.: "Device 78F936 has points { }, with a public key XYZZYZ"
  • Discovery Complete Event having event_no as negative value.

The Mapping Service subscribes to the udmi_target topic. When it receives a Discovery Complete event, it initiates the mapping process.

Key Workflow Steps

  • Device Mapping: The service first checks if the received discovery data corresponds to an existing device in the source repository.
  • Handling New Devices: If no matching device is found based on the family (bacnet/vendor, etc.) and address combination, a new device is created. The new device is named using the convention UNK-X, where UNK stands for "Unknown" and X is an increasing number starting from 1.
  • Updating Existing Devices: If a match is found, the service updates the existing device. New details from the Pointset Complete event are appended to the device's existing data.
  • Pushing Changes: All these updates and new device creations are then pushed to the discovery branch, which triggers the subsequent modeling phase.

Example Test Setup

A standalone test-setup can be used to emulate all the requisite parts of the system.

Cloud PubSub subscriptions (the defaults) on the udmi_target topic (need to be manually added):

  • mapping-service: To process discovery complete event and complete mapping process.

Local environment setup (e.g.):

  • project_id=test-gcp-project

Mock Device and Spotter

The pubber reference provides for both the device and spotter bits of functionality (AHU-1 in this case).

$ bin/pubber sites/udmi_site_model/ $project_id AHU-1 832172
...
INFO daq.pubber.Pubber - 2022-08-30T01:46:29Z Discovery scan starting virtual as 2022-08-30T01:46:29Z
...
INFO daq.pubber.Pubber - 2022-08-30T01:45:57Z Sent 1 discovery events from virtual for Mon Aug 29 18:45:43 PDT 2022
...

Mapping Agent

The mapping agent configures the on-prem discovery node (AHU-1) to perform periodic discovery runs.

$ validator/bin/mapping agent sites/udmi_site_model/ $project_id AHU-1
...
Received new family virtual generation Mon Aug 29 18:47:43 PDT 2022
...

Mapping Service

The mapping service receives discovery complete and mapping events to perform the mapping process.

$ services/bin/mapping_service //pubsub/bos-platform-dev/namespace //gbos/bos-platform-dev/namespace tmp/udmi/sites/ --local
...
Received discovery event for generation Mon Aug 29 18:48:43 PDT 2022
...