generated.md

UDMI / Docs / Specs / Sequences / Generated

Generated sequences

These are the exact sequences being checked by the sequence tool. They are programmatically generated so maybe a bit cryptic, but they accurately represent the specific steps required for each test.

• bad_point_ref: Error handling for badly formed gateway point ref Test skipped: Not a proxied device
• bad_target_address: Error handling for badly formed gateway target address Test skipped: Not a proxied device
• bad_target_family: Error handling for badly formed gateway target family Test skipped: Not a proxied device
• broken_config: Check that the device correctly handles a broken (non-json) config message.
• config_logging: Check that the device publishes minimum required log entries when receiving config
• device_config_acked: Check that the device MQTT-acknowledges a sent config.
• endpoint_connection_bad_alternate: Failed connection never uses alternate registry.
• endpoint_connection_bad_hash: Failed connection because of bad hash.
• endpoint_connection_error: Push endpoint config message to device that results in a connection error.
• endpoint_connection_retry: Check repeated endpoint with same information gets retried.
• endpoint_connection_success_alternate: Check connection to an alternate project.
• endpoint_connection_success_reconnect: Check a successful reconnect to the same endpoint.
• endpoint_failure_and_restart
• endpoint_redirect_and_restart
• enumerate_families: Check enumeration of network families
• enumerate_features: Check enumeration of device features
• enumerate_multi: Check enumeration of multiple categories
• enumerate_nothing: Check enumeration of nothing at all
• enumerate_pointset: Check enumeration of device points
• extra_config: Check that the device correctly handles an extra out-of-schema field
• family_ether_addr
• family_ipv4_addr
• family_ipv6_addr: Test skipped: No ipv6 address defined in metadata
• gateway_proxy_events: Check that a gateway proxies pointset events for indicated devices Test skipped: Not a gateway
• gateway_proxy_state: Check that a gateway proxies state updates for indicated devices Test skipped: Not a gateway
• pointset_publish: Check that a device publishes pointset events
• pointset_publish_interval: Check handling of sample_rate_sec and sample_limit_sec
• pointset_remove_point: Check that pointset state does not report an unconfigured point
• pointset_request_extraneous: Check error when pointset configuration contains extraneous point
• scan_periodic_now_enumerate: Check periodic scan on a fixed schedule and enumeration
• scan_single_future: Check results of a single scan scheduled soon
• scan_single_now: Check results of a single scan scheduled in the recent past including enumeration
• scan_single_past: Check that a scan scheduled in the past never starts
• state_make_model: Check that a device publishes correct make and model information in state messages
• state_software: Check that a device publishes correct software information in state messages
• system_last_update: Check that last_update state is correctly set in response to a config update.
• system_mode_restart: Restart and connect to same endpoint and expect it returns.
• valid_serial_no

bad_point_ref (PREVIEW)

Error handling for badly formed gateway point ref

Test skipped: Not a proxied device

bad_target_address (PREVIEW)

Error handling for badly formed gateway target address

Test skipped: Not a proxied device

bad_target_family (PREVIEW)

Error handling for badly formed gateway target family

Test skipped: Not a proxied device

broken_config (STABLE)

Check that the device correctly handles a broken (non-json) config message.

1. Update config to enable debug logging
   • Set system.min_loglevel = 100
2. logging Wait until system logs level NOTICE category system.config.apply
3. Update config to force broken (invalid JSON) configuration
4. status Wait until system status level is >= WARNING (400)
5. status Check that status level is exactly ERROR (500)
6. status Check that category matches system.config.parse
7. Check that device state last_config has not been updated
8. logging Wait until system logs level DEBUG category system.config.receive
9. logging Wait until system logs level ERROR category system.config.parse
10. logging Check that log level NOTICE (or greater) category system.config.apply was not logged
11. Reset config to clean version
12. (Log level is implicitly set to INFO through config reset)
13. status Wait until system status level is not >= WARNING (400)
14. logging Wait until system logs level NOTICE category system.config.apply
15. logging Check that log level DEBUG (or greater) category system.config.receive was not logged
16. logging Check that log level DEBUG (or greater) category system.config.parse was not logged
17. Check that device state last_config has been updated

Test passed.

config_logging (STABLE)

Check that the device publishes minimum required log entries when receiving config

1. Force config update to resend config to device
2. Wait until system logs level DEBUG category system.config.receive
3. Wait until system logs level DEBUG category system.config.parse
4. Wait until system logs level NOTICE category system.config.apply

Test passed.

device_config_acked (STABLE)

Check that the device MQTT-acknowledges a sent config.

1. Wait for config acked

Test passed.

endpoint_connection_bad_alternate (PREVIEW)

Failed connection never uses alternate registry.

1. Wait until initial last_config matches config timestamp
2. Update config before blobset phase is final and stateStatus is not null
   • Add blobset = { "blobs": { "_iot_endpoint_config": { "phase": final, "generation": blob generation, "sha256": blob data hash, "url": endpoint data } } }
3. Wait for blobset phase is final and stateStatus is not null
4. Wait for blobset phase is final and stateStatus is not null
5. Wait until alternate client connect delay
6. Wait for blobset phase is final and stateStatus is null
7. Wait until restored last_config matches config timestamp
8. Update config before endpoint config blobset state not defined
   • Remove blobset.blobs._iot_endpoint_config
9. Wait for endpoint config blobset state not defined

Test passed.

endpoint_connection_bad_hash (PREVIEW)

Failed connection because of bad hash.

1. Update config before blobset status is ERROR
   • Add blobset = { "blobs": { "_iot_endpoint_config": { "phase": final, "generation": blob generation, "sha256": invalid blob data hash, "url": endpoint data } } }
2. Wait for blobset status is ERROR

Test passed.

endpoint_connection_error (PREVIEW)

Push endpoint config message to device that results in a connection error.

1. Update config before blobset entry config status is error
   • Add blobset = { "blobs": { "_iot_endpoint_config": { "phase": final, "generation": blob generation, "sha256": blob data hash, "url": endpoint data } } }
2. Wait for blobset entry config status is error
3. Update config before endpoint config blobset state not defined
   • Remove blobset.blobs._iot_endpoint_config
4. Wait for endpoint config blobset state not defined

Test passed.

endpoint_connection_retry (PREVIEW)

Check repeated endpoint with same information gets retried.

1. Update config before blobset entry config status is error
   • Add blobset = { "blobs": { "_iot_endpoint_config": { "phase": final, "generation": blob generation, "sha256": blob data hash, "url": endpoint data } } }
2. Wait for blobset entry config status is error
3. Update config before blobset entry config status is error
   • Set blobset.blobs._iot_endpoint_config.generation = new generation
4. Wait for blobset entry config status is error
5. Update config before endpoint config blobset state not defined
   • Remove blobset.blobs._iot_endpoint_config
6. Wait for endpoint config blobset state not defined

Test passed.

endpoint_connection_success_alternate (PREVIEW)

Check connection to an alternate project.

1. Wait until initial last_config matches config timestamp
2. Update config mirroring config false
   • Add blobset = { "blobs": { "_iot_endpoint_config": { "phase": final, "generation": blob generation, "sha256": blob data hash, "url": endpoint data } } }
3. Wait for blobset phase is apply and stateStatus is null
4. Wait for blobset phase is final and stateStatus is null
5. Wait until alternate last_config matches config timestamp
6. Update config before endpoint config blobset state not defined
   • Remove blobset.blobs._iot_endpoint_config
7. Wait for endpoint config blobset state not defined
8. Update config mirroring config true
   • Add blobset.blobs._iot_endpoint_config = { "phase": final, "generation": blob generation, "sha256": blob data hash, "url": endpoint data }
9. Wait for blobset phase is apply and stateStatus is null
10. Wait for blobset phase is final and stateStatus is null
11. Wait until restored last_config matches config timestamp
12. Update config before endpoint config blobset state not defined
    • Remove blobset.blobs._iot_endpoint_config
13. Wait for endpoint config blobset state not defined

Test passed.

endpoint_connection_success_reconnect (PREVIEW)

Check a successful reconnect to the same endpoint.

1. Update config before blobset phase is final and stateStatus is null
   • Add blobset = { "blobs": { "_iot_endpoint_config": { "phase": final, "generation": blob generation, "sha256": blob data hash, "url": endpoint data } } }
2. Wait for blobset phase is final and stateStatus is null
3. Update config before endpoint config blobset state not defined
   • Remove blobset.blobs._iot_endpoint_config
4. Wait for endpoint config blobset state not defined

Test passed.

endpoint_failure_and_restart (PREVIEW)

1. Update config before blobset entry config status is error
   • Add blobset = { "blobs": { "_iot_endpoint_config": { "phase": final, "generation": blob generation, "sha256": blob data hash, "url": endpoint data } } }
2. Wait for blobset entry config status is error
3. Wait for last_start is not zero
4. Check that initial count is greater than 0
5. Update config before system mode is ACTIVE
   • Add system.operation.mode = active
6. Wait for system mode is ACTIVE
7. Update config before system mode is INITIAL
   • Set system.operation.mode = restart
8. Wait for system mode is INITIAL
9. Check that restart count increased by one
10. Update config before system mode is ACTIVE
    • Set system.operation.mode = active
11. Wait for system mode is ACTIVE
12. Wait for last_config is newer than previous last_config before abort
13. Wait for last_config is newer than previous last_config after abort
14. Wait for last_start is newer than previous last_start
15. Update config before endpoint config blobset state not defined
    • Remove blobset.blobs._iot_endpoint_config
16. Wait for endpoint config blobset state not defined

Test passed.

endpoint_redirect_and_restart (PREVIEW)

1. Wait until initial last_config matches config timestamp
2. Update config mirroring config false
   • Add blobset = { "blobs": { "_iot_endpoint_config": { "phase": final, "generation": blob generation, "sha256": blob data hash, "url": endpoint data } } }
3. Wait for blobset phase is apply and stateStatus is null
4. Wait for blobset phase is final and stateStatus is null
5. Wait until alternate last_config matches config timestamp
6. Update config before endpoint config blobset state not defined
   • Remove blobset.blobs._iot_endpoint_config
7. Wait for endpoint config blobset state not defined
8. Wait for last_start is not zero
9. Check that initial count is greater than 0
10. Update config before system mode is ACTIVE
    • Add system.operation.mode = active
11. Wait for system mode is ACTIVE
12. Update config before system mode is INITIAL
    • Set system.operation.mode = restart
13. Wait for system mode is INITIAL
14. Check that restart count increased by one
15. Update config before system mode is ACTIVE
    • Set system.operation.mode = active
16. Wait for system mode is ACTIVE
17. Wait for last_config is newer than previous last_config before abort
18. Wait for last_config is newer than previous last_config after abort
19. Wait for last_start is newer than previous last_start
20. Update config mirroring config true
    • Add blobset.blobs._iot_endpoint_config = { "phase": final, "generation": blob generation, "sha256": blob data hash, "url": endpoint data }
21. Wait for blobset phase is apply and stateStatus is null
22. Wait for blobset phase is final and stateStatus is null
23. Wait until restored last_config matches config timestamp
24. Update config before endpoint config blobset state not defined
    • Remove blobset.blobs._iot_endpoint_config
25. Wait for endpoint config blobset state not defined

Test passed.

enumerate_families (PREVIEW)

Check enumeration of network families

1. Update config before enumeration not active
   • Add discovery.enumerations = { "families": entries }
2. Wait for enumeration not active
3. Update config before matching enumeration generation
   • Add discovery.generation = generation start time
4. Wait for matching enumeration generation
5. Update config before cleared enumeration generation
   • Remove discovery.generation
6. Wait for cleared enumeration generation
7. Check that family enumeration size matches
8. Check that no feature enumeration exists
9. Check that no point enumeration exists

Test passed.

enumerate_features (PREVIEW)

Check enumeration of device features

1. Update config before enumeration not active
   • Add discovery.enumerations = { "features": entries }
2. Wait for enumeration not active
3. Update config before matching enumeration generation
   • Add discovery.generation = generation start time
4. Wait for matching enumeration generation
5. Update config before cleared enumeration generation
   • Remove discovery.generation
6. Wait for cleared enumeration generation
7. Check that no family enumeration exists
8. Check that feature enumeration matches metadata
9. Check that all enumerated features are official buckets
10. Check that no point enumeration exists

Test passed.

enumerate_multi (PREVIEW)

Check enumeration of multiple categories

1. Update config before enumeration not active
   • Add discovery.enumerations = { "features": details, "families": details, "points": details }
2. Wait for enumeration not active
3. Update config before matching enumeration generation
   • Add discovery.generation = generation start time
4. Wait for matching enumeration generation
5. Update config before cleared enumeration generation
   • Remove discovery.generation
6. Wait for cleared enumeration generation
7. Check that family enumeration size matches
8. Check that feature enumeration matches metadata
9. Check that all enumerated features are official buckets
10. Check that enumerated point count matches

Test passed.

enumerate_nothing (PREVIEW)

Check enumeration of nothing at all

1. Update config before enumeration not active
   • Add discovery.enumerations = { }
2. Wait for enumeration not active
3. Update config before matching enumeration generation
   • Add discovery.generation = generation start time
4. Wait for matching enumeration generation
5. Update config before cleared enumeration generation
   • Remove discovery.generation
6. Wait for cleared enumeration generation
7. Check that no family enumeration exists
8. Check that no feature enumeration exists
9. Check that no point enumeration exists

Test passed.

enumerate_pointset (PREVIEW)

Check enumeration of device points

1. Update config before enumeration not active
   • Add discovery.enumerations = { "points": entries }
2. Wait for enumeration not active
3. Update config before matching enumeration generation
   • Add discovery.generation = generation start time
4. Wait for matching enumeration generation
5. Update config before cleared enumeration generation
   • Remove discovery.generation
6. Wait for cleared enumeration generation
7. Check that no family enumeration exists
8. Check that no feature enumeration exists
9. Check that enumerated point count matches

Test passed.

extra_config (STABLE)

Check that the device correctly handles an extra out-of-schema field

1. Wait for last_config not null
2. Wait for system operational
3. Check that system status level is not >= WARNING (400)
4. Update config before system logs level DEBUG category system.config.receive
5. Wait until system logs level DEBUG category system.config.receive
6. Wait until last_config updated
7. Wait for system operational
8. Check that system status level is not >= WARNING (400)
9. Wait until system logs level DEBUG category system.config.parse
10. Wait until system logs level NOTICE category system.config.apply
11. Update config before system logs level DEBUG category system.config.receive
12. Wait until system logs level DEBUG category system.config.receive
13. Wait for last_config updated again
14. Wait for system operational
15. Wait until system logs level DEBUG category system.config.parse
16. Wait until system logs level NOTICE category system.config.apply

Test passed.

family_ether_addr (PREVIEW)

1. Wait until device state localnet family ether is available
2. Check that family ether address matches

Test passed.

family_ipv4_addr (PREVIEW)

1. Wait until device state localnet family ipv4 is available
2. Check that family ipv4 address matches

Test passed.

family_ipv6_addr (PREVIEW)

Test skipped: No ipv6 address defined in metadata

gateway_proxy_events (BETA)

Check that a gateway proxies pointset events for indicated devices

Test skipped: Not a gateway

gateway_proxy_state (PREVIEW)

Check that a gateway proxies state updates for indicated devices

Test skipped: Not a gateway

pointset_publish (STABLE)

Check that a device publishes pointset events

1. Update config before receive a pointset event
   • Set pointset.sample_rate_sec = 10
2. Wait for receive a pointset event

Test passed.

pointset_publish_interval (STABLE)

Check handling of sample_rate_sec and sample_limit_sec

1. Update config before receive at least 4 pointset events
   • Set pointset.sample_rate_sec = 8
   • Add pointset.sample_limit_sec = 5
2. Wait for receive at least 4 pointset events
3. Check that time period between successive pointset events is between 5 and 8 seconds
4. Update config before receive at least 4 pointset events
   • Set pointset.sample_rate_sec = 18
   • Set pointset.sample_limit_sec = 15
5. Wait for receive at least 4 pointset events
6. Check that time period between successive pointset events is between 15 and 18 seconds

Test passed.

pointset_remove_point (STABLE)

Check that pointset state does not report an unconfigured point

1. Update config before pointset state matches config
   • Set pointset.sample_rate_sec = 10
2. Wait until pointset state matches config
3. Wait until pointset event contains correct points
4. Update config before pointset state does not contain removed point
   • Remove pointset.points[random_point]
5. Wait for pointset state does not contain removed point
6. Wait until pointset state matches config
7. Wait until pointset event contains correct points
8. Update config before pointset state contains restored point
   • Add pointset.points[random_point] = point configuration
9. Wait for pointset state contains restored point
10. Wait until pointset state matches config
11. Wait until pointset event contains correct points

Test passed.

pointset_request_extraneous (STABLE)

Check error when pointset configuration contains extraneous point

1. Update config before pointset state matches config
   • Set pointset.sample_rate_sec = 10
2. Wait until pointset state matches config
3. Wait until pointset event contains correct points
4. Update config before pointset state contains extraneous point error
   • Add pointset.points[extraneous_point] = point configuration
5. Wait for pointset state contains extraneous point error
6. Wait until pointset state matches config
7. Wait until pointset event contains correct points
8. Update config before pointset state removes extraneous point error
   • Remove pointset.points[extraneous_point]
9. Wait for pointset state removes extraneous point error
10. Wait until pointset state matches config
11. Wait until pointset event contains correct points

Test passed.

scan_periodic_now_enumerate+vendor (PREVIEW)

Check periodic scan on a fixed schedule and enumeration

1. Update config before discovery families defined
   • Remove discovery.families.vendor
   • Remove discovery.families.bacnet
2. Wait for discovery families defined
3. Wait until discovery family keys match
4. Wait for no scans active
5. Update config before scan iterations
   • Add discovery.families.vendor = { "generation": family generation, "scan_interval_sec": 20, "depth": details, "scan_duration_sec": 20 }
6. Wait for scan iterations
7. Check that scan did not terminate prematurely
8. Check that all events have matching refs

Test passed.

scan_single_future+vendor (PREVIEW)

Check results of a single scan scheduled soon

1. Update config before discovery families defined
   • Remove discovery.families.vendor
   • Remove discovery.families.bacnet
2. Wait for discovery families defined
3. Wait until discovery family keys match
4. Wait for no scans active
5. Update config before scheduled scan pending
   • Add discovery.families.vendor = { "generation": family generation, "scan_duration_sec": 10 }
6. Wait until scheduled scan pending
7. Wait until scheduled scan active
8. Check that scan started at time
9. Wait until scheduled scan complete
10. Check that scan completed at time
11. Check that received expected number of discovery events
12. Check that no events have discovered refs
13. Check that discovery events were valid
14. Check that received all unique event numbers
15. Check that received proper discovery start event
16. Check that received proper last discovery event
17. Check that received proper discovery termination event
18. Check that all scan addresses are unique
19. Check that all expected addresses were found
20. Check that all expected networks were found

Test passed.

scan_single_now+vendor (PREVIEW)

Check results of a single scan scheduled in the recent past including enumeration

1. Update config before discovery families defined
   • Remove discovery.families.vendor
   • Remove discovery.families.bacnet
2. Wait for discovery families defined
3. Wait until discovery family keys match
4. Wait for no scans active
5. Update config before scheduled scan active
   • Add discovery.families.vendor = { "generation": family generation, "depth": details, "scan_duration_sec": 10 }
6. Wait until scheduled scan active
7. Check that scan started at time
8. Wait until scheduled scan complete
9. Check that scan completed at time
10. Check that received expected number of discovery events
11. Check that all events have matching refs
12. Check that discovery events were valid
13. Check that received all unique event numbers
14. Check that received proper discovery start event
15. Check that received proper last discovery event
16. Check that received proper discovery termination event
17. Check that all scan addresses are unique
18. Check that all expected addresses were found
19. Check that all expected networks were found

Test passed.

scan_single_past+vendor (PREVIEW)

Check that a scan scheduled in the past never starts

1. Update config before discovery families defined
   • Remove discovery.families.vendor
   • Remove discovery.families.bacnet
2. Wait for discovery families defined
3. Wait until discovery family keys match
4. Wait for no scans active
5. Update config before scan schedule initially not active
   • Add discovery.families.vendor = { "generation": family generation, "scan_duration_sec": 10 }
6. Wait until scan schedule initially not active
7. Wait until scan schedule still not active
8. Check that there were no received discovery events

Test passed.

state_make_model (STABLE)

Check that a device publishes correct make and model information in state messages

1. Check that make and model in state matches make in metadata

Test passed.

state_software (STABLE)

Check that a device publishes correct software information in state messages

1. Check that software in metadata matches state

Test passed.

system_last_update (STABLE)

Check that last_update state is correctly set in response to a config update.

1. Wait until state last_config matches config timestamp
2. subblocks Wait until state update complete
3. Force config update to trigger another config update
4. Wait until state last_config matches config timestamp
5. subblocks Wait until state update complete
6. Force config update to trigger another config update
7. Wait until state last_config matches config timestamp
8. subblocks Wait until state update complete

Test passed.

system_mode_restart (PREVIEW)

Restart and connect to same endpoint and expect it returns.

1. Wait for last_start is not zero
2. Check that initial count is greater than 0
3. Update config before system mode is ACTIVE
   • Add system.operation.mode = active
4. Wait for system mode is ACTIVE
5. Update config before system mode is INITIAL
   • Set system.operation.mode = restart
6. Wait for system mode is INITIAL
7. Check that restart count increased by one
8. Update config before system mode is ACTIVE
   • Set system.operation.mode = active
9. Wait for system mode is ACTIVE
10. Wait for last_config is newer than previous last_config before abort
11. Wait for last_config is newer than previous last_config after abort
12. Wait for last_start is newer than previous last_start

Test passed.

valid_serial_no (STABLE)

1. Wait for received serial number matches

Test passed.