Promote set of tests to stable (#922)

Author: noursaidi

docs/guides/development.md

@@ -26,6 +26,7 @@ template `etc/schema_readme_template.md`, and must match (case sensitive) the
 value of `$section`, otherwise the schema entry is inserted under the `Other`
 section
 
+
 ## Releases
 
 The `bin/upgrade_version` tool updates :

docs/specs/sequences/generated.md

@@ -32,38 +32,38 @@ Some caveats:
 
 <!-- START GENERATED, do not edit anything after this line! -->
 * [bad_target_family](#bad_target_family-preview): Error handling for badly formed target address family
-* [broken_config](#broken_config-beta): Check that the device correctly handles a broken (non-json) config message.
-* [config_logging](#config_logging-beta): Check that the device publishes minimum required log entries when receiving config
-* [device_config_acked](#device_config_acked-beta): Check that the device MQTT-acknowledges a sent config.
+* [broken_config](#broken_config-stable): Check that the device correctly handles a broken (non-json) config message.
+* [config_logging](#config_logging-stable): Check that the device publishes minimum required log entries when receiving config
+* [device_config_acked](#device_config_acked-stable): Check that the device MQTT-acknowledges a sent config.
 * [empty_enumeration](#empty_enumeration-preview): Check enumeration of nothing at all
 * [endpoint_connection_error](#endpoint_connection_error-preview): Push endpoint config message to device that results in a connection error.
 * [endpoint_connection_retry](#endpoint_connection_retry-preview): Check repeated endpoint with same information gets retried.
 * [endpoint_connection_success_alternate](#endpoint_connection_success_alternate-preview): Check connection to an alternate project.
 * [endpoint_connection_success_reconnect](#endpoint_connection_success_reconnect-preview): Check a successful reconnect to the same endpoint.
 * [endpoint_failure_and_restart](#endpoint_failure_and_restart-preview)
 * [endpoint_redirect_and_restart](#endpoint_redirect_and_restart-preview)
-* [extra_config](#extra_config-beta): Check that the device correctly handles an extra out-of-schema field
+* [extra_config](#extra_config-stable): Check that the device correctly handles an extra out-of-schema field
 * [family_ether_addr](#family_ether_addr-preview)
 * [family_ipv4_addr](#family_ipv4_addr-preview)
 * [family_ipv6_addr](#family_ipv6_addr-preview)
 * [feature_enumeration](#feature_enumeration-preview): Check enumeration of device features
 * [gateway_proxy_events](#gateway_proxy_events-beta): Check that a gateway proxies pointset events for indicated devices
-* [pointset_publish](#pointset_publish-beta): Check that a device publishes pointset events
-* [pointset_publish_interval](#pointset_publish_interval-beta): Check handling of sample_rate_sec and sample_limit_sec
-* [pointset_remove_point](#pointset_remove_point-beta): Check that pointset state does not report an unconfigured point
-* [pointset_request_extraneous](#pointset_request_extraneous-beta): Check error when pointset configuration contains extraneous point
-* [state_make_model](#state_make_model-beta): Check that a device publishes correct make and model information in state messages
-* [state_software](#state_software-beta): Check that a device publishes correct software information in state messages
+* [pointset_publish](#pointset_publish-stable): Check that a device publishes pointset events
+* [pointset_publish_interval](#pointset_publish_interval-stable): Check handling of sample_rate_sec and sample_limit_sec
+* [pointset_remove_point](#pointset_remove_point-stable): Check that pointset state does not report an unconfigured point
+* [pointset_request_extraneous](#pointset_request_extraneous-stable): Check error when pointset configuration contains extraneous point
+* [state_make_model](#state_make_model-stable): Check that a device publishes correct make and model information in state messages
+* [state_software](#state_software-stable): Check that a device publishes correct software information in state messages
 * [system_last_update](#system_last_update-stable): Check that last_update state is correctly set in response to a config update.
-* [valid_serial_no](#valid_serial_no-beta)
+* [valid_serial_no](#valid_serial_no-stable)
 
 ## bad_target_family (PREVIEW)
 
 Error handling for badly formed target address family
 
 1. Test skipped: Not a proxied device
 
-## broken_config (BETA)
+## broken_config (STABLE)
 
 Check that the device correctly handles a broken (non-json) config message.
 
@@ -87,7 +87,7 @@ Check that the device correctly handles a broken (non-json) config message.
 1. Check that log category `system.config.receive` level `DEBUG` not logged
 1. Check that log category `system.config.parse` level `DEBUG` not logged
 
-## config_logging (BETA)
+## config_logging (STABLE)
 
 Check that the device publishes minimum required log entries when receiving config
 
@@ -98,7 +98,7 @@ Check that the device publishes minimum required log entries when receiving conf
 1. Wait for log category `system.config.parse` level `DEBUG` to be logged
 1. Wait for log category `system.config.apply` level `NOTICE` to be logged
 
-## device_config_acked (BETA)
+## device_config_acked (STABLE)
 
 Check that the device MQTT-acknowledges a sent config.
 
@@ -238,7 +238,7 @@ Check a successful reconnect to the same endpoint.
     * Remove `blobset.blobs._iot_endpoint_config`
 1. Wait for endpoint config blobset state not defined
 
-## extra_config (BETA)
+## extra_config (STABLE)
 
 Check that the device correctly handles an extra out-of-schema field
 
@@ -298,13 +298,13 @@ Check that a gateway proxies pointset events for indicated devices
 
 1. Test skipped: Not a gateway
 
-## pointset_publish (BETA)
+## pointset_publish (STABLE)
 
 Check that a device publishes pointset events
 
 1. Wait for receive a pointset event
 
-## pointset_publish_interval (BETA)
+## pointset_publish_interval (STABLE)
 
 Check handling of sample_rate_sec and sample_limit_sec
 
@@ -319,7 +319,7 @@ Check handling of sample_rate_sec and sample_limit_sec
 1. Wait for receive at least 4 pointset events
 1. Check that time period between successive pointset events is between 15 and 18 seconds
 
-## pointset_remove_point (BETA)
+## pointset_remove_point (STABLE)
 
 Check that pointset state does not report an unconfigured point
 
@@ -336,7 +336,7 @@ Check that pointset state does not report an unconfigured point
 1. Wait for pointset state matches config
 1. Wait for pointset event contains correct points
 
-## pointset_request_extraneous (BETA)
+## pointset_request_extraneous (STABLE)
 
 Check error when pointset configuration contains extraneous point
 
@@ -355,13 +355,13 @@ Check error when pointset configuration contains extraneous point
 1. Wait for pointset state matches config
 1. Wait for pointset event contains correct points
 
-## state_make_model (BETA)
+## state_make_model (STABLE)
 
 Check that a device publishes correct make and model information in state messages
 
 1. Check that make and model in state matches make in metadata
 
-## state_software (BETA)
+## state_software (STABLE)
 
 Check that a device publishes correct software information in state messages
 
@@ -380,6 +380,6 @@ Check that last_update state is correctly set in response to a config update.
 1. Wait for state last_config matches config timestamp
 1. Wait for state update complete
 
-## valid_serial_no (BETA)
+## valid_serial_no (STABLE)
 
 1. Wait for received serial number matches

etc/schema.out

@@ -1,10 +1,7 @@
 RESULT pass schemas device_state_alpha ALPHA 5/5 Schema validation passed
-RESULT pass schemas device_state_beta BETA 5/5 Schema validation passed
 RESULT pass schemas device_state_stable STABLE 5/5 Schema validation passed
-RESULT pass schemas events_pointset_beta BETA 5/5 Schema validation passed
+RESULT pass schemas events_pointset_stable STABLE 5/5 Schema validation passed
 RESULT pass schemas events_system_alpha ALPHA 5/5 Schema validation passed
-RESULT pass schemas events_system_beta BETA 5/5 Schema validation passed
 RESULT pass schemas events_system_stable STABLE 5/5 Schema validation passed
 RESULT pass schemas state_update_alpha ALPHA 5/5 Schema validation passed
-RESULT pass schemas state_update_beta BETA 5/5 Schema validation passed
 RESULT pass schemas state_update_stable STABLE 5/5 Schema validation passed

etc/schema_itemized.out

@@ -4,23 +4,23 @@
 04 pointset_remove_point
 05 too_much_state
 06 feature_enumeration
-07 valid_serial_no RESULT pass schemas device_state_beta BETA 5/5 Schema validation passed
-07 valid_serial_no RESULT pass schemas events_system_beta BETA 5/5 Schema validation passed
-07 valid_serial_no RESULT pass schemas state_update_beta BETA 5/5 Schema validation passed
+07 valid_serial_no RESULT pass schemas device_state_stable STABLE 5/5 Schema validation passed
+07 valid_serial_no RESULT pass schemas events_system_stable STABLE 5/5 Schema validation passed
+07 valid_serial_no RESULT pass schemas state_update_stable STABLE 5/5 Schema validation passed
 08 writeback_success
 09 writeback_success
 10 writeback_success
 11 pointset_publish_interval
 12 system_mode_restart
 13 config_logging
 14 broken_config
-15 broken_config RESULT pass schemas device_state_beta BETA 5/5 Schema validation passed
-15 broken_config RESULT pass schemas events_system_beta BETA 5/5 Schema validation passed
-15 broken_config RESULT pass schemas state_update_beta BETA 5/5 Schema validation passed
+15 broken_config RESULT pass schemas device_state_stable STABLE 5/5 Schema validation passed
+15 broken_config RESULT pass schemas events_system_stable STABLE 5/5 Schema validation passed
+15 broken_config RESULT pass schemas state_update_stable STABLE 5/5 Schema validation passed
 16 broken_config
-17 broken_config RESULT pass schemas device_state_beta BETA 5/5 Schema validation passed
-17 broken_config RESULT pass schemas events_system_beta BETA 5/5 Schema validation passed
-17 broken_config RESULT pass schemas state_update_beta BETA 5/5 Schema validation passed
+17 broken_config RESULT pass schemas device_state_stable STABLE 5/5 Schema validation passed
+17 broken_config RESULT pass schemas events_system_stable STABLE 5/5 Schema validation passed
+17 broken_config RESULT pass schemas state_update_stable STABLE 5/5 Schema validation passed
 18 broken_config
 19 system_last_update RESULT fail schemas state_update_stable STABLE 5/5 Schema violations found
 19 system_last_update RESULT pass schemas device_state_stable STABLE 5/5 Schema validation passed

etc/sequencer.out

@@ -19,25 +19,25 @@ RESULT pass enumeration.pointset pointset_enumeration ALPHA 5/5 Sequence complet
 RESULT skip gateway bad_target_family PREVIEW 0/0 Not a proxied device
 RESULT skip gateway gateway_attach_handling ALPHA 0/0 Not a gateway
 RESULT skip gateway gateway_proxy_events BETA 0/0 Not a gateway
-RESULT pass pointset pointset_publish BETA 5/5 Sequence complete
-RESULT pass pointset pointset_publish_interval BETA 5/5 Sequence complete
-RESULT pass pointset pointset_remove_point BETA 5/5 Sequence complete
-RESULT pass pointset pointset_request_extraneous BETA 5/5 Sequence complete
-RESULT pass system broken_config BETA 5/5 Sequence complete
+RESULT pass pointset pointset_publish STABLE 5/5 Sequence complete
+RESULT pass pointset pointset_publish_interval STABLE 5/5 Sequence complete
+RESULT pass pointset pointset_remove_point STABLE 5/5 Sequence complete
+RESULT pass pointset pointset_request_extraneous STABLE 5/5 Sequence complete
+RESULT pass system broken_config STABLE 5/5 Sequence complete
 CPBLTY pass system broken_config.logging ALPHA 1/1 Capability supported
-RESULT pass system config_logging BETA 5/5 Sequence complete
-RESULT pass system device_config_acked BETA 5/5 Sequence complete
-RESULT pass system extra_config BETA 5/5 Sequence complete
+RESULT pass system config_logging STABLE 5/5 Sequence complete
+RESULT pass system device_config_acked STABLE 5/5 Sequence complete
+RESULT pass system extra_config STABLE 5/5 Sequence complete
 RESULT pass system family_ether_addr PREVIEW 5/5 Sequence complete
 RESULT pass system family_ipv4_addr PREVIEW 5/5 Sequence complete
 RESULT pass system family_ipv6_addr PREVIEW 5/5 Sequence complete
-RESULT pass system state_make_model BETA 5/5 Sequence complete
-RESULT pass system state_software BETA 5/5 Sequence complete
+RESULT pass system state_make_model STABLE 5/5 Sequence complete
+RESULT pass system state_software STABLE 5/5 Sequence complete
 RESULT pass system system_last_update STABLE 6/6 Sequence complete
 CPBLTY pass system system_last_update.subblocks ALPHA 1/1 Capability supported
 RESULT pass system system_min_loglevel ALPHA 5/5 Sequence complete
 RESULT pass system too_much_state ALPHA 5/5 Sequence complete
-RESULT pass system valid_serial_no BETA 5/5 Sequence complete
+RESULT pass system valid_serial_no STABLE 5/5 Sequence complete
 RESULT pass system.mode system_mode_restart ALPHA 5/5 Sequence complete
 RESULT pass writeback writeback_failure ALPHA 5/5 Sequence complete
 RESULT pass writeback writeback_invalid ALPHA 5/5 Sequence complete

etc/sequencer_local.out

@@ -8,20 +8,20 @@ RESULT pass enumeration empty_enumeration PREVIEW 5/5 Sequence complete
 RESULT pass enumeration.features feature_enumeration PREVIEW 5/5 Sequence complete
 RESULT skip gateway bad_target_family PREVIEW 0/0 Not a proxied device
 RESULT skip gateway gateway_proxy_events BETA 0/0 Not a gateway
-RESULT pass pointset pointset_publish BETA 5/5 Sequence complete
-RESULT pass pointset pointset_publish_interval BETA 5/5 Sequence complete
-RESULT pass pointset pointset_remove_point BETA 5/5 Sequence complete
-RESULT pass pointset pointset_request_extraneous BETA 5/5 Sequence complete
-RESULT pass system broken_config BETA 5/5 Sequence complete
+RESULT pass pointset pointset_publish STABLE 5/5 Sequence complete
+RESULT pass pointset pointset_publish_interval STABLE 5/5 Sequence complete
+RESULT pass pointset pointset_remove_point STABLE 5/5 Sequence complete
+RESULT pass pointset pointset_request_extraneous STABLE 5/5 Sequence complete
+RESULT pass system broken_config STABLE 5/5 Sequence complete
 CPBLTY pass system broken_config.logging ALPHA 1/1 Capability supported
-RESULT pass system config_logging BETA 5/5 Sequence complete
-RESULT fail system device_config_acked BETA 0/5 Timeout waiting for config acked
-RESULT pass system extra_config BETA 5/5 Sequence complete
+RESULT pass system config_logging STABLE 5/5 Sequence complete
+RESULT fail system device_config_acked STABLE 0/5 Timeout waiting for config acked
+RESULT pass system extra_config STABLE 5/5 Sequence complete
 RESULT pass system family_ether_addr PREVIEW 5/5 Sequence complete
 RESULT pass system family_ipv4_addr PREVIEW 5/5 Sequence complete
 RESULT pass system family_ipv6_addr PREVIEW 5/5 Sequence complete
-RESULT pass system state_make_model BETA 5/5 Sequence complete
+RESULT pass system state_make_model STABLE 5/5 Sequence complete
 RESULT pass system state_software BETA 5/5 Sequence complete
 RESULT pass system system_last_update STABLE 6/6 Sequence complete
 CPBLTY pass system system_last_update.subblocks ALPHA 1/1 Capability supported
-RESULT pass system valid_serial_no BETA 5/5 Sequence complete
+RESULT pass system valid_serial_no STABLE 5/5 Sequence complete

etc/sequencer_nostate.out

@@ -8,20 +8,20 @@ RESULT skip enumeration empty_enumeration PREVIEW 0/0 State testing disabled
 RESULT skip enumeration.features feature_enumeration PREVIEW 0/0 State testing disabled
 RESULT skip gateway bad_target_family PREVIEW 0/0 Not a proxied device
 RESULT skip gateway gateway_proxy_events BETA 0/0 Not a gateway
-RESULT pass pointset pointset_publish BETA 5/5 Sequence complete
-RESULT pass pointset pointset_publish_interval BETA 5/5 Sequence complete
-RESULT skip pointset pointset_remove_point BETA 0/0 State testing disabled
-RESULT skip pointset pointset_request_extraneous BETA 0/0 State testing disabled
-RESULT skip system broken_config BETA 0/0 State testing disabled
+RESULT pass pointset pointset_publish STABLE 5/5 Sequence complete
+RESULT pass pointset pointset_publish_interval STABLE 5/5 Sequence complete
+RESULT skip pointset pointset_remove_point STABLE 0/0 State testing disabled
+RESULT skip pointset pointset_request_extraneous STABLE 0/0 State testing disabled
+RESULT skip system broken_config STABLE 0/0 State testing disabled
 CPBLTY skip system broken_config.logging ALPHA 0/0 Never executed
-RESULT skip system config_logging BETA 0/0 State testing disabled
-RESULT skip system device_config_acked BETA 0/0 State testing disabled
-RESULT skip system extra_config BETA 0/0 State testing disabled
+RESULT skip system config_logging STABLE 0/0 State testing disabled
+RESULT skip system device_config_acked STABLE 0/0 State testing disabled
+RESULT skip system extra_config STABLE 0/0 State testing disabled
 RESULT skip system family_ether_addr PREVIEW 0/0 State testing disabled
 RESULT skip system family_ipv4_addr PREVIEW 0/0 State testing disabled
 RESULT skip system family_ipv6_addr PREVIEW 0/0 State testing disabled
-RESULT skip system state_make_model BETA 0/0 State testing disabled
-RESULT skip system state_software BETA 0/0 State testing disabled
+RESULT skip system state_make_model STABLE 0/0 State testing disabled
+RESULT skip system state_software STABLE 0/0 State testing disabled
 RESULT skip system system_last_update STABLE 0/0 State testing disabled
 CPBLTY skip system system_last_update.subblocks ALPHA 0/0 Never executed
-RESULT skip system valid_serial_no BETA 0/0 State testing disabled
+RESULT skip system valid_serial_no STABLE 0/0 State testing disabled