Merge pull request w3f#54 from davxy/improve-stats-vectors

Improve statistics vectors

Author: davxy

accumulate/README.md

@@ -16,6 +16,14 @@ artifact frequently varies, even when generated from identical source code. As
 a result, you can just rely on the code blob embedded in the test vectors, which
 is available within the `accounts` map.
 
+## Statistics
+
+A subset of service activity statistics ($π_S$) updated by the STF subsystem used
+to process these test vectors.
+
+In particular the `accumulate-count` (a.0), `accumulate-gas-used` (a.1),
+`on-transfers-count` (t.0) and `on-transfers-gas-used` (t.1).
+
 ## Tiny Vectors
 
 - [no_available_reports-1](./tiny/no_available_reports-1.json)

accumulate/accumulate.asn

@@ -39,19 +39,29 @@ AccountsMapEntry ::= SEQUENCE {
 }
 
 State ::= SEQUENCE {
+	-- [τ] Most recent block's timeslot.
     slot TimeSlot,
+	-- [n_0']  Posterior entropy accumulator.
     entropy Entropy,
+	-- [ϑ] Work packages ready to be accumulated.
     ready-queue ReadyQueue,
+	-- [ξ] Work packages recently accumulated.
     accumulated AccumulatedQueue,
+	-- [χ] Privileged service indices.
     privileges Privileges,
+	-- [π_S] Services statistics.
+	-- Note: Only stats related to accumulate STF are updated, in particular:
+	-- accumulate_count (a.0), accumulate_gas_used (a.1),
+	-- on_transfers_count (t.0), on_transfers_gas_used (t.1)
     statistics ServicesStatistics,
 	-- [δ] Relevant services account data. Refer to T(σ) in GP Appendix D.
     accounts SEQUENCE OF AccountsMapEntry
 }
 
 Input ::= SEQUENCE {
-    -- [H_t] Block's timeslot.
+	-- [H_t] Current time slot as found within the block header.
     slot TimeSlot,
+	-- Newly available work reports (W).
     reports SEQUENCE OF WorkReport
 }
 

jam-types-asn/jam-types.asn

@@ -254,7 +254,7 @@ BlocksHistory ::= SEQUENCE (SIZE(0..max-blocks-history)) OF BlockInfo
 -- Statistics
 --------------------------------------------
 
-ActivityRecord ::= SEQUENCE {
+ValidatorActivityRecord ::= SEQUENCE {
     blocks      U32,
     tickets     U32,
     pre-images  U32,
@@ -263,7 +263,7 @@ ActivityRecord ::= SEQUENCE {
     assurances  U32
 }
 
-ActivityRecords ::= SEQUENCE (SIZE(validators-count)) OF ActivityRecord
+ValidatorsStatistics ::= SEQUENCE (SIZE(validators-count)) OF ValidatorActivityRecord
 
 CoreActivityRecord ::= SEQUENCE {
 	-- Amount of bytes which are placed into either Audits or Segments DA.
@@ -323,8 +323,8 @@ ServicesStatisticsMapEntry ::= SEQUENCE {
 ServicesStatistics ::= SEQUENCE OF ServicesStatisticsMapEntry
 
 Statistics ::= SEQUENCE {
-    vals-current ActivityRecords,
-    vals-last    ActivityRecords,
+    vals-curr    ValidatorsStatistics,
+    vals-last    ValidatorsStatistics,
     cores        CoresStatistics,
     services     ServicesStatistics
 }

preimages/README.md

@@ -5,9 +5,12 @@ and this subsystem STF specific [schema](./preimages.asn).
 
 ## Statistics
 
-A subset of `ServiceActivityRecord` structure is updated by the STF involved processing these test vectors.
-In particular the `provided-count` and `provided-size` items are updated when new preimages are provided
-to be included in the chain storage via the extrinsic.
+A subset of service activity statistics ($π_S$) updated by the STF subsystem used
+to process these test vectors.
+
+In particular the `provided-count` (p.0) and `provided-size` (p.1) items are
+updated when new preimages are provided to be included in the chain storage via
+the extrinsic.
 
 ## Vectors
 

preimages/preimages.asn

@@ -36,16 +36,17 @@ AccountsMapEntry ::= SEQUENCE {
 }
 
 State ::= SEQUENCE {
-	-- Relevant services account data [δ]. Refer to T(σ) in GP Appendix D.
+	-- [δ] Relevant services account data. Refer to T(σ) in GP Appendix D.
     accounts SEQUENCE OF AccountsMapEntry,
-    -- Statistics updated by the preimages STF subsystem.
+	-- [π_S] Services statistics.
+	-- Note: Only stats related to preimages STF are updated, in particular:
+	-- provided_count (p.0), provided_size (p.1).
     statistics ServicesStatistics
 }
 
 Input ::= SEQUENCE {
     -- [E_P] Preimages extrinsic.
     preimages PreimagesExtrinsic,
-
     -- [H_t] Block's timeslot.
     slot TimeSlot
 }

reports/README.md

@@ -12,6 +12,16 @@ Any update to the pools is delegated to the dedicated ["authorizations"](../auth
 subsystem, which is tasked with removing consumed items from the authorization
 pools and introduce new authorizers from the authorization queue.
 
+## Statistics
+
+A subset of service ($π_S$) and core ($π_S$) activity statistics updated by the
+STF subsystem used to process these test vectors.
+
+In particular, for cores statistics we may update imports (i), exports (e),
+extrinsic_size (z), extrinsic_count (x), bundle_size (b), gas_used (u).
+For services statistics we may update refinement_count (r.0), refinement_gas_used (r.1),
+imports (i), exports (x), extrinsic_size (z), extrinsic_count (x).
+
 ## Tiny Vectors
 
 - [report_curr_rotation](./tiny/report_curr_rotation-1.json) 🟢

reports/reports.asn

@@ -51,10 +51,16 @@ State ::= SEQUENCE {
     -- [δ] Relevant services account data. Refer to T(σ) in GP Appendix D.
     accounts SEQUENCE OF AccountsMapEntry,
 
-    -- Cores statistics.
+	-- [π_C] Cores statistics.
+	-- Note: Only stats related to reports STF are updated, in particular:
+	-- imports (i), exports (e), extrinsic_size (z), extrinsic_count (x),
+	-- bundle_size (b), gas_used (u).
     cores-statistics CoresStatistics,
 
-    -- Services statistics.
+	-- [π_S] Services statistics.
+	-- Note: Only stats related to reports STF are updated, in particular:
+	-- refinement_count (r.0), refinement_gas_used (r.1), imports (i),
+	-- exports (x), extrinsic_size (z), extrinsic_count (x).
     services-statistics ServicesStatistics
 }
 

statistics/README.md

@@ -1,8 +1,19 @@
-# Statistics STF Test Vectors
+# Validators Statistics STF Test Vectors
 
 Both JSON and SCALE formats conform to the JAM ASN.1 [schema](../jam-types-asn/jam-types.asn)
 and this subsystem STF specific [schema](./statistics.asn).
 
+## Validators Statistics
+
+These vectors exclusively contribute to updating validator-related statistics
+($π_V$ and $π_L$).
+
+In contrast, service and core statistics ($π_S$ and $π_C$) are updated by
+vectors that more directly influence the state changes relevant to those
+metrics (i.e. see [preimages](../preimages/README.md#statistics),
+[reports](../reports/README.md#statistics) and
+[accumulate](../accumulate/README.md#statistics)).
+
 ## ⚠️Extrinsic Semantic Validity
 
 These vectors are intended just to advance the statistics of validators.