Persistency CFT [2024-04-17, 2:30 - 4:00 pm]

[guysagnes]

Participants

@johannes-esr (ACN)
@hbragge-acn (ACN)
@sbachmann-qorix (Qorix) ✅
@vinodreddy-g (Qorix) ✅
@armin-acn (ACN)
Uwe Maucher (BMW)
Ahmed Elsaka (BMW) ✅
@guysagnes (Conti)- ✅
Jannik Szwajczyk (Conti) ✅
@vbogdanb - Vasile-Bogdan Badicu (Conti)✅
@joshualicht -Joshua Licht (Conti)✅
@arsibo
Eva Peiker
@PandaeDo ✅

report: ✅ / ❌

Agenda

• Feedback Architecture workshop

  • https://github.com/orgs/eclipse-score/discussions/764

• presentation of the C++ API (Conti)

• presentation of KVS Tool (Conti)

  • Initial Implementation of the KVS_Tool inc_mw_per#5

• reuse the module_template reported under:
  -> https://github.com/eclipse-score/module_template

Meeting Minutes

Related Persistency CFT Board:
Persistency FT

Next Steps

• Proposal for C++ API
• Extension of the API to support file system access

open tasks - reported by @johannes-esr :

• Rebase https://github.com/eclipse-score/inc_mw_per repository on the https://github.com/eclipse-score/module_template repository, enable bazel build
• Re-work of the feature request, check also what requirements are currently not implemented
• Discuss on the underlying json library: Further work with tinyjson or switch to another library
• Definition of C++ API
• Development of a C++ wrapper on top of the Rust implementation
• Documentation, Architecture, … work

additional action to be refined @guysagnes :

• Epic: Functional gap analysis for feature "Persistency"
  • Security - impact on the API definition, alignment with possible unique context identifier in system
    separation of the data inside file system to be taken in account - Freedom from Interference on Flash.
  • Privacy - impact on encryption algorithm expected in system
    encryption to be taken in account on RAM / cached areas
  • Safety - System safety requirements expectation to be derived down to modules.
    Definition of the safety level file system, file, key entry
    Expected impact: Freedom from Interference (FFI), redundancy, data inversion, hash
    Additional impact: safety escalation in the system and expected recovery based on snapshot.

[joshualicht]

API - Findings to be addressed:

KVS-Version: 0.1.0 (ID: ec6982747eb1f48d1495f0fda4695f3b0e0bfdb0)

• Function flush_on_exit(self, flush_on_exit bool):
  Current Implementation:
  Ownership transfer of KVS to the function
  Expected Behavior:
  Borrowing of KVS without ownership transfer
  -> this approach is probably solved by builder approach - Fix requested @sbachmann-qorix

• Function snapshot_count(&self):
  Current Implementation:
  Provides number of snapshots available
  Suggestion for improvement:
  Additionally provide timestamp of each snapshot (e.g. extension of API here with new functions)
  -> proposed to get information about separated API - agreed @sbachmann-qorix

• Function key_exists(&self, key: &str) :
  Suggestion for improvement:
  Provide a more detailed description what "key_exists" means
  (e.g. key_exists reports the existence of a current value of a key)
  -> adaptation agreed @sbachmann-qorix

• Function is_value_default(&self, key: &str):
  Current Implementation:
  Function returns false if a current key is written and true if only a default value is available
  Suggestion for improvement:
  Refactor the function to "has_default_value" and return a bool type
  (true when a default value is available and false if not)
  -> adaptation agreed @sbachmann-qorix

• Data Separation between Applications:
  Current Implementation:
  Data separation is only indirectly managed by the file system
  Suggestion for improvement:
  Provide additional layers of separation (e.g. with an application unique identifier and separated read/write access)
  -> To be addressed in Architecture Community - @guysagnes / @vinodreddy-g Elsaka Ahmed
  Separation related to the process ID, an application will be addressed to a single folder.
  The folder may get the name of application or uuid?

• Default Values – Data Integrity:
  Current Implementation:
  Default values are not secured with Hash Files
  Expected Behavior:
  Data integrity check should not be limited to written keys and also apply to default values
  -> adaptation agreed @sbachmann-qorix @vinodreddy-g

• Reset to Default Values:
  Current Implementation:
  You have to remove the key, check if a default value is available and write it again with the retrieved default value
  Suggestion for improvement:
  Implementation of an API function to reset current values to default values
  -> agreed @guysagnes

• Deployment of Default Values:
  Current Implementation:
  Not implemented
  Suggestion for improvement:
  Implementation of a functionality to deploy (new) default values for an application (e.g. in an update process)
  -> proposition agreed, can be deployed in context of the configuration @vinodreddy-g
  to be taken in account that a unique way to prepare the data should be managed e.g. for encrypted data

• Conversion of SnapshotId in Application Context:
  Current Implementation:
  To use SnapshotIds a conversion needs to take place inside the application: SnapshotId::new(id as usize)
  Expected Behavior:
  Function should handle Snapshots directly without a specific conversion to SnapshotId
  -> possibility to adapt to autotrade @sbachmann-qorix
  Use case to be investigated for dedicated roll back use case @guysagnes / @sbachmann-qorix

• Conversion kvsvalue to type (get_value):
  Current Implementation:
  Panic inside Trait Implementation when trying to get a key with a wrong datatype (infallible-conversion).
  Suggestion for improvement:
  Return Errorcode instead of panic, so the application stays up (fallible conversions - e.g. with TryFrom)
  -> use tryFrom agreed @sbachmann-qorix

• Function get_value and get_default_value different Syntax:
  Current Implementation:
  get_value needs specific datatype and returns this type. get_default_value always returns a KvsValue. Handling of standard/primitive types and strings can be done without KvsValue-Handling, while more Complex datatypes (Array, Object) always need KvsValue handling from the application. When retrieving the default value, the application always needs to handle the KvsValue unless it writes it directly to the KVS
  Expected Behavior:
  The process of data retrieving should be consistent regardless of the operation itself (get_default_value <-> get_value)
  Suggestion for improvement:
  The functions get_value and get_default_value should always return a KvsValue Type and leave the responsibility of interpretation to the application itself
  -> use agreed @sbachmann-qorix

[guysagnes]

Organization

Overview about the EPIC and Workpacakges related to Persistency
https://github.com/eclipse-score/score/issues?q=is%3Aissue%20state%3Aopen%20label%3Acft%3Apersistency

Related Persistency CFT Board:
Persistency FT

• Requested merge of the features and requirements requested -> @guysagnes

Feedback from Workshop

https://github.com/orgs/eclipse-score/discussions/764

https://eclipse-score.github.io/score/main/_images/architecture.drawio.svg

• Json Feature to be checked for completness.
• Communication will be based additional to the zero-copy with gateway (SomeIP)
• Impact on the deviation to the Adaptative Autosar may be managed over the communication gateway

Template

• confirmation that the operation will change the current repository

Presentation of C++ API

• the data type will be managed inside baselibs changed to result
• check for use of fix types in context of the API

JSON Communication library - BMW Proposition API to be checked
The initial version will be provided for additional review to persistency CFT -> @vbogdanb

KVS Tool

• Presentation of KVS Tool (Conti)
  • Initial Implementation of the KVS_Tool inc_mw_per#5

KVS Tool
Usage
Options:
-h, --help Show this help message and exit
-o, --operation Specify the operation to perform
(setkey, getkey, removekey, listkeys, reset, snapshotcount, snapshotmaxcount,
snapshotrestore, getkvsfilename, gethashfilename, createtestdata)
-k, --key Specify the key to operate on (for key operations)
-p, --payload Specify the value to write (for set operations)
-t, --type Specify the value type for get operations
(number, bool, string, null, array, object or first letter as a short form: n = number (except NULL))
-s, --snapshotid Specify the snapshot ID for Snapshot operations

The tool can be extended and used for analysis and production phase.