- 
                Notifications
    
You must be signed in to change notification settings  - Fork 8.5k
 
[Dataset Quality] Indicate if failure store isn't enabled for data stream #221644
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Dataset Quality] Indicate if failure store isn't enabled for data stream #221644
Conversation
…tore-isnt-enabled-for-data-stram
| 
           /oblt-deploy  | 
    
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for taking this one @rStelmach.
I have two main comments here:
- I think we should change the approach here, from your changes I can infer that we are treating privileges (meaning user can not read failure store: 
read_failure_store) the same as a failure store not enabled for an specific dataStream. Take the following as an example (You can find everything configured in the PR deployment): 
I've set up a custom component template enabling failure store for data streams following the pattern logs-foo-*
script
PUT _component_template/logs-foo@custom
{
  "template": {
    "data_stream_options": {
      "failure_store": {
        "enabled": true
      }
    }
  }
}
Then I ingested some documents in logs-foo-1 (Failure store enabled) and logs-bar-1 (Failure store disabled)
script
### Failure store enabled
POST logs-foo-1/_doc/1?op_type=create
{
  "foo": "bar"
}
POST logs-foo-1/_doc/2?op_type=create
{
  "foo": { "bar": 123}
}
#### Failure Store not enabled
POST logs-bar-1/_doc/1?op_type=create
{
  "foo": "bar"
}
POST logs-bar-1/_doc/2?op_type=create
{
  "foo": { "bar": 123}
}
User has proper privileges to read failure store over all datastreams following logs-* pattern
And this is what I see in dataset quality page

This is what I should be seeing

but I can only get to that state if the user doesn't have read privileges over failure store.
- We should also consider what to do inside a specific dataStream in regards to failure Store, should we show something there (?) or instead we could use a banner at the top with a link to the specific documentation? (Something similar to what we do when user doesn't have privileges but now for enabling the failure store for the specific datastream)
 
@LucaWintergerst, @patpscal what are your thoughts on this one?
Note
I configured everything in the PR deployment, you can access with elastic, failure_store_some_enabled and failure_store_some_restricted with the same password generated by oblt-deploy command (🔒 deployment)
…s with/without failure store enabled
…tore-isnt-enabled-for-data-stram
…tore-isnt-enabled-for-data-stram
…tore-isnt-enabled-for-data-stram
…tore-isnt-enabled-for-data-stram
        
          
                x-pack/platform/plugins/shared/dataset_quality/public/utils/generate_datasets.ts
              
                Outdated
          
            Show resolved
            Hide resolved
        
              
          
                x-pack/platform/plugins/shared/dataset_quality/public/utils/generate_datasets.test.ts
          
            Show resolved
            Hide resolved
        
              
          
                ...platform/plugins/shared/dataset_quality/server/routes/data_streams/get_data_streams/index.ts
              
                Outdated
          
            Show resolved
            Hide resolved
        
      | 
           We are missing some api tests (integration) asserting that we are getting the right information regarding failureStore being enabled/disabled  | 
    
| 
           @yngrdyn apologies for the super slow reply here - this is good, thank you work implementing it like this  | 
    
          
 I think we could treat that either way!  | 
    
…lureStore`. remove unnecessary nullish coalescing operators
…tore-isnt-enabled-for-data-stram
…tore-isnt-enabled-for-data-stram
| ? i18n.translate('xpack.datasetQuality.failureStore.enable', { | ||
| defaultMessage: 'Set failure store', | ||
| }) | ||
| : 'N/A'} | 
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This should also be translated
| 
           /oblt-deploy  | 
    
| css={{ textDecoration: 'underline' }} | ||
| > | ||
| {i18n.translate('xpack.datasetQuality.enableFailureStore', { | ||
| defaultMessage: 'Enable failure store', | 
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Adding an icon would be beneficial here @patpscal?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
…tore-isnt-enabled-for-data-stram
          💚 Build Succeeded
 Metrics [docs]Async chunks
 
 Page load bundle
 
 History
  | 
    
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM, thanks for taking this one
Good job 🚀
| 
           Starting backport for target branches: 8.19  | 
    
…ream (elastic#221644) Added a tooltip and a link with documentation for Failed docs column when dataset does not have failure store enabled. https://github.com/user-attachments/assets/be65db9a-15c8-4087-b175-752b2fabab6e For now it awaits for the documentation PR to be merged : elastic/docs-content#1368 --------- Co-authored-by: kibanamachine <[email protected]> Co-authored-by: Yngrid Coello <[email protected]> (cherry picked from commit 36cbbb9)
          💚 All backports created successfully
 Note: Successful backport PRs will be merged automatically after passing CI. Questions ?Please refer to the Backport tool documentation  | 
    
…data stream (#221644) (#225007) # Backport This will backport the following commits from `main` to `8.19`: - [[Dataset Quality] Indicate if failure store isn't enabled for data stream (#221644)](#221644) <!--- Backport version: 9.6.6 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sorenlouv/backport) <!--BACKPORT [{"author":{"name":"Robert Stelmach","email":"[email protected]"},"sourceCommit":{"committedDate":"2025-06-24T07:49:36Z","message":"[Dataset Quality] Indicate if failure store isn't enabled for data stream (#221644)\n\nAdded a tooltip and a link with documentation for Failed docs column\nwhen dataset does not have failure store enabled.\n\n\n\nhttps://github.com/user-attachments/assets/be65db9a-15c8-4087-b175-752b2fabab6e\n\n\n\n\nFor now it awaits for the documentation PR to be merged :\nhttps://github.com/elastic/docs-content/pull/1368\n\n---------\n\nCo-authored-by: kibanamachine <[email protected]>\nCo-authored-by: Yngrid Coello <[email protected]>","sha":"36cbbb9bf8f141a54bb6c455e4479e93990d9626","branchLabelMapping":{"^v9.1.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:feature","Feature:Dataset Health","backport:version","v9.1.0","v8.19.0"],"title":"[Dataset Quality] Indicate if failure store isn't enabled for data stream","number":221644,"url":"https://github.com/elastic/kibana/pull/221644","mergeCommit":{"message":"[Dataset Quality] Indicate if failure store isn't enabled for data stream (#221644)\n\nAdded a tooltip and a link with documentation for Failed docs column\nwhen dataset does not have failure store enabled.\n\n\n\nhttps://github.com/user-attachments/assets/be65db9a-15c8-4087-b175-752b2fabab6e\n\n\n\n\nFor now it awaits for the documentation PR to be merged :\nhttps://github.com/elastic/docs-content/pull/1368\n\n---------\n\nCo-authored-by: kibanamachine <[email protected]>\nCo-authored-by: Yngrid Coello <[email protected]>","sha":"36cbbb9bf8f141a54bb6c455e4479e93990d9626"}},"sourceBranch":"main","suggestedTargetBranches":["8.19"],"targetPullRequestStates":[{"branch":"main","label":"v9.1.0","branchLabelMappingKey":"^v9.1.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/221644","number":221644,"mergeCommit":{"message":"[Dataset Quality] Indicate if failure store isn't enabled for data stream (#221644)\n\nAdded a tooltip and a link with documentation for Failed docs column\nwhen dataset does not have failure store enabled.\n\n\n\nhttps://github.com/user-attachments/assets/be65db9a-15c8-4087-b175-752b2fabab6e\n\n\n\n\nFor now it awaits for the documentation PR to be merged :\nhttps://github.com/elastic/docs-content/pull/1368\n\n---------\n\nCo-authored-by: kibanamachine <[email protected]>\nCo-authored-by: Yngrid Coello <[email protected]>","sha":"36cbbb9bf8f141a54bb6c455e4479e93990d9626"}},{"branch":"8.19","label":"v8.19.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"}]}] BACKPORT--> Co-authored-by: Robert Stelmach <[email protected]> Co-authored-by: Yngrid Coello <[email protected]>
…ream (elastic#221644) Added a tooltip and a link with documentation for Failed docs column when dataset does not have failure store enabled. https://github.com/user-attachments/assets/be65db9a-15c8-4087-b175-752b2fabab6e For now it awaits for the documentation PR to be merged : elastic/docs-content#1368 --------- Co-authored-by: kibanamachine <[email protected]> Co-authored-by: Yngrid Coello <[email protected]>


Added a tooltip and a link with documentation for Failed docs column when dataset does not have failure store enabled.
failed_docs_tooltip.mov
For now it awaits for the documentation PR to be merged : elastic/docs-content#1368