[Logs UX] Upgrade to 9.0 testing plan

[achyutjhunjhunwala]

Summary

Refer to the https://github.com/elastic/observability-dev/issues/4176 for full instructions.

Note

If there is only 9.0.0-SNAPSHOT version available when the test is executed, the entire flow needs to be done using -SNAPSHOT releases

Testing plan

Please make sure to navigate to all existing pages, tabs and Flyouts on each version, including the

Pages to test

• Dataset Quality Page (This would only be available in 8.x and 9.x)
• Logs Explorer appears until 8.x and then goes way in 9.x
• Logs settings page (Should we test any specific setting here)
• Shared Log Stream Component (Either from existing Integration with a Dashboard or a Custom with Log Stream Component)
• is there any other place to test

How to test

• Create 7.17 cluster (see testing utilities doc)
• Load logs data (Need to figure out what's best to use for this. 1st instinct is to go with Synthtrace)
• Navigate through above listed pages
• Upgrade to 8.lastest following the Kibana Upgrade Assistant
• Load more data (this can be done using synthtrace)
• Navigate through above listed pages
• Upgrade to 9.0.0 following the Kibana Upgrade Assistant
• Navigate through above listed pages

Test Results

https://docs.google.com/document/d/1efjsLctu7TTyyzkmiggyAhVBHRX4mdNAcEVuVkxTadQ/edit?usp=sharing

[elasticmachine]

Pinging @elastic/obs-ux-logs-team (Team:obs-ux-logs)

[achyutjhunjhunwala]

Steps Executed

1. Created a 7.17.28-SNAPSHOT cluster on QA

[gbamparop]

Thanks for creating the test plan! Adding some thoughts below:

Shared Log Stream Component (Either from existing Integration with a Dashboard or a Custom with Log Stream Component)

• Checking links to Discover to ensure that the right ad-hoc data views are created, including cases where the logs settings have changed
• Upgrade the integrations that installed dashboards that were using the log stream
• Following the migration assistant to migrate the old log sources to the new Kibana advanced setting

Other use cases:

• Log Stream: Feature flag and redirection to Logs Explorer
• Logs Explorer: Redirection to Discover
• Even though we haven't made any changes, it'll be good to check the ML pages (Logs Anomalies and Logs Categories) by creating machine learning jobs and upgrading to 8.x / 9.x

[achyutjhunjhunwala]

A detailed test plan document is created here to follow the steps.

https://docs.google.com/document/d/1efjsLctu7TTyyzkmiggyAhVBHRX4mdNAcEVuVkxTadQ/edit?usp=sharing

The document is open for review.

[achyutjhunjhunwala]

Round 1 Testing

Unfortunately instance got deleted before i could proceed with final 8.x to 9.x upgrade

Round 2 Testing

Version (initial)	Version (after upgrade)	Results	Notes
7.17.28 Snapshot	-	Installed APM Manually using Fleet, Nginx, Checkpoint and Cisco integration. Ingested Synthtrace data. Created ML Jobs for Anomalies and Categorisation	Synthtrace scenarios used - degraded_logs.ts, slash_logs.ts
7.17.28 Snapshot	8.18.0 Snapshot	Upgraded successfully ✅	-
8.18.0 Snapshot	-	Ingested more synthtrace data	Synthtrace scenarios used - degraded_logs.ts, slash_logs.ts, failed_logs.ts, trace_logs_entities.ts
8.18.0 Snapshot	9.0.0 Snapshot	Upgraded Failed 🔴	See Cloud UI Errors Below

Round 3 Testing

Version (initial)	Version (after upgrade)	Results	Notes
7.17.28 Snapshot	-	Installed APM Manually using Fleet, Nginx, Checkpoint and Cisco integration. Ingested Synthtrace data. Created ML Jobs for Anomalies and Categorisation	Synthtrace scenarios used - degraded_logs.ts, slash_logs.ts
7.17.28 Snapshot	8.18.0 Snapshot	Upgraded successfully ✅	-
8.18.0 Snapshot	-	Ingested more synthtrace data	Synthtrace scenarios used - degraded_logs.ts, slash_logs.ts, failed_logs.ts, trace_logs_entities.ts
8.18.0 Snapshot	9.0.0 Snapshot	Upgraded Failed 🔴	Upgrade Assistant Broken. See screenshot below

Round 4 Testing

Version (initial)	Version (after upgrade)	Results	Notes
7.17.28 Snapshot	-	Installed APM Manually using Fleet, Nginx, Checkpoint and Cisco integration. Ingested Synthtrace data. Created ML Jobs for Anomalies and Categorisation	Synthtrace scenarios used - degraded_logs.ts, slash_logs.ts
7.17.28 Snapshot	8.18.0 Snapshot	Upgraded successfully ✅	-
8.18.0 Snapshot	9.0.0 Snapshot	Upgraded Failed 🔴	Upgrade Assistant Broken. See screenshot below

Round 6 Testing

Version (initial)	Version (after upgrade)	Results	Notes
7.17.28 Snapshot	-	Installed APM Manually using Fleet, Nginx Integration, Ingested Synthtrace data. Created ML Jobs for Anomalies and Categorisation	Synthtrace scenarios used - degraded_logs.ts, slash_logs.ts
7.17.28 Snapshot	8.18.0 Snapshot	Upgraded successfully ✅	-
8.18.0 Snapshot	9.1.0 Snapshot	Upgraded Failed 🔴	Cloud UI Failed to Upgrade - 8.18 to 9.1 Upgrade path not supported

Round 7 Testing

Version (initial)	Version (after upgrade)	Results	Notes
7.17.28 Snapshot	-	Installed APM Manually using Fleet, Nginx Integration, Ingested Synthtrace data. Created ML Jobs for Anomalies and Categorisation	Synthtrace scenarios used - degraded_logs.ts, slash_logs.ts
7.17.28 Snapshot	8.18.0 Snapshot	Upgraded successfully ✅	-
8.18.0 Snapshot	9.0 Snapshot	Upgraded successfully ✅	-

[gmarouli]

Thank you for reporting the failures, we are fixing the cause of this specific error in elastic/elasticsearch#121263.

[achyutjhunjhunwala]

List of bugs/blockers found during testing

Bugs Found	Status	Blocker
Upgrade to 9.0 Broken Due to null template	✅ Resolved	Yes
Dataset Quality page shows infinite loading post 7.x to 8.18 migration	✅ Resolved	Yes
Upgrade Assistant - Documentation Link for fixing multiple re-indexes broken	Will be fixed once 8.18 docs are released	No
Upgrade Assistant - Still shows some data streams to be manually reindexed and some automatically	✅ Resolved	Sort of
Upgrade Assistant - ML Snapshot upgrades does not work. Only delete works	✅ Resolved	Depends
Logs Explorer still appears on 9.x Left hand navigation	✅ Resolved	No
Index Management does not load on 9.0	✅ Resolved	No
Logs Setting Error Boundary	🚫 UNRESOLVED	No

[achyutjhunjhunwala]

@gbamparop Is it fine to mark the testing as completed ? I was able to follow the Upgrade Assistant and follow the upgrade all the way from 7.17.28-SNAPSHOT -> 8.18-SNAPSHOT -> 9.0-SNAPSHOT.

Bugs have already been reported and respective teams are working.

[gbamparop]

@gbamparop Is it fine to mark the testing as completed ? I was able to follow the Upgrade Assistant and follow the upgrade all the way from 7.17.28-SNAPSHOT -> 8.18-SNAPSHOT -> 9.0-SNAPSHOT.

Bugs have already been reported and respective teams are working.

@achyutjhunjhunwala do you think it's worth testing with the latest 9.0 beta BC?

[achyutjhunjhunwala]

@achyutjhunjhunwala do you think it's worth testing with the latest 9.0 beta BC?

@gbamparop I just saw on Slack ping from Paul as well around this that 9.0 Beta is out for testing.

Given the number of bugs reported open. I would say we wait until those are fixed before we do another testing as i am pretty sure beta will also have those issues.

We can keep this ticket open for some time and check again next week once all the bugs atleast related to Logs UX are resolved

[gbamparop]

We can keep this ticket open for some time and check again next week once all the bugs atleast related to Logs UX are resolved

Sounds good to me, thanks.

[achyutjhunjhunwala]

Linking to the last dependent ticket we have - https://github.com/elastic/kibana-team/issues/1510