LogsDB data migration integration testing #112710
Conversation
Here we test reindexing logsdb indices, creating and restoring snapshots. Note that logsdb uses synthetic source and restoring source only snapshots fails due to missing _source.
salvatore-campagna
added
>test
|
Pinging @elastic/es-storage-engine (Team:StorageEngine) |
| assertOK(client.performRequest(reindexRequest)); | ||
| assertDataStreamBackingIndexMode("standard", 0, "standard-apache-kafka"); | ||
| assertThat( | ||
| entityAsMap(client.performRequest(new Request("POST", "/standard-apache-kafka/_count"))).get("count"), |
There was a problem hiding this comment.
Can we check that documents look correct?
There was a problem hiding this comment.
What do you exactly mean when saying "look correct"? I think it is not up to thins kind of test including that they match.
There was a problem hiding this comment.
I agree that this is not an ideal place to test reindex operation itself but AFAIK there are currently no tests for this. Do we have an item for that? I was thinking that such a simple document should be identical even with synthetic _source and therefore we could get at least an initial test for that behavior.
| putTemplate(client, "standard-template", STANDARD_TEMPLATE); | ||
| createDataStream(client, "standard-apache-kafka"); | ||
|
|
||
| // Index some documents in LogsDB |
There was a problem hiding this comment.
The name of the test is testStandardToLogsDBReindex but the test is doing the opposite - it reindexes form logsdb to standard.
| putTemplate(client, "standard-template", STANDARD_TEMPLATE); | ||
| createDataStream(client, "standard-apache-kafka"); | ||
|
|
||
| // Index some documents in LogsDB |
| Matchers.equalTo(10) | ||
| ); | ||
|
|
||
| // Reindex LogsDB data stream into a standard data stream |
| mountRequest.setJsonEntity("{\"index\": \"" + index + "\",\"renamed_index\": \"" + restoreIndex + "\"}"); | ||
| assertOK(client.performRequest(mountRequest)); | ||
|
|
||
| assertThat( |
There was a problem hiding this comment.
Same - can we check documents?
|
@lkts this was not ready for review yet...I pushed to see CI results. |
|
@salvatore-campagna lol sorry about that. Can you mark it as draft then? |
I usually wait at least for a round of CI to be green or, at least to see if there is anything wrong and then I ask for a review selecting reviewers. I do this to avoid wasting people time reviewing something that is not ready yet. I rarely use draft PRs. |
salvatore-campagna
requested review from
kkrik-es,
lkts and
martijnvg
and removed request for
kkrik-es
Here we test reindexing logsdb indices, creating and restoring snapshots. Note that logsdb uses synthetic source and restoring source only snapshots fails due to missing _source.
💚 All backports created successfully
Questions ?Please refer to the Backport tool documentation |
Here we test reindexing logsdb indices, creating and restoring snapshots. Note that logsdb uses synthetic source and restoring source only snapshots fails due to missing _source. (cherry picked from commit f7880ae)
5 participants
Here we test reindexing logsdb indices, creating and restoring snapshots. Note that logsdb uses synthetic source and restoring source-only snapshots fails due to missing _source.
Resolves #109803