Skip to content

Commit 5fb620e

Browse files
lobdelledownsrobannie3431dbbaughebowenlan-amzn
authored
Merge main to 1.x (#121)
* Fixes editing rollup delay (#82) * Fixes editing the delay amount and displaying the schedule for rollups Signed-off-by: Clay Downs <[email protected]> * Bumps version to 1.2 (#103) * Bumps version to 1.2 * Changes test workflows to follow Dashboards 1.x Signed-off-by: Clay Downs <[email protected]> * Fixes flakey tests (#105) * Fixes flakey cypress test 'rollup enable and disable test' * Fixes flakey unit test 'can delete a rollup job' Signed-off-by: Robert Downs <[email protected]> * Fixed Transforms geo_point bug and boolean render (#93) * Fixed Transforms geo_point bug and boolean render * Set default DefineTransforms table render behavior to JSON.stringify * Updated DefineTransforms null cell value check Signed-off-by: Eric Lobdell <[email protected]> * Transform bug fixes (#109) * Rename plugin helper file * Update TransformIndices.tsx * Backport commits from main (#90) * Release 1.1.0.0 (#83) * Release 1.1.0.0 Signed-off-by: bowenlan-amzn <[email protected]> * Correct copyright notices * UI fixes for new ISM UI (#84) * Removes X icon from action/transition flyout footer next to cancel Signed-off-by: Drew Baugher <[email protected]> * Adds tooltips to edit/delete icon buttons on the draggable action/transition components Signed-off-by: Drew Baugher <[email protected]> * Removes overlay for flyout so clicking outside doesn't close flyout and removes X close button on top right Signed-off-by: Drew Baugher <[email protected]> * Adds edit/delete tooltips for state component and removes underline from hovering states Signed-off-by: Drew Baugher <[email protected]> * Changes state flyout from X Close to Cancel Signed-off-by: Drew Baugher <[email protected]> * Adds JSON editor for allocation action Signed-off-by: Drew Baugher <[email protected]> * Adds isValid to action interface and implements in actions Signed-off-by: Drew Baugher <[email protected]> * Correctly removes unused rollover keys from rollover object, handles uncontrolled inputs and NaN minDocs Signed-off-by: Drew Baugher <[email protected]> * Adds error for state name already existing in policy Signed-off-by: Drew Baugher <[email protected]> * Fixes duplicated actions when editing as previously we had a new ID and matching logic didn't find an action Signed-off-by: Drew Baugher <[email protected]> * Fixes other action inputs that can return NaN and creates a no conditions option that is the default Signed-off-by: Drew Baugher <[email protected]> * Updates snapshot Signed-off-by: Drew Baugher <[email protected]> * Correctly show danger toast on update/create failures Signed-off-by: Drew Baugher <[email protected]> * Fixes rollup action nesting multiple ism_template keys Signed-off-by: Drew Baugher <[email protected]> * Fixes timeout/retry settings turning into uncontrolled inputs and NaN value Signed-off-by: Drew Baugher <[email protected]> * Removes text transformation on inputs and updates isValid methods and passes to form labels Signed-off-by: Drew Baugher <[email protected]> * Fixes allocation and rollup adding UIAction properties to policy JSON on updates Signed-off-by: Drew Baugher <[email protected]> * Updates release notes w/ new PR changes Signed-off-by: Drew Baugher <[email protected]> * Wraps actions/transitions in states component and updates default rollup to include ism_rollup key Signed-off-by: Drew Baugher <[email protected]> * Fixes some small UI touchups/issues for new ISM UI (#85) * Moves cancel secondary button next to primary on action/transition flyout Signed-off-by: Drew Baugher <[email protected]> * Removes default ISM template and increases width of empty prompt Signed-off-by: Drew Baugher <[email protected]> * Adds punctuation Signed-off-by: Drew Baugher <[email protected]> * Adds back X icon next to cancel for state flyout Signed-off-by: Drew Baugher <[email protected]> * Makes inputs in flyout full width, updates help text, and adds some spacing Signed-off-by: Drew Baugher <[email protected]> * Updates transition default to not include conditions empty object, and on change transition to delete conditions when selecting none Signed-off-by: Drew Baugher <[email protected]> * Moves edit button out of policy settings content panel and on to global page for view policy Signed-off-by: Drew Baugher <[email protected]> * Updates release note Signed-off-by: Drew Baugher <[email protected]> * Fixes broken link Signed-off-by: Drew Baugher <[email protected]> * Fixes small issues on new ISM UI (#88) * Fixes small issues on new ISM UI Signed-off-by: Drew Baugher <[email protected]> * Updates snapshots Signed-off-by: Drew Baugher <[email protected]> * Updates release notes Signed-off-by: Drew Baugher <[email protected]> * Updates workflows to trigger on 1.* branches Signed-off-by: Drew Baugher <[email protected]> Co-authored-by: Bowen Lan <[email protected]> * Support data filter when viewing sample data * Update TransformOptions.tsx * Add TransformService.test.ts * Draft of date histogram panel * Add helper method and move import * Split calendar and fixed interval to 2 panels * Add rendering test * Add close popover test * Update cypress-workflow.yml * Update links.yml * Update unit-tests-workflow.yml * Update TransformOptions.tsx * Update tests * Update DateHistogramPanel.test.tsx * Make scripted metrics editor larger * Expand code editor related panels * Undo changes to workflow Signed-off-by: Annie Lee <[email protected]> * Update TimeAggregation.tsx * Update DefineTransforms.tsx undo changes to style * Clean up code * Update snapshot Signed-off-by: Annie Lee <[email protected]> * Refactor the order of checking if name is defined before checking for duplicate name * Update CreateTransformForm.tsx * Update DefineTransforms.tsx Signed-off-by: Annie Lee <[email protected]> * Update DefineTransforms.tsx Signed-off-by: Annie Lee <[email protected]> * Update DefineTransforms.tsx * Update DefineTransforms.tsx * Update DefineTransforms.tsx Co-authored-by: Drew Baugher <[email protected]> Co-authored-by: Bowen Lan <[email protected]> * Fix the bugs in visual UI for legacy notification (#111) * Transforms unit and cypress tests (#92) * Transform cypress tests (#21) * Added Cypress tests for Transforms UI * License update * Unit tests initial commit * Added jest tests for Transforms, details, edit and create * Sample transform fix * Fixed cypress test transform and sample data load order Signed-off-by: Eric Lobdell <[email protected]> * Test fix debug * Simplified unit test * Updated transforms cypress test comments and success criteria Signed-off-by: Eric Lobdell <[email protected]> * Fixed quotation mark issue in test Signed-off-by: Eric Lobdell <[email protected]> * Removed debug mapping and index objects from unit test Signed-off-by: Eric Lobdell <[email protected]> * Cypress test improvements and completed edit transform unit test Signed-off-by: Eric Lobdell <[email protected]> * Integ test delete fix and EditTransforms jest test fix Signed-off-by: Eric Lobdell <[email protected]> * Restoring enable disable page refresh delay to fix failing integ test Signed-off-by: Eric Lobdell <[email protected]> * Resolve test constants conflict Signed-off-by: Eric Lobdell <[email protected]> * Fix pagination of managed indices page (#113) * Disables jitter for managed_indices_spec Cypress tests (#122) Signed-off-by: Robert Downs <[email protected]> Co-authored-by: Clay Downs <[email protected]> Co-authored-by: Annie Lee <[email protected]> Co-authored-by: Drew Baugher <[email protected]> Co-authored-by: Bowen Lan <[email protected]> Co-authored-by: Ravi <[email protected]>
1 parent 10ba1af commit 5fb620e

File tree

52 files changed

+4364
-297
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

52 files changed

+4364
-297
lines changed

.github/workflows/cypress-workflow.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,7 @@ on:
1010
- 1.*
1111
env:
1212
OPENSEARCH_DASHBOARDS_VERSION: '1.x'
13-
OPENSEARCH_VERSION: '1.1.0-SNAPSHOT'
13+
OPENSEARCH_VERSION: '1.2.0-SNAPSHOT'
1414
jobs:
1515
tests:
1616
name: Run Cypress E2E tests

.github/workflows/unit-tests-workflow.yml

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -7,8 +7,9 @@ on:
77
push:
88
branches:
99
- main
10-
- 1.*
11-
10+
- development-*
11+
env:
12+
OPENSEARCH_DASHBOARDS_VERSION: '1.x'
1213
jobs:
1314
tests:
1415
name: Run unit tests
@@ -22,7 +23,7 @@ jobs:
2223
uses: actions/checkout@v2
2324
with:
2425
repository: opensearch-project/OpenSearch-Dashboards
25-
ref: '1.0'
26+
ref: ${{ env.OPENSEARCH_DASHBOARDS_VERSION }}
2627
path: OpenSearch-Dashboards
2728
- name: Get node and yarn versions
2829
id: versions
Lines changed: 40 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,40 @@
1+
{
2+
"transform": {
3+
"enabled": true,
4+
"schedule": {
5+
"interval": {
6+
"period": 1,
7+
"unit": "Minutes",
8+
"start_time": 1602100553
9+
}
10+
},
11+
"description": "Test transform",
12+
"source_index": "opensearch_dashboards_sample_data_ecommerce",
13+
"target_index": "test_transform",
14+
"data_selection_query": {
15+
"match_all": {}
16+
},
17+
"page_size": 1000,
18+
"groups": [
19+
{
20+
"terms": {
21+
"source_field": "customer_gender",
22+
"target_field": "gender"
23+
}
24+
},
25+
{
26+
"terms": {
27+
"source_field": "day_of_week",
28+
"target_field": "day"
29+
}
30+
}
31+
],
32+
"aggregations": {
33+
"quantity": {
34+
"sum": {
35+
"field": "total_quantity"
36+
}
37+
}
38+
}
39+
}
40+
}

cypress/integration/managed_indices_spec.js

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -36,6 +36,8 @@ describe("Managed indices", () => {
3636
beforeEach(() => {
3737
// Set welcome screen tracking to false
3838
localStorage.setItem("home:welcome:show", "false");
39+
// Disable jitter so ISM jobs run without an additional delay
40+
cy.disableJitter();
3941

4042
cy.wait(3000);
4143

cypress/integration/rollups_spec.js

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -236,6 +236,9 @@ describe("Rollups", () => {
236236

237237
cy.contains(`${ROLLUP_ID}`);
238238

239+
// Disable button is enabled
240+
cy.get(`[data-test-subj="disableButton"]`).should("not.be.disabled");
241+
239242
// Click Disable button
240243
cy.get(`[data-test-subj="disableButton"]`).trigger("click", { force: true });
241244

Lines changed: 257 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,257 @@
1+
/*
2+
* SPDX-License-Identifier: Apache-2.0
3+
*
4+
* The OpenSearch Contributors require contributions made to
5+
* this file be licensed under the Apache-2.0 license or a
6+
* compatible open source license.
7+
*
8+
* Modifications Copyright OpenSearch Contributors. See
9+
* GitHub history for details.
10+
*/
11+
12+
import { PLUGIN_NAME } from "../support/constants";
13+
import sampleTransform from "../fixtures/sample_transform";
14+
15+
const TRANSFORM_ID = "test_transform_id";
16+
17+
describe("Transforms", () => {
18+
before(() => {
19+
// Delete all indices
20+
cy.deleteAllIndices();
21+
22+
// Load ecommerce data
23+
cy.request({
24+
method: 'POST',
25+
url:`${Cypress.env("opensearch_dashboards")}/api/sample_data/ecommerce`,
26+
headers: {
27+
'osd-xsrf': true
28+
}
29+
}).then((response) => {
30+
expect(response.status).equal(200);
31+
});
32+
});
33+
34+
beforeEach(() => {
35+
// delete test transform and index
36+
cy.request("DELETE", `${Cypress.env("opensearch")}/test_transform*`);
37+
cy.request({
38+
method: 'POST',
39+
url: `${Cypress.env("opensearch")}/_plugins/_transform/${TRANSFORM_ID}/_stop`,
40+
failOnStatusCode: false
41+
});
42+
cy.request({
43+
method: 'DELETE',
44+
url: `${Cypress.env("opensearch")}/_plugins/_transform/${TRANSFORM_ID} `,
45+
failOnStatusCode: false
46+
});
47+
48+
// Set welcome screen tracking to test_transform_target
49+
localStorage.setItem("home:welcome:show", true);
50+
51+
// Visit ISM Transforms Dashboard
52+
cy.visit(`${Cypress.env("opensearch_dashboards")}/app/${PLUGIN_NAME}#/transforms`);
53+
54+
// Common text to wait for to confirm page loaded, give up to 60 seconds for initial load
55+
cy.contains("Create transform", { timeout: 60000 });
56+
});
57+
58+
describe("can be created", () => {
59+
it("successfully", () => {
60+
// Confirm we loaded empty state
61+
cy.contains(
62+
"Transform jobs help you create a materialized view on top of existing data."
63+
);
64+
65+
// Route to create transform page
66+
cy.contains("Create transform").click({ force: true });
67+
68+
// Type in transform ID
69+
cy.get(`input[placeholder="my-transformjob1"]`).type(TRANSFORM_ID, { force: true });
70+
71+
// Get description input box
72+
cy.get(`textarea[data-test-subj="description"]`).focus().type("some description");
73+
74+
// Enter source index
75+
cy.get(`div[data-test-subj="sourceIndexCombobox"]`)
76+
.find(`input[data-test-subj="comboBoxSearchInput"]`)
77+
.focus()
78+
.type("opensearch_dashboards_sample_data_ecommerce{enter}");
79+
80+
// Enter target index
81+
cy.get(`div[data-test-subj="targetIndexCombobox"]`)
82+
.find(`input[data-test-subj="comboBoxSearchInput"]`)
83+
.focus()
84+
.type("test_transform{enter}");
85+
86+
// Click the next button
87+
cy.get("button").contains("Next").click({ force: true });
88+
89+
// Confirm that we got to step 2 of creation page
90+
cy.contains("Select fields to transform");
91+
92+
cy.get(`button[data-test-subj="category.keywordOptionsPopover"]`)
93+
.click({ force: true });
94+
95+
cy.contains("Group by terms").click({ force: true });
96+
97+
// Confirm group was added
98+
cy.contains("category.keyword_terms");
99+
100+
// Add aggregable field
101+
cy.contains("50 columns hidden").click({ force: true });
102+
cy.contains("taxless_total_price").click({ force: true });
103+
// Click out of the window
104+
cy.contains("Select fields to transform").click({ force: true });
105+
106+
cy.get(`button[data-test-subj="taxless_total_priceOptionsPopover"]`)
107+
.click({ force: true });
108+
109+
cy.contains("Aggregate by avg").click({ force: true });
110+
111+
// Confirm agg was added
112+
cy.contains("avg_taxless_total_price");
113+
114+
// Click the next button
115+
cy.get("button").contains("Next").click({ force: true });
116+
117+
// Confirm that we got to step 3 of creation page
118+
cy.contains("Job enabled by default");
119+
120+
// Click the next button
121+
cy.get("button").contains("Next").click({ force: true });
122+
123+
// Confirm that we got to step 4 of creation page
124+
cy.contains("Review and create");
125+
126+
// Click the create button
127+
cy.get("button").contains("Create").click({ force: true });
128+
129+
// Verify that sample data is add by checking toast notification
130+
cy.contains(`Transform job "${TRANSFORM_ID}" successfully created.`);
131+
cy.location('hash').should('contain', 'transforms');
132+
cy.get(`button[data-test-subj="transformLink_${TRANSFORM_ID}"]`);
133+
});
134+
});
135+
136+
describe("can be edited", () => {
137+
beforeEach(() => {
138+
cy.createTransform(TRANSFORM_ID, sampleTransform);
139+
cy.reload();
140+
});
141+
142+
it("successfully", () => {
143+
// Confirm we have our initial transform
144+
cy.contains(TRANSFORM_ID);
145+
146+
// Select checkbox for our transform
147+
cy.get(`#_selection_column_${TRANSFORM_ID}-checkbox`)
148+
.check({ force: true });
149+
150+
// Click on Actions popover menu
151+
cy.get(`[data-test-subj="actionButton"]`).click({ force: true });
152+
153+
// Click Edit button
154+
cy.get(`[data-test-subj="editButton"]`).click({ force: true });
155+
156+
// Wait for initial transform job to load
157+
cy.contains("Test transform");
158+
159+
cy.get(`textArea[data-test-subj="description"]`).focus().clear().type("A new description");
160+
161+
// Click Save changes button
162+
cy.get(`[data-test-subj="editTransformSaveButton"]`).click({ force: true });
163+
164+
// Confirm we get toaster saying changes saved
165+
cy.contains(`Changes to transform saved`);
166+
167+
// Click into transform job details page
168+
cy.get(`[data-test-subj="transformLink_${TRANSFORM_ID}"]`).click({ force: true });
169+
170+
// Confirm new description shows in details page
171+
cy.contains("A new description");
172+
});
173+
});
174+
175+
describe("can be deleted", () => {
176+
beforeEach(() => {
177+
cy.createTransform(TRANSFORM_ID, sampleTransform);
178+
cy.reload();
179+
});
180+
181+
it("successfully", () => {
182+
// Confirm we have our initial transform
183+
cy.contains(TRANSFORM_ID);
184+
185+
// Disable transform
186+
cy.get(`#_selection_column_${TRANSFORM_ID}-checkbox`).check({ force: true });
187+
cy.get(`[data-test-subj="disableButton"]`).click({ force: true });
188+
cy.contains(`"${TRANSFORM_ID}" is disabled`);
189+
190+
// Select checkbox for our transform job
191+
cy.get(`#_selection_column_${TRANSFORM_ID}-checkbox`).check({ force: true });
192+
193+
// Click on Actions popover menu
194+
cy.get(`[data-test-subj="actionButton"]`).click({ force: true });
195+
196+
// Click Delete button
197+
cy.get(`[data-test-subj="deleteButton"]`).click({ force: true });
198+
199+
// Type "delete" to confirm deletion
200+
cy.get(`input[placeholder="delete"]`).type("delete", { force: true });
201+
202+
// Click the delete confirmation button in modal
203+
cy.get(`[data-test-subj="confirmModalConfirmButton"]`).click();
204+
205+
// Confirm we got deleted toaster
206+
cy.contains(`"${TRANSFORM_ID}" successfully deleted`);
207+
208+
// Confirm showing empty loading state
209+
cy.contains(
210+
"Transform jobs help you create a materialized view on top of existing data."
211+
);
212+
});
213+
});
214+
215+
describe("can be enabled and disabled", () => {
216+
beforeEach(() => {
217+
cy.createTransform(TRANSFORM_ID, sampleTransform);
218+
cy.reload();
219+
});
220+
221+
it("successfully", () => {
222+
// Confirm we have our initial transform
223+
cy.contains(TRANSFORM_ID);
224+
225+
// Click into transform job details page
226+
cy.get(`[data-test-subj="transformLink_${TRANSFORM_ID}"]`).click({ force: true });
227+
228+
cy.contains(`${TRANSFORM_ID}`);
229+
230+
/* Wait required for page data to load, otherwise "Disable" button will
231+
* appear greyed out and unavailable. Cypress automatically retries,
232+
* but only after menu is open, doesn't re-render.
233+
*/
234+
cy.wait(1000);
235+
236+
// Click into Actions menu
237+
cy.get(`[data-test-subj="actionButton"]`).click({ force: true });
238+
239+
// Click Disable button
240+
cy.get(`[data-test-subj="disableButton"]`).click();
241+
242+
// Confirm we get toaster saying transform job is disabled
243+
cy.contains(`"${TRANSFORM_ID}" is disabled`);
244+
245+
cy.wait(1000);
246+
247+
// Click into Actions menu
248+
cy.get(`[data-test-subj="actionButton"]`).click({ force: true });
249+
250+
// Click Enable button
251+
cy.get(`[data-test-subj="enableButton"]`).click({ force: true });
252+
253+
// Confirm we get toaster saying transform job is enabled
254+
cy.contains(`"${TRANSFORM_ID}" is enabled`);
255+
});
256+
})
257+
});

cypress/support/commands.js

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -157,3 +157,21 @@ Cypress.Commands.add("deleteDataStreams", (names) => {
157157
Cypress.Commands.add("rollover", (target) => {
158158
cy.request("POST", `${Cypress.env("opensearch")}/${target}/_rollover`);
159159
});
160+
161+
Cypress.Commands.add("createTransform", (transformId, transformJSON) => {
162+
cy.request("PUT", `${Cypress.env("opensearch")}${API.TRANSFORM_JOBS_BASE}/${transformId}`, transformJSON);
163+
});
164+
165+
Cypress.Commands.add("disableJitter", () => {
166+
// Sets the jitter to 0 in the ISM plugin cluster settings
167+
const jitterJson = {
168+
persistent: {
169+
plugins: {
170+
index_state_management: {
171+
jitter: "0.0",
172+
},
173+
},
174+
},
175+
};
176+
cy.request("PUT", `${Cypress.env("opensearch")}/_cluster/settings`, jitterJson);
177+
});

cypress/support/constants.js

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -26,6 +26,7 @@
2626

2727
export const API_ROUTE_PREFIX = "/_plugins/_ism";
2828
export const API_ROUTE_PREFIX_ROLLUP = "/_plugins/_rollup";
29+
export const API_ROUTE_PREFIX_TRANSFORM = "/_plugins/_transform";
2930

3031
export const INDEX = {
3132
OPENDISTRO_ISM_CONFIG: ".opendistro-ism-config",
@@ -41,6 +42,7 @@ export const API = {
4142
REMOVE_POLICY_BASE: `${API_ROUTE_PREFIX}/remove`,
4243
CHANGE_POLICY_BASE: `${API_ROUTE_PREFIX}/change_policy`,
4344
ROLLUP_JOBS_BASE: `${API_ROUTE_PREFIX_ROLLUP}/jobs`,
45+
TRANSFORM_JOBS_BASE: `${API_ROUTE_PREFIX_TRANSFORM}`,
4446
};
4547

4648
export const PLUGIN_NAME = "opensearch_index_management_dashboards";

0 commit comments

Comments
 (0)