Skip to content

Conversation

@psainics
Copy link
Contributor

@psainics psainics commented Jan 31, 2024

Adding XLS UI elements for S3 source

Jira : Plugin-1730

Description

This PR adds UI element required for XLS Support in S3 Source.
Ref : XLS Support PR cdapio/hydrator-plugins#1835

Added Fields

  • Sample Size
  • Override
  • Terminate If Empty Row
  • Select Sheet Using
  • Sheet Value

A filter is used to show XLS specific values Terminate If Empty Row , Select Sheet Using, Sheet Value only when format XLS is selected.

Preview

image image

Copy link

@DJSagarAhire DJSagarAhire left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Most comments from data-integrations/google-cloud#1368 are also applicable here. Looks good apart from those.

@psainics
Copy link
Contributor Author

psainics commented May 7, 2025

Sanity Tests

Data Used

xlsx_bench.xlsx

  • A xlsx file with 3 sheets
  • Sheet 1 (data with headers)
  • Sheet 2 (data without headers)
  • Sheet 3 (data with headers and an empty line)

"Sheet1" [index 0]

image

"Sheet2" [index 1]

image

"Sheet3" [index 2]

image


Test 1 [Read a sheet with headers]

We will be testing can the plugin read a sheet with headers defined, also skip the first row while reading data.

  • Config

    • Turn on "Use First Row as Header"
      image
  • Deploy
    image

  • Output
    image

Test 2 [Read a sheet using it's name + Read a sheet other than the 1st + Read a sheet without headers]

We will be testing can the plugin read a sheet using the sheet's name.
We will be testing if the plugin can read any sheet other than the first one, we will also be testing if the plugin can read sheet without a header.

  • Config

    • Set "Select Sheet Using" to "Sheet Name"
    • Add the sheet name
      image
  • Deploy
    image

  • Output
    image

Test 3 [Do not stop reading when a row is missing]

We will be testing if the plugin can keep reading if an empty row is encountered.

  • Config

    • Turn off "Terminate Reading After Empty Row"
      image
  • Deploy
    image

  • Output
    image

Test 4 [Stop reading when a row is missing]

We will be testing if the plugin can stop reading if an empty row is encountered.

  • Config

    • Turn on "Terminate Reading After Empty Row"
      image
  • Deploy
    image

  • Output
    image

@psainics psainics force-pushed the patch-s3-xls-ui branch from 29e832a to 08a6d63 Compare May 7, 2025 11:41
@psainics psainics merged commit 3690027 into data-integrations:develop May 7, 2025
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants