Skip to content

Conversation

@lleadbet
Copy link
Contributor

What

The source-file connector's Reader Options supports the ability to pass in Panda options to the related type of data. This works for most types, as we use the underlying Panda function, however for Excel files we use a custom openpyxl_chunk_reader by default which does not.

This PR adds rudimentary support for sheet_name in the custom function to keep things consistent with our documentation, as well as allowing users to select a sheet to use (or more).

Our general implementation doesn't fully abide by Pandas's in that it notably flattens all columns whereas Pandas's does not, but it keeps it more consistent with our existing behavior.

How

Adds explicit support for the arg with basic filtering logic.

Review guide

Validate it works correctly.

User Impact

No changes for existing users.

Can this PR be safely reverted and rolled back?

  • YES 💚
  • NO ❌

@github-actions
Copy link
Contributor

👋 Greetings, Airbyte Team Member!

Here are some helpful tips and reminders for your convenience.

Helpful Resources

PR Slash Commands

Airbyte Maintainers (that's you!) can execute the following slash commands on your PR:

  • /format-fix - Fixes most formatting issues.
  • /bump-version - Bumps connector versions.
    • You can specify a custom changelog by passing changelog. Example: /bump-version changelog="My cool update"
    • Leaving the changelog arg blank will auto-populate the changelog from the PR title.
  • /run-cat-tests - Runs legacy CAT tests (Connector Acceptance Tests)
  • /build-connector-images - Builds and publishes a pre-release docker image for the modified connector(s).
  • JVM connectors:
    • /update-connector-cdk-version connector=<CONNECTOR_NAME> - Updates the specified connector to the latest CDK version.
      Example: /update-connector-cdk-version connector=destination-bigquery
    • /bump-bulk-cdk-version bump=patch changelog='foo' - Bump the Bulk CDK's version. bump can be major/minor/patch.
  • Python connectors:
    • /poe connector source-example lock - Run the Poe lock task on the source-example connector, committing the results back to the branch.
    • /poe source example lock - Alias for /poe connector source-example lock.
    • /poe source example use-cdk-branch my/branch - Pin the source-example CDK reference to the branch name specified.
    • /poe source example use-cdk-latest - Update the source-example CDK dependency to the latest available version.

📝 Edit this welcome message.

@github-actions
Copy link
Contributor

source-file Connector Test Results

67 tests   62 ✅  11s ⏱️
 2 suites   5 💤
 2 files     0 ❌

Results for commit 548b02b.

@lleadbet
Copy link
Contributor Author

Closing in favor of #69774.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants