Skip to content

v0.23.1

Choose a tag to compare

@nfx nfx released this 10 May 15:07
· 1002 commits to main since this release
7d07554
  • Improved error handling for migrate-tables workflows (#1674). This commit enhances the error handling for migrate-tables workflows by introducing new tests that cover specific scenarios where failures may occur during table migration. The changes include the creation of mock objects and injecting failures for the get_tables_to_migrate method of the TableMapping class. Three new tests have been added, each testing a specific scenario, including token errors when checking table properties, errors when trying to get properties for a non-existing table, and errors when trying to unset the upgraded_to property. The commit also asserts that specific error messages are logged during these failures. These improvements ensure better visibility and debugging capabilities during table migration. The code was manually tested, and unit tests were added and verified on a staging environment, ensuring that the new error handling mechanisms function as intended.
  • Improved error handling for all queries executed during table migration (#1679). This release includes improved error handling during table migration in our data workflow, resolving issue #167
  • Removed dependency on internal pathlib implementations (#1672). In this release, we have introduced a new custom _DatabricksFlavor class as a replacement for the internal pathlib._Flavor implementations, specifically designed for the Databricks environment. This class handles various functionalities including separation of paths, joining and parsing of parts, and casefolding of strings, among others. The make_uri method has also been updated to generate the correct URI for the workspace. This change removes the dependency on internal pathlib._Flavor implementations which were not available on Windows. As part of this change, the test_wspath.py file in the tests/integration/mixins directory has been updated, with the test_exists and test_mkdirs methods being modified to reflect the removal of _Flavor. These updates improve the compatibility and reliability of the codebase on Windows systems.
  • Updated databricks-labs-blueprint requirement from ~=0.4.3 to >=0.4.3,<0.6.0 (#1670). In this update, we have adjusted the requirement for databricks-labs-blueprint from version ~=0.4.3 to >=0.4.3,<0.6.0, ensuring the latest version can be installed while remaining below 0.6.0. This change is part of issue #1670 and includes the release notes and changelog in the commit message, highlighting improvements and updates in version 0.5.0. These enhancements consist of content assertion in MockInstallation, better handling of partial functions in parallel.Threads, and adjusted configurations aligned with the UCX project. The commit also covers various dependency updates and bug fixes, providing a more robust and efficient library experience for software engineers.

Dependency updates:

  • Updated databricks-labs-blueprint requirement from ~=0.4.3 to >=0.4.3,<0.6.0 (#1670).

Contributors: @nkvuong, @dependabot[bot], @nfx, @JCZuurmond