|
1 | 1 | # Version changelog |
2 | 2 |
|
| 3 | +## 0.32.0 |
| 4 | + |
| 5 | +* Added troubleshooting guide for self-signed SSL cert related error ([#2346](https://github.com/databrickslabs/ucx/issues/2346)). In this release, we have added a troubleshooting guide to the README file to address a specific error that may occur when connecting from a local machine to a Databricks Account and Workspace using a web proxy and self-signed SSL certificate. This error, SSLCertVerificationError, can prevent UCX from connecting to the Account and Workspace. To resolve this issue, users can now set the `REQUESTS_CA_BUNDLE` and `CURL_CA_BUNDLE` environment variables to force the requests library to set `verify=False`, and set the `SSL_CERT_DIR` env var pointing to the proxy CA cert for the urllib3 library. This guide will help users understand and resolve this error, making it easier to connect to Databricks Accounts and Workspaces using a web proxy and self-signed SSL certificate. |
| 6 | +* Code Compatibility Dashboard: Fix broken links ([#2347](https://github.com/databrickslabs/ucx/issues/2347)). In this release, we have addressed and resolved two issues in the Code Compatibility Dashboard of the UCX Migration (Main) project, enhancing its overall usability. Previously, the Markdown panel contained a broken link to the workflow due to an incorrect anchor, and the links in the table widget to the workflow and task definitions did not render correctly. These problems have been rectified, and the dashboard has been manually tested and verified in a staging environment. Additionally, we have updated the `invisibleColumns` section in the SQL file by changing the `fieldName` attribute to 'name', which will now display the `workflow_id` as a link. Before and after screenshots have been provided for visual reference. The corresponding workflow is now referred to as "Jobs Static Code Analysis Workflow". |
| 7 | +* Filter out missing import problems for imports within a try-except clause with ImportError ([#2332](https://github.com/databrickslabs/ucx/issues/2332)). This release introduces changes to handle missing import problems within a try-except clause that catches ImportError. A new method, `_filter_import_problem_in_try_except`, has been added to filter out import-not-found issues when they occur in such a clause, preventing unnecessary build failures. The `_register_import` method now returns an Iterable[DependencyProblem] instead of yielding problems directly. Supporting classes and methods, including Dependency, DependencyGraph, and DependencyProblem from the databricks.labs.ucx.source_code.graph module, as well as FileLoader and PythonCodeAnalyzer from the databricks.labs.ucx.source_code.notebooks.cells module, have been added. The ImportSource.extract_from_tree method has been updated to accept a DependencyProblem object as an argument. Additionally, a new test case has been included for the scenario where a failing import in a try-except clause goes unreported. Issue [#1705](https://github.com/databrickslabs/ucx/issues/1705) has been resolved, and unit tests have been added to ensure proper functionality. |
| 8 | +* Fixed `report-account-compatibility` cli command docstring ([#2340](https://github.com/databrickslabs/ucx/issues/2340)). In this release, we have updated the `report-account-compatibility` CLI command's docstring to accurately reflect its functionality, addressing a previous issue where it inadvertently duplicated the `sync-workspace-info` command's description. This command now provides a clear and concise explanation of its purpose: "Report compatibility of all workspaces available in the account." Upon execution, it generates a readiness report for the account, specifically focusing on workspaces where ucx is installed. This enhancement improves the clarity of the CLI's functionality for software engineers, enabling them to understand and effectively utilize the `report-account-compatibility` command. |
| 9 | +* Fixed broken table migration workflow links in README ([#2286](https://github.com/databrickslabs/ucx/issues/2286)). In this release, we have made significant improvements to the README file of our open-source library, including fixing broken links and adding a mermaid flowchart to demonstrate the table migration workflows. The table migration workflow has been renamed to the table migration process, which includes migrating Delta tables, non-Delta tables, external tables, and views. Two optional workflows have been added for migrating HiveSerDe tables in place and for migrating external tables using CTAS. Additionally, the commands related to table migration have been updated, with the table migration workflow being renamed to the table migration process. These changes are aimed at providing a more comprehensive understanding of the table migration process and enhancing the overall user experience. |
| 10 | +* Fixed dashboard queries fail when default catalog is not `hive_metastore` ([#2278](https://github.com/databrickslabs/ucx/issues/2278)). In this release, we have addressed an issue where dashboard queries fail when the default catalog is not set to `hive_metastore`. This has been achieved by modifying the existing `databricks labs ucx install` command to always include the `hive_metastore` namespace in dashboard queries. Additionally, the code has been updated to add the `hive_metastore` namespace to the `DashboardMetadata` object used in creating a dashboard from SQL queries in a folder, ensuring queries are executed in the correct database. The commit also includes modifications to the `test_install.py` unit test file to ensure the installation process correctly handles specific configurations related to the `ucx` namespace for managing data storage and retrieval. The changes have been manually tested and verified on a staging environment. |
| 11 | +* Improve group migration error reporting ([#2344](https://github.com/databrickslabs/ucx/issues/2344)). This PR introduces enhancements to the group migration dashboard, focusing on improved error reporting and a more informative user experience. The documentation widgets have been fine-tuned, and the failed-migration widget now provides formatted failure information with a link to the failed job run. The dashboard will display only failures from the latest workflow run, complete with logs. A new link to the job list has been added in the [workflows](/jobs) section of the documentation to assist users in identifying and troubleshooting issues. Additionally, the SQL query for retrieving group migration failure information has been refactored, improving readability and extracting relevant data using regular expressions. The changes have been tested and verified on the staging environment, providing clearer and more actionable insights during group migrations. The PR is related to previous work in [#2333](https://github.com/databrickslabs/ucx/issues/2333) and [#1914](https://github.com/databrickslabs/ucx/issues/1914), with updates to the UCX Migration (Groups) dashboard, but no new methods have been added. |
| 12 | +* Improve type checking in cli command ([#2335](https://github.com/databrickslabs/ucx/issues/2335)). This release introduces enhanced type checking in the command line interface (CLI) of our open-source library, specifically in the `lint_local_code` function of the `cli.py` file. By utilizing a newly developed local code linter object, the function now performs more rigorous and accurate type checking for potential issues in the local code. While the functionality remains consistent, this improvement is expected to prevent similar occurrences like issue [#2221](https://github.com/databrickslabs/ucx/issues/2221), ensuring more robust and reliable code. This change underscores our commitment to delivering a high-quality, efficient, and developer-friendly library. |
| 13 | +* Lint dependencies in context ([#2236](https://github.com/databrickslabs/ucx/issues/2236)). The `InheritedContext` class has been introduced to gather code fragments from parent files or notebooks during linting of child files or notebooks, addressing issues [#2155](https://github.com/databrickslabs/ucx/issues/2155), [#2156](https://github.com/databrickslabs/ucx/issues/2156), and [#2221](https://github.com/databrickslabs/ucx/issues/2221). This new feature includes the addition of the `InheritedContext` class, with methods for building instances from a route of dependencies, appending other `InheritedContext` instances, and finalizing them for use with linting. The `DependencyGraph` class has been updated to support the new functionality, and various classes, methods, and functions for handling the linter context have been added or updated. Unit, functional, and integration tests have been added to ensure the correct functioning of the changes, which improve the linting functionality by allowing it to consider the broader context of the codebase. |
| 14 | +* Make ucx pylsp plugin configurable ([#2280](https://github.com/databrickslabs/ucx/issues/2280)). This commit introduces the ability to configure the ucx pylsp plugin with cluster information, which can be provided either in a file or by a client and is managed by the pylsp infrastructure. The Spark Connect linter is now only applied to UC Shared clusters, as Single-User clusters run in Spark Classic mode. A new entry point `pylsp_ucx` has been added to the pylsp configuration file. The changes affect the pylsp plugin configuration and the application of the Spark Connect linter. Unit tests and manual testing have been conducted, but integration tests and verification on a staging environment are not included in this release. |
| 15 | +* New dashboard: group migration, showing groups that failed to migrate ([#2333](https://github.com/databrickslabs/ucx/issues/2333)). In this release, we have developed a new dashboard for monitoring group migration in the UCX Migration (Groups) workspace. This dashboard includes a widget displaying messages related to groups that failed to migrate during the `migrate-groups-experimental` workflow, aiding users in identifying and addressing migration issues. The group migration process consists of several steps, including renaming workspace groups, provisioning account-level groups, and replicating permissions. The release features new methods for displaying and monitoring migration-related messages, as well as links to documentation and workflows for assessing, validating, and removing workspace-level groups post-migration. The new dashboard is currently not connected to the existing system, but it has undergone manual testing and verification on the staging environment. The changes include the addition of a new SQL query file to implement the logic for fetching group migration failures and a new Markdown file displaying the Group Migration Failures section. |
| 16 | +* Support spaces in run cmd args ([#2330](https://github.com/databrickslabs/ucx/issues/2330)). The recent commit resolves an issue where the system had trouble handling spaces in command-line arguments when running subprocesses. The previous implementation only accepted a full command line, which it would split on spaces, causing problems when the command line contained arguments with spaces. The new implementation supports argument lists, which are passed `as is` to `Popen`, allowing for proper handling of command lines with spaces. This change is incorporated in the `run_command` function of the `utils.py` file and the `_install_pip` method of the `PythonLibraryResolver` class. The `shlex.join()` function has been replaced with direct string formatting for increased flexibility. The feature is intended for use with the `PythonLibraryResolver` class and is co-authored by Eric Vergnaud and Andrew Snare. Integration tests have been enabled to ensure the proper functioning of the updated code. |
| 17 | +* Updated error messages for SparkConnect linter ([#2348](https://github.com/databrickslabs/ucx/issues/2348)). The SparkConnect linter's error messages have been updated to improve clarity and precision. The term `UC Shared clusters` has been replaced with `Unity Catalog clusters in Shared access mode` throughout the codebase, affecting messages related to various unsupported functionalities or practices on these clusters. These changes include warnings about direct Spark log level setting, accessing the Spark Driver JVM or its logger, using `sc`, and employing RDD APIs. This revision enhances user experience by providing more accurate and descriptive error messages, enabling them to better understand and address the issues in their code. The functionality of the linter remains unchanged. |
| 18 | +* Updated sqlglot requirement from <25.8,>=25.5.0 to >=25.5.0,<25.9 ([#2279](https://github.com/databrickslabs/ucx/issues/2279)). In this update, we have updated the required version range of the `sqlglot` dependency in the 'pyproject.toml' file from 'sqlglot>=25.5.0,<25.8' to 'sqlglot>=25.5.0,<25.9'. This change allows the project to utilize any version of `sqlglot` that is greater than or equal to 25.5.0 and less than 25.9, including the latest version. The update also includes a changelog for the updated version range, sourced from 'sqlglot's official changelog. This changelog includes various bug fixes and new features for several dialects such as BigQuery, DuckDB, and tSQL. Additionally, the parser has undergone some refactors and improvements. The commits section lists the individual commits included in this update. |
| 19 | + |
| 20 | +Dependency updates: |
| 21 | + |
| 22 | + * Updated sqlglot requirement from <25.8,>=25.5.0 to >=25.5.0,<25.9 ([#2279](https://github.com/databrickslabs/ucx/pull/2279)). |
| 23 | + |
3 | 24 | ## 0.31.0 |
4 | 25 |
|
5 | 26 | * Added handling for corrupted dashboard state in installation process ([#2262](https://github.com/databrickslabs/ucx/issues/2262)). This commit introduces a new method, `_handle_existing_dashboard`, to manage various scenarios related to an existing dashboard during the installation process of UCX, including updating, upgrading from Redash to Lakeview, handling trashed dashboards, and recovering corrupted dashboard references. The `_create_dashboard` method now requires a non-null `parent_path` and has updated its docstring. New unit and integration tests have been added to verify the changes, with a particular focus on handling corrupted dashboard state during the installation process. These tests resolve issue [#2261](https://github.com/databrickslabs/ucx/issues/2261) and improve the reliability of the installation process. Additionally, the commit includes new fixtures such as `ws`, `any_prompt`, and `clusters` to create controlled test environments and ensure proper validation of the dashboard and removal of the database. The `DashboardMetadata` class has been removed from `databricks.labs.lsql.dashboards`, and the `ProductInfo` and `WheelsV2` classes have been updated in `databricks.labs.blueprint.wheels`. The `remove_database` method has also been added to the `MockBackend` class in `databricks.labs.lsql.backends`. This comprehensive update enhances the robustness of the installation process and the handling of different dashboard types and scenarios. |
|
0 commit comments