v0.46.0

• Added lazy_loader to known list (#2991). With this commit, the lazy_loader module has been added to the known list in the configuration file, addressing a portion of issue #193, which may have been caused by the discovery or loading of this module. The lazy_loader is a package or module that, once added to the known list, will be recognized and loaded by the system. This change does not affect any existing functionality or introduce new methods. The commit solely updates the known.json file to include lazy_loader with an empty list, indicating that it is ready for use. This modification will enable the correct loading and recognition of the lazy_loader module in the system.
• Added librosa to known list (#2992). In this update, we have added several open-source libraries to the known list in the configuration file, including librosa, llvmlite, msgpack, pooch, soundfile, and soxr. These libraries are commonly used in data engineering, machine learning, and scientific computing tasks. librosa is a Python library for audio and music analysis, while llvmlite is a lightweight Python interface to the LLVM compiler infrastructure. msgpack is a binary serialization format like JSON, pooch is a package for managing external data files, soundfile is a library for reading and writing audio files, and soxr is a library for high-quality audio resampling. Each library has an empty list next to it for specifying additional configuration related to the library. This update partially resolves issue #1931 by adding librosa to the known list, ensuring that these libraries will be properly recognized and utilized by the codebase.
• Added linkify-it-py to known list (#2993). In this release, we have added support for two new open-source packages, linkify-it-py and uc-micro-py, to enhance the software's functionality and compatibility. The addition of linkify-it-py and its constituent modules, as well as the incorporation of uc-micro-py with its modules and classes, aims to expand the software's capabilities. These changes are related to the resolution of issue #1931, and they will enable the software to work seamlessly with these packages, thereby providing a better user experience.
• Added lz4 to known list (#2994). In this release, we have added support for the LZ4 lossless data compression algorithm, which is known for its focus on compression and decompression speed. The implementation includes four variants: lz4, lz4.block, lz4.frame, and lz4.version, each providing different levels of compression and decompression speed and flexibility. This addition expands the range of supported compression algorithms, providing more options for users to choose from and partially addressing issue #1931 related to supporting additional compression algorithms. This improvement will be beneficial to software engineers working with data compression in their projects.
• Fixed SystemError: AST constructor recursion depth mismatch failing the entire job (#3000). This PR introduces more deterministic, Go-style, error handling for parsing Python code, addressing issues that caused the entire job to fail due to a SystemError: AST constructor recursion depth mismatch (#3000) and bug #2976. It includes removing the AstroidSyntaxError import, adding an import for SqlglotError, and updating the SqlParseError exception to SqlglotError in the lint method of the SqlLinter class. Additionally, abstract classes TablePyCollector and DfsaPyCollector and their respective methods for collecting tables and direct file system accesses have been removed. The PythonSequentialLinter class, previously handling multiple responsibilities, has also been removed, enhancing code modularity, understandability, maintainability, and testability. The changes affect the base.py, python_ast.py, and python_sequential_linter.py modules.
• Skip applying permissions for workspace system groups to Unity Catalog resources (#2997). This commit introduces changes to the ACL-related code in the databricks labs ucx create-catalog-schemas command and the migrate-table-* workflow, skipping the application of permissions for workspace system groups in the Unity Catalog. These system groups, which include 'admins', do not exist at the account level. To ensure the correctness of these modifications, unit and integration tests have been added, including a test that checks the proper handling of user privileges in system groups during catalog schema creation. The AccessControlResponse object has been updated for the admins and users groups, granting them specific permissions for a workspace and warehouse object, respectively, enhancing the system's functionality in multi-user environments with system groups.

Contributors: @pritishpai, @asnare, @JCZuurmond, @nfx