chore: cherry-pick fixes from main for 3.8.1 release #27110
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
* fix: show WriteLineError.error_message in ParseError display * fix: show source of reqwest::Error in influxdb3_client
) This PR backports several bug fixes and improvements related to retention and deletion from the `influxdata/influxdb_pro` repository: - **influxdb_pro PR # 1986**: Fix catalog to prevent deleting tables from already deleted databases - **influxdb_pro PR # 1991**: Update error message from "delete" to "modify" for `AlreadyDeleted` error - **influxdb_pro PR # 2043**: Add resource name to `AlreadyDeleted` error for better error messages - **influxdb_pro PR # 2046**: Set default retention period for `_internal` database to 7 days Changes: - Add check in soft_delete_table to return `AlreadyDeleted` error if database is already deleted - Change `CatalogError::AlreadyDeleted` to include resource name - Update all `AlreadyDeleted` error sites to include resource name - Add `INTERNAL_DB_RETENTION_PERIOD` constant (7 days) - Update `create_internal_db` to use retention period
* fix: add EOF marker to end of metrics scrape * fix: add EOF marker to end of metrics scrape
Bumps [rsa](https://github.com/RustCrypto/RSA) from 0.9.9 to 0.9.10. - [Changelog](https://github.com/RustCrypto/RSA/blob/v0.9.10/CHANGELOG.md) - [Commits](RustCrypto/RSA@v0.9.9...v0.9.10) --- updated-dependencies: - dependency-name: rsa dependency-version: 0.9.10 dependency-type: indirect ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
This change is based on an assumed issue in data which spans many months with very small wal files; we assume the data is present in most of the 10 minute table chunks over the time range; this is sparse data that has only a few rows per 10 minutes. The TableBuffer creates a MutableTableChunk for each 10 min chunk in the months range. With arrow's default 1024 element allocations for our tag and field information, this can be a substantial in-memory use for wal files that serialize to a few megs or less. It is akin to a zipbomb. If the wal files were larger, snapshots would have been triggered. Instead, the TableBuffer grows to many GBs. The changes here use the exact capacity that can be determined when rows are added to the MutableTableChunk. Some workloads that were coincidentally tuned to the 1024 element default will see more allocations but like Vecs the exponential reservations will catch up fast. * port of influxdata/influxdb_pro#2071
* chore: additional retention logging * feat: retention show output * chore: display retention period in human readable format Co-authored-by: Joe-Blount <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR preps for today's 3.8.1 release by cherry-pick various fixes from
mainonto3.8.