Closed
Conversation
…unctions (MaterializeInc#31097) This PR changes the MySQL source to support ingesting the `bit` type as `uint8`. It also adds two new Postgres functions, `bit_count(bytea)` and `get_bit(bytea, int32)` to making working with byte strings easier. ### Motivation Progress towards: MaterializeInc/database-issues#8891 ### Checklist - [x] This PR has adequate test coverage / QA involvement has been duly considered. ([trigger-ci for additional test/nightly runs](https://trigger-ci.dev.materialize.com/)) - [x] This PR has an associated up-to-date [design doc](https://github.com/MaterializeInc/materialize/blob/main/doc/developer/design/README.md), is a design doc ([template](https://github.com/MaterializeInc/materialize/blob/main/doc/developer/design/00000000_template.md)), or is sufficiently small to not require a design. <!-- Reference the design in the description. --> - [x] If this PR evolves [an existing `$T ⇔ Proto$T` mapping](https://github.com/MaterializeInc/materialize/blob/main/doc/developer/command-and-response-binary-encoding.md) (possibly in a backwards-incompatible way), then it is tagged with a `T-proto` label. - [x] If this PR will require changes to cloud orchestration or tests, there is a companion cloud PR to account for those changes that is tagged with the release-blocker label ([example](MaterializeInc/cloud#5021)). <!-- Ask in #team-cloud on Slack if you need help preparing the cloud PR. --> - [x] If this PR includes major [user-facing behavior changes](https://github.com/MaterializeInc/materialize/blob/main/doc/developer/guide-changes.md#what-changes-require-a-release-note), I have pinged the relevant PM to schedule a changelog post. --------- Co-authored-by: Dennis Felsing <dennis@felsing.org>
Enabled widely by default with zero consequences so far.
Don't need an explicit %N anymore
The materialized binary generates massive debuginfo: 8GB with full debuginfo and 3.5GB with limited debuginfo. (This is not the compressed size of the debuginfo on disk, but the amount of memory required to load the debuginfo into memory so that backtraces can be symbolized.) We've historically shipped the materialized binary with full debuginfo, so that we get rich backtraces for any crashes users run into when running the emulator. Unfortunately, the full debuginfo is so large that it is itself the *cause* of OOMs [0], which is unacceptable. (When a console query encounters certain routine errors, like a connection failing to validate, the adapter attempts to log a backtrace, which requires loading the debuginfo.) Even the limited debuginfo size (3.5GB) is unacceptable for a Docker image that's meant to be run on developer laptops. So, this commit adjusts the materialized image to strip all debuginfo from the binary. If a user reports a crash with an unsymbolized backtrace, it's still *possible* (just painful) to manually symbolize the backtrace as long as they give us the exact version of Materialize they were running. We'll just need to manually run `addr2line` on each address reported in the backtrace. (We do irrevocably lose access to frames for inlined functions, but that seems tolerable.) One silver lining here is that the Docker image will get much smaller. It's currently about 1GB. I expect this change to shave off a huge chunk of that. [0]: https://materializeinc.slack.com/archives/C07FX1W1Y03/p1737414021061139
…alizeInc#31127) This refactors the impl of `Blob` for Azure in a way that should be faster. The `BlobClient` we use from the `azure_storage_blob` crate returns a `Stream` that when `await`-ed sends a ranged GET request for a chunk of a blob. This PR refactors our impl so we await each ranged request in a `tokio::task` which increases the concurrency at which we fetch chunks of a `Part`. It also refactors how we handle the case when the `content-length` header is missing, and adds metrics so we can track how often this occurs. ### Motivation Maybe progress against MaterializeInc/database-issues#8892 ### Checklist - [ ] This PR has adequate test coverage / QA involvement has been duly considered. ([trigger-ci for additional test/nightly runs](https://trigger-ci.dev.materialize.com/)) - [ ] This PR has an associated up-to-date [design doc](https://github.com/MaterializeInc/materialize/blob/main/doc/developer/design/README.md), is a design doc ([template](https://github.com/MaterializeInc/materialize/blob/main/doc/developer/design/00000000_template.md)), or is sufficiently small to not require a design. <!-- Reference the design in the description. --> - [ ] If this PR evolves [an existing `$T ⇔ Proto$T` mapping](https://github.com/MaterializeInc/materialize/blob/main/doc/developer/command-and-response-binary-encoding.md) (possibly in a backwards-incompatible way), then it is tagged with a `T-proto` label. - [ ] If this PR will require changes to cloud orchestration or tests, there is a companion cloud PR to account for those changes that is tagged with the release-blocker label ([example](MaterializeInc/cloud#5021)). <!-- Ask in #team-cloud on Slack if you need help preparing the cloud PR. --> - [ ] If this PR includes major [user-facing behavior changes](https://github.com/MaterializeInc/materialize/blob/main/doc/developer/guide-changes.md#what-changes-require-a-release-note), I have pinged the relevant PM to schedule a changelog post.
Previously, INSPECT SHARD would panic if it was given an ID that didn't belong to any item. This commit fixes the issue by returning an error instead. Fixes #MaterializeInc/database-issues/issues/8910
We built this for the streaming iterator, but it works well here too.
In particular, don't decode K/Vs when we have an override set, and make sure every K/V only gets decoded once.
- Moves the code inside `datadriven_testcase()`, where there is already other similar code. - Places the code to after the normal (non-redacted) reparse check. It's better like this, because when the redacted check was before the normal check, it used to just complain about the redacted reparse even when the normal reparse would also fail, which was misleading me to think that there is something wrong specifically with the redaction.
Newlines are represented as a ⏎ character in the result specification.
…-linear-operators.td
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
13 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Just running CI on my cherry-pick state.
Motivation
Tips for reviewer
Checklist
$T ⇔ Proto$Tmapping (possibly in a backwards-incompatible way), then it is tagged with aT-protolabel.