Skip to content

make protobuf contentview bidirectional #4

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
mhils wants to merge 33 commits into
base: protobuf-contentview
Choose a base branch
from
Open

make protobuf contentview bidirectional #4

mhils wants to merge 33 commits into from

Conversation

@mhils
Copy link
Owner

@mhils mhils commented Mar 28, 2025

No description provided.

@mhils mhils mentioned this pull request Mar 28, 2025
Merged
1 task
emanuele-em
emanuele-em reviewed Mar 29, 2025
View reviewed changes
src/contentviews/hex_stream.rs Outdated

impl Reencode for HexStream {
fn reencode(&self, data: String) -> anyhow::Result<Vec<u8>, ReencodeError> {
fn reencode(&self, data: &str, original: &[u8]) -> anyhow::Result<Vec<u8>, ReencodeError> {
Copy link

@emanuele-em emanuele-em Mar 29, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mhils It's not clear to me why we need "original"

Copy link
Owner Author

@mhils mhils Mar 31, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Without going too much on a tangent:

  1. Protobuf has different ways to serialize numbers (fixed32, fixed64, varint).
  2. When reencoding, we want to serialize to the same format.
  3. The first implementation of this PR inspected the original message to figure out which format should be used for a particular YAML number.

I've since then changed two things:

  1. We now encode the number format in the YAML if it's not a varint, e.g. a fixed64-encoded number will be explicitly encoded as !fixed64 42 and not just 42.
  2. "original" has moved from being part of the data to being part of the metadata. For some contentviews it can still be potentially useful.

@mhils mhils force-pushed the protobuf-reencode branch from caafe7f to b4cc446 Compare March 30, 2025 12:38
errorxyz and others added 6 commits March 31, 2025 16:51
@errorxyz @autofix-ci @mhils
Wireguard: Bind server to specified address only (mitmproxy#242)
11bf9ab 
* Require host to be specified for WireGuardConf and set IPV6_V6ONLY

* [autofix.ci] apply automated fixes

* Reuse code from  for building wireguard udpsocket

* [autofix.ci] apply automated fixes

* Improve comments

* Accept all types that implement ToSocketAddrs trait for creating UdpSocket

* Make create_udp_socket pub(crate)

* [autofix.ci] apply automated fixes

* just use SocketAddr everywhere

* [autofix.ci] apply automated fixes

---------

Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: Maximilian Hils <[email protected]>
@dependabot
Bump windows from 0.60.0 to 0.61.1 in the windows group (mitmproxy#244)
2e4a8bb 
Bumps the windows group with 1 update: [windows](https://github.com/microsoft/windows-rs).


Updates `windows` from 0.60.0 to 0.61.1
- [Release notes](https://github.com/microsoft/windows-rs/releases)
- [Commits](https://github.com/microsoft/windows-rs/commits)

---
updated-dependencies:
- dependency-name: windows
  dependency-version: 0.61.1
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: windows
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
@dependabot
Bump the dependencies group with 3 updates (mitmproxy#246)
97827d6 
Bumps the dependencies group with 3 updates: [install-pinned/maturin-with-zig](https://github.com/install-pinned/maturin-with-zig), [install-pinned/maturin](https://github.com/install-pinned/maturin) and [install-pinned/pdoc](https://github.com/install-pinned/pdoc).


Updates `install-pinned/maturin-with-zig` from caba41a16e619c003b5adb81920d21ee51dfad00 to b1490b711b58bce2e99c1b49957651fefa9ed6b1
- [Release notes](https://github.com/install-pinned/maturin-with-zig/releases)
- [Commits](install-pinned/maturin-with-zig@caba41a...b1490b7)

Updates `install-pinned/maturin` from 18a9a3b28f101f1737ff0b83d09800c06c0685f7 to 43b39b11faf67ba5f8922a76d274cfe7795778fd
- [Release notes](https://github.com/install-pinned/maturin/releases)
- [Commits](install-pinned/maturin@18a9a3b...43b39b1)

Updates `install-pinned/pdoc` from 5a894a20c6b4e9db4d3fe07efd9680b0f753b0d1 to 69ba59f9699df21e1026110af4ec6b10a98cf5cd
- [Release notes](https://github.com/install-pinned/pdoc/releases)
- [Commits](install-pinned/pdoc@5a894a2...69ba59f)

---
updated-dependencies:
- dependency-name: install-pinned/maturin-with-zig
  dependency-version: b1490b711b58bce2e99c1b49957651fefa9ed6b1
  dependency-type: direct:production
  dependency-group: dependencies
- dependency-name: install-pinned/maturin
  dependency-version: 43b39b11faf67ba5f8922a76d274cfe7795778fd
  dependency-type: direct:production
  dependency-group: dependencies
- dependency-name: install-pinned/pdoc
  dependency-version: 69ba59f9699df21e1026110af4ec6b10a98cf5cd
  dependency-type: direct:production
  dependency-group: dependencies
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
@dependabot @mhils @autofix-ci
Bump the dependencies group with 13 updates (mitmproxy#245)
f0d51ed 
* Bump the dependencies group with 13 updates

Bumps the dependencies group with 13 updates:

| Package | From | To |
| --- | --- | --- |
| [log](https://github.com/rust-lang/log) | `0.4.26` | `0.4.27` |
| [once_cell](https://github.com/matklad/once_cell) | `1.20.3` | `1.21.3` |
| [tokio](https://github.com/tokio-rs/tokio) | `1.43.0` | `1.44.1` |
| [image](https://github.com/image-rs/image) | `0.25.5` | `0.25.6` |
| [tokio-util](https://github.com/tokio-rs/tokio) | `0.7.13` | `0.7.14` |
| [hickory-resolver](https://github.com/hickory-dns/hickory-dns) | `0.24.4` | `0.25.1` |
| [socket2](https://github.com/rust-lang/socket2) | `0.5.8` | `0.5.9` |
| [sysinfo](https://github.com/GuillaumeGomez/sysinfo) | `0.33.1` | `0.34.1` |
| [tun](https://github.com/meh/rust-tun) | `0.7.13` | `0.7.17` |
| [tempfile](https://github.com/Stebalien/tempfile) | `3.17.1` | `3.19.1` |
| [hickory-server](https://github.com/hickory-dns/hickory-dns) | `0.24.4` | `0.25.1` |
| [pyo3-log](https://github.com/vorner/pyo3-log) | `0.12.1` | `0.12.2` |
| [libc](https://github.com/rust-lang/libc) | `0.2.170` | `0.2.171` |


Updates `log` from 0.4.26 to 0.4.27
- [Release notes](https://github.com/rust-lang/log/releases)
- [Changelog](https://github.com/rust-lang/log/blob/master/CHANGELOG.md)
- [Commits](rust-lang/log@0.4.26...0.4.27)

Updates `once_cell` from 1.20.3 to 1.21.3
- [Changelog](https://github.com/matklad/once_cell/blob/master/CHANGELOG.md)
- [Commits](matklad/once_cell@v1.20.3...v1.21.3)

Updates `tokio` from 1.43.0 to 1.44.1
- [Release notes](https://github.com/tokio-rs/tokio/releases)
- [Commits](tokio-rs/tokio@tokio-1.43.0...tokio-1.44.1)

Updates `image` from 0.25.5 to 0.25.6
- [Changelog](https://github.com/image-rs/image/blob/main/CHANGES.md)
- [Commits](image-rs/image@v0.25.5...v0.25.6)

Updates `tokio-util` from 0.7.13 to 0.7.14
- [Release notes](https://github.com/tokio-rs/tokio/releases)
- [Commits](tokio-rs/tokio@tokio-util-0.7.13...tokio-util-0.7.14)

Updates `hickory-resolver` from 0.24.4 to 0.25.1
- [Release notes](https://github.com/hickory-dns/hickory-dns/releases)
- [Changelog](https://github.com/hickory-dns/hickory-dns/blob/main/OLD-CHANGELOG.md)
- [Commits](hickory-dns/hickory-dns@v0.24.4...v0.25.1)

Updates `socket2` from 0.5.8 to 0.5.9
- [Release notes](https://github.com/rust-lang/socket2/releases)
- [Changelog](https://github.com/rust-lang/socket2/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/socket2/commits)

Updates `sysinfo` from 0.33.1 to 0.34.1
- [Changelog](https://github.com/GuillaumeGomez/sysinfo/blob/master/CHANGELOG.md)
- [Commits](https://github.com/GuillaumeGomez/sysinfo/commits)

Updates `tun` from 0.7.13 to 0.7.17
- [Release notes](https://github.com/meh/rust-tun/releases)
- [Commits](meh/rust-tun@v0.7.13...v0.7.17)

Updates `tempfile` from 3.17.1 to 3.19.1
- [Changelog](https://github.com/Stebalien/tempfile/blob/master/CHANGELOG.md)
- [Commits](Stebalien/tempfile@v3.17.1...v3.19.1)

Updates `hickory-server` from 0.24.4 to 0.25.1
- [Release notes](https://github.com/hickory-dns/hickory-dns/releases)
- [Changelog](https://github.com/hickory-dns/hickory-dns/blob/main/OLD-CHANGELOG.md)
- [Commits](hickory-dns/hickory-dns@v0.24.4...v0.25.1)

Updates `pyo3-log` from 0.12.1 to 0.12.2
- [Changelog](https://github.com/vorner/pyo3-log/blob/main/CHANGELOG.md)
- [Commits](vorner/pyo3-log@v0.12.1...v0.12.2)

Updates `libc` from 0.2.170 to 0.2.171
- [Release notes](https://github.com/rust-lang/libc/releases)
- [Changelog](https://github.com/rust-lang/libc/blob/0.2.171/CHANGELOG.md)
- [Commits](rust-lang/libc@0.2.170...0.2.171)

---
updated-dependencies:
- dependency-name: log
  dependency-version: 0.4.27
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: dependencies
- dependency-name: once_cell
  dependency-version: 1.21.3
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: dependencies
- dependency-name: tokio
  dependency-version: 1.44.1
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: dependencies
- dependency-name: image
  dependency-version: 0.25.6
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: dependencies
- dependency-name: tokio-util
  dependency-version: 0.7.14
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: dependencies
- dependency-name: hickory-resolver
  dependency-version: 0.25.1
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: dependencies
- dependency-name: socket2
  dependency-version: 0.5.9
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: dependencies
- dependency-name: sysinfo
  dependency-version: 0.34.1
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: dependencies
- dependency-name: tun
  dependency-version: 0.7.17
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: dependencies
- dependency-name: tempfile
  dependency-version: 3.19.1
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: dependencies
- dependency-name: hickory-server
  dependency-version: 0.25.1
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: dependencies
- dependency-name: pyo3-log
  dependency-version: 0.12.2
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: dependencies
- dependency-name: libc
  dependency-version: 0.2.171
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: dependencies
...

Signed-off-by: dependabot[bot] <[email protected]>

* adjust to hickory changes

* [autofix.ci] apply automated fixes

---------

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Maximilian Hils <[email protected]>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
@mhils
cargo update (mitmproxy#248)
c419275
@mhils
suppress build warnings in dependency
0d3fcae
@mhils mhils force-pushed the protobuf-reencode branch 3 times, most recently from d949557 to 1fd1195 Compare April 5, 2025 17:12
@mhils mhils force-pushed the protobuf-reencode branch from bd5d2cd to 8f547fb Compare April 5, 2025 17:47
@mhils mhils force-pushed the protobuf-reencode branch from 55e1759 to 495f254 Compare April 8, 2025 07:22
@mhils mhils force-pushed the protobuf-reencode branch from 495f254 to 2d21be3 Compare April 8, 2025 07:35
@mhils mhils force-pushed the protobuf-reencode branch from 630968c to 073f3bb Compare April 8, 2025 08:47
@mhils mhils force-pushed the protobuf-reencode branch from 6c01119 to 4a181fb Compare April 11, 2025 13:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

1 more reviewer

@emanuele-em emanuele-em emanuele-em left review comments

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@mhils @emanuele-em @errorxyz