Skip to content

wirego_remote(rust): add support for Wirego implemented in Rust #37

Merged
NothNoth merged 19 commits intoquarkslab:mainfrom
whiskeyo:dev/initial-rust
May 27, 2025
Merged

wirego_remote(rust): add support for Wirego implemented in Rust #37
NothNoth merged 19 commits intoquarkslab:mainfrom
whiskeyo:dev/initial-rust

Conversation

@whiskeyo
Copy link
Contributor

@whiskeyo whiskeyo commented May 14, 2025

This MR is still WIP, but I am actively working to finish the implementation. Currently the minimal example based on the Python one, using the PCAP available in the Python example, shows the same results (at least I hope it does, the screenshot below shows it):

image

TODOs:

  • DONE add better error handling, so parsing failure does not crash the lib (if you could tell me which commands are "must haves" and should crash, and which of them can fail gracefully, it'll be great)
  • NOT VALID add the option to disable cache (but why would we want it?)
  • CAN BE DONE LATER add better logging with the option to enable/disable them
  • CAN BE DONE LATER / NOT VALID add unbinding the zeromq::RepSocket after the main loop is finished
  • CAN BE DONE LATER / NOT VALID add signal handling (it can probably wait, but you can tell how crucial it is)
  • introduce some CI for: building the library, running tests, and releasing the package
  • cleanup Cargo.toml from unused libraries

Limitations:

  • Sadly zeromq does not yet support the UDP, thus the library will not have the same functionality as you've implemented in Go or Python, but in case it is added in future - we shouldn't need lots of changes.

Once the MR is ready, I'll remove the [DRAFT] from its name :) I just need a bit of help from your side to fully understand how the Wirego Remote should work.

@whiskeyo whiskeyo marked this pull request as draft May 14, 2025 06:42
@NothNoth
Copy link
Collaborator

Hi !

Wow, that looks really great!

To anwser your questions:

  • Error handling and crash failures: the wirego plugin (on the Wireshark side) is pretty tolerant to errors. If at some point the plugin does not respond corretly or returns an error at the end of the day Wireshark will just.. show nothing. Thus a good and simple approche is just to return an error through the ZMQ tunnel and nothing else is something goes wrong.
  • Cache enable/disable : in some use cases you may want to disable the cache. Sometimes being able to parse a whole session multiple times can be useful, this allows you to update a packet descriptor based on what you see later on the session.
  • ZMQ/UDP: I don't think that a big issue, most people will use IPC when running local and TCP when running on a remote device anyway!

@NothNoth
Copy link
Collaborator

Feel free to ask any other questions and thanks again for your great work!

@whiskeyo
Copy link
Contributor Author

Thank you! I have some questions about your answer:

  • Error handling and crash failures: the wirego plugin (on the Wireshark side) is pretty tolerant to errors. If at some point the plugin does not respond corretly or returns an error at the end of the day Wireshark will just.. show nothing. Thus a good and simple approche is just to return an error through the ZMQ tunnel and nothing else is something goes wrong.

So sending back the message with a single frame "\x00" should be the right thing to do in every case, right?

  • Cache enable/disable : in some use cases you may want to disable the cache. Sometimes being able to parse a whole session multiple times can be useful, this allows you to update a packet descriptor based on what you see later on the session.

So it's allowing Wireshark to perform two pass (or multipass) run while dissecting packets?

  • ZMQ/UDP: I don't think that a big issue, most people will use IPC when running local and TCP when running on a remote device anyway!

That's great then :D

Now a few other questions from me, because I am curious about it:

  1. Should there be some CI job for building and testing the wirego remotes? Examples could be also included, and since tshark is already built, we could also have some regression jobs to verify that examples are working fine.
  2. Do you consider releasing the Wirego Remote in Python to PyPi and once the rust implementation is done - to cargo?
  3. When creating the instance of the WiregoListener, can the wirego_field_id be anything, or it has to be incrementally raising?
  4. Is there a chance of calling some internal Wireshark dissector from the plugin? I mean the embedded protocol inside another one, so I am implementing the foo dissector and I want to call the bar dissector that is already implemented in Wireshark.

@NothNoth
Copy link
Collaborator

Thank you! I have some questions about your answer:

  • Error handling and crash failures: the wirego plugin (on the Wireshark side) is pretty tolerant to errors. If at some point the plugin does not respond corretly or returns an error at the end of the day Wireshark will just.. show nothing. Thus a good and simple approche is just to return an error through the ZMQ tunnel and nothing else is something goes wrong.

So sending back the message with a single frame "\x00" should be the right thing to do in every case, right?

Absolutely, just reply with a 0x00 !

  • Cache enable/disable : in some use cases you may want to disable the cache. Sometimes being able to parse a whole session multiple times can be useful, this allows you to update a packet descriptor based on what you see later on the session.

So it's allowing Wireshark to perform two pass (or multipass) run while dissecting packets?

Actually Wireshark will always perform a two pass when loading a pcap. When the pcap is loaded, all packets are processed. When you scroll on the Window, all displayed packets are re-processed so that the plugin may eventually update the results based on the complete context of the capture.
The cache mechanism is really specific to Wirego and need to be implemented on the package side.
You can take a look on the Go/Python packages to see how it's done, but basically:

  • During the process_dissect_packet, fill a map with thje packet number as key, and the dissection results as a value
  • You can use this map when you are requested for the complete dissection results (result_get_protocol, etc.)
  • If the cache is enabled, during process_dissect_packet check first if the packets has already been processed in the cache map
  • ZMQ/UDP: I don't think that a big issue, most people will use IPC when running local and TCP when running on a remote device anyway!

That's great then :D

Now a few other questions from me, because I am curious about it:

1. Should there be some CI job for building and testing the wirego remotes? Examples could be also included, and since tshark is already built, we could also have some regression jobs to verify that examples are working fine.

Currently there's none, but that would be cool yes!

2. Do you consider releasing the Wirego Remote in Python to PyPi and once the rust implementation is done - to cargo?

Absolutely, when it will be a bit more "proof-read"

3. When creating the instance of the `WiregoListener`, can the `wirego_field_id` be anything, or it has to be incrementally raising?

It can be anything, as long as there's no duplicate values (which would be reject by wirego)

4. Is there a chance of calling some internal Wireshark dissector from the plugin? I mean the embedded protocol inside another one, so I am implementing the `foo` dissector and I want to call the `bar` dissector that is already implemented in Wireshark.

Mmhh that's a good question, I'm not really sure. If I set a "protocol" value to HTTP for example, will it trigger the HTTP dissector?

@whiskeyo
Copy link
Contributor Author

@NothNoth thank you again for answers! I have again some questions, answers, and ideas here, so here is a not-very-short message from me 😸

Part I - changes

I have added a few tests and created a simple CI pipeline for Rust, so the package is now verified 😄

  1. The first thing that I want to mention is the cache - are you fine with leaving the "cache-enabled" version for now (since the MR is way too big already) and let me implement it in a near future?
  2. Similar to the point above - are you fine with leaving some "debug logs" enabled by default in the current code?

Part II - embedded protocols and calling external dissectors

Now, moving to my question and your question:

Is there a chance of calling some internal Wireshark dissector from the plugin? I mean the embedded protocol inside another one, so I am implementing the foo dissector and I want to call the bar dissector that is already implemented in Wireshark.
Mmhh that's a good question, I'm not really sure. If I set a "protocol" value to HTTP for example, will it trigger the HTTP dissector?

I'll use a few examples from the Wireshark to show what I mean, that will be way easier to implement.

"Interaction" of eCPRI and ORAN FH-CUS

The WG4: Open Fronthaul Interfaces Workgroup specification of ORAN FH-CUS, more specifically O-RAN Control, User and Synchronization Plane Specification 17.01 (you can find the pdf on the page) says that eCPRI messages of type 0 (IQ Data) and 2 (Real-Time Control Data) can carry the payload of ORAN FH-CUS.

That means, if you have some data and you are sure that in your protocol (eCPRI) is some data from the other protocol (ORAN FH-CUS), you may call the find_dissector("some_other_dissector_name") to get the handle to that dissector. Then, having the other_dissector_handle, you may call call_dissector_only(other_dissector_handle, ...) to call the embedded dissector to parse your data.

All of that can be found in the eCPRI dissector implementation - I will not give you particular lines, since that code may change in basically any day.

The possibilty of extending dissectors

Some protocols, such as GTP (TS 29.281), have the "reserved fields" for vendor specific usage, i.e. values 1, 2, 3 are defined by the GTP, but you can do anything you want with "reserved" values 10, 11, 12.

To implement that, Wireshark lets you call the gtp_hdr_ext_dissector_table = register_dissector_table("gtp.hdr_ext", ...) function when registering the protocol, and then you may use the dissector_try_uint_with_data(gtp_hdr_ext_dissector_table, ...) function while dissecting the packet to call some external dissector (e.g. your private "gtp.something_very_mine") from the other place, if you of course match it with the "gtp.hdr_ext" value. Then, all packets are dissected by GTP first, and then by some other dissector.

As above, you may find the GTP dissector implementation here.

The question

Now, coming to the final part - is it possible to do it with the current implementation of Wirego, or it is something that is not implemented yet? Do you have plans of adding support for that? It is not something that I really need, but I am curious about it 😄

Part III - some improvements

  1. I've analysed existing CI jobs and I found that they do not use the paths in on section - that means that any change, even in doc directory, will trigger building of everything, which makes no sense. I might open some issues to track it, and I am open to fix it :)
  2. Forcing some coding guidelines. This might sound a bit "harsh" at first, but to keep the code clean, you may consider adding some tools to make the development easier: some formatters, linters, etc. so the code is always formatted in the same manner for everyone.
  3. Adding tests for Python package - another simple thing, but worth adding before getting into PyPi.

Part IV - merging changes

There are still a few TODOs in the code, but I am getting in some kind of loop to improve everything at once, which makes the MR bigger and bigger. Could you please review the change, so I can merge what we have here, and I'll proceed with further improvements later on? Thank you!

@whiskeyo whiskeyo changed the title [DRAFT] wirego_remote(rust): add support for Wirego implemented in Rust wirego_remote(rust): add support for Wirego implemented in Rust May 17, 2025
@whiskeyo whiskeyo marked this pull request as ready for review May 17, 2025 21:39
@NothNoth
Copy link
Collaborator

Hi @whiskeyo !

Here are a few answers to your questions/remarks:

Part 1 :
Sure, we can release a first version without the cache feature. It's a bonus and we can clearly do without
For the logs: can you eventually add a static flag somewhere in the code to enable/disable the logs?

Part 2:
That's a pretty cool feature. I do not plan to add it to Wirego for now. That's a tricky question actually: I want to keep wirego simple. There are many plugin features that are not available when using wirego. If I add them all, at the end of the day creating a wirego plugin will be quite complex and a bit intimidating.
My current position is to stick with the current set of features, unless there's a real need for something specific

Part 3:
That's right! Can you create some improvement issues so that we can keep track?

Part 4:
No problem, I'll take a look this week so that we can merge a first version of the rust package

Thanks again!

@NothNoth
Copy link
Collaborator

Hi @whiskeyo !

I'm currently looking for someone at Quarkslab more fluent than me in Rust for the review :)

One additional question: I'll be talking about Wirego in a few weeks at SSTIC 25 (https://www.sstic.org/2025/news/) and Pass The Salt (https://2025.pass-the-salt.org/). Do you mind if I cite your name for the Rust contribution?

@whiskeyo
Copy link
Contributor Author

Hi @NothNoth 😀

That's good to know! My knowledge is Rust isn't very big too, so getting a review from experienced Rust dev will be a pleasure to get.

When it comes to the conference and citing my name - go for it, I'm glad to be included there.


- `get_name` should return the protocol name, e.g. "eCPRI 2.0"
- `get_filter` should return the protocol filter, e.g. "ecpri"
- `get_fields` should return all fields that can be dissected by Wireshark
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should return all fields that may be returned by your Wireshark plugin

fn get_fields(&self) -> Vec<wirego::types::WiresharkField> {
vec![
wirego::types::WiresharkField {
wirego_field_id: 1,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of using "1" here (and 2,3,4...) can you define an enum instead so that you can reuse it later during dissect?

@NothNoth
Copy link
Collaborator

Hi!

I just finished the review.
There's just a few things to fix/update, nothing too serious

Thanks!

@whiskeyo
Copy link
Contributor Author

Hi @NothNoth, thank you! I'll try to do it today or tomorrow, depending on how well I feel :)

@whiskeyo
Copy link
Contributor Author

Hey @NothNoth! I believe that I fixed all issues that you've pointed out. I am not sure what way of working you prefer in case of comment reviews, thus I left all of them unresolved and in some of them I added my own comments. If you agree with these changes, LMK if I should close these threads or you'll do it :) Thank you!

Copy link
Collaborator

@NothNoth NothNoth left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perfect, thanks!

@NothNoth NothNoth merged commit 3dd2d8c into quarkslab:main May 27, 2025
1 check passed
@NothNoth
Copy link
Collaborator

That's merged!
Thanks a lot for your work, I'm super happy that we now have a Rust package for Wirego, that's a great addition!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants