|
1 |
| -### Feature Implementation Writeup: HurlParser Wrapper for API Dash |
| 1 | +# Flutter Rust Bridge Experiment for Parsing Hurl |
2 | 2 |
|
3 |
| -I initially started with the approach of writing my own parser using `petitparser` and creating a custom implementation. However, after about a week, the maintainer recommended against this approach. They highlighted that building a custom parser would lack future-proofing if `hurl.dev` were to implement changes. While I abandoned this route, the code is still available in a branch. My custom parser had progressed to the point where requests and assertions were working, though it was far from complete. |
| 3 | +## Background and Initial Approach |
4 | 4 |
|
5 |
| -#### Transition to Hurl Parser Wrapper Using Flutter Rust Bridge |
| 5 | +When I first tackled this project, I started by writing a custom parser using `petitparser`. After spending about a week on this approach, I received valuable feedback from the maintainer who advised against it. Their main concern was future compatibility - if `hurl.dev` were to implement changes, my custom parser would quickly become outdated. While I ultimately moved away from this approach, the code remains available in a branch, where I had successfully implemented request and assertion parsing, though it wasn't complete. |
6 | 6 |
|
7 |
| -Following the maintainer's suggestion, I began developing a wrapper for the Hurl parser using `flutter_rust_bridge`. Since the documentation for the library can be sparse, I’ve documented my steps for clarity: |
| 7 | +## Understanding Flutter Rust Bridge |
8 | 8 |
|
9 |
| -1. **Creating the Library:** |
| 9 | +### What is Flutter Rust Bridge? |
10 | 10 |
|
11 |
| - - I ran the command: |
12 |
| - `flutter_rust_bridge_codegen create hurl --template plugin` |
13 |
| - - This initialized the project. |
| 11 | +Flutter Rust Bridge serves as a powerful tool for developing Flutter plugins using Rust. It automatically generates all the necessary code for both Flutter and Rust, creating a seamless bridge between the two languages. This approach is particularly valuable when you want to harness Rust's performance and safety features within Flutter projects. |
14 | 12 |
|
15 |
| -2. **Adding the Parse Function:** |
| 13 | +The official documentation highlights three key features: |
| 14 | +1. Write standard Rust code, including complex features like arbitrary types, closures, mutable references, async functions, and traits |
| 15 | +2. Call Rust code from Flutter as if it were native Flutter code |
| 16 | +3. Automatic generation of all intermediate code required for communication |
16 | 17 |
|
17 |
| - - I added the library code and the `parse_hurl` function in `rust/src/api/simple.rs`. |
| 18 | +### Why Choose Flutter Rust Bridge? |
18 | 19 |
|
19 |
| -3. **Generating the Flutter Side Code:** |
| 20 | +Two main factors drove my decision to use Flutter Rust Bridge: |
20 | 21 |
|
21 |
| - - I ran: |
22 |
| - `flutter_rust_bridge_codegen generate` |
23 |
| - - This generated all the necessary code for Flutter and all the targeted platforms. |
| 22 | +1. **Automatic Updates with Upstream Changes**: By using the official Hurl Rust library, any changes made to the `hurl.dev` parser would automatically propagate to our plugin. This ensures long-term maintainability and compatibility. |
24 | 23 |
|
25 |
| -`flutter_rust_bridge` uses a tool called `cargokit` to manage dependencies, acting as a glue layer between Flutter and Rust. Unfortunately, `cargokit` is still experimental, with little documentation available. The [blog post by Matej Knopp](https://matejknopp.com/post/flutter_plugin_in_rust_with_no_prebuilt_binaries/) provided valuable insights. One crucial takeaway was avoiding the Homebrew installation of Rust and instead using `rustup`. This resolved platform compilation issues for me, though the project is still not compiling entirely. |
| 24 | +2. **Performance Benefits**: Rust's renowned performance characteristics and safety guarantees make it ideal for parsing large Hurl files efficiently. |
26 | 25 |
|
27 |
| -#### Implementing the Wrapper |
| 26 | +## Implementation Process |
28 | 27 |
|
29 |
| -I postponed multi-platform compilation issues to focus on writing the wrapper code. By reviewing the `hurl.dev` documentation and examples, I identified the required structure: |
| 28 | +### Setting Up the Flutter Rust Bridge Plugin |
30 | 29 |
|
31 |
| -- `HurlFile` → List of `Entries` |
32 |
| -- `Entries` → `Request` and `Response` |
| 30 | +While there are several examples of Flutter Rust Bridge usage available, such as [flutter smooth](https://github.com/fzyzcjy/flutter_smooth) and the [official documentation demo](https://cjycode.com/flutter_rust_bridge/demo/), I found that plugin-specific examples were scarce. Here's a detailed breakdown of my setup process: |
33 | 31 |
|
34 |
| -Based on this, I created models in Dart using the `freezed` package. The goal was to convert the JSON output from the Hurl parser into Dart data models. I am confident this part turned out well. |
| 32 | +1. **Project Initialization** |
| 33 | +Instead of using the standard Flutter plugin template (`flutter create --template=plugin hurl`), Flutter Rust Bridge provides its own initialization command. From the apidash/packages directory, I ran: |
| 34 | +```bash |
| 35 | +flutter_rust_bridge_codegen create hurl --template plugin |
| 36 | +``` |
35 | 37 |
|
36 |
| -After writing tests for Hurl parsing and model generation, I shifted focus to building the project. |
| 38 | +This command generated three main directories: |
37 | 39 |
|
38 |
| -#### Compilation Challenges |
| 40 | +1. `cargokit/`: Contains build infrastructure |
| 41 | + - build_tool/: Build system utilities |
| 42 | + - cmake/: CMake configuration files |
| 43 | + - Pod creation scripts for iOS |
| 44 | + - Build scripts for various platforms |
| 45 | + - Gradle configuration for Android |
39 | 46 |
|
40 |
| -During compilation, I encountered an issue, which I reported on the [Hurl repository](https://github.com/Orange-OpenSource/hurl/issues/3603). The Hurl parser depends on the `libxml2` crate for XML parsing. To resolve this, I had to add: |
| 47 | +2. `rust/`: The Rust project structure |
| 48 | + - src/ |
| 49 | + - api/: Contains the core Rust implementation |
| 50 | + - simple.rs: Main implementation file |
| 51 | + - mod.rs: Module definitions |
| 52 | + - frbgenerated.rs: Auto-generated bridge code |
| 53 | + - lib.rs: Library entry point |
| 54 | + - cargo.toml: Rust dependencies and configuration |
| 55 | + - .gitignore |
41 | 56 |
|
42 |
| -`OTHER_LDFLAGS => -force_load ${BUILT_PRODUCTS_DIR}/libhurl.a -lxml2` |
| 57 | +3. `lib/`: Flutter code |
| 58 | + - rust/: Generated Rust bindings |
| 59 | + - hurl.dart: Package export file |
43 | 60 |
|
44 |
| -This fixed the macOS build. |
| 61 | +### Implementation Details |
45 | 62 |
|
46 |
| -However, testing on iOS and Android proved problematic, as I didn’t have access to Windows or Linux systems. For iOS, adding similar flags didn’t work. I tried various fixes, such as modifying the header search paths, linking frameworks, and changing build phases in Xcode, but none succeeded. |
| 63 | +The core parsing functionality was implemented in `rust/src/api/simple.rs`: |
47 | 64 |
|
48 |
| -#### Investigation into `libxml2` |
| 65 | +```rust |
| 66 | +/// Parses a Hurl file content and returns its JSON representation |
| 67 | +#[flutter_rust_bridge::frb(sync)] |
| 68 | +pub fn parse_hurl_to_json(content: String) -> Result<String, String> { |
| 69 | + match parse_hurl_file(&content) { |
| 70 | + Ok(hurl_file) => { |
| 71 | + // Convert to JSON using hurlfmt's format_json |
| 72 | + Ok(format_json(&hurl_file)) |
| 73 | + } |
| 74 | + Err(errors) => { |
| 75 | + let mut error_msg = String::new(); |
| 76 | + write!(error_msg, "Failed to parse Hurl file: {:?}", errors).unwrap(); |
| 77 | + Err(error_msg) |
| 78 | + } |
| 79 | + } |
| 80 | +} |
| 81 | +``` |
49 | 82 |
|
50 |
| -I learned that `libxml2` is not a system library but a crate that wraps platform-specific libraries. Unfortunately, `libxml2` does not support Android or iOS, causing persistent errors. After consulting the maintainers of Hurl and `flutter_rust_bridge`, I received the following suggestion from the Hurl maintainer: |
| 83 | +For dependencies, I added hurlfmt to `cargo.toml`: |
| 84 | + |
| 85 | +```toml |
| 86 | +[package] |
| 87 | +name = "hurl" |
| 88 | +version = "0.1.0" |
| 89 | +edition = "2021" |
| 90 | + |
| 91 | +[lib] |
| 92 | +crate-type = ["cdylib", "staticlib"] |
| 93 | + |
| 94 | +[dependencies] |
| 95 | +flutter_rust_bridge = "=2.7.0" |
| 96 | +hurlfmt = "6.0.0" |
| 97 | +hurl_core = "6.0.0" |
| 98 | +``` |
| 99 | + |
| 100 | +### Dart Model Implementation |
| 101 | + |
| 102 | +Based on the Hurl documentation, I identified the core data structures: |
| 103 | +- HurlFile: Contains a list of Entries |
| 104 | +- Entries: Composed of Request and Response objects |
| 105 | + |
| 106 | +I implemented these models using the `freezed` package in Dart, creating a clean mapping between the JSON output from the Hurl parser and our Dart data structures. This part of the implementation proved particularly successful and robust. |
| 107 | + |
| 108 | +## Compilation Challenges and Platform Support |
| 109 | + |
| 110 | +### Build Process Overview |
| 111 | + |
| 112 | +The compilation process varies by platform, but generally follows these steps: |
| 113 | + |
| 114 | +1. For local testing in the `hurl/tests` directory, building the Rust component requires: |
| 115 | +```bash |
| 116 | +cd apidash/packages/hurl/rust |
| 117 | +cargo build --release |
| 118 | +``` |
| 119 | + |
| 120 | +2. For actual deployment, cargokit manages the platform-specific build process during Flutter's standard build phase. The process involves: |
| 121 | + - On iOS: Generating and building Pod files |
| 122 | + - On Android: Creating platform-specific binaries |
| 123 | + - On macOS/Linux: Building dynamic libraries |
| 124 | + - On Windows: Generating appropriate DLLs |
| 125 | + |
| 126 | +### Understanding Flutter Rust Bridge on Mobile Platforms |
| 127 | + |
| 128 | +Flutter Rust Bridge generally works seamlessly across platforms as long as the Rust dependencies don't rely on platform-specific implementations. The bridge itself handles the complexity of cross-platform compilation through cargokit, which manages the build process for each target platform. However, when Rust crates depend on system libraries or have platform-specific requirements, complications can arise. |
| 129 | + |
| 130 | +For iOS and Android specifically, the build process involves: |
| 131 | +1. For iOS: Compiling Rust code to a static library that gets embedded in the iOS app bundle |
| 132 | +2. For Android: Creating platform-specific .so files for each supported architecture (arm64-v8a, armeabi-v7a, x86_64) |
| 133 | + |
| 134 | +The challenge comes when dependencies like `libxml2` expect certain system libraries to be available, which may not exist or behave differently on mobile platforms. |
| 135 | + |
| 136 | +### Platform-Specific Challenges |
| 137 | + |
| 138 | +#### macOS Implementation |
| 139 | + |
| 140 | +The main challenge on macOS involved the `libxml2` dependency. I reported this issue on the [Hurl repository](https://github.com/Orange-OpenSource/hurl/issues/3603). The solution required adding specific linker flags to the `apidash/packages/hurl/macos/hurl.podspec`: |
| 141 | + |
| 142 | +```ruby |
| 143 | +spec.pod_target_xcconfig = { |
| 144 | + 'OTHER_LDFLAGS' => '-force_load ${BUILT_PRODUCTS_DIR}/libhurl.a -lxml2' |
| 145 | +} |
| 146 | +``` |
| 147 | + |
| 148 | +#### iOS and Android Challenges |
| 149 | + |
| 150 | +The iOS and Android implementations faced more significant hurdles, primarily due to `libxml2` compatibility issues. Unlike macOS, these platforms don't include `libxml2` as a system library, and the Rust crate wrapper doesn't support mobile platforms. |
| 151 | + |
| 152 | +The following approaches were attempted for iOS: |
| 153 | +1. Modifying header search paths |
| 154 | +2. Adding framework linkage |
| 155 | +3. Adjusting Xcode build phases |
| 156 | +4. Experimenting with different linking configurations |
| 157 | + |
| 158 | +None of these attempts proved successful, leading to a deeper investigation of the `libxml2` dependency. |
| 159 | + |
| 160 | +#### Maintainer Feedback and Investigation |
| 161 | + |
| 162 | +After reaching out to both the Hurl and Flutter Rust Bridge maintainers, I received valuable feedback from the Hurl maintainer: |
51 | 163 |
|
52 | 164 | > _"I'm not an expert in iOS/Android, but what I would do is try to build something smaller, maybe just calling libxml through a Flutter component, then trying to build with libxml but with the Rust crate wrapper, etc."_ ([GitHub comment](https://github.com/Orange-OpenSource/hurl/issues/3603#issuecomment-2611159759))
|
53 | 165 |
|
54 |
| -#### Current Status and Potential Solutions |
| 166 | +This suggestion highlighted a crucial debugging approach: isolating the `libxml2` integration from the rest of the Hurl parsing functionality. The idea is to: |
| 167 | + |
| 168 | +1. First verify that we can call `libxml2` directly from Flutter |
| 169 | +2. Then attempt to use the Rust crate wrapper |
| 170 | +3. Finally integrate it with the full Hurl parser |
| 171 | + |
| 172 | +This systematic approach could help identify exactly where the build process breaks down on mobile platforms and potentially lead to a workable solution. |
| 173 | + |
| 174 | +### Current Status and Potential Solutions |
| 175 | + |
| 176 | +After extensive testing and consultation with both the Hurl and Flutter Rust Bridge maintainers, I've identified several potential paths forward: |
| 177 | + |
| 178 | +1. **Dart Native FFIs**: This approach is currently experimental but shows promise. The Flutter Rust Bridge maintainer suggested this option in this [Github comment](https://github.com/fzyzcjy/flutter_rust_bridge/issues/2510#issuecomment-2608469143). |
55 | 179 |
|
56 |
| -At this point, I’ve committed all my code to the repository in a fork. Here are some potential solutions I believe might work: |
| 180 | +2. **Legacy Hurlfmt Version**: We could use a version of the library from before the `libxml` transition. This would avoid the platform compatibility issues but might miss out on recent performance improvements. |
57 | 181 |
|
58 |
| -1. **Use Dart Native FFIs**: Currently under an experimental flag, as suggested by the `flutter_rust_bridge` maintainer. |
59 |
| -2. **Revert to an Older Version of Hurlfmt**: Use a version of the library from before it transitioned to `libxml` for performance improvements. |
60 |
| -3. **Revisit My Custom Parser**: Complete the parser I started building with `petitparser`. |
| 182 | +3. **Custom Parser Completion**: Returning to and completing the `petitparser`-based implementation could provide a more controlled, albeit maintenance-heavy solution. |
61 | 183 |
|
62 |
| -If anyone knows a solution to these challenges or has suggestions, I’d appreciate the help! |
| 184 | +## Code Availability |
63 | 185 |
|
64 |
| -Current Branches in my fork: |
| 186 | +The current implementation is available in two branches of my fork: |
| 187 | +- [Hurl Parser with flutter_rust_bridge](https://github.com/WrathOP/apidash/tree/hurl-parser-rust) |
| 188 | +- [Hurl Parser with petiteparser](https://github.com/WrathOP/apidash/tree/hurl-parser-added) |
65 | 189 |
|
66 |
| -- [Hurl Parser with flutter_rust_bridge](https://github.com/WrathOP/apidash/tree/hurl-parser-rust) |
67 |
| -- [Hurl Parser with petiteparser](https://github.com/WrathOP/apidash/tree/hurl-parser-added) |
| 190 | +Feel free to contribute ideas or suggestions for addressing these challenges! |
0 commit comments