Skip to content

Commit c58178e

Browse files
committed
Edits after the review
- Tried to make it more informative - Added more links and code snippets
1 parent 9deb666 commit c58178e

File tree

1 file changed

+160
-37
lines changed

1 file changed

+160
-37
lines changed

doc/dev_guide/hurl_attempt.md

Lines changed: 160 additions & 37 deletions
Original file line numberDiff line numberDiff line change
@@ -1,67 +1,190 @@
1-
### Feature Implementation Writeup: HurlParser Wrapper for API Dash
1+
# Flutter Rust Bridge Experiment for Parsing Hurl
22

3-
I initially started with the approach of writing my own parser using `petitparser` and creating a custom implementation. However, after about a week, the maintainer recommended against this approach. They highlighted that building a custom parser would lack future-proofing if `hurl.dev` were to implement changes. While I abandoned this route, the code is still available in a branch. My custom parser had progressed to the point where requests and assertions were working, though it was far from complete.
3+
## Background and Initial Approach
44

5-
#### Transition to Hurl Parser Wrapper Using Flutter Rust Bridge
5+
When I first tackled this project, I started by writing a custom parser using `petitparser`. After spending about a week on this approach, I received valuable feedback from the maintainer who advised against it. Their main concern was future compatibility - if `hurl.dev` were to implement changes, my custom parser would quickly become outdated. While I ultimately moved away from this approach, the code remains available in a branch, where I had successfully implemented request and assertion parsing, though it wasn't complete.
66

7-
Following the maintainer's suggestion, I began developing a wrapper for the Hurl parser using `flutter_rust_bridge`. Since the documentation for the library can be sparse, I’ve documented my steps for clarity:
7+
## Understanding Flutter Rust Bridge
88

9-
1. **Creating the Library:**
9+
### What is Flutter Rust Bridge?
1010

11-
- I ran the command:
12-
`flutter_rust_bridge_codegen create hurl --template plugin`
13-
- This initialized the project.
11+
Flutter Rust Bridge serves as a powerful tool for developing Flutter plugins using Rust. It automatically generates all the necessary code for both Flutter and Rust, creating a seamless bridge between the two languages. This approach is particularly valuable when you want to harness Rust's performance and safety features within Flutter projects.
1412

15-
2. **Adding the Parse Function:**
13+
The official documentation highlights three key features:
14+
1. Write standard Rust code, including complex features like arbitrary types, closures, mutable references, async functions, and traits
15+
2. Call Rust code from Flutter as if it were native Flutter code
16+
3. Automatic generation of all intermediate code required for communication
1617

17-
- I added the library code and the `parse_hurl` function in `rust/src/api/simple.rs`.
18+
### Why Choose Flutter Rust Bridge?
1819

19-
3. **Generating the Flutter Side Code:**
20+
Two main factors drove my decision to use Flutter Rust Bridge:
2021

21-
- I ran:
22-
`flutter_rust_bridge_codegen generate`
23-
- This generated all the necessary code for Flutter and all the targeted platforms.
22+
1. **Automatic Updates with Upstream Changes**: By using the official Hurl Rust library, any changes made to the `hurl.dev` parser would automatically propagate to our plugin. This ensures long-term maintainability and compatibility.
2423

25-
`flutter_rust_bridge` uses a tool called `cargokit` to manage dependencies, acting as a glue layer between Flutter and Rust. Unfortunately, `cargokit` is still experimental, with little documentation available. The [blog post by Matej Knopp](https://matejknopp.com/post/flutter_plugin_in_rust_with_no_prebuilt_binaries/) provided valuable insights. One crucial takeaway was avoiding the Homebrew installation of Rust and instead using `rustup`. This resolved platform compilation issues for me, though the project is still not compiling entirely.
24+
2. **Performance Benefits**: Rust's renowned performance characteristics and safety guarantees make it ideal for parsing large Hurl files efficiently.
2625

27-
#### Implementing the Wrapper
26+
## Implementation Process
2827

29-
I postponed multi-platform compilation issues to focus on writing the wrapper code. By reviewing the `hurl.dev` documentation and examples, I identified the required structure:
28+
### Setting Up the Flutter Rust Bridge Plugin
3029

31-
- `HurlFile` → List of `Entries`
32-
- `Entries``Request` and `Response`
30+
While there are several examples of Flutter Rust Bridge usage available, such as [flutter smooth](https://github.com/fzyzcjy/flutter_smooth) and the [official documentation demo](https://cjycode.com/flutter_rust_bridge/demo/), I found that plugin-specific examples were scarce. Here's a detailed breakdown of my setup process:
3331

34-
Based on this, I created models in Dart using the `freezed` package. The goal was to convert the JSON output from the Hurl parser into Dart data models. I am confident this part turned out well.
32+
1. **Project Initialization**
33+
Instead of using the standard Flutter plugin template (`flutter create --template=plugin hurl`), Flutter Rust Bridge provides its own initialization command. From the apidash/packages directory, I ran:
34+
```bash
35+
flutter_rust_bridge_codegen create hurl --template plugin
36+
```
3537

36-
After writing tests for Hurl parsing and model generation, I shifted focus to building the project.
38+
This command generated three main directories:
3739

38-
#### Compilation Challenges
40+
1. `cargokit/`: Contains build infrastructure
41+
- build_tool/: Build system utilities
42+
- cmake/: CMake configuration files
43+
- Pod creation scripts for iOS
44+
- Build scripts for various platforms
45+
- Gradle configuration for Android
3946

40-
During compilation, I encountered an issue, which I reported on the [Hurl repository](https://github.com/Orange-OpenSource/hurl/issues/3603). The Hurl parser depends on the `libxml2` crate for XML parsing. To resolve this, I had to add:
47+
2. `rust/`: The Rust project structure
48+
- src/
49+
- api/: Contains the core Rust implementation
50+
- simple.rs: Main implementation file
51+
- mod.rs: Module definitions
52+
- frbgenerated.rs: Auto-generated bridge code
53+
- lib.rs: Library entry point
54+
- cargo.toml: Rust dependencies and configuration
55+
- .gitignore
4156

42-
`OTHER_LDFLAGS => -force_load ${BUILT_PRODUCTS_DIR}/libhurl.a -lxml2`
57+
3. `lib/`: Flutter code
58+
- rust/: Generated Rust bindings
59+
- hurl.dart: Package export file
4360

44-
This fixed the macOS build.
61+
### Implementation Details
4562

46-
However, testing on iOS and Android proved problematic, as I didn’t have access to Windows or Linux systems. For iOS, adding similar flags didn’t work. I tried various fixes, such as modifying the header search paths, linking frameworks, and changing build phases in Xcode, but none succeeded.
63+
The core parsing functionality was implemented in `rust/src/api/simple.rs`:
4764

48-
#### Investigation into `libxml2`
65+
```rust
66+
/// Parses a Hurl file content and returns its JSON representation
67+
#[flutter_rust_bridge::frb(sync)]
68+
pub fn parse_hurl_to_json(content: String) -> Result<String, String> {
69+
match parse_hurl_file(&content) {
70+
Ok(hurl_file) => {
71+
// Convert to JSON using hurlfmt's format_json
72+
Ok(format_json(&hurl_file))
73+
}
74+
Err(errors) => {
75+
let mut error_msg = String::new();
76+
write!(error_msg, "Failed to parse Hurl file: {:?}", errors).unwrap();
77+
Err(error_msg)
78+
}
79+
}
80+
}
81+
```
4982

50-
I learned that `libxml2` is not a system library but a crate that wraps platform-specific libraries. Unfortunately, `libxml2` does not support Android or iOS, causing persistent errors. After consulting the maintainers of Hurl and `flutter_rust_bridge`, I received the following suggestion from the Hurl maintainer:
83+
For dependencies, I added hurlfmt to `cargo.toml`:
84+
85+
```toml
86+
[package]
87+
name = "hurl"
88+
version = "0.1.0"
89+
edition = "2021"
90+
91+
[lib]
92+
crate-type = ["cdylib", "staticlib"]
93+
94+
[dependencies]
95+
flutter_rust_bridge = "=2.7.0"
96+
hurlfmt = "6.0.0"
97+
hurl_core = "6.0.0"
98+
```
99+
100+
### Dart Model Implementation
101+
102+
Based on the Hurl documentation, I identified the core data structures:
103+
- HurlFile: Contains a list of Entries
104+
- Entries: Composed of Request and Response objects
105+
106+
I implemented these models using the `freezed` package in Dart, creating a clean mapping between the JSON output from the Hurl parser and our Dart data structures. This part of the implementation proved particularly successful and robust.
107+
108+
## Compilation Challenges and Platform Support
109+
110+
### Build Process Overview
111+
112+
The compilation process varies by platform, but generally follows these steps:
113+
114+
1. For local testing in the `hurl/tests` directory, building the Rust component requires:
115+
```bash
116+
cd apidash/packages/hurl/rust
117+
cargo build --release
118+
```
119+
120+
2. For actual deployment, cargokit manages the platform-specific build process during Flutter's standard build phase. The process involves:
121+
- On iOS: Generating and building Pod files
122+
- On Android: Creating platform-specific binaries
123+
- On macOS/Linux: Building dynamic libraries
124+
- On Windows: Generating appropriate DLLs
125+
126+
### Understanding Flutter Rust Bridge on Mobile Platforms
127+
128+
Flutter Rust Bridge generally works seamlessly across platforms as long as the Rust dependencies don't rely on platform-specific implementations. The bridge itself handles the complexity of cross-platform compilation through cargokit, which manages the build process for each target platform. However, when Rust crates depend on system libraries or have platform-specific requirements, complications can arise.
129+
130+
For iOS and Android specifically, the build process involves:
131+
1. For iOS: Compiling Rust code to a static library that gets embedded in the iOS app bundle
132+
2. For Android: Creating platform-specific .so files for each supported architecture (arm64-v8a, armeabi-v7a, x86_64)
133+
134+
The challenge comes when dependencies like `libxml2` expect certain system libraries to be available, which may not exist or behave differently on mobile platforms.
135+
136+
### Platform-Specific Challenges
137+
138+
#### macOS Implementation
139+
140+
The main challenge on macOS involved the `libxml2` dependency. I reported this issue on the [Hurl repository](https://github.com/Orange-OpenSource/hurl/issues/3603). The solution required adding specific linker flags to the `apidash/packages/hurl/macos/hurl.podspec`:
141+
142+
```ruby
143+
spec.pod_target_xcconfig = {
144+
'OTHER_LDFLAGS' => '-force_load ${BUILT_PRODUCTS_DIR}/libhurl.a -lxml2'
145+
}
146+
```
147+
148+
#### iOS and Android Challenges
149+
150+
The iOS and Android implementations faced more significant hurdles, primarily due to `libxml2` compatibility issues. Unlike macOS, these platforms don't include `libxml2` as a system library, and the Rust crate wrapper doesn't support mobile platforms.
151+
152+
The following approaches were attempted for iOS:
153+
1. Modifying header search paths
154+
2. Adding framework linkage
155+
3. Adjusting Xcode build phases
156+
4. Experimenting with different linking configurations
157+
158+
None of these attempts proved successful, leading to a deeper investigation of the `libxml2` dependency.
159+
160+
#### Maintainer Feedback and Investigation
161+
162+
After reaching out to both the Hurl and Flutter Rust Bridge maintainers, I received valuable feedback from the Hurl maintainer:
51163

52164
> _"I'm not an expert in iOS/Android, but what I would do is try to build something smaller, maybe just calling libxml through a Flutter component, then trying to build with libxml but with the Rust crate wrapper, etc."_ ([GitHub comment](https://github.com/Orange-OpenSource/hurl/issues/3603#issuecomment-2611159759))
53165
54-
#### Current Status and Potential Solutions
166+
This suggestion highlighted a crucial debugging approach: isolating the `libxml2` integration from the rest of the Hurl parsing functionality. The idea is to:
167+
168+
1. First verify that we can call `libxml2` directly from Flutter
169+
2. Then attempt to use the Rust crate wrapper
170+
3. Finally integrate it with the full Hurl parser
171+
172+
This systematic approach could help identify exactly where the build process breaks down on mobile platforms and potentially lead to a workable solution.
173+
174+
### Current Status and Potential Solutions
175+
176+
After extensive testing and consultation with both the Hurl and Flutter Rust Bridge maintainers, I've identified several potential paths forward:
177+
178+
1. **Dart Native FFIs**: This approach is currently experimental but shows promise. The Flutter Rust Bridge maintainer suggested this option in this [Github comment](https://github.com/fzyzcjy/flutter_rust_bridge/issues/2510#issuecomment-2608469143).
55179

56-
At this point, I’ve committed all my code to the repository in a fork. Here are some potential solutions I believe might work:
180+
2. **Legacy Hurlfmt Version**: We could use a version of the library from before the `libxml` transition. This would avoid the platform compatibility issues but might miss out on recent performance improvements.
57181

58-
1. **Use Dart Native FFIs**: Currently under an experimental flag, as suggested by the `flutter_rust_bridge` maintainer.
59-
2. **Revert to an Older Version of Hurlfmt**: Use a version of the library from before it transitioned to `libxml` for performance improvements.
60-
3. **Revisit My Custom Parser**: Complete the parser I started building with `petitparser`.
182+
3. **Custom Parser Completion**: Returning to and completing the `petitparser`-based implementation could provide a more controlled, albeit maintenance-heavy solution.
61183

62-
If anyone knows a solution to these challenges or has suggestions, I’d appreciate the help!
184+
## Code Availability
63185

64-
Current Branches in my fork:
186+
The current implementation is available in two branches of my fork:
187+
- [Hurl Parser with flutter_rust_bridge](https://github.com/WrathOP/apidash/tree/hurl-parser-rust)
188+
- [Hurl Parser with petiteparser](https://github.com/WrathOP/apidash/tree/hurl-parser-added)
65189

66-
- [Hurl Parser with flutter_rust_bridge](https://github.com/WrathOP/apidash/tree/hurl-parser-rust)
67-
- [Hurl Parser with petiteparser](https://github.com/WrathOP/apidash/tree/hurl-parser-added)
190+
Feel free to contribute ideas or suggestions for addressing these challenges!

0 commit comments

Comments
 (0)