Skip to content

Network req data export improvements #9423

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 3 commits into
base: master
Choose a base branch
from
Open

Network req data export improvements #9423

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
c995ef4
added mime checks, minor logic improvements in recording
hrajwade96 Apr 12, 2025
f8f0a67
mime fixes
hrajwade96 May 24, 2025
d3eb915
added new helper functions, moved one function
hrajwade96 Sep 3, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 10 additions & 1 deletion packages/devtools_app/lib/src/screens/network/har_data_entry.dart
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -119,6 +119,9 @@ class HarDataEntry {
};
}).toList();

final isBinary = !isTextMimeType(e.type);
final responseBodyBytes = e.encodedResponse;

return <String, Object?>{
NetworkEventKeys.startedDateTime.name: e.startTimestamp
.toUtc()
Expand Down Expand Up @@ -153,8 +156,14 @@ class HarDataEntry {
NetworkEventKeys.content.name: <String, Object?>{
NetworkEventKeys.size.name: e.responseBody?.length,
NetworkEventKeys.mimeType.name: e.type,
NetworkEventKeys.text.name: e.responseBody,
if (responseBodyBytes != null && isBinary) ...{
NetworkEventKeys.text.name: base64.encode(responseBodyBytes),
'encoding': 'base64',
} else if (e.responseBody != null) ...{
NetworkEventKeys.text.name: e.responseBody,
},
},

NetworkEventKeys.redirectURL.name: '',
NetworkEventKeys.headersSize.name: calculateHeadersSize(
e.responseHeaders,
Expand Down
10 changes: 6 additions & 4 deletions packages/devtools_app/lib/src/screens/network/network_controller.dart
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -78,9 +78,8 @@ class NetworkController extends DevToolsScreenController
debugPrint('No valid request data to export');
return '';
}

// Build the HAR object
try {
// Build the HAR object
final har = HarNetworkData(_httpRequests!);
return ExportController().downloadFile(
json.encode(har.toJson()),
Expand Down Expand Up @@ -206,8 +205,11 @@ class NetworkController extends DevToolsScreenController
shouldLoad: (data) => !data.isEmpty,
loadData: (data) => loadOfflineData(data),
);
}
if (serviceConnection.serviceManager.connectedState.value.connected) {
} else if (serviceConnection
.serviceManager
.connectedState
.value
.connected) {
await startRecording();
}
}
Expand Down
35 changes: 33 additions & 2 deletions packages/devtools_app/lib/src/shared/http/http_request_data.dart
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -101,15 +101,46 @@ class DartIOHttpRequestData extends NetworkRequest {
);
_request = updated;
final fullRequest = _request as HttpProfileRequest;
_responseBody = utf8.decode(fullRequest.responseBody!);
_requestBody = utf8.decode(fullRequest.requestBody!);
var responseMime =
responseHeaders?['content-type']?.toString().split(';').first;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we pull this into a helper function (e.g. getHeadersMimeType)? That way we can use it both here and below. Please add it to network/utils/http_utils.dart and add tests. Thanks!

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added a helper function, will be adding the tests soon.

final requestMime =
requestHeaders?['content-type']?.toString().split(';').first;

if (fullRequest.responseBody != null) {
responseMime = normalizeContentType(responseHeaders?['content-type']);

if (isTextMimeType(responseMime)) {
_responseBody = utf8.decode(fullRequest.responseBody!);
} else {
_responseBody = base64.encode(fullRequest.responseBody!);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this encode and not decode?

Copy link
Contributor Author

@hrajwade96 hrajwade96 Sep 7, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For text-based data like json, HTML, or XML, I've used utf8.decode since it can be directly represented and rendered without additional serialization.

For binary data such as Pdf, Pngs, or other non-text formats, I've used Base64 encode on the raw bytes so they can be safely included in the JSON-based HAR export.

When importing the HAR into a tool e.g. Chrome DevTools, the tool will automatically decode this Base64 back into its original binary form (guided by the "encoding": "base64" field present in the HAR).

}
}

if (fullRequest.requestBody != null) {
if (isTextMimeType(requestMime)) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Similarly to comment above, create helper function to use here and above.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added a helper function, will be adding the tests soon.

_requestBody = utf8.decode(fullRequest.requestBody!);
} else {
_requestBody = base64.encode(fullRequest.requestBody!);
}
}

notifyListeners();
}
} finally {
isFetchingFullData = false;
}
}

//TODO check if all cases are handled correctly
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please address the TODO, thanks!

String? normalizeContentType(dynamic header) {
if (header is List && header.isNotEmpty) {
return header.first.toString().split(';').first.trim().toLowerCase();
} else if (header is String) {
return header.split(';').first.trim().toLowerCase();
}
return null;
}

static List<Cookie> _parseCookies(List<String>? cookies) {
if (cookies == null) return [];
return cookies.map((cookie) => Cookie.fromSetCookieValue(cookie)).toList();
Expand Down
18 changes: 18 additions & 0 deletions packages/devtools_app/lib/src/shared/primitives/utils.dart
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1143,3 +1143,21 @@ String devtoolsAssetsBasePath({required String origin, required String path}) {
pathParts.removeLast();
return '$trimmedOrigin${pathParts.join(separator)}';
}

/// Returns `true` if the given [mimeType] is considered textual and can be
/// safely decoded as UTF-8 without base64 encoding.
///
/// This function is useful for determining whether the content of an HTTP
/// request or response can be directly included in a HAR or JSON file as
/// human-readable text.
bool isTextMimeType(String? mimeType) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this should belong in network/utils/http_utils.dart. Please also add tests. Thank you!

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

moved the helper function, will be adding the tests soon.

if (mimeType == null) return false;

// Strip charset if present
final cleanedMime = mimeType.split(';').first.trim().toLowerCase();

return cleanedMime.startsWith('text/') ||
cleanedMime == 'application/json' ||
cleanedMime == 'application/javascript' ||
cleanedMime == 'application/xml';
}
Loading