Skip to content

Commit 6ca568c

Browse files
bpotchikbpotchik
committed
Add Container Browser documentation.
1 parent 7f97cc3 commit 6ca568c

File tree

5 files changed

+293
-0
lines changed

5 files changed

+293
-0
lines changed

‎docs/dev/containertransforms.md‎

Lines changed: 218 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,218 @@
1+
# Container Transforms
2+
3+
Container Transforms are specialized transforms that enable Binary Ninja to extract and navigate files within container formats such as ZIP archives, disk images, and other multi-file structures. Unlike simple encoding transforms (Base64, Hex, etc.), container transforms can produce multiple output files and interact with the Container Browser UI.
4+
5+
You can list all available container transforms (those with detection support) using:
6+
7+
```python
8+
>>> [x.name for x in Transform if getattr(x, "supports_detection", False)]
9+
['Gzip', 'Zlib', 'Zip', 'CaRT', 'IntelHex', 'SRec', 'TiTxt', 'IMG4', 'LZFSE']
10+
```
11+
12+
## Overview
13+
14+
The Transform API provides the foundation for creating custom container decoders. Container transforms differ from standard transforms in that they:
15+
16+
1. Support **context-aware decoding** via `perform_decode_with_context()`
17+
2. Can produce **multiple output files** from a single input
18+
3. Support **password protection** and other interactive parameters
19+
4. Integrate with the **Container Browser** UI for file selection
20+
21+
## Basic Transform Structure
22+
23+
All transforms, including container transforms, inherit from the `Transform` base class. Here's a minimal example:
24+
25+
```python
26+
from binaryninja import Transform, TransformType, TransformCapabilities
27+
28+
class MyContainerTransform(Transform):
29+
transform_type = TransformType.DecodeTransform
30+
capabilities = TransformCapabilities.TransformSupportsContext | TransformCapabilities.TransformSupportsDetection
31+
name = "MyContainer"
32+
long_name = "My Container Format"
33+
group = "Container"
34+
35+
def can_decode(self, input):
36+
"""Check if this transform can decode the input"""
37+
# Check for magic bytes or other signatures
38+
head = input.read(0, 4)
39+
return head == b"MYCN" # Your format's magic bytes
40+
41+
def perform_decode_with_context(self, context, params):
42+
"""Context-aware extraction for multi-file containers"""
43+
# Implementation details below
44+
pass
45+
46+
# Register the transform
47+
MyContainerTransform.register()
48+
```
49+
50+
## Container Extraction Protocol
51+
52+
Container transforms typically operate in **two phases**:
53+
54+
### Phase 1: Discovery
55+
56+
During discovery, the transform enumerates all available files and populates `context.available_files`:
57+
58+
```python
59+
def perform_decode_with_context(self, context, params):
60+
# Parse the container format
61+
container = parse_my_format(context.input)
62+
63+
# Phase 1: Discovery
64+
if not context.has_available_files:
65+
file_list = [entry.name for entry in container.entries]
66+
context.set_available_files(file_list)
67+
return False # More user interaction needed
68+
```
69+
70+
Returning `False` indicates that the Container Browser should present these files to the user for selection.
71+
72+
### Phase 2: Extraction
73+
74+
Once the user selects files, the transform extracts them and creates child contexts:
75+
76+
```python
77+
def perform_decode_with_context(self, context, params):
78+
container = parse_my_format(context.input)
79+
80+
# Phase 1: Discovery (as above)
81+
if not context.has_available_files:
82+
# ... discovery code ...
83+
return False
84+
85+
# Phase 2: Extraction
86+
requested = context.requested_files
87+
if not requested:
88+
return False # No files selected yet
89+
90+
complete = True
91+
for filename in requested:
92+
try:
93+
data = container.extract(filename)
94+
context.create_child(DataBuffer(data), filename)
95+
except Exception as e:
96+
# Create child with error status
97+
context.create_child(
98+
DataBuffer(b""),
99+
filename,
100+
result=TransformResult.TransformFailure,
101+
message=str(e)
102+
)
103+
complete = False
104+
105+
return complete # True if all files extracted successfully
106+
```
107+
108+
## Complete Example: ZipPython
109+
110+
Binary Ninja includes a reference implementation of a ZIP container transform in `api/python/transform.py`.
111+
112+
113+
114+
## Transform Results and Error Handling
115+
116+
Use `TransformResult` values to communicate extraction status:
117+
118+
- `TransformResult.TransformSuccess`: Extraction completed successfully
119+
- `TransformResult.TransformNotAttempted`: Extraction not attempted
120+
- `TransformResult.TransformFailure`: Generic extraction failure
121+
- `TransformResult.TransformRequiresPassword`: File is encrypted and needs a password
122+
123+
Set results on individual child contexts:
124+
125+
```python
126+
context.create_child(
127+
data=databuffer.DataBuffer(extracted_data),
128+
filename="file.bin",
129+
result=TransformResult.TransformSuccess,
130+
message="" # Optional success message
131+
)
132+
```
133+
134+
## Working with Passwords
135+
136+
Container transforms should integrate with Binary Ninja's password management system:
137+
138+
```python
139+
# Get passwords from settings
140+
passwords = Settings().get_string_list('files.container.defaultPasswords')
141+
142+
# Check for password in transform parameters
143+
if "password" in params:
144+
p = params["password"]
145+
pwd = p.decode("utf-8", "replace") if isinstance(p, (bytes, bytearray)) else str(p)
146+
passwords.insert(0, pwd)
147+
148+
# Try each password
149+
for password in passwords:
150+
try:
151+
content = extract_with_password(container, filename, password)
152+
break # Success!
153+
except PasswordError:
154+
continue # Try next password
155+
```
156+
157+
When a file requires a password that wasn't provided, use `TransformResult.TransformRequiresPassword` to signal the UI to prompt the user.
158+
159+
## Metadata and Virtual Paths
160+
161+
Container transforms automatically create metadata that tracks the extraction chain:
162+
163+
```python
164+
# After opening a file extracted through containers:
165+
>>> bv.parent_view.auto_metadata['container']
166+
{
167+
'chain': [
168+
{'transform': 'Zip'},
169+
{'transform': 'Base64'}
170+
],
171+
'virtualPath': 'Zip(/path/to/archive.zip)::Base64(encoded_file)::extracted'
172+
}
173+
```
174+
175+
You can also add custom metadata to child contexts:
176+
177+
```python
178+
child = context.create_child(data, filename)
179+
if child.metadata_obj:
180+
child.metadata_obj["custom_field"] = "value"
181+
```
182+
183+
## Testing Container Transforms
184+
185+
When testing your container transform, you can use the Python API directly:
186+
187+
```python
188+
from binaryninja import TransformSession
189+
190+
# Test with a file
191+
session = TransformSession("test_container.bin")
192+
193+
# Process and check results
194+
if session.process():
195+
print(f"Extraction complete: {session.current_context.filename}")
196+
else:
197+
print("User interaction required")
198+
ctx = session.current_context
199+
if ctx.parent and ctx.parent.has_available_files:
200+
print(f"Available files: {ctx.parent.available_files}")
201+
```
202+
203+
For interactive testing in the UI:
204+
205+
1. **Full Mode**: Settings → `files.container.mode` → "Full"
206+
- Opens your container and shows all extracted files immediately
207+
2. **Interactive Mode**: Settings → `files.container.mode` → "Interactive"
208+
- Requires clicking through each level of the container hierarchy
209+
210+
## API Reference
211+
212+
For complete API documentation, see:
213+
214+
- [`Transform`](https://api.binary.ninja/binaryninja.transform-module.html#binaryninja.transform.Transform) - Base transform class
215+
- [`TransformContext`](https://api.binary.ninja/binaryninja.transform-module.html#binaryninja.transform.TransformContext) - Container extraction context
216+
- [`TransformSession`](https://api.binary.ninja/binaryninja.transform-module.html#binaryninja.transform.TransformSession) - Multi-stage extraction workflow
217+
- [`TransformResult`](https://api.binary.ninja/binaryninja.enums-module.html#binaryninja.enums.TransformResult) - Extraction result codes
218+
- [`TransformCapabilities`](https://api.binary.ninja/binaryninja.enums-module.html#binaryninja.enums.TransformCapabilities) - Transform capability flags

‎docs/dev/index.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -11,6 +11,7 @@ The Binary Ninja API is available through a [Core API](#core-api), through the [
1111
The Python API is the most common third-party API and is used in many [public plugins](https://github.com/vector35/community-plugins). Here's a list of the most important Python API documentation resources:
1212

1313
- [Writing Python Plugins](plugins.md)
14+
- [Container Transforms](containertransforms.md) - Creating custom container/archive decoders
1415
- [Applying Annotations](annotation.md)
1516
- [Script Cookbook](cookbook.md) with common examples and concepts explained
1617
- [Python API Reference](https://api.binary.ninja/) (available offline via the Help menu)

‎docs/guide/index.md‎

Lines changed: 73 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -97,6 +97,79 @@ While Binary Ninja defaults to opening most files with sane defaults without pro
9797

9898
Items 1, 3, 5, 6 in the above list all describe methods you can use to override the default settings and request an "Open with Options" dialog.
9999

100+
### Container Browser
101+
102+
![container browser](../img/container-browser.png "Container Browser"){ width="800" }
103+
104+
The Container Browser provides an interactive way to explore and extract files from container formats such as ZIP archives, encrypted containers, and other nested file structures. When opening files that contain nested content, Binary Ninja can automatically detect and decode these containers, presenting a hierarchical tree view of all available files.
105+
106+
Binary Ninja includes built-in support for the following container formats:
107+
108+
- **Zip**: ZIP archives (including password-protected)
109+
- **Gzip**: Gzip compressed files
110+
- **Zlib**: Zlib compressed data
111+
- **CaRT**: Custom archive format for malware analysis with metadata support
112+
- **IntelHex**: Intel HEX format files
113+
- **SRec**: Motorola S-record format files
114+
- **TiTxt**: Texas Instruments TXT format
115+
- **IMG4**: Apple IMG4 container format
116+
- **LZFSE**: Apple LZFSE compressed data
117+
118+
#### Container Detection Modes
119+
120+
Binary Ninja offers three container detection modes, configurable via the [`files.container.mode`](settings.md#files.container.mode) setting:
121+
122+
- **Full** (default): Automatically discovers all nested paths and builds a complete context tree before requesting user selection. This mode provides the most complete view of the container structure upfront.
123+
- **Interactive**: Requires user interaction at each level of the container hierarchy. This mode is useful when working with deeply nested containers or when you want more control over the extraction process.
124+
- **Disabled**: Opens the file as-is without attempting to unwrap container formats.
125+
126+
#### Working with Containers
127+
128+
When you open a container file in Full or Interactive mode, the Container Browser dialog displays:
129+
130+
- **Name**: The file or entry name within the container
131+
- **Type**: The detected format
132+
- **Size**: File size in bytes
133+
- **Path**: The hierarchical path within the container structure
134+
135+
The browser supports:
136+
137+
- **Filtering**: Use the search box at the top to filter by name, type, or path
138+
- **Password Protection**: Binary Ninja will attempt common passwords (configurable via [`files.container.defaultPasswords`](settings.md#files.container.defaultPasswords)) before prompting for manual entry
139+
- **Custom Extraction**: Right-click any entry to access "Extract With" options, allowing you to apply different transforms (Base64, Hex, etc.) to decode content
140+
- **Metadata Display**: For containers that include embedded metadata, the associated information is displayed in the preview pane on the right.
141+
142+
#### Virtual Paths
143+
144+
Files opened through the Container Browser maintain a virtual path that tracks the full extraction chain. For example:
145+
146+
```python
147+
>>> bv.file.virtual_path
148+
'Zip(.../papi_b64.zip)::Base64(papi_b64)::extracted'
149+
```
150+
151+
This virtual path is also stored in the file's metadata for the 'Raw' BinaryView and can be accessed programmatically:
152+
153+
```python
154+
>>> bv.parent_view.auto_metadata['container']
155+
{'chain': [{'transform': 'Zip'}, {'transform': 'Base64'}], 'virtualPath': 'Zip(.../papi_b64.zip)::Base64(papi_b64)::extracted'}
156+
```
157+
158+
???+ Note "Note"
159+
The virtual path and associated metadata are not persisted when saving to a database (`.bndb` file). Additionally, the format of the virtual path string may change in future releases.
160+
161+
#### Settings
162+
163+
The following settings control Container Browser behavior:
164+
165+
- [`files.container.mode`](settings.md#files.container.mode): Controls container detection mode (Full/Interactive/Disabled)
166+
- [`files.container.autoOpen`](settings.md#files.container.autoOpen): Automatically opens files when there is exactly one extraction path with no required input
167+
- [`files.container.defaultPasswords`](settings.md#files.container.defaultPasswords): List of passwords to attempt for encrypted containers
168+
169+
#### Adding Custom Container Support
170+
171+
Binary Ninja's container system is extensible through the Transform API. Developers can create custom container decoders for proprietary or specialized formats. For information on implementing custom container transforms, see the [Container Transforms developer guide](../dev/containertransforms.md).
172+
100173
## Saving Files
101174

102175
![save choices >](../img/save-choices.png "Save Menu Choices"){ width="400" }

‎docs/img/container-browser.png‎

138 KB
Loading

‎mkdocs.yml‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -129,6 +129,7 @@ nav:
129129
- 'dev/index.md'
130130
- Cookbook: 'dev/cookbook.md'
131131
- Writing Plugins: 'dev/plugins.md'
132+
- Container Transforms: 'dev/containertransforms.md'
132133
- Automation: 'dev/batch.md'
133134
- BNIL / Architectures:
134135
- BNIL Guide: Overview: 'dev/bnil-overview.md'

0 commit comments

Comments
 (0)