Skip to content

Commit 28f9e79

Browse files
committed
Minor changes to README and comments.
1 parent 21bd762 commit 28f9e79

File tree

3 files changed

+17
-18
lines changed

3 files changed

+17
-18
lines changed

README.md

Lines changed: 9 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -225,17 +225,17 @@ python setup.py develop
225225

226226
#### Filtering layers
227227

228-
In order to improve the responsiveness of the tool when visualizing larger models or inputs, you may wish to filter the attention to a subset of layers
229-
by setting the `include_layers` parameter, which specifies which layers to include in the visualization. This option is supported by the head view and model
230-
view currently.
228+
To improve the responsiveness of the tool when visualizing larger models or inputs, you may set the `include_layers`
229+
parameter to restrict the visualization to a subset of layers (zero-indexed). This option is available in the head view and model
230+
view.
231231

232-
**Example:** Render model view with only layers 5 and 6 (zero-indexed) displayed:
232+
**Example:** Render model view with only layers 5 and 6 displayed:
233233
```
234234
model_view(attention, tokens, include_layers=[5, 6])
235235
```
236236

237-
Additionally, the model view accepts an `include_heads` parameter, which specifies which heads to include in the visualization. This may
238-
be set in combination with `include_layers` to display an even smaller subset of attention heads.
237+
For the model view, you may also restrict the visualization to a subset of attention heads (zero-indexed) by setting the
238+
`include_heads` parameter.
239239

240240
#### Dark / light mode
241241

@@ -248,8 +248,8 @@ model_view(attention, tokens, display_mode="light")
248248

249249
#### Setting default layer/head(s)
250250

251-
For the head view, you may choose a specific `layer` and collection of `heads` as the default selection when the
252-
visualization renders. (Note that this is different from the `include_heads`/`include_layers` parameter (above), which
251+
In the head view, you may choose a specific `layer` and collection of `heads` as the default selection when the
252+
visualization first renders. (Note that this is different from the `include_heads`/`include_layers` parameter (above), which
253253
removes layers and heads from the visualization completely.)
254254

255255
**Example:** Render head view with layer 2 and heads 3 and 5 preselected:
@@ -259,8 +259,7 @@ head_view(attention, tokens, layer=2, heads=[3,5])
259259

260260
You may also pre-select a specific `layer` and single `head` for the neuron view.
261261

262-
263-
#### Non-huggingface models
262+
#### Non-Huggingface models
264263

265264
The `head_view` and `model_view` functions may technically be used to visualize self-attention for any Transformer model,
266265
as long as the attention weights are available and follow the format specified in `model_view` and `head_view` (which is the format

bertviz/head_view.py

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -40,10 +40,10 @@ def head_view(
4040
decoder_tokens: list of tokens for decoder input
4141
For all models:
4242
prettify_tokens: indicates whether to remove special characters in wordpieces, e.g. Ġ
43-
layer: index (zero-based) of default layer to show in visualization when first loads. If non specified, defaults to layer 0.
44-
heads: indices (zero-based) of default heads to show in visualization when first loads. If non specified, defaults to all.
45-
include_layers: list of indices (zero-based) of layers to include in visualization (optional). Default
46-
is to include all layers. Filtering the layers may improve performance of the visualization.
43+
layer: index (zero-based) of initial selected layer in visualization. Defaults to layer 0.
44+
heads: Indices (zero-based) of initial selected heads in visualization. Defaults to all heads.
45+
include_layers: Indices (zero-based) of layers to include in visualization. Defaults to all layers.
46+
Note: filtering layers may improve responsiveness of the visualization for long inputs.
4747
"""
4848

4949
attn_data = []

bertviz/model_view.py

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -41,10 +41,10 @@ def model_view(
4141
For all models:
4242
prettify_tokens: indicates whether to remove special characters in wordpieces, e.g. Ġ
4343
display_mode: 'light' or 'dark' display mode
44-
include_layers: list of indices (zero-based) of layers to include in visualization (optional). Default
45-
is to include all layers. Filtering the layers may improve performance of the visualization.
46-
include_heads: list of indices (zero-based) of heads to include in visualization (optional). Default
47-
is to include all heads. Filtering the heads may improve performance of the visualization.
44+
include_layers: indices (zero-based) of layers to include in visualization. Defaults to all layers.
45+
Note: filtering layers may improve responsiveness of the visualization for long inputs.
46+
include_heads: indices (zero-based) of heads to include in visualization. Defaults to all heads.
47+
Note: filtering heads may improve responsiveness of the visualization for long inputs.
4848
"""
4949

5050
attn_data = []

0 commit comments

Comments
 (0)