Displacy renderer for token level heatmap

[galtay]

Feature description

The displacy visualization tool is great. The parser and named entity visualizations are very useful. Would it be possible to add a renderer for token level "heatmap" style visualizations? This could be used (for example) to show token level model activation or to show the strength of an attention like vector over a set of tokens. I imagine it could be very similar to the named entity renderer (with text and token offsets as input) but then also a way to specify a color (or maybe an index into an arbitrary color map) for each token.

[ines]

This is a nice idea! I've actually been thinking about heatmap-style visualizations for a while and found one of my old drafts that looked something like this. It's a slightly different intepretation of "heatmap", though, and I think for the use case you describe, the implementation of the NER visualizer makes more sense.

Assuming that you have extracted the values for each token and mapped them to an index (say, on a scale of 1 to 5), you could probably already build a hacky version of this using the current displacy:

1. Add one Span per token to the doc.ents and use (the string names of) your scale as labels, e.g. label="1".
2. Add custom colors for the labels via the options.
3. Set a custom teplate that doesn't show the label (to make it easier to read).
4. Visualize the Doc with style="ent".

[galtay]

OH, nice! I tried the hacky thing with new entity types that represented the color bins (and just ignored the entity label when viewing it) but didn't realize there was a template kwarg. Time to upgrade to 2.2 and test my front end knowledge. Could I potentially just remove the label span from this template?

https://github.com/explosion/spaCy/blob/master/spacy/displacy/templates.py#L46

If I get a nice example together tonight I'll post it here.

Thanks!

[galtay]

Just what I was looking for, thanks again :)

https://nbviewer.jupyter.org/github/galtay/displacy_token_heat/blob/master/token_heat.ipynb

[ines]

@galtay Nice! This isn't even that hacky 😃

Also just had a more general idea for how we could support this type of use case better: We could let the visualizer take an optional iterable of spans that should be used instead of the doc.ents. So you could also pass in the doc.sents or doc.noun_chunks, or any other custom selection you've created. The only constraint is that the spans can't overlap.

[galtay]

"not even that hacky" is totally my level of front end skill! :)

Yea, totally agree with you that the span is the better level of abstraction for this (as opposed to token).