Import hooks to set up inline backend when matplotlib is loaded

[takluyver]

Addresses ipython/ipython#6424

We believe that most people using matplotlib from a Jupyter frontend want plots to appear inline. Particularly when introducing new users to Python, we end up telling them to run %matplotlib inline with a "don't worry about what this does...", which feels wrong when we're trying to help them understand what they're doing.

This is an attempt to make inline plots the default when you use matplotlib in an IPython kernel. It installs import hooks to watch for matplotlib being imported, and sets up the inline backend when it is. There is a config option (IPKernelApp.mpl_plots_inline) to disable this behaviour. Specifying a backend with %matplotlib or %pylab should also override it.

We've discussed before whether this belongs in IPython or in matplotlib. I don't have a strong preference, but I got the impression that other people preferred it to be in IPython. If we decide to go the other way, I'm happy to help mpl do it.

Ping @mdboom and @tacaswell

[mdboom]

👍 to the idea, but don't you think the default should be the interactive notebook backend? It's really the most powerful way to use matplotlib in a notebook these days and is pretty performant, even over remote connections. (And if it's not performant, it's no worse in functionality than inline).

I don't have a strong preference about where this lives. However, I don't like that this creates a second config file location to set the backend in addition to matplotlibrc. I think what I'd prefer is to change behavior based on the matplotlibrc backend value (i.e. the value of matplotlib.get_backend() immediately after importing matplotlib).

• If a non-interactive backend (Agg, Pdf, Svg etc.) use Jupyter's inline backend. Bonus points for using svg instead of png if the user requests it
• If a GUI backend, use the GUI framework (and not inline)
• If set to nbagg, use the interactive notebook backend.

And then we get rid of any configuration on the Jupyter side. This gives the user one place to set the backend and Jupyter will follow it. %matplotlib and %pylab would continue to override.

[takluyver]

We discussed the possibility of using nbagg before, but I think it only works in a notebook. The inline backend is more basic, but it should work from any Jupyter frontend.

I don't want to do it based off the backend already set in mpl, because if you haven't specifically configured a backend, you'll probably get a GUI backend (on my system, the default is TkAgg). The aim is to avoid people having to do configuration to get plots inline.

If I was to do this in mpl, I'd add a config option called something like ipython_kernel_backend, which would default to the IPython inline backend. Then the user could override that to any other backend, or a special 'same' value to tell it to use the same backend as it would outside an IPython kernel.

[mdboom]

We discussed the possibility of using nbagg before, but I think it only works in a notebook. The inline backend is more basic, but it should work from any Jupyter frontend.

Oh, of course. Sorry I missed that.

If I was to do this in mpl, I'd add a config option called something like ipython_kernel_backend, which would default to the IPython inline backend. Then the user could override that to any other backend, or a special 'same' value to tell it to use the same backend as it would outside an IPython kernel.

Considering all that, I suppose that is the best option. A middling alternative is to have the ipython backend setting live in the ipython config but instead of being boolean as proposed here, be "notebook", "inline", "same" or any of the other matplotlib built-in backends.

[takluyver]

You can already configure the mpl backend from IPython, but for historical reasons that works by loading mpl as soon as IPython is started, so we haven't set any default value for that. If we do keep this in IPython, I like the idea of it being configurable to any backend on mpl import as well.

Do you think this would be better as a feature in matplotlib? It would need some knowledge of IPython to set up the hooks the inline backend relies on. I hope some of the others will weigh in on this too.

[tacaswell]

Couldn't we tack a rich repr on to nbagg that will fall back to just
displaying a png of its current state if it hits a non web front end?

I am unclear on why anyone would prefer a static figure to an interactive
one.

On Thu, Dec 3, 2015, 11:42 Thomas Kluyver [email protected] wrote:

You can already configure the mpl backend from IPython, but for historical
reasons that works by loading mpl as soon as IPython is started, so we
haven't set any default value for that. If we do keep this in IPython, I
like the idea of it being configurable to any backend on mpl import as well.

Do you think this would be better as a feature in matplotlib? It would
need some knowledge of IPython to set up the hooks the inline backend
relies on. I hope some of the others will weigh in on this too.

—
Reply to this email directly or view it on GitHub
#82 (comment).

[takluyver]

If nbagg sent a plain png fallback along with the interactive HTML version, I think we'd be happy for that to be the default.

[minrk]

Setting the hook on matplotlib makes it impossible for

import matplotlib
matplotlib.use('something')

to work. Is there something else we can look for that wouldn't preempt explicit backend selection? I've used pyplot in my own version of this, but I'm not sure if it's sufficiently general.

[takluyver]

I think the only problem is that the import hook loads our backend (and pyplot) in order to set up the config changes and post-execute hooks that the inline backends need. If we can delay that stuff (the call to configure_inline_support()) until the backend is actually loaded, I think it would be fine to set the backend on loading matplotlib, because the setting isn't committed until matplotlib.backends is loaded.

[tacaswell]

There was also discussion at scipy with @WeatherGod to delay actually forcing the back end until the first figure is created.

[cswarth]

@tacaswell There are a wide range of applications where in interactive graphic is simply not performant. Large graphs that plot thousands or millions of points can easily be viewed as a static png because the data is far larger than the pixels used to show it. A simple interactive backend like notebook tries to ship each and every point to the browser just in case you might want to zoom in on them. Doing this with large datasets will bring your browser to a crawl pretty quickly. Doing this 'right' is complicated and requires the server to slice and dice the data on demand. Think about how google maps allows you to view the map at various resolutions - it doesn't just send all world coordinates to your browser at once. This is also why genomic viewers are so specialized, because genomes are so damn big and deep sequencing experiments generate many millions of data points per experiment.

[tacaswell]

@cswarth That is not how notebook works. The image is rendered to a png on the kernel side and a png is shipped over the wire to the notebook. The js side in the browser captures mouse and keyboard events which it ships to the kernel side where they are processed by the mpl event framework, possibly triggering a new png to be shipped to the browser.

There can be performance issues inherent to mpl (ex, 1M points in a scatter can be slow), but inline will have the same performance constraints.

[cswarth]

Thank you for the correction! I was assuming it was implemented like a number of other SVG-based systems that fail miserably for large data. I should have checked before commenting.

[mdboom]

@cswarth: No worries. We haven't done a very good job communicating about how it works -- and it is very different from most other systems.

[minrk]

Reviving this, since I would like to ship 4.3 very soon. Sorry for the long delays.

@mdboom @tacaswell is there any module we can put this import detection on that we we can know will trigger automatic backend selection? I'd really like to avoid breaking matplotlib.use('backend'), which this does, currently.

In my version I detect pyplot, but that's because I always import pyplot.

The general logic I'd like:

if importing_something_in_matplotlib_would_trigger_backend_selection:
    if backend_isnt_already_selected:
        %matplotlib inline

So we need to know:

1. what module(s) force backend selection on import, and how stable is that statement
2. how do we ask if matplotlib.use('backend') has been called

If we have that, I think we are set and can take care of the rest on our side.

[Carreau]

ping @takluyver

[takluyver]

I think we were waiting for answers to @minrk's questions, to work out what we can reliably watch for to override automatic backend selection.

[wernight]

Would it be simpler to change the logic to:

if importing_something_in_matplotlib_would_trigger_backend_selection and not_initialized:
    %matplotlib inline

where not_initialized just means the kernel hasn't been restarted.

[takluyver]

I think that's roughly what we're aiming for already. It's the importing_something_in_matplotlib_would_trigger_backend_selection that we're not sure about.

[tacaswell]

There is also talk of deferring the pinning of the backend until you use
it, not import pyplot.

On Tue, Apr 26, 2016, 08:38 Thomas Kluyver [email protected] wrote:

I think that's roughly what we're aiming for already. It's the
importing_something_in_matplotlib_would_trigger_backend_selection that
we're not sure about.

—
You are receiving this because you were mentioned.

Reply to this email directly or view it on GitHub
#82 (comment)

[takluyver]

Does that seem likely? As part of that change, could you create a 'get the default backend' hook that we could plug into, rather than using import hooks?

[WeatherGod]

I would say that it is likely to happen at some point, but it isn't going
to be in our v2.0 release. Might be feasible to collaborate on it at SciPy,
though, if that gets you a sense of the timeline.

On Tue, Apr 26, 2016 at 9:01 AM, Thomas Kluyver [email protected]
wrote:

Does that seem likely? As part of that change, could you create a 'get the
default backend' hook that we could plug into, rather than using import
hooks?

—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
#82 (comment)

[takluyver]

Thanks! I think we've waited long enough for this one that we can wait a few more months, especially since there doesn't seem to be a good way to do it at present.

I won't be at SciPy this year, unfortunately, but other IPython people certainly will be.

[minrk]

Closing in favor of the lighter #159. Getting close to this one!