Mantid logs appearing in Jupyter notebook cells

[rbauststfc]

Original reporter: Andrew and Christy from the POLREF instrument at ISIS. They have asked to be kept up to date on progress with this.

Describe the bug

In the last cycle, when using the Mantid 6.7 kernel in IDAaaS (Jupyterlab in Jupyter environment), the logging that appears in the terminal is now also appearing in the cell output. This issue can be replicated simply by running the following command in a Jupyter notebook cell:

from mantid.simpleapi import *

In order to try and work out why this is happening Andrew has run a few tests and provided the following information:

Firstly, when using the Mantid 6.6 kernel in IDAaaS, the issue is not there, and it behaves as expected - only python printouts are sent to the cell output and the logging is only in the terminal.

Interestingly, when I use one of my own conda environments on IDAaaS (I have a separate miniconda install for development work), with what I believe is a newer version of Jupyter than potentially what is being used in the 6.6 IDAaaS Kernel (and with an older version of Mantid 6.4), the logging issue appears. Again, I believe this didn't used to happen with this environment, however, I can't confirm if jupyter in the environment had been updated or something in the background in IDAaaS has been updated.

Finally, when using a conda environment with Mantid 6.7 in windows with a very recent version of Jupyter, it all seems fine - i.e. no logging issues and things working as expected.

From this, it seems something has changed with the way IDAaaS interacts with Jupyterlab such that either certain versions of Jupyterlab or the way it is configured, needs to be different than it was before so that the logging works correctly.

IDAaaS support have done some investigation into this and haven't yet been able to find anything obvious. They asked us to test if this issue occurs on a Linux system other than IDAaaS, so I tested on a Linux ubuntu-focal-20.04 instance on STFC cloud. I found that if I conda install Mantid 6.7 and try to run it in the latest (pip installed) version of Jupyter notebooks then I see the logs in the notebook. If I do the same with Mantid 6.6 then I don't see the issue. It would be good to investigate this further to see if we can explain the difference.

Note that it's possible to change the logging behaviour in Mantid (see https://docs.mantidproject.org/v6.7.0/concepts/PropertiesFile.html#logging-properties) but neither Andrew or myself have been able to get any of these settings to stop the logs printing to the Jupyter notebook.

To Reproduce

1. Conda install Mantid 6.7.
2. Pip install Jupyter notebooks into the conda environment.
3. Open Jupyter notebooks, start a new notebook and in the first cell type from mantid.simpleapi import *. The log information should be printed in the notebook. You can also see the effect if, after importing the simpleapi, you call DownloadInstrument() from a cell too.

Expected behavior

The logs appear only in the terminal, not in the Jupyter notebook.

Platform/Version (please complete the following information):

• OS: Linux, IDAaaS. It could be handy to check the behaviour on Mac.
• Mantid Version 6.7

[jhaigh0]

Right so as a temporary measure, you can set the logging level to be one less like this

%config Application.log_level='WARN'
import logging
logging.getLogger().setLevel(logging.WARN)

All the Mantid output seems to come in at level INFO which I think is the default.

[jhaigh0]

Actually, this more Mantid way is better

ConfigService.Instance().setLogLevel("warning", True)

Although this does mute the console output as well.

[jhaigh0]

Okay so we've tracked this down to ipykernel updating between mantid versions. In particular, it was introduced intentionally in ipykernel 6.6.0 in this pr.
ipython/ipykernel#630

Will look further into what we can do about this.

[rbauststfc]

Sounds like great progress, thanks!

[jhaigh0]

Okay, so they added a config option to turn the behaviour off here:
ipython/ipykernel#752

So if you want to use this option (IPKernelApp.capture_fd_output = False) you will need to introduce it either using a config profile or on the command line, see here:
https://ipython.org/ipython-doc/3/config/intro.html

I'm hopeful that on Idaas we can set the mantid kernel's up to use this option by default.

[github-actions]

This issue has been automatically marked as stale because it has not had activity in 6 months. It will be closed in 7 days if no further activity occurs.
Allowing issues to close as stale helps us filter out issues which can wait for future development time. All issues closed by stale bot act like normal issues; they can be searched for, commented on or reopened at any point.
If you'd like a closed stale issue to be considered, feel free to either re-open the issue directly or contact a developer.
To extend the lifetime of an issue please comment below, it helps us see that this is still affecting you and you want it fixed in the near-future. Extending the lifetime of an issue may cause the development team to prioritise it over other issues, which may be closed as stale instead.