Add packages to Jupyter org on PyPI

[Carreau]

I realized this is not part of the Jupyter org,

Plus this could receives ~$100/month on tidelift, which I think the EC is working on

[consideRatio]

@yuvipanda @minrk, do you approve migrating this package to live under the Jupyter PyPI org?

I checked if I could technically transfer this, but I'm not in the Jupyter org on PyPI, maybe @yuvipanda or @minrk are?

[Carreau]

See jupyter-governance/ec-team-compass#101 then.

[manics]

I've added this to the agenda for tomorrow's meeting. @consideRatio Will you be able to explain the background for transferring this and other relevant repos during the meeting?

[Carreau]

In general for various maintenance and audit task we want an easy way to list all jupyter related projects on PyPI, that is the point of the Jupyter Org, it allows anybody to know that "this is an official jupyter project", it is not that much a "Transfer", but a "should be listed on", the current owners should still have access.

Personally for my security work, I have script that check all the package on the org, and discoverd that this one was not listed.

Note that I opened jupyter-governance/ec-team-compass#39 9 month ago and still nothing has been done.

[consideRatio]

@manics I lack insights into this, good call adding it to the agenda! Thank you for providing some context @Carreau, if you have more to link out to it could be helpful.

I don't yet know anything about the tidelift context btw, is there some good issue about that to look into?

[minrk]

Yeah, pretty much all of our packages should be under the Jupyter org on PyPI.

There isn't much to know, other than it adds this:

to the PyPI listing (as seen on jupyterlab) and enables some centralized administrative tasks. Mostly it's just the visual "this is official from Jupyter" indicator, and showing up in the organization listing.

[Carreau]

Be careful when you add to an org, you can be sometime removed as a maintainer and lose rights (until added back to a proper team).

It is also nice to add yourself back as a maintainer, otherwise your name is not listed on the package page anymore and becomes annoying to find (even for other jupyter org members).

[Carreau]

Here is a rough heuristic based of repo names in jupyterhub org that seem to match a Pypi package, names but are not in the Pypi jupyter org:

• https://github.com/jupyterhub/alabaster-jupyterhub -> https://pypi.org/project/alabaster-jupyterhub
• https://github.com/jupyterhub/batchspawner -> https://pypi.org/project/batchspawner
• https://github.com/jupyterhub/binderhub -> https://pypi.org/project/binderhub
• https://github.com/jupyterhub/chartpress -> https://pypi.org/project/chartpress
• https://github.com/jupyterhub/configurable-http-proxy -> https://pypi.org/project/configurable-http-proxy
• https://github.com/jupyterhub/design -> https://pypi.org/project/design
• https://github.com/jupyterhub/docker-image-cleaner -> https://pypi.org/project/docker-image-cleaner
• https://github.com/jupyterhub/dockerspawner -> https://pypi.org/project/dockerspawner
• https://github.com/jupyterhub/gh-scoped-creds -> https://pypi.org/project/gh-scoped-creds
• https://github.com/jupyterhub/jupyter-remote-desktop-proxy -> https://pypi.org/project/jupyter-remote-desktop-proxy
• https://github.com/jupyterhub/jupyter-rsession-proxy -> https://pypi.org/project/jupyter-rsession-proxy
• https://github.com/jupyterhub/jupyter-server-proxy -> https://pypi.org/project/jupyter-server-proxy
• https://github.com/jupyterhub/jupyterhub -> https://pypi.org/project/jupyterhub
• https://github.com/jupyterhub/jupyterhub-idle-culler -> https://pypi.org/project/jupyterhub-idle-culler
• https://github.com/jupyterhub/jupyterhub-sphinx-theme -> https://pypi.org/project/jupyterhub-sphinx-theme
• https://github.com/jupyterhub/nbgitpuller -> https://pypi.org/project/nbgitpuller
• https://github.com/jupyterhub/nbgitpuller-downloader-dropbox -> https://pypi.org/project/nbgitpuller-downloader-dropbox
• https://github.com/jupyterhub/nbgitpuller-downloader-generic-web -> https://pypi.org/project/nbgitpuller-downloader-generic-web
• https://github.com/jupyterhub/nbgitpuller-downloader-googledrive -> https://pypi.org/project/nbgitpuller-downloader-googledrive
• https://github.com/jupyterhub/nullauthenticator -> https://pypi.org/project/nullauthenticator
• https://github.com/jupyterhub/oauthenticator -> https://pypi.org/project/oauthenticator
• https://github.com/jupyterhub/pytest-jupyterhub -> https://pypi.org/project/pytest-jupyterhub
• https://github.com/jupyterhub/simpervisor -> https://pypi.org/project/simpervisor
• https://github.com/jupyterhub/sudospawner -> https://pypi.org/project/sudospawner
• https://github.com/jupyterhub/wrapspawner -> https://pypi.org/project/wrapspawner

In addition if it help, the list of repo for which it did not automatically found a pypi package match:

https://github.com/jupyterhub/.github
https://github.com/jupyterhub/action-k3s-helm
https://github.com/jupyterhub/action-k8s-await-workloads
https://github.com/jupyterhub/action-k8s-namespace-report
https://github.com/jupyterhub/action-major-minor-tag-calculator
https://github.com/jupyterhub/binder-billing
https://github.com/jupyterhub/binder-data
https://github.com/jupyterhub/dummyauthenticator
https://github.com/jupyterhub/firstuseauthenticator
https://github.com/jupyterhub/grafana-dashboards
https://github.com/jupyterhub/helm-chart
https://github.com/jupyterhub/hubshare
https://github.com/jupyterhub/jhub-proposals
https://github.com/jupyterhub/jupyterhub-deploy-docker
https://github.com/jupyterhub/jupyterhub-deploy-hpc
https://github.com/jupyterhub/jupyterhub-deploy-teaching
https://github.com/jupyterhub/jupyterhub-example-kerberos
https://github.com/jupyterhub/jupyterhub-on-hadoop
https://github.com/jupyterhub/jupyterhub-python-repo-template
https://github.com/jupyterhub/jupyterhub-the-hard-way
https://github.com/jupyterhub/jupyterhub-tutorial
https://github.com/jupyterhub/jupyterhub.github.io
https://github.com/jupyterhub/jupyterlab-hub
https://github.com/jupyterhub/katacoda-scenarios
https://github.com/jupyterhub/kerberosauthenticator
https://github.com/jupyterhub/kubespawner
https://github.com/jupyterhub/ldapauthenticator
https://github.com/jupyterhub/ltiauthenticator
https://github.com/jupyterhub/mybinder-tools
https://github.com/jupyterhub/mybinder.org-deploy
https://github.com/jupyterhub/mybinder.org-user-guide
https://github.com/jupyterhub/nativeauthenticator
https://github.com/jupyterhub/nbgitpuller-downloader-plugins
https://github.com/jupyterhub/outreachy
https://github.com/jupyterhub/pebble-helm-chart
https://github.com/jupyterhub/repo2docker
https://github.com/jupyterhub/repo2docker-action
https://github.com/jupyterhub/research-facilities
https://github.com/jupyterhub/simplespawner
https://github.com/jupyterhub/systemdspawner
https://github.com/jupyterhub/team-compass
https://github.com/jupyterhub/the-littlest-jupyterhub
https://github.com/jupyterhub/tmpauthenticator
https://github.com/jupyterhub/traefik-proxy
https://github.com/jupyterhub/yarnspawner
https://github.com/jupyterhub/zero-to-jupyterhub-k8s

script is in this PR: https://github.com/jupyter/security/pull/91/files

[manics]

I've moved this to the JupyterHub/team-compass since we're talking about all repos, not just simpervisor

[krassowski]

to the PyPI listing (as seen on jupyterlab) and enables some centralized administrative tasks. Mostly it's just the visual "this is official from Jupyter" indicator, and showing up in the organization listing.

If API for it becomes available, we might also end up using it to verify official extensions in JupyterLab/Jupyter Notebook in Extension Manager (currently there is not REST API for organizations though).

[KirstieJane]

Hi @Carreau and @krassowski !

We just had our Binder / JupyterHub monthly "collaboration cafe" and we added this issue to our agenda.

There are three open questions that unfortunately we didn't have the expertise in the call to be able to answer. (Likely @minrk has some of them but this month's call is at a time that doesn't work for his childcare responsibilities!)

(Shout out to @consideRatio and his little team mate who joined us! Multitasking Jitsi calls with a toddler is HARD!)

Q1. Please can we invite someone to come to our next collab cafe to explain this request a little more?

To be clear - no one is coming up with any problems that we can see. We just generally felt that we weren't well enough informed to know who should make the decision or how to make the decision!

There's a broader point here about communications across Jupyter which also led us into talking about "How to use Foundation funds to support the Jupyter community": jupyter-governance/jupyter-foundation-governing-board#1. If you have thoughts about how we (JupyterHub) can better connect with the broader project around these types of decisions / coordinated activities then please let us know!

Q2. What are the responsibilities that need to be allocated once the packages are moved over to Jupyter on PyPi?

I think another of our blockers here is that we don't know what happens after the move? What are the explicit asks for the maintainers? Should we have a dedicated person for each package (can be the same person multiple times!)?

Again, this is kind of a bigger question about JupyterHub governance and roles and very much downstream from Q1 (above). So any guidance here is really helpful.

Q3. What's the tidelift connection?

We were following the ask pretty well but no one in the conversation knew how Tidelift fitted in. This may also be well documented in the answer to Q1 😄

Thank you! Sorry if these are irritating - please send us links to read!
Kx

[krassowski]

The tidelift conversation is a separate one from the PyPI one, but I suspect it comes about from jupyter/security#91.

Let me give you some back story: about two years ago @Carreau proposed that JupyterLab could claim funds which are donated to JupyterLab on tidelift (and should by definition be earmarked for security) and that we could use them to help patch vulnerabilities (around that time Jupyter partook in bug bounty programme where reporters were funded for finding bugs, but no one really was paid to fix them). @Carreau opened:

• Tidelift and JupyterLab jupyterlab/frontends-team-compass#224

but during discussion on frontends call some members, were not fully comfortable with making this decision (there were questions like, are we even allowed to make this decision as a subrpoject, or do we need a blessing from EC), and this lead to creation of a ticket on ec-team-compass:

• Potential JupyterLab partnership with Tidelift jupyter-governance/ec-team-compass#20

I followed with EC multiple times to establish how JupyterLab could receive these funds, which downstream users of Jupyter pay to tidelift anyways and which currently month after month for years are basically lost. There is a couple of questions we need to answer, like:

• where should funds be directed
• who is going to sign the agreement with tidelift
• how should these be distributed

But we are not concerned with any of those three here, as these are really for EC, security group and subprojects to settle (please do chime in jupyter-governance/ec-team-compass#20 if you have any thoughts). Instead, there is another issue that came up when discussing the topic with EC: is it even worth pursuing this funding avenue, and which packages should be enrolled.

This is because for some reason the tidelift interface is hard to coerce to display the pypi/npm projects based on search criteria. I came up with one solution (filtering by upload bot), Matthias wrote another one in jupyter/security#91 (checking all projects of a Jupyter PyPI organization), which is why this two issues are associated. Otherwise, tidelift and PyPI org are disjoint issues.

[krassowski]

Please can we invite someone to come to our next collab cafe to explain this request a little more?

We just generally felt that we weren't well enough informed to know who should make the decision or how to make the decision

I would not know either! My assumption is that subprojects have autonomy to make the decision to move to Jupyter org on PyPI, and that security group can encourage this as to enable monitoring of 2FA enforcement, but I think this is not required and I believe subprojects should have a say which packages get added into the PyPI org.

There are some practical things I can show by joining the collab cafe - I could show how it looks on PyPI from either admin or manager level.