JupyterHub and BinderHub Team Meeting#
Date: Thursday 16th December 2021
GitHub issue: jupyterhub/team-compass#472
Calendar for future meetings: https://jupyterhub-team-compass.readthedocs.io/en/latest/meetings.html
Welcome to the Team Meeting#
If you are joining the team video meeting, sign in below so we know who was here. Roll call:
Min / Simula / @minrk
Samuel / Idiap / @sgaist
Sarah / 2i2c / @sgibson91
Simon / University of Dundee / @manics
Tim / Binder / @betatim
Erik / Sundell open source / @consideRatio
Yuvi / UC Berkeley & 2i2c / @yuvipanda
Callum / ATI / @callummole
60 second updates on things you have been up to, questions you have, or developments you think people should know about. Please add yourself, and if you do not have an update to share, you can pass.
Sarah: Tony Fast asked me to make sure you were all aware of the Jupyter Accessibility Workshop happening later on today. Everyone is welcome to attend!
Callum: Introduce Luke as the next Turing Cluster administrator (though I still plan to be involved!)
Reports and celebrations#
This is a place to make announcements (without a need for discussion). This is also a great place to give shout-outs to contributors! We’ll read through these at the beginning of the meeting.
Min: Feedback and testing on roles, especialy the default ‘user’ role not being applied. Fixes I’d like to get into 2.0.1 PRs:
Min: Working on extending roles & scopes to single-user servers w/ RTC team
Let’s collect all potential agenda items here before the start of the meeting. We will then attempt to create a coherent agenda that fits in the 60m meeting slot. If there are similar items try and group them together.
Tim (5min): what kind of things could happen if mybinder.org becomes a popular ipfs hoster?
Action: create a new issue template that people can use to request that a port is opened
obvious ports related to abuse will not be opened (FTP, SMTP, IMAP, POP, ??)
other ports will probably be opened if you make a reasonable case for it and can help us understand the risks
ipfs is a “distributed storage thing” (interplanetary file system)
is that even possible?
Yuvi has context:
pangeo folks are exploring using IPFS to store data
Need to clarify that binder should work as a “leecher” but not “seeder” in BitTorrent terminology. Seems that we may already have this if we don’t allow inbound connections.
My concern is based around “running a bittorrent seed” which ends up generating a lot of egress traffic and attracts people to your nodes. Maybe outdated?
Simon: How do we remain “fair” to other users requesting open ports for other services (we’ve rejected some in the past)
We have blocked requests in the past
Currently we are a bit ad-hoc
Related to IPFS - there are IPFS bridges, does that help?
If we do that, lose advantages of multi-host pull and other p2p advantages of the IPFS protocol
Min says it is ok to be liberal about opening ports, as most mining pools listen on port 443 as well
Samuel (10m): Multi-authenticator status update
Can mix local PAM and OAuth
Issue in PR where login url comes from Hub instead of authenticator
Ready for testing
Q: is oauthenticator the right repo?
Min: primary use case is multiple oauth providers, so I think it makes sense for the first generation. As it matures, can move to its own repo or upstream to JupyterHub.
Samuel (10m): Could Amalthea simplify the handling of Jupyter server on Kubernetes and thus KubeSpawner.
Might simplify KubeSpawner / be an alternative to KubeSpawner?
Uses CRD / Operator approach.
Big change, probalby hard to transition for z2jh/kubespawner
Support for template approach, would simplify things
kubernetes server-side apply allows creating patches from server-computed diff, which is relevant to template approach
Min: not sure CRDs add much when we have to have KubeSpawner anyway, unless we can use Amalthea exactly as-is.
Yuvi (5m) Latest RStudio and R now supported on JupyterHub via jupyter-rsession-proxy 2.0! (Yay to Ryan Lovett and @maresb)
Upstream Rocker binder image now fixed, was broken for a while. I made a template repo using that at yuvipanda/rstudio-binder-template
R support in repo2docker needs some love
Tim: what do people think of using conda to install R itself? Ubuntu/APT packages for R versions that aren’t “the latest” doesn’t seem to be a thing
APT packages is what we are doing now but it breaks “all the time” :(
Yuvi: packagemanager.rstudio.com is what we should move to - it gives all of CRAN (so current installation methods work), but provides prebuilt binary packages! Makes everything much faster, works with the package management R users are used to
Tim: can this install R itself as well?
Picking R version seems to be an unsolved problem with apt
Check-in what’s available for picking R version w/ packagemanager.rstudio.com
Sarah’s doing a survey a the collaborations workshop 2022 (March)
ask R folks what they need / want
Min: “automating existing community practices” means we should do what R folks are doing. If that means always using system R, that’s what we should do.
Erik: Does latest RStudio mean a version
>1.4.*? (Yuvi: Yes, they use calendar versioning now. 2021.07.x works)
Erik: Any known problems of updating to 2021.07.x?
Yuvi: Fairly new, I’m upgrading a few Berkeley hubs tomorrow. Will find out :)
Erik: Is it correct that what is now solved is the issue about redirecting to the RStudio’s own sign in and such?
Yuvi (5m) PR to have a stricter network policies by default for single user pods on z2jh
Inspired by the fantastic breakdown of the Azure CosmosDB vulnerability exploited via their Jupyter Notebooks (read here)
Prevents access to all Private IPs while allowing all access to the internet. Currently we don’t restrict access to private IPs, and this can leave unguarded services (like prometheus running on the cluster) exposed
Erik: Please review the suggested configuration we are about to implement in that PR, see: jupyterhub/zero-to-jupyterhub-k8s#2508
Erik (2-5m) Status update about nbgitpuller PR about plugin mechanism to download content from non-git sources developed by Sean Morris
Erik (2-5m) Status of Z2JH
Concrete work towards release remain, without blockers in other projects.
Breaking network policy changes described by Yuvi above
Heads up, a breaking change in kubespawner 2 will make anyone that supports
sudoneed to do an extra step, to be documented in the changelog
Erik (2-5m) Status of TLJH
Maintenance need is high still
No versioning of the distribution is a challenge now that we want to update to JupyterHub 2.