JupyterHub and BinderHub Team Meeting - August

  • Date: Thursday 20th August, 2020

  • Time: 8am UTC

    • Your timezone: https://arewemeetingyet.com/UTC/2020-08-20/08:00/Hubs-Team-Meeting

  • This HackMD: https://hackmd.io/@sgibson91/hubs-team-meeting

  • GitHub issue: https://github.com/jupyterhub/team-compass/issues/319

  • Calendar for future meetings: https://jupyterhub-team-compass.readthedocs.io/en/latest/meetings.html

Welcome to the Team Meeting

Hello!

If you are joining the team video meeting, sign in below so we know who was here. Roll call:

  • name / institution / GitHub handle

  • Richard Darst / Aalto / @rkdarst

  • Tim Head / Binder / @betatim

  • Maria Eriksson / UCL Institute of Child Health & Great Ormond Street Hospital

  • Yuvi Panda / UC Berkeley / @yuvipanda

  • Min RK / Simula / @minrk

  • Ivana Huskova / Simula / @IvanaH8

  • Simon Li / OME/University of Dundee / @manics

  • Sarah Gibson / Turing Institute / @sgibson91

  • Chia-liang Kao / InfuseAI / @clkao

Quick updates

60 second updates on things you have been up to, questions you have, or developments you think people should know about. Please add yourself, and if you do not have an update to share, you can pass.

  • Sarah: I have been working to get more Azure credits for the Turing cluster. Hopefully should be back online soon

  • YuviPanda: I’ve been working on MyBinder Terminal - a pure-frontend web-terminal interface to mybinder, where users can share sessions & collaborate realtime (with tmux / screen)

  • Min: Welcome to Ivana Huskova, working at Simula on auth, RBAC, & Projects for Sloan colaboration through Cal Poly

  • Min: Outreachy proposals for winter are due in September. Min plans to mentor this year project ideas welcome!.

Reports and celebrations

This is a place to make announcements (without a need for discussion). This is also a great place to give shout-outs to contributors! We’ll read through these at the beginning of the meeting.

  • DockerHub image expiration and new terms of service https://github.com/jupyterhub/team-compass/issues/330

  • 🎉 YAML linting for mybinder.org-deploy, for all of us who don’t have a yaml parser in our head

    • cool work by Sarah!

  • JupyterCon

    • Min: two posters accepted for JupyterCon:

      • Vilde’s summer work on repo2docker reproducibility

      • Min’s poster on limits to scaling JupyterHub

    • rkdarst: JupyterHub proposal got accepted. team-compass issue about to come, will be happy to have co-authors.

    • Yuvi: 3 talk participants

      • 12min talk on jupyter-server-proxy

      • Talk with Zach on Jupyter Telemetry

      • Fernando, Chris Holdgraf, Cathryn Carson, Jim Colliander, Ryan Abernathey on https://2i2c.org

    • clkao: 12min talk on enterprise jupyterhub and mlops with primehub

    • Sarah: Arnim and Kenan’s talk on persistant BinderHub was also accepted

Agenda items

Let’s collect all potential agenda items here before the start of the meeting. We will then attempt to create a coherent agenda that fits in the 60m meeting slot. If there are similar items try and group them together.

  • HPC/batchspawner/etc. Maintenance of these spawners, when to collect docs and use cases?

    • batchspawner maintenance

      • rkdarst: summary in batchspawner issue

      • We’ve added more maintainers and discussed how to keep things going when there is not enough time, and testing is hard

    • How to make HPC-type use cases more visible?

      • jupyturhub-deploy-hpc is rather stagnant and not a good entry point.

      • should batchspawner sphinx docs or jupyterhub sphinx docs be expanded?

    • Thoughts from others:

      • where to store the docs is maybe less important than that they exist and pointed to

      • hard thing is that every HPC is different from each other HPC

      • a good resource is JupyterHub-from-scratch

      • Answer the question “so your users want Jupyter on their HPC cluster, where do you go from that? What are the options?”

      • batchspawner and research facility people are welcome and should definitely come to this meeting as well

      • https://github.com/yuvipanda/the-batchiest-jupyterhub

  • Time based tagging of repo2docker and BinderHub code ()Tim)

    • https://github.com/jupyterhub/binderhub/issues/1141

    • what do people think of us starting to tag the code of repo2docker and BinderHub once a month to create a “release”

    • This is related to the dockerhub image expiry

    • nothing changes about uprgade/migration promises

      • “continous upgrading is the only supported form of upgrades”

    • goal is to provide easier to read tags

    • Ideas:

  • What is jupyter-server and what does it mean for repo2docker/binder/etc that it is going to be used by lab? ()Tim)

    • Tim doesn’t know much, but from various issues it seems we need to change things

    • Will URL based switching between lab and notebook still work?

    • Will jupyter-server make all UIs we offer “first class”?

      • RStudio, VS Code, lab, classic

    • Maybe someone is a bit more involved in this topic and could give a short intro

    • find some related issues via this

    • batchspawner also makes a wrapper batchspawner-singleuser (sends remote port to hub), how does jupyter-server relate there?

  • GKE credits ()Tim)

    • Chris(?) sent something (a proposal) to someone (at Google), does anyone know more?

      • It’s to this person, working on HPC stuff at Google: https://twitter.com/sdksb (Yuvi)

      • As of a week ago, he’s got an internal proposal sent to GCP

    • Is there something others can do to support?

      • People in the email thread are Fernando, Chris & Yuvi

      • Yuvi will follow up

        • Should we add more people? Min: add me, if that’s okay, if there’s further discussion required

    • Should we attempt to contact “all the people” at Google that we know or wait for this request to take its course?

      • lets wait to see what happens first

    • For reference, OVH provides constant 16 CPUs + 64 GB RAM and hosts ~10% of mybinder.org

  • MyBinder Terminal at terminal.mybinder.org ()Yuvi)

    • Show and build alternate interfaces for mybinder.org - not just GUIs

    • Launch a gist with tmux / screen installed. Once there, you can grab the ‘current session’ link, and share with someone else. If you are running ‘tmux’, they can attach with ‘tmux attach’ and now you’re both sharing the session realtime!

    • Any mybinder.org link will work - change ‘mybinder.org’ domain to ‘terminal.mybinder.org.yuvi.in’. https://terminal.mybinder.org.yuvi.in/v2/gh/gvanrossum/patma/master is the link to Guido’s Python 3.10 branch, for example

    • Works on all mybinder.org repos, doesn’t require any particular installation in the image

    • TODO: Have a link generator on the home page

    • I want to put this on terminal.mybinder.org