JupyterHub and BinderHub Team Meeting - September

Meeting details

Date: Thursday, August 15, 7am UTC Videoconference link: https://calpoly.zoom.us/my/jupyter

Attendees

  • name | institution | GitHub handle

  • Tim | Binder | @betatim

  • Zach | Cal Poly | @Zsailer

  • Kenan | GESIS | @bitnik

  • Kirstie | Alan Turing Institute | @KirstieJane

  • Chico Venancio | BMCG K.K | @chicocvenancio

  • Min | simula | @minrk

  • Arnim | GESIS | @arnim

  • Erik | Sandvik CODE | @consideRatio

Quick updates

60 second updates on things you have been up to, questions you have, or developments you think people should know about. Please add yourself, and if you do not have an update to share, you can pass.

  • [name=myname] My update!

  • [name=Chico Venancio] ECR and CodeCommit work done, (PRs #920 #940). Current work on autosaving conflict with BinderHub.

  • [name=Tim] we now support different R versions in repo2docker

Reports and celebrations

This is a place to make announcements (without a need for discussion). This is also a great place to give shout-outs to contributors! We’ll read through these at the beginning of the meeting.

  • [name=myname] My report!

  • [name=Tim] [name=Kenan] [name=Sarah] [name=Erik] Thanks for the awesome workshop Min!

  • [name=Sarah] I think the BinderHub workshop at RSE Con UK went well. See feedback here: bit.ly/RSEConBinderHub

  • [name=Tim] Thank you Sarah for all the work representing Binder at RSE Con UK by running a workshop and giving a talk and giving a lightning talk and taking the time to go to the event in the first place!

Agenda items

This is what we’ll cover in the meeting (we have about 60m in total). We’ll copy the proposed agenda topics above here just before the meeting.

  • Add agenda item here [name, estimated time for conversation]

  • Report from the workshop, all those who attended

    • notes and linked pads: https://hackmd.io/a2i1WfvmTJW4RJJeCrwdYA

    • we worked and discussed (summary pad):

      • documentation

        • create a “Hub for managers” guide

        • started “jupyterhub the hard way” tutorial

      • BinderHub UI

        • mobile friendly, get inspiration from PyPI

        • material UI as visual toolkit?

      • repo2docker PR triage fest

        • dealt with old PRs 🎉

      • BinderHub-button-for-GKE

        • one click deployment!!

      • Discussed persistent BinderHub

        • GESIS have a working demo!

      • Azure pipelines v1 for repo2docker

      • Recognised the need for “start a binder from an expired binder session”

        • ideas for how to do it, difficulty level “tricky”

      • Many Z2JH PRs viability discussed, as documented in issues

      • some called it “The annual workshop” ;)

    • [name=Kirstie]: What was the biggest thing that the team didn’t get to at the workshop?

      • Didn’t get to release some of the packages - there are a few that are ready to go but we didn’t get to that.

      • The changelog is kind of a blocker on this - a clearer process on what is needed for a release would make this easier.

        • Would be cool to make it clearer for people who aren’t working from master what will change when they update to a new release.

      • (Kirstie: As a bit of background on this question: I think its often hard to balance the high level discussions and the actual “doing” of things like updating code etc. I’m planning a sprint for a different project at the moment and I can feel these tensions in the planning!)

        • focussing on discussions of particular issues, clearing old PRs, and then discussing small bits of code was nice (Tim)

  • Share your canned responses, Tim, 10minutes

    • GitHub lets each user have “prepared responses” for issues and threads

    • we often have to reply to similar issues (wrong repo, needs more details, out of scope, angry user, welcome a user, is this PR stalled, etc)

    • At the team meeting in Oslo we discovered that people already have cheat sheets that they use

    • I’d be interested in creating a team cheat sheat that all of us can crib from to reduce the effort required (especially for tricky situations)

      • [name=Sarah] “Unfortunately, mybinder.org doesn’t support FEATURE, but you can enable this by deploying your own BinderHub! link to docs

    • Tim will create a hackmd and start adding ones he uses, others welcoms, others welcome to add theirs and reuse them

    • Maybe not copy&paste each response but customise it when you use it

      • add your own touch to it and maybe also customise it when you use it for that particular situation

  • Gesis joining the Binder federation (Tim)

    • deployment not from mybinder.org-deploy

    • use a bot to keep up to date (similar to henchbot)

    • assign gesis.mybinder.org as sub-domain

    • @bitnik as contact person

    • currently at 1% of mybinder.org traffic

      • goal capacity?

      • [name=Kenan] 1% -> 5% -> 10% -> 20%

    • [name=Kenan] We should remove or bring to the default buttons in Noteboook UI, right?

      • one thing to work on is to make them work better on mobile

      • maybe implement them as a Jupyter extension instead of via repo2docker appendix

    • decision

      • probably yes

    • [name=Erik] Question: will the user experience be remain similar?

      • Important that the experience for visitors to mybinder.org is seamless - don’t want them to have different usage experiences depending on where they’re assigned.

      • There’s a minor customisation at the moment, but agree that there’s a little difference (buttons in top right corner of notebook, see above)

      • how do we keep the instances in sync?

        • repo2docker updates are straight forward

        • BinderHub updates are easy but major changes (in particular HTML templates) take some time

        • if a member of the federation is “too far behind” we automatically stop sending traffic

    • Test: https://notebooks-test.gesis.org

  • Turing joining the Binder federation (Tim)

    • deployment from mybinder.org-deploy

    • assign turing.mybinder.org and hub.turing.mybinder.org (for jhub)

    • @sgibson91 as contact person

    • start off with 1% of traffic to debug then ramp up

      • goal capacity?([name=Sarah] 10-20%?)

        • [name=Kirstie] I think following the Gesis plan should work :smile:

        • 1% -> 5% -> 10% -> 20%

        • We have a budget of $10,000 for the next year. I think if we can show good usage and impact we would be able to find some more in the future.

    • decision

      • probably yes

    • [name=Kirstie]: What comms should we have for this?

      • The Turing comms team would like very much to be involved and they tend to need a decent amount of lead time.

      • Will we do a flying under the radar start off? And then launch once we’ve figured out some bugs?

      • a “launch event” would be good for Binder, Turing, etc in terms of attention,

      • a blog post on the jupyter blog

      • Turing will have their own but their post will be very short

    • [name=Kirstie] Re:comms - I think its useful to think about why we want to have these comms - for Turing it’s definitely showing that they’re investing in the most innovative and exciting data science. For Jupyter I think it might be about bringing in more contributors? More operators? More users? Money :moneybag:? What about for Gesis?

      • Tim thinks this is a very good input!

      • Kirstie’s happy to hack on this in a side channel (issue, hackmd) :smile_cat:

    • Our application for funding, for reference: https://hackmd.io/SxF25aeRRCS1ZptyShYxtg?view#Research-Justification

  • Eventlogging in JupyterHub. Min to the rescue :smiley: (THANK YOU!!!) [name=Zach, 5 min]

    • Merge?

    • ~~Came across a confusing issue in JupyterHub tests:~~

      • ~~unit tests affect each other; maybe not properly torn down?~~

      • ~~MockHub instances are the likely culprit.~~

  • Hubshare - 10min.

    • Should we move forward with this project?

      • Mixed signal across issues in HubShare.

        1. Implement HubShare using WebDAV for storage and a REST API on top. hubshare#12

        2. Just use other existing tools. hubshare#14

    • Explored using WebDAV for storage API (see this branch).

    • Question: What is the decision process for this? How do we balance time on this vs time on something else?

      • In general, its a very cool idea!

    • Unhelpful comment: (Kirstie likes the current README comment “Important: This repo is in early development (it doesn’t work).” :laughing:)