JupyterHub and BinderHub Team Meeting - January

Date: 17 January 2019 at 6pm Zurich time

Videoconference link: https://calpoly.zoom.us/my/jupyter Link to prior meeting’s virtual meeting report

Welcome to the Team Meeting

Hello!

If you are joining the team video meeting, sign in below so we know who was here. Roll call:

  • name / institution / GitHub handle

  • Zach / Cal Poly / @Zsailer

  • Tim / WTT / @betatim

  • Carol / Project Jupyter / @willingc

  • Craig Willis / NCSA / @craig-willis

  • Joe Hamman / NCAR-Pangeo / @jhamman

  • Yuvi Panda / UCB / @yuvipanda

  • Lindsey Heagy / UCB / @lheagy

  • Kacper Kowalik / NCSA / @xarthisius

  • Brian / Cal Poly / @ellisonbg

Agenda items

Let’s collect all potential agenda items here before the start of the meeting (closing time 24h before). We will then attempt to create a coherent agenda that fits in the 60m meeting slot. If there are similar items try and group them already.

  • Resources definitions/requests by the repository creator or repository launcher, following up on December 2018 [name=@jhamman]

    • relevant issues: https://github.com/jupyterhub/binderhub/issues/731, https://github.com/jupyterhub/binderhub/issues/759, https://github.com/jupyterhub/binderhub/issues/712

    • Relevant questions

      • What resource fields do we need?

        • Can use the Kubernetes docs as a reference (since ultimately we’ll be passing the values of these fields to k8s)

    • check if we can read docker image LABELs without pulling the image? If yes then storing resource requests in the repo and repo2docker will store it in LABELs

    • Q from Craig: Why encode things in the URL parameter, why not allow them to be encoded in the repository itself (e.g., so an author could define the resources needed for a repo to run)

      • The URL parameter stuff wouldn’t preclude this from being possible - one idea is to let authors store this in the repository, and set some kind of “defaults” in the built docker image than could be overridden by URL parameters.

  • Rust End of year thoughts/what is next style process, see https://discourse.jupyter.org/t/end-of-year-thoughts-from-the-rust-community/295/7?u=betatim

    • propose to create a post on the discourse forum asking for blogs/tweets/gists/replies for the next four(?) weeks which then get compiled into the Binder Roadmap for the next 12months. [name=@betatim]

    • When would we start this?

      • Maybe after dev meeting in March.

    • Make it clear that this is about getting input and ideas, taking the temperature, not about formal proposals and deliverables

    • How do we collect suggestions from the many different mediums?

      • Use discourse as the primary list and allow others to link to their ideas from there.

    • Reach out to the Rust community to ask for adivce on mechanics and such

      • Tim contacts Luiz Irber (Determine time to produce, resources that did the work re: paid/volunteer)

    • If you don’t have a highly structured format for such ideas, it becomes difficult to evaluate suggestions from the community. Maybe suggest a format/questions to make it easier to digest?

  • JupyterHub Helm Chart v 0.8 nearly ready to go [name=@choldgraf]

    • Has been about 7 months since last release

    • Only remaining issue is to find a list of the JupyterHub versions for each helm chart. Does anybody know this? (https://github.com/jupyterhub/zero-to-jupyterhub-k8s/issues/1094)

    • Need to update the change log.

    • Anything else to be done?

  • (no need to discuss) Chris would like help putting together a “how to spot a bitcoin miner” page in the SRE guide. He opened an issue to put together a list of tips (https://github.com/jupyterhub/mybinder.org-deploy/issues/879)

    • +100 Tim plans to write what he knows in the PR with the “find high CPU pods and processes”

  • (no need to discuss) Chris would like help finishing the BinderHub@NeurIPS post: https://hackmd.io/ZEVbmiWHQiOuRUQzRmjnNg?edit

  • Graphics, diagrams, drawings, “zines”

    • Tim is commissioning work to help explain what Binder is, how to use it, etc +100

    • Will probably select a small agency in Germany based on recommendations

    • They will pitch a concept mid February

    • Ideas for what kind of artefacts would be useful for the project welcome

    • Need to figure out how to integrate this into the normal GitHub based workflow. I don’t think they’ve ever worked with an open-source project like us.

      • Figma, is like sketch but web-based good for collaborating and reviewing work

    • Cool binder drawing: https://pbs.twimg.com/media/Dwe4jVdU0AA2Ss-.jpg:large

    • Apparently figma and slack paid the same design firm for their logo :-)

      • https://i.imgur.com/LI1VgZs.png

      • https://i.imgur.com/F9vZuVC.png

    • Three one pagers

      • architecture and ecosystem for JupyterHub

      • architecture and ecosystem for BinderHub

      • how does it work and ecosystem for mybinder.org

  • repo2docker.version PR check-in - Craig

    • Is there anything useful to discuss that isn’t covered on Github?

    • https://github.com/jupyter/repo2docker/pull/550

    • use local repo2docker to fetch source

    • local repo2docker starts a containerised repo2docker-in-specified-version with a local directory content provider

Items added after the cut-off will be in the “bonus material” section. We will go through the ordinary agenda items first and then cover bonus items if the clock hasn’t run out.

Reports

Things people should know about.

  • Cost per student for Berkeley’s hub (of 4.5k students) is between 1$ / month / user to 2$ / month / user based on how active a student is before you consider them a ‘user’. Open (anonymized) data + cool graphs at https://github.com/berkeley-dsep-infra/datahub-usage-analysis/blob/master/notebooks/03-visualize-cost-and-usage.ipynb. Mean daily active users is 1006 students! (Yuvi)

    • how does the cost scale if your cluster has less users?

      • if I only have 100 students, do I still only pay $1 per student or more because there are “fixed” costs

  • Yuvi working on getting Space Telescope Sciences Institute to buy out some of his time at Berkeley to focus more on JupyterHub / BinderHub

  • RStudio has explicitly granted us trademark permission on mybinder.org

Our next meeting is February 21st, 2019 at 6pm Zurich time.