JupyterHub and BinderHub Team Meeting - May

Meeting details

Date: Thursday, 16th May 2019 at 5pm UTC Videoconference link: https://calpoly.zoom.us/my/jupyter

The call will start with a quick round of updates from everyone present (you can pass if you want to), followed by bigger agenda items. There will be no feedback/questions on the 60s updates. Questions/discussion on the larger agenda items will be moderated by a speaker list (see end of this hackmd).

Agenda items

Let’s collect all potential agenda items here before the start of the meeting (closing time 24h before). We will then attempt to create a coherent agenda that fits in the 60m meeting slot. If there are similar items try and group them already.

  • Add agenda item here [name, estimated time for conversation]

Welcome to the Team Meeting

Hello!

If you are joining the team video meeting, sign in below so we know who was here. Roll call:

  • name / institution / GitHub handle

  • Min / Simula / @minrk

  • Chico Venancio / BMCG KK / @chicocvenancio

  • Chris Holdgraf / Jupyter@Berkeley / @choldgraf

  • Erik Sundell / Sandvik / @consideratio

  • Sarah Gibson / Alan Turing Institute / @sgibson91

  • Georgiana Dolocan / Simula / @GeorgianaElena

  • Joe Hamman / NCAR / @jhamman

  • James Colliander / PIMS / @colliand

  • Ian Allison / PIMS / @ianabc

  • Lindsey Heagy / UC Berkeley / @lheagy

Speaking queue

If you have a question, comment, or thought about the current agenda topic being discussed, please add your name to checklist below. The meeting moderator will call on you when it’s your turn in the queue. The list will reset each time we move to a new topic. Thanks!

  • [ ] name

  • [ ]

Quick updates

60 second updates on things you have been up to, questions you have, or developments you think people should know about. Please add yourself, and if you do not have an update to share, you can pass.

  • name: update or pass

  • Min: update (Georgiana working on Traefik)

  • Chris: binder talk @ csvconf, voila docs, new repo2docker version

  • Sarah: Question about contribution from/collaboration with Microsoft team

    • Folks I know of (CH)

      • Tania Allard

      • Safia Abdalla

  • Erik: update (jhub templates, repo2docker Pipfile, z2jh autoscaling cost simulator, pangeo curious)

  • Joe: two new Pangeo binders on aws, lots of k8s optimizations for our Jhubs, and a blog post.

Reports and celebrations

This is a place to make announcements (without a need for discussion). This is also a great place to give shout-outs to contributors! We’ll read through these at the beginning of the meeting.

  • add item here

  • Thanks to Kirstie Whitaker for adding issue templates in #655

  • Thanks to Kirstie + Bastian Greshake Tzovaras for giving JupyterHub + binder a shoutout in their CSVConf talks. Talk

  • Erik: Thanks to Simon Li (@manics) for being a friendly and helpful presence and also coding for multiple JupyterHub related repositories!

  • Erik: Thanks to Sarah Gibson (@sgibson91) for many additions to documentation in BinderHub

  • @GeorgianaElena will be continuing her work on JupyterHub + Traefik this summer through a contract with Simula

  • Chico Venancio will be working on behalf of BMC KK for Swiss RI Institute to customize binderhub for their needs. Initial work with AWS CodeCommit and ECR (Chico might not be present, see next note)

  • Chico is travelling to Wikimedia Hackathon and will present about PAWS, a JupyterHub deployment for Wikimedia.

  • What does it cost to run Pangeo? Joe wrote a blog post.

    • Note: inspired from Joe’s costs blog post, Yuvi started auto-publishing our mybinder.org costs here: https://archive.analytics.mybinder.org/cloud-costs.jsonl

    • Erik: me and a colleague is working towards a z2jh autoscaling cloud cost simulator. See this.

Agenda items

This is what we’ll cover in the meeting (we have about 60m in total). We’ll copy the proposed agenda topics above here just before the meeting.

  • Add agenda item here [name, estimated time for conversation]

  • Discuss adopting Black for repo2docker (or other jupyterhub repos). (5-10 min for discussing what it’s been like in JupyterHub or other projects, then a quick show of hands)

    • Min: the point is that it removes the style discussion

    • Erik: Q: git pre-commit hook? - runs locally

      • e.g. linters, autoformatting, etc

      • pre-commit: package you install to manage hooks

      • getting started?

        • install dev-requirements

      • benefits of pre-commit hook

        • If the repository changes its pre-commit hooks, this will automatically happen for each person that commits on that repo

        • However this is not strictly necessary, you can run Black from a code editor, and we’d add a Black validation step to the CI/CD

        • TODO: Chris H opens an issue about Black documentation and usage in jupyterhub (now here)

  • CZI has a call for funding for open source projects. Is anybody interested in pursuing this on behalf of JupyterHub or Binder? (not sure how long this would take, depends on interest from folks but may require input from folks not present)

    • scoped at a year with the possibility to renew

    • ~ 1 salary for 1 year based on previous grants

    • TODO: issue on team compass repo (now here)

  • PIMS operates a federated JupyterHub service (syzygy.ca) that currently serves 20 universities and several research teams.

    • universities interested in service level agreements

    • vanilla deployments on Compute Canada

    • Canadian not-for-profit to manage the hubs and take over role of PIMS in managing deployments of hubs

    • funds available: Millions - 10’s of Millions

    • interested in international collaborations

    • tentative name: 2i2c - International Interactive Computing Collaboration

    • scale: ~100 simultaneous users at a given university, 21 uiversities,

      • 15000 users

      • 5000 users @ UBC, maximum simultaneous load ~10%

      • asking to scale towards 1000 simultaneous users

    • tech challenges: different auth systems

    • QuantEcon team connections

    • Bespoke hubs

      • SFU: ireceptor - genomics research

      • UBC collaboration with industry, with NDA needed

    • models to look to: international collaborations on telescopes, particle accelerators

      • different pooling of resources

    • timeline?

      • pedagogical demands @ UBC - happening now

      • CFI: call for proposals happening over the summer

      • New Frontiers: has been announced, but hasn’t launched - expected launch in the summer, ~ Sept / Oct deadline

    • $572 million to rebuild Compute Canada