JupyterHub and BinderHub Team Meeting - July

Goal: Continuous improvement

Date: 19th July 2018

Welcome to the Team Meeting

Hello!

If you are joining the team video meeting, sign in below so we know who was here. Roll call:

  • name / institution / GitHub handle

  • min / simula / @minrk

  • tim / wild tree tech / @betatim

  • erik / sandvik code / @consideRatio

  • j forde / @jzf2101

  • Brian / Cal Poly / ellisonbg

  • Chris / UC Berkeley / @choldgraf

Actions

  1. Help or discussion needed; Agenda Items for Monthly Meeting 1.1. Open PRs 2. #758, node pools, k8s 1.9+, scheduler, etc… 3.2. Open issues 4. last call for camera version of icml workshop writeup plz sign up on openreview http://openreview.net/

  2. Decisions needed

    • betatim

      • what if I was to receive $20000 (or more) in cloud credits, how could they go to running mybinder.org?

        • would require juggling billing accounts

        • a reason to get started with federation so we can handle this under the hood?

        • reasons not to take credits on other platforms is that it adds complexity that costs human effort

        • jzf says related: grant opportunities for additional funding

        • add a public demo of binderhub with more resources to convince people we can do more to encourage them to deploy their own (fancy.mybinder.org)

          • maybe do impressive demos like pangeo where users can spin up big clusters

        • Jessica submits a proposal for AWS credits in the next few days for the demo of fancy.mybinder.org

        • “ai.mybinder.org” in order to host fancier compute and highlight machine learning use-cases

          • can this grant also include some engineering time from SageMaker/AWS team?

          • geo.mybinder.org, yourdiscipline.mybinder.org as a way to expand this idea

        • create a call for hosting mybinder.org, wait till after AWS grant

          • if you are against this write here

          • like this idea of a call, but we might want to make it non-exclusive, to allow multiple vendors to participate.

      • how do we try out https://cloud.google.com/compute/docs/instances/preemptible to see how disruptive it would be to users and how much we would end up saving?

        • Tim will create an issue to follow this up

        • One idea: split users between two nodepools once we have the ability nodepool

      • Wild Tree Tech will start saying on their webpage that we offer hosted/managed JupyterHub and BinderHub instances

    • choldgraf

      • I’d like to get some momentum behind the “binder sustainability” conversation. Or at least to recap what we discussed @ the dev meeting.

      • Update on the scipy presentation: http://bit.ly/scipy-2018-binder

        • start using the SciPy paper once it is out instead of the eLife blog post?

        • what about the JOSS paper that is stalled? Waiting for 0.1

      • @jzf2101 should give an update on the reproducible environments paper she presented!

        • if you want your name on this sign up for openreview.org

      • Update on the PEARC conference tutorial that we’ll be giving in pittsburgh

        • link to agenda: https://pearc18.conference-program.com/?page_id=10&id=tut149&sess=sess103

        • workshop will focus on using it on Jetstream and ComputeCanada instead of commercial clouds

      • Cost model + projections for our $$$ burn rate

        • we aren’t sure how long we have before we run out of money

        • we give 1GB of RAM at least, 10min idle time out, max run time 8h, scaling is determined by RAM required not CPU needs

      • Design: https://github.com/jupyterhub/design

        • we have been using the wrong logo all along!!!11!

    • consideRatio

    • minrk

      • we should try adopting the multi-node-pool + nodeAffinity deployment strategy, which may help a lot with scale-down (harder to say without packing for Binder’s round-the-clock visitors, but works great for users in a single timezone)

        • ERIK: This is what #758 is about, in an optional but recommended manner.

    • ellisonbg

      • Zach Sailer will be starting full time at Cal Poly around August 6th.

      • He will be working mostly full time on JupyterHub things, starting with better integration with directory services (AD, LDAP, etc.) and the idea of “projects” for JupyterHub.

  3. Next actions (team)

    • concrete actions

News, Information, and Thanks

  1. Team Metrics

  2. Team News and Informational items

  • 5.1 Organization highlights

  • 5.2 BinderHub projects

    • https://gitlab.com/gitlab-org/gitlab-ce/issues/48236 Tim isn’t sure what exactly they are planning but I made a suggestion for what I think would be cool.

  • 5.3 JupyterHub projects

  • 5.4 Related projects (repo2docker, nbgrader, others)

  1. Thanks, Things to Celebrate, and anything else

  • We got some shout outs in the CERN High School teacher summer school https://indico.cern.ch/event/651996/contributions/2994003/attachments/1689835/2718654/WG5_Open_Data_Presentation.pdf and their repo https://github.com/cms-opendata-education/HST-2018

  • Tim will be speaking about Binder at PyconDE in October