JupyterHub HPC Meeting - February 2021

  • Date: 2021-02-03

  • Time: 8:30 AM PST

    • Your timezone: https://arewemeetingyet.com/Los%20Angeles/2021-02-03/08:30/JupyterHub-HPC

  • GitHub issue:

  • Calendar for future meetings: https://jupyterhub-team-compass.readthedocs.io/en/latest/meetings.html

Welcome to the Meeting

Hello! If you are joining the team video meeting, sign in below so we know who was here. Roll call:

  • name / institution / GitHub handle

  • Rollin / NERSC / @rcthomas

  • Michael / MSI @ UMN / @mbmilligan

  • Kevin / NCAR / @kmpaul

  • Félix-Antoine / Université Laval / @cmd-ntrf

  • Richard / Aalto University

  • Jeffrey / ORNL / @millerjl1701

Quick updates

60 second updates on things you have been up to, questions you have, or developments you think people should know about. This is also a chance to suggest a future presentation if you’ve got work currently in progress you might want to share. Please add yourself, and if you do not have an update to share, you can pass.

  • NAME: What you’d like to update on

  • Michael MSI rolled out a big Jupyterhub update this month! PBS -> Slurm, Hub (something ancient) -> 1.1, Puppet 3 -> Puppet 5

  • Kevin: Starting work on an AWS deployment of a dev/testing platform for HPC services, based on what we’ve learned from Magic Castle. (Thanks for the suggestion! Reach out if you want at kpaul@ucar.edu.)

  • Félix-Antoine: E-CAM is working on a project named LearnHPC. I will invite the project PI Alan O’Cais to our next meeting.

Reports and celebrations

This is a place to make announcements (without a need for discussion). This is also a great place to give shout-outs to contributors! We’ll read through these at the beginning of the meeting.

  • Kevin: Thanks everyone for the suggestions to look at Magic Castle!!!

Agenda items

Let’s collect all potential agenda items here before the start of the meeting. We will then attempt to create a coherent agenda that fits in the 60m meeting slot. If there are similar items try and group them together.

  • JupyterLab 3 migrations

    • Rollin/NERSC

      • Dev deployment running, planning to deploy in March

      • Notes:

        • Change cmd to jupyter-labhub

        • Add JUPYTERHUB_SINGLEUSER_APP to environment as “jupyter_server.serverapp.ServerApp”

          • Needed for some labextensions

          • Is the future default

        • Centrally-deployed labextension migrations:

          • jupyterlab-toc went to core

          • ipympl/matplotlib is just ipympl via pip/conda

          • plotly+plotlywidgets stays the same (labextension install)

          • jupyterlab-git is waiting on nbdime

          • nbresuse -> jupyter-resource-usage

            • Reliable way to get the upper limit?

          • local projects: jupyterlab-{recents,favorites,slurm} will be migrated

        • User labextensions:

          • Some users are trying to test theirs, may have reports…

    • Felix: JupyterLab-lmod extension seemed to work without modification

      • Is there any plan or supporting 2 and 3 at the same time?

      • Doesn’t seem to be anything there.

    • Spencer@ORNL: Acceptance testing going on now

      • Config changes as above

    • Other?

  • Batchspawner check-in

    • In December, discussed a new release

    • This month, had a bunch of PRs merged by Erik

    • All things that would have been merged

    • Mike asks does it look like BatchSpawner is in a good place for release?

      • No objections heard

      • Bleeding-edge batchspawner is running at MSI OK

      • Enough features accumulated so the minor version can increment => 1.1

      • Mike says he’ll do it this month!

  • Wrapspawner check-in

    • Release pending

      • Haven’t ever done one but are in a good place to do it

    • Do we need release notes / other documentations?

      • Is README OK for release? Yes

    • Version number?

      • 1.0 this month

    • No action on Rollin’s issue with environment settings (close)

    • Issue with auth_state stuff:

      • Issue with remote authenticator

      • Login form never shows so hooks don’t fire

      • Had to hack around and uses all the hooks

      • The auth state hook fires on auto update

  • Jupyter Docker stacks?

    • Dependency hell?

    • Talking about automating builds

    • repo2docker at the system level of customizations?

    • mpi4py for instance needing fairly recent MPICH, user can’t add onto that afterward unless they’re careful