JupyterHub HPC Meeting - February 2021#

Welcome to the Meeting#

Hello! If you are joining the team video meeting, sign in below so we know who was here. Roll call:

  • name / institution / GitHub handle

  • Rollin / NERSC / @rcthomas

  • Michael / MSI @ UMN / @mbmilligan

  • Kevin / NCAR / @kmpaul

  • Félix-Antoine / Université Laval / @cmd-ntrf

  • Richard / Aalto University

  • Jeffrey / ORNL / @millerjl1701

Quick updates#

60 second updates on things you have been up to, questions you have, or developments you think people should know about. This is also a chance to suggest a future presentation if you’ve got work currently in progress you might want to share. Please add yourself, and if you do not have an update to share, you can pass.

  • NAME: What you’d like to update on

  • Michael MSI rolled out a big Jupyterhub update this month! PBS -> Slurm, Hub (something ancient) -> 1.1, Puppet 3 -> Puppet 5

  • Kevin: Starting work on an AWS deployment of a dev/testing platform for HPC services, based on what we’ve learned from Magic Castle. (Thanks for the suggestion! Reach out if you want at kpaul@ucar.edu.)

  • Félix-Antoine: E-CAM is working on a project named LearnHPC. I will invite the project PI Alan O’Cais to our next meeting.

Reports and celebrations#

This is a place to make announcements (without a need for discussion). This is also a great place to give shout-outs to contributors! We’ll read through these at the beginning of the meeting.

  • Kevin: Thanks everyone for the suggestions to look at Magic Castle!!!

Agenda items#

Let’s collect all potential agenda items here before the start of the meeting. We will then attempt to create a coherent agenda that fits in the 60m meeting slot. If there are similar items try and group them together.

  • JupyterLab 3 migrations

    • Rollin/NERSC

      • Dev deployment running, planning to deploy in March

      • Notes:

        • Change cmd to jupyter-labhub

        • Add JUPYTERHUB_SINGLEUSER_APP to environment as “jupyter_server.serverapp.ServerApp”

          • Needed for some labextensions

          • Is the future default

        • Centrally-deployed labextension migrations:

          • jupyterlab-toc went to core

          • ipympl/matplotlib is just ipympl via pip/conda

          • plotly+plotlywidgets stays the same (labextension install)

          • jupyterlab-git is waiting on nbdime

          • nbresuse -> jupyter-resource-usage

            • Reliable way to get the upper limit?

          • local projects: jupyterlab-{recents,favorites,slurm} will be migrated

        • User labextensions:

          • Some users are trying to test theirs, may have reports…

    • Felix: JupyterLab-lmod extension seemed to work without modification

      • Is there any plan or supporting 2 and 3 at the same time?

      • Doesn’t seem to be anything there.

    • Spencer@ORNL: Acceptance testing going on now

      • Config changes as above

    • Other?

  • Batchspawner check-in

    • In December, discussed a new release

    • This month, had a bunch of PRs merged by Erik

    • All things that would have been merged

    • Mike asks does it look like BatchSpawner is in a good place for release?

      • No objections heard

      • Bleeding-edge batchspawner is running at MSI OK

      • Enough features accumulated so the minor version can increment => 1.1

      • Mike says he’ll do it this month!

  • Wrapspawner check-in

    • Release pending

      • Haven’t ever done one but are in a good place to do it

    • Do we need release notes / other documentations?

      • Is README OK for release? Yes

    • Version number?

      • 1.0 this month

    • No action on Rollin’s issue with environment settings (close)

    • Issue with auth_state stuff:

      • Issue with remote authenticator

      • Login form never shows so hooks don’t fire

      • Had to hack around and uses all the hooks

      • The auth state hook fires on auto update

  • Jupyter Docker stacks?

    • Dependency hell?

    • Talking about automating builds

    • repo2docker at the system level of customizations?

    • mpi4py for instance needing fairly recent MPICH, user can’t add onto that afterward unless they’re careful