JupyterHub HPC Meeting - December 2023#

Welcome to the Meeting#

Hello! If you are joining the team video meeting, sign in below so we know who was here. Roll call:

  • name / institution / GitHub handle

  • Rollin Thomas / NERSC / @rcthomas

  • Matt Henderson / LBNL / @mlhenderson

  • Michael Milligan / MSI @ UMN / @mbmilligan

  • Ben Winjum / UCLA / @benjum

  • Félix-Antoine Fortin / Université Laval / @cmd-ntrf

Quick updates#

Things you have been up to, questions you have, or developments you think people should know about. This is also a chance to suggest a future presentation if you’ve got work currently in progress you might want to share. Please add yourself, and if you do not have an update to share, you can pass.

  • Name: Your update

Agenda items#

Let’s collect all potential agenda items here before the start of the meeting. We will then attempt to create a coherent agenda that fits in the 60m meeting slot. If there are similar items try and group them together.

  • All: Standing project items:

    • Batchspawner check-in, issues and PRs

      • release ready as of last month - still puzzled by new release procedures

      • will give it another try this week, otherwise will release manually to get changes published

    • Wrapspawner check-in, issues and PRs

    • Discourse

  • Workshop/conference news:

    • KubeCon NA Chicago Nov. 2023

      • November 6-9

      • HPC track

      • Report?

    • SC23

      • HPPSS High Performance Python for Science at Scale

        • 4 demo talks

        • Panel: NVIDIA RAPIDS, Siu Lam (Numba), Travis Oliphant, Torsten Hofler, Fernanda Foertter

      • Interactive and Urgent HPC BOF

      • Jupyter in the background

      • Magic Castle Tutorial went well, maybe 17 participants

      • Rollin talked to Mathworks about their Matlab integration and we’ll be testing it out

    • Others?

  • Demo - JupyterLab extension for Shutdown Alerts

    • Matt H.

    • A very in-your-face mechanism for getting users’ attention that the hardware you’re on is getting rebooted

    • Feedback — does this sound OK, are there other ways people want to configure this?

      • Can we configure the message? Yes.

      • Could we use the same banner to alert the user that their job was about to run out of time in BatchSpawner

        • Generalize the LabExtension at the point where you need to decide it’s time to post the banner?

        • You could periodically ask Slurm how much time you’ve got left?

        • Could send a signal before the job ends, trap that and post.

        • W/Slurm a simple approach would be trap a SBATCH --signal=... that touches the trigger file

    • Works for Lab 3 and Lab 4, code’s a little different, so there will be 2 releases

  • Ben from UCLA

    • Several Hubs deployed on Google Cloud following z2jh w/k8s

    • Bare metal on sandbox

    • HPC cluster?

      • Users can use a shell script to do an ssh tunnel

      • Sets up an interactive job for them

      • More convenient to set up a JupyterHub?

      • UGE scheduler

    • Where can the Hub run and how is that hardware configured

    • Security