JupyterHub HPC Meeting - December 2023#
Date: 2023-12-06
Time: 8:30 AM PST
Where you can find these and past meeting notes: https://jupyterhub-team-compass.readthedocs.io/en/latest/meetings/hpc/
Calendar for future meetings: https://jupyterhub-team-compass.readthedocs.io/en/latest/meetings/
Jupyter Code of Conduct: https://jupyter.org/governance/conduct/code_of_conduct.html
Welcome to the Meeting#
Hello! If you are joining the team video meeting, sign in below so we know who was here. Roll call:
name / institution / GitHub handle
Rollin Thomas / NERSC / @rcthomas
Matt Henderson / LBNL / @mlhenderson
Michael Milligan / MSI @ UMN / @mbmilligan
Ben Winjum / UCLA / @benjum
Félix-Antoine Fortin / Université Laval / @cmd-ntrf
Quick updates#
Things you have been up to, questions you have, or developments you think people should know about. This is also a chance to suggest a future presentation if you’ve got work currently in progress you might want to share. Please add yourself, and if you do not have an update to share, you can pass.
Name: Your update
Agenda items#
Let’s collect all potential agenda items here before the start of the meeting. We will then attempt to create a coherent agenda that fits in the 60m meeting slot. If there are similar items try and group them together.
All: Standing project items:
Batchspawner check-in, issues and PRs
release ready as of last month - still puzzled by new release procedures
will give it another try this week, otherwise will release manually to get changes published
Wrapspawner check-in, issues and PRs
Discourse
Workshop/conference news:
KubeCon NA Chicago Nov. 2023
November 6-9
HPC track
Report?
SC23
HPPSS High Performance Python for Science at Scale
4 demo talks
Panel: NVIDIA RAPIDS, Siu Lam (Numba), Travis Oliphant, Torsten Hofler, Fernanda Foertter
Interactive and Urgent HPC BOF
Jupyter in the background
Magic Castle Tutorial went well, maybe 17 participants
Rollin talked to Mathworks about their Matlab integration and we’ll be testing it out
Others?
Demo - JupyterLab extension for Shutdown Alerts
Matt H.
A very in-your-face mechanism for getting users’ attention that the hardware you’re on is getting rebooted
Feedback — does this sound OK, are there other ways people want to configure this?
Can we configure the message? Yes.
Could we use the same banner to alert the user that their job was about to run out of time in BatchSpawner
Generalize the LabExtension at the point where you need to decide it’s time to post the banner?
You could periodically ask Slurm how much time you’ve got left?
Could send a signal before the job ends, trap that and post.
W/Slurm a simple approach would be trap a
SBATCH --signal=...
that touches the trigger file
Works for Lab 3 and Lab 4, code’s a little different, so there will be 2 releases
Ben from UCLA
Several Hubs deployed on Google Cloud following z2jh w/k8s
Bare metal on sandbox
HPC cluster?
Users can use a shell script to do an ssh tunnel
Sets up an interactive job for them
More convenient to set up a JupyterHub?
UGE scheduler
Where can the Hub run and how is that hardware configured
Security