JupyterHub HPC Meeting - October 2021#
Date: 2021-10-06
Time: 8:30 AM PDT
GitHub issue:
Calendar for future meetings: https://jupyterhub-team-compass.readthedocs.io/en/latest/meetings.html
Welcome to the Meeting#
Hello! If you are joining the team video meeting, sign in below so we know who was here. Roll call:
name / institution / GitHub handle
Aric Werner / NCAR / @aawerner
Félix-Antoine Fortin / Université Laval / @cmd-ntrf
Simon Li / University of Dundee / @manics
Richard Darst / Aalto University / @rkdarst
Zach Price / ORNL /
Michael Milligan / MSI - U of Minnesota / @mbmilligan
Rick Wagner / UCSD / @rpwagner
Quick updates#
60 second updates on things you have been up to, questions you have, or developments you think people should know about. This is also a chance to suggest a future presentation if you’ve got work currently in progress you might want to share. Please add yourself, and if you do not have an update to share, you can pass.
Name: Your update
Reports and celebrations#
This is a place to make announcements (without a need for discussion). This is also a great place to give shout-outs to contributors! We’ll read through these at the beginning of the meeting.
Name: Your report or celebration
Agenda items#
Let’s collect all potential agenda items here before the start of the meeting. We will then attempt to create a coherent agenda that fits in the 60m meeting slot. If there are similar items try and group them together.
Aric: are there any other HPC sites using Cylc/CylcHub, and are you able to integrate this into your existing JupyterHub instance?
NERSC has had one request, evaluating as one of several approaches to workflow management, need to decide and set expectations on support
Integrating other apps besides Jupyter into JupyterHub has come up a few times, another example is MatLab, being able to start MatLab sessions from Jupyter/JupyterLab (a little bit different jupyter-server-proxy); what about license server?
Mathworks jupyter-matlab-proxy: mathworks/jupyter-matlab-proxy
RStudio started putting things in that make it harder to launch from JupyterHub, seem to be going their own way; network isolation
Félix-Antoine: do we know anyone at NCSA interested in JupyterHub and JupyterHub for HPC?
Kay Avila kayavila@illinois.edu
Zach: JupyterHub Services suggestions? We’ve deployed VSCode to good reception, looking for other suggestions
Deployment is through k8s, each container has ssh access to submit HPC jobs, they don’t run notebooks on the cluster (but they cross mount filesystems).
ARM deployment:
Matlab, TensorFlow
JupyterLab-Slurm, NVDashboard
Mike question about OOD vs JupyterHub:
Main discussion point was around the breadth and depth of the JupyterHub community and how it represents leverage to HPC deployments
Are there places that have done both? None come to mind due to burden of support issues.
Skill set difference between the two (e.g. Python vs Rails) for your staff
Put together a couple of slides for the next convenient Hubs meeting (is there a role for Hub to become generalized proxy or is that out of scope), also potentially have OOD
what about improving documentation of tools like the session proxy (from the point of view of computing centers). I have used it for several things, but it is magic to me. And some sort of general list of “what all can be proxied through jupyter” ?
What about inviting the OOD/OSC people to come to this meeting to present, to hear from them?
ideonate/jhsingle-native-proxy is something else you might be interested in, instead of being a proxy run inside jupyter it replaces the singleuser-server
Just thought of another cool example of a proxied app jupyterhub/jupyter-remote-desktop-proxy
Jupyter Community Workshop on Security:
OSC had agreed to host April/May but…
Current sense about state of travel and permission to travel is very uncertain still
May shift to doing something online/virtual
In any case goals for the workshop are making progress in some sense…
All: Standing project items:
Batchspawner check-in: Issues and PRs
Wrapspawner check-in: Issues and PRs