JupyterHub HPC Meeting - October 2021

  • Date: 2021-10-06

  • Time: 8:30 AM PDT

    • Your timezone: https://arewemeetingyet.com/Los%20Angeles/2021-10-06/08:30/JupyterHub-HPC

  • GitHub issue:

  • Calendar for future meetings: https://jupyterhub-team-compass.readthedocs.io/en/latest/meetings.html

Welcome to the Meeting

Hello! If you are joining the team video meeting, sign in below so we know who was here. Roll call:

  • name / institution / GitHub handle

  • Aric Werner / NCAR / @aawerner

  • Félix-Antoine Fortin / Université Laval / @cmd-ntrf

  • Simon Li / University of Dundee / @manics

  • Richard Darst / Aalto University / @rkdarst

  • Zach Price / ORNL /

  • Michael Milligan / MSI - U of Minnesota / @mbmilligan

  • Rick Wagner / UCSD / @rpwagner

Quick updates

60 second updates on things you have been up to, questions you have, or developments you think people should know about. This is also a chance to suggest a future presentation if you’ve got work currently in progress you might want to share. Please add yourself, and if you do not have an update to share, you can pass.

  • Name: Your update

Reports and celebrations

This is a place to make announcements (without a need for discussion). This is also a great place to give shout-outs to contributors! We’ll read through these at the beginning of the meeting.

  • Name: Your report or celebration

Agenda items

Let’s collect all potential agenda items here before the start of the meeting. We will then attempt to create a coherent agenda that fits in the 60m meeting slot. If there are similar items try and group them together.

  • Aric: are there any other HPC sites using Cylc/CylcHub, and are you able to integrate this into your existing JupyterHub instance?

    • NERSC has had one request, evaluating as one of several approaches to workflow management, need to decide and set expectations on support

    • Integrating other apps besides Jupyter into JupyterHub has come up a few times, another example is MatLab, being able to start MatLab sessions from Jupyter/JupyterLab (a little bit different jupyter-server-proxy); what about license server?

    • Mathworks jupyter-matlab-proxy: https://github.com/mathworks/jupyter-matlab-proxy

    • RStudio started putting things in that make it harder to launch from JupyterHub, seem to be going their own way; network isolation

  • Félix-Antoine: do we know anyone at NCSA interested in JupyterHub and JupyterHub for HPC?

    • Kay Avila kayavila@illinois.edu

  • Zach: JupyterHub Services suggestions? We’ve deployed VSCode to good reception, looking for other suggestions

    • Deployment is through k8s, each container has ssh access to submit HPC jobs, they don’t run notebooks on the cluster (but they cross mount filesystems).

    • ARM deployment:

      • https://api.github.com/repos/cdr/code-server/releases/latest

      • https://github.com/dirkcgrunwald/jupyter_codeserver_proxy-.git

      • UX:

    • https://www.gitpod.io/blog/openvscode-server-launch

    • https://github.com/betatim/vscode-binder

    • Matlab, TensorFlow

    • JupyterLab-Slurm, NVDashboard

  • Mike question about OOD vs JupyterHub:

    • Main discussion point was around the breadth and depth of the JupyterHub community and how it represents leverage to HPC deployments

    • Are there places that have done both? None come to mind due to burden of support issues.

    • Skill set difference between the two (e.g. Python vs Rails) for your staff

    • Put together a couple of slides for the next convenient Hubs meeting (is there a role for Hub to become generalized proxy or is that out of scope), also potentially have OOD

    • what about improving documentation of tools like the session proxy (from the point of view of computing centers). I have used it for several things, but it is magic to me. And some sort of general list of “what all can be proxied through jupyter” ?

    • What about inviting the OOD/OSC people to come to this meeting to present, to hear from them?

    • https://github.com/ideonate/jhsingle-native-proxy is something else you might be interested in, instead of being a proxy run inside jupyter it replaces the singleuser-server

    • Just thought of another cool example of a proxied app https://github.com/jupyterhub/jupyter-remote-desktop-proxy/

  • Jupyter Community Workshop on Security:

    • OSC had agreed to host April/May but…

    • Current sense about state of travel and permission to travel is very uncertain still

    • May shift to doing something online/virtual

    • In any case goals for the workshop are making progress in some sense…

  • All: Standing project items:

    • Batchspawner check-in: Issues and PRs

    • Wrapspawner check-in: Issues and PRs