JupyterHub HPC Meeting - May 2021#
Date: 2021-05-04
Time: 8:30 AM PDT
GitHub issue:
Calendar for future meetings: https://jupyterhub-team-compass.readthedocs.io/en/latest/meetings.html
Welcome to the Meeting#
Hello! If you are joining the team video meeting, sign in below so we know who was here. Roll call:
name / institution / GitHub handle
Rollin / NERSC / @rcthomas
Shane / NERSC / @scanon
Zach Price / ORNL
Richard / Aalto / rkdarst
Aric Werner / NCAR / @aawerner
Jared Baker / NCAR / @jbaksta
Rick Wagner / UCSD / @rpwagner
Félix-Antoine Fortin / Université Laval /@cmd-ntrf
Mary Thomas / UCSD, SDSC / @marypthomas
Scott Sakai / UCSD, SDSC / @scottsakai
Jens Henrik Goebbert / FZJ, JSC / @jhgoebbert
Thomas Mendoza / LLNL / @tgmachina
Quick updates#
60 second updates on things you have been up to, questions you have, or developments you think people should know about. This is also a chance to suggest a future presentation if you’ve got work currently in progress you might want to share. Please add yourself, and if you do not have an update to share, you can pass.
NAME: What you’d like to update on
Rick, Rollin Jupyter Community Workshop series is coming back to life, “plans for plans” for security-focused workshop will be coming together soon.
Want to announce dates and place in Sep
Need to make sure people can travel, be in same place when we have workshop
If you have ideas to share on security please feel free to let us know about it by email
Reports and celebrations#
This is a place to make announcements (without a need for discussion). This is also a great place to give shout-outs to contributors! We’ll read through these at the beginning of the meeting.
Wrapspawner release 1.0
Michael is out this month on paternity leave
Agenda items#
Let’s collect all potential agenda items here before the start of the meeting. We will then attempt to create a coherent agenda that fits in the 60m meeting slot. If there are similar items try and group them together.
SDSC Jupyter Proxy Service
Project is called “Satellite”
Jupyter initially appeared as “plain-text back door into the cluster”
“Make the right things easier than the wrong things”
Satellite:
Invoke client, get URL, notebook shows up
Self-service https reverse-proxy
Shell-based utility to orchestrate RP and Slurm
Reverse proxy:
RP provides SSL termination
users create/destroy mappings
short-lived random subdomains
satellite api only usable by wget/curl
start-jupyter, Galyleo:
Batch jobs, issues with templates and integrating Singularity
Open OnDemand portal on Expanse (have Matlab, RStudio, want to integrate Satellite)
Galyleo repo: sdsc-hpc-training-org/satellite
–transport=ipc
Streetwise guide: https://docs.google.com/presentation/d/1X_0hHYpQcCwrwbPMGSHuTeys7KPW3bC1/edit#slide=id.g8cad2b19f4_0_1024
Shane is trying it on expanse during the call :)
Questions:
What proxy service are you using (traefik, CHP?)
Looks like it is dynamically configuring apache
Is their an open repo for the API? (Answered)
Is there any user documentation? I couldn’t quickly find it via Google. (Answered)
Rather obvious one: Hub enables subdomains, provides proxy; what were the issues w/Hub that made you think you needed your own approach?
Already have trust established (Q: e2e SSL)
Issues local to SDSC when starting w/Hub
Want to prevent users from running Jupyter themselves full stop (if they go around Hub)
Concern about where hub lives with respect to trust boundary of cluster, can be addresses
Slides available? distributable?
yes, and yes
Standing Items:
Batchspawner check-in
Need to check in with outstanding PRs now that autoformat PR is merged, will need to rebase
Wrapspawner check-in
At last call, goal was release, then deep dive on issues
What are the top issues?
JupyterLab 3 deployment news
NERSC
Pushed back to May 19 at NERSC but will happen
~All needed extensions are working now
One issue found and fixed in jupyter_server