--- tags: meeting, notes --- # JupyterHub HPC Meeting - November 2021 - **Date:** 2021-11-03 - **Time:** 8:30 AM PDT - **Your timezone:** https://arewemeetingyet.com/Los%20Angeles/2021-11-03/08:30/JupyterHub-HPC - **GitHub issue:** - **Calendar for future meetings:** https://jupyterhub-team-compass.readthedocs.io/en/latest/meetings.html ## Welcome to the Meeting Hello! If you are joining the team video meeting, sign in below so we know who was here. Roll call: - **name** / **institution** / **GitHub handle** - Min RK / Simula / @minrk - Jens Henrik Göbbert / Jülich Supercomputing Centre / @j.goebbert - Zach Price / ORNL / - Shreyas Cholia / LBL / @shreddd - Richard Darst / Aalto / @rkdarst - Michael Milligan / MSI @ UMN / @mbmilligan - Félix-Antoine Fortin / Compute Canada - Université Laval / @cmd-ntrf - Simon Li / University of Dundee / @manics ## Quick updates 60 second updates on things you have been up to, questions you have, or developments you think people should know about. This is also a chance to suggest a future presentation if you've got work currently in progress you might want to share. Please add yourself, and if you do not have an update to share, you can pass. - **Name:** Your update - ## Reports and celebrations This is a place to make announcements (without a need for discussion). This is also a great place to give shout-outs to contributors! We'll read through these at the beginning of the meeting. - **Name:** Your report or celebration - ## Agenda items Let's collect all potential agenda items here before the start of the meeting. We will then attempt to create a coherent agenda that fits in the 60m meeting slot. If there are similar items try and group them together. - **@j.goebbert** redesign of JupyterHub landing page - More and more features on the JSC landing/home page - Users were getting confused, getting passed from one page to another - The options spawner-type functionality comes up as a modal on the home page - For every system, for every JupyterLab, for login nodes, for every kind of compute node - Each user starts with a short list of defaults and then they can come up with more options - What's available depends on what's been installed on the file system - Spawner starts a script that decides what to do - Ability to let users add their own JupyterLab with their own weird extensions - Can a user share a configuration with with other users? - The hub picks up the installation so users can share with each other this way - Installation on a project's space on the file system shares it with everone else on the project - Is there a standard "base" set of extensions that power users can use to create their own? - Based on easy build so users can base off that - Curating some extensions (taking away...) - Lots of extensions are a burden... - Modules used to extend the labextension search path - Done dynamically in the start script - Not part of the install/configuration - **Simon** BinderHub in a Box - Runs on k8s, scales well, really powerful, but it is a pain to install locally for testing - Try to host BinderHub just on Docker? Turns out not as difficult as you might think - Started a bigger discussion about abstraction layers? - Need: r2d, hub (nothing that says k8s is intrinsic) - Main PR: https://github.com/jupyterhub/binderhub/pull/1364 - Demo: https://katacoda.com/manics/scenarios/binderhub-in-a-box - Platform for infrastructure learning - Spins up a VM... actually look at the demo, it's very nicely documented - Repo2docker runs _locally_, so for more than a few users it needs to be pretty beefy VM - Demo is done with docker, but you can use podman (podman daemon spawner, and fake docker registry) - BinderHub is running as a JupyterHub service here - Minimizing Privilege for Building HPC Containers: https://arxiv.org/pdf/2104.07508.pdf - k3s some exploration, it can be done, but teaching k8s is tedious :) - SC21 activities? - No Interactive HPC workshop - Felix is demo'ing latest Magic Castle at Training and Tools HPC workshop - ISC22 - Program committee? - Shreyas find people will do everything from his group - - JupyterHub 2.0 - **PEOPLE SHOULD TEST** - [Feedback topic on discourse](https://discourse.jupyter.org/t/jupyterhub-2-0-beta/10841) - RBAC - Roles and scopes, how permissions work in JupyterHub - More granular permissions: useful for collaboration and RTC in JupyterLab - Spawning changes - Small change, JupyterHub will not pass its own args as command line -> environment variables - The only ones passed are the ones that user configuration has explicitly given - If you look for --ip then you may want to watch out - => batchspawner-singleuser - Arguments are injected for remote port selection but - Not parsing the command line - code: ``` cmd_path = which(sys.argv[1]) sys.argv = sys.argv[1:] + ["--port={}".format(port)] run_path(cmd_path, run_name="__main__") ``` - Environment variable information carrying port info should defer to command line but needs test - ... can we get port selection on the remote side (the singleuser executable) - Richard has a draft PR - 2 exclusive cases... need a scheme that works for both! - Can't move everything over to the server because it doesn't always have the info it needs (servers running in contairers know the bind port but not the connect port) - it can be handled but needs careful thought. - Could have a property of the spawner. - https://github.com/jupyterhub/jupyterhub/pull/2727 (old, needs someone with either knoweledge or time to concentrate) - - **All:** Standing project items: - Batchspawner check-in: Issues and PRs - Felix says he'll try it out with Hub 2 - Report to Discourse - Wrapspawner check-in: Issues and PRs