JupyterHub and BinderHub Weekly Report#
Goal: Continuous improvement
Opening Date: 05 Feb 2018
Closing Date: 12 Feb 2018
Next meeting date: Feb 8th, 6pm Zurich time or your timezone
Videoconference link: https://calpoly.zoom.us/my/jupyter
Welcome to the Team Meeting#
If you are joining the team video meeting, sign in below so we know who was here. Roll call:
your name / affiliation / github
Tim / Wild Tree Tech / @betatim
J Forde / Project Jupyter / @jzf2101
Carol / Cal Poly SLO - Project Jupyter / @willingc
Erik Sundell / Academedia / @consideRatio
Chris H / UC Berkeley / @choldgraf
M Pacer / UC Berkeley /@mpacer
Opening question, while we wait#
(silent hackmd’ing): Are you a rusher or a take-enough-time-and-am-few-minutes-early-and-still-stressed-out (or something else)?
Tim is a rusher because he likes optimising things … but could have a more relaxed life by just being a few minutes too early to things.
Chris is a “wherever coffee is closest-er”. As such he is in a coffee shop right now.
Yuvi just woke up and what is this words on the screen
Carol is early or really, really late
M is a right skewed distribution with a median couple of minutes late, always rushing in any case
Help or discussion needed; Agenda Items for Monthly Meeting
1.1 Open PRs
1.2 Open issues
[CH] We should discuss Matthias’ PR about Federation, with the goal of having a clear path forward to merging! (PR)
two points: technical and governance
for technical things comment on the PR, especially if you know about cookies
governance related: there have been some discussions by email already. Can we create a new issue to discuss the governance related aspects of federation to not combine it with tehcnical discussions
good idea: looking at the grant to get guidance as to what has been discussed re: federation
[CH] If anybody would like access to any of the accounts they don’t have access to, please say so and we’ll get you connected! (e.g. twitter, google analytics, pip rights, google cloud, etc) [issue](jupyterhub/team-compass#20)
go to the issue if you would like to get access
What are the possible forms of Binder governance? (maybe start the discussion, idea gathering not decide)
is Binder its own entity? Is it part of Jupyter or …?
What are possible models of companies/others donating time/people? What do they get in return? (maybe start the discussion, idea gathering not decide)
how does it work for Jupyter (this applies to JupyterHub)?
what is the intended fiscal relationship between Binder and the rest of the project?
[CH] more generally, let’s brainstorm (or start an issue where we brainstorm later) specific ways we can make it easier for people to get involved in the JupyterHub + Binder communities. E.g., improvements to documentation, clarity in the issues tags, etc. Just want to make sure we’re being friendly to new contributors in a sustainable way.
should we create a bit of a ROADMAP to see where the project is heading? (+1)
CH: I think that’s a great idea a while back we had been using this github project to manage long-term plan but it hasn’t been used as much since the initial release. We can improve this stuff!
implement the idea of office hours as a way to increase the number of people who contribute (+1)
create a hackpad for the office hours that becomes community driven documentation
can we provide starting points for each of the technologies you need to know to contribute? jupyterhub/zero-to-jupyterhub-k8s#480
[ES] can we provide the curious with help on envisioning the potential of a JupyterHub?
example notebooks utilized for teaching
Provide a recommended learning progress
kubernetes - reading the cooncept section on was awesome
helm - i had to read the helm documentation
Note, we have this issue just opened a few days ago: jupyterhub/zero-to-jupyterhub-k8s#480 <- would be a great place to iterate on!
There may be more than one collection of skills depending on the role the person wants to take on (dev ops vs. educator)(+1)
[CH] Related to the above, we should discuss ways to distribute the workload more effectively for the
mybinder.orgdeployment. Right now we are still largely bottlenecked on Yuvi when fires pop up. We need to either grow the kubernetes / devops skills within the team, or include members of the ops community in the project somehow (probably we need to do both of these things).
Consider using the binder grant to pay for kubernetes / cloud training for interested team members?
Another idea is to have a google project that’s specifically used for sandboxing / learning (paid for by the binder grant)
maybe not have a persistent binderhub that is the playground
do not use staging as playground
what does it take to create a project that is linked to the billing account?
Does it need to be a new project or an extra cluster on the staging project?
Should we deploy to staging, production and development?
Finalize SciPy proposal(s): jupyterhub/binderhub#422 (BinderHub SciPy submission is here) (CH: I don’t think we’re in a place to finalize this but hashing out a general plan/outline would be great)
JupyterCon CFP submissions anyone?
[CW] May submit a new education talk that I’ll be giving in March JupyterDays Atlanta STEAM workshops with JupyterHub and Binder
Deadline is March 6th
one talk on repo2docker
one talk on mybinder.org
one talk on zero2jupyterhub
Open issue to track
PyData London CFP is open, Tim might go, might submit a talk about binder.
eLife innovation sprint May 10-11 in Cambridge
Tim to find etherpad link
[CH] Jeremy Freeman (original Binder team, now at the Chan Zuckerburg Initiative) is going to stop by BIDS next week. They’re interested in ways they could help with the JupyterHub world. We should discuss how to make this an effective relationship!
[CH] Related to above: I put together a wishlist document where team members can put anything that they think would be a useful direction for these projects. Would love if you could think about it / add thoughts from time to time! here’s a link
MRK Stackdriver logging costs link
how to limit log output to reduce cost for stackdriver
Needs to be dealt with this month before it starts costing money
turn down CHP log level
maybe only store bhub and jhub logs
link to the issue: link
We need to get a move on with this a bit because of European privacy laws (GDPR!)
Keep in mind that we might be “tracking” users if we implement a way to re-use binders in the context of textbook pages launching new binders (should be ok, need to figure out definition of ‘tracking’)
This is probably also something we should consider in discussions around federation (e.g. another binderhub service would also need to adhere to DNT to be part of the federation model)
(Yuvi) We should consider merging the binderhub chart into the z2jh chart so turning binder ‘on’ is just an option in z2jh - jupyterhub/binderhub#435
Just a note: I’ve spoken with a number of JHub people that want Binder-building functionality, and a number of Binder folks that would like persistent storage, so I think this would make them happier
note we do have a v0.1 milestone for binder (link)
Should make sure that this isn’t the only
$M$ Storing proposals/papers/templates/&c., should this be a new github repo?
Let’s use team-compass repo for this (+1, we already have too many repos!)
$M$ for the discussion or the location for the actual content? -1 as the location for the actual content as it is a separate aim and type of activity from general organisation/governance/project management/communication amongst the team
use the team-compass as a landing page and link out to actual content
Next actions (team)
Create an issue specifically for discussing governance and group decisions around how to proceed with federation (separate from technical challenges)
Worth specifically pointing out short-term and long-term federation plans
Create a “playground” binderhub setup on other cloud providers
Create a “playground/learning” project on google cloud
Create a place where we store Binder / JupyterHub proposals, papers, talks given, etc.
Our ungodly amount of logs
Figure out what is the application on our deployment that’s generating all of these logs
It sounds like BinderHub+JupyterHub won’t be enough to put us over the free limit, so we need to do a logs audit and disable whatever thing is generating the insane number of logs we are getting.
News, Information, and Thanks#
Team News and Informational items
5.1 Organization highlights
[CW] Traveling from Tuesday until end of month (I’ll be a bit slower to respond)
5.2 BinderHub projects
New experimental deployment by Enol Fernandez from EGI; this is meant as proof of concept for an official EOSC (European Open Science Cloud infrastructure) binder deployment.
The BinderHub JOSS submission is here. Please leave comments/thoughts/additions/etc!
5.3 JupyterHub projects
5.4 Related projects (repo2docker, nbgrader, others)
Thanks, Things to Celebrate, and anything else
Submitted Mozilla Mini Grant. full text
Tim -> Chris, grant is about teaching people to use binder, you hacked some teaching materials at BIDS, are they relevant? Where to find them?
Zenodo integration: zenodo/zenodo#1416 Tim thinks this would be pretty cool
Thanks to Yuvi for the awesome work on upfront code documentation and developer docs for repo2docker. Should be a big time saver and great resource for all. :tada:
We appreciate the community members who are active on the gitter channel. Thanks for helping others with configuration issues and answering questions. :tada:
Happy Birthday Yuvi tomorrow!!