2021

September 1st 2021

If you are joining the team video meeting, sign in below so we know who was here. Roll call:

  • Name / Affiliation / Handle

  • Steve Purves / Curvenote / stevejpurves

  • Rowan Cockett / Curvenote / rowanc1

  • Matt McKay / ANU / mmcky

  • Chris Holdgraf / 2i2c / choldgraf

  • Damián Avila / 2i2c / damianavila

  • Chris Sewell / EPFL / chrisjsewell

Reports, updates, and celebrations

This is a place to make announcements (without a need for discussion), short updates about what you’ve been up to, and shout-outs to contributors! We’ll read through these at the beginning of the meeting.

  • THEBE:

    • Some Thebe PRs in recently, and refactoring work started, other PRs ongoing

    • Opened(opening) tickets on activity board for dicussion

    • Refactoring, improving test coverage and docs

    • Extended kernel support for Juptyerlite/pyodide, local proxy and better kernel selection

  • Some blog post about MyST in my personal blog (Damián)

    • Part 1

    • Part 2

    • Eventually, a part 3 with some coding experiment/stuff

  • Sphinx 4 Upgrade (Matt)

  • ReadTheDocs integration (Chris S)

    • That is now working

    • PR is open - just waiting on documenting it up

    • Action: Needs Doc Update

Agenda items

Let’s collect all potential agenda items here before the start of the meeting. We will then attempt to create a coherent agenda that fits in the 60m meeting slot. If there are similar items try and group them together.

  • Steve: Thebe directions, and future focus

    • Setup a Project board on the thebe repo to better plan out work there, in more detail that the Activity Board maybe?

    • A lot of issues in the repository

      • Was a JupyterCon Sprint + a board from that

    • Would like to do a bit of planning about what issues etc to tackle next

    • There is definitely the opportunity now to move a bit more quickly and make changes before opening up for contributions

    • Priority definely to get the CSS issue fixed so jupyter-book can move to the latest

  • Rowan: Thoughts on how to move JS-Myst forward (dev branch?)

    • Few-no current dependencies on the work, and could possibly merge/deploy faster than some other repos.

    • Might just be an off month?!

    • Is there a way that we can ensure a consistent momentum of PRs so that we don’t constantly block on people’s reviews etc?

    • Right now we are at an important early design moment on the JS side. Longer term it should be faster as the PRs will be less complex.

    • In the meantime, Chris S will prioritize his EBP stuff on making sure these PRs get reviewed.

  • Chris H: in-line markdown execution?

    • Long conversation about this here: https://discourse.jupyter.org/t/inline-variable-insertion-in-markdown/10525/37

    • The difficulty is in the specification

    • mmcky: Related to adding execution to exercise nodes (which we are prototyping) and working out how to get execution embedded in directives. Current approach is through gated directives (which wouldn’t work obviously for roles)

    • An easy first step would be to do “read only inline markdown”

    • This is a really popular feature of RMarkdown / Quarto Markdown

    • Can you start with the restriction “we should be able to still run every code cell”

    • From a user’s perspective, you’d just want them to be able to put in variables with some kind of syntax e.g. {{ }}

      • You’d need to go through the markdown cells and look for the {{ }} syntax and know what kinds of variables that you need.

      • You’d then do the execution, and at the end of the time that

05th August 2021

If you are joining the team video meeting, sign in below so we know who was here. Roll call:

  • Matthew McKay / ANU / mmcky

  • Steve Purves / Curvenote / stevejpurves

  • Chris Sewell

  • Rowan Cockett / Curvenote / rowanc1

  • Damián Avila / 2i2c / damianavila

Reports, updates, and celebrations

This is a place to make announcements (without a need for discussion), short updates about what you’ve been up to, and shout-outs to contributors! We’ll read through these at the beginning of the meeting.

  • Chris Sewell

    • Sphinx Designs - better designed panels, will come into jupyter book at some point

    • Jupyter Cache (for executing notebooks –> myst-nb –> JupyterBook) - rewritten

      • Decides if code cells are changed and caches outputs. Then can grab the outputs from a local database. There is a CLI!

      • There is also multi-processing on the notebooks! 💥

      • There is a new PR draft

    • Javascript!

      • Added a new repo docutils

      • There are same directives and roles from Python

        • Same arguments/content

      • This is now over to the vscode extension

    • MyST as a “standard”

      • What roles, directives, blocks to we need that are copied over in another language

      • We should get the critical ones in a list somewhere!

        • Forming specs in a collaborative way (Identify Good Tools for this?)

  • Matt McKay: Taken over maintaining sphinx-proof and sphinx-exercise (looking into enabling execution through gated directives as a proof of concept)

    • “enabling execution in directives” (https://github.com/executablebooks/sphinx-exercise/issues/24)

      • Adding a directive that is a question, and then how do you add code?

        • Directive at the top level and then code and exercise.

          • Violates the core design of the Jupyter Notebook – nested code cells

          • Got rid of “nested” concepts

        • So how do you have “Gated” directive (open/close) around a few code cells

          • This is more compliant with other tools like Jupytext

        • Not sure if there is a Gated directive, or prior art in the Sphinx world?

      • Other options:

        • Metadata? Adding tags, etc. (Similar to nbgrader notebooks)

        • Glue? (currently restricted to python, cather than cellId or something)

          • Similar ideas playing with at Curvenote! (copy/paste)

      • Consideration is mostly from an authorship perspective

  • Steve Purves: Thebe

    • got started and made some improvements to the build and docs – yarn, CI issues, local kernel convenience commands for development

    • PR for a small feature - progress indicator

    • Collaborating a bit with LibreText folks, would like to more and understand how they are using Thebe better

    • LibreText made a big change to utilise the @jupyterlab/manager in place of a shim+custom code – so waiting on that PR coming in before pushing on

    • Next up:

      • Connecting to many different kernals, and having some basic management on that!

  • Rowan: MyST in JS

    • Working on myst in JS. Migrating work over to docutils and cleaning up along the way

    • Still basics of various libraries

    • 10 or so PRs up at the moment!

      • Basics:

        • sub/sup/abbr

        • internationalization? [each role/directive has a different name]

          • Can generate various dicts to export

      • Admonitions

      • Math

        • amsmath, dollar, math directive

      • Figures

      • Referencing/numbering PR

        • numref/eq/ref

    • Working to remove all of the markdown-it-myst functionality –> docutils w/ review from ChrisS

    • How are we working with styles?

      • There is some copying back and forth of the .css

  • Damián

    • Learning Myst internals working in the Nikola + Myst story

    • Probably writing a series of blog posts to push discussion forward about things I have found and engage the community

    • MyST as a “standard” > What roles, directives, blocks do we need copied over Python (independent from Docutils/Sphinx)? Do we need a docultils/sphinx free Py representation of the “standard”?

Agenda items

Let’s collect all potential agenda items here before the start of the meeting. We will then attempt to create a coherent agenda that fits in the 60m meeting slot. If there are similar items try and group them together.

  • NAME: ITEM (EXPECTED TIME TO DISCUSS)

  • Discuss team activity board proposal (https://github.com/executablebooks/meta/issues/441)

    • The goal of this board is to coordinate and plan our major work items, to make it clear who is working on what, and to help signal boost items for review/discussion.

    • Is this board structure worth trying as-is or are there major changes to make to it?

    • If we want to try this process, perhaps we should use this meeting to add some initial items to the board?

Discussion:

  1. This was discussed with the team and we agreed to use it this month to see how it works.

  2. Any suggestions to simplify or improve the Activity Board was encouraged.

  3. We focused on keeping the activity board pretty high level for cross-cutting communication and to help coordinate.

  4. We also discussed keeping an eye on the Request for Review board to get visibility on PR’s that need review and get more distributed reviews across the team.

  5. We should add the Activity Board as a standing discussion point for the monthly meeting

  • Steve: Next Steps for Thebe

    • Refactoring the one big file into more manageable chunks

    • More tests - current test coverage is poor, PRs go green but not much really covered which is maybe worse than no tests at all

    • Kernel Status & Management - as part of refactor want to make it easier to get status of the kernel and lay ground work for the issues asking for: Kernel Status, Multiple Kernels and Jupyterlite connectivity; 412, 349, 271

Discussion:

  1. This was discussed in the Reports section.

July 1st, 2021

If you are joining the team video meeting, sign in below so we know who was here. Roll call:

  • Matthew McKay / Australian National University (ANU) / mmcky

  • Chris Sewell / EPFL / chrisjsewell

  • Steve Purves / Curvenote / stevejpurves

  • John Stachurski / ANU / jstac

  • Chris Holdgraf / 2i2c / choldgraf

  • Damián Avila / 2i2c / damianavila

  • Aakash Gude / ANU / AakashGfude

Reports, updates, and celebrations

This is a place to make announcements (without a need for discussion), short updates about what you’ve been up to, and shout-outs to contributors! We’ll read through these at the beginning of the meeting.

QuantEcon / ANU

  • QuantEcon: 4/5 Projects running Jupyter Book (remaining: julia)

  • ANU Team: Working on Support for Individual Page PDF (via LaTeX) generation (Lead: @mmcky)

    • Chris H notes that we should include updates about QuantEcon lectures conversion etc

    • @mmcky to add links to executable books gallery to QuantEcon jupyter-book powered projects

Chris S.

Recent work

  • Released sphinx-external-toc and integrated in Jupyter-Book

    • Removes a lot of code from jupyter-book 😄, moving towards JB just being a thin wrapper for sphinx-extensions (last part is basically the _config.yml)

    • Bit of a balancing act between; ease of use/understandanding, having a clear schema, having all the functionality of sphinx toctrees

    • Integration in JB also includes addition of “pluggable” CLI

  • Released markdown-it-py v1.0, i.e. fully stable

  • Released myst-parser v0.15.0, with sphinx v4 support

    • Should be moving all repositories to sphinx v4

  • Released rst-to-myst v0.3

    • Facile conversion of RST to MyST Markdown for a whole project

    • Utilises mdformat + mdformat-myst to create the Markdown from markdown-it tokens

    • Still working on mdformat-myst, but this essentially allows round trips Markdown text -> markdown-it tokens -> Markdown text. Could potentially be used for e.g. modifying links in downloadable notebooks

  • Created markdown-it-plugin-template; template repository with best practices for TypeScript library

Intended work

  • Update MyST VS Code extension to use new markdown-it plugins

  • Create sphinx-design: An iteration on sphinx-panels, but with the benefit of hindsight, and to more closely follow aspects of Materials Design and Material-UI

    • class names that do not clash with sphinx ones (i.e. container)

    • easier to set principle colors and width defaults

  • Either in myst-parser or sphinx-design have nice way to generate stylised post-header banners, with e.g. author, date, read time (see https://github.com/executablebooks/MyST-Parser/issues/372)

  • Improvements to myst-nb (+ jupyter-cache)

    • Been gradually working on it, but lot of interconnected parts 😬

    • Revise the AST and transforms (post-transform -> transform)

    • Drop nbconvert, jupyter-sphinx dependencies

    • Better support for Markdown code output

    • Better support for “multi-level” configuration (cell level > notebook level > conf.py level)

    • Improve notebook execution and (maybe) drop auto mode in favour of cache

      • Support for dynamic kernel names

    • How to handle glue; ideally in a kernel agnostic manner, possily with the new substitution syntax

    • Plugin CLI in jupyter-book for notebook execution

Steve

  • Would like to:

    • meet the team

    • hear about thoughts on Thebelab as I’m prepping to do some work there

Berkeley team

  • :wave: to Damian, onboarding etc

  • Damian is playing around with the MyST parser and learning how it is structured, experimenting with a Nikola implementation of the parser.

  • Chris is going to focus more of his time on team documentation, tool documentation for both users and developers, etc.

Agenda items

Didn’t get to the following items

John S:

  • Hiring casual RAs?

Chris H:

  • Communicating the work we’re doing - blog posts, tweets, etc? @executablebooks handle?

  • Coordinating around deliverables, workstreams, requests for reviews, etc?