MyST Markdown is an extensible, semantic, and community-driven flavor of markdown designed for scientific and computational narratives. It was created for Jupyter Book, and has gained wide adoption in Python’s Sphinx documentation ecosystem.
Over the years we have come to believe that MyST has the right technical and social foundation to serve as a truly community-driven, cross-platform standard for authoring scientific content. In this post, we’re happy to announce MyST Markdown as a standalone project and standard from the Executable Books community.
The rest of this post provides ideas on where this project is heading and fleshes out some details. We are still exploring what is possible, and would love feedback about what you’re excited about.
Learn more and get involved with MyST
What is MyST Markdown?#
Two years ago, the Executable Books project was awarded a grant from the Sloan Foundation to build open source tooling for authoring and reading computational narratives. We focused our efforts around Jupyter Book, a tool for writing and sharing computational narratives as websites and PDFs. This project was built on the Sphinx ecosystem, an open source community that is at the foundation of documentation tools with Python.
MyST, short for Markedly Structured Text, was designed to combine the fluid experience of writing Markdown with the programmable extensibility of reStructuredText. We needed these properties in order to meet the needs of Jupyter users in research and scholarly communication. From this starting point, MyST was designed with the following goals in mind:
Be extensible: We wanted a flavor of markdown that had extensibility built in from the start. This way users could extend its functionality in an intentional and structured way.
Be easy to read: This extra extensibility should come without sacrificing the simplicity and readability of CommonMark markdown syntax.
Be semantic and structured: We believe that the future of scholarly communication involves semantic and well-structured data, and wanted MyST to have a clearly defined specification and underlying data structure that authors and readers could leverage to connect MyST documents with one another.
Be community driven: We believe that standards for scholarly communication require community governance and leadership, rather than being driven by one stakeholder. We framed MyST and the Jupyter Book stack as a collaboration between many organizations and will develop open governance that follows the principles of open scholarly infrastructure.
The first implementation of MyST was the MyST Markdown Python parser, a parser that allows users to take advantage of Sphinx’s powerful directives and roles syntax in building their documentation.
Over the years we have seen interest in MyST raise considerably - it allows authors and researchers to express themselves more efficiently than traditional source text languages like LaTeX. MyST has also become a staple in the Python documentation ecosystem, and is the backbone of authoring content in Jupyter Book.
Because the original parser for MyST is written entirely in Python (and strongly tied to Sphinx), we realized that it would be difficult for the impact of this implementation of MyST to make its way out of the Jupyter Book and Python communities.
Here are a few major new steps summarizing this work over the coming months:
MyST is now a top-level project in Executable Books#
We’ll treat MyST as another major effort of the Executable Books project, similar to Jupyter Book. It will have its own strategy and goals. Jupyter Book will remain a consumer of MyST markdown, and it will remain one of many stakeholders in the MyST ecosystem.
A new website!
We have set up a dedicated site to describe the MyST project at mystmd.org.
You can find the documentation for MyST JS at js.mystmd.org. You can also find an issue from Rowan discussing major new planned functionality here.
MyST will have a JupyterLab extension#
For example, the
jupyterlab-myst extension brings native MyST rendering into JupyterLab including admonitions, cross-references, and figure numbering using
mystjs as the document engine.
Learn more about the
jupyterlab-myst project at
MyST will have an implementation-agnostic specification#
We are creating a formal definition for MyST at spec.mystmd.org. This will allow us to extend MyST’s syntax through a community-driven enhancement proposal process, and allow implementations of MyST to bring in new syntax as they wish. Modifications to this specification will follow a MyST Enhancement Proposals process (MEPs for short). We are still working out the details of how this spec will be structured, and what the process looks like, so please provide feedback if you have ideas for how to build a healthy and inclusive process around the MyST language.
The MyST Enhancement Proposals repository has the latest information about the MEP process and any active MEPs.
MyST-JS will complement Jupyter Book and focus on articles#
We think that MyST-JS and Jupyter Book will be able to complement one another in the use-cases they focus on. The Jupyter Book project (and its Sphinx stack) are focused on building multi-page books and project documentation. It will continue to lean heavily into the use-cases that Sphinx was designed for. It also has major functionality that is missing in MyST-JS (for example, full internationalization support and extensions). We will continue to improve and maintain this stack (both as individual projects in the Sphinx ecosystem and as the Jupyter Book distribution).
Over time, we will continue to explore the potential of both of these ecosystems. As an implementation-agnostic markdown standard, users should be able to switch back and forth between MyST parsers with low friction.
Formalizing organizational structure for the Executable Books project#
Finally, we have begun formally defining the organizational structure and governance of the Executable Books project. Our goal is to transition this from a small grant-funded team into a community-driven project that has multiple stakeholders as well as clear pathways for participation and leadership. We will continue to refine these organizational practices moving forward, and invite you to join our efforts! More on this effort in a subsequent update!
See this GitHub issue for a long list of ideas and suggested improvements to make as we formalize our processes and structure.