diff --git a/.gitignore b/.gitignore index b6e4761..f6aa1e9 100644 --- a/.gitignore +++ b/.gitignore @@ -127,3 +127,6 @@ dmypy.json # Pyre type checker .pyre/ + +# Documentation-specific files +docs/development/conventions.txt diff --git a/docs/about.md b/docs/about/history.md similarity index 76% rename from docs/about.md rename to docs/about/history.md index 175a86d..77b07f9 100644 --- a/docs/about.md +++ b/docs/about/history.md @@ -2,8 +2,10 @@ This is a brief history of the Executable Books project, written with the intention of encoding the provenance and rationale behind our current governance structure. -The Executable Books project began as a collaboration between three universities funded from a single grant. -This team used these resources to develop a collection of tools and standards for communicating ideas with computational narratives in the Jupyter ecosystem and beyond. +The Executable Books project began as a grant-funded collaboration between groups at [The Australian National University](https://anu.edu.au), +[Northern Arizona University](https://nau.edu/), and +[The University of California at Berkeley](https://www.berkeley.edu/). +These teams represented many open source projects in the scientific community, such as [QuantEcon](https://quantecon.org), [QIIME 2](https://qiime2.org/), and [The Jupyter Project](https://jupyter.org/). This group followed a traditional Principal Investigator model where the vast majority of development on the project came from dedicated time paid for by the grant. Towards the end of the grant, the team discussed ways to create more pathways for community participation and leadership on the project, and to [transition towards more co-creation of a broader group of people](https://github.com/executablebooks/meta/issues/493), rather than being driven primarily by one grant-funded team. diff --git a/docs/about/support.md b/docs/about/support.md new file mode 100644 index 0000000..9c4d582 --- /dev/null +++ b/docs/about/support.md @@ -0,0 +1,31 @@ +(contributions)= +# Support for the project + +The following organizations have provided financial and in-kind support for this project. +For a list of individuals that have contributed to the project, see [](team.md). + +## Financial support + +These organizations have provided direct financial support for this project. + +:::::{grid} + +::::{grid-item} +:columns: 3 + +:::{image} https://pbs.twimg.com/profile_images/1226944724365447169/MzFpwY5P_400x400.png +:class: float-left mr-2 rounded +:width: 100px +::: +:::: + +::::{grid-item} +:columns: 9 +This project was launched via [a three-year grant from the Sloan Foundation](https://sloan.org/grant-detail/9231). +:::: + +::::: + +## In-kind support + +_In the future we wish to create a formal definition of what warrants in-kind support for the project, and list them here_. diff --git a/docs/team.md b/docs/about/team.md similarity index 55% rename from docs/team.md rename to docs/about/team.md index 63a82e2..54b80d6 100644 --- a/docs/team.md +++ b/docs/about/team.md @@ -1,5 +1,8 @@ -# Team Members +(team)= +# Team members +This project has had contributions from many different people and organizations. +Below we list each, including their present or past roles. ## Steering Council @@ -13,4 +16,10 @@ We expect the Steering Council to expand in the future, and this initial Steerin ## Core Team -Currently the core team is [defined in this list of team members](https://executablebooks.org/en/latest/team.html). +The core team is [defined in this list of team members](https://executablebooks.org/en/latest/team.html). + +## Emeritus Team Members + +These are team members that were once on the Core Team, but no longer wish to have Core Team responsibilities. + +_Note: this section is currently empty as no team member has formally left the project, we leave it here to encourage these practices moving forward_. diff --git a/docs/code-of-conduct.md b/docs/code-of-conduct.md index 42d6bad..ee9d5fb 100644 --- a/docs/code-of-conduct.md +++ b/docs/code-of-conduct.md @@ -24,3 +24,13 @@ The Executable Book Project follows a community [Code of Conduct](https://github.com/executablebooks/.github/blob/master/CODE_OF_CONDUCT.md). Please use that document as the source of truth for interacting with the EBP community. + +```{button-link} https://github.com/executablebooks/.github/blob/master/CODE_OF_CONDUCT.md +:color: primary +Code of Conduct +``` + +## Future plans + +In the future we aim to create a more formal Code of Conduct reporting process and team of people to respond to CoC violations. +Please monitor and provide feedback in [the `team-compass/` repository](https://github.com/executablebooks/team-compass) as we explore the best path forward. diff --git a/docs/conf.py b/docs/conf.py index 3e8bc00..a98f0ee 100644 --- a/docs/conf.py +++ b/docs/conf.py @@ -23,7 +23,7 @@ html_theme = "sphinx_book_theme" html_logo = "_static/logo.svg" html_favicon = "_static/logo-square.png" -html_title = None +html_title = "" html_static_path = ["_static"] html_theme_options = { @@ -39,3 +39,30 @@ "jb": ("https://jupyterbook.org/en/latest", None), "meta": ("https://executablebooks.org/en/latest/", None), } + +# -- Extra scripts at build time --------------------------------------------- + +from pathlib import Path +from sphinx.application import Sphinx +import os +from sphinx.util import logging +import requests + +LOGGER = logging.getLogger("conf") + +# Functions that this calls are below +def setup(app: Sphinx): + app.connect("builder-inited", update_contributing) + + +def update_contributing(app: Sphinx): + """Downloads the latest version of the contributing guide.""" + path_contributing = Path(app.srcdir) / "development/conventions.txt" + if path_contributing.exists(): + LOGGER.info("Contributing page exists, delete and re-build to update...") + return + LOGGER.info("Updating conventions page...") + # Grab the latest contributing docs + url_contributing = "https://raw.githubusercontent.com/executablebooks/.github/master/CONTRIBUTING.md" + resp = requests.get(url_contributing, allow_redirects=True) + path_contributing.write_bytes(resp.content) diff --git a/docs/development/conventions.md b/docs/development/conventions.md new file mode 100644 index 0000000..dc61cf6 --- /dev/null +++ b/docs/development/conventions.md @@ -0,0 +1,12 @@ +# Development guide + +This is a guide with helpful guidelines and suggestions for contribution. +It is located [in our `executablebooks/.github` repository](https://github.com/executablebooks/.github) for easy access, and copied below. + +--- + +% This is automatically updated in conf.py +% Skip the first line so we don't double up the headers +```{include} conventions.txt +:start-line: 1 +``` diff --git a/docs/governance.md b/docs/governance.md index d6f538f..3423de7 100644 --- a/docs/governance.md +++ b/docs/governance.md @@ -53,7 +53,6 @@ Individuals who are particularly interested in the EBP community and have demons This is intentionally blank for now, we will add more information in the coming weeks. ``` - #### Expectations ```{note} diff --git a/docs/index.md b/docs/index.md index ad89554..a660276 100644 --- a/docs/index.md +++ b/docs/index.md @@ -3,9 +3,17 @@ This documentation serves as the **Source of Truth** (SoT) for the Executable Books community policy and strategy, and defines its organizational structure and governance. It also aims to provide resources that facilitate contribution and coordination among team members. +:::{admonition} Work in progress! +:class: caution + +This site is a work in progress, as the project begins to formally define its structure and governance. +Some information may be duplicated or contradictory with information at [executablebooks.org](https://executablebooks.org) - if it is, treat the information here as the source of truth. +[See this GitHub issue to track our progress on this migration](https://github.com/executablebooks/meta/issues/857) +::: + ## Goals of the Team Compass -The Team Compass should make our organizational policy and power structures explicit. +The Team Compass should make our organizational policy, structure, and team practices explicit. It also delegates authority to other groups, processes, and locations. See the sections below for more information.[^1] @@ -15,11 +23,29 @@ The Team Compass is always evolving as our community learns and changes. To propose a change to the Team Compass, see [](governance:policy-decision). ```{toctree} +:caption: Policies and standards :maxdepth: 2 code-of-conduct governance -team -about +about/team +strategy +``` + +```{toctree} +:caption: Contributor resources +:maxdepth: 2 +resources/accounts +resources/communication +development/conventions +meetings/index +``` + +```{toctree} +:caption: About the project +:maxdepth: 2 + +about/support +about/history ``` ## License diff --git a/docs/meetings/2021.md b/docs/meetings/2021.md new file mode 100644 index 0000000..a0a0e0b --- /dev/null +++ b/docs/meetings/2021.md @@ -0,0 +1,398 @@ +# 2021 + +## Dec 1st 2021 + +### Attending + +- Aakash / Australian National University / aakashgc +- Chris H / 2i2c / choldgraf +- Rowan / Curvenote / rowanc1 +- Damián / 2i2c / damianavila +- Chris S / EPFL / chrisjsewell + +**Apologies:** + +- Matt McKay / Australian National University / mmcky (Some updates below @Aakash can speak to them if needed) + +### Team updates + + +- [sphinx-exercise] has been re-factored to make the extension more maintainable. + - https://github.com/executablebooks/sphinx-exercise/pull/37 + - no major user facing changes (except some style updates to output) + - focus: use sphinx internals and AST as much as possible + - comments: found numref integration pretty hard to get right as numref is implemented (in part) in the translator phase + - reviews: Welcome any and all comments, also would love to chat with anyone with knowledge on Sphinx references (ref and numref) + - preview: https://61a58d957549766565b7eb2a--peaceful-feynman-936ca4.netlify.app/intro.html + - todo: write developer notes and lessons learnt for a general "admonition factory" with support for: + - custom styling + - custom options + - ref and numref integration + - autonumbering of nodes (i.e. numref) +- Rowan + - Javascript PR merges in + - Low hanging fruit is done on the JS side +- Chris S + - Working on VSCode, then JupyterLab integration with the MyST JS parser +- Steve + - Thebe - underlying infrastructure and moving away from JQuery +- Damian + - Working on 2 major PRs from the PyData Sphinx Theme +- Aakash + - Hasn't had time to work on EBP stuff much + - Interested in working on JTex for single-page PDFs + +### Agenda items + +- Discuss using `jtex` to build single-page PDFs of pages + - A quick low-hanging fruit would be to improve the CSS for printing pages in the book theme. + - However this won't satisfy most needs for publishing articles etc, so that would be a good usecase for `jtex` to tackle after dealing with low-hanging fruit. + - https://github.com/executablebooks/sphinx-book-theme/issues/435 + +## October 3rd 2021 + +### Attending + +- Rowan C / Curvenote / rowanc1 +- Steve Purves / Curvenote +- Chris H / 2i2c / choldgraf +- Anton Akhmerov / TU Delft / akhmerov +- Chris S / EPFL / chrisjsewell +- John S. / ANU +- Matt M. / ANU + +### Team updates + +- Chris H + - Community strategy for the EOL on our Sloan grant: https://github.com/executablebooks/meta/issues/493 + - Focusing most of my time trying to document things + un-block people + - Have played around with Pradyun's `sphinx-basic-ng` theme and really like it +- Steve P + - Progress on thebe: + - Major styles bug fixed + - First cut of status and activate button out + - Changed the docs & examples to use latest build - Thanks to recent RTD changes + - Started a Jupyterlite example - have JupyterLiteServer running in browser + - Next Steps: + - Test & Release + - we need to get an update out, as some projects are pinning earlier releases because of that CSS bug. (will coordinate with Chris H.) + - Refactoring + - separate jquery layer from low level api (opens up more options for integration) + - document jquery and low level api + - follow through on rename + - Include Jupyterlite kernel option (and generalise) + - Tackle enhancements from Chris S. in thebe / sphinx-thebe +- Matt M + - Sphinx V4 Support in https://github.com/executablebooks/jupyter-book/pull/1448 + - Refactor of sphinx-exercise (include LaTeX support for nodes) in final testing + - Would it be useful to have a generalized function / tool to quickly create semantic admonitions that are well-structured? Common Infrastructure? Might be a good way to document admonition -> theme style relationships. + - General agreement that this would be useful :-) + - Added LaTeX support to sphinx-proof + - Example: https://jstac.github.io/continuous_time_mcs/kolmogorov_bwd.html + - Semantic Admonitions! +- Chris S + - +1 for `sphinx-basic-ng` and merging focus of themes + - Participating in https://discourse.jupyter.org/t/inline-variable-insertion-in-markdown/10525 + - Made PoC PR to nbclient and nbformat + - Participated in Jupyter Community Meeting and spoke after with Angus Goose and Tony Fast and maybe set up further talks + - But will it go stale 😬 + - Added `skip-execution` cell tag functionality in nbclient: https://github.com/jupyter/nbclient/pull/151 + - Created myst-nb documentation for sqlalchemy tutorial and fed back to thebe + - Also added merging streams in myst-nb + - Might work with them further on their documentation: https://github.com/sqlalchemy/sqlalchemy/discussions/7138 + - Working with `cpitclaudel` to add "proper" docutils support for myst-parser + - Working with `tanelli` to update markdown-it-py to latest version of markdown-it (and finally fix the more complex url parsing) + - Will also eventually get back to improving jupyter-cache (https://github.com/executablebooks/jupyter-cache/pull/74) and long awaited myst-nb "refactor" +- Rowan + - Have been working on some other open source libraries for templating and exporting to word/latex/pdf from myst. + - Need to get back to myst in js work, next week! + - As implementation starts to happen on the TS/JS/Curvenote side, then it will open opportunities to have discussions around the "core vocabulary" of MyST - what admonitions should be supported everywhere + +### Agenda items + +#### Activity board + +- switch to new GitHub project boards? Demo here: https://github.com/orgs/executablebooks/projects/9/views/1 + +#### Community proposal doc +- Issue for comments/discussion: https://github.com/executablebooks/meta/issues/493 +- Proposal text: https://hackmd.io/@choldgraf/HkgAIHtQY +- What are some major next steps? + - Investigate the major steps we'd need to take to integrate with the Jupyter community + - Sticking points? + - We'll need to demonstrate that we have a good maintainability story + - Might have questions around the different parts of EBP + - Start to get feedback in the Jupyter ecosystem + - Threads in the discourse + the governance repo + - Talk to stakeholders in that community and see if there are any things we should think about it + - Crystallize the community strategy doc into some concrete next steps + - Decide on an order of operations for things to tackle + +## September 1st 2021 + +If you are joining the team video meeting, sign in below so we know who was here. Roll call: + +- Name / Affiliation / Handle +- Steve Purves / Curvenote / stevejpurves +- Rowan Cockett / Curvenote / rowanc1 +- Matt McKay / ANU / mmcky +- Chris Holdgraf / 2i2c / choldgraf +- Damián Avila / 2i2c / damianavila +- Chris Sewell / EPFL / chrisjsewell + +### Reports, updates, and celebrations + +This is a place to make announcements (without a need for discussion), short updates about what you've been up to, and shout-outs to contributors! We'll read through these at the beginning of the meeting. + +- THEBE: + - Some Thebe PRs in recently, and refactoring work started, other PRs ongoing + - Opened(opening) tickets on activity board for dicussion + - Refactoring, improving test coverage and docs + - Extended kernel support for Juptyerlite/pyodide, local proxy and better kernel selection + +- Some blog post about MyST in my personal blog (Damián) + - [Part 1](https://damianavila.github.io/blog/posts/a-deep-dive-into-myst-part-1-the-myst-parser-python-api-usage-in-nikola.html) + - [Part 2](https://damianavila.github.io/blog/posts/a-deep-dive-into-myst-part-2-the-myst-parser-docutils-and-sphinx.html) + - Eventually, a part 3 with some coding experiment/stuff + +- Sphinx 4 Upgrade (Matt) + - myst-nb PR https://github.com/executablebooks/MyST-NB/pull/356 + - Action: Cut a new release right after merge + +- ReadTheDocs integration (Chris S) + - That is now working + - PR is open - just waiting on documenting it up + - Action: Needs Doc Update + +### Agenda items + +Let's collect all potential agenda items here before the start of the meeting. We will then attempt to create a coherent agenda that fits in the 60m meeting slot. If there are similar items try and group them together. + +- Steve: Thebe directions, and future focus + - Setup a Project board on the thebe repo to better plan out work there, in more detail that the Activity Board maybe? + - A lot of issues in the repository + - Was a JupyterCon Sprint + a board from that + - Would like to do a bit of planning about what issues etc to tackle next + - There is definitely the opportunity now to move a bit more quickly and make changes before opening up for contributions + - Priority definely to get the CSS issue fixed so jupyter-book can move to the latest + +- Rowan: Thoughts on how to move JS-Myst forward (dev branch?) + - Few-no current dependencies on the work, and could possibly merge/deploy faster than some other repos. + - Might just be an off month?! + - Is there a way that we can ensure a consistent momentum of PRs so that we don't constantly block on people's reviews etc? + - Right now we are at an important early design moment on the JS side. Longer term it should be faster as the PRs will be less complex. + - In the meantime, Chris S will prioritize his EBP stuff on making sure these PRs get reviewed. + +- Chris H: in-line markdown execution? + - Long conversation about this here: https://discourse.jupyter.org/t/inline-variable-insertion-in-markdown/10525/37 + - The difficulty is in the specification + - mmcky: Related to adding execution to exercise nodes (which we are prototyping) and working out how to get execution embedded in directives. Current approach is through gated directives (which wouldn't work obviously for roles) + - An easy first step would be to do "read only inline markdown" + - This is a really popular feature of RMarkdown / Quarto Markdown + - Can you start with the restriction "we should be able to still run every code cell" + - From a user's perspective, you'd just want them to be able to put in variables with some kind of syntax e.g. `{{ }}` + - You'd need to go through the markdown cells and look for the `{{ }}` syntax and know what kinds of variables that you need. + - You'd then do the execution, and at the end of the time that + + +# 05th August 2021 + +If you are joining the team video meeting, sign in below so we know who was here. Roll call: + +- Matthew McKay / ANU / mmcky +- Steve Purves / Curvenote / stevejpurves +- Chris Sewell +- Rowan Cockett / Curvenote / rowanc1 +- Damián Avila / 2i2c / damianavila + +## Reports, updates, and celebrations + +This is a place to make announcements (without a need for discussion), short updates about what you've been up to, and shout-outs to contributors! We'll read through these at the beginning of the meeting. + +- Chris Sewell + - Sphinx Designs - better designed panels, will come into jupyter book at some point + - Jupyter Cache (for executing notebooks --> myst-nb --> JupyterBook) - rewritten + - Decides if code cells are changed and caches outputs. Then can grab the outputs from a local database. There is a CLI! + - There is also multi-processing on the notebooks! 💥 + - There is a new PR [draft](https://github.com/executablebooks/jupyter-cache/pull/74) + - Javascript! + - Added a new repo [docutils](https://github.com/executablebooks/markdown-it-docutils) + - There are same directives and roles from Python + - Same arguments/content + - This is now over to the vscode extension + - MyST as a "standard" + - What roles, directives, blocks to we need that are copied over in another language + - We should get the critical ones in a list somewhere! + - Forming specs in a collaborative way (Identify Good Tools for this?) + +- Matt McKay: Taken over maintaining sphinx-proof and sphinx-exercise (looking into enabling execution through gated directives as a proof of concept) + - "enabling execution in directives" (https://github.com/executablebooks/sphinx-exercise/issues/24) + - Adding a directive that is a question, and then how do you add code? + - Directive at the top level and then code and exercise. + - Violates the core design of the Jupyter Notebook -- nested code cells + - Got rid of "nested" concepts + - So how do you have "Gated" directive (open/close) around a few code cells + - This is more compliant with other tools like Jupytext + - Not sure if there is a Gated directive, or prior art in the Sphinx world? + - Other options: + - Metadata? Adding tags, etc. (Similar to nbgrader notebooks) + - Glue? (currently restricted to python, cather than cellId or something) + - Similar ideas playing with at Curvenote! (copy/paste) + - Consideration is mostly from an authorship perspective + +- Steve Purves: Thebe + - got started and made some improvements to the build and docs -- yarn, CI issues, local kernel convenience commands for development + - PR for a small feature - progress indicator + - Collaborating a bit with LibreText folks, would like to more and understand how they are using Thebe better + - LibreText made a big change to utilise the @jupyterlab/manager in place of a shim+custom code -- so waiting on that PR coming in before pushing on + - Next up: + - Connecting to many different kernals, and having some basic management on that! + +- Rowan: MyST in JS + - Working on myst in JS. Migrating work over to docutils and cleaning up along the way + - Still basics of various libraries + + - 10 or so PRs up at the moment! + - Basics: + - sub/sup/abbr + - internationalization? [each role/directive has a different name] + - Can generate various dicts to export + - Admonitions + - Math + - amsmath, dollar, math directive + - Figures + - Referencing/numbering PR + - numref/eq/ref + - Working to remove all of the markdown-it-myst functionality --> docutils w/ review from ChrisS + - How are we working with styles? + - There is some copying back and forth of the .css + +- Damián + - Learning Myst internals working in the Nikola + Myst story + - Probably writing a series of blog posts to push discussion forward about things I have found and engage the community + - MyST as a “standard” > What roles, directives, blocks do we need copied over Python (independent from Docutils/Sphinx)? Do we need a docultils/sphinx free Py representation of the "standard"? + +## Agenda items + +Let's collect all potential agenda items here before the start of the meeting. We will then attempt to create a coherent agenda that fits in the 60m meeting slot. If there are similar items try and group them together. + +- NAME: ITEM (EXPECTED TIME TO DISCUSS) + +- Discuss team activity board proposal (https://github.com/executablebooks/meta/issues/441) + - The goal of this board is to coordinate and plan our major work items, to make it clear who is working on what, and to help signal boost items for review/discussion. + - Is this board structure worth trying as-is or are there major changes to make to it? + - If we want to try this process, perhaps we should use this meeting to add some initial items to the board? + +**Discussion:** +1. This was discussed with the team and we agreed to use it this month to see how it works. +2. Any suggestions to simplify or improve the Activity Board was encouraged. +3. We focused on keeping the activity board pretty high level for cross-cutting communication and to help coordinate. +4. We also discussed keeping an eye on the `Request for Review` board to get visibility on PR's that need review and get +more distributed reviews across the team. +5. We should add the Activity Board as a standing discussion point for the monthly meeting + + - Steve: Next Steps for Thebe + - Refactoring the one big file into more manageable chunks + - More tests - current test coverage is poor, PRs go green but not much really covered which is maybe worse than no tests at all + - Kernel Status & Management - as part of refactor want to make it easier to get status of the kernel and lay ground work for the issues asking for: Kernel Status, Multiple Kernels and Jupyterlite connectivity; 412, 349, 271 + +**Discussion:** +1. This was discussed in the Reports section. + +## July 1st, 2021 + +If you are joining the team video meeting, sign in below so we know who was here. Roll call: + +- Matthew McKay / Australian National University (ANU) / mmcky +- Chris Sewell / EPFL / chrisjsewell +- Steve Purves / Curvenote / stevejpurves +- John Stachurski / ANU / jstac +- Chris Holdgraf / 2i2c / choldgraf +- Damián Avila / 2i2c / damianavila +- Aakash Gude / ANU / AakashGfude + +### Reports, updates, and celebrations + +This is a place to make announcements (without a need for discussion), short updates about what you've been up to, and shout-outs to contributors! We'll read through these at the beginning of the meeting. + +#### QuantEcon / ANU + +- QuantEcon: 4/5 Projects running Jupyter Book (remaining: julia) +- ANU Team: Working on Support for Individual Page PDF (via LaTeX) generation (Lead: @mmcky) + - Chris H notes that we should include updates about QuantEcon lectures conversion etc + - @mmcky to add links to executable books gallery to QuantEcon jupyter-book powered projects + +#### Chris S. + +**Recent work** +- Released [sphinx-external-toc](https://sphinx-external-toc.readthedocs.io) and integrated in Jupyter-Book + - Removes a lot of code from jupyter-book 😄, moving towards JB just being a thin wrapper for sphinx-extensions (last part is basically the `_config.yml`) + - Bit of a balancing act between; ease of use/understandanding, having a clear schema, having all the functionality of sphinx toctrees + - Integration in JB also includes addition of "pluggable" CLI +- Released markdown-it-py v1.0, i.e. fully stable +- Released myst-parser v0.15.0, with sphinx v4 support + - Should be moving all repositories to sphinx v4 +- Released [rst-to-myst](https://github.com/executablebooks/rst-to-myst) v0.3 + - Facile conversion of RST to MyST Markdown for a whole project + - Utilises mdformat + mdformat-myst to create the Markdown from markdown-it tokens + - Still working on mdformat-myst, but this essentially allows round trips Markdown text -> markdown-it tokens -> Markdown text. Could potentially be used for e.g. modifying links in downloadable notebooks +- Created [markdown-it-plugin-template](https://github.com/executablebooks/markdown-it-plugin-template); template repository with best practices for TypeScript library + - Used this to create: [markdown-it-amsmath](https://github.com/executablebooks/markdown-it-amsmath), with demo website: and [markdown-it-dollarmath](https://github.com/executablebooks/markdown-it-dollarmath), with demo website: + - Intend to make all plugins for parity with [mdit-py-plugins](https://github.com/executablebooks/mdit-py-plugins), and thus myst-parser extensions + +**Intended work** +- Update [MyST VS Code extension](https://github.com/executablebooks/myst-language-support) to use new markdown-it plugins + - Plus hopefully LSP support + - Look into recently finalised Notebook API: https://code.visualstudio.com/api/extension-guides/notebook +- Create sphinx-design: An iteration on sphinx-panels, but with the benefit of hindsight, and to more closely follow aspects of [Materials Design](https://material.io/design) and [Material-UI](https://material-ui.com/) + - class names that do not clash with sphinx ones (i.e. container) + - easier to set principle colors and width defaults +- Either in myst-parser or sphinx-design have nice way to generate stylised post-header banners, with e.g. author, date, read time (see https://github.com/executablebooks/MyST-Parser/issues/372) +- Improvements to myst-nb (+ jupyter-cache) + - Been gradually working on it, but lot of interconnected parts 😬 + - Revise the AST and transforms (post-transform -> transform) + - Drop nbconvert, jupyter-sphinx dependencies + - Better support for Markdown code output + - Better support for "multi-level" configuration (cell level > notebook level > conf.py level) + - Improve notebook execution and (maybe) drop `auto` mode in favour of `cache` + - Support for dynamic kernel names + - How to handle glue; ideally in a kernel agnostic manner, possily with the new substitution syntax + - Plugin CLI in jupyter-book for notebook execution + +#### Steve + +- Would like to: + - meet the team + - hear about thoughts on Thebelab as I'm prepping to do some work there + +#### Berkeley team + +- :wave: to Damian, onboarding etc +- Damian is playing around with the MyST parser and learning how it is structured, experimenting with a Nikola implementation of the parser. +- Chris is going to focus more of his time on team documentation, tool documentation for both users and developers, etc. + +### Agenda items + +- [sphinx-multitoc-numbering PR](https://github.com/executablebooks/jupyter-book/pull/1326) default behaviours (Lead: @mmcky) +- Workshop for Theme Specification (Prerequisites?, Preparation?, Timing?) (Lead: @mmcky) +- Any new features we should be focussing on from: https://executablebooks.org/en/latest/feature-vote.html + - One possibility: `sphinx-margin` (https://github.com/executablebooks/meta/discussions/409) + +**Didn't get to the following items** +- Bug issues in Jupyter Book: any important issues to address from: https://github.com/executablebooks/jupyter-book/issues?q=is%3Aissue+is%3Aopen+sort%3Aupdated-desc+label%3Abug +- Jupyter-book in RTD: how can we make it happen!? + - Chris H can give a short update from a conversation he had with Eric and Juan at RTD a few days ago. +- Theming: I am still keen on creating a shared infrastructure (https://github.com/executablebooks/meta/issues/279) and not necessarily being directly dependent on pydata-sphinx-theme + - Although @choldgraf mentioned about possible funding from NumFocus for pydata-sphinx-theme +- MyST as specification: what does that mean, how to "de-couple" from sphinx? + - What roles/directives/extensions are "core"? + - How to support additional roles/directives/extensions + - How does this tie in with "JavaScript" work (i.e. editor tools in VS Code, JupyterLab, etc)? + - Some notes in https://github.com/executablebooks/markdown-it-myst/blob/directives/docs/design.md + +John S: +- Hiring casual RAs? + +Chris H: +- Communicating the work we're doing - blog posts, tweets, etc? @executablebooks handle? +- Coordinating around deliverables, workstreams, requests for reviews, etc? diff --git a/docs/meetings/2022.md b/docs/meetings/2022.md new file mode 100644 index 0000000..c1704d9 --- /dev/null +++ b/docs/meetings/2022.md @@ -0,0 +1,171 @@ +# 2022 + +## April + +### Atending + +- Matthew McKay / ANU / mmcky +- Rowan Cockett / Curvenote / rowanc1 +- Chris Sewell / EPFL / chrisjsewell +- Chris Holdgraf / 2i2c / choldgraf +- Damián Avila / 2i2c / damianavila +- Aakash Gupta / ANU / aakashgc + +### Notes + +- `myst-spec` - @fwkoch, @rowanc1, @chrisjsewell + - over the past month, we've put together a language-agnostic repo of JSON schema types for the MyST AST nodes, building on existing `mdast` types. + - github: https://github.com/executablebooks/myst-spec + - dist: https://unpkg.com/browse/myst-spec/dist/ + - This also includes test cases, similar to the commonmark spec, which may be used to validate implementations of the spec + - `mystjs`, the markdown-it javascript implementation, already consumes myst spec + - [`unified-myst`](https://github.com/executablebooks/unified-myst), the new unified ecosystem implementation, will also be "spec compliant"! + - python side hasn't been adressed quite yet. +- [Unified MyST](https://github.com/executablebooks/unified-myst) + - There are some drawbacks of markdown-it + - It doesn't have a well specified AST + - myst-spec uses `mdast`, but that's a "unified ecosystem" thing, not a markdown-it thing + - Unified ecosystem has their own parser called [micromark](https://github.com/micromark/micromark) (and that is bundled in [remark](https://remarkjs.com/)) + - The [unified-myst](https://github.com/executablebooks/unified-myst) parser uses this framework instead of markdown-it + - Q: **What's the relationship between all of these parsers?** + - MyST-js would use the unified parser + - VSCode uses markdown-it-docutils (not even mystjs) + - Unified ecosystem is well-maintained with many extensions + - Q: **how do all of these pieces fit together for the end-user experience?** + - We should have a strategy meeting about the direction of these pieces so that we can align on a path forward. + - There is not currently a "product" at the end that is driving all of this forward (from an end-user's perspective). + - Q: **This seems like a lot of stuff to maintain. How can we sustain our efforts?** + - We have a bunch of stuff we've added to grow our maintenance burden. + - Can we tie the JS myst stuff to end-users in a way that we can fund? + - General agreement that we need a better story around the strategy and maintainability of this stack. + - Product strategy meeting? + - Bring in multiple stakeholders around a "product vision / strategy" for where the ecosystem could move going forward. + - Could we bring the learnings from the JS world back into the Python world to improve Jupyter Book. +- Curvenote renderer for multiple composable "JupyterBooks" + - Hosting a conference at end of April! + - https://transform.softwareunderground.org/ +- CH: New book theme is out, will update Jupyter Book to use it soon! + +**Note: we did not get to the following topics, but there are some notes** + +- [Gated Directives (sphinx-exercise)](https://ebp-sphinx-exercise.readthedocs.io/en/latest/syntax.html#alternative-gated-syntax). There has been some comments and interest in making this syntax more general so it could be used for other directives. + - Best Strategy? As an extension? + - Mention this with work being done on `myst-spec` +- First Developer Interview: "Deep Dive into Parsing" (with Chris Sewell) + - https://github.com/executablebooks/meta/issues/672#issue-1147568262 + - cover myst-parser, markdown-it-py + - Any questions? Please add it to the issue. + - Will post a recording to share more widely. +- JupyterLab Myst Extension as you go + - Editing experience in jupyter is very critical! +- MyST Docs + - New repository? Migrate and refurbish content? + +## March + +### Attending + +- Matt McKay / Australian National University / mmcky +- Rowan Cockett / Curvenote / @rowanc1 +- Chris Sewell / EPFL / @chrisjsewell +- Franklin Koch / Curvenote / @fwkoch +- Chris Holdgraf / 2i2c / @choldgraf +- Damián Avila / 2i2c / @damianavila +- Will Lachance / Voltus / @wlach + +### Short updates + +- mystjs initial release! + - https://github.com/executablebooks/mystjs + - https://executablebooks.github.io/mystjs + - And many other PRs updates in the JS world (e.g. docutils state, packaging) +- release of myst-parser v0.17.0 (https://myst-parser.readthedocs.io/en/latest/develop/\_changelog.html), jupyter-cache v0.5.0 (https://jupyter-cache.readthedocs.io/) + - Both feed into https://github.com/executablebooks/MyST-NB/pull/380 +- soon-to-be-released sphinx-book-theme (v0.3). in pre-release now but haven't gotten any negative feedback. https://github.com/executablebooks/sphinx-book-theme/releases/tag/v0.3.0rc1 +- sphinx-togglebutton also released w/ smaller footprint: https://github.com/executablebooks/sphinx-togglebutton/releases/tag/v0.3.0 + +### Notes + +#### mystjs / myst-spec discussion (@rowanc1) + +- Functionality, demo, myst "spec", reflections and thoughts on next steps! + +- Slides: https://docs.google.com/presentation/d/1lLJUgILhBAZeLyUXucHtQGsA9OjqlSoIjop5_bFVTWg/edit + +- Questions to answer + + - Should MyST exist as a first-class project within Executable Books? + - General yes + - Where should its "project" documentation live? + - `myst.tools` ? + - `spec.myst.tools`? + - Agree to do this + - Chris H will get the domain (UPDATE: Chris got the domain, now we just need to point something to it). + - Figure out the following two later: + - `python.myst.tools` -> myst-parser docs? (maybe we rename to `myst-python`) + - `js.myst.tools` -> `mystjs` docs? + - No decisions made here, we'll get to it later. + - How do we avoid duplicating a bunch of documentation etc. + - Define the core myst spec in a single repo, along with implementation-agnostic syntax + - Unit tests of the spec - where do they live? + - Use `chrisjsewell/myst-spec` for this? + - Generally agree this is a good place to do it, need to consolidate docs somehow. + +- myst-spec: https://myst-spec.readthedocs.io + + - Did a quick demo of the spec to compare w/ mystjs + +#### myst-nb / embedding code outputs (@chrisjsewell) + +- https://github.com/executablebooks/meta/issues/681 +- https://github.com/executablebooks/MyST-NB/pull/380 is basically ready to go apart from this + - Perhaps merge, then work on this separately? + +#### Developer interviews (@mmcky) + +- I will be coordinating a series of developer interviews (recorded via Zoom) on a range of topics +- Proposal: Deep dive into Parsing: myst-parser, markdown-it-py and MyST Specification +- Developer: @chrisjsewell +- Compiling questions which I will organise into a logical interview. Please add to https://github.com/executablebooks/meta/issues/672 +- Target Date: Mid-March 2022 + +## February + +### Attending + +- Matt McKay / ANU / @mmcky +- Steve Purves / Curvenote / @stevejpurves +- Chris H / 2i2c / @choldgraf +- Franklin Koch / Curvenote / @fwkoch +- Leif Walsh / Two Sigma / @leifwalsh +- Rowan Cockett / Curvenote / @rowanc1 +- Aakash Gupta / ANU / @aakashgc + +### Reports, updates, and celebrations + +- `sphinx-exercise==0.2.1` minor release with bug fixes and style updates +- `sphinx-book-theme` refactor is ready for others to take a look: https://github.com/executablebooks/sphinx-book-theme +- Aakash got the sphinx-theme-builder working for our theme: https://github.com/executablebooks/sphinx-book-theme/pull/469 + +### Agenda items + +- \[Matt\] Planning to place an emphasis on Bug/Issue reduction for February (perhaps after the refactors for myst-parser, myst-nb) + [Issue/Bug Reduction Priorities](https://github.com/executablebooks/meta/issues/649) is an issue on the meta repo to record any issues/bugs you are: + 1. able to work on + 1. really want fixed to move something forward +- \[Matt\] Organise a session with Chris S on "The architecture of Jupyter Book" to flesh out some of the areas I am not familiar with such as: markdown-it / markdown-it-py etc. to make improvements for developer docs. + - Can you record this?! (@mmcky -- good idea) + - Three Common Ways to Extend Sphinx + - Overview of JupyterBook +- \[Steve\] Thebe next steps - driven by requirements for hookup to interactive web components + - refactor to a (Typescript) core, update current js library to use it. + - library also made available on npm. + - Issue open for discussion here https://github.com/executablebooks/thebe/issues/536 + - @mmcky If you need beta testers, we would be happy to help test on QuantEcon projects such as: https://python-programming.quantecon.org/intro.html +- \[Chris H\] Feedback on book theme structure / visuals? + - Demo: https://sphinx-book-theme--472.org.readthedocs.build/en/472/ + - General agreement that this looks a lot better than the current released version. +- \[Chris H\] Feedback on carets / toggle buttons + - Demo: https://readthedocs.org/projects/sphinx-togglebutton/builds/15966583/ + - Chevrons instead of + signs? + - Widescreen toggles - could we re-use from sphinx-design ? diff --git a/docs/meetings/index.md b/docs/meetings/index.md new file mode 100644 index 0000000..8e6961c --- /dev/null +++ b/docs/meetings/index.md @@ -0,0 +1,36 @@ +# Team Meetings + +We occasionally have team meetings to discuss important topics in-person and to invite broader participation from others. + +## Meeting structure + +We use [this HackMD for an agenda](https://hackmd.io/THymMOAmSICp8rJdB6_Z1w?edit) and relevant meeting information. +If you'd like to discuss something at a meeting, add an entry in the HackMD before the meeting. + +We generally go through agenda items as they're written, though others are welcome to bring items to discuss during the meeting and discuss if we have time. + +After the meeting, we post meeting notes in a public location so that others can read and discuss asynchronously. + +## Meeting dates and calendar + +We try to meet monthly, on the **first Wednesday / Thursday of the month (depending on time zone) for 1 hour**. +Double-check in [our forum](https://github.com/executablebooks/meta/discussions) if you intend on joining, as we don't _always_ have the meeting depending on our availability and capacity. + +This calendar has dates for upcoming events in the Executable Books community. +You can [find the calendar URL here](https://calendar.google.com/calendar/embed?src=2nbh00hh9020u622nt0p5qhbek%40group.calendar.google.com&ctz=America%2FLos_Angeles). + +:::{dropdown} Meeting calendar + + + +::: + +## Meeting notes + +Below are notes from past meetings. + +```{toctree} +:maxdepth: 2 +2022.md +2021.md +``` diff --git a/docs/resources/accounts.md b/docs/resources/accounts.md new file mode 100644 index 0000000..9b5c6aa --- /dev/null +++ b/docs/resources/accounts.md @@ -0,0 +1,14 @@ +# Accounts and Services + +There are a few resources that the team shares to reduce redundancy, save time, and standardize our practices. +This page has a few major pieces. + +## Design Assets + +The Executable Books team uses [this Figma board](https://www.figma.com/file/ZptZUfzGznnT8GhIplnZBU/icon) to store its logo and other visual design assets. +Anybody can access these assets, though only core team members can edit them. + +## Google Drive + +We have [a shared Google Drive](https://drive.google.com/drive/folders/1lg4YpS3BpMnra4bVmkwfaTmYGYn42l3d?usp=sharing) where we store some assets like presentations and documents. +Access to this Google Drive is restricted to core team members. diff --git a/docs/resources/communication.md b/docs/resources/communication.md new file mode 100644 index 0000000..ebc7543 --- /dev/null +++ b/docs/resources/communication.md @@ -0,0 +1,35 @@ +# Communication Channels + +There are a few channels for communication depending on what you're trying to do. +Here are some of the major pieces: + +## Discussion Forum + +We have [a GitHub Discussions Forum](https://github.com/orgs/executablebooks/discussions) for all of our GitHub repositories (rather than using one forum per repository). +Our forum has general conversation about how to use our tools, free-form questions, ideas, discussions, etc. +Generally it is *not* a place for "project planning" and coordination (those are in [GitHub Issues](comms:issues)) +This is open to anybody, and is a space where people can ask questions and help one another. +Please feel free to provide support, advice, etc to anybody on this forum. + +(comms:issues)= +## GitHub Issues + +For conversations around specific actions, tasks, features, etc, we use GitHub issues in each repository. +Use these issues as a general place to understand what work is on our radar, and feel free to comment and add 👍 reactions to voice your support for certain changes. + +## Team Slack + +Our team communicates via a Slack channel at [ebp.slack.com/](https://ebp.slack.com/). +If you'd like access, please reach out to a core team member to discuss! + +## Email + +% ref: https://github.com/executablebooks/meta/discussions/412 to share password/access +We have an e-mail address at `executablebooks@gmail.com`. +All core team members should have access to this email, though it is not currently actively monitored by the team. + +## Team Meetings + +We also conduct (roughly) monthly team meetings via video chat. +Anybody is welcome to join these meetings! +For more information, see [](meetings/index.md). diff --git a/docs/strategy.md b/docs/strategy.md new file mode 100644 index 0000000..b4cc403 --- /dev/null +++ b/docs/strategy.md @@ -0,0 +1,56 @@ +# Goals and strategy + +The goal of the EBP is to build tools that facilitate creating +professional computational narratives (books, lecture series, articles, etc.) +using open source tools. We want users in the scientific, academic, +and data science communities to be able to do the following: + +* **Write their content in either markdown text files, or Jupyter Notebooks**. + These files include rich content - outputs from running code, references + and cross-references, equations, etc. +* **Execute content and cache the results**. Intelligent caching means + that only modified code cells are re-run. +* **Combine cached outputs with content files with a document model**. Using + the excellent [Sphinx](https://www.sphinx-doc.org/en/master/) documentation + stack, documents can include many features for publishing, such as + equations, cross-references, and citations. +* **Build interactive HTML or publication-quality PDF outputs**. Sometimes + users wish to create rich and interactive websites, other times they want to + send a high-quality PDF to a publisher. This system will treat both as + equal citizens. +* **Control everything above with a simple command-line interface**. Most + users should not have to know anything about Sphinx, caching, etc. A simple + user interface will hide most of the complexity of this process. + +## Guiding principles and constraints + +In running this project, we aim to adhere to several principles that we believe +will result in higher-quality technology that aligns with the core principles +of the open source community. Here are a few key components: + +* **Give equal support to casual users and power users**. Complicating the feature + space to support a "power user feature" should be done with great care. +* **Build modular components that are useful elsewhere**. Rather than building + a single vertical stack, find parts of the workflow that naturally separate. + Create modular tools for these parts so that they may benefit the community + outside of the context of building interactive books. +* **Use pre-existing technology where possible**. Rather than re-inventing + the wheel, make every effort to utilize pre-existing open source tech. +* **Use pre-existing standards where possible**. In the event that we must + create new patterns of content creation or tooling, utilize prior art in + the open source community as much as possible, especially when it comes to + markup languages. +* **Contribute improvements upstream**. Where we utilize pre-existing tools, + contribute improvements to them as we build off of them for this project. +* **Design for the future**. While we have a bit of funding now, it won't last + forever. This means the technology should be easy for potential developers + to read, understand, and modify/improve. +* **Users should not need to know anything about the build system**. If they + want to dig into the guts of our infrastructure, they can, but knowledge + of Sphinx, Latex, and so on should not be a requirement. They should only need to + use a simple tool to control the process. +* **Don't try to do everything**. Focus our tools on publishing + computational documents with reasonable choices made for the user. + +Browse the rest of this site for more information about what we're working +on and our plans for what's coming next. \ No newline at end of file diff --git a/tox.ini b/tox.ini index 3097220..e0917d5 100644 --- a/tox.ini +++ b/tox.ini @@ -23,9 +23,6 @@ deps = -rrequirements.txt setenv = # GITHUB_TOKEN is used by ghapi to update the issues voting GITHUB_TOKEN = {env:GITHUB_TOKEN:} -passenv = - SKIP_CONTRIBUTE - SKIP_TEAM whitelist_externals = rm echo