From 9ed7a2c97d9c965ed8bd5e1f0593f4d52ae64b81 Mon Sep 17 00:00:00 2001 From: Julia Kent <46687291+jukent@users.noreply.github.com> Date: Thu, 10 Jul 2025 09:35:53 -0600 Subject: [PATCH 01/15] add blog post --- CONTRIBUTING.md | 4 +-- portal/myst.yml | 1 + portal/posts/mystification.md | 65 +++++++++++++++++++++++++++++++++++ 3 files changed, 68 insertions(+), 2 deletions(-) create mode 100644 portal/posts/mystification.md diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 7d3ed1427..179493b87 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -12,12 +12,12 @@ Within the `portal/posts/` folder add your `.md` blog file with the following he --- date: YYYY-MM-DD -author: First Last +author: github-handle tags: [sample-tag] --- ``` -The post will automatically be recognized and displayed when you build the portal site. +To display the post, you need to add the file path in the `myst.yml` configuration file under `toc:` -> `- title: Blog` -> `children:`. Please add the newest blog post at the top of the blog file list. This is temporary until we have better blog infrastructure. ## Instructions for building the portal site diff --git a/portal/myst.yml b/portal/myst.yml index 7e1a285f6..c68099785 100644 --- a/portal/myst.yml +++ b/portal/myst.yml @@ -14,6 +14,7 @@ project: children: # - pattern: posts/*.md # Temporary until we have blog infrastructure: explicit list of posts by date (newest first) + - file: posts/mystification.md - file: posts/cookoff2025-website.md - file: posts/binderhub_status.md - file: posts/new-cookbooks.md diff --git a/portal/posts/mystification.md b/portal/posts/mystification.md new file mode 100644 index 000000000..bd120c928 --- /dev/null +++ b/portal/posts/mystification.md @@ -0,0 +1,65 @@ +--- +title: Reflections on MyST-ification +subtitle: Project Pythia’s transition to MyST and JupyterBook2 architecture +date: 2025-07-11 +authors: jukent, brian-rose, dcamron, kafitzgerald +tags: [myst] +--- + +Project Pythia recently transitioned from a [Sphinx-based JupyterBook](https://jupyterbook.org/en/stable/intro.html) architecture to using [MyST Markdown](https://mystmd.org/), which serves as the foundation for the upcoming [JupyterBook 2](https://next.jupyterbook.org/)! + +## Our Motivation for MyST +We began the process of transitioning to MyST in the summer of 2024 at the annual [Project Pythia Cook-off hackathon](https://projectpythia.org/posts/new-cookbooks). At that event, members of the MyST team demonstrated the current [alpha-version of the technology](https://executablebooks.org/en/latest/blog/2024-06-14-project-pythia-mystmd/) and coached us through the boilerplate code necessary to make some of our key resources build with MyST. + +The new MyST architecture was very appealing for several reasons: +Sustainability. Our current Sphinx-based architecture was becoming clunky and hard to maintain as members joined or left the project, and required too much boilerplate code in individual cookbook repos which presented a barrier to would-be new contributors. MyST offered a much more streamlined alternative to keep our community project growing. +Staying on the leading edge of best practices. We are an open-source community resource that teaches open-source coding practices, so it’s important that our own sites continue to be useful models for the broader community. +Making cookbooks better! A lot of the new functionality in MyST is really well suited to the cookbooks, including things like [cross-referencing](https://mystmd.org/guide/cross-references) and embedding content (Embed & Include Content - MyST Markdown) and automated [bibliographies](https://mystmd.org/guide/citations). +Cross-pollination with the core developers! Having the MyST developers invested in our use-case as a demo as they learn, understand, and develop functionality that will be particularly useful to us (and users that come after) is a really nice feedback loop from both a community and technological stand point. + +## MyST for sustainability +### Our aging infrastructure +One example of our maintainability challenge was keeping our bespoke [Pythia-Sphinx theme](https://github.com/ProjectPythia/sphinx-pythia-theme) up-to-date. Upstream dependency updates and cascading syntax changes will always be a concern for the open source community. Combine that with browser default settingschanging since the birth of this project in 2020 (particularly for dark-mode), and there were many HTML and CSS customizations that were no longer displaying as intended. For this reason, we decided to stick as closely to the default [MyST book-theme](https://mystmd.org/guide/website-templates) as serves our purposes. The fewer moving pieces for a new contributor in our open source community to have to be spun up on the better. And with our current collaborations with the MyST team, it’s better to put effort into helping to improve the core tools rather than create unique new customizations. + +### Repository sprawl +Another maintainability challenge was propagating changes across many GitHub repositories. Within the [Project Pythia Github organization](https://github.com/ProjectPythia/) we currently have 75 different repositories, the vast majority of which contain some website source under the big trenchcoat masquerading as one single Project Pythia website. Each repository is deployed within the domain, but there are separate repositories for our [home page](https://projectpythia.org/), [Foundations book](https://foundations.projectpythia.org/), [resource](https://projectpythia.org/resource-gallery/) and [Cookbooks galleries](https://cookbooks.projectpythia.org/), and for each individual Cookbook. With the Sphinx infrastructure, while the site theming could be abstracted into its own package, other changes to the site configuration or appearance, specifically of the links included in the top nav-bar or footer, would have to be individually updated in every single repository for consistency. We could update our Cookbook Template repository, but GitHub has no one way of sending those divergent-git-history changes to the various Cookbook repositories that leveraged that template. The MyST [`extends` keyword](https://mystmd.org/guide/frontmatter#composing-myst-yml) in the configuration file allows us to not only abstract theming, but also configuration commands and content. Future changes to the site navbar will only have to be made in one place, and individual Cookbook authors will be able to focus on their own content with much reduced boilerplate! + +## Making the Switch +### Getting to yes +Before we could get to the point of “only having to make changes in one place” we had to make a lot of changes in ALL of our repositories. Initially the process was exploratory. We had a separate MyST GitHub organization to host the MyST versions of our home page and galleries. Various team members MyST-ified a Cookbook repository or two just to test the process. Over the fall, winter, and spring we stayed in touch with the MyST devs and helped scope out updates and releases that would meet our needs. Come spring, we faced a decision: Do we complete the transition before or after our [August 2025 Cook-off Hackathon](https://projectpythia.org/pythia-cookoff-2025/)? +Ultimately, we decided to flip the switch, transitioning fully to MyST before the hackathon. Reasons for this decision included: stability for contributors (you wouldn’t want to work on a project for it to immediately become unrecognizable to you afterwards) and fewer newly created repositories to track down that don’t match our [new template](https://github.com/ProjectPythia/cookbook-template). With having everyone at the hackathon working within the MyST ecosystem, we decided to transition incomplete Cookbooks as well as current published ones. + +### Mystification of our collaborative workflows +The first task was to adapt the shared GitHub Actions workflows that we rely on for building and deploying all our sites. These unique workflows are a key piece of Pythia’s collaboration infrastructure because they enable rich previews of pull requests rendered through the same machinery that builds the actual sites, so we don’t have to guess what the end product will look like – whether we’re changing website formatting or executable content in notebooks. Adapting the workflows to play well with MyST included changing the default build command[^buildcommand] and some new logic to handle MyST’s `BASE_URL` requirement[^baseurl]. With the mystified collaborative workflows in place we were ready for the big, coordinated push this transition would require. + +### Sprint to the finish line +In mid-June, we met over the course of several hack days to tinker, to test, to review each other’s pull requests, and to deploy the new websites. We had a spreadsheet to track our progress that listed 57 repositories (including 52 Cookbooks) that needed to be Myst-ified or cleaned up. Knowing that there were paths that were pointing to the old sandbox “with myST” GitHub organization, even Cookbooks that were previously transitioned to MyST would need to be combed through to make sure all updated changes were reflected. These changes included adding in the new boilerplate code to leverage MyST, updating all action calls, removing any stale code, and other miscellaneous changes to the README.md files and contribution guides. A typical Cookbook had additions, modifications, and deletions of 12 total files ([example](https://github.com/ProjectPythia/xbatcher-ML-1-cookbook/pull/16/files)), which is a total of over 600 files changed in a day. We could not have completed the process so quickly without the rapid response from the upstream MyST MD team. + +## Looking Ahead +After the Sprint day, continued work went into documentation for the new contribution guides, working with the MyST team to identify bugs and new feature requests, and finishing up various tasks. We are currently working diligently on the Footer, which will be manually added to 4 key sites while we wait for a MyST release that allows for the extension of site-parts in the configuration. Other key features we hope to include are: gallery extensions for custom gallery card filtering, custom internal vs external link treatment and checking, and better blog functionality. Even without these features, we are thrilled and proud to have rolled out the new infrastructure and a fresh new look for our websites in advance of our summer Cook-off Hackathon. + +We hope you’ll find the Cookbook creation process to be streamlined! **Test out our new architecture at this year’s [Cook-off Hackathon](https://projectpythia.org/pythia-cookoff-2025/)** (August 5-8 at the NSF NCAR Mesa Lab in Boulder). Register by July 20th. Or drop by any of our open community meetings (https://projectpythia.org/#join-us). + +## Attribution +Thanks to everyone who was involved in our sprint day (alphabetical): +Drew Camron +Rowan Cockett +Katelyn FitzGerald +Robert Ford +Julia Kent +Angus Hollands +Christian Okyere +Brian Rose +Kevin Tyle + +[^buildcommand]: Our shared [build-book.yaml workflow](https://github.com/ProjectPythia/cookbook-actions/blob/main/.github/workflows/build-book.yaml) accepts an arbitrary build_command input argument to accommodate using different HTML building tools. We switched the default build_command from jupyter-book build . to myst build –execute –html. You can restore previous behavior by explicitly setting the input argument from your workflow file like this: +``` +jobs: + build: + uses: ProjectPythia/cookbook-actions/.github/workflows/build-book.yaml@main + with: + build_commandl: 'jupyter-book build .' +``` +Reach out if this is causing problems! + +[^baseurl]: Unlike Sphinx, [MyST needs to know where a site is being deployed at build time](https://mystmd.org/guide/deployment-github-pages#base-url-configuration-for-github-pages). That means we have to pass different arguments depending on whether we are building a preview or the actual site since the previews are deployed to subdirectories of each site’s base URL. From 5f5292816efcea892806df49652ed48b7ceae5aa Mon Sep 17 00:00:00 2001 From: Julia Kent <46687291+jukent@users.noreply.github.com> Date: Thu, 10 Jul 2025 09:43:48 -0600 Subject: [PATCH 02/15] render author names and footnote --- portal/posts/mystification.md | 36 +++++++++++++++++------------------ 1 file changed, 18 insertions(+), 18 deletions(-) diff --git a/portal/posts/mystification.md b/portal/posts/mystification.md index bd120c928..c0c28ef5b 100644 --- a/portal/posts/mystification.md +++ b/portal/posts/mystification.md @@ -2,7 +2,7 @@ title: Reflections on MyST-ification subtitle: Project Pythia’s transition to MyST and JupyterBook2 architecture date: 2025-07-11 -authors: jukent, brian-rose, dcamron, kafitzgerald +authors: [jukent, brian-rose, dcamron, kafitzgerald] tags: [myst] --- @@ -42,24 +42,24 @@ We hope you’ll find the Cookbook creation process to be streamlined! **Test ou ## Attribution Thanks to everyone who was involved in our sprint day (alphabetical): -Drew Camron -Rowan Cockett -Katelyn FitzGerald -Robert Ford -Julia Kent -Angus Hollands -Christian Okyere -Brian Rose -Kevin Tyle +- Drew Camron +- Rowan Cockett +- Katelyn FitzGerald +- Robert Ford +- Julia Kent +- Angus Hollands +- Christian Okyere +- Brian Rose +- Kevin Tyle [^buildcommand]: Our shared [build-book.yaml workflow](https://github.com/ProjectPythia/cookbook-actions/blob/main/.github/workflows/build-book.yaml) accepts an arbitrary build_command input argument to accommodate using different HTML building tools. We switched the default build_command from jupyter-book build . to myst build –execute –html. You can restore previous behavior by explicitly setting the input argument from your workflow file like this: -``` -jobs: - build: - uses: ProjectPythia/cookbook-actions/.github/workflows/build-book.yaml@main - with: - build_commandl: 'jupyter-book build .' -``` -Reach out if this is causing problems! + ``` + jobs: + build: + uses: ProjectPythia/cookbook-actions/.github/workflows/build-book.yaml@main + with: + build_commandl: 'jupyter-book build .' + ``` + Reach out if this is causing problems! [^baseurl]: Unlike Sphinx, [MyST needs to know where a site is being deployed at build time](https://mystmd.org/guide/deployment-github-pages#base-url-configuration-for-github-pages). That means we have to pass different arguments depending on whether we are building a preview or the actual site since the previews are deployed to subdirectories of each site’s base URL. From 341ab8cdcbb43d175328ab89ffec6ed590e3712f Mon Sep 17 00:00:00 2001 From: Julia Kent <46687291+jukent@users.noreply.github.com> Date: Thu, 10 Jul 2025 09:57:52 -0600 Subject: [PATCH 03/15] fix date --- portal/posts/mystification.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/portal/posts/mystification.md b/portal/posts/mystification.md index c0c28ef5b..62168208b 100644 --- a/portal/posts/mystification.md +++ b/portal/posts/mystification.md @@ -1,7 +1,7 @@ --- title: Reflections on MyST-ification subtitle: Project Pythia’s transition to MyST and JupyterBook2 architecture -date: 2025-07-11 +date: 2025-07-10 authors: [jukent, brian-rose, dcamron, kafitzgerald] tags: [myst] --- From 7e27d3777b501413c32a5deddcb5cd9a0984d951 Mon Sep 17 00:00:00 2001 From: Julia Kent <46687291+jukent@users.noreply.github.com> Date: Thu, 10 Jul 2025 14:38:22 -0600 Subject: [PATCH 04/15] Update portal/posts/mystification.md Co-authored-by: Brian Rose --- portal/posts/mystification.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/portal/posts/mystification.md b/portal/posts/mystification.md index 62168208b..31ac3d5c8 100644 --- a/portal/posts/mystification.md +++ b/portal/posts/mystification.md @@ -14,7 +14,7 @@ We began the process of transitioning to MyST in the summer of 2024 at the annua The new MyST architecture was very appealing for several reasons: Sustainability. Our current Sphinx-based architecture was becoming clunky and hard to maintain as members joined or left the project, and required too much boilerplate code in individual cookbook repos which presented a barrier to would-be new contributors. MyST offered a much more streamlined alternative to keep our community project growing. Staying on the leading edge of best practices. We are an open-source community resource that teaches open-source coding practices, so it’s important that our own sites continue to be useful models for the broader community. -Making cookbooks better! A lot of the new functionality in MyST is really well suited to the cookbooks, including things like [cross-referencing](https://mystmd.org/guide/cross-references) and embedding content (Embed & Include Content - MyST Markdown) and automated [bibliographies](https://mystmd.org/guide/citations). +Making cookbooks better! A lot of the new functionality in MyST is really well suited to the cookbooks, including things like [cross-referencing](https://mystmd.org/guide/cross-references) and [embedding content](Embed & Include Content - MyST Markdown) and automated [bibliographies](https://mystmd.org/guide/citations). Cross-pollination with the core developers! Having the MyST developers invested in our use-case as a demo as they learn, understand, and develop functionality that will be particularly useful to us (and users that come after) is a really nice feedback loop from both a community and technological stand point. ## MyST for sustainability From 22b98fb181279b78d32077bb8d6403ef7f772683 Mon Sep 17 00:00:00 2001 From: Julia Kent <46687291+jukent@users.noreply.github.com> Date: Thu, 10 Jul 2025 14:38:51 -0600 Subject: [PATCH 05/15] Update portal/posts/mystification.md Co-authored-by: Brian Rose --- portal/posts/mystification.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/portal/posts/mystification.md b/portal/posts/mystification.md index 31ac3d5c8..3a59bfcca 100644 --- a/portal/posts/mystification.md +++ b/portal/posts/mystification.md @@ -58,7 +58,7 @@ Thanks to everyone who was involved in our sprint day (alphabetical): build: uses: ProjectPythia/cookbook-actions/.github/workflows/build-book.yaml@main with: - build_commandl: 'jupyter-book build .' + build_command: 'jupyter-book build .' ``` Reach out if this is causing problems! From 933ff04151c39711e0ca6039fbcc26a1abd97cf9 Mon Sep 17 00:00:00 2001 From: Julia Kent <46687291+jukent@users.noreply.github.com> Date: Thu, 10 Jul 2025 14:39:06 -0600 Subject: [PATCH 06/15] Update portal/posts/mystification.md Co-authored-by: Brian Rose --- portal/posts/mystification.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/portal/posts/mystification.md b/portal/posts/mystification.md index 3a59bfcca..2fa2c468c 100644 --- a/portal/posts/mystification.md +++ b/portal/posts/mystification.md @@ -38,7 +38,7 @@ In mid-June, we met over the course of several hack days to tinker, to test, to ## Looking Ahead After the Sprint day, continued work went into documentation for the new contribution guides, working with the MyST team to identify bugs and new feature requests, and finishing up various tasks. We are currently working diligently on the Footer, which will be manually added to 4 key sites while we wait for a MyST release that allows for the extension of site-parts in the configuration. Other key features we hope to include are: gallery extensions for custom gallery card filtering, custom internal vs external link treatment and checking, and better blog functionality. Even without these features, we are thrilled and proud to have rolled out the new infrastructure and a fresh new look for our websites in advance of our summer Cook-off Hackathon. -We hope you’ll find the Cookbook creation process to be streamlined! **Test out our new architecture at this year’s [Cook-off Hackathon](https://projectpythia.org/pythia-cookoff-2025/)** (August 5-8 at the NSF NCAR Mesa Lab in Boulder). Register by July 20th. Or drop by any of our open community meetings (https://projectpythia.org/#join-us). +We hope you’ll find the Cookbook creation process to be streamlined! **Test out our new architecture at this year’s [Cook-off Hackathon](https://projectpythia.org/pythia-cookoff-2025/)** (August 5-8 at the NSF NCAR Mesa Lab in Boulder). Register by July 20th. Or drop by any of our [open community meetings](https://projectpythia.org/#join-us). ## Attribution Thanks to everyone who was involved in our sprint day (alphabetical): From 283b4eaa95793ec6bb54c756617a3ac8541fa886 Mon Sep 17 00:00:00 2001 From: Julia Kent <46687291+jukent@users.noreply.github.com> Date: Thu, 10 Jul 2025 14:39:59 -0600 Subject: [PATCH 07/15] Update portal/posts/mystification.md Co-authored-by: Katelyn FitzGerald <7872563+kafitzgerald@users.noreply.github.com> --- portal/posts/mystification.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/portal/posts/mystification.md b/portal/posts/mystification.md index 2fa2c468c..8487cc301 100644 --- a/portal/posts/mystification.md +++ b/portal/posts/mystification.md @@ -36,7 +36,7 @@ The first task was to adapt the shared GitHub Actions workflows that we rely on In mid-June, we met over the course of several hack days to tinker, to test, to review each other’s pull requests, and to deploy the new websites. We had a spreadsheet to track our progress that listed 57 repositories (including 52 Cookbooks) that needed to be Myst-ified or cleaned up. Knowing that there were paths that were pointing to the old sandbox “with myST” GitHub organization, even Cookbooks that were previously transitioned to MyST would need to be combed through to make sure all updated changes were reflected. These changes included adding in the new boilerplate code to leverage MyST, updating all action calls, removing any stale code, and other miscellaneous changes to the README.md files and contribution guides. A typical Cookbook had additions, modifications, and deletions of 12 total files ([example](https://github.com/ProjectPythia/xbatcher-ML-1-cookbook/pull/16/files)), which is a total of over 600 files changed in a day. We could not have completed the process so quickly without the rapid response from the upstream MyST MD team. ## Looking Ahead -After the Sprint day, continued work went into documentation for the new contribution guides, working with the MyST team to identify bugs and new feature requests, and finishing up various tasks. We are currently working diligently on the Footer, which will be manually added to 4 key sites while we wait for a MyST release that allows for the extension of site-parts in the configuration. Other key features we hope to include are: gallery extensions for custom gallery card filtering, custom internal vs external link treatment and checking, and better blog functionality. Even without these features, we are thrilled and proud to have rolled out the new infrastructure and a fresh new look for our websites in advance of our summer Cook-off Hackathon. +After the sprint day, continued work went into documentation for the new contribution guides, working with the MyST team to identify bugs and new feature requests, and finishing up various tasks. We are currently working diligently on the footer, which will be manually added to 4 key sites while we wait for a MyST release that allows for the extension of site-parts in the configuration. Other key features we hope to include are: gallery extensions for custom gallery card filtering, custom internal vs external link treatment and checking, and better blog functionality. Even without these features, we are thrilled and proud to have rolled out the new infrastructure and a fresh new look for our websites in advance of our summer Cook-off Hackathon. We hope you’ll find the Cookbook creation process to be streamlined! **Test out our new architecture at this year’s [Cook-off Hackathon](https://projectpythia.org/pythia-cookoff-2025/)** (August 5-8 at the NSF NCAR Mesa Lab in Boulder). Register by July 20th. Or drop by any of our [open community meetings](https://projectpythia.org/#join-us). From e21e6a08d7864fb4aa97240d40c8a428ed4f53e3 Mon Sep 17 00:00:00 2001 From: Julia Kent <46687291+jukent@users.noreply.github.com> Date: Thu, 10 Jul 2025 14:40:56 -0600 Subject: [PATCH 08/15] alphabet --- portal/posts/mystification.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/portal/posts/mystification.md b/portal/posts/mystification.md index 8487cc301..b96174044 100644 --- a/portal/posts/mystification.md +++ b/portal/posts/mystification.md @@ -46,8 +46,8 @@ Thanks to everyone who was involved in our sprint day (alphabetical): - Rowan Cockett - Katelyn FitzGerald - Robert Ford -- Julia Kent - Angus Hollands +- Julia Kent - Christian Okyere - Brian Rose - Kevin Tyle From 221be451ca58942a111f28a15bbc842f832a66e1 Mon Sep 17 00:00:00 2001 From: Julia Kent <46687291+jukent@users.noreply.github.com> Date: Thu, 10 Jul 2025 15:23:28 -0600 Subject: [PATCH 09/15] Update portal/posts/mystification.md Co-authored-by: Katelyn FitzGerald <7872563+kafitzgerald@users.noreply.github.com> --- portal/posts/mystification.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/portal/posts/mystification.md b/portal/posts/mystification.md index b96174044..c4b2d221a 100644 --- a/portal/posts/mystification.md +++ b/portal/posts/mystification.md @@ -33,7 +33,7 @@ Ultimately, we decided to flip the switch, transitioning fully to MyST before th The first task was to adapt the shared GitHub Actions workflows that we rely on for building and deploying all our sites. These unique workflows are a key piece of Pythia’s collaboration infrastructure because they enable rich previews of pull requests rendered through the same machinery that builds the actual sites, so we don’t have to guess what the end product will look like – whether we’re changing website formatting or executable content in notebooks. Adapting the workflows to play well with MyST included changing the default build command[^buildcommand] and some new logic to handle MyST’s `BASE_URL` requirement[^baseurl]. With the mystified collaborative workflows in place we were ready for the big, coordinated push this transition would require. ### Sprint to the finish line -In mid-June, we met over the course of several hack days to tinker, to test, to review each other’s pull requests, and to deploy the new websites. We had a spreadsheet to track our progress that listed 57 repositories (including 52 Cookbooks) that needed to be Myst-ified or cleaned up. Knowing that there were paths that were pointing to the old sandbox “with myST” GitHub organization, even Cookbooks that were previously transitioned to MyST would need to be combed through to make sure all updated changes were reflected. These changes included adding in the new boilerplate code to leverage MyST, updating all action calls, removing any stale code, and other miscellaneous changes to the README.md files and contribution guides. A typical Cookbook had additions, modifications, and deletions of 12 total files ([example](https://github.com/ProjectPythia/xbatcher-ML-1-cookbook/pull/16/files)), which is a total of over 600 files changed in a day. We could not have completed the process so quickly without the rapid response from the upstream MyST MD team. +In mid-June, we met over the course of several hack days to tinker, to test, to review each other’s pull requests, and to deploy the new websites. We had a spreadsheet to track our progress that listed 57 repositories (including 52 Cookbooks) that needed to be MyST-ified or cleaned up. Knowing that there were paths that were pointing to the old sandbox “with MyST” GitHub organization, even Cookbooks that were previously transitioned to MyST would need to be combed through to make sure all updated changes were reflected. These changes included adding in the new boilerplate code to leverage MyST, updating all action calls, removing any stale code, and other miscellaneous changes to the README.md files and contribution guides. A typical Cookbook had additions, modifications, and deletions of 12 total files ([example](https://github.com/ProjectPythia/xbatcher-ML-1-cookbook/pull/16/files)), which is a total of over 600 files changed in a day. We could not have completed the process so quickly without the rapid response from the upstream MyST Markdown team. ## Looking Ahead After the sprint day, continued work went into documentation for the new contribution guides, working with the MyST team to identify bugs and new feature requests, and finishing up various tasks. We are currently working diligently on the footer, which will be manually added to 4 key sites while we wait for a MyST release that allows for the extension of site-parts in the configuration. Other key features we hope to include are: gallery extensions for custom gallery card filtering, custom internal vs external link treatment and checking, and better blog functionality. Even without these features, we are thrilled and proud to have rolled out the new infrastructure and a fresh new look for our websites in advance of our summer Cook-off Hackathon. From 22c7bb16f776bb02c3ad42496b7ed1a46189fa8e Mon Sep 17 00:00:00 2001 From: Julia Kent <46687291+jukent@users.noreply.github.com> Date: Fri, 11 Jul 2025 10:10:03 -0600 Subject: [PATCH 10/15] new file structure --- CONTRIBUTING.md | 2 +- portal/myst.yml | 16 ++++++++-------- portal/posts/{ => 2023}/cookoff2023.md | 0 .../posts/{ => 2023}/cookoff2024-savethedate.md | 0 portal/posts/{ => 2023}/fundraiser.md | 0 portal/posts/{ => 2024}/cookoff2024-website.md | 0 portal/posts/{ => 2025}/binderhub_status.md | 0 portal/posts/{ => 2025}/cookoff2025-website.md | 0 portal/posts/{ => 2025}/mystification.md | 0 portal/posts/{ => 2025}/new-cookbooks.md | 0 10 files changed, 9 insertions(+), 9 deletions(-) rename portal/posts/{ => 2023}/cookoff2023.md (100%) rename portal/posts/{ => 2023}/cookoff2024-savethedate.md (100%) rename portal/posts/{ => 2023}/fundraiser.md (100%) rename portal/posts/{ => 2024}/cookoff2024-website.md (100%) rename portal/posts/{ => 2025}/binderhub_status.md (100%) rename portal/posts/{ => 2025}/cookoff2025-website.md (100%) rename portal/posts/{ => 2025}/mystification.md (100%) rename portal/posts/{ => 2025}/new-cookbooks.md (100%) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 179493b87..6350b0ab7 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -6,7 +6,7 @@ contribution guidelines (such as detailed description of Project Pythia structure, forking, repository cloning, branching, etc.). ## Instructions for adding a blog post -Within the `portal/posts/` folder add your `.md` blog file with the following heading: +Within the `portal/posts/[YEAR]` folder add your `.md` blog file with the following heading: ``` --- diff --git a/portal/myst.yml b/portal/myst.yml index c68099785..e49c852e3 100644 --- a/portal/myst.yml +++ b/portal/myst.yml @@ -14,14 +14,14 @@ project: children: # - pattern: posts/*.md # Temporary until we have blog infrastructure: explicit list of posts by date (newest first) - - file: posts/mystification.md - - file: posts/cookoff2025-website.md - - file: posts/binderhub_status.md - - file: posts/new-cookbooks.md - - file: posts/cookoff2024-website.md - - file: posts/cookoff2024-savethedate.md - - file: posts/fundraiser.md - - file: posts/cookoff2023.md + - file: posts/2025/mystification.md + - file: posts/2025/cookoff2025-website.md + - file: posts/2025/binderhub_status.md + - file: posts/2025/new-cookbooks.md + - file: posts/2024/cookoff2024-website.md + - file: posts/2023/cookoff2024-savethedate.md + - file: posts/2023/fundraiser.md + - file: posts/2023/cookoff2023.md - file: contributing.md - file: cookbook-guide.md - file: quick-cookbook-guide.md diff --git a/portal/posts/cookoff2023.md b/portal/posts/2023/cookoff2023.md similarity index 100% rename from portal/posts/cookoff2023.md rename to portal/posts/2023/cookoff2023.md diff --git a/portal/posts/cookoff2024-savethedate.md b/portal/posts/2023/cookoff2024-savethedate.md similarity index 100% rename from portal/posts/cookoff2024-savethedate.md rename to portal/posts/2023/cookoff2024-savethedate.md diff --git a/portal/posts/fundraiser.md b/portal/posts/2023/fundraiser.md similarity index 100% rename from portal/posts/fundraiser.md rename to portal/posts/2023/fundraiser.md diff --git a/portal/posts/cookoff2024-website.md b/portal/posts/2024/cookoff2024-website.md similarity index 100% rename from portal/posts/cookoff2024-website.md rename to portal/posts/2024/cookoff2024-website.md diff --git a/portal/posts/binderhub_status.md b/portal/posts/2025/binderhub_status.md similarity index 100% rename from portal/posts/binderhub_status.md rename to portal/posts/2025/binderhub_status.md diff --git a/portal/posts/cookoff2025-website.md b/portal/posts/2025/cookoff2025-website.md similarity index 100% rename from portal/posts/cookoff2025-website.md rename to portal/posts/2025/cookoff2025-website.md diff --git a/portal/posts/mystification.md b/portal/posts/2025/mystification.md similarity index 100% rename from portal/posts/mystification.md rename to portal/posts/2025/mystification.md diff --git a/portal/posts/new-cookbooks.md b/portal/posts/2025/new-cookbooks.md similarity index 100% rename from portal/posts/new-cookbooks.md rename to portal/posts/2025/new-cookbooks.md From 06527ff011eb9352719e107ee37e8b62b06d7d63 Mon Sep 17 00:00:00 2001 From: Julia Kent <46687291+jukent@users.noreply.github.com> Date: Fri, 11 Jul 2025 10:12:20 -0600 Subject: [PATCH 11/15] brian edit and file paths on home --- portal/index.md | 4 ++-- portal/posts/2025/mystification.md | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/portal/index.md b/portal/index.md index 67e99adae..86125f653 100644 --- a/portal/index.md +++ b/portal/index.md @@ -19,10 +19,10 @@ Project Pythia is hosting a Cookbook Cook-Off August 5-8, 2025. {button}`Cook-Off 2025` -{button}`2024 Cookbooks<./posts/new-cookbooks.md>` +{button}`2024 Cookbooks<./posts/2025/new-cookbooks.md>` Donate to support Project Pythia! -{button}`Donate now` +{button}`Donate now` +++ { "kind": "justified"} diff --git a/portal/posts/2025/mystification.md b/portal/posts/2025/mystification.md index c4b2d221a..afe11a37c 100644 --- a/portal/posts/2025/mystification.md +++ b/portal/posts/2025/mystification.md @@ -58,7 +58,7 @@ Thanks to everyone who was involved in our sprint day (alphabetical): build: uses: ProjectPythia/cookbook-actions/.github/workflows/build-book.yaml@main with: - build_command: 'jupyter-book build .' + build_command: 'jupyter-book build .' ``` Reach out if this is causing problems! From cc1bca717037673e0b1b23bbf9368714587d0f4f Mon Sep 17 00:00:00 2001 From: Julia Kent <46687291+jukent@users.noreply.github.com> Date: Fri, 11 Jul 2025 10:13:20 -0600 Subject: [PATCH 12/15] bold --- portal/posts/2025/mystification.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/portal/posts/2025/mystification.md b/portal/posts/2025/mystification.md index afe11a37c..e842f4952 100644 --- a/portal/posts/2025/mystification.md +++ b/portal/posts/2025/mystification.md @@ -15,7 +15,7 @@ The new MyST architecture was very appealing for several reasons: Sustainability. Our current Sphinx-based architecture was becoming clunky and hard to maintain as members joined or left the project, and required too much boilerplate code in individual cookbook repos which presented a barrier to would-be new contributors. MyST offered a much more streamlined alternative to keep our community project growing. Staying on the leading edge of best practices. We are an open-source community resource that teaches open-source coding practices, so it’s important that our own sites continue to be useful models for the broader community. Making cookbooks better! A lot of the new functionality in MyST is really well suited to the cookbooks, including things like [cross-referencing](https://mystmd.org/guide/cross-references) and [embedding content](Embed & Include Content - MyST Markdown) and automated [bibliographies](https://mystmd.org/guide/citations). -Cross-pollination with the core developers! Having the MyST developers invested in our use-case as a demo as they learn, understand, and develop functionality that will be particularly useful to us (and users that come after) is a really nice feedback loop from both a community and technological stand point. +**Cross-pollination with the core developers!** Having the MyST developers invested in our use-case as a demo as they learn, understand, and develop functionality that will be particularly useful to us (and users that come after) is a really nice feedback loop from both a community and technological stand point. ## MyST for sustainability ### Our aging infrastructure From 1243b386c19a6e11d0821889a038927aa1ce8bba Mon Sep 17 00:00:00 2001 From: Julia Kent <46687291+jukent@users.noreply.github.com> Date: Fri, 11 Jul 2025 10:18:45 -0600 Subject: [PATCH 13/15] all bold --- portal/posts/2025/mystification.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/portal/posts/2025/mystification.md b/portal/posts/2025/mystification.md index e842f4952..84834ead7 100644 --- a/portal/posts/2025/mystification.md +++ b/portal/posts/2025/mystification.md @@ -12,9 +12,9 @@ Project Pythia recently transitioned from a [Sphinx-based JupyterBook](https://j We began the process of transitioning to MyST in the summer of 2024 at the annual [Project Pythia Cook-off hackathon](https://projectpythia.org/posts/new-cookbooks). At that event, members of the MyST team demonstrated the current [alpha-version of the technology](https://executablebooks.org/en/latest/blog/2024-06-14-project-pythia-mystmd/) and coached us through the boilerplate code necessary to make some of our key resources build with MyST. The new MyST architecture was very appealing for several reasons: -Sustainability. Our current Sphinx-based architecture was becoming clunky and hard to maintain as members joined or left the project, and required too much boilerplate code in individual cookbook repos which presented a barrier to would-be new contributors. MyST offered a much more streamlined alternative to keep our community project growing. -Staying on the leading edge of best practices. We are an open-source community resource that teaches open-source coding practices, so it’s important that our own sites continue to be useful models for the broader community. -Making cookbooks better! A lot of the new functionality in MyST is really well suited to the cookbooks, including things like [cross-referencing](https://mystmd.org/guide/cross-references) and [embedding content](Embed & Include Content - MyST Markdown) and automated [bibliographies](https://mystmd.org/guide/citations). +**Sustainability**: Our current Sphinx-based architecture was becoming clunky and hard to maintain as members joined or left the project, and required too much boilerplate code in individual cookbook repos which presented a barrier to would-be new contributors. MyST offered a much more streamlined alternative to keep our community project growing. +**Staying on the leading edge of best practices**: We are an open-source community resource that teaches open-source coding practices, so it’s important that our own sites continue to be useful models for the broader community. +**Making cookbooks better!** A lot of the new functionality in MyST is really well suited to the cookbooks, including things like [cross-referencing](https://mystmd.org/guide/cross-references) and [embedding content](Embed & Include Content - MyST Markdown) and automated [bibliographies](https://mystmd.org/guide/citations). **Cross-pollination with the core developers!** Having the MyST developers invested in our use-case as a demo as they learn, understand, and develop functionality that will be particularly useful to us (and users that come after) is a really nice feedback loop from both a community and technological stand point. ## MyST for sustainability From 910847ecac8a93b5cc4f5977c55f546729b6b0cd Mon Sep 17 00:00:00 2001 From: Julia Kent <46687291+jukent@users.noreply.github.com> Date: Fri, 11 Jul 2025 10:23:06 -0600 Subject: [PATCH 14/15] preserve bullets --- portal/posts/2025/mystification.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/portal/posts/2025/mystification.md b/portal/posts/2025/mystification.md index 84834ead7..d0f1b7d69 100644 --- a/portal/posts/2025/mystification.md +++ b/portal/posts/2025/mystification.md @@ -12,10 +12,10 @@ Project Pythia recently transitioned from a [Sphinx-based JupyterBook](https://j We began the process of transitioning to MyST in the summer of 2024 at the annual [Project Pythia Cook-off hackathon](https://projectpythia.org/posts/new-cookbooks). At that event, members of the MyST team demonstrated the current [alpha-version of the technology](https://executablebooks.org/en/latest/blog/2024-06-14-project-pythia-mystmd/) and coached us through the boilerplate code necessary to make some of our key resources build with MyST. The new MyST architecture was very appealing for several reasons: -**Sustainability**: Our current Sphinx-based architecture was becoming clunky and hard to maintain as members joined or left the project, and required too much boilerplate code in individual cookbook repos which presented a barrier to would-be new contributors. MyST offered a much more streamlined alternative to keep our community project growing. -**Staying on the leading edge of best practices**: We are an open-source community resource that teaches open-source coding practices, so it’s important that our own sites continue to be useful models for the broader community. -**Making cookbooks better!** A lot of the new functionality in MyST is really well suited to the cookbooks, including things like [cross-referencing](https://mystmd.org/guide/cross-references) and [embedding content](Embed & Include Content - MyST Markdown) and automated [bibliographies](https://mystmd.org/guide/citations). -**Cross-pollination with the core developers!** Having the MyST developers invested in our use-case as a demo as they learn, understand, and develop functionality that will be particularly useful to us (and users that come after) is a really nice feedback loop from both a community and technological stand point. +- **Sustainability**: Our current Sphinx-based architecture was becoming clunky and hard to maintain as members joined or left the project, and required too much boilerplate code in individual cookbook repos which presented a barrier to would-be new contributors. MyST offered a much more streamlined alternative to keep our community project growing. +- **Staying on the leading edge of best practices**: We are an open-source community resource that teaches open-source coding practices, so it’s important that our own sites continue to be useful models for the broader community. +- **Making cookbooks better!** A lot of the new functionality in MyST is really well suited to the cookbooks, including things like [cross-referencing](https://mystmd.org/guide/cross-references) and [embedding content](Embed & Include Content - MyST Markdown) and automated [bibliographies](https://mystmd.org/guide/citations). +- **Cross-pollination with the core developers!** Having the MyST developers invested in our use-case as a demo as they learn, understand, and develop functionality that will be particularly useful to us (and users that come after) is a really nice feedback loop from both a community and technological stand point. ## MyST for sustainability ### Our aging infrastructure From 2fdb1d36195d893712ba8a5055e99dbc639c2aa1 Mon Sep 17 00:00:00 2001 From: Julia Kent <46687291+jukent@users.noreply.github.com> Date: Fri, 11 Jul 2025 10:24:31 -0600 Subject: [PATCH 15/15] oops missing a file path --- portal/posts/2025/mystification.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/portal/posts/2025/mystification.md b/portal/posts/2025/mystification.md index d0f1b7d69..f260a7818 100644 --- a/portal/posts/2025/mystification.md +++ b/portal/posts/2025/mystification.md @@ -9,7 +9,7 @@ tags: [myst] Project Pythia recently transitioned from a [Sphinx-based JupyterBook](https://jupyterbook.org/en/stable/intro.html) architecture to using [MyST Markdown](https://mystmd.org/), which serves as the foundation for the upcoming [JupyterBook 2](https://next.jupyterbook.org/)! ## Our Motivation for MyST -We began the process of transitioning to MyST in the summer of 2024 at the annual [Project Pythia Cook-off hackathon](https://projectpythia.org/posts/new-cookbooks). At that event, members of the MyST team demonstrated the current [alpha-version of the technology](https://executablebooks.org/en/latest/blog/2024-06-14-project-pythia-mystmd/) and coached us through the boilerplate code necessary to make some of our key resources build with MyST. +We began the process of transitioning to MyST in the summer of 2024 at the annual [Project Pythia Cook-off hackathon](https://projectpythia.org/posts/2025/new-cookbooks). At that event, members of the MyST team demonstrated the current [alpha-version of the technology](https://executablebooks.org/en/latest/blog/2024-06-14-project-pythia-mystmd/) and coached us through the boilerplate code necessary to make some of our key resources build with MyST. The new MyST architecture was very appealing for several reasons: - **Sustainability**: Our current Sphinx-based architecture was becoming clunky and hard to maintain as members joined or left the project, and required too much boilerplate code in individual cookbook repos which presented a barrier to would-be new contributors. MyST offered a much more streamlined alternative to keep our community project growing.