GSOC Project: Add Translation System to DISCOVER Cookbook

[aterrel]

I'm happy to announce that @speco29 has been accepted as a student for GSOC. The proposal can be at #209.

We can keep this thread open for @speco29 to post on the progress of the project.

[speco29]

Thank you @aterrel for this oppurtunity.
looking forward to a productive summer:)

[speco29]

Hello @aterrel!
I think most of the issues of the Cookbook are resolved and almost every issue have an attached PR to it.
I would like to start working on the project now, is there anything in specific that I should start working on.
Also, for the translations, how many languages should be there. I was thinking of adding Chinese, Japanese for now as our top priority.

@OriolAbril is there anything in specific that I should start with regarding translations?
As I mentioned before for adding switches via HTM, CSS, JS but we want something custom made, could you help me out a bit to do some research regarding these also?

[OriolAbril]

is there anything in specific that I should start with regarding translations?

In addition to participating in the coordination discussion, I would recommend starting by building the cookbook in a different language as a standalone website. That is:

1. get the translatable text from the cookbook content in a suitable format, gettext (pot/po) files are probably the most common
2. translate part of a couple pages out of that, this is only to make sure the translated parts are pulled in correctly, any plain text editor should do
3. Build the translated version of the website with jupyterbook/sphinx
4. Publish it as a standalone website. The github pages of your personal fork of this repo should be enough for this.

In my opinion, these 4 steps are necessary independently of whatever other choices we do elsewhere. i.e. how and when to get the translatable content is susceptible to be automated (partially or fully) to aim for constant updating or to happen only when a release/tag is made... but it will need to happen either way, same for the other steps.

As I mentioned before for adding switches via HTM, CSS, JS but we want something custom made, could you help me out a bit to do some research regarding these also?

In general, I think the goal should be to aim for minimal (or ideally 0) disruption of the existing content and also keeping the amount of new/custom code low, as it will have to be maintained later on. I understand it might be necessary to implement custom html templates with some css styling and js code for the switchers for example, but I wouldn't feel comfortable doing that without the maintainers of the repo agreeing to that knowing they'll have to maintain it, otherwise it will probably stop working in a couple years but it might never be fixed. This is also a part with significant overlap with the versioning project, so I would wait until we reach a decision on #317 before looking too much into it. We are still in the community bonding period after all. It won't be feasible to attempt independent switchers that are not aware of each other but we'll also want a common style for the switchers so as much as possible their html/css/js should be shared if it needs to be added here from scratch. I am also not sure I'll be of much help here, I can defend myself with html and css but I have never used javascript

[speco29]

In my opinion, these 4 steps are necessary independently of whatever other choices we do elsewhere. i.e. how and when to get the translatable content is susceptible to be automated (partially or fully) to aim for constant updating or to happen only when a release/tag is made... but it will need to happen either way, same for the other steps.

Ok @OriolAbril
I will look into it and will update soon.

[speco29]

🗓️ Weekly Update – Community Bonding Week 1 & 2

This week, I focused on understanding different translation platforms, comparing workflows in Crowdin and Transifex to evaluate their approaches. I also had a discussion with my mentor about how translations are preserved across documentation projects, examining strategies for maintaining consistency and avoiding redundant work. Additionally, I explored the CPython documentation, analyzing how their language switcher functions and how files are organized within their repository.

Summary of Activities

• Explored translation services like Crowdin and Transifex, analyzing their file structures and automation workflows.
• Had a mentor discussion on best practices for maintaining translations and ensuring consistency across versions.
• Examined CPython documentation, studying how their language switcher is implemented.

Next Steps
📝 Drafting a guide for translators and language managers, outlining best practices for handling multilingual content.

[speco29]

Greetings @aterrel @OriolAbril !

Localization in Jupyter Book – Using Sphinx’s gettext

I've investigated how Jupyter Book handles localization and found that while it is built on Sphinx, it does not expose gettext functionality directly. To enable gettext, we need to convert _config.yml to conf.py so that Sphinx can properly handle translations.

Steps Completed:

• Ran sphinx-quickstart, which generated the necessary Sphinx project structure.

• Translated key settings from _config.yml to conf.py to align with Sphinx’s requirements.
  
  

• Successfully extracted translatable content, and the index.pot file is now generated in build/gettext/.
  

[OriolAbril]

I have used sphinx directly way more than I have used the jupyterbook command line so that will probably make things easier for me too going forward. If you want me to take a look you can share the link to the branch in your PR this this changes or open a draft PR and tag me. We should make the switch from yaml to python config without any change to other files (maybe the requirement ones).

Translated key settings from _config.yml to conf.py to align with Sphinx’s requirements.

Did you do it manually or throug the jupyterbook command line?

[speco29]

i did it manually

[speco29]

If you want me to take a look you can share the link to the branch

https://github.com/speco29/DISCOVER-Cookbook/tree/translation-docs
Will this link work?

[speco29]

I think i have done something wrong.
here, in the contribute box, at first it was showing 'Discard 5 commits' and 'Open pull request'. I choose the discard 5 commits option because the branch I pushed had commits which were not relevant to the translation branch, I thought clicking on discard option will remove those irrelevant commits but its showing something like this.
Please help me out here

[OriolAbril]

If you still have the translation-docs branch with the changes locally you should "force push" it. From the command line this would look like:

git status
# check the output says "on translation-docs branch"
git push --force

You have to avoid "pulling", "merging" or "rebasing" as these will take the changes on GitHub and bring them to your local computer which is the opposite of what you want.

If force pushing were not an option, you should be able to create a new branch from translation-docs and push that one.

---

For some more context, these diagrams from the turing way can help: https://book.the-turing-way.org/reproducible-research/vcs/vcs-workflow-branches.

"Feature B" is akin to your translation-docs branch. You created it from main and added commits to it, and while you did that, other people also created their own branches, opened PRs and new commits where added to main. In the diagram, "Feature B" is 2 commits ahead of main and 6 commits behind. As long as the commits in main don't change the same things you are changing this is not an issue, but it is good practice to update your branch from time to time with the changes in main. We can go over git/github workflow on the call to make sure we are all on the same page.

GitHub has some checks to make sure your branch is up to date and has a PR open if there are changes there not in main. And it looks like one of the options to get your branch up to date with main was to remove the orange commits so your branch is exactly main

[speco29]

I have opened a draft PR and has tagged you in it.
let me know if its okay.

[speco29]

Greetings @aterrel , @OriolAbril, @kamila-NF, @emmasaroyan !

Weekly Update - Localization Setup Progress

This week, I focused on refining the localization setup for the website, ensuring a smooth transition to Sphinx-based workflows. Key tasks included generating translation files and streamlining configurations.

Summary of Activities

Set up Sphinx for the website, transitioning from the previous setup.

• Generated the locale folder, containing .po files with Spanish messages.
• Created config.py, necessary for the Sphinx setup, and removed _config.yml, which was no longer needed.
• Updated all files from the Jupyter Book build setup to Sphinx for better workflow alignment.
• Deleted unnecessary files to streamline project efficiency.
• Created Translation.md, a guide detailing the steps to generate .po files for translation.
• All the work done can be reviewed by examining this Draft PR: translable files #326

Next Steps

📝 Begin translating the website into Spanish using Transifex.

---

[OriolAbril]

Thanks for the report! I can start some bare bones translation to Spanish so you have some content that changes when testing out building the translated website. But I can do that directly on the .po files.

I think we should try out the different alternatives before deciding to stick with one of them. I think at the very least we should try out local translations (like the python-docs-es do), transifex and crowdin. Collaborative cloud services might be friendlier for translators who are not familiar with git/github, but will probably make automation more complex (we'd need to push/pull translations from there every now and then) as well as difficult keeping the same attribution process. @aterrel and @kamila-NF probably have a better sense of the governance of this repo, but we might need to generate a report on pros/cons of each for the editorial/maintenance team to choose which alternative better fits this repository.

[speco29]

Should we divide like which portion is getting translated by whom, as myself don't know spanish so I'll be taking help from different AI models for translations.

[OriolAbril]

I think for now we only need to have minimal translations so we can 1) test the translation workflow and 2) test the generated previews (which should work and show differences with the English version once a handful of messages have been translated). No need to have a full translation of the website ready (either now or as an output of the project, the goal is the translation infrastructure).

Re AI, I personally prefer translating myself to reviewing AI translations[1] but we can evaluate them as a first step to help translators/reviewers at some point.

1: I have never found an AI translation of minimal complexity to be usable as-is, it needs fixing, but as you start from a "proposal" you end up focusing on fixing the proposal rather than generating the best translation out of the initial prompt, so the output often sounds quite weird even after reviewing.

[speco29]

Hello @OriolAbril,
Here's the link of some translations that I'm doing for the website https://app.transifex.com/numfocus/discover-cookbook/translate/#es/01_aboutpo/580044808, is there a way to reach out to a community who does Translations because in the very beginning of the project @aterrel suggested that if we just pipe in the AI translations that will not add much value to the project, rather we should call translators to do the translations. Either way let me know the proceedings.

[OriolAbril]

I can't access the link (independently of being logged in or not) nor I can access https://explore.transifex.com/numfocus. The 2nd is a bit more worrying as I can access https://explore.transifex.com/arviz/, https://explore.transifex.com/sphinx-doc/ or https://explore.transifex.com/pymc/ when not logged in and from private navigation windows. During this week's call we should double check the settings to make sure you are on the OSS plan, send invites to all mentors and transfer ownership to @kamila-NF given the organization is named numfocus.

[speco29]

I can't access the link (independently of being logged in or not) nor I can access https://explore.transifex.com/numfocus.

I think it happened because I choose the organization's setting as private. I have changed it now. I think it will work now: https://app.transifex.com/numfocus/

[speco29]

During this week's call we should double check the settings to make sure you are on the OSS plan, send invites to all mentors and transfer ownership to @kamila-NF given the organization is named numfocus.

I have figured out on how to invite collaborators, I will assign @kamila-NF as organization administrator and @aterrel as Project maintainer, but there are more roles such as: Team manager, Language coordinator, Translator and Reviewer. I am a bit confused on whom to assign what.

[speco29]

Hello @OriolAbril,
Does this page look good?

[speco29]

It looks good yes. Once we have it set up we can iterate on it to improve it. I think the main thing we'll have to be careful about is we are using subfolders to host the different languages, but everything is still a single website, which means a single 404 page.

The 404 page can be translated into different languages too so when a person is using website in (for example) Spanish, and the link breaks, 404 page lands in Spanish Translation.

[speco29]

It looks good yes.

Should I add this 404 html page with other html pages?

[OriolAbril]

Open a PR yes.

The 404 page can be translated into different languages too so when a person is using website in (for example) Spanish, and the link breaks, 404 page lands in Spanish Translation.

While that would be ideal, I don't think it is possible in our case where different languages are different subfolders on a hosted website instead of hosting each language as their own website. We should definitely test it though, we can discuss in the weekly meeting how to host a clone of the website through the github pages of you fork so we can play around without affecting the cookbook itself.

[speco29]

While that would be ideal, I don't think it is possible in our case where different languages are different subfolders on a hosted website instead of hosting each language as their own website. We should definitely test it though, we can discuss in the weekly meeting how to host a clone of the website through the github pages of you fork so we can play around without affecting the cookbook itself.

Definitely!

[speco29]

Open a PR yes.

In order to add the 404.html, first its markdown file is needed to be added which is in this PR: #332
I guess in tomorrow's meet, we will have to discuss about the merging of important PRs too.

[speco29]

@OriolAbril
Added the proposal for language switcher: #334

[OriolAbril]

Thanks!

[speco29]

Hello @OriolAbril, @aterrel, @kamila-NF and @emmasaroyan!

404 Page Behavior Testing Summary
After testing GitHub Pages fallback behavior across localized directories (/en, /es), I’ve confirmed that placing a 404.html file inside individual folders does not trigger localized fallback pages (e.g., /en/404.html for /en/unknown.html). Instead, GitHub Pages only supports a global 404.html placed at the root of the gh-pages branch. Any unmatched route, regardless of folder, redirects to this single root-level 404 page.

[OriolAbril]

We should design a multi-language 404 page then I think. And as part of the "add a new language" instructions explain how to update it. When you open the PR to correctly add the 404 page tag me and I will comment the spanish translation of the content in there

[speco29]

@OriolAbril
I need a little bit of guidance, I created the markdown file 404.md then by using jupyter-book build DISCOVER/, the 404.html file was generated based on that html file, I pushed the changes but when it tried to test it, it did not worked.
I also tried creating a 404.html file manually in root directory but the test again failed, the 404 page is not getting deployed.
I don't know if its because I am not using gh-pages branch or the url is wrong😓
Also as the 404.html page is based on 404.md page, the page looks like this now:
[

[OriolAbril]

I pushed the changes but when it tried to test it

where did you push to?

[speco29]

I pushed the changes but when it tried to test it

where did you push to?

I pushed the changes but when it tried to test it

where did you push to?

I FINALLYYY deployed the page
Researched a bit but the page is finally deployed 😭
This is what it looks like:

[speco29]

Discussion on translation workflows and different alternatives (transifex, crowdin and poedit). Discussing pros and cons of each translation service

1. Transifex

Best for: Scalable software projects and continuous localization

Pros:

• Supports continuous integration and automation (great for CI/CD pipelines)
• Collaborative UI with roles and review workflows

Cons:

• UI can feel a bit dense for casual contributors
• Pricing is tiered and might be steep for smaller open-source projects without sponsorship

Special strength: Works well for projects managing multiple branches or string contexts — excellent for multilingual web apps

2. Crowdin

Best for: Interactive, contributor-friendly localization with integrations

Pros:

• Friendly UI for translators, great for engaging communities
• Deep integrations: GitHub, GitLab, Bitbucket, Figma, etc.
• Auto-sync between repo and Crowdin — changes can be live in minutes
• Supports screenshots to provide context during translation

Cons:

• Free plan has limited collaborators and features
• UI has a learning curve for managing branches in large projects

Special strength: Community-sourced translations with modern UX — good if we're inviting non-technical contributors.

3. Poedit

Best for: Lightweight, desktop-based gettext (.po/.mo) translation

Pros:

• Offline use — fast, focused interface for translators
• Easy to install and doesn’t require a server or cloud account
• Supports translation memory for reuse

Cons:

• No built-in collaboration — you'd rely on external tools for syncing and reviewing
• Doesn’t support non-gettext formats (e.g. JSON or XLIFF) well

Special strength: Quick edits and reviews for smaller teams or solo workflows — helpful if you're just testing i18n before scaling.

Choosing Between Them

• If we're focused on automation, Transifex fits well with your CI/CD setup.
• If we’re building community involvement around localization, Crowdin offers a friendlier experience.
• If we're handling one-off edits or legacy .po workflows, Poedit remains fast and effective.

@aterrel @OriolAbril @kamila-NF @emmasaroyan
We can decide which of the translation service we would want to use through out the project, as per everyone's comfort and familarity with the services. I have tried them all and each of them stands similar to each other, but ofcourse some of the provided features are different.

[OriolAbril]

@speco29 can you make that a new discussion so it is easier for other contributors to participate? I think to help keep things on topic it will help if you add an overview of what is the goal of the discussion, then have one post/answer for each alternative so it is possible to comment/ask for extra details on specific alternatives in a more organized way

[speco29]

Sure

[speco29]

Weekly Update

This week, I focused on starting discussions, issues and developing the structure of Cookbook.

Summary of Activities

• Raised an issue to describe the structure of the language switcher that we would want to implement in our website: Issue.
• Researched and started a discussion on what translation service would be better to move forward with for our website specifying each service's pros and cons: DIscussion.
• Structured 404 page for the website:
  -- Custom 404 page.
  -- 404 page build from markdown file: Sphinx build 404 page.

---

Next step: Update the Guidance material for translations and prepare proper documentation

[speco29]

Greetings @aterrel , @OriolAbril, @kamila-NF, @emmasaroyan !

Weekly Update

This week, I focused on deeply learning the Translaton services and changing the format of 404 page.

Summary of Activities

• Upgraded the discussion that was opened to discuss and choose the translation service for our project.
• Dived deeper into the plans that each services provide for free for open source projects.
• Tested and finished the 404 page.

---

[speco29]

Greetings @aterrel, @OriolAbril, @kamila-NF, @emmasaroyan!

Weekly Update

This week, I focused on configuring the Transifex CLI integration with the DISCOVER-Cookbook repository, aligning translation workflows with the 2.0-translations branch, and troubleshooting .tx/config compatibility issues.

---

Summary of Activities

• Refactored and tested .tx/config blocks for all English .po files to support source uploads to Transifex.
• Synced my fork with upstream/2.0-translations and reset my working branch to match that baseline.
• Migrated source .po files from translation-docs into the updated branch and pushed them to GitHub.
• Debugged persistent tx push -s errors caused by Transifex CLI though a new bug got discovered in between and I am still trying to figure it out
• Cleaned .tx/config of formatting and parsing issues using script-based diagnostics.
• Finalized the translation service for Cookbook i.e., Transifex.

---

Next Steps

• Begin uploading English .po files to Transifex and add Spanish (es) as a target language.
• Update or create new Translation file with contributor-friendly instructions for pushing/pulling translations and CLI setup.

[OriolAbril]

thanks for the update

[speco29]

@aterrel @OriolAbril @kamila-NF @emmasaroyan

Summary of the project

This document outlines the work completed so far on the DISCOVER Cookbook project, focusing on localization and translation infrastructure.

---

✅ Localization Workflow Setup

• Extracted Translatable Strings

  • Used sphinx-build -b gettext to generate .po files from the documentation.
  • Output stored in:

    DISCOVER/_build/gettext
    

• Generated and Updated .po Files

  • Used sphinx-intl to update .po files for multiple languages:

    sphinx-intl update -p DISCOVER/_build/gettext -l <language-code>

• Switched from _config.yml to config.py

• Structured Locales Directory

  • Organized translation files under:

    locales/
      └── <lang>/
          └── LC_MESSAGES/
              └── <section>.po
    

---

Designed the look for language switcher and how it should work with the translation

Transifex Integration

• Installed Go-Based Transifex CLI (v1.6.17)

  • Configured .transifexrc with API token authentication.

• Created .tx/config

  • Defined mappings for each .po file as a separate resource.
  • Used consistent source_file and file_filter paths.

• Verified Push Functionality

  • Pushed source files:

    tx push -s

  • Pushed translations:

    tx push -t

Yet to verify:

• Pulled all translations:

  tx pull -a

---

Documentation Progress

• Translation Workflow Guide (TRANSLATION.md)

  • Covers:
    • .po updates
    • Compilation and previewing
    • Transifex CLI setup and usage

• Next Step

  • Finalize a dedicated document for translators:
    • Best practices
    • Glossary
    • Contribution tips

---

If there is anything else needed to be add to do for GSoC project specifically, it can be shared here:)
From my side, these things should cover the project