Merge branch 'master' into nhussein11/follow-up-moonbeam-xcm-sdk-contribution

Author: nhussein11

.CONTRIBUTING.md

@@ -30,9 +30,9 @@ You may want to spin up a local version of the documentation site to preview you
 
 Building and serving the site requires cloning two repositories:
 
-- **[Polkadot MkDocs](https://github.com/papermoonio/polkadot-mkdocs)** - contains the MkDocs configuration files, theme overrides, and custom CSS for the Polkadot documentation site
+- **[Polkadot MkDocs](https://github.com/papermoonio/polkadot-mkdocs)**: Contains the MkDocs configuration files, theme overrides, and custom CSS for the Polkadot documentation site.
 
-- **[Polkadot Docs](https://github.com/polkadot-developers/polkadot-docs)** - the actual content is stored in the `polkadot-docs` repository and pulled into the `polkadot-mkdocs` directory during build
+- **[Polkadot Docs](https://github.com/polkadot-developers/polkadot-docs)**: The actual content is stored in the `polkadot-docs` repository and pulled into the `polkadot-mkdocs` directory during build.
 
 For everything to work correctly, the file structure needs to be as follows:
 
@@ -90,16 +90,16 @@ When adding new pages to the documentation site, it is important to ensure all r
 
 To create a new section of pages, you will first need to create a subdirectory with the desired name of the section. The root directory, and every subdirectory, must contain the following files:
 
-- `.pages` - defines the structure of the documentation site
-- `index.md` - a landing page which can be used to provide more context about the content in the section
+- **`.pages`**: Defines the structure of the documentation site.
+- **`index.md`**: A landing page which can be used to provide more context about the content in the section.
 
 #### Example `.pages` File
 
 ```markdown
 title: Application Developers
 nav: 
   - index.md
-  - 'Polkadot SDK': 'polkadot-sdk.md'
+  - **'Polkadot SDK'**: 'polkadot-sdk.md'
   - interact
   - tooling
 ```
@@ -108,10 +108,10 @@ This example would result in adding 'Application Developers' as a section with t
 
 Some important things to note:
 
-- The `title` field at the top of the page represents the display name for the section. This is displayed on the left-side navigation bar when viewing the site
-- The `index.md` page should always be the first item in the `nav` list
-- Files follow the convention of `'Display Name': 'file-name.md'`
-- Sections are listed by their directory name in the source code. For example, the Tooling section will be added to the navigation simply by using the directory name: `tooling`
+- The `title` field at the top of the page represents the display name for the section. This is displayed on the left-side navigation bar when viewing the site.
+- The `index.md` page should always be the first item in the `nav` list.
+- Files follow the convention of `'Display Name': 'file-name.md'`.
+- Sections are listed by their directory name in the source code. For example, the Tooling section will be added to the navigation simply by using the directory name: `tooling`.
 
 #### Example `index.md` File
 
@@ -126,22 +126,22 @@ description: Learn how to build and customize blockchains with Polkadot SDK. Exp
 
 Some important things to note:
 
-- The `title` represents the `<title>` tag and is used for SEO purposes
-- The `description` represents the meta-description and is also used for SEO purposes
+- The `title` represents the `<title>` tag and is used for SEO purposes.
+- The `description` represents the meta-description and is also used for SEO purposes.
 
 ### Adding Pages to Existing Section
 
 If you are adding pages to an existing section, the steps are simplified. However, it's essential to ensure you complete these steps to display the new page and its title on the documentation site correctly:
 
 - Add the new markdown page to the appropriate section. Note that the filename becomes part of the URL for this page. See the [style guide](https://github.com/papermoonio/documentation-style-guide/blob/main/style-guide.md#naming-conventions) for additional guidance on naming conventions.
 - Ensure the new content page includes the following:
-    - **`title`** - represents the `<title>` tag and is used for SEO purposes (not displayed on the published site) Titles have a maximum length of 45 characters.
-    - **`description`** - represents the meta-description and is also used for SEO purposes (not displayed on the published site). Descriptions should be 120-160 characters and should provide a preview into the page topic.
-    - **Page title** - an H1 heading title to be displayed at the top of the page
+    - **`title`**: Represents the `<title>` tag and is used for SEO purposes (not displayed on the published site) Titles have a maximum length of 45 characters.
+    - **`description`**: Represents the meta-description and is also used for SEO purposes (not displayed on the published site). Descriptions should be 120-160 characters and should provide a preview into the page topic.
+    - **Page title**: An H1 heading title to be displayed at the top of the page.
 
-    - **`## Checking Prerequisites` section** - if the guide requires the user to have specific developer tools installed, for example, Docker or MetaMask, it should be listed here
+    - **`## Checking Prerequisites` section**: If the guide requires the user to have specific developer tools installed, for example, Docker or MetaMask, it should be listed here.
 
-- Add the 'Display Name': 'file-name.md' for the new page to the `.pages` folder in the same subdirectory where you added the new page
+- **Add the 'Display Name'**: 'file-name.md' for the new page to the `.pages` folder in the same subdirectory where you added the new page.
 
 An example new content page might look like:
 
@@ -172,10 +172,10 @@ More resources for [SEO Optimization](#search-engine-optimization-seo) of titles
     git checkout -b INSERT_NEW_BRANCH_NAME
     ```
 
-2. Modify content as desired. Remember to place images and code snippets in the appropriate folders (see the following sections for details)
-3. Review the [style guide](https://github.com/papermoonio/documentation-style-guide/blob/main/style-guide.md) to ensure your new content meets the guidelines
-3. Once you commit and push all of your changes, open a pull request for the new content branch against the `main` branch
-4. Monitor notifications and pull requests for feedback from code owners. At least one approval is required before merging content
+2. Modify content as desired. Remember to place images and code snippets in the appropriate folders (see the following sections for details).
+3. Review the [style guide](https://github.com/papermoonio/documentation-style-guide/blob/main/style-guide.md) to ensure your new content meets the guidelines.
+3. Once you commit and push all of your changes, open a pull request for the new content branch against the `main` branch.
+4. Monitor notifications and pull requests for feedback from code owners. At least one approval is required before merging content.
 
 If your additions or modifications are limited to content on an existing page, there is no need to worry about the [`.pages`](#example-pages-file) or [`index.md`](#example-indexmd-file) files, as changes to page content don't affect these files.
 
@@ -222,11 +222,11 @@ In general, titles should be between 50 and 60 characters and descriptions shoul
 
 There are a few tools you may find useful for proofreading and editing your contributions:
 
-- **[Vale](https://vale.sh/)** - the `polkadot-mkdocs` repository contains configuration for Vale, an open source NLP-powered linter for text. The configuration is a work in progress with a growing custom dictionary tailored toward software engineering, blockchain, and Polkadot terms. Running Vale against your files locally can serve as a first round of feedback to speed up the review process
+- **[Vale](https://vale.sh/)**: The `polkadot-mkdocs` repository contains configuration for Vale, an open source NLP-powered linter for text. The configuration is a work in progress with a growing custom dictionary tailored toward software engineering, blockchain, and Polkadot terms. Running Vale against your files locally can serve as a first round of feedback to speed up the review process.
 
 To use Vale locally to screen your work:
 
-1. Visit the Vale site and follow the [installation instructions](https://vale.sh/docs/vale-cli/installation/) 
+1. Visit the Vale site and follow the [installation instructions](https://vale.sh/docs/vale-cli/installation/).
 2. From the `polkadot-mkdocs` directory, run the following in your terminal:
 
     ```bash
@@ -237,8 +237,8 @@ To use Vale locally to screen your work:
 
     ![Vale sample terminal output](images/contributing/vale-output-01.webp)
 
-3. You can use CMD+click to open the file with the flagged items. This is especially helpful if you run Vale against a directory with multiple files
+3. You can use CMD+click to open the file with the flagged items. This is especially helpful if you run Vale against a directory with multiple files.
 
-4. Each flag tells you the line and location of the flagged item, the level of the flag (error, warning, or suggestion), and a suggestion for how to resolve the flag
+4. Each flag tells you the line and location of the flagged item, the level of the flag (error, warning, or suggestion), and a suggestion for how to resolve the flag.
 
-5. Once you have addressed the flagged items and made edits as needed, you can complete the normal steps to commit your changes and open a pull request to review for merge
+5. Once you have addressed the flagged items and made edits as needed, you can complete the normal steps to commit your changes and open a pull request to review for merge.

.github/scripts/check_dependencies.py

@@ -57,6 +57,29 @@ def get_latest_pypi_version(package_name):
         print(f"Error fetching PyPI package version for {package_name}: {e}")
     return None, None
 
+def check_sub_dependencies(name, info, outdated_dependencies):
+    """Check sub-dependencies within a repository entry"""
+    # Check if there's a subdependencies section
+    subdependencies = info.get("subdependencies", {})
+    
+    if not subdependencies:
+        return
+    
+    for key, current_version in subdependencies.items():
+        if key.endswith("_version"):
+            # Extract crate name from the key (remove _version suffix)
+            crate_name = key.replace("_version", "").replace("_", "-")
+            latest_version, latest_url = get_latest_crate_version(crate_name)
+            
+            if latest_version and latest_version != current_version:
+                outdated_dependencies.append({
+                    "name": f"{name}.{crate_name}",
+                    "category": "sub-dependency",
+                    "current_version": current_version,
+                    "latest_version": latest_version,
+                    "latest_release_url": latest_url,
+                })
+
 def check_releases(releases_source_file):
     try:
         with open(releases_source_file, "r") as file:
@@ -88,6 +111,10 @@ def check_releases(releases_source_file):
                     "latest_version": latest_version,
                     "latest_release_url": latest_url,
                 })
+            
+            # Check sub-dependencies for repositories
+            if category == "repositories":
+                check_sub_dependencies(name, info, outdated_dependencies)
 
     return outdated_dependencies
 

.github/workflows/check-llms.yml

@@ -1,67 +1,13 @@
-name: Validate llms.txt
+name: Validate llms-full.txt
 
 on:
   pull_request:
-    branches:
-      - master
+    branches: [master]
 
 jobs:
-  check-llms:
-    runs-on: ubuntu-latest
-
-    steps:
-      - name: Checkout code
-        uses: actions/checkout@v3
-
-      - name: Set up Python
-        uses: actions/setup-python@v4
-        with:
-          python-version: '3.x'
-
-      - name: Install dependencies
-        run: |
-          python -m pip install --upgrade pip
-          pip install pyyaml requests
-
-      - name: Get SHA256 hash of the PR's llms.txt file
-        id: pr_llms_hash
-        run: |
-          if [ -f "llms.txt" ]; then
-            # Calculate SHA256 hash of the llms.txt file in the PR branch (checked out by default)
-            sha256sum llms.txt | awk '{ print $1 }' > pr_sha256.txt
-          else
-            echo "0000" > pr_sha256.txt
-          fi
-          cat pr_sha256.txt
-
-      - name: Generate llms.txt using the Python script
-        run: python3 scripts/generate_llms.py
-
-      - name: Calculate SHA256 hash of the generated llms.txt
-        id: generated_llms_hash
-        run: |
-          # Full path to the generated llms.txt file
-          llms_file="/home/runner/work/polkadot-docs/polkadot-docs/llms.txt"
-
-          # Check if the file exists before calculating the hash
-          if [ -f "$llms_file" ]; then
-            sha256sum "$llms_file" | awk '{ print $1 }' > generated_sha256.txt
-          else
-            echo "Error: llms.txt not found at $llms_file"
-            exit 1
-          fi
-          cat generated_sha256.txt
-
-      - name: Compare the SHA256 hashes
-        run: |
-          generated_hash=$(cat generated_sha256.txt)
-          pr_hash=$(cat pr_sha256.txt)
-
-          if [ "$generated_hash" != "$pr_hash" ]; then
-            echo "Error: SHA256 hashes do not match. The generated llms.txt file differs from the one in the PR."
-            echo "You need to run the generate LLMS script:"
-            echo "python3 scripts/generate_llms.py"
-            exit 1
-          else
-            echo "SHA256 hashes match. The llms.txt file is consistent."
-          fi
+  validate:
+    uses: papermoonio/workflows/.github/workflows/validate-llms.yml@main
+    with:
+      llms_filename: "llms-full.txt"
+      generator_command: "python3 scripts/generate_llms.py"
+      require_pr_file: false

.gitignore

@@ -1,4 +1,5 @@
 node_modules
 .vscode
 target
-.DS_Store
+.DS_Store
+scripts/__pycache__/*