How do I create developer-only extensions for shared components?

[coatless]

Description

I have a series of components that I want to use across multiple extensions. However, these components will not be exposed to user. Instead, they'll be used within the extension. The goal of versioning the components in this way is to allow for targeted upgrades.

To that end, I wrote an experiment a few months back called {quarto-dependson} that nests a dependency underneath an existing extension. Before I go further, I'm interested in learning:

1. Is this the best way to nest a filter extension of utility filters?
2. Should a developer extension filter be registered under the user-facing Lua filter?
3. Are there any simplifications for retrieving the embedded extensions path?

Planned nesting structure

The dependson project is structured in the following manner:

.
├── example.qmd
├── _extensions
│   └── dependson
│       ├── dependson.lua
│       ├── _extensions
│       │   └── coatless-quarto
│       │       └── thedependency
│       │           ├── _extension.yml
│       │           └── thedependency.lua
│       ├── _extension.yml
│       ├── next-level
│       │   └── nested.lua
│       └── same-directory.lua
└── README.md

Inside of the _extension.yml, we then have:

title: dependson
contributes:
  filters:
    - dependson.lua
    - same-directory.lua
    - next-level/nested.lua
    - _extensions/coatless-quarto/thedependency/thedependency.lua # <-- dependency

To access the Lua filter, I'm then using a relative Lua path load into where the extension is:

-- Load the a filter from a dependency extension
local dependency = require("_extensions.coatless-quarto.thedependency.thedependency")

Then, function exported in the filter's table can be accessed with:

-- Check to see if the table call to the check_function is present
quarto.log.output(dependency.check_function())

[mcanouil]

Have you considered using Git submodules using the same structure as extensions embedding works?
you could then use the Dependabot GitHub Action to update those submodules.

It should be completely transparent for Quarto and users.

[coatless]

@mcanouil my modus operandi has been to avoid submodules whenever possible. But, in this case, it may make sense to pick up the dependabot benefits.

I suppose the best way to keep these dependencies up-to-date would be to use the following dependabot approach:

dependabot.yml

version: 2
updates:
- package-ecosystem: github-actions
  directory: /
  schedule: {interval: daily}

with:

update-submodules.yml

name: Update submodules
on:
  schedule: [cron: 0 6 * * 0]
  workflow_dispatch:
jobs:
  update-submodules:
    runs-on: ubuntu-latest
    steps:
    - name: Checkout repository
      uses: actions/checkout@v4
      with:
        submodules: true
        fetch-depth: 0
    - name: Update submodules
      run: |
        if git commit -S -am"Update submodules (automated)

        $(git submodule -q foreach '

          # latest release
          tag="$(gh api repos/{owner}/{repo}/releases/latest -q .tag_name)"

          cur="$(git describe --tags)"
          if [ "$cur" != "$tag" ] && printf "%s\n%s" "$cur" "$tag" | sort -VC
          then
            git switch -dq -- "$tag"
            printf "%s\n" "$name: $cur -> $tag"
          fi'
        )"; then
          git show --raw
          git push
        fi

based on discussions in: dependabot/dependabot-core#2192 (comment)

[mcanouil]

You can do:

version: 2
updates:
  - package-ecosystem: "gitsubmodule"
    directory: "/"
    schedule:
      interval: "weekly"

This will update to the latest commit on the default branch.
I am not sure, using releases is worth it, but that's up to you.
Hopefully, GitHub will support update via releases.

Edit: If you use the update-submodules.yml you don't really need Dependabot.
Here you configuration, is to update GHA in your workflows, for instance Dependabot will check for actions/checkout updates and that's it.