Merging listing metadata

[andrewpbray]

Description

I have a website with several listings but some of the contents I'd like to store in a separate meta data file. I've tried an implementation from the default Quarto blog that adds a new _metadata.yml at the project root along with a new post called extra.qmd. The contents of the metadata file is:

listing:
  contents:
    - extra.qmd

When I render the website, it succeeds in adding extra.qmd to the blog listing but does so twice:

Any leads as to what is going on here?

Reprex here: https://github.com/andrewpbray/mwe-blog.

This is on Quarto 1.3.433.

[andrewpbray]

An interesting and less surprising addendum:

I tried renaming _metadata.yml > _xetadata.md, adding --- to the first and last line of that file, then including it the body of index.qmd with {{< include _xetadata.md >}}. No dice: Extra post doesn't show up. I guess this just demonstrates that the include shortcode is run after the listing yaml/json is built.

[mcanouil]

I don't know what you try to do with include but it seems very complicated and not reliable while contents accepts a lot of things:

  contents:
    - path: https://github.com/mcanouil/quarto-invoice
      image: /projects/quarto-extension.png
      title: "Invoice (Typst)"
      description: "Quarto Typst template extension to make invoices."
      date: "2023-11-24"
      categories: [Quarto, Extension, HTML]

contents: posts

contents:
    - publications.yml

Etc.

Edit: Note also and as described in the documentation, _metadata.yml is a directory level file, while _quarto.yml is a project/root level.

I am not in front of a laptop, so that's the obvious things wrong.
(I am not saying that maybe Quarto should safeguard against those things not following the documentation and how things were designed)

[andrewpbray]

Fair question!

I have a highly-functional but highly-complex website for a course that I teach. Other instructors at my institution have reached out to see about borrowing some of the functionality for their own course websites. To get there, I need to make interacting with this functionality as simple for a novice as possible and hopefully keeping consistent with things they'll learn to do through the Quarto docs. To that end, I'm centralizing the configuration options into a single file / schema (a yaml file) that is processed by scripts and/or filters.

In this case, there is a schedule.yaml file that contains a list of all assignment types and course assets as well as their publish date. A snippet looks like this:

- week: 1
  days:
    - display-date: "Aug 1"
      date: 8/1/23
      topic: "Understanding the World"
      items:
        - type: Notes
          href: foo/notes.qmd
        - type: Slides
          href: foo/slides.qmd
        - type: PS
          href: foo/ps.qmd
    - display-date: "Aug 3"
      date: 8/3/23
      topic: "Taxonomy of Data"
      items:
        - type: Notes
          href: foo2/notes.qmd
        - type: Slides
          href: foo2/slides.qmd
        - type: PS
          href: foo2/ps.qmd

There is an ejs template that reads this in to make a course calendar. There is also a script that ignores files with dates in the future. When hooked up to CI, the effect is an auto-publish feature.

I'd like to draw from the same file to make listings (i.e. a "Problem Sets" page). The original plan was to write a script to pull out all href of the same type and have it write it into a temporary _metadata.yml file. The advantage here is that the problem-sets.qmd could have a normally functioning listing yaml header, so "turning on" the listings-from-schedule feature doesn't change anything about the way that those listings pages' source looks.

Another option is to have the script write a ps-items.yml that is hard-coded into the contents of the problem-sets.qmd page. One reason I didn't want to do this is that I'd like to us some of the default ejs templates, which access metadata that results from filters that Quarto runs (reading time, image, etc). I don't know how to run those filters within my script so that the ps-items.yml that I'd be making would have all of the needed metadata fields.

[mcanouil]

Why not provide listing content as YAML file without using the _metadata.yml which is a special file?
You could even use Quarto profile to modify contents.
To me, the use of _metadata.yml here is wrong.

On a related note, see:

• profile-specific metadata files not being incorporated #6859

Alternatively, it might be possible for you to share your project as a template extension, which could make things a lot easier without all the circumvolution.

[andrewpbray]

I think that'd work for most fields but how would I generate listing content such as reading-time?

And indeed, I'm aiming towards a template extension but I'm a ways off from that right now.

[mcanouil]

Really hard to say as the picture is far from complete.
For instance, it's easy (from what you said) to write your listing content as a YAML file, and use that file inside the Quarto document hosting the listing.
This way, it is not your listing that is changing, but its content and there is nothing to try to merge at different level.

  contents:
    - default.yml
    - your-course.yml

EDIT: I never tried to set two YAML, but I don't see any reason for it to not work.

[dragonstyle]

Not 100% sure whether this has already been covered, but contents should be able to be pointed a yml file which can itself contain paths to qmd files which will pass through into the contents list- we resolve a path property which points to a qmd, rmd or md file into a content item with metadata as if it were being pointed to directly (if that makes sense - would handle things like reading time, reading a description from the contents of the document, etc...)?

So making the file:

_file.yml like:

- path: extra.qmd

and the listing like:

---
title: "Mwe-blog"
listing:
  contents:
    - posts
    - _files.yml
  sort: "date desc"
  type: default
  categories: true
  sort-ui: false
  filter-ui: false
page-layout: full
title-block-banner: true
---

I think will get you there with an externalized file list.

[andrewpbray]

Yep, I think this method will work just fine.

Thanks very much for helping me think this through!

[andrewpbray]

@dragonstyle I just checked that and was kinda surprised (but in a good way). I had been under the impression that contents added via yaml file had to be the fields referenced in the ejs. Since some of those rely on output of Quarto filters, I thought I was out of luck, but looks like if you just give it path, those filters get run downstream, so you still have access to them in the ejs.

This works - great!

[mcanouil]

Take a look at my listing content as YAML:

• https://github.com/mcanouil/mickael.canouil.fr/blob/main/publications.yml
• https://github.com/mcanouil/mickael.canouil.fr/blob/main/publications.qmd