Ignored render targets behave differently depending on method of ignore

[andrewpbray]

I've observed some surprising behavior on a website that I have and I'm curious whether this is the intended behavior or not.

Here's a simplification of the website that I'm working with.

/my_website
├── _quarto.yml
├── index.qmd
├── unit-1
│   └── post.qmd
│   └── other-files.qmd
├── unit-2
│   └── post.qmd
│   └── other-files.qmd

The index.qmd has a listing on it structured as

---
listing:
  contents:
      - "*/post.qmd"
---

My goal is to only render the material in the unit-1 directory. Two approaches that I've tried.

1. Exclude unit-2 by specifying render targets.
2. Change unit-2/ to _unit-2.

Approach 1 involves changing _quarto.yml to read:

project:
  website:
    render:
      - "*.qmd"
      - "!unit-2/"

The result is that none of the .qmd in unit-2/ are rendered, however, index.qmd throws this warning:

WARNING: File /unit-2/post.qmd in the listing 'listing' contains no metadata.

When I view the webpage, there is a blank listing field for the second post, reflecting that lack of metadata.

Approach 2 also prevents the rendering of anything in unit-2/, however there is no warning about the listing and it shows up as preferred, without the blank spot for the second post.

My guess is that this is a similar issue to the one that came up in #3879 . The order of operations on quarto render seems to be.

1. Build up the listing files (which ignores files that start with _)
2. Run pre-render scripts
3. Run quarto render on all of the qmd files controlled by _quarto.yml.
4. Run post-render scripts

Is this correct? If so, is it intentional that 1. doesn't consult _quarto.yml for ignored files?

Thanks for your help!

[mcanouil]

Did you know you can ignore files by setting a .quartoignore file?

I believe the negative glob is not really working as it should or not handled expectedly by Quarto when rendering the website, i.e., likely a bug to me.

[cderv]

I believe the negative glob is not really working as it should or not handled expectedly by Quarto when rendering the website, i.e., likely a bug to me.

Not implemented yet - follow #4061

[mcanouil]

In '.quartoignore' but according to the documentation for render YAML key it should be already implemented.

[dragonstyle]

I don't believe that the render key is valid under the website key. I believe it belongs here:

project:
  type: website
  render:
    - "*.qmd"
    - "!welcome/*.qmd"

website:
  title: "Blog"
  navbar:
    right:
      - about.qmd
      - icon: github
        href: https://github.com/
      - icon: twitter
        href: https://twitter.com
format:
  html:
    theme: cosmo
    css: styles.css

The above example works with our default blog to exclude the welcome post from being rendered, which appears to in turn not allow it to appear in a listing. I've attached a zip of that sample project so you can compare to see if there are differences that may explain this.

blog.zip

Depending upon your goal, you might also consider using the draft: true front matter of the document that you'd like to exclude from the list. This will keep it in the render list (and deploy / publish list), but exclude it from listings and search, so it would only be visible to someone who knows the URL (basically the equivalent of being 'unlisted').

[andrewpbray]

@dragonstyle Good catch about the render key. I think that was a typo on my part.

When I render your blog, it works a-ok. When I modified it to replicate features of my project structure, it returned to the warning message and the empty listing. Those changes were.

1. Renaming the post files from index.qmd to post.qmd.
2. Changing the listing on the index page to read.

---
title: "Blog"
listing:
  contents: "posts/*/post.qmd"
  # etc.
---

The reason for my approach to paths and naming is I'd like my repo to be organized according to a structure that makes the most sense for course materials: all of the .qmd files - blog post, slides, handouts - to be in the same directory for that day of class.

Here's that modified blog:

blog-empty-listing.zip

[dragonstyle]

Ok, that test project really helped me narrow down the issue, thank you. The basic cause of the different behavior is how we treat globs in listing to allow users to just create a listing of files on disk (not just inputs in the project). Basically, we treat paths to files as resolving using the raw file systems, and paths to folders as resolving against the inputs. We could try to be a little smarter than this (for example, treating globs to qmd files as inputs), but I think the best short term thing would be to just update your listing glob, if this works for you:

---
title: "Blog"
listing:
  contents: "posts"   # << this will be interpreted as `posts/**` against render list
  sort: "date desc"
  type: default
  categories: true
  sort-ui: false
  filter-ui: false
page-layout: full
title-block-banner: true
---

Would that work for you?

Sorry for the confusing behavior, we are trying to balance a number of different goals with that listing glob and it can make for confusion that way it is currently implemented.

[andrewpbray]

Unfortunately my equivalent of the posts directory has all sorts of .qmd files in addition to actual posts so I need to be able to target each post.qmd specifically and yank it into a listing.

I'm sure there's tons of different use cases you're needing to balance in designing this stuff, so no worries. Approach 2 works just fine. The only downside is that the folders that I least want to see now appear at the top of the directory due to alphanumeric ordering. But life could be worse =)

Thanks for your help getting to the bottom of it!

[andrewpbray]

@dragonstyle A related issue came up for me today when trying to build a listing of problem sets. The listing pulls its contents from "*/ps.qmd". Those files exist in directories across the repo and are generally files with format: pdf and displayed in this custom ejs listing:

<ul>
<% for (let i = 0; i < items.length; i++) { %>
  <li><a href="<%- items[i].path %>">PS <%= i+1 %>: <%= items[i].title %></a></li>
<% } %>
</ul>

It has generally worked great: it dynamically lists out the problem sets and numbers them appropriately.

The puzzle that I haven't been able to solve is how to include in that listing a ps.qmd file that is not rendered. This is a problem set that I'd like to display as the raw qmd file instead of the rendered pdf to give them a template file to work from. I've tried several things, with no success:

1. Change the file name to _ps.qmd and add */_ps.qmd to the contents list: doesn't show up in the listing.
2. Move the file to */_qmd-ps/ps.qmd: doesn't show up in the listing.
3. Add *.qmd and the pull path to the ps to project: render: preceded by !: appears in the listing but the ejs template doesn't find items[i].title

I'd appreciate any guidance in how to think about what's going on here. There's no identity-render like format: qmd, is there?