🐛 Bug - tina-astro-starter returns 404 on new content

[tinanoob]

🐛 Describe the Bug

When running the "tina-astro-starter" locally in development mode,
new entries added to a collection aren't rendered properly after Save.
The entry appears in the list of entries in the collection but clicking on the entry shows a 404 page instead.
This condition persists until tina/astro is restarted manually.

I did not try any other integration, nor did I try Tina Cloud.

🧑‍💻 Reproduction Steps

1. cd /tmp
2. pnpx create-tina-app@latest --template tina-astro-starter
   ✔ Which package manager would you like to use? › pnpm
   ✔ What is your project named? … my-tina-app
3. cd my-tina-app
4. pnpm run dev
5. open http://localhost:4321/admin/index.html#/collections/blog/~
6. press Add Files
7. Enter 'New' for Title, Description, Body and Filename. Choose todays for Publication Date and Updated Date.
8. Press Save
9. Observe "New" appears in the Blogs list
10. Click on the "New" blog just created
11. Observe browser navigates to url http://localhost:4321/admin/index.html#/~/blog/New
12. Observe the astro logo and "404: Not found", "Path: /blog/New"
13. Observe error in the terminal

12:22:46 [WARN] [router] A `getStaticPaths()` route pattern was matched, but no matching static path was found for requested path `/blog/New`.
Possible dynamic routes being matched: src/pages/blog/[...slug].astro, src/pages/[...slug].astro.
12:22:46 [404] /blog/New 3ms

14. In the terminal, press Ctrl-C
15. pnpm run dev
16. Refresh the page in the browser
17. Observe the http://localhost:4321/admin/index.html#/~/blog/New page renders without error

🎯 Expected Behaviour

The new content should appear after it's created.
The user should not have to manually restart tina/astro after adding a new entry.

If this is a known limitation of the tina-astro integartion, the documentation
https://tina.io/docs/frameworks/astro should point it out.

Possibly related

tinacms/tinacms#2534 reports that restart is required
when adding a new collection, but in this case a restart is required when adding
a new entry to a collection.

[tinanoob]

I tried following equivalent steps using https://tina.io/docs/introduction/using-starter

1. cd /tmp
2. npx create-tina-app@latest
   ✔ Which package manager would you like to use? › pnpm
   ✔ What is your project named? … my-tina-next
   ✔ What starter code would you like to use? › ⭐ NextJS starter
3. cd my-tina-next
4. pnpm run dev
5. ... follow equivalent steps to create a "New" blog entry
6. Observe newly created entry appears when clicked
7. Observe terminal output

 GET / 200 in 40616ms
 GET / 200 in 73ms
 GET / 200 in 64ms
 ○ Compiling /posts/[...urlSegments] ...
 ✓ Compiled /posts/[...urlSegments] in 3.4s (11228 modules)
 GET /posts/New 200 in 4585ms   ◀───── this produced the error in the tina-astro-starter 

Since there is no error here, this leads me to believe the problem is more likely in the astro integration.

[tinanoob]

I experimented with this integration https://github.com/rafay99-epic/Astro-Portfolio-Blog

After cloning the repository and running 'nvm install' I saw some warnings about my node version.
So I did nvm use v20.10.0 and checked out a clean workspace.
I fixing a missing \" in the package.json file for the dev script: "dev": "tinacms dev -c \"astro dev\""
I was able to start it with npm run dev.
This appears to be a slightly older version of Tina/Astro.
Creating a new "Article" and clicking on it leaves the user in the edit view.
It seems the tina visual editing was not supported on this version.
Navigating to the page directly worked as expected. There were no 404 errors.

 astro  v5.5.5 ready in 2431 ms
┃ Local    http://localhost:4321/
┃ Network  use --host to expose
13:42:02 watching for file changes...
13:42:22 [200] / 662ms
13:43:32 [glob-loader] Reloaded data from New.mdx
13:43:32 [glob-loader] Reloaded data from New.mdx
13:45:20 [200] / 115ms
13:45:22 [200] /blog 14ms
13:45:25 [200] /blog/new/ 94ms ◀───── The new article rendered 

This older integration didn't produce the 404 but doesn't appear to support the tina visual editing.

This is odd. It appears this integration is using latest tinacms

    "astro": "^5.5.5",
    "tinacms": "^2.7.3"

but visual editing isn't working in the expected way here.
There appear to be signficant differences in the tina/ directory
between this and the tina-astro-starter so perhaps this
integration isn't setup to use the visual editing.

[tinanoob]

Looking closer at tina-astro-starter and Astro-Portfolio-Blog.
It seems tina-astro-starter defines the blog collection with a loader
but the collection defined by Astro-Portfolio-Blog does not specify a loader.

If I understand correctly this means the (working?) Astro-Portfolio-Blog isn't caching the static routes in the way tina-astro-starter does.

It also seems that editing a typescript file or changing the content.config.ts in the tina-astro-starter
will cause astro to revalidate it's cache

14:49:12 [content] Syncing content
14:49:12 [content] Content config changed
14:49:12 [content] Clearing content store
14:49:12 [content] Synced content

then the page for the newly added entry loads as expected.

Since the Tina admin can list and edit newly created entries, I suspect the problem lies
in the astro loader used by tina-astro-starter. If this loader's cache isn't invalidated somehow
when a new item is created then Astro won't know to serve it.

[tinanoob]

Here is what https://astro.build/blog/content-layer-deep-dive/#content-layer-lifecycle says

Astro components and pages can then use getCollection or getEntry to query the data. The data is immutable at that point, so the pages are all querying the same snapshot, compiled at build time. This applies whether the page is being prerendered at build time, or server rendered on-demand. The important thing to note here is that the data store is only updated at build time: a deployed site cannot change the data store. If a data source needs to update a collection it must do this by triggering a new build.

Although immutable in production, when running astro dev the data store can be updated on demand by users with the s+enter hotkey, or by integrations. They can do this in many ways, such as registering a development refresh endpoint or opening a socket to a CMS to listen for updates.

So in dev mode an integration needs to explicitly tell astro to refresh when there is new content.
I can't see where tina-astro-starter is doing that.

[tinanoob]

Mentioned the issue on https://discord.com/channels/835168149439643678/1357493630184591560
Going to keep researching to see what integrations can do to let astro know about new content.

[tinanoob]

Looks like this may be a more fundamental issue.
Opening tinacms/tinacms#5615

[tinanoob]

Examining the astro source, it appears astro dev offers what they call a "shortcut" to "sync content layer".
https://github.com/withastro/astro/blob/main/packages/astro/src/core/dev/restart.ts#L197

Pressing the "s" key followed by Enter/Return in the terminal where astro dev is running appears to cause it to rescan the files in the collection. In the tina-astro-starter the following seems to work:

• create a Blog entry with Add Files
• press "s" and Enter/Return in the terminal
• click on the new entry in the Tina admin view
• observe Astro opens the newly added blog entry and allows visual editing in Tina.

This may offer a workaround.
Perhaps the integration could provide a setting to do that automatically when new content is added.

[tinanoob]

Looking closer at the Astro Integration API, it appears there is a refreshContent api for this sort of situation. The documentation there gives an example which adds a middleware to provide a /_refresh endpoint which provides access to Astro's refreshContent api.

I tested it with a small alteration to the astro-tina-directive/register.js of the tina-astro-starter after adding a pages/404.astro file which included a <a href="/_refresh">Refresh Content</a> link.

With this in place I was able to

• create a Blog entry with Add Files
• click on the new entry in the Tina admin view
• observe the expected 404 page
• click on the Refresh Content link
• press the back button followed by the browser refresh page button
• observe Astro opens the newly added blog entry and allows visual editing in Tina.

This approach provides a small improvement over using the "s" shortcut since the user won't have to leave the browser to continue working with the content.

Ideally the tina-astro-starter could extend this. The refreshContent api appears to allow more selective refreshing.
If Tina offered some kind of Save hook after Add Files then I could envision making this selective refresh completely transparent to the user.

/**
 * @type {() => import('astro').AstroIntegration}
 */
export default () => ({
  name: "client:tina",
  hooks: {
    "astro:config:setup": ({ addClientDirective }) => {

      addClientDirective({
        name: "tina",
        entrypoint: "./astro-tina-directive/tina.js", 
      });
    },

    // adapted from https://docs.astro.build/en/reference/integrations-reference/#refreshcontent-option
    // uses GET instead of POST
    "astro:server:setup": ({ server, refreshContent }) => {

      // Register a dev server webhook endpoint
      server.middlewares.use('/_refresh', async (req, res) => {
        if (req.method !== 'GET') {
          res.statusCode = 405
          res.end('Method Not Allowed');
          return
        }
        try {
          await refreshContent({});

          res.writeHead(200, { 'Content-Type': 'application/json' });
          res.end(JSON.stringify({ message: 'Content refreshed successfully' }));

        } catch (error) {

          res.writeHead(500, { 'Content-Type': 'application/json' });
          res.end(JSON.stringify({ error: 'Failed to refresh content: ' + error.message }));
        }
      });
    }
  }
});

[JackDevAU]

Hey @tinanoob,

This looks great! Feel free to open a PR, and we'll get it merged in. 🚀

[OliverSpeir]

@tinanoob nice work, do you plan on sending a PR?

[tinanoob]

@tinanoob nice work, do you plan on sending a PR?

@OliverSpeir @JackDevAU I'm currently blocked but I would be happy to work on one once I have the information I need.

What I've done so far is a bit of an experimental hack so I was hoping someone here, in the tinacms/tinacms#5615 discussion or in the Tina/Astro gives 404 for newly added content discord discussion with a little more knowledge of the tina / astro architecture could offer some insights and suggest the right way to do this.

Astro's refreshContent API appears to provide a way to pass an arbitrary context object to a list of named loaders. The code I wrote makes no use of these so when the "/_refresh" middleware endpoint calls refreshContent presumably all the Astro loaders would refresh all collections. This might technically work but seems a bit heavy handed. Ideally only the loader associated with the newly created collection entry should be called and it should make use of the context to update the appropriate state.

I've looked around a bit to learn more about Astro and came across astro-loader-obsidian. This loader makes use of a syncData method within an onChange callback which is invoked when the content is loaded or the underlying file is modified. syncData directly updates the underlying store with the new content.

In contrast the tina-astro-starter loaders leverage the generated tina client to obtain collection data. I'm unaware of whether or not the generate tina client supports something similar to astro-loader-obsidian's onChange callback.

If the generated tina client has some similar way to provide notifications when the underlying content is changed then presumably the tina-astro-starter loaders could hook up a function similar to syncData and there would be no need for any middleware endpoint like mine to call refreshContent.

But a middleware hook could make sense if the generated tina client provides no such notification mechanism. In that case the next question that comes to mind is whether or not Tina provides any sort of "post-create" lifecycle function called after new content is created. If so, that function could call the middleware hook and pass sufficient context to enable the Astro refreshContent api to record the new data.

So in summary my questions are:

1. does the generated tina client provide a notification api which loaders could use to update the corresponding Astro store?
2. if so, is there any reason (e.g. data race, lack of context, etc) not to take that approach?
3. if not, does tina offer any kind of post-create lifecycle function we could leverage?
4. if so, where can I find more about it?
5. if not, would it make sense to add compensation logic in the 404 path. e.g. before showing a 404 see if the content actually exists, trigger the refreshContent and refresh the page.
6. any other ideas?

I'm new to both Astro and Tina so please correct me if I'm wrong about any of this.

[Aierie]

Came from the other issue. Leaving an alternative here that can be refined - hook into Vite's dev server's watcher and refresh content when markdown or mdx changes:

"astro:server:setup": ({ server, refreshContent }) => {
      server.watcher.on('unlink', async (filePath) => {
        let extension = path.extname(filePath);
        if (extension == '.md' || extension == '.mdx') {
          await refreshContent();
        }
      });
      server.watcher.on('add', async (filePath) => {
        let extension = path.extname(filePath);
        if (extension == '.md' || extension == '.mdx') {
          await refreshContent();
        }
      });
    },