feature request: option to publish "clean" to gh-pages?

[ijlyttle]

As best I can tell from examining the code (and behavior I see), if I have an existing gh-pages branch, quarto publish gh-pages does something like this:

1. Pulls gh-pages branch into a temp directory
2. Renders into _site (or wherever) in local directory
3. Copies local _site into temp directory
4. Commits changes into gh-pages branch
5. Pushes gh-pages

If a file exists in the gh-pages branch, pre-render, that does not exist in the rendered local _site, then that file will continue to exist in the gh-pages branch, despite it not existing in the rendered local _site.

Assuming my assessment is correct (big assumption, I know), could there be a way to tell publish to replace completely with the new _site, keeping nothing from the old _site?

Does this option exist, and if not, could it?

Thanks!

[mcanouil]

You could write a pre-render script that does the cleaning.

or before running quarto publish, something like below:

git push origin --delete gh-pages
git checkout --orphan gh-pages
# preview files to be deleted
git rm -rf --dry-run .
# actually delete the files
git rm -rf .
git push gh-pages

[ijlyttle]

Thanks!

I can see how a pre-render script would work, but, if I understand correctly, it would also wipe out my published site time I rendered locally. I'll look into the manual script, though.

Holding out hope that the quarto devs might consider an option for quarto publish gh-pages

[mcanouil]

You do not have many options there to remove obsolete pages, either you remove everything and push a clean version, or you manually perform the cleaning.

For example, in my (long) work in progress website, I use a pre-render script to create a listing page, which actually does not exist before rendering, if some kind of cleaning process was to be added by default, I am pretty sure, it would remove my previous listing everytime before pushing a new version.

In my opinion, deleting files should not be automatic but carefully done by the user.

[ijlyttle]

To be clear, I am not asking to make "clean" publish the default; I agree that it should be done with consideration.

I am proposing that it could be an option when publishing.

[mcanouil]

This is doable.
Looking at the code, it should be straightforward to add a --clean option which will add a single git command such as git rm -r * just before the render code if the option is present (
https://github.com/quarto-dev/quarto-cli/blob/main/src/publish/gh-pages/gh-pages.ts#L144-L158)

Something like below, but with some upstream changes for the option to get parsed and send to internals scripts (I do not know enough the TypeScript codebase to try to make a PR at this point).
The "publish" code is from @jjallaire, so he'll know better if and how to add such option.

if (options.clean) {
  await gitCmds(input, [
    ["rm", "-r", "*"],
  ]);
}

Note: There are some files that should not be deleted if they exist in the gh-pages such as the .nojekyll and CNAME files.

[ijlyttle]

That sort of thing is exactly what I have in mind, thanks!

[ijlyttle]

Given new features for board_url() in pins-r (#661, #681), I am trying out a new project workflow:

• analysis done using R using Quarto
• results written using pins::board_folder()
• pins directory incorporated into the Quarto _site
• publish using gh-pages

This allows me to make "the data" available as pins to whomever wants to use a board_url() to consume.

One sticking point is the seeming inability to specify a "clean" update to the gh-pages branch.

Following @mcanouil's suggested code for cleaning the temporary directory, I had a look at how a --clean option could be implemented. Here's my (assuredly incomplete) assessment:

• add clean?: boolean; to:

  quarto-cli/src/command/publish/options.ts

  Lines 8 to 15 in 2d59cd2

  	export interface PublishCommandOptions {
  	token?: string;
  	server?: string | null;
  	id?: string;
  	render?: boolean;
  	prompt?: boolean;
  	browser?: boolean;
  	}

• add something like:

   .option( 
     "--clean [clean:boolean]", 
     "Valid only for gh-pages, specify all files to be overwritten",
    {default: false} ,
   ) 

  to:

  quarto-cli/src/command/publish/cmd.ts

  Lines 70 to 73 in 2d59cd2

  	.option(
  	"--token <token:string>",
  	"Access token for publising provider",
  	)

• add clean: boolean; to:

  quarto-cli/src/publish/types.ts

  Lines 22 to 30 in 2d59cd2

  	export type PublishOptions = {
  	input: ProjectContext | string;
  	server?: string | null;
  	token?: string;
  	id?: string;
  	render: boolean;
  	prompt: boolean;
  	browser: boolean;
  	};

• add @mcanouil's code:

  if (options.clean) {
    await gitCmds(input, [
      ["rm", "-r", "*"],
    ]);
  }

  to (just before rendering):

  quarto-cli/src/publish/gh-pages/gh-pages.ts

  Lines 150 to 157 in 2d59cd2

  	// render
  	const renderResult = await renderForPublish(
  	render,
  	"gh-pages",
  	type,
  	title,
  	target?.url,
  	);

There would have to be a corresponding PR to quarto-web to update the docs.

A few questions:

• Is this a reasonable change?
• I have been through the gh-pages code, but not through the other platforms. Do they operate differently, i.e. is the default to add files to an existing set of files?
• Is --clean the right option? Maybe --no-add, otherwise?

What do you think, @cscheid, @jjallaire? If you think it feasible, I'd be happy to start a PR.

Thanks for letting me impose!

[mcanouil]

Cleaning all the time means there is no way to host compositional or multiple websites/pages.
I think cleaning should be the default but a "no-clean" or whatever argument name should be made available to allow more complex setup.

For example, I could have a listing using iframes to include Reveal.js presentations rendered separately but published alongside the website in which the listing is.

[jjallaire]

This feature though is about managing a gh-pages branch on your behalf that has the entire contents of a site. I think if people want to do fancier things than that they should do so manually with their own scripts. i.e. this feature is intended as a "hands free" publish my site -- you can do many more elaborate things w/ gh-pages and I'd just assume not take all of these on in this feature.

[mcanouil]

It makes sense.
(I am not even using the publish action in my CI/CD workflows 😅)

[cscheid]

@ijlyttle is this enough for you to start a PR?

[ijlyttle]

Yes - thanks, it sounds like this will be the "minimal" version to make it only clean, using the code suggested by @jjallaire, provided by @mcanouil.

I'll spin something up locally to make sure it works as advertised, then make a PR, adding an item to news/changelog-1.3.md.

[mcanouil]

@ijlyttle Maybe mark this discussion as "resolved"/"answered" since your PR (#3692) was merged.