Set up cron job with actions to compile automatically

[FichteFoll]

And a feed, so Dash and Zeal can update them within the app.

[FichteFoll]

Problem: The contents of the website are copyrighted and unless we are granted a license, we aren't allowed to redistribute. Will need to ask Jon about this directly.

[michaelblyons]

Problem: The contents of the website are copyrighted and unless we are granted a license, we aren't allowed to redistribute. Will need to ask Jon about this directly.

Resolved in #10

[michaelblyons]

How do you imagine this works? GitHub Actions has

on:
 schedule:
   - cron:  "42 7 15 * *"

We could have it spin up once a day/week/etc., check @maliayas's documentation diff-er (Atom feed) to see whether there's something new, spool up a VM, and build the docset.

[FichteFoll]

I believe a regular schedule should be fine, with some diff checks between the previous run. With that, a run every day or so would be fast enough to be updated along with a new release without putting any noticable load on the website.

I'm unsure if we need human review in form of a PR or similar before releasing it to the public, though. The review process has the advantage of not publishing a broken version in case the docs are updated in a majorly incompatible way, but on the other hand that requires more work to set up and review (which could be a bottleneck). Generally, I doubt we will run into breakage a lot and would rather clean up any potential mess after it happens for this project.

(That's of course assuming that we get into the Dash/Zeal feed for auto-updates.)

[michaelblyons]

Maybe it could generate a pre-release automatically, and you promote it (if possible) or re-release it (if not) as a stable version after review.

[FichteFoll]

I would then rather prefer the PR with a diff and then automatically creating a release from master.

[michaelblyons]

But there would be no code to put in the PR. Maybe open an issue whose text include the diff between old docs and new docs? Then you can build locally, verify that the new items are indexed (if not, then make a PR to merge), and make a release?

[FichteFoll]

Right, not in the default branch anyway. We could use a separate branch, though, think gh-branches, that's entirely separate from the code and contains only the built version which will be used for distribution. That would be reviewable and diffable.

I just don't believe fetching a built docset and comparing it locally is the way to go here (or even generate a diff otherwise because then the version to diff it with would need to be stored somewhere, too, in which case we might as well go the PR way).

[maliayas]

In case it's useful for anyone, here is the command I use on https://github.com/maliayas/SublimeText_Documentation

wget \
    --mirror \
    --no-parent \
    --page-requisites \
    --convert-links \
    --continue \
    --adjust-extension \
    --user-agent="" \
    --execute="robots=off" \
    --wait=1 \
    --append-output="mirror.log" \
    --no-verbose \
    --max-redirect=0 \
    https://www.sublimetext.com/docs/index.html

And I want to add that, this process requires a review. Sometimes network fails, sometimes ST web site fails, so every now and then unintentional commits occur. I locally undo them on my PC and never push to GH. On the other hand, I try to separate trivial and serious changes into separate commits and only git-tag the latter (also a review process).

P.S. Thanks @kapitanluffy for the ping. I've just found your email in my spam box. Since my email is sort of broken, I'm replying here.

Cheers!

[michaelblyons]

If this issue is strictly about GitHub being able to generate a docset, that is handled at #16.

If we are also talking about automatic generation for new Documentation site changes, maybe we can make @maliayas' repo a submodule of this one. (Edit: #18) He's already done the spidering. The options we currently use that he doesn't are --timestamping and --recursive, and they may not be important.

Then, maybe we can get some service to make PRs here whenever his repo changes. The repos can build automatically for the PR, and if we're happy, we merge.

An additional benefit is that builds are clearly tied to specific versions of the Doc site, and you can return to one with a revert.