make apply-patches work for core package contributions

[a-teammate]

the problem

contributors to core packages want to test their contributions, but usually they don't have it merged into elm-janitor "stack-XY" branches.

So instead they would have it in
a) a local copy of the elm-core repo
b) a pull request to the elm-core repo
c) sometimes in a branch to elm-janitor (bundeling some PRs to the respective elm-core package)

currently the only way to use apply-patches is, if you have access to elm-janitor and the guts to create a stack-1.0.x branch (which will also potentially break all current users of apply-patches, as your new branch would be picked up for them).

implementation

i guess this would require some major rewrites, as currently the implementation is passing around a "version".

Challenges:

1. add code for not downloading but copying from a local folder
2. add solution for telling the version of unpublished packages

Especially the second I see as a little challenging:
they don't have a bumped version in elm.json as this is the responsibility of evan (after merging in some PRs, he will bump the version). So instead the version in the repo would be incremented?

I think this has the potential of making the elm-janitor contributions a little easier as well.
As all the "stack-1.0.x" branches would not be required anymore, but just the elm-janitor/main branch would be enough.
But that is also why I cant take over that task as novice, as it will probably require some deeper thoughts from people who already set up that system.

[marc136]

Thank you for your comment @a-teammate.

I wanted this script to be something that I can run in CI with few dependencies.
Its aim is to download and unpack stable patch versions (from the branch stack-x.z.y), and to mark the last commit on this branch inside the elm-janitor-commit.json file to track changes between local and remote.

It would be possible to add another script to download code from other remote repositories, or to copy it from a local directory into the proper place inside $ELM_HOME, but as you already mentioned there would be many possibilities.
So I also consider the first challenge you mentioned a hard one.

If this repo were to include such a script, it would be separate (because it behaves very different to the download-only one), and I would prefer to have it in another repo and CLI tool instead.

I like your idea of incrementing a value/version with every change, but for the patches it made sense to show which "official version" was changed, so stack-1.0.5 means that a stack of changes was applied on top of the 1.0.5 release of the upstream repository, so there should be no confusion what base version was extended.

The commit id does not directly tell you how many versions have passed between the locally installed one and the latest, but that information is at least recoverable (unless the branch is rebased, where it would only tell us that it is now different). But if we increment a value, there would need to be a single source of truth, which would be at least a problem when mixing local changes from the file system and changes downloaded with git.

The main benefit of having stack-x.y.z branches is that multiple versions could be patched.
If some people rely on the janitor-patched version of core/1.0.5, but Evan would for instance only include some of the fixes on a next v2.0.0 with other major breaking changes, we can still install stack-1.0.5 and a stack-2.0.0 version from elm-janitor and get those fixes for both upstream versions.
Or if v1.0.6 addresses only one bug in a different way (than the patch in janitor), the other changes from the stack could be applied again.

For working with the core packages, I would use another approach, and consider addressing the problems you listed out of scope of the apply-patches script.

[marc136]

Working with the core packages and the challenges you mention reminded me of a repository that I showed during elm-camp but did not publish yet. There I started creating tests for each patch included into the elm-janitor stacks.

I was trying to get it in a publishable state (so I could also include a link) but you are already waiting two weeks for an answer, and I did not yet manage that, so I wanted to at least respond.
I hope to finish cleaning up the repo next weekend.

---

To test the janitor fixes and mess with the core packages, I created a repository and added every forked core package of elm-janitor as a git submodule.
Then I also added the original elm/* repositories as another remote repo to each.

When I now link the submodules into a custom $ELM_HOME directory, I can switch between different branches (official, stack-*, PRs) without much hassle, and still ensure that my custom versions are not a requirement for Elm projects that don't need the changes.

For example:

# The Elm compiler respects this environment variable
ELM_HOME=/tmp/elm-home

ln -s `pwd`/packages/core $ELM_HOME/0.19.1/packages/elm/core/1.0.5

# remove (potentially outdated) build artifacts created by the Elm compiler
rm -rf packages/*/artifacts.dat

# and remove local Elm caches from the current project
rm -rf elm-stuff

# then I can make a build using the patched versions
elm make src/Main.elm

If you are confused about the many branches, Rupert documented the approach for branch naming in
https://github.com/elm-janitor/manifesto/blob/master/git-help.md

I only added a few bash scripts to help with tedious tasks like clearing the caches, or linking repositories, but maybe it makes sense to create something platform-independent.

[marc136]

Hello @a-teammate, I actually managed to upload my test suite.

https://github.com/elm-janitor/test-suite#readme

If you are interested in developing bugfixes to Elm core repositories, please give it a try.

[a-teammate]

ah, sorry for the late reply! I don't have github notifications activated..
very cool! thank you! will use