[4.2.0][Feature] Publish CSS & JS assets upon composer update

[martijnb92]

Bug report

What I did

Upgrading Backpack to 4.0.18 for the updated packages
composer update

What I expected to happen

By upgrading Backpack I expect the new packages to be copied to the public folder

What happened

No updated packages are copied

What I've already tried to fix it

Previously in the upgrade guide v3 to v4 there was a post-update command for the composer.json file
This command is now removed, so I also updated my composer file to reflect these changes.
Why has this been done?
After each update I manually run:
php artisan vendor:publish --provider="Backpack\CRUD\BackpackServiceProvider" --tag=public --force
This copies the updated packages. Am I missing something or is this auto update command removed on purpose?

Backpack, Laravel, PHP, DB version

Backpack 4.0.18
Laravel 6.5.1
PHP 7.3.8

[tabacitu]

Hi @martijnb92 ,

I can confirm php artisan vendor:publish --provider="Backpack\CRUD\BackpackServiceProvider" --tag=public --force is the right command to publish all assets again.

Indeed, we've removed it from the installation guide, because it had an unintended side-effect: every time you ran a composer update, it published up to 69MB of new assets to your public folder. Even if there was nothing new inside packages. Since most people have their public inside a git repo, this made the git repos of all projects that use Backpack very large, very fast. That's really unfortunate, because it was a very good way for us to push changes to the public assets.

I think a good solution for this would be to have a custom command instead of it, something like php artisan backpack:update, that would:

• check if there are differences between the public/packages folder that the user has, and the one in vendor; we could do that using a checksum, or a public/packages/backpack/assets_version file that can be generated upon npm update;
• if there are differences, ask the user if he wants to publish everything all over again; wait 10 seconds, if no reply, assume they don't;
• if there are no differences, do nothing;

This solution would:

• only publish the public assets if there have been changes to them;
• allow us to push JS & CSS changes in patch versions (4.0.x);

Win-win, I think. What do you think about this plan @martijnb92 ?

[pxpm]

Hello guys!

I would go with with the artisan command also.

But what other things will the update command do that we need to ask the user?

• If the user runs that command he wants to publish something. 👍
• If we have the diff files, does that make sense to publish everything again if we get for sure which files we need to update ?

Why would the user wants to publish everything again ?

Best,
Pedro

[martijnb92]

@tabacitu That sound like a great idea.

However, this might be a stupid question, isn't git perfect for this? If one asset is replaced with an identical asset, nothing should happen in git. I understand it's not very efficient to publish everything and let git sort out the differences but for me it worked fine this morning while updating, 110 modified assets, nowhere near 60+mb. Or am I missing something pretty obvious? Not very familiar with git..
Other options, assuming the packages folder is only in use for Backpack:

• symlink the packages folder to the vendor/backpack/src/public/packages one
• Add a .gitignore file to the packages folder

[tabacitu]

I'm sorry I haven't replied in a while @martijnb92 . This is still an issue I am not comfortable with, and that I'd like to to fix. You're right, Git is great at this. So most of the time people won't be getting a 60+ MB size bump, they'd get a smaller one. But:

• when you do composer update repeatedly, you'd be almost sure we have some new CSS or JS asset; and even though you don't necessarily need it, you'd get it, and you .git folder would get a little bigger;
• do this a bunch of times during development, and you .git folder could become pretty big; pretty fast; I might just be overreacting to this, or being too cautious; It's possible 😄 Let me know if you think so;
• I'm just not sure people expect Backpack assets to be published every time they run composer update; I don't know of any other package that does this, so the behaviour itself is... unexpected; and it could become annoying to have to fix merge conflicts for CSS&JS files you don't own, after every composer update;

I don't know why, but I'm still not really comfortable with instructing people to add php artisan vendor:publish --provider="Backpack\CRUD\BackpackServiceProvider" --tag=public --force to their post-update, in composer.json. But we could easily do it now when we launch 4.1.

I think more people need to see this, and pitch in. Something triggers my spidey-sense. Thoughts anyone? Should we publish assets again upon composer update? Feedback? @Laravel-Backpack/veterans ?

PS. I would love to be able to symlink to our vendor/backpack/src/public/packages folder, and not publish anything, ever. Would love it! I tried to put together a solution a while ago and it didn't work very well - can't remember if the issue was unix-vs-windows, or access to subfolders, or access to parent folders, but I remember I scrapped it. Do you happen to know any other packages that use this symlink method @martijnb92 , so we can take inspiration from them?

[tabacitu]

Hmm... I've just tried to symlink again, using php artisan tinker:

$target = base_path('vendor/backpack/crud/src/public/packages');
$link = public_path('packages');
symlink($target, $link);

And... it works like a charm... I don't know what problems I had with this before, but... it looks like a very good solution...

Of course, we'd still need to make some changes:

• we need to create a command that does this symlink; and call this command as standard when installing Backpack, instead of publishing the assets;
• we can't symlink the entire packages directory, because other packages might want to push things there (ex: our FileManager package); so we might have to
  • (A) rename it to something like backpack-dependencies or backpack-packages or... something very particular;
  • (B) publish those assets in the addons somewhere else...
  • (C) create a symlink for each dependency inside packages...

I don't know, public/packages seems like something people might want to use. And something other packages might want to use. Thoughts anyone?

But... it looks like we might finally have a working solution for this... Which would:

• reduce the project size (all of Backpack's CSS and JS is in one place instead of two);
• allow us to push JS & CSS changes just like we push PHP changes;
• prevent people from having to publish CSS & JS files all the time, and pollute their git history;

[martijnb92]

I would go for option A, since the packages folder might be in use. Creating a symlink for each dependency looks messy to me.

Interesting discussion over here #1404

[eduardoarandah]

@tabacitu

the "laravel way" for assets would be doing the same thing as laravel/ui

https://laravel.com/docs/7.x/frontend

php artisan backpack:assets
npm install
npm run dev

and backpack using the output from laravel mix, let's say... public/dist/backpack

/public/dist/backpack/app.js
/public/dist/backpack/app.css
/public/dist/backpack/images/icons.svg

using webpack.mix.js wouldn't be too much to ask for the users, that's something that has to be done anyway....

let's say the command adds something like this to webpack.mix.js :

mix
  .js('resources/js/backpack.js', 'public/dist/backpack/app.js')
  .sass('resources/sass/backpack.scss', 'public/dist/backpack/app.scss');

[lotarbo]

@eduardoarandah this mean, we need installed node.js right?
But not all my projects needs laravel mix, and some projects doesn't have node on server, because its mb small project, or fronted has his separate way to develop his job.

[eduardoarandah]

@eduardoarandah this mean, we need installed node.js right?
But not all my projects needs laravel mix, and some projects doesn't have node on server, because its mb small project, or fronted has his separate way to develop his job.

Node? Not at all.
Build process is made locally or in CI/CD.
Then rsync assets to production.

This is the recommended way to do it.
Check roots/sage. In that project, you rsync /dist folder

[OwenMelbz]

Node? Not at all.

Build process is made locally or in CI/CD.

Then rsync assets to production.

This is the recommended way to do it.

Check roots/sage. In that project, you rsync /dist folder

This does reaffirm his point, if it’s a small project that can’t have node on the server, then it’s even less likely there’s a CI/CD in the mix which handles deployments which most likely will cost the dev money to do

[eduardoarandah]

@OwenMelbz Laravel is a framework, not a CMS, it’s expected to be built locally and then uploaded to the server.

CICD is an option.

Node is not supposed to be used in the server.

Using node to build in production server is a terrible idea btw, you may consume a lot of memory and trigger the oom killer.

Building locally also opens the possibility of customization via sass variables. (Think of a dark theme)

[OwenMelbz]

@OwenMelbz Laravel is a framework, not a CMS, it’s expected to be built locally and then uploaded to the server.

CICD is an option.

Node is not supposed to be used in the server.

Using node to build in production server is a terrible idea btw, you may consume a lot of memory and trigger the oom killer.

Building locally also opens the possibility of customization via sass variables. (Think of a dark theme)

I’m not disputing that.

I’m referring to how @lotarbo was saying requiring node can complicate certain projects, and how the response of “use a CI” to solve it, isn’t a solution everybody around the world has the luxury to do.

If anything should go down a similar route to Nova with providing a prebuilt published version. Then if people want to delete the prebuilt one and run their own build pipeline they have the choice.

There’s a balance that should be made between encouraging a good developer experience, making the tool and project as accessible as possible for everybody. And then “doing it the right way” Laravel and Rails are built on a mantra of providing solid tools with a good developer experience. Otherwise we’d all be working with complex code bases like Magento.

Empathy for those less fortunate goes along way, so considering how those will work with the tool is important to prevent any sense of elitism or tribalism and alienating others

[eronisko]

I don't find recommending to use Node.js unreasonable at all. I think that Laravel Mix is a really good fit for this use case and Laravel does come with a package.json file included by default. (BTW, Nova also has an expectation for Node.js)

FYI, Rails does make it a little easier to add a 3rd party package (gem) to your project without having to "publish" its assets and pollute your own codebase. But that's only because it comes with a pre-packaged asset pipeline (Sprockets), which does pretty much what Laravel Mix does by default (e.g. asset compilation).

In Laravel, there seems to be a lack of a standard asset pipeline, but I'd say that Mix is getting pretty close.

My 2c: it would be nice if Backpack worked similar to how the other 'standard' Laravel front-end dependencies (e.g. Bootstrap, Vue) are set up:

1. with a dev dependency added to package.json, and
2. a require/@import statment added to the default app.js/app.scss files

For projects not using Laravel Mix, perhaps it would make sense to publish the assets onto a CDN or make them readily available for including in the public/ directory.

[tabacitu]

That's the thing... I kind of agree with ALL of you 😄

Yes like @eduardoarandah & @eronisko mentioned, using NPM and Laravel Mix is the "standard way" of doing things - and it would be easy to adopt it - Backpack already uses Laravel Mix under the hood, it has a mix file and a package.json file.

But, like @OwenMelbz said, we should also take into account that not everybody want or knows how to install and use NPM. One thing I've always tried to keep in Backpack is a very simple stack - I think that greatly improves the developer experience. KISS. And though there's a super-simple analogy between composer and NPM, imho one is great and the other sucks 😆 Add asset compilation and minification to the mix (pun intended) and you've got yourself a whole new thing you need to learn and understand, before you use Backpack.

That's why it's a tough nut to crack 😄

This is something I've wanted to take a stab at for quite a while now. And now that 4.1 is out the door, we can finally do this for the 4.2 release. Here's what I currently think about this, what we can do right now, in 4.1, as non-breaking changes:

• for the actual publishing of assets, we can easily create a php artisan backpack:assets --publish command (detailed here) that would just re-publish the assets from the vendor folder to the public folder, by running php artisan vendor:publish --provider="Backpack\CRUD\BackpackServiceProvider" --tag=public --force; easy-peasy; would solve this problem; whoever wants this, can add it to their composer.json's scripts inside post-update-cmd and it would fix the publishing for them, they'd always have fresh CSS & JS assets after a composer update;
• for those who actually prefer to run their own build process, or want to update assets faster than we do inside the main backpack/crud package, we should be able to create some instructions on how to leverage the package.json and mix file that Backpack already contains, inside the main project; I mean, they're already there, it shouldn't be too difficult to re-use them in the main project; maybe we can even create a command that will actually run npm update and publish newer assets, in one line; something like php artisan backpack:assets --update; this could also be put inside post-update-cmd as far as I'm concerned;

But a more forward-looking solution, inside 4.2:

• for the problem that currently backpack/crud is too large of a vendor dependency, because it includes all those css & js assets, and that it's difficult to push JS & CSS updates without re-publishing all assets, I had this idea https://github.com/Laravel-Backpack/CRUD/issues/2682 which is either brilliant or extremely stupid - but I don't know which one yet 😆 ; I'd appreciate feedback if you have thoughts on it;

I'm eager to see what you think about #313 , and if I've correctly identified the problems - or misunderstood somehow.

Cheers!

[martijnb92]

What about creating a separate ready to use package which includes all dependencies / builds for those who want it, which can be included via composer backpack\dependencies. And for those who want to run their own build create some instructions on how to use the already available package.json and webpack.mix.js files?

[jcastroa87]

I will close this discussion since we have now basset to handle assets on Backpack.

Cheers.