Licensing issues with big amount of production dependencies

[Purii]

Summary

It's quite hard to find out which Dependencies are actually bundled and shipped with a Gatsby App.

I have the issue, that some (nested) dependencies of gatsby-cli seem to be published under licenses that might be critical for a commercial app (Copyleft).

I'm aware that Gatsby is a generator. But I'm wondering if it's possible to differ between some kind of "Gatsby Core" (shipped as part of a bundled App) and gatsby-cli. Both published as separate packages. That would allow to list gatsby-cli as devDependency. The current gatsby package could still bundle "Gatsby Core" and gatsby-cli to prevent breaking anything.

[servrox]

Can we please have an update on this issue?

For most projects, we list gatsby-cli as devDependency anyway..
Creating a gatsby-core package definitely would make sense!

[wardpeet]

it's going to be tricky as Gatsby is both a development server and build server so what would the package gatsby-core have? Would it only have the files that are used in production builds?

Would it help to give you a list of what we have today in production builds? A good indicator is using source-map-explorer and the hello world starter

[Purii]

Hi @wardpeet!

Yes, you are right. Exactly these two parts (development server and build server) actually belong together.
But gatsby-link is neither needed by the development server, nor by the build server, but part of the production bundle (output from development server and build server).

gatsby-link and gatsby-cli are not on the same level / type, but both are listed as dependencies of gatsby.
To check for license compliance one need to know exactly which dependencies are eventually part of a production bundle and shipped to the customers.

I see three types of dependencies in the gatsby package:

• Development Dependencies (currently devDeps) to develop gatsby itself further, like eslint,…
• „Build“ Dependencies (currently deps) needed to develop a gatsby site and create a production bundle, like autoprefixer, gatsby-cli, @typescript/, dotenv,…
• Production Dependencies (currently deps) bundled as part of a gatsby app (which is created using the „build dependencies“), like gatsby-link, @reach/router,…

To separate these „Build Dependencies“ and „Production Dependencies“ I suggest to introduce a new package. To stay compatible the „Build dependencies“ can simply depend on the „Production dependencies“.
Unfortunately I don’t know the internals, which dependencies the gatsby build process injects into the production bundle (like core-js).

-> QUESTION: Can you point me to the specific files? Then I can play around and possibly suggest a concrete solution.

Random Examples of Dependencies

• gatsby > gatsby-cli > gatsby-recipes > react-icons: Comes with a lot of different licenses, because of the included icons
• gatsby > gatsby-cli > gatsby-recipes > urql > wonka: Contains references to GPL licenses (/esy.lock/**)
• gatsby > webpack-dev-server > selfsigned > node-forge: Comes with a lot of different licenses (/LICENSE), like GPL

There are similar issues with some plugins, like gatsby-plugin-sharp: > imagemin-pngquant > pngquant: GPL.
I guess for these issues I have to check each plugin individually and move that to the devDeps, if nothing gets bundled.

Sourcecode-map-explorer

Sourcecode-map-explorer is a great idea, but I guess by scanning the bundle we cannot be 100% sure to identify all bundled dependencies.
Simply because they are bundled.
The normal way to scan licenses is to traverse though the lock files (yarn.lock or package-json.lock) and then scan every single line of the dependency (deep scan). Unfortunately just scanning the license, which is declared in the package.json is not enough. Sometimes a dependency is dual licensed, or the declared package license is different from a license comment in the header of a single file deep in the package.

[kitten]

@Purii At least for the wonka package, which I'm maintaining, I can tell you that the files in esy.lock are not relating to licences of the library itself. In fact, outside of native builds with OPAM/Esy they aren't even installed, so they're just there to indicate which packages would be installed for esy as development-only tools.

I'm aware that some tools erroneously detect these licences and since publishing these lockfiles isn't necessary anymore, I've removed them in the latest patch release, which you can upgrade to. For comparison:

• [email protected]/esy.lock/
• [email protected]/esy.lock/

On a side note, IANAL, but using tools like pngquant to compress images in development, does not have any negative effect on your own licensing. However, react-icons here is likely more questionable? It's likely hard to tell whether it ever in any form ends up in the bundle when a recipe is used.

[Purii]

@kitten Thanks for that note and really awesome to patch your package!

Some of the scanners ("most accurate ones") scan the whole repository instead of only the files published to npm. I'm about to dig deeper on why they're doing that. But scanning the whole source code instead of only package.json or LICENSE file makes total sense to me (see my comment above).

Point is with libs like pngquant that we would need to check every single dependency on how it is used. Yes we can flag these. But in order to make it correctly we need to recheck that on every dependency update. Thats a lot of overhead.
As far as I understand the recipes package, react-icons are used by the CLI/Tool itself. Packages installed using recipes would bring their own reference to react-icons.

[Purii]

Any updates @wardpeet ?

[Purii]

Thanks for cleaning up a bit @wardpeet and @kitten ! 🚀 77143e8