feat: add multi-language support with matrix scanning examples#39
feat: add multi-language support with matrix scanning examples#39
Conversation
- Add language, build_mode, and build_command inputs for multi-language repos - Include usage examples for both single-language and matrix scanning - Document java-kotlin support with manual build configuration
There was a problem hiding this comment.
In this approach, we will be required to make a change in each repository's config to populate the languages matrix which will take a significant amount of effort.
While we have control over the MetaMask/action-security-code-scanner@v1 from a central location, as of today, the rest of the inputs, and job level maintenance continues to lie within the repo as seen with this . This makes updates like this challenging to make.
If we are going to be making repository changes, we should try to do so in a way that abstracts further job configuration that we wish to do in the future. This can be done by creating the code scanner as a reusable workflow which then contains the configuration required to spawn additional jobs
To illustrate this, here is a diagram roughly showing the setup we have today:
Here you can see each repo is responsible for defining their YML, configuring the jobs to run, and then invoking our composite action. We as a team, only have centralized control over a single job worker.
By adjusting the workflow to be a reusable workflow, we can instead only rely on the repo level configuration file to pass us inputs. The rest we would have control over behind the scenes on our end. This can be seen in the image below:
--
We can get into this state by instead having the matrix definition be in our reusable worklow defined in this repo, and have the language specific build commands passed in as inputs. Then we can simply update each GitHub repo's config file to point to our reusable workflow with those optional inputs. This will then allow us to make changes in the future without restriction, and only need to update repo specific configs when we need to bump the version of the workflow, or if we need to modify inputs
| matrix: | ||
| include: | ||
| - language: javascript-typescript | ||
| - language: typescript | ||
| - language: go | ||
| - language: java-kotlin | ||
| build-mode: manual | ||
| build-command: './gradlew :coordinator:app:build' |
There was a problem hiding this comment.
As a matrix like this is only needed for a very select few languages, can we only require this for those languages and auto detect the rest?
For an example, if we know Kotlin / C# are the only two languages that require build command (and make up a very small portion of our repos), rather than requiring all other repos who don't use these languages to be explicit about their config, it would be better for those edge cases to be capable of providing this information. That way users of go, typescript, python etc. don't require a config, and our scanner can automatically adapt as the codebase does.
| build-mode: manual | ||
| build-command: './gradlew :coordinator:app:build' | ||
|
|
||
| name: Security Code Scan (${{ matrix.language }}) |
There was a problem hiding this comment.
Having the whole MetaMask/action-security-code-scanner run multiple times will mean that semgrep scans will also upload multiple times. Do we know if GitHub's API is smart enough to know these are duplicates? Ideally, we only need to run individual jobs for certain Code Scanning Engines (e.g CodeQL) as its not necessary to do so for things like semgrep which scan all languages at once.
|
closing in favour of the updated v2 aproach |
Summary
language,build_mode, andbuild_commandinputsChanges
New Inputs
language: Supportsjavascript-typescript,typescript,go, andjava-kotlinbuild_mode: Set tomanualfor compiled languages requiring explicit build commandsbuild_command: Required build command when using manual build mode (e.g.,./gradlew build)Documentation Updates
Examples
examples/security-code-scanner.ymlfor single-language usageexamples/security-code-scanner-matrix.ymldemonstrating matrix strategy for multi-language reposTest Plan
Notes
https://github.com/witmicko/lll/actions/workflows/security-code-scanner-matrix.yml
is my unlinked Linea-monorepo fork with changed name, you can run the scans in this repo as a test.