Skip to content

feat: add copyright to evidence collection #1338

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 6 commits into
base: master
Choose a base branch
from
Draft

feat: add copyright to evidence collection #1338

Changes from 1 commit
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
84c258a
feat(Extractor): add copyright to evidence collection
Frozen-byte Dec 3, 2024
ccaaa75
fix(Logger): copy-pasta error
Frozen-byte Dec 3, 2024
4bf5d1a
Merge branch 'remote_master' into with_copyright_evidences
Frozen-byte Apr 24, 2025
7b48862
chore: refactor using new exported CopyrightRepository from core lib
Frozen-byte Apr 24, 2025
44da118
fix(Issue#1310): update test snapshots
Frozen-byte Apr 24, 2025
c7e8507
Merge branch 'master' into with_copyright_evidences
Frozen-byte May 15, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
44 changes: 44 additions & 0 deletions src/extractor.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -111,6 +111,9 @@ export class Extractor {
component.evidence = new CDX.Models.ComponentEvidence({
licenses: new CDX.Models.LicenseRepository(this.getLicenseEvidence(dirname(pkg.path), logger))
})
for (const line of this.getCopyrightEvidence(dirname(pkg.path), logger)) {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd like use the same Method as lin 112, but sadly the Stringable is not exported via CDX.Models and the only way I found to create that Set is with the Component Evidence constructor. Maybe someone else have a more elegant way to iterate.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why would you need the Stringable being exported?

Copy link
Contributor Author

@Frozen-byte Frozen-byte Dec 3, 2024 •
edited by jkowalleck
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Basically I think its more elegant to use something like the following:

      component.evidence = new CDX.Models.ComponentEvidence({
        licenses: new CDX.Models.LicenseRepository(this.getLicenseEvidence(dirname(pkg.path), logger)),
        copyright: new SortableStringables(copyrights)
      })

The SortableStringables is nowhere exported via the CDX object. Importing directly from '_helpers/sortable' will not work since its not exported in any barrel.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see.
If you feel that it is a good idea to export the SortableStringables, then feel free to write an issue for it in the library's repository: https://github.com/CycloneDX/cyclonedx-javascript-library

Copy link
Member

@jkowalleck jkowalleck Dec 19, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about adding and exporting a dedicated type for the purpose new CDX.Models.CopyrightRepository?
The thing is, that the SortableStringables is internal for a good reason. it is just no stable/public interface, yet.
see CycloneDX/cyclonedx-javascript-library#1192

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Frozen-byte with version 7.1.0 of the JS library, the needed model was introduced.
see https://github.com/CycloneDX/cyclonedx-javascript-library/releases/tag/v7.1.0

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this dependency is intended to be bumped to ^7.0 via #1331

please remember to bump to ^7.1 with this PR 👍

Copy link
Contributor Author

@Frozen-byte Frozen-byte Apr 24, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since this PR was idle for a long time, v8 was already used on the main branch.

Updated to the more elegant way, that uses no iteration loop.

component.evidence.copyright.add(line)
}
}

component.purl = this.#purlFactory.makeFromComponent(component)
Expand All @@ -130,6 +133,46 @@ export class Extractor {
}
}

readonly #COPYRIGHT_FILENAME_PATTERN = /^(?:UN)?LICEN[CS]E|.\.LICEN[CS]E$|^NOTICE$|^COPYRIGHTNOTICE$/i

public * getCopyrightEvidence (packageDir: string, logger?: WebpackLogger): Generator<string> {
let pcis
try {
pcis = readdirSync(packageDir, {withFileTypes: true})
} catch (e) {
logger?.warn('collecting license evidence in', packageDir, 'failed:', e)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"license evidence"? Probably a copy/paste error

return
}
for (const pci of pcis) {
if (
!pci.isFile() ||
!this.#COPYRIGHT_FILENAME_PATTERN.test(pci.name)
) {
continue
}
const fp = join(packageDir, pci.name)
try {
// Add copyright evidence
const linesStartingWithCopyright = readFileSync(fp).toString('utf-8')
.split(/\r\n?|\n/)
.map(line => line.trimStart())
.filter(trimmedLine => {
return trimmedLine.startsWith('opyright', 1) && // include copyright statements
!trimmedLine.startsWith('opyright notice', 1) && // exclude lines from license text
!trimmedLine.startsWith('opyright and related rights', 1) &&
!trimmedLine.startsWith('opyright license to reproduce', 1)
})
.filter((value, index, list) => index === 0 || value !== list[0]) // remove duplicates

for (const line of linesStartingWithCopyright) {
yield line
}
} catch (e) { // may throw if `readFileSync()` fails
logger?.warn('collecting copyright evidences from', fp, 'failed:', e)
}
}
}

readonly #LICENSE_FILENAME_PATTERN = /^(?:UN)?LICEN[CS]E|.\.LICEN[CS]E$|^NOTICE$/i

public * getLicenseEvidence (packageDir: string, logger?: WebpackLogger): Generator<CDX.Models.License> {
Expand All @@ -150,6 +193,7 @@ export class Extractor {

const contentType = getMimeForTextFile(pci.name)
if (contentType === undefined) {
logger?.warn(`could not determine content-type for ${pci.name}`)
continue
}

Expand Down