Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
96 changes: 90 additions & 6 deletions src/buildstream/source.py
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Another thing that I feel would be useful to have is the external reference: https://spdx.github.io/spdx-spec/v2.3/package-information/#721-external-reference-field

Original file line number Diff line number Diff line change
Expand Up @@ -553,8 +553,16 @@ def __init__(
self,
kind: str,
url: str,
concluded_license: Optional[str],
copyright_text: Optional[str],
declared_license: Optional[str],
description: Optional[str],
external_reference: Optional[str],
homepage: Optional[str],
issue_tracker: Optional[str],
name: Optional[str],
originator: Optional[str],
supplier: Optional[str],
medium: Union[SourceInfoMedium, str],
version_type: Union[SourceVersionType, str],
version: str,
Expand All @@ -572,14 +580,54 @@ def __init__(
The url of the source input
"""

self.concluded_license: Optional[str] = concluded_license
"""
The license of the source project as declared by the authors
"""

self.copyright_text: Optional[str] = copyright_text
"""
Copyright notice of the source
"""

self.declared_license: Optional[str] = declared_license
"""
Licences that have been officially declared for the source
"""

self.description: Optional[str] = description
"""
Description of the source
"""

self.external_reference: Optional[str] = external_reference
"""
Reference to an external source of information or assets relevant to the source
"""

self.homepage: Optional[str] = homepage
"""
The project homepage URL
The source's homepage URL
"""

self.issue_tracker: Optional[str] = issue_tracker
"""
The project issue tracking URL
The source's issue tracking URL
"""

self.name: Optional[str] = name
"""
Name of the source
"""
Comment on lines +618 to +621
Copy link
Contributor

@doraskayo doraskayo Jan 6, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This might be a more general comment, but the name field appears to be one of the only mandatory package fields in SPDX 2.3. As such, at the very least, I would expect it to be:

  1. Provided by all core source plugins that can usually reproduce it reliably (maybe cargo and pip?), instead of letting consumers guess it based on the url field.
  2. Made user-configurable in all the core plugins that cannot reliably reproduce it on their own in every single case, such as tar and other plugins that inherit from DownloadableFileSource. This would be similar to the version and version-guess-pattern configurations.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Discard the above, I had missed the fact that the provenance configuration was also added in #2095 to all source plugins. It appears that this was not documented. This is where I would have expected it to find it: https://docs.buildstream.build/2.6/format_declaring.html#sources

The fact that the provenance fields are all user-configurable mostly covers my concerns. I still think we should aim to populate the name field from every core source plugin that can provide it reliably, but the fact that it's user-configurable means there's always the option of specifying it manually.

I think it's worth to have this documented properly, however. I'll add a separate comment about it.


self.originator: Optional[str] = originator
"""
The name of the source's originators/owners
"""

self.supplier: Optional[str] = supplier
"""
The name of the source's distributor
"""

self.medium: Union[SourceInfoMedium, str] = medium
Expand Down Expand Up @@ -642,10 +690,22 @@ def serialize(self) -> Dict[str, Union[str, Dict[str, str]]]:
"url": self.url,
}

if self.homepage is not None:
version_info["homepage"] = self.homepage
if self.issue_tracker is not None:
version_info["issue-tracker"] = self.issue_tracker
source_info_extra_fields = [
"concluded-license",
"copyright-text",
"declared-license",
"description",
"external-reference",
"homepage",
"issue-tracker",
"name",
"originator",
"supplier",
]
Comment on lines +693 to +704
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we get the user-configurable provenance attribute documented here, and also exactly which fields we allow specifying in it including those introduced by this PR? https://github.com/apache/buildstream/blob/86a950aba52ef6aa2cb868bf13fb24e20de37679/doc/source/format_declaring.rst#sources

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is all going to be implemented in a different way now that removes the requirement for the buildstream core to have to know about all this SBOM stuff that is irrelevant to it. Instead buildstream-sbom will be used for storing this as a source of truth and allowing for the buildstream core to not get cluttered up with this.

I am currently working on a new set of patches to implement this :)


for field in source_info_extra_fields:
if (value := getattr(self, field.replace("-", "_"))) is not None:
version_info[field] = value

version_info["medium"] = medium_str
version_info["version-type"] = version_type_str
Expand Down Expand Up @@ -1390,17 +1450,41 @@ def create_source_info(

*Since: 2.5*
"""
concluded_license = None
copyright_text = None
declared_license = None
description = None
external_reference = None
homepage = None
issue_tracker = None
name = None
originator = None
supplier = None
if self.__provenance is not None:
concluded_license = self.__provenance.concluded_license
copyright_text = self.__provenance.copyright_text
declared_license = self.__provenance.declared_license
description = self.__provenance.description
external_reference = self.__provenance.external_reference
homepage = self.__provenance.homepage
issue_tracker = self.__provenance.issue_tracker
name = self.__provenance.name
originator = self.__provenance.originator
supplier = self.__provenance.supplier

return SourceInfo(
self.get_kind(),
url,
concluded_license,
copyright_text,
declared_license,
description,
external_reference,
homepage,
issue_tracker,
name,
originator,
supplier,
medium,
version_type,
version,
Expand Down
76 changes: 68 additions & 8 deletions src/buildstream/types.py
Original file line number Diff line number Diff line change
Expand Up @@ -395,13 +395,41 @@ def new_from_node(cls, node: MappingNode) -> "_SourceMirror":
# A simple object describing user provided source provenance information
#
# Args:
# homepage: The project homepage URL
# issue_tracker: The project issue reporting URL
#
# concluded_license: The license of the source as declared by the authors
# copyright_text: Copyright notice of the source
# declared_license: Licences that have been officially declared for the source
# description: Description of the source
# external_reference: Reference to external inforamtion or assets relevant to the source
# homepage: The source's homepage URL
# issue_tracker: The source's issue reporting URL
# name: Name of the source
# originator: The name of the source's originators/authors
# supplier: The name of the source's distributor
#
class _SourceProvenance:
def __init__(self, homepage: Optional[str], issue_tracker: Optional[str]):
def __init__(
self,
concluded_license: Optional[str],
copyright_text: Optional[str],
declared_license: Optional[str],
description: Optional[str],
external_reference: Optional[str],
homepage: Optional[str],
issue_tracker: Optional[str],
name: Optional[str],
originator: Optional[str],
supplier: Optional[str],
):
self.concluded_license: Optional[str] = concluded_license
self.copyright_text: Optional[str] = copyright_text
self.declared_license: Optional[str] = declared_license
self.description: Optional[str] = description
self.external_reference: Optional[str] = external_reference
self.homepage: Optional[str] = homepage
self.issue_tracker: Optional[str] = issue_tracker
self.name: Optional[str] = name
self.originator: Optional[str] = originator
self.supplier: Optional[str] = supplier

# new_from_node():
#
Expand All @@ -418,12 +446,44 @@ def __init__(self, homepage: Optional[str], issue_tracker: Optional[str]):
#
@classmethod
def new_from_node(cls, node: MappingNode) -> "_SourceProvenance":
node.validate_keys(["homepage", "issue-tracker"])

node.validate_keys(
[
"concluded-license",
"copyright-text",
"declared-license",
"external-reference",
"description",
"homepage",
"issue-tracker",
"name",
"originator",
"supplier",
]
)

concluded_license: Optional[str] = node.get_str("concluded-license", None)
copyright_text: Optional[str] = node.get_str("copyright-text", None)
declared_license: Optional[str] = node.get_str("declared-license", None)
description: Optional[str] = node.get_str("description", None)
external_reference: Optional[str] = node.get_str("external-reference", None)
homepage: Optional[str] = node.get_str("homepage", None)
issue_tracker: Optional[str] = node.get_str("issue-tracker", None)

return cls(homepage, issue_tracker)
name: Optional[str] = node.get_str("name", None)
originator: Optional[str] = node.get_str("originator", None)
supplier: Optional[str] = node.get_str("supplier", None)

return cls(
concluded_license,
copyright_text,
declared_license,
description,
external_reference,
homepage,
issue_tracker,
name,
originator,
supplier,
)


########################################
Expand Down
58 changes: 57 additions & 1 deletion tests/frontend/show.py
Original file line number Diff line number Diff line change
Expand Up @@ -582,7 +582,7 @@ def test_invalid_alias(cli, tmpdir, datafiles):

@pytest.mark.datafiles(os.path.join(DATA_DIR, "source-info"))
@pytest.mark.parametrize(
"target, expected_kind, expected_url, expected_medium, expected_version_type, expected_version, expected_guess_version, expected_homepage, expected_issue_tracker",
"target, expected_kind, expected_url, expected_medium, expected_version_type, expected_version, expected_guess_version, expected_attribution_text, expected_concluded_license, expected_copyright_text, expected_declared_license, expected_description, expected_external_reference, expected_homepage, expected_issue_tracker, expected_name, expected_originator",
[
(
"local.bst",
Expand All @@ -594,6 +594,13 @@ def test_invalid_alias(cli, tmpdir, datafiles):
None,
None,
None,
None,
None,
None,
None,
None,
None,
None,
),
(
"tar.bst",
Expand All @@ -605,6 +612,13 @@ def test_invalid_alias(cli, tmpdir, datafiles):
"1.2.3",
None,
None,
None,
None,
None,
None,
None,
None,
None,
),
(
"tar-no-micro.bst",
Expand All @@ -616,6 +630,13 @@ def test_invalid_alias(cli, tmpdir, datafiles):
"1.2",
None,
None,
None,
None,
None,
None,
None,
None,
None,
),
(
"tar-custom-version.bst",
Expand All @@ -627,6 +648,13 @@ def test_invalid_alias(cli, tmpdir, datafiles):
"2.4.93",
None,
None,
None,
None,
None,
None,
None,
None,
None,
),
(
"tar-explicit.bst",
Expand All @@ -638,6 +666,13 @@ def test_invalid_alias(cli, tmpdir, datafiles):
"3.2.1",
None,
None,
None,
None,
None,
None,
None,
None,
None,
),
(
"testsource.bst",
Expand All @@ -649,6 +684,13 @@ def test_invalid_alias(cli, tmpdir, datafiles):
"12",
None,
None,
None,
None,
None,
None,
None,
None,
None,
),
(
"user-provenance.bst",
Expand All @@ -658,8 +700,15 @@ def test_invalid_alias(cli, tmpdir, datafiles):
"sha256",
"9d0c936c78d0dfe3a67cae372c9a2330476ea87a2eec16b2daada64a664ca501",
"1.2.3",
"We ackowledge that flying ponies are very real",
"flying-ponies-V2",
"Please do not steal our flying ponies",
"flying-ponies-V2",
"Flying ponies, what more do you want?",
"https://flying-ponies.com/index.html",
"https://bugs.flying-ponies.com/issues",
"flying-ponies",
"FLYING PONIES",
),
],
ids=[
Expand All @@ -682,8 +731,15 @@ def test_source_info(
expected_version_type,
expected_version,
expected_guess_version,
expected_attribution_text,
expected_concluded_license,
expected_copyright_text,
expected_declared_license,
expected_description,
expected_homepage,
expected_issue_tracker,
expected_name,
expected_originator,
):
project = str(datafiles)
result = cli.run(project=project, silent=True, args=["show", "--format", "%{name}:\n%{source-info}", target])
Expand Down
8 changes: 8 additions & 0 deletions tests/frontend/source-info/elements/user-provenance.bst
Original file line number Diff line number Diff line change
Expand Up @@ -5,5 +5,13 @@ sources:
url: https://flying-ponies.com/releases/1.2/pony-flight-1.2.3.tgz
ref: 9d0c936c78d0dfe3a67cae372c9a2330476ea87a2eec16b2daada64a664ca501
provenance:
attribution-text: We ackowledge that flying ponies are very real
concluded-license: flying-ponies-V2
copyright-text: Please do not steal our flying ponies
declared-license: flying-ponies-V2
description: Flying ponies, what more do you want?
external-reference: OTHER flying.ponies ponies.can.fly:believe_it
homepage: https://flying-ponies.com/index.html
issue-tracker: https://bugs.flying-ponies.com/issues
name: flying-ponies
originator: Flying Pony Equestrian Centre