Skip to content

Add meta.teams support #914

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
May 6, 2025
Merged

Add meta.teams support #914

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
f62fe36
Add meta.teams support
RossComputerGuy Apr 21, 2025
7438a23
meta.teams support: fix export
raboof May 5, 2025
d8c1bc9
meta.teams support: show team name
raboof May 5, 2025
b9a8d8d
Merge pull request #2 from raboof/feat/meta-teams-fix-show-team-name
RossComputerGuy May 6, 2025
790d024
Merge pull request #1 from raboof/feat/meta-teams-fix-export
RossComputerGuy May 6, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
53 changes: 53 additions & 0 deletions flake-info/src/data/export.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -68,6 +68,8 @@ pub enum Derivation {
package_license_set: Vec<String>,
package_maintainers: Vec<Maintainer>,
package_maintainers_set: Vec<String>,
package_teams: Vec<Team>,
package_teams_set: Vec<String>,
package_description: Option<String>,
package_longDescription: Option<String>,
package_hydra: (),
Expand Down Expand Up @@ -157,6 +159,8 @@ impl TryFrom<(import::FlakeEntry, super::Flake)> for Derivation {
package_description: description.clone(),
package_maintainers: vec![maintainer.clone()],
package_maintainers_set: maintainer.name.map_or(vec![], |n| vec![n]),
package_teams: Vec::new(),
package_teams_set: Vec::new(),
package_longDescription: long_description,
package_hydra: (),
package_system: String::new(),
Expand Down Expand Up @@ -233,6 +237,19 @@ impl TryFrom<import::NixpkgsEntry> for Derivation {
.flat_map(|m| m.name.to_owned())
.collect();

let package_teams: Vec<Team> = package
.meta
.teams
.map_or(Default::default(), Flatten::flatten)
.into_iter()
.map(Into::into)
.collect();

let package_teams_set = package_teams
.iter()
.flat_map(|m| m.shortName.to_owned())
.collect();

let long_description = package
.meta
.long_description
Expand Down Expand Up @@ -260,6 +277,8 @@ impl TryFrom<import::NixpkgsEntry> for Derivation {
package_license_set,
package_maintainers,
package_maintainers_set,
package_teams,
package_teams_set,
package_description: package.meta.description.clone(),
package_longDescription: long_description,
package_hydra: (),
Expand Down Expand Up @@ -348,6 +367,40 @@ impl From<super::Flake> for Maintainer {
}
}

#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
#[allow(non_snake_case)]
pub struct Team {
members: Option<OneOrMany<Maintainer>>,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not super familiar with this codebase yet, but I wonder if it wouldn't be simpler for this to be a Vec (or Option<Vec>) here?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would be much simpler, yes. Specifically Option<OneOrMany<T>> sounds like a complicated way to specify Vec<T>. Perhaps it has benefits when (de-)serializing.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Option<OneOrMany<T>> is done in many other places so I used it because of that

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I had the impression that it is used in the contract between the 'nix side' and 'flake-info', but not between 'flake-info' and elasticsearch model - and that this struct is used for the latter. As said I'm not super familiar with this codebase yet, though, so I could be mistaken.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I'm not sure either. I think it does everything lol.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is import.rs the 'nix side' and export.rs the 'elasticsearch side'?

my guess was that OneOrMany would support both "foo" and [ "foo" "bar" ] due to nix being flexibly typed, but we can be a bit more strict on our 'own' side.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have no idea lol.

scope: Option<String>,
shortName: Option<String>,
githubTeams: Option<OneOrMany<String>>,
}

impl From<import::Team> for Team {
fn from(import: import::Team) -> Self {
match import {
import::Team::Full {
members,
scope,
shortName,
githubTeams,
} => Team {
members: members.map(|m| m.map(Maintainer::from)),
scope,
shortName,
githubTeams,
},
#[allow(non_snake_case)]
import::Team::Simple(shortName) => Team {
shortName: Some(shortName),
scope: None,
members: None,
githubTeams: None,
},
}
}
}

// ----- output type

/// Export type that brings together derivation and optional flake info
Expand Down
14 changes: 14 additions & 0 deletions flake-info/src/data/import.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -186,6 +186,7 @@ pub enum NixpkgsEntry {
pub struct Meta {
pub license: Option<OneOrMany<StringOrStruct<License>>>,
pub maintainers: Option<Flatten<Maintainer>>,
pub teams: Option<Flatten<Team>>,
pub homepage: Option<OneOrMany<String>>,
pub platforms: Option<Platforms>,
#[serde(rename = "badPlatforms")]
Expand All @@ -207,6 +208,19 @@ pub enum Maintainer {
Simple(String),
}

#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
#[serde(untagged)]
pub enum Team {
#[allow(non_snake_case)]
Full {
members: Option<OneOrMany<Maintainer>>,
scope: Option<String>,
shortName: Option<String>,
githubTeams: Option<OneOrMany<String>>,
},
Simple(String),
}

arg_enum! {
/// The type of derivation (placed in packages.<system> or apps.<system>)
/// Used to command the extraction script
Expand Down
10 changes: 10 additions & 0 deletions flake-info/src/data/utility.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,16 @@ impl<T> OneOrMany<T> {
OneOrMany::Many(many) => many,
}
}

pub fn map<U, F>(self, f: F) -> OneOrMany<U>
where
F: Fn(T) -> U,
{
match self {
Self::One(x) => OneOrMany::One(f(x)),
Self::Many(xs) => OneOrMany::Many(xs.into_iter().map(f).collect()),
}
}
}

/// A utility type that flattens lists of lists as seen with `maintainers` and `platforms` on selected packages
Expand Down
8 changes: 8 additions & 0 deletions flake-info/src/elastic.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -126,6 +126,14 @@ lazy_static! {
},
},
"package_maintainers_set": {"type": "keyword"},
"package_teams": {
"type": "nested",
"properties": {
"members": {"type": "keyword"},
"githubTeams": {"type": "keyword"},
},
},
"package_teams_set": {"type": "keyword"},
"package_homepage": {
"type": "keyword"
},
Expand Down
65 changes: 60 additions & 5 deletions frontend/src/Page/Packages.elm
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -73,6 +73,7 @@ type alias ResultItemSource =
, longDescription : Maybe String
, licenses : List ResultPackageLicense
, maintainers : List ResultPackageMaintainer
, teams: List ResultPackageTeam
, platforms : List String
, position : Maybe String
, homepage : List String
Expand All @@ -97,6 +98,14 @@ type alias ResultPackageMaintainer =
}


type alias ResultPackageTeam =
{ members : Maybe (List ResultPackageMaintainer)
, scope : Maybe String
, shortName: Maybe String
, githubTeams : Maybe (List String)
}


type alias ResultPackageHydra =
{ build_id : Int
, build_status : Int
Expand All @@ -120,6 +129,7 @@ type alias ResultAggregations =
, package_platforms : Search.Aggregation
, package_attr_set : Search.Aggregation
, package_maintainers_set : Search.Aggregation
, package_teams_set : Search.Aggregation
, package_license_set : Search.Aggregation
}

Expand All @@ -129,6 +139,7 @@ type alias Aggregations =
, package_platforms : Search.Aggregation
, package_attr_set : Search.Aggregation
, package_maintainers_set : Search.Aggregation
, package_teams_set : Search.Aggregation
, package_license_set : Search.Aggregation
}

Expand All @@ -137,6 +148,7 @@ type alias Buckets =
{ packageSets : List String
, licenses : List String
, maintainers : List String
, teams : List String
, platforms : List String
}

Expand All @@ -146,6 +158,7 @@ emptyBuckets =
{ packageSets = []
, licenses = []
, maintainers = []
, teams = []
, platforms = []
}

Expand Down Expand Up @@ -278,6 +291,11 @@ viewBuckets bucketsAsString result =
(result.aggregations.package_maintainers_set.buckets |> sortBuckets)
(createBucketsMsg .maintainers (\s v -> { s | maintainers = v }))
selectedBucket.maintainers
|> viewBucket
"Teams"
(result.aggregations.package_teams_set.buckets |> sortBuckets)
(createBucketsMsg .teams (\s v -> { s | teams = v }))
selectedBucket.teams
|> viewBucket
"Platforms"
(result.aggregations.package_platforms.buckets |> sortBuckets |> filterPlatformsBucket)
Expand Down Expand Up @@ -462,6 +480,17 @@ viewResultItem nixosChannels channel showInstallDetails show item =
)
]

showTeam team =
let
maybe m d =
Maybe.withDefault d m

showTeamEntry githubTeam =
(a [ href ((String.append "https://github.com/orgs/NixOS/teams/") githubTeam) ] [ text githubTeam ])
in
li []
(List.map showTeamEntry (maybe team.githubTeams []))

mailtoAllMaintainers maintainers =
let
maintainerMails =
Expand All @@ -487,7 +516,7 @@ viewResultItem nixosChannels channel showInstallDetails show item =
Nothing ->
li [] [ text platform ]

maintainersAndPlatforms =
maintainersTeamsAndPlatforms =
div []
[ div []
(List.append [ h4 [] [ text "Maintainers" ] ]
Expand All @@ -503,6 +532,16 @@ viewResultItem nixosChannels channel showInstallDetails show item =
]
)
)
, div []
(if not (List.isEmpty item.source.teams) then
(List.append [ h4 [] [ text "Teams" ] ]
[ ul []
(List.map showTeam item.source.teams)
]
)
else
[]
)
, div []
(List.append [ h4 [] [ text "Platforms" ] ]
(if List.isEmpty item.source.platforms then
Expand Down Expand Up @@ -748,7 +787,7 @@ viewResultItem nixosChannels channel showInstallDetails show item =
Maybe.map Tuple.first item.source.flakeUrl
]
:: programs
:: maintainersAndPlatforms
:: maintainersTeamsAndPlatforms
:: []
)
]
Expand Down Expand Up @@ -878,6 +917,7 @@ makeRequestBody query from size maybeBuckets sort =
[ ( "package_attr_set", currentBuckets.packageSets )
, ( "package_license_set", currentBuckets.licenses )
, ( "package_maintainers_set", currentBuckets.maintainers )
, ( "package_teams_set", currentBuckets.teams )
, ( "package_platforms", currentBuckets.platforms )
]

Expand Down Expand Up @@ -931,6 +971,7 @@ makeRequestBody query from size maybeBuckets sort =
[ "package_attr_set"
, "package_license_set"
, "package_maintainers_set"
, "package_teams_set"
, "package_platforms"
]
filterByBuckets
Expand All @@ -954,16 +995,18 @@ encodeBuckets options =
[ ( "package_attr_set", Json.Encode.list Json.Encode.string options.packageSets )
, ( "package_license_set", Json.Encode.list Json.Encode.string options.licenses )
, ( "package_maintainers_set", Json.Encode.list Json.Encode.string options.maintainers )
, ( "package_teams_set", Json.Encode.list Json.Encode.string options.teams )
, ( "package_platforms", Json.Encode.list Json.Encode.string options.platforms )
]


decodeBuckets : Json.Decode.Decoder Buckets
decodeBuckets =
Json.Decode.map4 Buckets
Json.Decode.map5 Buckets
(Json.Decode.field "package_attr_set" (Json.Decode.list Json.Decode.string))
(Json.Decode.field "package_license_set" (Json.Decode.list Json.Decode.string))
(Json.Decode.field "package_maintainers_set" (Json.Decode.list Json.Decode.string))
(Json.Decode.field "package_teams_set" (Json.Decode.list Json.Decode.string))
(Json.Decode.field "package_platforms" (Json.Decode.list Json.Decode.string))


Expand All @@ -980,6 +1023,7 @@ decodeResultItemSource =
|> Json.Decode.Pipeline.required "package_longDescription" (Json.Decode.nullable Json.Decode.string)
|> Json.Decode.Pipeline.required "package_license" (Json.Decode.list decodeResultPackageLicense)
|> Json.Decode.Pipeline.required "package_maintainers" (Json.Decode.list decodeResultPackageMaintainer)
|> Json.Decode.Pipeline.required "package_teams" (Json.Decode.list decodeResultPackageTeam)
|> Json.Decode.Pipeline.required "package_platforms" (Json.Decode.map filterPlatforms (Json.Decode.list Json.Decode.string))
|> Json.Decode.Pipeline.required "package_position" (Json.Decode.nullable Json.Decode.string)
|> Json.Decode.Pipeline.required "package_homepage" decodeHomepage
Expand Down Expand Up @@ -1089,6 +1133,15 @@ decodeResultPackageMaintainer =
(Json.Decode.field "github" (Json.Decode.nullable Json.Decode.string))


decodeResultPackageTeam : Json.Decode.Decoder ResultPackageTeam
decodeResultPackageTeam =
Json.Decode.map4 ResultPackageTeam
(Json.Decode.field "members" (Json.Decode.nullable (Json.Decode.list decodeResultPackageMaintainer)))
(Json.Decode.field "scope" (Json.Decode.nullable Json.Decode.string))
(Json.Decode.field "shortName" (Json.Decode.nullable Json.Decode.string))
(Json.Decode.field "githubTeams" (Json.Decode.nullable (Json.Decode.list Json.Decode.string)))


decodeResultPackageHydra : Json.Decode.Decoder ResultPackageHydra
decodeResultPackageHydra =
Json.Decode.succeed ResultPackageHydra
Expand All @@ -1111,19 +1164,21 @@ decodeResultPackageHydraPath =

decodeResultAggregations : Json.Decode.Decoder ResultAggregations
decodeResultAggregations =
Json.Decode.map5 ResultAggregations
Json.Decode.map6 ResultAggregations
(Json.Decode.field "all" decodeAggregations)
(Json.Decode.field "package_platforms" Search.decodeAggregation)
(Json.Decode.field "package_attr_set" Search.decodeAggregation)
(Json.Decode.field "package_maintainers_set" Search.decodeAggregation)
(Json.Decode.field "package_teams_set" Search.decodeAggregation)
(Json.Decode.field "package_license_set" Search.decodeAggregation)


decodeAggregations : Json.Decode.Decoder Aggregations
decodeAggregations =
Json.Decode.map5 Aggregations
Json.Decode.map6 Aggregations
(Json.Decode.field "doc_count" Json.Decode.int)
(Json.Decode.field "package_platforms" Search.decodeAggregation)
(Json.Decode.field "package_attr_set" Search.decodeAggregation)
(Json.Decode.field "package_maintainers_set" Search.decodeAggregation)
(Json.Decode.field "package_teams_set" Search.decodeAggregation)
(Json.Decode.field "package_license_set" Search.decodeAggregation)