Skip to content

Commit 82c57ac

Browse files
gaspergromgaspergrom
authored
feat: project insights (IN-1009) (#3903)
Signed-off-by: Gašper Grom <gasper.grom@gmail.com>
1 parent 2993353 commit 82c57ac

File tree

3 files changed

+102
-7
lines changed

3 files changed

+102
-7
lines changed

‎services/libs/tinybird/datasources/project_insights_copy_ds.datasource‎

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5,6 +5,16 @@ DESCRIPTION >
55
- `id` column is the primary key identifier for the project.
66
- `name` column is the human-readable project name.
77
- `slug` column is the URL-friendly identifier used in routing and filtering.
8+
- `logoUrl` column is the URL to the project's logo image.
9+
- `isLF` column indicates whether the project is a Linux Foundation project (1 = true, 0 = false).
10+
- `contributorCount` column is the total number of contributors for the project.
11+
- `organizationCount` column is the total number of organizations for the project.
12+
- `softwareValue` column is the estimated economic value of the software.
13+
- `contributorDependencyCount` column is the number of contributors making up 51% of contributions (bus factor).
14+
- `contributorDependencyPercentage` column is the combined contribution percentage of dependent contributors.
15+
- `organizationDependencyCount` column is the number of organizations making up 51% of contributions.
16+
- `organizationDependencyPercentage` column is the combined contribution percentage of dependent organizations.
17+
- `achievements` column is an array of tuples (leaderboardType, rank, totalCount) representing project achievements.
818
- `healthScore` column is the overall health score for the project (0-100).
919
- `firstCommit` column is the timestamp of the first commit in the project.
1020
- `starsLast365Days` column is the count of stars in the last 365 days.
@@ -22,7 +32,16 @@ SCHEMA >
2232
`id` String,
2333
`name` String,
2434
`slug` String,
35+
`logoUrl` String,
36+
`isLF` UInt8,
37+
`contributorCount` UInt64,
38+
`organizationCount` UInt64,
2539
`softwareValue` UInt64,
40+
`contributorDependencyCount` UInt64,
41+
`contributorDependencyPercentage` Float64,
42+
`organizationDependencyCount` UInt64,
43+
`organizationDependencyPercentage` Float64,
44+
`achievements` Array(Tuple(String, UInt64, UInt64)),
2645
`healthScore` Nullable(Float64),
2746
`firstCommit` Nullable(DateTime64(3)),
2847
`starsLast365Days` UInt64,

‎services/libs/tinybird/pipes/project_insights.pipe‎

Lines changed: 24 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,9 +1,12 @@
11
DESCRIPTION >
22
- `project_insights.pipe` serves project insights data for a specific project.
3-
- Returns comprehensive metrics including health score, first commit, and activity metrics for both current and previous 365-day periods.
3+
- Returns comprehensive metrics including project metadata (name, logoUrl, isLF), health score, contributor count, software value, contributor and organization dependency metrics, leaderboard achievements, and activity metrics for both current and previous 365-day periods.
44
- Parameters:
5-
- `slug`: Required string for project slug (e.g., 'kubernetes', 'tensorflow')
6-
- Response: Single project record with all insights metrics
5+
- `slug`: Optional string for a single project slug (e.g., 'kubernetes')
6+
- `slugs`: Optional array of project slugs for multi-project query (e.g., ['kubernetes', 'tensorflow'])
7+
- `ids`: Optional array of project ids for multi-project query
8+
- At least one of `slug`, `slugs`, or `ids` should be provided.
9+
- Response: Project records with all insights metrics including achievements as array of (leaderboardType, rank, totalCount) tuples
710
TAGS ""Insights, Widget", "Project""
811

912
NODE project_insights_endpoint
@@ -13,7 +16,16 @@ SQL >
1316
id,
1417
name,
1518
slug,
19+
logoUrl,
20+
isLF,
21+
contributorCount,
22+
organizationCount,
1623
softwareValue,
24+
contributorDependencyCount,
25+
contributorDependencyPercentage,
26+
organizationDependencyCount,
27+
organizationDependencyPercentage,
28+
achievements,
1729
healthScore,
1830
firstCommit,
1931
starsLast365Days,
@@ -28,5 +40,13 @@ SQL >
2840
WHERE
2941
1 = 1
3042
{% if defined(slug) %}
31-
AND slug = {{ String(slug, description="Project slug", required=True) }}
43+
AND slug = {{ String(slug, description="Project slug", required=False) }}
44+
{% end %}
45+
{% if defined(slugs) %}
46+
AND slug
47+
IN {{ Array(slugs, 'String', description="Filter by project slug list", required=False) }}
48+
{% end %}
49+
{% if defined(ids) %}
50+
AND id
51+
IN {{ Array(ids, 'String', description="Filter by project id list", required=False) }}
3252
{% end %}

‎services/libs/tinybird/pipes/project_insights_copy.pipe‎

Lines changed: 59 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,11 +1,56 @@
11
NODE project_insights_copy_base_projects
22
DESCRIPTION >
3-
Returns base project information (id, name, slug, segmentId, softwareValue, healthScore, firstCommit)
3+
Returns base project information (id, name, slug, segmentId, logoUrl, isLF, contributorCount, organizationCount, softwareValue, healthScore, firstCommit)
44

55
SQL >
6-
SELECT id, name, slug, segmentId, softwareValue, healthScore, firstCommit
6+
SELECT
7+
id,
8+
name,
9+
slug,
10+
segmentId,
11+
logoUrl,
12+
isLF,
13+
contributorCount,
14+
organizationCount,
15+
softwareValue,
16+
healthScore,
17+
firstCommit
718
FROM insights_projects_populated_ds
8-
GROUP BY id, name, slug, segmentId, softwareValue, healthScore, firstCommit
19+
GROUP BY
20+
id,
21+
name,
22+
slug,
23+
segmentId,
24+
logoUrl,
25+
isLF,
26+
contributorCount,
27+
organizationCount,
28+
softwareValue,
29+
healthScore,
30+
firstCommit
31+
32+
NODE project_insights_copy_dependency_metrics
33+
DESCRIPTION >
34+
Get contributor and organization dependency metrics from health_score_copy_ds
35+
36+
SQL >
37+
SELECT
38+
slug,
39+
contributorDependencyCount,
40+
contributorDependencyPercentage,
41+
organizationDependencyCount,
42+
organizationDependencyPercentage
43+
FROM health_score_copy_ds
44+
45+
NODE project_insights_copy_achievements
46+
DESCRIPTION >
47+
Aggregate leaderboard achievements per project from leaderboards_copy_ds (latest snapshot)
48+
49+
SQL >
50+
SELECT slug, groupArray(tuple(leaderboardType, rank, totalCount)) AS achievements
51+
FROM leaderboards_copy_ds
52+
WHERE snapshotId = (SELECT max(snapshotId) FROM leaderboards_copy_ds)
53+
GROUP BY slug
954

1055
NODE project_insights_copy_last_365_days_metrics
1156
DESCRIPTION >
@@ -50,7 +95,16 @@ SQL >
5095
base.id AS id,
5196
base.name AS name,
5297
base.slug AS slug,
98+
base.logoUrl AS logoUrl,
99+
base.isLF AS isLF,
100+
base.contributorCount AS contributorCount,
101+
base.organizationCount AS organizationCount,
53102
base.softwareValue AS softwareValue,
103+
COALESCE(dep.contributorDependencyCount, 0) AS contributorDependencyCount,
104+
COALESCE(dep.contributorDependencyPercentage, 0) AS contributorDependencyPercentage,
105+
COALESCE(dep.organizationDependencyCount, 0) AS organizationDependencyCount,
106+
COALESCE(dep.organizationDependencyPercentage, 0) AS organizationDependencyPercentage,
107+
COALESCE(ach.achievements, []) AS achievements,
54108
base.healthScore AS healthScore,
55109
base.firstCommit AS firstCommit,
56110
l365.starsLast365Days AS starsLast365Days,
@@ -62,6 +116,8 @@ SQL >
62116
p365.activeContributorsPrevious365Days AS activeContributorsPrevious365Days,
63117
p365.activeOrganizationsPrevious365Days AS activeOrganizationsPrevious365Days
64118
FROM project_insights_copy_base_projects AS base
119+
LEFT JOIN project_insights_copy_dependency_metrics AS dep ON base.slug = dep.slug
120+
LEFT JOIN project_insights_copy_achievements AS ach ON base.slug = ach.slug
65121
LEFT JOIN project_insights_copy_last_365_days_metrics AS l365 USING (segmentId)
66122
LEFT JOIN project_insights_copy_previous_365_days_metrics AS p365 USING (segmentId)
67123

0 commit comments

Comments
 (0)