Skip to content

Commit ddac6e7

Browse files
Add v3 scoring model (Diet Egg) and Fresh Egg advisory (#48)
* Add v3 scoring model (Diet Egg) and Fresh Egg advisory v3 uses alltime merge rate as the sole scoring input, dropping graph score and log account age from the v2 logistic regression. Fresh accounts (under 365 days) get an advisory surfaced in all output formats without affecting the score. Bots are excluded from the advisory since their synthetic profiles have unreliable age data. - Add FreshAccountAdvisory model and TrustScore.fresh_account field - Add _score_v3() using merge_rate as raw and normalized score - Add _build_fresh_account_advisory() called in all scoring paths - Change default scoring_model from v1 to v3 - Add "Diet Egg" brand name for v3 in all formatters - Widen v2 component breakdown checks to include v3 - Add v3 to CLI, action, and MCP server validation - Update tests: 332 passing, 94% coverage * Run action from local checkout with v3 scoring model Point the Good Egg workflow at ./ instead of the published @v1 tag so the PR runs the branch code. Set scoring-model to v3. * Restore action workflow to use published @v1 tag Remove the local checkout used for testing. Drop the explicit scoring-model since v3 is now the default. * Update docs and examples for v3 (Diet Egg) default - README: update scoring models table, add Fresh Egg section, update description to reflect v3 as default - docs/configuration.md: add v3 to scoring model section, YAML schema, env var table, and scoring_model description - docs/github-action.md: update inputs/outputs tables, replace "Using Better Egg" with general "Selecting a Scoring Model" section, add diet-egg-workflow to example list - docs/library.md: replace v2-only config section with general scoring model selection, add fresh_account to TrustScore field table, update raw_score description - docs/mcp-server.md: update all parameter tables and return examples to show v3 as default - docs/methodology.md: add Diet Egg (v3) section with motivation, Fresh Egg advisory, and component scores; update scoring section to lead with v3 - docs/troubleshooting.md: add v3 and Fresh Egg troubleshooting sections - examples: add diet-egg-workflow.yml, update .good-egg.yml with v3 comments, update library_usage.py with v3 and fresh_account usage - assets: replace PR comment screenshot with Diet Egg version * Address review comments on formatter docstring and threshold constant - Update _brand_name docstring to include Diet Egg - Use FreshAccountAdvisory.model_fields default instead of hardcoded 365 * Address PR review: changelog, explicit v1 model, stale comment, tests - Add CHANGELOG entry for v3 (Diet Egg) and Fresh Egg advisory - Set scoring_model="v1" explicitly in v1 return for consistency - Add comment explaining why bot short-circuit omits fresh_account - Fix stale "v2 component score breakdown" comment to include v3 - Add test for MEDIUM trust boundary with 50% merge rate - Add test for fresh account at 364 days (boundary confirmation)
1 parent 75da8e4 commit ddac6e7

25 files changed

Lines changed: 831 additions & 107 deletions

.github/workflows/good-egg.yml

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -14,5 +14,4 @@ jobs:
1414
- uses: 2ndSetAI/good-egg@v1
1515
with:
1616
github-token: ${{ secrets.GITHUB_TOKEN }}
17-
scoring-model: v2
1817
skip-known-contributors: 'false'

CHANGELOG.md

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5,6 +5,24 @@ All notable changes to this project will be documented in this file.
55
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
66
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
77

8+
## [Unreleased]
9+
10+
### Added
11+
12+
- **Diet Egg (v3) scoring model** -- uses alltime merge rate as the sole
13+
scoring signal, dropping graph score and log account age from v2.
14+
- **Fresh Egg advisory** -- accounts under 365 days old receive an
15+
informational "Fresh Egg" note in all output formats. Does not affect
16+
the score. Bots are excluded since their synthetic profiles have
17+
unreliable age data.
18+
- `FreshAccountAdvisory` model and `fresh_account` field on `TrustScore`.
19+
- "Diet Egg" branding on PR comments when using v3.
20+
- Example workflow for v3: `examples/diet-egg-workflow.yml`.
21+
22+
### Changed
23+
24+
- Default `scoring_model` changed from `v1` to `v3`.
25+
826
## [1.0.0] - 2026-02-23
927

1028
### Added

README.md

Lines changed: 18 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -3,8 +3,7 @@
33
Good Egg
44
</h1>
55

6-
Trust scoring for GitHub PR authors using graph-based analysis of
7-
contribution history.
6+
Trust scoring for GitHub PR authors based on contribution history.
87

98
## Why
109

@@ -122,23 +121,31 @@ See [docs/mcp-server.md](https://github.com/2ndSetAI/good-egg/blob/main/docs/mcp
122121

123122
## Scoring Models
124123

125-
Good Egg supports two scoring models:
124+
Good Egg supports three scoring models:
126125

127126
| Model | Name | Description |
128127
|-------|------|-------------|
129-
| `v1` | Good Egg (default) | Graph-based scoring from contribution history |
128+
| `v3` | Diet Egg (default) | Alltime merge rate as sole signal |
130129
| `v2` | Better Egg | Graph score + merge rate + account age via logistic regression |
130+
| `v1` | Good Egg | Graph-based scoring from contribution history |
131131

132-
To use v2, set `scoring_model: v2` in your `.good-egg.yml`, pass
133-
`--scoring-model v2` on the CLI, or set `scoring-model: v2` in the action
134-
input. See [Methodology](https://github.com/2ndSetAI/good-egg/blob/main/docs/methodology.md#better-egg-v2) for how the
135-
v2 model works.
132+
v3 is the default. To use an older model, set `scoring_model: v1` or
133+
`scoring_model: v2` in your `.good-egg.yml`, pass `--scoring-model v1` on
134+
the CLI, or set `scoring-model: v1` in the action input. See
135+
[Methodology](https://github.com/2ndSetAI/good-egg/blob/main/docs/methodology.md) for how each model works.
136+
137+
### Fresh Egg Advisory
138+
139+
Accounts less than 365 days old receive a "Fresh Egg" advisory in the
140+
output. This is informational only and does not affect the score. Fresh
141+
accounts correlate with lower merge rates in the validation data.
136142

137143
## How It Works
138144

139-
Good Egg builds a weighted contribution graph from a user's merged PRs and
140-
runs personalized graph scoring to produce a trust score relative to your
141-
project. See [Methodology](https://github.com/2ndSetAI/good-egg/blob/main/docs/methodology.md) for details.
145+
The default v3 model (Diet Egg) scores contributors by their alltime merge
146+
rate: merged PRs divided by total PRs (merged + closed). Older models (v1,
147+
v2) build a weighted contribution graph and run personalized graph scoring.
148+
See [Methodology](https://github.com/2ndSetAI/good-egg/blob/main/docs/methodology.md) for details.
142149

143150
## Trust Levels
144151

action.yml

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -23,7 +23,7 @@ inputs:
2323
required: false
2424
default: 'false'
2525
scoring-model:
26-
description: 'Scoring model to use (v1 or v2)'
26+
description: 'Scoring model to use (v1, v2, or v3)'
2727
required: false
2828
skip-known-contributors:
2929
description: 'Skip scoring for authors with merged PRs in the repo (true/false)'
@@ -40,7 +40,7 @@ outputs:
4040
description: 'GitHub username that was scored'
4141
value: ${{ steps.score.outputs.user }}
4242
scoring-model:
43-
description: 'Scoring model that was used (v1 or v2)'
43+
description: 'Scoring model that was used (v1, v2, or v3)'
4444
value: ${{ steps.score.outputs.scoring-model }}
4545
skipped:
4646
description: 'Whether scoring was skipped for an existing contributor (true/false)'

assets/pr-comment-screenshot.png

7.51 KB
Loading

docs/configuration.md

Lines changed: 22 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -27,23 +27,24 @@ Configuration values are resolved in this order (highest priority first):
2727

2828
## Scoring Model
2929

30-
Good Egg supports two scoring models. Set the model at the top level of the
31-
config file:
30+
Good Egg supports three scoring models. Set the model at the top level of
31+
the config file:
3232

3333
```yaml
34-
scoring_model: v1 # default -- graph-based scoring only
34+
scoring_model: v3 # default -- Diet Egg -- alltime merge rate as sole signal
3535
scoring_model: v2 # Better Egg -- graph + external features via logistic regression
36+
scoring_model: v1 # Good Egg -- graph-based scoring only
3637
```
3738
38-
When using v2, PR comments are branded "Better Egg" instead of "Good Egg".
39-
See [methodology.md](methodology.md#better-egg-v2) for how the v2 model
39+
PR comments are branded "Diet Egg", "Better Egg", or "Good Egg" depending
40+
on the model. See [methodology.md](methodology.md) for how each model
4041
works.
4142
4243
## Full YAML Schema
4344
4445
```yaml
45-
# Scoring model selection: v1 (default) or v2
46-
scoring_model: v1
46+
# Scoring model selection: v3 (default), v2, or v1
47+
scoring_model: v3
4748

4849
# Skip scoring for authors who already have merged PRs in the target repo.
4950
# When true (the default), existing contributors get an EXISTING_CONTRIBUTOR
@@ -127,12 +128,19 @@ v2:
127128
128129
### scoring_model
129130
130-
Selects the scoring model. Set to `v1` (default) for graph-only scoring or
131-
`v2` for the Better Egg combined model. When set to `v2`, the parameters
132-
under the `v2:` block are used and the graph construction is simplified
133-
(no self-contribution penalty, no language normalization in repo quality, no
134-
diversity/volume adjustment). Language match personalization weighting
135-
(`same_language_weight`) is retained in v2.
131+
Selects the scoring model. `v3` (default, Diet Egg) uses alltime merge rate
132+
as the sole signal with no graph construction. `v2` (Better Egg) combines a
133+
simplified graph score with merge rate and account age via logistic
134+
regression. `v1` (Good Egg) uses graph-based scoring only.
135+
136+
When set to `v2`, the parameters under the `v2:` block are used and the
137+
graph construction is simplified (no self-contribution penalty, no language
138+
normalization in repo quality, no diversity/volume adjustment). Language
139+
match personalization weighting (`same_language_weight`) is retained in v2.
140+
141+
v3 does not use graph construction, so the `graph_scoring`, `recency`,
142+
`edge_weights`, and `language_normalization` sections have no effect. The
143+
`thresholds` section still controls trust level classification.
136144

137145
### v2 (Better Egg)
138146

@@ -215,7 +223,7 @@ The following environment variables override individual config values:
215223
| `GOOD_EGG_HIGH_TRUST` | `thresholds.high_trust` | float |
216224
| `GOOD_EGG_MEDIUM_TRUST` | `thresholds.medium_trust` | float |
217225
| `GOOD_EGG_HALF_LIFE_DAYS` | `recency.half_life_days` | int |
218-
| `GOOD_EGG_SCORING_MODEL` | `scoring_model` | str (`v1` or `v2`) |
226+
| `GOOD_EGG_SCORING_MODEL` | `scoring_model` | str (`v1`, `v2`, or `v3`) |
219227
| `GOOD_EGG_SKIP_KNOWN_CONTRIBUTORS` | `skip_known_contributors` | bool (`true`/`false`) |
220228

221229
## Programmatic Configuration

docs/github-action.md

Lines changed: 11 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -37,7 +37,7 @@ This posts a trust score comment on each pull request:
3737
| `comment` | No | `true` | Post a PR comment with the trust score |
3838
| `check-run` | No | `false` | Create a check run with the trust score |
3939
| `fail-on-low` | No | `false` | Fail the action if trust level is LOW |
40-
| `scoring-model` | No | `v1` | Scoring model: `v1` (Good Egg) or `v2` (Better Egg) |
40+
| `scoring-model` | No | `v3` | Scoring model: `v3` (Diet Egg), `v2` (Better Egg), or `v1` (Good Egg) |
4141
| `skip-known-contributors` | No | `true` | Skip scoring for authors with merged PRs in the repo |
4242

4343
## Outputs
@@ -47,7 +47,7 @@ This posts a trust score comment on each pull request:
4747
| `score` | Normalized trust score (0.0 - 1.0) |
4848
| `trust-level` | Trust level: HIGH, MEDIUM, LOW, UNKNOWN, BOT, or EXISTING_CONTRIBUTOR |
4949
| `user` | GitHub username that was scored |
50-
| `scoring-model` | Scoring model used: `v1` (Good Egg) or `v2` (Better Egg) |
50+
| `scoring-model` | Scoring model used: `v3` (Diet Egg), `v2` (Better Egg), or `v1` (Good Egg) |
5151
| `skipped` | Whether scoring was skipped for an existing contributor (`true`/`false`) |
5252

5353
## Custom Configuration
@@ -179,9 +179,10 @@ jobs:
179179

180180
You can check whether scoring was skipped via the `skipped` output.
181181

182-
## Using Better Egg (v2)
182+
## Selecting a Scoring Model
183183

184-
To use the v2 scoring model, set the `scoring-model` input:
184+
The default model is v3 (Diet Egg), which scores by alltime merge rate. To
185+
use an older model, set the `scoring-model` input:
185186

186187
```yaml
187188
jobs:
@@ -191,12 +192,12 @@ jobs:
191192
- uses: 2ndSetAI/good-egg@v0
192193
with:
193194
github-token: ${{ secrets.GITHUB_TOKEN }}
194-
scoring-model: v2
195+
scoring-model: v2 # or v1
195196
```
196197

197-
When using v2, PR comments are branded "Better Egg" and include a component
198-
score breakdown showing graph score, merge rate, and account age
199-
contributions. The `scoring-model` output reflects which model was used.
198+
PR comments are branded according to the model: "Diet Egg" for v3, "Better
199+
Egg" for v2, "Good Egg" for v1. v2 and v3 include a component score
200+
breakdown. The `scoring-model` output reflects which model was used.
200201

201202
You can also set `scoring-model` via the `GOOD_EGG_SCORING_MODEL` environment
202203
variable, but the input takes precedence.
@@ -209,5 +210,7 @@ See the [examples/](../examples/) directory for complete workflow files:
209210
that posts a PR comment
210211
- [strict-workflow.yml](../examples/strict-workflow.yml) -- comment, check
211212
run, and fail-on-low
213+
- [diet-egg-workflow.yml](../examples/diet-egg-workflow.yml) -- v3
214+
scoring model (default) with component breakdown
212215
- [better-egg-workflow.yml](../examples/better-egg-workflow.yml) -- v2
213216
scoring model with component breakdown

docs/library.md

Lines changed: 21 additions & 23 deletions
Original file line numberDiff line numberDiff line change
@@ -114,26 +114,22 @@ result = await score_pr_author(
114114
)
115115
```
116116

117-
### v2 (Better Egg) Configuration
117+
### Scoring Model Selection
118118

119-
To use the v2 scoring model, set `scoring_model` on the config:
119+
The default model is v3 (Diet Egg). To use an older model, set
120+
`scoring_model` on the config:
120121

121122
```python
122123
from good_egg import GoodEggConfig, score_pr_author
123124

124-
config = GoodEggConfig(
125-
scoring_model="v2",
126-
v2={
127-
"graph": {"half_life_days": 180, "max_age_days": 730},
128-
"features": {"merge_rate": True, "account_age": True},
129-
"combined_model": {
130-
"intercept": -0.8094,
131-
"graph_score_weight": 1.9138,
132-
"merge_rate_weight": -0.7783,
133-
"account_age_weight": 0.1493,
134-
},
135-
},
136-
)
125+
# v3 (default) -- merge rate only
126+
config = GoodEggConfig()
127+
128+
# v2 -- graph + merge rate + account age
129+
config = GoodEggConfig(scoring_model="v2")
130+
131+
# v1 -- graph only
132+
config = GoodEggConfig(scoring_model="v1")
137133

138134
result = await score_pr_author(
139135
login="octocat",
@@ -142,12 +138,13 @@ result = await score_pr_author(
142138
config=config,
143139
)
144140

145-
# v2 results include component scores
141+
# v3 and v2 results include component scores
146142
if result.component_scores:
147-
print(f"Graph score: {result.component_scores['graph_score']:.3f}")
148-
print(f"Merge rate: {result.component_scores['merge_rate']:.3f}")
149-
print(f"Log account age: {result.component_scores['log_account_age']:.3f}")
150-
print(f"Normalized score: {result.normalized_score:.3f}")
143+
print(f"Merge rate: {result.component_scores.get('merge_rate')}")
144+
145+
# v3 includes a fresh account advisory
146+
if result.fresh_account and result.fresh_account.is_fresh:
147+
print(f"Fresh account: {result.fresh_account.account_age_days} days old")
151148

152149
print(f"Scoring model: {result.scoring_model}")
153150
```
@@ -176,7 +173,7 @@ the following fields:
176173
|-------|------|-------------|
177174
| `user_login` | `str` | GitHub username that was scored |
178175
| `context_repo` | `str` | Repository used as scoring context |
179-
| `raw_score` | `float` | Pre-normalization score: graph score (v1) or logit (v2) |
176+
| `raw_score` | `float` | Pre-normalization score: merge rate (v3), logit (v2), or graph score (v1) |
180177
| `normalized_score` | `float` | Normalized score (0.0 - 1.0) |
181178
| `trust_level` | `TrustLevel` | HIGH, MEDIUM, LOW, UNKNOWN, BOT, or EXISTING_CONTRIBUTOR |
182179
| `account_age_days` | `int` | Age of the GitHub account in days |
@@ -185,9 +182,10 @@ the following fields:
185182
| `top_contributions` | `list[ContributionSummary]` | Top repositories contributed to |
186183
| `language_match` | `bool` | Whether the user's top language matches the context repo |
187184
| `flags` | `dict[str, bool]` | Flags (is_bot, is_new_account, etc.) |
188-
| `scoring_model` | `str` | Scoring model used: `v1` or `v2` |
189-
| `component_scores` | `dict[str, float] \| None` | Component breakdown (v2 only): `graph_score`, `merge_rate`, `log_account_age` |
185+
| `scoring_model` | `str` | Scoring model used: `v1`, `v2`, or `v3` |
186+
| `component_scores` | `dict[str, float]` | Component breakdown (v3: `merge_rate`; v2: `graph_score`, `merge_rate`, `log_account_age`) |
190187
| `scoring_metadata` | `dict[str, Any]` | Internal scoring details |
188+
| `fresh_account` | `FreshAccountAdvisory \| None` | Advisory for accounts under 365 days old (None for bots and existing contributors) |
191189

192190
`TrustScore` is a Pydantic model, so you can serialize it:
193191

0 commit comments

Comments
 (0)