Skip to content

Conversation

@bencap
Copy link
Collaborator

@bencap bencap commented Nov 19, 2024

No description provided.

@bencap bencap added type: feature New feature workstream: pillar Task relates to pillar project app: backend Task implementation touches the backend app: database Task implementation requires database changes labels Nov 19, 2024
@bencap bencap linked an issue Nov 19, 2024 that may be closed by this pull request
Base automatically changed from release-2024.4.2 to main November 21, 2024 22:55
@bencap bencap changed the base branch from main to release-2024.4.3 November 21, 2024 23:01
@bencap bencap changed the base branch from release-2024.4.3 to release-2024.5.0 December 10, 2024 22:36
@bencap bencap marked this pull request as ready for review December 17, 2024 20:18
@bencap bencap requested a review from jstone-dev December 17, 2024 20:18
Copy link
Collaborator

@jstone-dev jstone-dev left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we somehow document or validate the expected calibration keys?

Apart from this, no comments except for the one typo.


if len(set((num_es, num_st, num_plr))) != 1:
raise ValidationError(
"Calibration object must provide the same number of evidence strenghts, score thresholds, and positive likelihood ratios. "
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Spelling typo

@bencap bencap force-pushed the feature/bencap/313/contributor-provided-score-thresholds branch from efcca99 to 000819a Compare December 31, 2024 04:40
@bencap
Copy link
Collaborator Author

bencap commented Dec 31, 2024

Should we somehow document or validate the expected calibration keys?

Apart from this, no comments except for the one typo.

@jstone-dev Are you talking about calibration keys in the sense of pillar_project: {calibrations} or the set of thresholds, evidence_strengths, positive_likelihood_ratios?

For the former, I guess I just don't know what those will be at the moment but it does probably make sense to have a list of them which are valid-- especially before we open up an endpoint for contributors to add calibrations (if that ever happens). For the latter, I think the pydantic model validation should be sufficient for API users at least, although we should certainly work on improving our documentation in the upcoming months.

@bencap bencap force-pushed the feature/bencap/313/contributor-provided-score-thresholds branch from 000819a to 740f8aa Compare December 31, 2024 04:48
@bencap bencap merged commit 1e223c3 into release-2024.5.0 Jan 24, 2025
5 checks passed
@bencap bencap deleted the feature/bencap/313/contributor-provided-score-thresholds branch January 24, 2025 17:22
@bencap bencap mentioned this pull request Feb 25, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

app: backend Task implementation touches the backend app: database Task implementation requires database changes type: feature New feature workstream: pillar Task relates to pillar project

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Score Thresholds from Contributors

3 participants