Skip to content

fix: use correct DNS for evalhub#134

Merged
tarilabs merged 1 commit intoopendatahub-io:incubationfrom
tarilabs:tarilabs-20260325
Mar 25, 2026
Merged

fix: use correct DNS for evalhub#134
tarilabs merged 1 commit intoopendatahub-io:incubationfrom
tarilabs:tarilabs-20260325

Conversation

@tarilabs
Copy link
Copy Markdown
Member

@tarilabs tarilabs commented Mar 25, 2026

Description

small fix reverse DNS

How Has This Been Tested?

not needed

Merge criteria:

  • The commits are squashed in a cohesive manner and have meaningful messages.
  • Testing instructions have been added in the PR body (for PRs involving changes that are not immediately obvious).
  • The developer has manually tested the changes and verified that the changes work

Summary by CodeRabbit

  • Chores
    • Updated artifact configuration identifiers to align with new naming convention.

Signed-off-by: tarilabs <matteo.mortari@gmail.com>
@coderabbitai
Copy link
Copy Markdown

coderabbitai bot commented Mar 25, 2026

📝 Walkthrough

Walkthrough

Changes to main.py migrate OCI artifact annotation key namespaces from org.evalhub.* to io.github.eval-hub.*. Three annotation keys are affected: benchmark, model, and job_id. Annotation values and artifact creation logic remain unchanged. This constitutes a namespace rebranding for OCI artifact metadata.

Estimated code review effort

🎯 1 (Trivial) | ⏱️ ~3 minutes

🚥 Pre-merge checks | ✅ 1 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Title check ⚠️ Warning The title describes changing DNS configuration for evalhub, but the actual changes involve updating OCI artifact annotation namespaces from org.evalhub.* to io.github.eval-hub.*—a namespace/registry change, not a DNS configuration change. Update the title to accurately reflect the namespace change, such as 'fix: update OCI artifact annotations to io.github.eval-hub namespace' or similar.
✅ Passed checks (1 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.


Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Copy Markdown

@mariusdanciu mariusdanciu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

Copy link
Copy Markdown

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@main.py`:
- Around line 381-388: The update currently writes new OCI annotation keys via
coords.annotations.update; to remain backward-compatible also write the legacy
org.evalhub.* keys alongside the new io.github.eval-hub.* and
org.opencontainers.image.created keys so consumers reading the old namespace
still see metadata. Modify the coords.annotations.update payload to include
duplicate legacy keys (e.g., "org.evalhub.image.created" or matching legacy
names for created/benchmark/model/job_id) set to the same values
(datetime.now(UTC).isoformat(), benchmark_id, model_name, job_id) so both
namespaces are written during migration; remove the legacy keys in a future,
explicitly versioned change.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Central YAML (base), Organization UI (inherited)

Review profile: CHILL

Plan: Pro

Run ID: 22c55c55-907e-433e-bb00-a34c9d7be680

📥 Commits

Reviewing files that changed from the base of the PR and between c8d8109 and b54e2d7.

📒 Files selected for processing (1)
  • main.py

Comment on lines 381 to 388
coords.annotations.update(
{
"org.opencontainers.image.created": datetime.now(UTC).isoformat(),
"org.evalhub.benchmark": benchmark_id,
"org.evalhub.model": model_name,
"org.evalhub.job_id": job_id,
"io.github.eval-hub.benchmark": benchmark_id,
"io.github.eval-hub.model": model_name,
"io.github.eval-hub.job_id": job_id,
}
)
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

Add backward-compatible annotation keys to avoid metadata contract break

This change silently renames externally consumed OCI annotation keys. If any consumer still reads org.evalhub.*, artifact processing can fail (integration reliability break). Write both namespaces during migration, then remove legacy keys in a later versioned change.

Proposed compatibility patch
                 coords.annotations.update(
                     {
                         "org.opencontainers.image.created": datetime.now(UTC).isoformat(),
+                        # Legacy namespace (backward compatibility)
+                        "org.evalhub.benchmark": benchmark_id,
+                        "org.evalhub.model": model_name,
+                        "org.evalhub.job_id": job_id,
+                        # New reverse-DNS namespace
                         "io.github.eval-hub.benchmark": benchmark_id,
                         "io.github.eval-hub.model": model_name,
                         "io.github.eval-hub.job_id": job_id,
                     }
                 )

As per coding guidelines, "REVIEW PRIORITIES: 2. Architectural issues and anti-patterns; 3. Bug-prone patterns and error handling gaps."

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@main.py` around lines 381 - 388, The update currently writes new OCI
annotation keys via coords.annotations.update; to remain backward-compatible
also write the legacy org.evalhub.* keys alongside the new io.github.eval-hub.*
and org.opencontainers.image.created keys so consumers reading the old namespace
still see metadata. Modify the coords.annotations.update payload to include
duplicate legacy keys (e.g., "org.evalhub.image.created" or matching legacy
names for created/benchmark/model/job_id) set to the same values
(datetime.now(UTC).isoformat(), benchmark_id, model_name, job_id) so both
namespaces are written during migration; remove the legacy keys in a future,
explicitly versioned change.

@tarilabs tarilabs merged commit b37fa96 into opendatahub-io:incubation Mar 25, 2026
4 of 5 checks passed
ruivieira pushed a commit to ruivieira/lm-evaluation-harness that referenced this pull request Mar 26, 2026
Signed-off-by: tarilabs <matteo.mortari@gmail.com>
ruivieira pushed a commit to ruivieira/lm-evaluation-harness that referenced this pull request Mar 26, 2026
Signed-off-by: tarilabs <matteo.mortari@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants