Skip to content

Route against-direction objective constraints to outcome constraints in MOO#5239

Open
saitcakmak wants to merge 1 commit into
facebook:mainfrom
saitcakmak:export-D109856433
Open

Route against-direction objective constraints to outcome constraints in MOO#5239
saitcakmak wants to merge 1 commit into
facebook:mainfrom
saitcakmak:export-D109856433

Conversation

@saitcakmak

Copy link
Copy Markdown
Contributor

Summary:
D109061377 relaxed MultiObjectiveOptimizationConfig validation so that, in multi-objective optimization, an outcome constraint can be placed on an objective metric (e.g. flops >= 42 while minimizing flops) to bound the objective against its optimization direction. Objective.get_unconstrainable_metric_names() now returns [] for MOO.

However, optimization_config_from_string (used by Client.configure_optimization) was never updated. It unconditionally converted any single-metric constraint on an objective metric into an objective threshold. Objective thresholds can only bound in the direction of optimization, so a constraint like flops >= 42.50 while minimizing flops was turned into an invalid threshold and tripped check_objective_thresholds_match_objectives:

UserInputError: Objective threshold on flops bounds from below but flops is being minimized.

This diff fixes the conversion so a single-metric constraint on an objective metric becomes an objective threshold only when it bounds the objective in its optimization direction (upper bound on a minimized objective, lower bound on a maximized one). A constraint that bounds against the optimization direction is kept as a true outcome constraint, which MOO now supports.

For objective="-flops, -train_ne" with ["flops >= 42.50", "flops <= 94.38", "train_ne <= 0.62938"]:

  • flops >= 42.50 -> outcome constraint (against the minimize direction)
  • flops <= 94.38 -> objective threshold (aligned with minimize)
  • train_ne <= 0.62938 -> objective threshold (aligned with minimize)

Differential Revision: D109856433

…in MOO

Summary:
D109061377 relaxed `MultiObjectiveOptimizationConfig` validation so that, in multi-objective optimization, an outcome constraint can be placed on an objective metric (e.g. `flops >= 42` while minimizing `flops`) to bound the objective against its optimization direction. `Objective.get_unconstrainable_metric_names()` now returns `[]` for MOO.

However, `optimization_config_from_string` (used by `Client.configure_optimization`) was never updated. It unconditionally converted any single-metric constraint on an objective metric into an *objective threshold*. Objective thresholds can only bound in the direction of optimization, so a constraint like `flops >= 42.50` while minimizing `flops` was turned into an invalid threshold and tripped `check_objective_thresholds_match_objectives`:

  UserInputError: Objective threshold on flops bounds from below but flops is being minimized.

This diff fixes the conversion so a single-metric constraint on an objective metric becomes an objective threshold only when it bounds the objective in its optimization direction (upper bound on a minimized objective, lower bound on a maximized one). A constraint that bounds against the optimization direction is kept as a true outcome constraint, which MOO now supports.

For `objective="-flops, -train_ne"` with `["flops >= 42.50", "flops <= 94.38", "train_ne <= 0.62938"]`:
- `flops >= 42.50` -> outcome constraint (against the minimize direction)
- `flops <= 94.38` -> objective threshold (aligned with minimize)
- `train_ne <= 0.62938` -> objective threshold (aligned with minimize)

Differential Revision: D109856433
@meta-cla meta-cla Bot added the CLA Signed Do not delete this pull request or issue due to inactivity. label Jun 26, 2026
@meta-codesync

meta-codesync Bot commented Jun 26, 2026

Copy link
Copy Markdown

@saitcakmak has exported this pull request. If you are a Meta employee, you can view the originating Diff in D109856433.

@codecov-commenter

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 96.57%. Comparing base (025d7e5) to head (fe8702e).

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #5239   +/-   ##
=======================================
  Coverage   96.56%   96.57%           
=======================================
  Files         619      619           
  Lines       70273    70282    +9     
=======================================
+ Hits        67861    67875   +14     
+ Misses       2412     2407    -5     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA Signed Do not delete this pull request or issue due to inactivity. meta-exported

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants