Skip to content

Add production dependency validation via validate_needs()#109

Draft
Copilot wants to merge 2 commits intov0.8-previewfrom
copilot/add-production-dependency-validation
Draft

Add production dependency validation via validate_needs()#109
Copilot wants to merge 2 commits intov0.8-previewfrom
copilot/add-production-dependency-validation

Conversation

Copy link

Copilot AI commented Feb 19, 2026

Catch misconfigured needs chains before job submission by validating that each declared dependency will actually produce the data products required by the downstream analysis.

Pipeline metadata (pipeline.py)

  • Added available_outputs and required_inputs class-level list attributes to Pipeline — declare what a pipeline produces/consumes
  • Added get_actual_outputs(production) and get_actual_inputs(production) methods returning those lists by default; subclasses can override for conditional logic based on production configuration

Dependency validation (analysis.py)

  • Added validate_needs() to the Analysis base class — resolves dependency names to analysis objects, then checks each required input against the outputs advertised by every dependency, issuing a UserWarning per unsatisfied requirement
# In a pipeline subclass:
class BayesWave(Pipeline):
    available_outputs = ["psd"]

class Bilby(Pipeline):
    required_inputs = ["psd"]

# At build/submission time:
bilby_analysis.validate_needs()
# UserWarning: Production 'Prod1' requires 'psd' but no dependency provides it

The method is a no-op when the pipeline declares no required_inputs, preserving backward compatibility with all existing pipelines.

Original prompt

This section details on the original issue you should resolve

<issue_title>Add Production Dependency Validation</issue_title>
<issue_description>Summary: Validate that productions in needs will provide required data products before submission.

Description:

Add validate_needs() method to Analysis class that:

  • Checks if productions listed in needs provide the required inputs (we can find this via the enhancements described in Add Pipeline Input/Output Metadata System #94)
  • Warns (or errors) if required data products won't be available
  • Runs at build/submission time, not runtime

Example validation:

def validate_needs(self):
    required = self.pipeline.get_actual_inputs(self)
    for requirement in required:
        satisfied = any(
            requirement in dep.pipeline.get_actual_outputs(dep)
            for dep in self.needs
        )
        if not satisfied:
            warnings.warn(f"Production requires '{requirement}' but no dependency provides it")

Benefits:

  • Catch configuration errors before wasting compute time
  • Better user feedback about missing dependencies
  • Foundation for smarter workflow orchestration</issue_description>

Comments on the Issue (you are @copilot in this section)


💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.

…eline

Co-authored-by: transientlunatic <4365778+transientlunatic@users.noreply.github.com>
Copilot AI changed the title [WIP] Add production dependency validation to Analysis class Add production dependency validation via validate_needs() Feb 19, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants