Skip to content

Create asimov v0.7.0#78

Draft
transientlunatic wants to merge 195 commits intomainfrom
v0.7-preview
Draft

Create asimov v0.7.0#78
transientlunatic wants to merge 195 commits intomainfrom
v0.7-preview

Conversation

@transientlunatic
Copy link
Collaborator

No description provided.

transientlunatic and others added 4 commits January 20, 2026 10:14
* Initial plan

* Fix dependency resolution order and graph visualization bugs

Co-authored-by: transientlunatic <4365778+transientlunatic@users.noreply.github.com>

* Add status persistence fix, results links, and subject analysis visualization

Co-authored-by: transientlunatic <4365778+transientlunatic@users.noreply.github.com>

* Enhance SubjectAnalysis handling and visualization

- Added source analyses display in Analysis class.
- Implemented source analyses readiness check in SubjectAnalysis.
- Updated manage.py to handle source analyses completion.
- Improved monitor.py to update analysis status in ledger.
- Enhanced report.py with styling for source analysis dependencies.
- Modified event.py to include source analysis names and statuses.
- Cleaned up PESummary command output.

* Fix YAMLLedger initialization to use absolute path for ledger file

* Fix job submission to capture cluster ID and update job ID handling

* Fix condor interaction bug

* Refactor analysis resolution and dependency handling in SubjectAnalysis and ProjectAnalysis; enhance PESummary asset collection and processing detection

* Enhance dependency resolution and validation in analysis and pipeline classes; add staleness detection for SubjectAnalysis and improve error handling for resolved dependencies

* Remove unnecessary comment handling in SubjectAnalysis class

* Improve source analyses handling in SubjectAnalysis by checking for iterable attributes

* Apply suggestions from code review

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: transientlunatic <4365778+transientlunatic@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
…rogrammatic API, and pipeline-specific handlers (#72)

* Initial plan

* Add state machine architecture and refactor monitor loop

Co-authored-by: transientlunatic <4365778+transientlunatic@users.noreply.github.com>

* Add comprehensive unit tests for monitor state machine

Co-authored-by: transientlunatic <4365778+transientlunatic@users.noreply.github.com>

* Add documentation and .gitignore for monitor state machine

Co-authored-by: transientlunatic <4365778+transientlunatic@users.noreply.github.com>

* Address code review feedback: fix imports, error handling, and ledger updates

Co-authored-by: transientlunatic <4365778+transientlunatic@users.noreply.github.com>

* Add plugin system for custom monitor states via entry points

Co-authored-by: transientlunatic <4365778+transientlunatic@users.noreply.github.com>

* Add programmatic monitor API and custom states from v0.6

Co-authored-by: transientlunatic <4365778+transientlunatic@users.noreply.github.com>

* Add tests for programmatic API and custom states

Co-authored-by: transientlunatic <4365778+transientlunatic@users.noreply.github.com>

* Address code review feedback: keyword-only params and performance improvements

Co-authored-by: transientlunatic <4365778+transientlunatic@users.noreply.github.com>

* Add pipeline-specific state handlers with fallback to defaults

Co-authored-by: transientlunatic <4365778+transientlunatic@users.noreply.github.com>

* Fix duplicate test content in test_monitor_helpers.py

Co-authored-by: transientlunatic <4365778+transientlunatic@users.noreply.github.com>

* Add initial manifest file for Test Project storage

* Fix WaitState test and add programmatic monitor API end-to-end test

Co-authored-by: transientlunatic <4365778+transientlunatic@users.noreply.github.com>

* Apply suggestions from code review

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: transientlunatic <4365778+transientlunatic@users.noreply.github.com>
Co-authored-by: Daniel Williams <mail@daniel-williams.co.uk>
Co-authored-by: Daniel Williams <daniel.williams@ligo.org>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
…d ProjectAnalysis (#48)

* Initial plan

* Add testing pipelines for SimpleAnalysis, SubjectAnalysis, and ProjectAnalysis

Co-authored-by: transientlunatic <4365778+transientlunatic@users.noreply.github.com>

* Address code review feedback: fix YAML formatting and reduce code duplication

Co-authored-by: transientlunatic <4365778+transientlunatic@users.noreply.github.com>

* Polish code style: use modern super() syntax and remove trailing whitespace

Co-authored-by: transientlunatic <4365778+transientlunatic@users.noreply.github.com>

* Add config template files for testing pipelines to fix TemplateNotFound error

Co-authored-by: transientlunatic <4365778+transientlunatic@users.noreply.github.com>

* Add build_dag method to testing pipelines to fix AttributeError

Co-authored-by: transientlunatic <4365778+transientlunatic@users.noreply.github.com>

* Fix testing pipelines to immediately create results files on submit

The testing pipelines now create their results files (results.dat,
combined_results.dat, population_results.dat) immediately in submit_dag
instead of waiting for after_completion. This simulates instantly-completing
jobs and allows the GitHub Actions workflow file watchers to detect completion.

Co-authored-by: transientlunatic <4365778+transientlunatic@users.noreply.github.com>

* Rework testing pipelines to actually submit jobs to HTCondor scheduler

Changed approach so testing pipelines properly submit to HTCondor:
- build_dag() now creates executable shell scripts, HTCondor submit files, and DAG files
- Shell scripts create results files when they run (not immediately)
- submit_dag() calls condor_submit_dag to actually submit to the scheduler
- Jobs run on HTCondor and create results files, properly testing the infrastructure
- All three pipelines (Simple, Subject, Project) updated with this approach

Co-authored-by: transientlunatic <4365778+transientlunatic@users.noreply.github.com>

* Fix SubjectTestPipeline to use production.event instead of production.subject

The SubjectAnalysis class exposes the subject/event via the 'event' attribute,
consistent with how other pipelines access it. Changed batch-name construction
from production.subject.name to production.event.name to fix AttributeError.

Co-authored-by: transientlunatic <4365778+transientlunatic@users.noreply.github.com>

* Fix test blueprints to use correct 'kind' values for apply command

Changed subject_test_pipeline.yaml from 'kind: subject_analysis' to 'kind: analysis'
and project_test_pipeline.yaml from 'kind: project_analysis' to 'kind: ProjectAnalysis'.

The apply command in asimov only recognizes specific kind values:
- 'analysis' for regular and subject analyses
- 'ProjectAnalysis' for project analyses
- 'event', 'postprocessing', 'configuration'

SubjectAnalysis is distinguished by its pipeline type and internal category attribute,
not by a special 'kind' value in the YAML.

Co-authored-by: transientlunatic <4365778+transientlunatic@users.noreply.github.com>

* Fix ProjectAnalysis blueprint to use correct analyses filter format

Changed from 'name: test-simple-pipeline' to 'name:test-simple-pipeline'.
The analyses field in ProjectAnalysis blueprints uses attribute:value format
(e.g., 'pipeline:bayeswave', 'status:finished', 'name:analysis-name'),
not YAML key-value pairs with spaces.

Co-authored-by: transientlunatic <4365778+transientlunatic@users.noreply.github.com>

* Fix DAG file path in submit_dag methods for all testing pipelines

Changed from absolute/relative paths to just filenames since condor_submit_dag
is executed after changing to the rundir. All three pipelines (Simple, Subject,
Project) now use just the DAG filename (e.g., 'test.dag') in the submit command.

Co-authored-by: transientlunatic <4365778+transientlunatic@users.noreply.github.com>

* Fix ProjectTestPipeline to use absolute path for results file creation

Changed the shell script to use absolute path when creating population_results.dat
instead of relying on relative path resolution. Also added debug logging to show
working directory and target file path to help diagnose any future issues.

This ensures the results file is created in the correct location even if there are
issues with initialdir or working directory resolution in HTCondor.

Co-authored-by: transientlunatic <4365778+transientlunatic@users.noreply.github.com>

* Fix shell script syntax error in ProjectTestPipeline

Fixed syntax error in the echo command that was causing the shell script to fail.
Changed from f-string with unescaped quotes to proper double quotes with command
substitution: echo "Working directory: $(pwd)"

This was preventing the job from running correctly on HTCondor, which explains
why the population_results.dat file was never created.

Co-authored-by: transientlunatic <4365778+transientlunatic@users.noreply.github.com>

* Fix ProjectTestPipeline to use relative path for results file

Co-authored-by: transientlunatic <4365778+transientlunatic@users.noreply.github.com>

* Add absolute path for results file in testing pipelines and enhance job scripts with directory information

* Add rundir property to SubjectAnalysis and update job script in SubjectTestPipeline to use absolute path for results file

* Update rundir property in ProjectAnalysis to return absolute path

* Apply suggestions from code review

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: transientlunatic <4365778+transientlunatic@users.noreply.github.com>
Co-authored-by: Daniel Williams <daniel.williams@ligo.org>
Co-authored-by: Daniel Williams <mail@daniel-williams.co.uk>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings January 22, 2026 12:18
@transientlunatic transientlunatic marked this pull request as draft January 22, 2026 12:18
Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR introduces version 0.7.0 of asimov, a major release that adds several significant features:

Purpose: Adds a Python API for project management, refactored prior handling system, monitor state machine architecture, testing pipeline infrastructure, and various improvements to dependency management and HTML reporting.

Changes:

  • New Python API for programmatic project creation and management
  • Prior specification and validation system using pydantic
  • Monitor state machine refactoring with plugin support
  • Testing pipelines for SimpleAnalysis, SubjectAnalysis, and ProjectAnalysis
  • Dependency system enhancements (optional dependencies, property-based dependencies)
  • Directory structure change from "C01_offline" to "analyses"

Reviewed changes

Copilot reviewed 99 out of 106 changed files in this pull request and generated 14 comments.

Show a summary per file
File Description
asimov/project.py New Project class for Python API with context manager support
asimov/priors.py Pydantic-based prior specification and validation system
asimov/pipeline.py Enhanced pipeline base class with prior interface support
asimov/monitor_helpers.py Helper functions for monitor loop refactoring
asimov/pipelines/testing/*.py New testing pipelines for all analysis types
tests/*.py Comprehensive test coverage for new features
pyproject.toml Updated dependencies including pydantic>=2.0.0
docs/source/*.rst Extensive documentation for new features

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.


self.assertEqual(len(samples), 1)
self.assertTrue(os.path.exists(samples[0]))
self.assertTrue("posterior_samples.dat" in samples[0])
Copy link

Copilot AI Jan 22, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

assertTrue(a in b) cannot provide an informative message. Using assertIn(a, b) instead will give more informative messages.

Copilot uses AI. Check for mistakes.

# Check if stale (dependencies changed)
is_stale = hasattr(node, 'is_stale') and node.is_stale
is_refreshable = hasattr(node, 'is_refreshable') and node.is_refreshable
Copy link

Copilot AI Jan 22, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Variable is_refreshable is not used.

Suggested change
is_refreshable = hasattr(node, 'is_refreshable') and node.is_refreshable

Copilot uses AI. Check for mistakes.

# Check if stale (dependencies changed)
is_stale = hasattr(node, 'is_stale') and node.is_stale
is_refreshable = hasattr(node, 'is_refreshable') and node.is_refreshable
Copy link

Copilot AI Jan 22, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Variable is_refreshable is not used.

Suggested change
is_refreshable = hasattr(node, 'is_refreshable') and node.is_refreshable

Copilot uses AI. Check for mistakes.
Comment on lines +108 to +113
def _webdir_for(subject_name, production_name):
webroot = Path(config.get("general", "webroot"))
if not webroot.is_absolute():
webroot = Path(config.get("project", "root")) / webroot
return webroot / subject_name / production_name / "pesummary"

Copy link

Copilot AI Jan 22, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Variable _webdir_for is not used.

Suggested change
def _webdir_for(subject_name, production_name):
webroot = Path(config.get("general", "webroot"))
if not webroot.is_absolute():
webroot = Path(config.get("project", "root")) / webroot
return webroot / subject_name / production_name / "pesummary"

Copilot uses AI. Check for mistakes.
Comment on lines +114 to +124
def _has_pesummary_outputs(webdir: Path) -> bool:
"""Detect PESummary outputs when the default sentinel is missing."""
posterior = webdir / "samples" / "posterior_samples.h5"
if posterior.exists():
return True
# Accept legacy pesummary.dat as fallback
legacy = webdir / "samples" / f"{webdir.parent.name}_pesummary.dat"
if legacy.exists():
return True
return False

Copy link

Copilot AI Jan 22, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Variable _has_pesummary_outputs is not used.

Suggested change
def _has_pesummary_outputs(webdir: Path) -> bool:
"""Detect PESummary outputs when the default sentinel is missing."""
posterior = webdir / "samples" / "posterior_samples.h5"
if posterior.exists():
return True
# Accept legacy pesummary.dat as fallback
legacy = webdir / "samples" / f"{webdir.parent.name}_pesummary.dat"
if legacy.exists():
return True
return False

Copilot uses AI. Check for mistakes.
Comment on lines +10 to +14
from asimov.pipelines.testing import (
SimpleTestPipeline,
SubjectTestPipeline,
ProjectTestPipeline
)
Copy link

Copilot AI Jan 22, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Import of 'SubjectTestPipeline' is not used.
Import of 'ProjectTestPipeline' is not used.

Copilot uses AI. Check for mistakes.
result_pages = []
if webdir and rundir:
# Extract just the directory name from the full rundir path
import os
Copy link

Copilot AI Jan 22, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This import of module os is redundant, as it was previously imported on line 5.

Suggested change
import os

Copilot uses AI. Check for mistakes.
result_pages = []
if webdir and rundir:
# Extract just the directory name from the full rundir path
import os
Copy link

Copilot AI Jan 22, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This import of module os is redundant, as it was previously imported on line 5.

Suggested change
import os

Copilot uses AI. Check for mistakes.
+ f" {prod.name} refresh failed: {exc}"
)
except Exception:
pass
Copy link

Copilot AI Jan 22, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

'except' clause does nothing but pass and there is no explanatory comment.

Suggested change
pass
logger.exception(
"Unexpected error while processing production %s for event %s",
getattr(prod, "name", "<unknown>"),
getattr(event, "name", "<unknown>"),
)

Copilot uses AI. Check for mistakes.
raise ValueError("Test exception")

# Verify that the subject was not saved
events = project.get_event()
Copy link

Copilot AI Jan 22, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This statement is unreachable.

Copilot uses AI. Check for mistakes.
…matrices (#75)

* Initial plan

* Implement strategy expansion functionality with tests

Co-authored-by: transientlunatic <4365778+transientlunatic@users.noreply.github.com>

* Add documentation and examples for strategy feature

Co-authored-by: transientlunatic <4365778+transientlunatic@users.noreply.github.com>

* Address code review feedback - fix input mutation and naming issues

Co-authored-by: transientlunatic <4365778+transientlunatic@users.noreply.github.com>

* Use fully qualified parameter names in strategy templates

Co-authored-by: transientlunatic <4365778+transientlunatic@users.noreply.github.com>

* Refactor event handling in apply_page to use event_obj for consistency

* Add validation and error handling for strategy parameters

Co-authored-by: transientlunatic <4365778+transientlunatic@users.noreply.github.com>

* Update documentation with validation notes and boolean handling

Co-authored-by: transientlunatic <4365778+transientlunatic@users.noreply.github.com>

* Fix parameter names in strategy examples for consistency and clarity

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: transientlunatic <4365778+transientlunatic@users.noreply.github.com>
Co-authored-by: Daniel Williams <daniel.williams@ligo.org>
Copilot AI and others added 16 commits January 22, 2026 15:56
…ent execution (#40)

* Initial plan

* Add LALInference testing to HTCondor workflow

Co-authored-by: transientlunatic <4365778+transientlunatic@users.noreply.github.com>

* Enable concurrent execution of Bilby and LALInference jobs

Co-authored-by: transientlunatic <4365778+transientlunatic@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: transientlunatic <4365778+transientlunatic@users.noreply.github.com>
#43)

* Initial plan

* Implement lazy logging to prevent log file creation for help/version commands

Co-authored-by: transientlunatic <4365778+transientlunatic@users.noreply.github.com>

* Add log rotation to prevent large log files

Co-authored-by: transientlunatic <4365778+transientlunatic@users.noreply.github.com>

* Add tests for logging interface improvements

Co-authored-by: transientlunatic <4365778+transientlunatic@users.noreply.github.com>

* Add tests/tmp/ to .gitignore

* Address code review feedback - add permissions and warning logging

Co-authored-by: transientlunatic <4365778+transientlunatic@users.noreply.github.com>

* Fix logging config key mismatch - use 'location' instead of 'directory'

Co-authored-by: transientlunatic <4365778+transientlunatic@users.noreply.github.com>

* Fix remaining references to logging.directory - change to logging.location

Co-authored-by: transientlunatic <4365778+transientlunatic@users.noreply.github.com>

* Add debug output to test_init_creates_log_in_logs_directory for CI diagnosis

Co-authored-by: transientlunatic <4365778+transientlunatic@users.noreply.github.com>

* Fix config reload issue - reload config after project creation to pick up new settings

Co-authored-by: transientlunatic <4365778+transientlunatic@users.noreply.github.com>

* Fix test isolation issue - reset global _file_handler in setUp

Co-authored-by: transientlunatic <4365778+transientlunatic@users.noreply.github.com>

* Fix logging.directory reference in project.py after merge

Co-authored-by: transientlunatic <4365778+transientlunatic@users.noreply.github.com>

* Address code review feedback: improve error handling, thread safety, and test coverage

Co-authored-by: transientlunatic <4365778+transientlunatic@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: transientlunatic <4365778+transientlunatic@users.noreply.github.com>
Co-authored-by: Daniel Williams <daniel.williams@ligo.org>
Co-authored-by: Daniel Williams <mail@daniel-williams.co.uk>
* Add prior ranges to all priors

* Increase timeout and interval for Bilby result monitoring

* Increase timeout for Bilby result monitoring from 1200 to 1800 seconds
* Start work on making a blueprint validator.

* More updates to the schemafication

* Add more blueprint types.

* Apply suggestions from code review

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Add initial implementation of the scheduler module

* Start working on the scheduler.

* Initial plan

* Complete scheduler module implementation with HTCondor and Slurm placeholder

Co-authored-by: transientlunatic <4365778+transientlunatic@users.noreply.github.com>

* Refactor condor.py to use new scheduler module with backward compatibility

Co-authored-by: transientlunatic <4365778+transientlunatic@users.noreply.github.com>

* Add .gitignore and remove __pycache__ files

Co-authored-by: transientlunatic <4365778+transientlunatic@users.noreply.github.com>

* Add scheduler configuration documentation and usage examples

Co-authored-by: transientlunatic <4365778+transientlunatic@users.noreply.github.com>

* Address code review feedback: improve exception handling and resource mapping

Co-authored-by: transientlunatic <4365778+transientlunatic@users.noreply.github.com>

* Fix duplicate code and use specific HTCondor exceptions

Co-authored-by: transientlunatic <4365778+transientlunatic@users.noreply.github.com>

* Fix resource mapping usage and replace all bare except clauses

Co-authored-by: transientlunatic <4365778+transientlunatic@users.noreply.github.com>

* Add scheduler integration to pipelines and monitor loop

Co-authored-by: transientlunatic <4365778+transientlunatic@users.noreply.github.com>

* Fix create_job_from_dict to not modify input dictionary

Co-authored-by: transientlunatic <4365778+transientlunatic@users.noreply.github.com>

* Refactor pipeline DAG submission to use scheduler API instead of CLI

Co-authored-by: transientlunatic <4365778+transientlunatic@users.noreply.github.com>

* Fix code review issues: move import to top and remove redundant path construction

Co-authored-by: transientlunatic <4365778+transientlunatic@users.noreply.github.com>

* Integrate queue monitoring into scheduler interface with JobList class

Co-authored-by: transientlunatic <4365778+transientlunatic@users.noreply.github.com>

* Fix code review issues: remove duplicate imports and ensure consistent job storage

Co-authored-by: transientlunatic <4365778+transientlunatic@users.noreply.github.com>

* Apply suggestions from code review

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Apply suggestions from code review

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Add htcondor2 compatibility with modern API migration (#80)

* Initial plan

* Add htcondor2/classad2 compatibility to scheduler and pipeline modules

Co-authored-by: transientlunatic <4365778+transientlunatic@users.noreply.github.com>

* Update to htcondor2-compatible API using schedd.submit() instead of transaction/queue

Co-authored-by: transientlunatic <4365778+transientlunatic@users.noreply.github.com>

* Improve exception handling to include HTCondor-specific errors

Co-authored-by: transientlunatic <4365778+transientlunatic@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: transientlunatic <4365778+transientlunatic@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: transientlunatic <4365778+transientlunatic@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Enhance apply_page function to support dynamic ledger initialization and improve event handling in analysis bundles

* Address code review feedback: improve error handling, security, and code organization (#77)

* Initial plan

* Address PR review comments: improve imports, naming, error handling, and security

Co-authored-by: transientlunatic <4365778+transientlunatic@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: transientlunatic <4365778+transientlunatic@users.noreply.github.com>

* Fix UnboundLocalError from bad merge in event handling (#79)

* Initial plan

* Fix merge-related UnboundLocalError by correcting variable names

Co-authored-by: transientlunatic <4365778+transientlunatic@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: transientlunatic <4365778+transientlunatic@users.noreply.github.com>

---------

Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com>
Co-authored-by: transientlunatic <4365778+transientlunatic@users.noreply.github.com>
* Refactor olivaw command to enforce project presence in ProjectAwareGroup

* Update CHANGELOG for 0.7.0-alpha2 release with new features and fixes
* Initial plan

* Add minimum frequency field to Waveform and update validation

Co-authored-by: transientlunatic <4365778+transientlunatic@users.noreply.github.com>

* Update config templates to use waveform minimum frequency

Co-authored-by: transientlunatic <4365778+transientlunatic@users.noreply.github.com>

* Move validation before pipeline init and improve error handling

Co-authored-by: transientlunatic <4365778+transientlunatic@users.noreply.github.com>

* Fix meta_defaults access in validation code

Co-authored-by: transientlunatic <4365778+transientlunatic@users.noreply.github.com>

* Fix type definition and add waveform variable to templates

- Change minimum_frequency type to dict[str, float] only (remove float option)
- Add waveform variable assignment to bilby.ini and lalinference.ini templates
- Use waveform shorthand in lalinference.ini for consistency

Co-authored-by: transientlunatic <4365778+transientlunatic@users.noreply.github.com>

* Add robust error handling for minimum frequency validation

- Add type checking in bayeswave.py flow property
- Add type checking in pesummary.py for both SubjectAnalysis and single analysis paths
- Add type checking in pipeline.py PESummaryPipeline
- Ensure all code paths validate that minimum frequency is a non-empty dict before calling .values()

Co-authored-by: transientlunatic <4365778+transientlunatic@users.noreply.github.com>

* Split external blueprint tests to separate CI job

- Create local copies of external blueprints with correct v0.7 waveform structure
  in tests/test_data/blueprints/
- Refactor test_specific_events.py to use local blueprints (must pass) with
  __file__-based absolute paths to be robust against cwd pollution from other tests
- Add tests/external_blueprint_compat.py for external URL tests (not auto-discovered
  by unittest discover since file doesn't match test*.py pattern)
- Add test-external-blueprints CI job to python-app.yml with continue-on-error: true
  (runs on single Python version only)
- Fix bayeswave.py: use setdefault() for quality dict to avoid KeyError when quality
  section doesn't exist in meta
- Fix analysis.py: calculate maximum frequency even when quality section is absent
  from event blueprint

Co-authored-by: transientlunatic <4365778+transientlunatic@users.noreply.github.com>

* Update changelog and fix all tests to use local blueprints

- Add 'Unreleased' section to CHANGELOG.rst documenting the breaking change
  for minimum frequency location with migration guide and PR reference
- Update tests/blueprints.py helper module with absolute paths to local blueprint files
- Create tests/test_data/blueprints/production-pe-priors.yaml (local copy)
- Create tests/test_data/blueprints/gwtc-2-1/GW150914_095045.yaml (v0.7 compliant)
- Update test_dag.py, test_cli_manage.py, test_yaml.py, test_review.py, and
  tests/test_pipelines/test_bilby.py to use local blueprints via the blueprints module
  instead of external git.ligo.org URLs

Co-authored-by: transientlunatic <4365778+transientlunatic@users.noreply.github.com>

* Fix test_build_submit_dryruns: add copy frames and osg to bayeswave defaults

The local production-pe.yaml was missing copy frames: True and osg: True in
the bayeswave scheduler section. These settings are present in the external
production-pe.yaml and are needed for the --copy-frames and --transfer-files
flags to appear in the BayesWave build_dag dryrun output, which
test_build_submit_dryruns asserts are present.

Co-authored-by: transientlunatic <4365778+transientlunatic@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: transientlunatic <4365778+transientlunatic@users.noreply.github.com>
* Fix the application of analyses

* Updated event apply
* Improve cache handling in CondorJobList and JobList to manage YAML loading errors

* Apply suggestions from code review

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

releases PRs which make releases

Projects

Status: Todo

Development

Successfully merging this pull request may close these issues.

3 participants