Feature Request: Support for OpenPipeline Configurations

[flo-mair]

What would you like?

Support to validate matchers and DQL in OpenPipeline
Create OpenPipeline Configurations (via settings)
Read for OpenPipeline Configurations (via settings)

Example usage

dtctl exec openpipeline --dql (--matcher)'<some DQL>'

dtctl apply openpipeline --routing (--pipeline, --ingest-source) -f config.json

dtctl get openpipeline --routing (--pipeline, --ingest-source)

[WulfAI]

Strong +1 on this. OpenPipeline config management via CLI would be a significant enabler for automated observability pipelines.

One use case that makes this especially compelling: agent observability at scale. When running multi-agent systems (think: fleets of AI agents with different roles — orchestration, execution, quality gates), each agent type ideally routes through its own OpenPipeline configuration for specialized processing, enrichment, and routing.

Right now in agentic-enterprise (an open-source framework for governing multi-agent organizations), we're defining an OTel-first observability contract that specifies correlation attributes like agent.layer, mission.id, agent.type etc. The natural ingestion path for these traces is through OpenPipeline custom endpoints (as discussed in dtctl#44).

Being able to dtctl apply openpipeline would close the loop: you could version-control your OpenPipeline configs alongside your agent definitions, and dtctl apply them as part of your deployment pipeline. Config-as-code all the way down.

A few thoughts on the proposed interface:

• dtctl exec openpipeline --dql for validation is great — dry-run validation before apply is essential when you're automating pipeline config changes
• For dtctl get openpipeline, it would be useful to support output formats (JSON/YAML) that can be round-tripped back through dtctl apply — enables "export → modify → apply" workflows
• Would dtctl diff openpipeline make sense as a companion? Showing what would change before applying, similar to Terraform plan

cc @discostu105 — this connects well to the dtctl send event discussion in #44.

[discostu105]

Yes this makes sense. Contribution would be welcome. Please api-design first.

OpenPipeline is managed via Settings which are already managable via dtctl. However, putting conveniece functions on top of it makes sense.