Skip to content

Add workflow to run SDGym multi-table benchmark monthly and publish results #516

@R-Palazzo

Description

@R-Palazzo

Problem Description

Once #515 is resolved, it should be possible to run the multi-table SDGym benchmark on GCP using enterprise synthesizers. The goal of this issue is to add GitHub Actions workflows that automatically run the multi-table benchmark and upload the results, mirroring the existing automation for single-table benchmarks.

Expected behavior

  • Add a run_benchmark_multi_table.yaml file
  • Add an upload_benchmark_results_multi_table.yaml

Additional context

The new workflows should closely follow the existing single-table benchmark workflows, with the following differences:

  • Credentials handling:
    • Retrieve AWS, GCP, and SDV Enterprise credentials from SDGym GitHub secrets
    • Prepare these credentials in a format compatible with _benchmark_multi_table_compute_gcp
  • Benchmark execution:
    • Use _benchmark_multi_table_compute_gcp to launch the benchmark runs
    • Launch:
      • One GCP instance running HMASynthesizer
      • One GCP instance running HSASynthesizer and IndependentSynthesizer
  • Result handling:
    • Benchmark artifacts and results are uploaded to AWS (S3), consistent with the existing SDGym workflow

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

Relationships

None yet

Development

No branches or pull requests

Issue actions