-
Notifications
You must be signed in to change notification settings - Fork 65
Open
Description
Problem Description
Once #515 is resolved, it should be possible to run the multi-table SDGym benchmark on GCP using enterprise synthesizers. The goal of this issue is to add GitHub Actions workflows that automatically run the multi-table benchmark and upload the results, mirroring the existing automation for single-table benchmarks.
Expected behavior
- Add a
run_benchmark_multi_table.yamlfile - Add an u
pload_benchmark_results_multi_table.yaml
Additional context
The new workflows should closely follow the existing single-table benchmark workflows, with the following differences:
- Credentials handling:
- Retrieve AWS, GCP, and SDV Enterprise
credentialsfrom SDGym GitHub secrets - Prepare these credentials in a format compatible with _benchmark_multi_table_compute_gcp
- Retrieve AWS, GCP, and SDV Enterprise
- Benchmark execution:
- Use
_benchmark_multi_table_compute_gcpto launch the benchmark runs - Launch:
- One GCP instance running HMASynthesizer
- One GCP instance running HSASynthesizer and IndependentSynthesizer
- Use
- Result handling:
- Benchmark artifacts and results are uploaded to AWS (S3), consistent with the existing SDGym workflow
Metadata
Metadata
Assignees
Labels
feature requestRequest for a new featureRequest for a new feature