Skip to content

Conversation

@huydhn
Copy link
Contributor

@huydhn huydhn commented Sep 27, 2024

This updates the mobile job to return the list of artifacts from AWS to the caller to use as they see fit. One of the use case is for ExecuTorch to extract the benchmark results JSON from CUSTOMER_ARTIFACT and TESTSPEC_OUTPUT (2 types of artifacts from AWS).

My plan here is to:

  • Get the list of artifacts (this PR)
  • On ExecuTorch side, extract the benchmark JSON from the artifacts Upload Android benchmark results to OSS benchmark database executorch#5808
  • Create a new GitHub action on test-infra to upload the JSON (This could be done on ExecuTorch side too, but I plan to make this generic and reusable by other projects, so it must be on test-infra)

Minor fixes:

  • Also update the console output to print out the context of TESTSPEC_OUTPUT, which is basically the main output from the device.
  • Print additional information about the job name (the device name from AWS in disguise) and app type (Android, iOS)
  • Some lint fix here and there.

@vercel
Copy link

vercel bot commented Sep 27, 2024

The latest updates on your projects. Learn more about Vercel for Git ↗︎

1 Skipped Deployment
Name Status Preview Comments Updated (UTC)
torchci ⬜️ Ignored (Inspect) Visit Preview Oct 3, 2024 0:33am

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Sep 27, 2024
@huydhn huydhn changed the title Upload mobile benchmark results Return the list of artifacts from mobile job Sep 28, 2024
@huydhn huydhn marked this pull request as ready for review September 28, 2024 04:34
@huydhn huydhn requested a review from clee2000 October 1, 2024 19:42
huydhn added a commit to pytorch/executorch that referenced this pull request Oct 2, 2024
@huydhn huydhn merged commit 0b034ed into main Oct 3, 2024
9 checks passed
@huydhn huydhn mentioned this pull request Oct 3, 2024
4 tasks
huydhn added a commit that referenced this pull request Oct 5, 2024
This adds a GHA to upload benchmark results. I'm using the existing
`torchci-oss-ci-benchmark` database for now until the new generic
database from https://fburl.com/gdoc/ossgtvte is ready. So, it's backed
by dynamoDB instead of S3 for now.

I also include a quick test with some dummy Android benchmark records.

* [x] Get the list of artifacts
#5727
* [x] On ExecuTorch side, extract the benchmark JSON from the artifacts
pytorch/executorch#5808
* [x] Create a new GitHub action on test-infra to upload the JSON
#5742
* [x] Create a new GHA role to upload the results
meta-pytorch/pytorch-gha-infra#483
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants