Executorch OSS CI tracking issue (2023)

[huydhn]

I'm creating this initial issue to keep track of executorch OSS CI progress in 2023. P0 needs to be done before Oct.

• (P0) Integrate executorch with GHA and PyTorch-org shareable runners (AWS, linux). This is needed to provide the infrastructure to run OSS CI.
• (P0) Integrate executorch with GitHub webhook and Rockset DB. This is needed to keep track of which is run on OSS CI.
• (P0) Setup e2e build and test on Linux runner. This requires BUCK OSS to run.
  • (P0) Check with Executorch team if they need support on getting the BUCK OSS setup.
  • (P0) Also setup one for Mac M1
  • (P0) Add e2e testing for the rest of the sample models T160756042
• (P1) Setup devices (Android, iOS) to run benchmark tests. The prefer option is https://aws.amazon.com/device-farm
• (P1) Sync with Executorch and TorchBench on how to benchmark Executor.
• (P1) Setup benchmark CI for Executor.
• (P1) Add OSS alert for Executorch when CI workflow fails

---

This is probably not needed for now, only list them here for reference.

• (P2) Linter
• (P2) Binaries release
• (P2) GitHub First features such as merge bot, Dr.CI

cc @malfet @seemethere @mergennachin @dbort @clee2000

[huydhn]

Please feel free to update the issue or post your comments if there are other items that you have in mind.

[larryliu0820]

(P0) Check with Executorch team if they need support on getting the BUCK OSS setup.

This is done just now. We have BUCK2 working correctly and we are able to run the following command:

buck2 build //test:size_test_all_ops

We should add this to CI.

[huydhn]

Thank @dbort and @cccclai for the setup doc https://www.internalfb.com/intern/staticdocs/executorch/docs/tutorials/setting_up_executorch/

[huydhn]

Executorch goes public today, so we can close this one as the first phase is done