[Flaky] test-demo-ios / macos-job

[guangy10]

I've seen this job being flaky in many PRs. For example,

• [Ez] Fix vulkan_backend linking in Android demo app #2872
• delete test_quant_lowering_custom_backend_pass.py as there are no tests in it #2809
• integrate qnn backend to runner #2806

2024-04-05T18:47:27.4678510Z 	MobileNetClassifierTest.testV3WithCoreMLBackend()
2024-04-05T18:47:27.4678840Z 
2024-04-05T18:47:27.4678950Z ** TEST FAILED **
2024-04-05T18:47:27.4679220Z 
2024-04-05T18:47:27.4679280Z Testing started
2024-04-05T18:47:27.4679840Z Test suite 'MobileNetClassifierTest' started on 'Clone 1 of executorch - xctest (83121)'
2024-04-05T18:47:27.4680680Z Test case 'MobileNetClassifierTest.testV3WithCoreMLBackend()' failed on 'Clone 1 of executorch - xctest (83121)' (5.452 seconds)
2024-04-05T18:47:27.4681790Z Test case 'MobileNetClassifierTest.testV3WithMPSBackend()' passed on 'Clone 1 of executorch - xctest (83121)' (10.237 seconds)
2024-04-05T18:47:27.4682870Z Test case 'MobileNetClassifierTest.testV3WithPortableBackend()' passed on 'Clone 1 of executorch - xctest (83121)' (1.780 seconds)
2024-04-05T18:47:27.4684040Z Test case 'MobileNetClassifierTest.testV3WithXNNPACKBackend()' passed on 'Clone 1 of executorch - xctest (83121)' (0.424 seconds)
2024-04-05T18:47:27.4684680Z 
2024-04-05T18:47:27.4685100Z 	** Deleting Simulator **
2024-04-05T18:47:27.4685470Z 
2024-04-05T18:47:27.4685800Z 
2024-04-05T18:47:27.4686100Z 	** FAILED **

[cymbalrush]

@guangy10 are you still seeing this?

[guangy10]

Per the HUD https://hud.pytorch.org/hud/pytorch/executorch/main/1?per_page=50&name_filter=test-demo-ios&mergeLF=true, it's been running pretty stable. Close now