"Building an ExecuTorch iOS Demo App" Feedback

[jathu]

📚 The doc issue

• The doc is outdated since we do not need to build ExecuTorch from source for CoreML and XNNPACK backends anymore
• Some of the steps are confusing and out of order. For example, the doc suggests to download two repos (et, and et-examples), but then doesn't mention where to run the example commands
• The final Xcode project does not build

Suggest a potential alternative/fix

• Usepip install executorch and just focus on XNNPACK and CoreML. Remove references to MPS (for now)
• Restructure the steps to be more clear and verbose on which folder/repo we should run commands
• Fix the failing Xcode build project

[metascroy]

@shoumikhin is this something you could help clean up?

[shoumikhin]

@jathu can you elaborate please what's up with MPS? The demo app needs a model exported with MPS to run inference and tests.
Doc link for clarity.
Also, can you elaborate on no need to build Core ML or XNNPACK backends anymore?
Did you try to run the commands exactly as outlines in the doc in that particular order? Which one had failed?

[jathu]

@shoumikhin

can you elaborate please what's up with MPS? The demo app needs a model exported with MPS to run inference and tests.

I was just suggesting we should remove the MPS export example to keep the instructions simple. Perhaps we can add another section at the bottom that expands on how to export an MPS backend?

Also, can you elaborate on no need to build Core ML or XNNPACK backends anymore?

We now include xnnpack and coreml exporting support in the macOS pip packages. So just be running pip install executorch, people should be able to export to those two backends

Did you try to run the commands exactly as outlines in the doc in that particular order? Which one had failed?

All the commands executed successfully, but what did fail was opening the Xcode project (open $APP_PATH.xcodeproj) and building the project. It was failing with libraries not found error

[shoumikhin]

@jathu sorry, still not clear on MPS part. To provide more context, the demo app features the MPS support along with Core ML and XNNPACK. You can see it on a screenshot as one of the modes the app can run inference on the images. If we remove it, we drop that feature pretty much. Also the app has tests for MPS that we run as part of the CI.

I guess I'm missing some recent updates on installation flow. You're saying users may not need to clone ET at all now, but just pip install it? And that it comes with all the extra deps for Core ML (like coremltools) and XNNPACK with it, able to export models to those backends? Does that mean that ideally pip install executorch and then python3 -m examples.apple.coreml.scripts.export -m <model name> should be enough?

And given the PyPI doesn't come with a prebuilt MPS backend export tools (I guess?), that means we still gotta clone and install ET?

[jathu]

@shoumikhin I see, I didn't realize the demo app requires the MPS export as well. Are you willing to consider stripping MPS out in favor of keeping the demo as simple as possible for new developers? So the demo app will only contain xnnpack + coreml. cc @mergennachin @byjlw

You're saying users may not need to clone ET at all now, but just pip install it?

Yes, on macOS it will install coremltools as a dependency so you should be able to export models out of the box. See: #9483 (CoreML), and #9773 (xnnpack)

[mergennachin]

@shoumikhin

Yeah, pip install executorch nows install coremltools automatically. We don't need to do build from source at all when exporting a model.

You can test by doing this today:

pip install --extra-index-url https://download.pytorch.org/whl/test/cpu executorch==0.6.0 torch==2.7.0 torchaudio==2.7.0 torchvision==0.22.0

Can we have two versions of the app with CoreML+XNNPACK only (which means, people don't need to build clone at all. just use pip and SwiftPM) and MPS version (which requires people to clone and build from source).

[shoumikhin]

On other hand, how hard is it to add the necessary code from MPS backend to executorch PyPI? Iirc, they don't have much deps.

[shoumikhin]

@mergennachin I mean technically we can do that, e.g. add #ifdefs and stuff to the app code, change how xcode project bundles resources (exported models), fix CI, etc. and properly test both flows: w/ and w/o MPS.
Although, think about it: in the readme we'll now have to describe the two unrelated setup flows instead of one: pip install and git clone. Where for the latter we'll need to add an extra export command for MPS and also mention that users gotta set a particular #define to get MPS-related code compiled too.
Maybe we can leave the demo app workflow as is for this release and add MPS into PyPI for the next?

[shoumikhin]

Leaving the demo app as is, ie. building ET from source, due to the lack of MPS support in PyPI package, since changing the app at this point would take more effort.