Qualcomm AI Engine Direct - Delegate mutable buffer and fix the mutable buffer issue

[shewu-quic]

Summary:

• Add a parameter to support mutable buffer delegation in QNN Backend
  • Set the same memory address for I/O of mutable buffer at runtime
  • Ref: Qualcomm AI Engine Direct - Delegated mutable buffer #6727
• Avoid annotating the input node because mutable buffers will be folded during the convert_pt2e process.
• Deprecated use_legacy_export in executorch llama

cc @cccclai @winskuo-quic @cbilgin

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/11782

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

✅ You can merge normally! (1 Unrelated Failure)

As of commit 5fde193 with merge base 44d2643 ():

BROKEN TRUNK - The following job failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

• pull / test-moshi-linux / linux-job (gh) (trunk failure)
  test_exported_decoder_xnnpack

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[github-actions]

This PR needs a release notes: label

If your change should be included in the release notes (i.e. would users of this library care about this change?), please use a label starting with release notes:. This helps us keep track and include your important work in the next release notes.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "release notes: none"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.

[cccclai]

Avoid annotating the input node because mutable buffers will be folded during the convert_pt2e process.

Is the input node still folded after we land pytorch/ao#2345?

[shewu-quic]

Avoid annotating the input node because mutable buffers will be folded during the convert_pt2e process.

Is the input node still folded after we land pytorch/ao#2345?

Yes, unless we apply run_decomposition after export. I think we can wait until run_decomposition becomes a pass and doesn't require re-tracing. After that we can change it back to annotate mutable buffer. What do you think?

[shewu-quic]

BTW, in previous, we have submitted a PR to deprecated convert_bmm_to_matmul pass. It will result in multiple partitions for Meta's llama due to not using to_edge_transform_and_lower_to_qnn. So, I add it back and set False for activation as default value.

[facebook-github-bot]

@cccclai has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

[cccclai]

I ran into this error for some internal use cases.

TypeError: LLMEdgeManager.__init__() got an unexpected keyword argument 'use_legacy_export'

Can we turn if off in this PR, and I will have another PR to remove this arg?

[facebook-github-bot]

@cccclai has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.