-
Notifications
You must be signed in to change notification settings - Fork 960
Partitioner to not delegate mixed dtype operators #9023
Copy link
Copy link
Closed
keyprocedure/executorch
#1Labels
good first issueGood for newcomersGood for newcomersmodule: xnnpackIssues related to xnnpack delegation and the code under backends/xnnpack/Issues related to xnnpack delegation and the code under backends/xnnpack/triagedThis issue has been looked at a team member, and triaged and prioritized into an appropriate moduleThis issue has been looked at a team member, and triaged and prioritized into an appropriate module
Milestone
Metadata
Metadata
Assignees
Labels
good first issueGood for newcomersGood for newcomersmodule: xnnpackIssues related to xnnpack delegation and the code under backends/xnnpack/Issues related to xnnpack delegation and the code under backends/xnnpack/triagedThis issue has been looked at a team member, and triaged and prioritized into an appropriate moduleThis issue has been looked at a team member, and triaged and prioritized into an appropriate module
Type
Projects
Status
Done
Status
Done
Problem
Currently there is no check that the XnnpackPartitioner does not partition mixed dtype ops for example a multiply that takes in a bf16 and fp32 input (these are not supported by xnnpack).
Context
In the XnnpackPartitioner we have configs for operators that describe how operators should be partitioned, and what constraints they must pass. The below link should show constraints that must be passed by all operators
executorch/backends/xnnpack/partition/config/xnnpack_config.py
Line 116 in 1a9a59b
We should add a check in this constraint to fail partitions that take in mixed dtypes as inputs. The refactoring done in this issue might be a good way to help enforce this more easily for binary inputs. #9024
Verification/Testing
Add a test with a pytorch module for operations that take in different dtypes (fp32, bf16, fp16, int8). Check that these are not partitioned:
Inverse of this:
executorch/backends/arm/test/ops/test_cat.py
Lines 69 to 70 in 1a9a59b
in that we want to check that there are no call_delegate nodes and the original node is still in the graph
Test for the following representative ops:
Resources
https://discord.gg/a9r5KZDNfZ
cc @digantdesai @cbilgin