[microNPU][ETHOSU] Channel pad offloaded to NPU#14765
Conversation
| return None | ||
| if ( | ||
| list(pad_width[0]) != [0, 0] | ||
| or list(pad_width[1]) != [0, 0] |
There was a problem hiding this comment.
Are there networks that have paddings in height, width and channels? If there are such, then it would be possible to remove width and height restrictions and add width and height padding processing to the legalization using depthwise convolution as it is done for pad2d.
There was a problem hiding this comment.
Yes, you are right, spatial and channel pad can of course occur in neural networks. This is a separate task, we discussed that it will be useful in the future. We plan to solve it when we have time or a network with such pad appears.
|
@tvm-bot rerun |
ekalda
left a comment
ekalda
left a comment
There was a problem hiding this comment.
Thanks @sergio-grovety! Left some comments...
|
|
||
|
|
||
| class ChannelPadRewriter(DFPatternCallback): | ||
| """Convert ethos-u.pad2d composite function to the Relay concatenate operation""" |
There was a problem hiding this comment.
| """Convert ethos-u.pad2d composite function to the Relay concatenate operation""" | |
| """Convert ethos-u.channel-pad composite function to the Relay concatenate operation""" |
| params.ifm.tensor = post.args[0] | ||
|
|
||
| concat_args = list() | ||
| # Activations requiring LUT is currently not supported, so setting it to an empty list |
There was a problem hiding this comment.
I know every operator here has this copy pasted legacy comment, but let's remove it... Firstly LUT based activations are supported and secondly it could leave an impression that implementing something like fused pad + sigmoid is a TODO.
| axis = 3 | ||
| return relay.op.concatenate(relay.Tuple(concat_args), axis=axis) |
There was a problem hiding this comment.
Since it is not used elsewhere, maybe just
| axis = 3 | |
| return relay.op.concatenate(relay.Tuple(concat_args), axis=axis) | |
| return relay.op.concatenate(relay.Tuple(concat_args), axis=3) |
|
|
||
| def are_pad_on_graph(self, subgraph) -> bool: | ||
| """ | ||
| This function recursively visits the graph and checks if 'nn.pad' op is ongraph |
There was a problem hiding this comment.
Nit:
| This function recursively visits the graph and checks if 'nn.pad' op is ongraph | |
| This function recursively visits the graph and checks if 'nn.pad' op is on graph |
| mod = partition_ethosu_by_table(mod, conv2d_pattern_table) | ||
|
|
||
| mod["tvmgen_default_ethos_u_main_0"] = dataflow_pattern.rewrite( | ||
| legalize.Conv2DRewriter(), mod["tvmgen_default_ethos_u_main_0"] | ||
| ) | ||
|
|
||
| verify(mod["tvmgen_default_ethos_u_main_0"]) |
There was a problem hiding this comment.
I'm a bit confused about what that test does... It creates a TFLite graph with channel pad and conv2d, then partitions them for microNPU, then legalizes the conv2d into ethosu.conv2d and then checks that the Relay pad is still there?
There was a problem hiding this comment.
Hi @ekalda, thank you for the review!
Yes, it is. Here we check that the pad by channel does not merge with conv2d, as it happens with the spatial pad.
| @pytest.mark.parametrize("ifm_shape", [(1, 55, 55, 3), (1, 23, 32, 7)]) | ||
| @pytest.mark.parametrize("channel_padding", [(0, 1), (1, 1), (5, 2)]) | ||
| @pytest.mark.parametrize("const_value", [0, 5, 125, -5]) | ||
| def test_tflite_separate_channel_padding_legalize(ifm_shape, channel_padding, const_value): |
There was a problem hiding this comment.
Shouldn't this test be using ChannelPadRewriter and then check in verify that the concatenates got created?
There was a problem hiding this comment.
Thanks for the comment, the test really did not correspond to the tested functionality. I corrected the test.
|
@tvm-bot rerun |
|
Thanks @arina-grovety, @sergio-grovety and @Aleksei-grovety, this is now merged! |
A separate channel-dimension nn.pad relay operator is rewritten as Relay concatenate operation. --------- Co-authored-by: Sergey Smirnov <89378719+sergey-grovety@users.noreply.github.com> Co-authored-by: arina.naumova <naumova@grovety.com>
5 participants
A separate channel-dimension nn.pad relay operator is rewritten as Relay concatenate operation.