Image.filter gives wrong result when using asymmetric kernels

[Perfec-specops]

Problem

When filtering an array like [[0,0,0],[0,1,0],[0,0,0]](the central element is 1 and the others are 0) with a kernel like [1, 2, 3, 4, 5, 6, 7, 8, 9], a correlation will generate [[9,8,7],[6,5,4],[3,2,1]] and a convolution will generate [[1, 2, 3], [4, 5, 6], [7, 8, 9]], if we don't consider the border.

However, Pillow generates neither of them.

This behavior won't affect results of symmetric kernels we use all the time, so only in some rare cases it will cause wrong result.

from PIL import Image, ImageFilter
import numpy as np

kernel=[1, 2, 3, 4, 5, 6, 7, 8, 9]
image=np.array([
       [ 0.,  0.,  0.,  0.,  0.,  0.,  0.],
       [ 0.,  0.,  0.,  0.,  0.,  0.,  0.],
       [ 0.,  0.,  0.,  0.,  0.,  0.,  0.],
       [ 0.,  0.,  0.,  1.,  0.,  0.,  0.],
       [ 0.,  0.,  0.,  0.,  0.,  0.,  0.],
       [ 0.,  0.,  0.,  0.,  0.,  0.,  0.],
       [ 0.,  0.,  0.,  0.,  0.,  0.,  0.]], 'uint8')

img=Image.fromarray(image,'L')
img=img.filter(ImageFilter.Kernel((3,3),kernel,1))

print(np.array(img))

# result:
#[[0 0 0 0 0 0 0]
# [0 0 0 0 0 0 0]
# [0 0 3 2 1 0 0]
# [0 0 6 5 4 0 0]
# [0 0 9 8 7 0 0]
# [0 0 0 0 0 0 0]
# [0 0 0 0 0 0 0]]

# expect:
# the correlation
#[[ 0.  0.  0.  0.  0.  0.  0.]
# [ 0.  0.  0.  0.  0.  0.  0.]
# [ 0.  0.  9.  8.  7.  0.  0.]
# [ 0.  0.  6.  5.  4.  0.  0.]
# [ 0.  0.  3.  2.  1.  0.  0.]
# [ 0.  0.  0.  0.  0.  0.  0.]
# [ 0.  0.  0.  0.  0.  0.  0.]]
# or the convolution
#[[ 0.  0.  0.  0.  0.  0.  0.]
# [ 0.  0.  0.  0.  0.  0.  0.]
# [ 0.  0.  1.  2.  3.  0.  0.]
# [ 0.  0.  4.  5.  6.  0.  0.]
# [ 0.  0.  7.  8.  9.  0.  0.]
# [ 0.  0.  0.  0.  0.  0.  0.]
# [ 0.  0.  0.  0.  0.  0.  0.]]

[homm]

You are right, this is confusing behavior, I don't know the original intents why this is done this way. Many Pillow bugs are legacy from PIL and this is one of this. Recently I have rewritten kernel filter implementation but haven't fixed it for compatibility reasons.

@wiredfool Do you think we can change filter direction (and all built-in filters in ImageFilter.py) in 5.2 or we should wait for 6.0?

By the way, despite the fact that at first glance the "convolution order" looks more correct, this is not so obvious.

# convolution order
[[0 0 0 0 0 0 0]
 [0 0 0 0 0 0 0]
 [0 0 1 2 3 0 0]
 [0 0 4 5 6 0 0]
 [0 0 7 8 9 0 0]
 [0 0 0 0 0 0 0]
 [0 0 0 0 0 0 0]]

# correlation order
[[0 0 0 0 0 0 0]
 [0 0 0 0 0 0 0]
 [0 0 9 8 7 0 0]
 [0 0 6 5 4 0 0]
 [0 0 3 2 1 0 0]
 [0 0 0 0 0 0 0]
 [0 0 0 0 0 0 0]]

When you on [2,2] element, you have the kernel:

[[1, 2, 3],
 [4, 5, 6],
 [7, 8, 9]]

and the image:

[[0, 0, 0],
 [0, 0, 0],
 [0, 0, 1]]

So simply multiplying each image element on the appropriate kernel element and sum them, you'll get 9. So correlation order looks more correct and also it is simple to fix.

[radarhere]

Closing as a duplicate of #3678

[radarhere]

As per #3678 (comment), PR #7204 has documented this behaviour.