[CUTLASS] Refactor cutlass kernel generation and selection

[masahi]

Currently, when we enumerate cutlass kernels for profiling, for each parameter config we generate all variants of the kernel with different epilogues. See for example

tvm/python/tvm/contrib/cutlass/gen_gemm.py

Lines 67 to 106 in 1afcf36

	op = GemmOperation(
	tile_description.minimum_compute_capability,
	tile_description,
	A,
	B,
	C,
	element_epilogue,
	EpilogueFunctor.LinearCombination,
	swizzling_functor,
	)
	op_bias = GemmOperation(
	tile_description.minimum_compute_capability,
	tile_description,
	A,
	B,
	C,
	element_epilogue,
	EpilogueFunctor.LinearCombinationBias,
	swizzling_functor,
	)
	op_bias_relu = GemmOperation(
	tile_description.minimum_compute_capability,
	tile_description,
	A,
	B,
	C,
	element_epilogue,
	EpilogueFunctor.LinearCombinationRelu,
	swizzling_functor,
	)
	op_bias_gelu = GemmOperation(
	tile_description.minimum_compute_capability,
	tile_description,
	A,
	B,
	C,
	element_epilogue,
	EpilogueFunctor.LinearCombinationGelu,
	swizzling_functor,
	)

After profiling, we select which variant of epilogue to use based on the pattern name:

tvm/python/tvm/contrib/cutlass/build.py

Lines 219 to 230 in 1afcf36

	if op_type == "cutlass.conv2d":
	cutlass_op_def = out["opdef"]
	elif op_type == "cutlass.conv2d_bias":
	cutlass_op_def = out["opdef_bias"]
	elif op_type == "cutlass.conv2d_bias_relu":
	cutlass_op_def = out["opdef_bias_relu"]
	elif op_type == "cutlass.conv2d_bias_sigmoid":
	cutlass_op_def = out["opdef_bias_sigmoid"]
	elif op_type == "cutlass.conv2d_bias_silu":
	cutlass_op_def = out["opdef_bias_silu"]
	elif op_type == "cutlass.conv2d_bias_hardswish":
	cutlass_op_def = out["opdef_bias_hardswish"]

This approach simply doesn't work when we introduce support for residual connection fusion, because there are so many different kinds of epilogues.

The idea of this change is to split kernel generation into two steps:
(1) First, we generate all kernels without any epilogue. This is used for profiling
(2) After profiling decides the best parameter configuration, use that information to generate a single kernel with the required epilogue.

Overall I believe this refactoring of kernel generation and selection have made things much cleaner, and makes us well-prepared for residual block fusion.

cc @comaniac @Laurawly

[masahi]

@comaniac Can you take a look (no functional change, should be easy)? The cutlass side change to enable residual block fusion was merged yesterday in NVIDIA/cutlass#391, so I'm ready to send residual fusion support (with good speed up!)

[Laurawly]

LGTM

[comaniac]

Sorry I was on vacation. Thanks @masahi @Laurawly