Hello!
I'm currently rewriting my LM-pipeline with TE. And I noticed that the following line doesn't work as expected. I.e for the first forward of my layer this cat noop function goes to this point (i.e. all checks of layout were passed), but out tensor instead of being concatenation of all weights looks like cat of tensor[0] and uninitialised memory.
After some debugs I found that tensors[i].data_ptr() + tensors[i].size(dim) * data_ptr_stride = tensors[I+1].data_ptr(), but tensors[i].untyped_storage().data_ptr() are different (but storages located sequentially). And as far as I know here is expected that tensors[i].untyped_storage().data_ptr() would be the same. Am I wrong?
Hello!
I'm currently rewriting my LM-pipeline with TE. And I noticed that the following line doesn't work as expected. I.e for the first forward of my layer this cat noop function goes to this point (i.e. all checks of layout were passed), but out tensor instead of being concatenation of all weights looks like cat of tensor[0] and uninitialised memory.
After some debugs I found that tensors[i].data_ptr() + tensors[i].size(dim) * data_ptr_stride = tensors[I+1].data_ptr(), but tensors[i].untyped_storage().data_ptr() are different (but storages located sequentially). And as far as I know here is expected that tensors[i].untyped_storage().data_ptr() would be the same. Am I wrong?