We need to decide to what level of fidelity we want to enforce alignment. Currently, we enforce nothing but then occasionally run into interop issues with Arrow. To work around these, we ensure that Arrow does the allocation (e.g., on the VarBinView builder). This is kludgy and error-prone.
An alternative option would be to require or encourage all buffers allocated by the project to be aligned to some value (e.g., the platform-specific arrow_buffer::alloc::ALIGNMENT, or the max of that and 64/128 bytes, etc).
If we require all of our buffers to be aligned, then that will imply copying unaligned Vecs passed by users (arrow-rs doesn't copy), so we should probably make it obvious & less ergonomic to do so.
We need to decide to what level of fidelity we want to enforce alignment. Currently, we enforce nothing but then occasionally run into interop issues with Arrow. To work around these, we ensure that Arrow does the allocation (e.g., on the VarBinView builder). This is kludgy and error-prone.
An alternative option would be to require or encourage all buffers allocated by the project to be aligned to some value (e.g., the platform-specific
arrow_buffer::alloc::ALIGNMENT, or the max of that and 64/128 bytes, etc).If we require all of our buffers to be aligned, then that will imply copying unaligned
Vecs passed by users (arrow-rsdoesn't copy), so we should probably make it obvious & less ergonomic to do so.