[Python] Segfault orcWriter write table

[asfimport]

pyarrow segfault when trying to write an orc from a table containing nullArray.

 

from pyarrow import orc
import pyarrow as pa

a = pa.array([1, None, 3, None])
b = pa.array([None, None, None, None])
table = pa.table({"int64": a, "utf8": b})
orc.write_table(table, 'test.orc')

zsh: segmentation fault  python3

Reporter: patrice
Assignee: Raúl Cumplido / @raulcd

Related issues:

• [Python] Calling nonexistent method of pyarrow.orc.ORCWriter causes segfault (Blocked)
• [Python] Calling nonexistent method of pyarrow.orc.ORCWriter causes segfault (is fixed by)

_{Note: This issue was originally created as ARROW-15723. Please see the migration documentation for further details.}

[asfimport]

Joris Van den Bossche / @jorisvandenbossche:
Thanks for the report. There are potentially multiple issues here.

First, writing null arrays is not actually supported (yet). When using the OrcWriter API directly, we can see this (using the table from the code snippet above):

In [3]: writer = orc.ORCWriter("test.orc")

In [4]: writer.write(table)
...
ArrowNotImplementedError: Unknown or unsupported Arrow type: null
../src/arrow/adapters/orc/util.cc:1062  GetOrcType(*arrow_child_type)
../src/arrow/adapters/orc/adapter.cc:811  GetOrcType(*(table.schema()))

But, it seems that for some reason this error is not bubbled up when using write_table (which uses this ORCWriter in a context manager).

Then, it further seems that the segfault comes from trying to write (close) an empty file. This can be reproduced with the following as well:

In [1]: from pyarrow import orc

In [2]: writer = orc.ORCWriter("test.orc")

In [3]: writer.close()
Segmentation fault (core dumped)

[asfimport]

Raúl Cumplido / @raulcd:
The segmentation fault is fixed as part of ARROW-16025. I have added a test that ensures the error from null type not being supported is correctly raised.

[asfimport]

Raúl Cumplido / @raulcd:
This has been fixed and tested as part of ARROW-16025