In ARROW-11843 we found that we still can't enable parallel column conversion for Parquet datasets because the Arrow Parquet reader uses OptionalParallelFor, which does a parallel wait. We should provide an async version to avoid nested parallelism.
Reporter: David Li / @lidavidm
Assignee: David Li / @lidavidm
Related issues:
PRs and other links:
Note: This issue was originally created as ARROW-12597. Please see the migration documentation for further details.
In ARROW-11843 we found that we still can't enable parallel column conversion for Parquet datasets because the Arrow Parquet reader uses OptionalParallelFor, which does a parallel wait. We should provide an async version to avoid nested parallelism.
Reporter: David Li / @lidavidm
Assignee: David Li / @lidavidm
Related issues:
PRs and other links:
Note: This issue was originally created as ARROW-12597. Please see the migration documentation for further details.