Skip to content

[R] glimpse() errors if there is a UDF #20371

@asfimport

Description

@asfimport

Using the example from ARROW-17437:

register_scalar_function(
  "test", 
  function(context, x) paste(x, collapse=","), 
  utf8(), 
  utf8(), 
  auto_convert=TRUE
)

Table$create(x = c("a", "b", "c")) |>
  transmute(test(x)) |>
  glimpse()

# Table (query)
# 3 rows x 1 columns
# Error in `dplyr::collect()`:
# ! NotImplemented: Call to R (resolve scalar user-defined function output data type) from a non-R thread from an unsupported context
# Run `rlang::last_error()` to see where the error occurred.

A variety of things could fix this:

  • Supporting UDFs in any query (I think there's a draft PR open for this)
  • The limit operator (FetchNode?) so that head() is handled in the ExecPlan and we don't need to use the RecordBatchReader workaround to get it efficiently (also PR in the works)
  • Worse case, error more informatively

Reporter: Neal Richardson / @nealrichardson

Related issues:

Note: This issue was originally created as ARROW-17438. Please see the migration documentation for further details.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions