datafusion-cli should trap interrupt signal and abort execution

[jimexist]

Is your feature request related to a problem or challenge? Please describe what you are trying to do.

datafusion-cli should trap interrupt signal and abort execution

Describe the solution you'd like

using rustyline signal handling and kill async operation instead of exiting

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Additional context
Add any other context or screenshots about the feature request here.

[Jefffrey]

I tried to take an initial shot at this: Jefffrey@4761d16

But have run into a roadblock, where in the exec_and_print() function it can potentially block whilst printing, especially for a very large result set.

async fn exec_and_print(
    ctx: &mut SessionContext,
    print_options: &PrintOptions,
    sql: String,
) -> Result<()> {
    let now = Instant::now();
    let df = ctx.sql(&sql).await?;
    let results = df.collect().await?;
    print_options.print_batches(&results, now)?; // can block here!

    Ok(())
}

Since it is blocking, you can't abort the async function whilst it executes. Even spawning it in a separate task with spawn_blocking() doesn't look like it'll help, as though it would return control to let you enter more commands when you interrupt with CTRL-C, eventually that synchronous blocking print function will complete anyway and print onto the console (even though you interrupted it).

Perhaps could spawn it in a separate thread and kill it, rather than trying to relying on async cancellation?

(Edit: also relevant for why can't exactly use rustyline for this task: kkawakam/rustyline#125)

[qrilka]

@Jefffrey what about adding an option to make print_batches cancellable e.g. checking a flag after every batch?

[Jefffrey]

That could be possible @qrilka, though I believe might require changes in arrow-rs as I think that's where the iterative print code is

Definitely something worth exploring

[qrilka]

@Jefffrey my first thought was about having "cancellation points" after/before every batch i.e. in print_batches itself. But I guess that would be too coarse-grained and it looks like ideally we need AsyncWrite implemented for every printer...

[Jefffrey]

Yeah that might be too coarse-grained and I think would vary depending on size of batches? So could still have cases where user hits CTRL-C and still has a batch print afterwards, messing with their prompt.

I'll see if I have time to look into this a bit more, though I haven't made any progress on this since my initial stab at it.

[alamb]

the code that does the formatting in Arrow is here: https://docs.rs/arrow-cast/40.0.0/src/arrow_cast/pretty.rs.html#73-108

I have wanted a way (also in IOx) to "stream" results to the output rather than have to have all the record batches before we can even start formatting them...

maybe we could make such an API 🤔
https://docs.rs/arrow/latest/arrow/util/display/struct.ArrayFormatter.html

[Jefffrey]

Closed by #8739