[R] Make head/tail lazy on datasets and queries

[asfimport]

Reporter: Neal Richardson / @nealrichardson
Assignee: Neal Richardson / @nealrichardson

PRs and other links:

• GitHub Pull Request #11373

_{Note: This issue was originally created as ARROW-13893. Please see the migration documentation for further details.}

[asfimport]

Jonathan Keane / @jonkeane:
The current behavior is a bit surprising even when it "should" be defined:

> library(arrow)
> library(dplyr)
> 
> tab <- Table$create(mtcars)
> 
> tab %>% 
+   arrange(mpg) %>% 
+   head(4) %>% 
+   collect()
   mpg cyl disp  hp drat    wt  qsec vs am gear carb
1 21.0   6  160 110 3.90 2.620 16.46  0  1    4    4
2 21.0   6  160 110 3.90 2.875 17.02  0  1    4    4
3 22.8   4  108  93 3.85 2.320 18.61  1  1    4    1
4 21.4   6  258 110 3.08 3.215 19.44  1  0    3    1
> 
> mtcars %>% 
+   arrange(mpg) %>% 
+   head(4) %>% 
+   collect()
                     mpg cyl disp  hp drat    wt  qsec vs am gear carb
Cadillac Fleetwood  10.4   8  472 205 2.93 5.250 17.98  0  0    3    4
Lincoln Continental 10.4   8  460 215 3.00 5.424 17.82  0  0    3    4
Camaro Z28          13.3   8  350 245 3.73 3.840 15.41  0  0    3    4
Duster 360          14.3   8  360 245 3.21 3.570 15.84  0  0    3    4

I know there's some work to determine if head() could/should be defined int he absence of an explicit order, but after an arrange() it should work.

[asfimport]

Weston Pace / @westonpace:
I'd kind of expect it to work too given our current threading model but, in general, and specifically with some of the threading models being proposed, you would need resequencing/serialization and not just an implicit order.

Otherwise it is perfectly reasonable for the following ordering:

Sort node emits batch 1
Sort node emits batch 2
Project node emits batch 2
Project node emits batch 1

[asfimport]

Weston Pace / @westonpace:
Actually...I suppose sort / order by happens after projection, so I retract my previous statement.

[asfimport]

Neal Richardson / @nealrichardson:
Issue resolved by pull request 11373
#11373