[SPARK-35378][SQL][FOLLOWUP] isLocal should consider CommandResult#32963
[SPARK-35378][SQL][FOLLOWUP] isLocal should consider CommandResult#32963beliefer wants to merge 3 commits into
Conversation
|
Kubernetes integration test unable to build dist. exiting with code: 1 |
|
Test build #139984 has finished for PR 32963 at commit
|
|
cc @cloud-fan |
| */ | ||
| def isLocal: Boolean = logicalPlan.isInstanceOf[LocalRelation] | ||
| def isLocal: Boolean = logicalPlan.isInstanceOf[LocalRelation] || | ||
| logicalPlan.isInstanceOf[CommandResult] |
There was a problem hiding this comment.
+1, this API has no test at all, we can add one in DataFrameSuite
There was a problem hiding this comment.
Thank you for reminder.
|
Gentle ping, @beliefer . |
|
|
||
| test("isLocal should consider CommandResult and LocalRelation") { | ||
| withTable("t1") { | ||
| val df = sql("CREATE TABLE t USING PARQUET AS SELECT 1 as a") |
There was a problem hiding this comment.
can we use SHOW TABLES which returns data? Then we don't need to create a table and wrap with withTable.
|
Kubernetes integration test unable to build dist. exiting with code: 1 |
|
Kubernetes integration test starting |
|
Kubernetes integration test status failure |
dongjoon-hyun
left a comment
dongjoon-hyun
left a comment
There was a problem hiding this comment.
+1, LGTM. Thank you, @beliefer and @cloud-fan .
|
Test build #140115 has finished for PR 32963 at commit
|
|
thanks, merging to master! |
|
@dongjoon-hyun @cloud-fan Thanks for review. |
|
Test build #140117 has finished for PR 32963 at commit
|
4 participants
What changes were proposed in this pull request?
#32513 added the case class
CommandResultso as we can eagerly execute command locally. But we forgot to updateisLocalofDataset.Why are the changes needed?
Dataset.isLocalshould considerCommandResult.Does this PR introduce any user-facing change?
Yes. If the SQL plan is
CommandResult,Dataset.isLocalmust return true.How was this patch tested?
No test.