Skip to content

[SPARK-6452] [SQL] Checks for missing attributes and unresolved operator for all types of operator#5129

Closed
liancheng wants to merge 2 commits into
apache:masterfrom
liancheng:spark-6452
Closed

[SPARK-6452] [SQL] Checks for missing attributes and unresolved operator for all types of operator#5129
liancheng wants to merge 2 commits into
apache:masterfrom
liancheng:spark-6452

Conversation

@liancheng

@liancheng liancheng commented Mar 22, 2015

Copy link
Copy Markdown
Contributor

In CheckAnalysis, Filter and Aggregate are checked in separate case clauses, thus never hit those clauses for unresolved operators and missing input attributes.

This PR also removes the prettyString call when generating error message for missing input attributes. Because result of prettyString doesn't contain expression ID, and may give confusing messages like

resolved attributes a missing from a

cc @rxin

Review on Reviewable

@SparkQA

SparkQA commented Mar 22, 2015

Copy link
Copy Markdown

Test build #28965 has started for PR 5129 at commit 46facb9.

  • This patch merges cleanly.

@SparkQA

SparkQA commented Mar 22, 2015

Copy link
Copy Markdown

Test build #28965 timed out for PR 5129 at commit 46facb9 after a configured wait of 120m.

@AmplabJenkins

AmplabJenkins commented Mar 22, 2015

Copy link
Copy Markdown

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/28965/
Test FAILed.

rxin
rxin reviewed Mar 22, 2015
View reviewed changes
Comment thread sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/analysis/AnalysisSuite.scala Outdated

@rxin rxin Mar 22, 2015

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should throw AnalysisException ? otherwise it is an incomplete sentence

@liancheng liancheng mentioned this pull request Mar 23, 2015
Closed
@SparkQA

SparkQA commented Mar 23, 2015

Copy link
Copy Markdown

Test build #28983 has started for PR 5129 at commit 0c49010.

  • This patch merges cleanly.

@SparkQA

SparkQA commented Mar 23, 2015

Copy link
Copy Markdown

Test build #28983 has finished for PR 5129 at commit 0c49010.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@AmplabJenkins

AmplabJenkins commented Mar 23, 2015

Copy link
Copy Markdown

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/28983/
Test PASSed.

@SparkQA

SparkQA commented Mar 23, 2015

Copy link
Copy Markdown

Test build #28991 has started for PR 5129 at commit 52cdc69.

  • This patch merges cleanly.

@liancheng

liancheng commented Mar 23, 2015

Copy link
Copy Markdown
Contributor Author

Rebased after #5132 being merged. All comments are addressed. Thanks everyone for the review!

@SparkQA

SparkQA commented Mar 23, 2015

Copy link
Copy Markdown

Test build #28991 has finished for PR 5129 at commit 52cdc69.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@AmplabJenkins

AmplabJenkins commented Mar 23, 2015

Copy link
Copy Markdown

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/28991/
Test PASSed.

asfgit pushed a commit that referenced this pull request Mar 24, 2015
@liancheng @marmbrus
[SPARK-6452] [SQL] Checks for missing attributes and unresolved opera…
6f10142 
…tor for all types of operator

In `CheckAnalysis`, `Filter` and `Aggregate` are checked in separate case clauses, thus never hit those clauses for unresolved operators and missing input attributes.

This PR also removes the `prettyString` call when generating error message for missing input attributes. Because result of `prettyString` doesn't contain expression ID, and may give confusing messages like

> resolved attributes a missing from a

cc rxin

<!-- Reviewable:start -->
[<img src="https://reviewable.io/review_button.png" height=40 alt="Review on Reviewable"/>](https://reviewable.io/reviews/apache/spark/5129)
<!-- Reviewable:end -->

Author: Cheng Lian <lian@databricks.com>

Closes #5129 from liancheng/spark-6452 and squashes the following commits:

52cdc69 [Cheng Lian] Addresses comments
029f9bd [Cheng Lian] Checks for missing attributes and unresolved operator for all types of operator

(cherry picked from commit 1afcf77)
Signed-off-by: Michael Armbrust <michael@databricks.com>
@marmbrus

marmbrus commented Mar 24, 2015

Copy link
Copy Markdown
Contributor

Thanks! Merged to master and 1.3

@asfgit asfgit closed this in 1afcf77 Mar 24, 2015
@liancheng liancheng deleted the spark-6452 branch March 24, 2015 15:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

8 participants

@liancheng @SparkQA @AmplabJenkins @marmbrus @rxin @chenghao-intel @cloud-fan @scwf