[SPARK-2235][SQL]Spark SQL basicOperator add Intersect operator by YanjieGao · Pull Request #1150 · apache/spark

YanjieGao · 2014-06-20T07:20:34Z

Hi all,
I want to submit a basic operator Intersect
For example , in sql case
select * from table1
intersect
select * from table2
So ,i want use this operator support this function in Spark SQL
This operator will return the the intersection of SparkPlan child table RDD .
JIRA:https://issues.apache.org/jira/browse/SPARK-2235

Hi all, I want to submit a basic operator Intersect For example , in sql case select * from table1 intersect select * from table2 So ,i want use this operator support this function in Spark SQL This operator will return the the intersection of SparkPlan child table RDD .

AmplabJenkins · 2014-06-20T07:24:58Z

Can one of the admins verify this patch?

YanjieGao · 2014-06-23T08:18:33Z

hi all ,I have finished the INTERSECT operator ,and has update the 6 files this operator needed .
Now i need your help to review the code .Thanks a lot !

YanjieGao · 2014-07-03T05:53:10Z

Hi all, I have resolved the conflict .

marmbrus · 2014-07-04T09:51:50Z

This is conflicting now that we merged the other PR.

test this please

marmbrus · 2014-07-04T09:57:12Z

Does hive support this? if so we should add a test to HiveQuerySuite too.

Hive doesn't support ,I will remove this

YanjieGao · 2014-07-04T10:45:32Z

I don't know why this line always red

Hi, it says you deleted some whitespace after Nil.

YanjieGao · 2014-07-04T10:46:26Z

Hi Michael,I have resolve the conflict .And modify as your suggestion

YanjieGao · 2014-07-05T04:55:25Z

Thanks ,I add three blank after that line ,Now there aren't red lines.

marmbrus · 2014-07-05T20:16:34Z

Space between ) and }

marmbrus · 2014-07-05T20:17:11Z

Space after ,

marmbrus · 2014-07-05T20:19:03Z

Thanks ,I add three blank after that line ,Now there aren't red lines.

If there is trailing whitespace we should remove it. It is okay if it is not code you added in the PR.

YanjieGao · 2014-07-06T00:31:12Z

Thanks a lot, I have reformat the code style.

marmbrus · 2014-07-08T02:40:57Z

Thanks, I've merged this into master. Did not apply cleanly to 1.0 so we'll backport it later.

YanjieGao · 2014-07-08T02:45:25Z

Hi, Michael I also have a Skew Join pr need to be reviewed and need some suggestions .Could you help me review it ?I have test it can pass the testsuite .Thanks a lot

#1134

Hi all, I want to submit a basic operator Intersect For example , in sql case select * from table1 intersect select * from table2 So ,i want use this operator support this function in Spark SQL This operator will return the the intersection of SparkPlan child table RDD . JIRA:https://issues.apache.org/jira/browse/SPARK-2235 Author: Yanjie Gao <gaoyanjie55@163.com> Author: YanjieGao <396154235@qq.com> Closes apache#1150 from YanjieGao/patch-5 and squashes the following commits: 4629afe [YanjieGao] reformat the code bdc2ac0 [YanjieGao] reformat the code as Michael's suggestion 3b29ad6 [YanjieGao] Merge remote branch 'upstream/master' into patch-5 1cfbfe6 [YanjieGao] refomat some files ea78f33 [YanjieGao] resolve conflict and add annotation on basicOperator and remove HiveQl 0c7cca5 [YanjieGao] modify format problem a802ca8 [YanjieGao] Merge remote branch 'upstream/master' into patch-5 5e374c7 [YanjieGao] resolve conflict in SparkStrategies and basicOperator f7961f6 [Yanjie Gao] update the line less than bdc4a05 [Yanjie Gao] Update basicOperators.scala 0b49837 [Yanjie Gao] delete the annotation f1288b4 [Yanjie Gao] delete annotation e2b64be [Yanjie Gao] Update basicOperators.scala 4dd453e [Yanjie Gao] Update SQLQuerySuite.scala 790765d [Yanjie Gao] Update SparkStrategies.scala ac73e60 [Yanjie Gao] Update basicOperators.scala d4ac5e5 [Yanjie Gao] Update HiveQl.scala 61e88e7 [Yanjie Gao] Update SqlParser.scala 469f099 [Yanjie Gao] Update basicOperators.scala e5bff61 [Yanjie Gao] Spark SQL basicOperator add Intersect operator

…s" message appears when spark session is starting (apache#1150)

YanjieGao changed the title ~~Spark SQL basicOperator add Intersect operator~~ [SPARK-2235][SQL]Spark SQL basicOperator add Intersect operator Jun 22, 2014

YanjieGao added 6 commits June 23, 2014 16:04

Update basicOperators.scala

469f099

Update SqlParser.scala

61e88e7

Update HiveQl.scala

d4ac5e5

Update basicOperators.scala

ac73e60

Update SparkStrategies.scala

790765d

Update SQLQuerySuite.scala

4dd453e

YanjieGao added 5 commits June 24, 2014 11:14

Update basicOperators.scala

e2b64be

delete annotation

f1288b4

delete the annotation

0b49837

Update basicOperators.scala

bdc4a05

update the line less than

f7961f6

YanjieGao added 3 commits July 3, 2014 14:07

resolve conflict in SparkStrategies and basicOperator

5e374c7

Merge remote branch 'upstream/master' into patch-5

a802ca8

modify format problem

0c7cca5

YanjieGao mentioned this pull request Jul 4, 2014

[SPARK-2234][SQL]Spark SQL basicOperators add Except operator #1151

Closed

marmbrus reviewed Jul 4, 2014
View reviewed changes

YanjieGao added 2 commits July 4, 2014 18:27

resolve conflict and add annotation on basicOperator and remove HiveQl

ea78f33

refomat some files

1cfbfe6

YanjieGao reviewed Jul 4, 2014
View reviewed changes

marmbrus reviewed Jul 5, 2014
View reviewed changes

Comment thread sql/core/src/main/scala/org/apache/spark/sql/execution/SparkStrategies.scala Outdated

marmbrus Jul 5, 2014

Copy link
Copy Markdown

Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Space after ,

Merge remote branch 'upstream/master' into patch-5

3b29ad6

YanjieGao added 2 commits July 6, 2014 08:39

reformat the code as Michael's suggestion

bdc2ac0

reformat the code

4629afe

asfgit closed this in 50561f4 Jul 8, 2014

udaynpusa pushed a commit to mapr/spark that referenced this pull request Jan 30, 2024

MapR [SPARK-1228] "Warning: Ignoring non-Spark config property: maprf…

af1f597

…s" message appears when spark session is starting (apache#1150)

mapr-devops pushed a commit to mapr/spark that referenced this pull request May 8, 2025

MapR [SPARK-1228] "Warning: Ignoring non-Spark config property: maprf…

e02d9f7

…s" message appears when spark session is starting (apache#1150)

Uh oh!

Conversation

YanjieGao commented Jun 20, 2014

Uh oh!

AmplabJenkins commented Jun 20, 2014

Uh oh!

YanjieGao commented Jun 23, 2014

Uh oh!

YanjieGao commented Jul 3, 2014

Uh oh!

marmbrus commented Jul 4, 2014

Uh oh!

marmbrus Jul 4, 2014

Choose a reason for hiding this comment

Uh oh!

YanjieGao Jul 4, 2014

Choose a reason for hiding this comment

Uh oh!

YanjieGao Jul 4, 2014

Choose a reason for hiding this comment

Uh oh!

ueshin Jul 5, 2014

Choose a reason for hiding this comment

Uh oh!

YanjieGao commented Jul 4, 2014

Uh oh!

YanjieGao commented Jul 5, 2014

Uh oh!

marmbrus Jul 5, 2014

Choose a reason for hiding this comment

Uh oh!

marmbrus Jul 5, 2014

Choose a reason for hiding this comment

Uh oh!

marmbrus commented Jul 5, 2014

Uh oh!

YanjieGao commented Jul 6, 2014

Uh oh!

marmbrus commented Jul 8, 2014

Uh oh!

YanjieGao commented Jul 8, 2014

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants