Skip to content

Conversation

@yhuai
Copy link
Contributor

@yhuai yhuai commented Sep 1, 2015

https://issues.apache.org/jira/browse/SPARK-10377

#8252 (comment) renames TakeOrdered to TakeOrderedAndProject (this is an internal API), which breaks third-party code using Spark 1.4 branch (e.g. Cassandra connector https://github.com/datastax/spark-cassandra-connector/blob/v1.4.0-M3/spark-cassandra-connector/src/main/scala/org/apache/spark/sql/cassandra/CassandraSQLContext.scala#L90). This PR renames this class back to TakeOrdered.

@yhuai
Copy link
Contributor Author

yhuai commented Sep 1, 2015

@frodeso This is the change to rename it back.

@yhuai yhuai changed the title [SPARK-10377] [SQL] Rename TakeOrderedAndProject back to TakeOrdered. [SPARK-10377] [SQL] [BRANCH-1.4] Rename TakeOrderedAndProject back to TakeOrdered. Sep 1, 2015
@cloud-fan
Copy link
Contributor

Our physical operator TakeOrderedAndProject is also public, @frodeso do you use this?

@SparkQA
Copy link

SparkQA commented Sep 1, 2015

Test build #41851 has finished for PR 8545 at commit 8e659eb.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@JoshRosen
Copy link
Contributor

I believe that we've also made other strategy renames in recent patches, including a rename of the join selection strategy (it's no longer called HashJoinStrategy). Does this imply that we need to change that name back as well?

@yhuai
Copy link
Contributor Author

yhuai commented Sep 1, 2015

Is that in branch 1.4? Looks like we still have HashJoin in 1.4 branch. This PR changes the name of TakeOrderedAndProject back TakeOrdered for 1.4 branch. So, users use it do not need to change code for a minor update (when 1.4.2 is released).

@JoshRosen
Copy link
Contributor

Ah, gotcha. They're still going to have to update for 1.5, though, and will need to use reflection if they want to be compatible across 1.4.x and 1.5.x (or ship multiple versions of their library).

@yhuai
Copy link
Contributor Author

yhuai commented Sep 1, 2015

Yeah, I think it makes sense to update their lib since those Strategies used by our planner are all private APIs.

@yhuai
Copy link
Contributor Author

yhuai commented Sep 1, 2015

@cloud-fan I did a quick of their repo. Looks like they have their own SQLContext and refer to the TakeOrdered strategy at there. Looks like it is fine that we do not rename the operator.

@frodeso
Copy link

frodeso commented Sep 1, 2015

@cloud-fan @yhuai I just did the same and came to the same conclusion. We are not actually involved in the connector, just a user of it so no guarantees though.

Thanks for the quick response!

asfgit pushed a commit that referenced this pull request Sep 1, 2015
… TakeOrdered.

https://issues.apache.org/jira/browse/SPARK-10377

#8252 (comment) renames `TakeOrdered` to `TakeOrderedAndProject` (this is an internal API), which breaks third-party code using Spark 1.4 branch (e.g. Cassandra connector https://github.com/datastax/spark-cassandra-connector/blob/v1.4.0-M3/spark-cassandra-connector/src/main/scala/org/apache/spark/sql/cassandra/CassandraSQLContext.scala#L90). This PR renames this class back to `TakeOrdered`.

Author: Yin Huai <[email protected]>

Closes #8545 from yhuai/SPARK-10377.
@yhuai
Copy link
Contributor Author

yhuai commented Sep 1, 2015

@frodeso This one has been merged. When we release 1.4.2, the name of that strategy will be the same.

@frodeso
Copy link

frodeso commented Sep 1, 2015

@yhuai Thanks, much appreciated!

@yhuai yhuai closed this Sep 1, 2015
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants