[SPARK-34773][BUILD] change-scala-version.sh fails to modify pom.xml if it changes the version from 2.13 to 2.12

[sarutak]

What changes were proposed in this pull request?

This PR fixes an issue that If we change the Scala version to 2.13 and then change the version to 2.12 using change-scala-version.sh, it fails to modify pom.xml and fails to build.

The reason seems that the following logic never be able to fetch 2.12.0 from pom.xml after it changes 2.13.5.

# Update <scala.version> in parent POM
# First find the right full version from the profile's build
SCALA_VERSION=`build/mvn help:evaluate -Pscala-${TO_VERSION} -Dexpression=scala.version -q -DforceStdout`
sed_i '1,/<scala\.version>[0-9]*\.[0-9]*\.[0-9]*</s/<scala\.version>[0-9]*\.[0-9]*\.[0-9]*</<scala.version>'$SCALA_VERSION'</' \
  "$BASEDIR/pom.xml"

Further, I don't think this logic and the following logic are necessary because those versions are changed by -Pscala-2.13 if we'd like to build with Scala 2.13.

# Also update <scala.binary.version> in parent POM
# Match any scala binary version to ensure idempotency
sed_i '1,/<scala\.binary\.version>[0-9]*\.[0-9]*</s/<scala\.binary\.version>[0-9]*\.[0-9]*</<scala.binary.version>'$TO_VERSION'</' \
  "$BASEDIR/pom.xml"

So I removed them to fix this issue.

Why are the changes needed?

It's inconvenient if we'd like to change the version back to 2.12 in development.

Does this PR introduce any user-facing change?

No.

How was this patch tested?

I confirmed build for Scala 2.13 and 2.12 successfully finishes with both sbt and Maven.

./dev/change-scala-version 2.13
./build/sbt -Pyarn -Pmesos -Pkubernetes -Phive -Phive-thriftserver -Phadoop-cloud -Pkinesis-asl -Pdocker-integration-tests -Pkubernetes-integration-tests -Pspark-ganglia-lgpl -Pscala-2.13 compile test:compile 

./build/sbt -Pyarn -Pmesos -Pkubernetes -Phive -Phive-thriftserver -Phadoop-cloud -Pkinesis-asl -Pdocker-integration-tests -Pkubernetes-integration-tests -Pspark-ganglia-lgpl -Pscala-2.13 -DskipTests compile test-compile

./dev/change-scala-version 2.12
./build/sbt -Pyarn -Pmesos -Pkubernetes -Phive -Phive-thriftserver -Phadoop-cloud -Pkinesis-asl -Pdocker-integration-tests -Pkubernetes-integration-tests -Pspark-ganglia-lgpl  compile test:compile 

./build/sbt -Pyarn -Pmesos -Pkubernetes -Phive -Phive-thriftserver -Phadoop-cloud -Pkinesis-asl -Pdocker-integration-tests -Pkubernetes-integration-tests -Pspark-ganglia-lgpl  -DskipTests compile test-compile

I also confirmed Scala 2.13.5 is used.

sbt:spark-parent> scalaVersion
[info] core / scalaVersion
[info] 	2.13.5
[info] docker-integration-tests / scalaVersion
[info] 	2.13.5
[info] assembly / scalaVersion
[info] 	2.13.5
[info] mllib-local / scalaVersion
[info] 	2.13.5
[info] sketch / scalaVersion
[info] 	2.13.5
[info] sql-kafka-0-10 / scalaVersion
[info] 	2.13.5
[info] network-shuffle / scalaVersion
[info] 	2.13.5
[info] streaming-kafka-0-10-assembly / scalaVersion
[info] 	2.13.5
[info] streaming-kinesis-asl / scalaVersion
[info] 	2.13.5
[info] streaming-kinesis-asl-assembly / scalaVersion
[info] 	2.13.5
[info] kvstore / scalaVersion
[info] 	2.13.5
[info] token-provider-kafka-0-10 / scalaVersion
[info] 	2.13.5
[info] network-yarn / scalaVersion
[info] 	2.13.5
[info] graphx / scalaVersion
[info] 	2.13.5
[info] hive-thriftserver / scalaVersion
[info] 	2.13.5
[info] streaming-kafka-0-10 / scalaVersion
[info] 	2.13.5
[info] streaming / scalaVersion
[info] 	2.13.5
[info] tools / scalaVersion
[info] 	2.13.5
[info] unsafe / scalaVersion
[info] 	2.13.5
[info] mllib / scalaVersion
[info] 	2.13.5
[info] network-common / scalaVersion
[info] 	2.13.5
[info] repl / scalaVersion
[info] 	2.13.5
[info] yarn / scalaVersion
[info] 	2.13.5
[info] examples / scalaVersion
[info] 	2.13.5
[info] avro / scalaVersion
[info] 	2.13.5
[info] hadoop-cloud / scalaVersion
[info] 	2.13.5
[info] catalyst / scalaVersion
[info] 	2.13.5
[info] ganglia-lgpl / scalaVersion
[info] 	2.13.5
[info] sql / scalaVersion
[info] 	2.13.5
[info] mesos / scalaVersion
[info] 	2.13.5
[info] kubernetes / scalaVersion
[info] 	2.13.5
[info] kubernetes-integration-tests / scalaVersion
[info] 	2.13.5
[info] tags / scalaVersion
[info] 	2.13.5
[info] hive / scalaVersion
[info] 	2.13.5
[info] launcher / scalaVersion
[info] 	2.13.5
[info] scalaVersion
[info] 	2.13.5

[SparkQA]

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/40729/

[SparkQA]

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/40729/

[srowen]

This logic is essential to make the final POM correct when published - see original issue. The point is that profiles don't affect the final POM. (try it)

There may be a problem reversing it, can check, but you can also rollback git changes. May not help the release script though yeah

[sarutak]

Ah, I understand. #31865 seems to be prominent.