Skip to content

AWS script fix for EMR-7.0.0#2004

Closed
lachezar-n wants to merge 5 commits into
apache:mainfrom
lachezar-n:main
Closed

AWS script fix for EMR-7.0.0#2004
lachezar-n wants to merge 5 commits into
apache:mainfrom
lachezar-n:main

Conversation

@lachezar-n

Copy link
Copy Markdown
Contributor

Following the pull request for updating the script to use emr-7.0.0, here is a fix to cover these issues:

  • EMR uses by default Java 8 for Spark: Settings to enforce Java 11 were added
  • Launching the cluster from outside the AWS client region was not configured originally: Here that is fixed for all shell scripts.

Successfully tested:

  • Spin up the cluster, run DML script and terminate cluster from eu-centra-1 client into eu-central-1
  • Spin up the cluster, run DML script and terminate cluster from eu-centra-1 client into us-central-1

@mboehm7

mboehm7 commented Mar 16, 2024

Copy link
Copy Markdown
Contributor

LGTM - thanks for the patch @lachezar-n. I resolved the merge conflicts (in the future please do not include the previous PR), and we should keep an eye open if the hard-coded /usr/lib/jvm/jre-11 is always valid.

@mboehm7 mboehm7 closed this in 0950852 Mar 16, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

2 participants