Skip to content

[fix][functions] Fix netty.DnsResolverUtil compat issue on JDK9+ for the function Runtimes - #16423

Merged
eolivelli merged 1 commit into
apache:masterfrom
cbornet:functions-jdk11
Jul 19, 2022
Merged

[fix][functions] Fix netty.DnsResolverUtil compat issue on JDK9+ for the function Runtimes#16423
eolivelli merged 1 commit into
apache:masterfrom
cbornet:functions-jdk11

Conversation

@cbornet

@cbornet cbornet commented Jul 6, 2022

Copy link
Copy Markdown
Contributor

Motivation

DnsResolverUtil fails to get existing DNS TTL settings. This happens in Java 17.
This has been fixed for the pulsar-client and pulsar launcher scripts in #15349 and #15540
This PR is to do the same for the Runtime java command.

Modifications

Detect the java version of the function worker and add the JVM flag --add-opens java.base/sun.net=ALL-UNNAMED if the version is 9+

Documentation

Check the box below or label this PR directly.

Need to update docs?

  • doc-required
    (Your PR needs to update docs and you will update later)

  • doc-not-needed
    fix

  • doc
    (Your PR contains doc changes)

  • doc-complete
    (Docs have been already added)

@github-actions github-actions Bot added the doc-not-needed Your PR changes do not impact docs label Jul 6, 2022

// Needed for netty.DnsResolverUtil on JDK9+
if (!System.getProperty("java.version").startsWith("1.")) {
args.add("--add-opens java.base/sun.net=ALL-UNNAMED");

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you want to add other modules too? at least look at the changes that resulted in these and see if they are applicable to the functions

$ ggrep -rI "\-\-add-opens"

buildtools/pom.xml:      --add-opens java.base/jdk.internal.loader=ALL-UNNAMED
buildtools/pom.xml:      --add-opens java.base/java.lang=ALL-UNNAMED <!--Mockito-->
buildtools/dependency-reduced-pom.xml:    <test.additional.args>--add-opens java.base/jdk.internal.loader=ALL-UNNAMED
buildtools/dependency-reduced-pom.xml:      --add-opens java.base/java.lang=ALL-UNNAMED</test.additional.args>
bin/bookkeeper:# Start --add-opens options
bin/bookkeeper:# '--add-opens' option is not supported in jdk8
bin/bookkeeper:  OPTS="$OPTS --add-opens java.base/java.io=ALL-UNNAMED --add-opens java.base/java.util.zip=ALL-UNNAMED"
bin/bookkeeper:  OPTS="$OPTS --add-opens java.base/java.nio=ALL-UNNAMED --add-opens java.base/jdk.internal.misc=ALL-UNNAMED"
bin/function-localrunner:# Start --add-opens options
bin/function-localrunner:# '--add-opens' option is not supported in jdk8
bin/function-localrunner:  OPTS="$OPTS --add-opens java.base/java.nio=ALL-UNNAMED --add-opens java.base/jdk.internal.misc=ALL-UNNAMED"
bin/pulsar:# Start --add-opens options
bin/pulsar:# '--add-opens' option is not supported in jdk8
bin/pulsar:  OPTS="$OPTS --add-opens java.base/java.io=ALL-UNNAMED --add-opens java.base/java.util.zip=ALL-UNNAMED"
bin/pulsar:  OPTS="$OPTS --add-opens java.base/java.nio=ALL-UNNAMED --add-opens java.base/jdk.internal.misc=ALL-UNNAMED"
bin/pulsar:  OPTS="$OPTS --add-opens java.base/sun.net=ALL-UNNAMED --add-opens java.management/sun.management=ALL-UNNAMED"
bin/pulsar-client:# Start --add-opens options
bin/pulsar-client:# '--add-opens' option is not supported in jdk8
bin/pulsar-client:  OPTS="$OPTS --add-opens java.base/sun.net=ALL-UNNAMED"
pom.xml:      --add-opens java.base/jdk.internal.loader=ALL-UNNAMED
pom.xml:      --add-opens java.base/java.lang=ALL-UNNAMED <!--Mockito-->
pom.xml:      --add-opens java.base/java.io=ALL-UNNAMED <!--Bookkeeper NativeIO -->
pom.xml:      --add-opens java.base/sun.net=ALL-UNNAMED <!--netty.DnsResolverUtil-->
pom.xml:      --add-opens java.management/sun.management=ALL-UNNAMED <!--JvmDefaultGCMetricsLogger-->

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

java.base/sun.netshould be enough for Pulsar client since it's the same configuration used in pulsar-client CLI tool.

args.add("-Dio.netty.tryReflectionSetAccessible=true");

// Needed for netty.DnsResolverUtil on JDK9+
if (!System.getProperty("java.version").startsWith("1.")) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what about using commons-lang3 utility
SystemUtils.isJavaVersionAtLeast(JavaVersion.JAVA_9) ?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we should have a consistent approach across the code base, do you see other cases in Java code where we detect the Java version ?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No. That's the only place I found

@cbornet
cbornet force-pushed the functions-jdk11 branch 2 times, most recently from 095032d to 23ab6be Compare July 7, 2022 11:49
@cbornet
cbornet requested a review from nicoloboschi July 7, 2022 11:51
@cbornet
cbornet force-pushed the functions-jdk11 branch 2 times, most recently from 3e43cf8 to 0c306ca Compare July 7, 2022 13:47
@cbornet
cbornet requested a review from nicoloboschi July 7, 2022 13:48

@eolivelli eolivelli left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@cbornet

cbornet commented Jul 7, 2022

Copy link
Copy Markdown
Contributor Author

/pulsarbot rerun-failure-checks

2 similar comments
@cbornet

cbornet commented Jul 8, 2022

Copy link
Copy Markdown
Contributor Author

/pulsarbot rerun-failure-checks

@cbornet

cbornet commented Jul 8, 2022

Copy link
Copy Markdown
Contributor Author

/pulsarbot rerun-failure-checks

@cbornet
cbornet force-pushed the functions-jdk11 branch from 0c306ca to 09d995c Compare July 8, 2022 14:53
@cbornet

cbornet commented Jul 16, 2022

Copy link
Copy Markdown
Contributor Author

/pulsarbot rerun-failure-checks

@cbornet
cbornet force-pushed the functions-jdk11 branch 2 times, most recently from e1691be to 9cace6d Compare July 18, 2022 10:33
@cbornet

cbornet commented Jul 18, 2022

Copy link
Copy Markdown
Contributor Author

/pulsarbot rerun-failure-checks

@lhotari lhotari left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@eolivelli
eolivelli merged commit b9249d7 into apache:master Jul 19, 2022
@eolivelli eolivelli added this to the 2.11.0 milestone Jul 19, 2022
@cbornet
cbornet deleted the functions-jdk11 branch July 19, 2022 13:43
cbornet added a commit to cbornet/pulsar that referenced this pull request Aug 31, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/function doc-not-needed Your PR changes do not impact docs

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants