Skip to content

Proposed fix for https://github.com/networknt/openapi-parser/issues/43#56

Merged
stevehu merged 6 commits into
masterfrom
unknown repository
Sep 23, 2024
Merged

Proposed fix for https://github.com/networknt/openapi-parser/issues/43#56
stevehu merged 6 commits into
masterfrom
unknown repository

Conversation

@ghost
Copy link
Copy Markdown

@ghost ghost commented Sep 17, 2024

Proposed fix for #43 along the lines of what I described there:

Using Java Reflection API, dynamically loading either the "jakarta" (1st prio) or the "javax" (2nd prio) implementation of the target classes as the delegate objects, and putting delegating proxies in front of them as part of the "openapi-parser" project which forward all calls to the respective delegates...

@stevehu stevehu self-requested a review September 18, 2024 11:46
@ghost
Copy link
Copy Markdown
Author

ghost commented Sep 19, 2024

Just fixed (de5e134) missing delegation to three methods that I had accidentally overlooked...

@stevehu
Copy link
Copy Markdown
Contributor

stevehu commented Sep 23, 2024

@AndreasALoew Is it completed? If yes, I will merge it and try it out. Thanks.

@ghost
Copy link
Copy Markdown
Author

ghost commented Sep 23, 2024

Yes, it's complete now and works fine. We passed all our project-specific tests as well 😄

You can now (by commenting in/out version property and dependency) choose between

  • com.sun.mail:jakarta-mail:1.6.7 (the "javax" world of Java EE 8 / Spring Boot up to 2.7.x)
  • jakarta.mail:jakarta.mail-api:2.1.3 (the "jakarta" world of Jakarta EE 9+ / Spring Boot 3.x)

Happy testing! 👍

@stevehu stevehu merged commit 0f8c89f into networknt:master Sep 23, 2024
@stevehu
Copy link
Copy Markdown
Contributor

stevehu commented Oct 29, 2024

@AndreasALoew One of our users reports an issue with light-4j 2.1.38-SNAPSHOT version due to this change. With Java 11, there are some warning message like below.

WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by com.networknt.oas.validator.mail.InternetAddress (file:/home/steve/.m2/repository/com/networknt/openapi-parser/2.1.38-SNAPSHOT/openapi-parser-2.1.38-SNAPSHOT.jar) to method java.lang.Object.finalize()
WARNING: Please consider reporting this to the maintainers of com.networknt.oas.validator.mail.InternetAddress
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release

With Java 17 and above, the server cannot start anymore with the following error messages.

2024-10-29T08:45:03.872 [main]   ERROR com.networknt.handler.Handler:426 initStringDefinedHandler - Could not instantiate handler class class com.networknt.openapi.OpenApiHandler
java.lang.reflect.InvocationTargetException: null
	at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:77)
	at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
	at java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:500)
	at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:481)
	at com.networknt.handler.Handler.initStringDefinedHandler(Handler.java:423)
	at com.networknt.handler.Handler.initHandlers(Handler.java:82)
	at com.networknt.handler.Handler.init(Handler.java:64)
	at com.networknt.server.Server.start(Server.java:181)
	at com.networknt.server.Server.init(Server.java:132)
	at com.networknt.server.Server.main(Server.java:105)
Caused by: java.lang.ExceptionInInitializerError: null
	at com.networknt.oas.validator.ValidatorBase.checkEmail(ValidatorBase.java:166)
	at com.networknt.oas.validator.ValidatorBase.validateStringField(ValidatorBase.java:71)
	at com.networknt.oas.validator.ValidatorBase.validateEmailField(ValidatorBase.java:160)
	at com.networknt.oas.validator.ValidatorBase.validateEmailField(ValidatorBase.java:152)
	at com.networknt.oas.validator.impl.ContactValidator.runObjectValidations(ContactValidator.java:27)
	at com.networknt.oas.validator.ObjectValidatorBase.runValidations(ObjectValidatorBase.java:18)
	at com.networknt.oas.validator.ValidatorBase.validate(ValidatorBase.java:44)
	at com.networknt.oas.validator.ValidatorBase.validateField(ValidatorBase.java:204)
	at com.networknt.oas.validator.impl.InfoValidator.runObjectValidations(InfoValidator.java:28)
	at com.networknt.oas.validator.ObjectValidatorBase.runValidations(ObjectValidatorBase.java:18)
	at com.networknt.oas.validator.ValidatorBase.validate(ValidatorBase.java:44)
	at com.networknt.oas.validator.ValidatorBase.validateField(ValidatorBase.java:204)
	at com.networknt.oas.validator.impl.OpenApi3Validator.runObjectValidations(OpenApi3Validator.java:24)
	at com.networknt.oas.validator.ObjectValidatorBase.runValidations(ObjectValidatorBase.java:18)
	at com.networknt.oas.validator.ValidatorBase.validate(ValidatorBase.java:44)
	at com.networknt.oas.model.impl.OpenApi3Impl.validate(OpenApi3Impl.java:44)
	at com.networknt.oas.OpenApiParser.parse(OpenApiParser.java:96)
	at com.networknt.oas.OpenApiParser.parse(OpenApiParser.java:85)
	at com.networknt.oas.OpenApiParser.parse(OpenApiParser.java:35)
	at com.networknt.oas.OpenApiParser.parse(OpenApiParser.java:28)
	at com.networknt.openapi.OpenApiHelper.<init>(OpenApiHelper.java:58)
	at com.networknt.openapi.OpenApiHandler.<init>(OpenApiHandler.java:126)
	at com.networknt.openapi.OpenApiHandler.<init>(OpenApiHandler.java:142)
	... 11 common frames omitted
Caused by: java.lang.reflect.InaccessibleObjectException: Unable to make protected void java.lang.Object.finalize() throws java.lang.Throwable accessible: module java.base does not "opens java.lang" to unnamed module @15ecfcb1
	at java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:354)
	at java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:297)
	at java.base/java.lang.reflect.Method.checkCanSetAccessible(Method.java:200)
	at java.base/java.lang.reflect.Method.setAccessible(Method.java:194)
	at com.networknt.oas.validator.mail.InternetAddress.<clinit>(InternetAddress.java:119)
	... 34 common frames omitted
2024-10-29T08:45:03.873 [main]   ERROR com.networknt.server.Server:135 init - Server is not operational! Failed with exception
java.lang.RuntimeException: Could not instantiate handler class: class com.networknt.openapi.OpenApiHandler
	at com.networknt.handler.Handler.initStringDefinedHandler(Handler.java:427)
	at com.networknt.handler.Handler.initHandlers(Handler.java:82)
	at com.networknt.handler.Handler.init(Handler.java:64)
	at com.networknt.server.Server.start(Server.java:181)
	at com.networknt.server.Server.init(Server.java:132)
	at com.networknt.server.Server.main(Server.java:105)
Failed to start server:Could not instantiate handler class: class com.networknt.openapi.OpenApiHandler
2024-10-29T08:45:03.874 [Thread-0]   INFO  com.networknt.server.Server:391 shutdown - Cleaning up before server shutdown

It looks like we cannot use reflection or we need to change the way we use reflection. What do you think? Thanks.

@ghost
Copy link
Copy Markdown
Author

ghost commented Oct 29, 2024

Hi @stevehu ,

good news: both user reports issues are easily resolved and do not require any changes to the codebase.

The first one is due to the fact that from Java 9 (with the introduction of the module system), reflective access between packages needs to either be included in the module info files (which we obviously cannot do for JDK packages, as we don't own them), or otherwise "opened up" by JVM command-line options on startup:

For older versions such as JDK11, it is sufficient (though not great practice) to simply use the default of --illegal-access=permit which exactly causes the warning reported by your user: "The first reflective-access operation to any such package causes a warning to be issued. However, no warnings are issued after the first occurrence."

To get fully rid of the warning, you need to "open up" the JDK modules for access by "unnamed module" (such as in our case openapi-parser and/or light-4j):

In line 270 added here:

<argLine>--add-opens=java.base/java.lang=ALL-UNNAMED</argLine>

you can find the option: --add-opens=java.base/java.lang=ALL-UNNAMED that is needed to do this opening without causing an error message.

From Java 17, the --illegal-access=permit option has been completely removed, so the intended (and nowadays, only!) way to do this is a proper --add-opens=(...) as shown above.

So can you please try to add --add-opens=java.base/java.lang=ALL-UNNAMED with both JDK11 and JDK17 to the command line starting the Java VM? The errors should just vanish... 😄

Of course, you also need to take care of whether you want/need the "javax" or the "jakarta" version of the mail implementation in your classpath - change the dependency from

openapi-parser/pom.xml

Lines 141 to 145 in 0f8c89f

<dependency>
<groupId>com.sun.mail</groupId>
<artifactId>jakarta.mail</artifactId>
<version>${version.javamail}</version>
</dependency>

to

openapi-parser/pom.xml

Lines 147 to 151 in 0f8c89f

<dependency>
<groupId>jakarta.mail</groupId>
<artifactId>jakarta.mail-api</artifactId>
<version>${version.jakartamail}</version>
</dependency>

to switch from javax (latest is 1.6.x) to jakarta (2.1.y) versions...

Please report back on your success... 🤞

Best regards
Andreas

@stevehu
Copy link
Copy Markdown
Contributor

stevehu commented Oct 30, 2024

Given that we have some users on Java 17 and Java 21, we have to roll back the changes so that they can use the snapshot version for internal tests. As more and more users are moving to the Graalvm for native images, reflection is not just an issue for now but a bigger issue in the future. I am wondering if we can pick up one of the libraries without reflection. In our current use case, we are using javax.mail and it is sufficient enough as we are not using JEE.

@ghost
Copy link
Copy Markdown
Author

ghost commented Oct 30, 2024

OK, @stevehu , while I can understand your point, it still is sad and a pity. 😢

If your main goal indeed is to get fully rid of/stay compeletly free of Java reflection (the usage of which btw, can also be "announced"/configured beforehead for GraalVM native images: https://www.graalvm.org/latest/reference-manual/native-image/metadata/#reflection ), then I am at my wits' end - and our ways should likely spread up here, me doing just another fork of your "openapi-parser":

Our scenario is plain "good old" Java/Jakarta EE, where we MUST switch to Jakarta EE 10 and thus, "jakarta.mail" in the next couple of weeks. Staying with the old "javax.mail" as you propose is a true NO-GO for us even for legal reasons (Oracle is not allowing jaxax packages owned by Jakarta EE later than Java/Jakarta EE8 any more!).

But I can perfectly understand that if light-4j is still on the "javax" side of things (IMHO, it can't stay there forever!), you don't like this.

Sorry - but if want to keep the code base free of the use of Reflection, the only way you can do is split up your repo into two major version branches: one old version linking with "javax.mail" without reflection, and one new version linking with "jakarta.mail" without reflection. Many third-party products do this by opening up a new major version (i.e. keep an old - still maintained - 2.1.x branch using "javax", but create a new 3.y branch including main for "jakarta").

But if you want to stay in "main" with "javax.mail", this is a valid statement. So if you prefer to revert "my" commit, I would move forward and fork your repo one more time for Jakarta EE usage - just let me know.

Thanks & best regards
Andreas

@stevehu
Copy link
Copy Markdown
Contributor

stevehu commented Oct 30, 2024

@AndreasALoew My goal is to support as many users as possible. Since the InternetAddress is used only to validate the email in the specification, we can find an alternative way to do that. I am looking at the following link and there are two good candidates (apache-validation and regex). I haven't spent time digging into the details yet but will spend some time later. What do you think about this direction? Thanks.

@ghost
Copy link
Copy Markdown
Author

ghost commented Oct 30, 2024

OK, well - that opens up the space to look for more and indeed intersting solutions.

Unfortunately, you didn't include the link you mentioned, but I would NOT opt to try and "build your own" using regexs, as the RFCs behind are awfully complex and they define the offical rules set that must be matched.

I myself have already run across https://github.com/RohanNagar/jmail - which you can even check out at https://www.rohannagar.com/jmail/ testing for its behaviour (make sure to tick at least some of the Advanced Options, otherwise IMHO it is too lax!), but reading through its README markdown document, it both has the spec-related "RFC" background as well as even more configurability than you may ever need... 😉

So if you can provide some more information to me as to which of the Advanced Options you would like to check for (see docs at https://github.com/RohanNagar/jmail/blob/master/README.md and the testing page), I can then again create a proposed patch; this time by replacing "{javax, jakarta}.mail.impl" by "jmail"...

Looking forward to your opinion on this solution path...😄

@stevehu
Copy link
Copy Markdown
Contributor

stevehu commented Oct 30, 2024

Sorry. This is the link I was read. https://stackoverflow.com/questions/624581/what-is-the-best-java-email-address-validation-method

I also asked AI and it gave me the following answer.

public static boolean isValidEmail(String email) {
    String emailRegex = "^[a-zA-Z0-9_+&*-]+(?:\\.[a-zA-Z0-9_+&*-]+)*@(?:[a-zA-Z0-9-]+\\.)+[a-zA-Z]{2,7}$";
    Pattern pat = Pattern.compile(emailRegex);
    if (email == null)
        return false;
    return pat.matcher(email).matches();
}

I think this is good enough. What we are doing is to validate the email in the contact section of the specification. As long as the format is correct, we really don't care if the email address is real or not. What do you think?

@ghost
Copy link
Copy Markdown
Author

ghost commented Oct 30, 2024

Well, if I were in your shoes, I'd rather be careful and NOT follow AI advice (if AI has read some nonsense frequently enough, it thinks it's "true" as it has the highest probability - only true if all information sources were equally valid, which they ARE NOT)... :sad:

I'd rather choose jmail as already described in my last post (https://github.com/RohanNagar/jmail), it's also currently well maintained, its last release is from May 2024...

There also is https://github.com/bbottema/email-rfc2822-validator BUT it still has an optional (and only "javax"-style, NOT "jakarta" YET...) optional dependency to the "javax.mail" implementation, which means we would not win anything: "Note: Jakarta Mail is now an optional dependency, which you need to add yourself, but only if you use the parsing facilities of this library (rather than only the validation function)"... :sad:

So if you would be so kind as to tell me which of jmail's optional features you would want to activate, I volunteer to create a new proposed patch of "openapi-parser", replacing "javax/jakarta mail" by "jmail" and send you a new PR...

Thanks & best regards,
Andreas

@ghost
Copy link
Copy Markdown
Author

ghost commented Oct 30, 2024

... and please also look at the comparison table between these options in the lower half of this page:
https://www.rohannagar.com/jmail/ - "Comparison to Other Java Libraries" - where from the examples plus the result of differnt libraries, it should make quite clear that IMHO this would be the way to go... 😉

(don't know how the AI proposed simple regex setup would finish here, but I assume last place...)

@stevehu
Copy link
Copy Markdown
Contributor

stevehu commented Oct 30, 2024

@AndreasALoew Yes. Jmail looks very good. Let's use it to replace the javax.mail. That is a very good pick.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant