Remove bouncy castle shaded module to avoid bring error of verifySingleJar by jiazhai · Pull Request #7453 · apache/pulsar

jiazhai · 2020-07-05T03:36:24Z

Motivation

shade bouncy castle will cause some signature errors, this PR tries to remove the bouncy castle shaded module.

Here is the related error stack:

10:01:34.257 [pulsar-client-io-33-1] ERROR org.apache.pulsar.client.impl.ConsumerImpl - MessageCryptoBc may not included in the jar. e:
java.lang.SecurityException: JCE cannot authenticate the provider BC
	at javax.crypto.Cipher.getInstance(Cipher.java:657) ~[?:1.8.0_121]
	at javax.crypto.Cipher.getInstance(Cipher.java:596) ~[?:1.8.0_121]
	at org.apache.pulsar.client.impl.crypto.MessageCryptoBc.<init>(MessageCryptoBc.java:147) ~[classes/:?]
	at org.apache.pulsar.client.impl.ConsumerImpl.<init>(ConsumerImpl.java:270) ~[classes/:?]
	at org.apache.pulsar.client.impl.ConsumerImpl.newConsumerImpl(ConsumerImpl.java:209) ~[classes/:?]
	at org.apache.pulsar.client.impl.PulsarClientImpl.lambda$doSingleTopicSubscribeAsync$5(PulsarClientImpl.java:364) ~[classes/:?]
	at java.util.concurrent.CompletableFuture.uniAccept(CompletableFuture.java:656) ~[?:1.8.0_131]
	at java.util.concurrent.CompletableFuture$UniAccept.tryFire(CompletableFuture.java:632) ~[?:1.8.0_131]
...

Caused by: java.util.jar.JarException: file:/Users/jia/.m2/repository/org/apache/pulsar/bouncy-castle-bc-shaded/2.7.0-SNAPSHOT/bouncy-castle-bc-shaded-2.7.0-SNAPSHOT.jar has unsigned entries - org/bouncycastle/cert/AttributeCertificateHolder.class
	at javax.crypto.JarVerifier.verifySingleJar(JarVerifier.java:500) ~[?:1.8.0_121]
	at javax.crypto.JarVerifier.verifyJars(JarVerifier.java:361) ~[?:1.8.0_121]
	at javax.crypto.JarVerifier.verify(JarVerifier.java:289) ~[?:1.8.0_121]
	at javax.crypto.JceSecurity.verifyProviderJar(JceSecurity.java:159) ~[?:1.8.0_121]
	at javax.crypto.JceSecurity.getVerificationResult(JceSecurity.java:185) ~[?:1.8.0_121]
	at javax.crypto.Cipher.getInstance(Cipher.java:653) ~[?:1.8.0_121]

Modifications

Remove bouncy castle shaded module, avoid package bouncy castle into a dependency jar.
enhance test case to identify this error.

Verifying this change

ut passed.

documentation

put doc changes in a separate PR #7454. so it is easier for the release manager to cherry-pick this PR.

codelipenghui · 2020-07-05T09:35:20Z

/pulsarbot run-failure-checks

jiazhai · 2020-07-05T23:11:18Z

/pulsarbot run-failure-checks

sijie · 2020-07-14T01:55:13Z

/pulsarbot run-failure-checks

Jennifer88huang-zz · 2020-07-14T03:30:14Z

/pulsarbot run-failure-checks

merlimat · 2020-07-15T20:46:00Z

/pulsarbot run-failure-checks

codelipenghui · 2020-07-15T23:59:56Z

/pulsarbot run-failure-checks

codelipenghui · 2020-07-16T12:56:13Z

/pulsarbot run-failure-checks

sijie · 2020-07-17T04:00:55Z

ping @jiazhai

jiazhai · 2020-07-18T13:35:00Z

rebased with latest master

jiazhai · 2020-07-18T15:37:31Z

/pulsarbot run-failure-checks

jiazhai · 2020-07-23T06:07:10Z

/pulsarbot run-failure-checks

jiazhai · 2020-07-23T08:01:07Z

/pulsarbot run-failure-checks

jiazhai · 2020-07-23T08:50:09Z

/pulsarbot run-failure-checks

wolfstudy · 2020-10-21T01:14:04Z

/pulsarbot run-failure-checks

eolivelli · 2020-10-21T06:36:23Z

In #8270 with the BK upgrade we will bring in BouncyCastle again.
this change is coming with perfect timing

jiazhai · 2020-10-22T13:51:48Z

/pulsarbot run-failure-checks

jiazhai · 2020-10-22T23:45:20Z

/pulsarbot run-failure-checks

jiazhai · 2020-10-23T03:15:55Z

/pulsarbot run-failure-checks

jiazhai · 2020-10-23T06:07:51Z

/pulsarbot run-failure-checks

jiazhai · 2020-10-23T06:34:08Z

would like to remove the 2.6.2 tag, since this pr changed some pom files

### Motivation The docs of bc need an update after merge PR #7453, also we could include bc in the pulsar client to avoid reference it in other modules. ### Modifications - fix the docs after fix PR 7453; - make pulsar client include bc module; avoid other modules to include bc. - fix the pulsar-function bc jar issue. ### Verifying this change all the tests passed.

…leJar (apache#7453) ### Motivation shade bouncy castle will cause some signature errors, this PR tries to remove the bouncy castle shaded module. Here is the related error stack: ``` 10:01:34.257 [pulsar-client-io-33-1] ERROR org.apache.pulsar.client.impl.ConsumerImpl - MessageCryptoBc may not included in the jar. e: java.lang.SecurityException: JCE cannot authenticate the provider BC at javax.crypto.Cipher.getInstance(Cipher.java:657) ~[?:1.8.0_121] at javax.crypto.Cipher.getInstance(Cipher.java:596) ~[?:1.8.0_121] at org.apache.pulsar.client.impl.crypto.MessageCryptoBc.<init>(MessageCryptoBc.java:147) ~[classes/:?] at org.apache.pulsar.client.impl.ConsumerImpl.<init>(ConsumerImpl.java:270) ~[classes/:?] at org.apache.pulsar.client.impl.ConsumerImpl.newConsumerImpl(ConsumerImpl.java:209) ~[classes/:?] at org.apache.pulsar.client.impl.PulsarClientImpl.lambda$doSingleTopicSubscribeAsync$5(PulsarClientImpl.java:364) ~[classes/:?] at java.util.concurrent.CompletableFuture.uniAccept(CompletableFuture.java:656) ~[?:1.8.0_131] at java.util.concurrent.CompletableFuture$UniAccept.tryFire(CompletableFuture.java:632) ~[?:1.8.0_131] ... Caused by: java.util.jar.JarException: file:/Users/jia/.m2/repository/org/apache/pulsar/bouncy-castle-bc-shaded/2.7.0-SNAPSHOT/bouncy-castle-bc-shaded-2.7.0-SNAPSHOT.jar has unsigned entries - org/bouncycastle/cert/AttributeCertificateHolder.class at javax.crypto.JarVerifier.verifySingleJar(JarVerifier.java:500) ~[?:1.8.0_121] at javax.crypto.JarVerifier.verifyJars(JarVerifier.java:361) ~[?:1.8.0_121] at javax.crypto.JarVerifier.verify(JarVerifier.java:289) ~[?:1.8.0_121] at javax.crypto.JceSecurity.verifyProviderJar(JceSecurity.java:159) ~[?:1.8.0_121] at javax.crypto.JceSecurity.getVerificationResult(JceSecurity.java:185) ~[?:1.8.0_121] at javax.crypto.Cipher.getInstance(Cipher.java:653) ~[?:1.8.0_121] ``` ### Modifications - Remove bouncy castle shaded module, avoid package bouncy castle into a dependency jar. - enhance test case to identify this error. ### Verifying this change ut passed. * remove dep of bc-shaded from other module * remove bc-shaded module * enhance testECDSAEncryption and testRSAEncryption to cover error case * fix license check * remove bc-shaded module * build a jar in jar to avoid break bc signature * use new bc dependency by classifier in maven * build pulsar-all docker image instead of pull from dockerhub in integration tests * remove nar * fix licence, fix error brings in apache#7640 * add bc when broker/client is referenced in pom * add missing bc reference in pom * change ci back to not build docker image

### Motivation The docs of bc need an update after merge PR apache#7453, also we could include bc in the pulsar client to avoid reference it in other modules. ### Modifications - fix the docs after fix PR 7453; - make pulsar client include bc module; avoid other modules to include bc. - fix the pulsar-function bc jar issue. ### Verifying this change all the tests passed.

jiazhai requested review from codelipenghui, merlimat and sijie July 5, 2020 03:36

jiazhai self-assigned this Jul 5, 2020

jiazhai added release/2.5.3 release/2.6.1 area/security labels Jul 5, 2020

jiazhai mentioned this pull request Jul 5, 2020

[doc] fix doc after remove bc-shaded #7454

Merged

codelipenghui approved these changes Jul 5, 2020

View reviewed changes

codelipenghui added this to the 2.7.0 milestone Jul 5, 2020

sijie approved these changes Jul 6, 2020

View reviewed changes

Jennifer88huang-zz closed this in #7454 Jul 6, 2020

Jennifer88huang-zz reopened this Jul 14, 2020

codelipenghui closed this Jul 16, 2020

codelipenghui reopened this Jul 16, 2020

jiazhai force-pushed the remove_bc_shaded_module branch from 461d80d to 34e9cfc Compare July 18, 2020 13:34

jiazhai force-pushed the remove_bc_shaded_module branch from c63aef3 to 8a545e7 Compare July 23, 2020 07:20

remove nar

9ced015

jiazhai force-pushed the remove_bc_shaded_module branch from d61e0a0 to 9ced015 Compare October 21, 2020 03:02

jiazhai changed the title ~~Remove bouncy castle shaded module to avoid bring error of verifySingleJar~~ [WIP]Remove bouncy castle shaded module to avoid bring error of verifySingleJar Oct 21, 2020

jiazhai force-pushed the remove_bc_shaded_module branch 2 times, most recently from 86e53ff to 368b2e9 Compare October 21, 2020 13:57

fix licence, fix error brings in apache#7640

264ac78

jiazhai force-pushed the remove_bc_shaded_module branch from 368b2e9 to 264ac78 Compare October 21, 2020 14:20

add bc when broker/client is referenced in pom

a3b57ec

jiazhai closed this Oct 21, 2020

jiazhai reopened this Oct 21, 2020

add missing bc reference in pom

4de8c90

jiazhai closed this Oct 22, 2020

jiazhai reopened this Oct 22, 2020

change ci back to not build docker image

b6bb132

jiazhai changed the title ~~[WIP]Remove bouncy castle shaded module to avoid bring error of verifySingleJar~~ Remove bouncy castle shaded module to avoid bring error of verifySingleJar Oct 23, 2020

jiazhai removed release/2.5.3 release/2.6.2 labels Oct 23, 2020

jiazhai merged commit ef614ad into apache:master Oct 23, 2020

jiazhai mentioned this pull request Nov 9, 2020

[Bouncy Castle] include bc into pulsar-client fix docs #8491

Merged

Uh oh!

Conversation

jiazhai commented Jul 5, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Motivation

Modifications

Verifying this change

documentation

Uh oh!

codelipenghui commented Jul 5, 2020

Uh oh!

jiazhai commented Jul 5, 2020

Uh oh!

sijie commented Jul 14, 2020

Uh oh!

Jennifer88huang-zz commented Jul 14, 2020

Uh oh!

merlimat commented Jul 15, 2020

Uh oh!

codelipenghui commented Jul 15, 2020

Uh oh!

codelipenghui commented Jul 16, 2020

Uh oh!

sijie commented Jul 17, 2020

Uh oh!

jiazhai commented Jul 18, 2020

Uh oh!

jiazhai commented Jul 18, 2020

Uh oh!

jiazhai commented Jul 23, 2020

Uh oh!

jiazhai commented Jul 23, 2020

Uh oh!

jiazhai commented Jul 23, 2020

Uh oh!

wolfstudy commented Oct 21, 2020

Uh oh!

eolivelli commented Oct 21, 2020

Uh oh!

jiazhai commented Oct 22, 2020

Uh oh!

jiazhai commented Oct 22, 2020

Uh oh!

jiazhai commented Oct 23, 2020

Uh oh!

jiazhai commented Oct 23, 2020

Uh oh!

jiazhai commented Oct 23, 2020

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

jiazhai commented Jul 5, 2020 •

edited

Loading