[improve][cpp] Upgrade OpenSSL to version 1.1.1n - #17538
Merged
Merged
Conversation
RobertIndie
added
type/enhancement
BewareMyPower
requested review from
BewareMyPower,
Demogorgon314,
jiazhai and
merlimat
September 8, 2022 07:15
BewareMyPower
approved these changes
Sep 8, 2022
Demogorgon314
approved these changes
Sep 8, 2022
nodece
approved these changes
Sep 8, 2022
Jason918
approved these changes
Sep 9, 2022
Contributor
|
@RobertIndie Please check CI. |
shibd
approved these changes
Sep 13, 2022
michaeljmarshall
approved these changes
Sep 13, 2022
merlimat
approved these changes
Sep 13, 2022
Member
|
@RobertIndie Please fix this issue: Building debian packages fails in https://github.com/apache/pulsar/actions/runs/3019434098/jobs/4855451903#step:8:29495 That slipped into master since the CI wasn't preventing merging of builds with failing checks temporarily and this PR wasn't rebased. |
Contributor
|
FYI, I will handle this issue. |
4 tasks
BewareMyPower
added a commit
to BewareMyPower/pulsar
that referenced
this pull request
Sep 13, 2022
### Motivation See apache#17538 (comment) The root cause is when libcurl is built from source, it uses [`ld`](https://linux.die.net/man/1/ld) to check if the `libcurl.so` links to the correct dependencies in runtime. In Linux, a dynamic library links to the paths of `/etc/ld.so.conf` by default. However, different from other images like `centos:7` and `alpine`, this file includes `/usr/lib/x86_64-linux-gnu` in `debian:9`. ```bash $ cat /etc/ld.so.conf include /etc/ld.so.conf.d/*.conf $ cat /etc/ld.so.conf.d/*.conf /usr/lib/x86_64-linux-gnu/libfakeroot # libc default configuration /usr/local/lib # Multiarch support /lib/x86_64-linux-gnu /usr/lib/x86_64-linux-gnu ``` When libcurl is compiled, it links to the install path of libopenssl via the `--with-ssl` option: https://github.com/apache/pulsar/blob/1f50366768e76f1a5f7084f7972167f989ddd0af/pulsar-client-cpp/pkg/deb/Dockerfile#L85 i.e. `/usr/local/ssl/lib/libopenssl.so`. However, after the `libcurl.so` is built, it links to `/usr/lib/x86_64-linux-gnu/libssl.so.1.1`, see the following output: ```bash $ ldd /usr/local/lib/libcurl.so /usr/local/lib/libcurl.so: /usr/lib/x86_64-linux-gnu/libssl.so.1.1: version `OPENSSL_1_1_1' not found (required by /usr/local/lib/libcurl.so) ``` In `debian:9`, the default libopenssl version is 1.1.0: ```bash $ strings /usr/lib/x86_64-linux-gnu/libssl.so.1.1 | grep OpenSSL OpenSSL 1.1.0l 10 Sep 2019 ``` The ABI compatibility is not guaranteed between 1.1.0l and 1.1.1n, see https://abi-laboratory.pro/index.php?view=timeline&l=openssl. ### Modifications Set the `LD_LIBRARY_PATH` to `/usr/local/ssl/lib` in the Dockerfile to build deb package. Actually it's not required for other images like `centos:7`, but it's also good to add the `LD_LIBRARY_PATH` to them. So this PR set the environment variable to them as well.
4 tasks
lhotari
added a commit
that referenced
this pull request
Sep 13, 2022
* [fix][cpp] Fix libcurl build failure when building deb package ### Motivation See #17538 (comment) The root cause is when libcurl is built from source, it uses [`ld`](https://linux.die.net/man/1/ld) to check if the `libcurl.so` links to the correct dependencies in runtime. In Linux, a dynamic library links to the paths of `/etc/ld.so.conf` by default. However, different from other images like `centos:7` and `alpine`, this file includes `/usr/lib/x86_64-linux-gnu` in `debian:9`. ```bash $ cat /etc/ld.so.conf include /etc/ld.so.conf.d/*.conf $ cat /etc/ld.so.conf.d/*.conf /usr/lib/x86_64-linux-gnu/libfakeroot # libc default configuration /usr/local/lib # Multiarch support /lib/x86_64-linux-gnu /usr/lib/x86_64-linux-gnu ``` When libcurl is compiled, it links to the install path of libopenssl via the `--with-ssl` option: https://github.com/apache/pulsar/blob/1f50366768e76f1a5f7084f7972167f989ddd0af/pulsar-client-cpp/pkg/deb/Dockerfile#L85 i.e. `/usr/local/ssl/lib/libopenssl.so`. However, after the `libcurl.so` is built, it links to `/usr/lib/x86_64-linux-gnu/libssl.so.1.1`, see the following output: ```bash $ ldd /usr/local/lib/libcurl.so /usr/local/lib/libcurl.so: /usr/lib/x86_64-linux-gnu/libssl.so.1.1: version `OPENSSL_1_1_1' not found (required by /usr/local/lib/libcurl.so) ``` In `debian:9`, the default libopenssl version is 1.1.0: ```bash $ strings /usr/lib/x86_64-linux-gnu/libssl.so.1.1 | grep OpenSSL OpenSSL 1.1.0l 10 Sep 2019 ``` The ABI compatibility is not guaranteed between 1.1.0l and 1.1.1n, see https://abi-laboratory.pro/index.php?view=timeline&l=openssl. ### Modifications Set the `LD_LIBRARY_PATH` to `/usr/local/ssl/lib` in the Dockerfile to build deb package. Actually it's not required for other images like `centos:7`, but it's also good to add the `LD_LIBRARY_PATH` to them. So this PR set the environment variable to them as well. * Fix workflow so that cpp-tests isn't skipped * Revisit workflow fix to cover doc only workflows too * Fix multi-line condition Co-authored-by: Lari Hotari <lhotari@apache.org>
tisonkun
pushed a commit
to tisonkun/pulsar
that referenced
this pull request
Sep 14, 2022
tisonkun
pushed a commit
to tisonkun/pulsar
that referenced
this pull request
Sep 14, 2022
…e#17614) * [fix][cpp] Fix libcurl build failure when building deb package ### Motivation See apache#17538 (comment) The root cause is when libcurl is built from source, it uses [`ld`](https://linux.die.net/man/1/ld) to check if the `libcurl.so` links to the correct dependencies in runtime. In Linux, a dynamic library links to the paths of `/etc/ld.so.conf` by default. However, different from other images like `centos:7` and `alpine`, this file includes `/usr/lib/x86_64-linux-gnu` in `debian:9`. ```bash $ cat /etc/ld.so.conf include /etc/ld.so.conf.d/*.conf $ cat /etc/ld.so.conf.d/*.conf /usr/lib/x86_64-linux-gnu/libfakeroot # libc default configuration /usr/local/lib # Multiarch support /lib/x86_64-linux-gnu /usr/lib/x86_64-linux-gnu ``` When libcurl is compiled, it links to the install path of libopenssl via the `--with-ssl` option: https://github.com/apache/pulsar/blob/1f50366768e76f1a5f7084f7972167f989ddd0af/pulsar-client-cpp/pkg/deb/Dockerfile#L85 i.e. `/usr/local/ssl/lib/libopenssl.so`. However, after the `libcurl.so` is built, it links to `/usr/lib/x86_64-linux-gnu/libssl.so.1.1`, see the following output: ```bash $ ldd /usr/local/lib/libcurl.so /usr/local/lib/libcurl.so: /usr/lib/x86_64-linux-gnu/libssl.so.1.1: version `OPENSSL_1_1_1' not found (required by /usr/local/lib/libcurl.so) ``` In `debian:9`, the default libopenssl version is 1.1.0: ```bash $ strings /usr/lib/x86_64-linux-gnu/libssl.so.1.1 | grep OpenSSL OpenSSL 1.1.0l 10 Sep 2019 ``` The ABI compatibility is not guaranteed between 1.1.0l and 1.1.1n, see https://abi-laboratory.pro/index.php?view=timeline&l=openssl. ### Modifications Set the `LD_LIBRARY_PATH` to `/usr/local/ssl/lib` in the Dockerfile to build deb package. Actually it's not required for other images like `centos:7`, but it's also good to add the `LD_LIBRARY_PATH` to them. So this PR set the environment variable to them as well. * Fix workflow so that cpp-tests isn't skipped * Revisit workflow fix to cover doc only workflows too * Fix multi-line condition Co-authored-by: Lari Hotari <lhotari@apache.org>
Contributor
|
@RobertIndie Can you help cherry-pick this to branch-2.10? There are some conflicts. |
Contributor
|
We should cherry-pick #17614 as well because it's a fix on this PR. |
Jason918
pushed a commit
that referenced
this pull request
Sep 17, 2022
(cherry picked from commit 06bac43)
Jason918
pushed a commit
that referenced
this pull request
Sep 17, 2022
* [fix][cpp] Fix libcurl build failure when building deb package See #17538 (comment) The root cause is when libcurl is built from source, it uses [`ld`](https://linux.die.net/man/1/ld) to check if the `libcurl.so` links to the correct dependencies in runtime. In Linux, a dynamic library links to the paths of `/etc/ld.so.conf` by default. However, different from other images like `centos:7` and `alpine`, this file includes `/usr/lib/x86_64-linux-gnu` in `debian:9`. ```bash $ cat /etc/ld.so.conf include /etc/ld.so.conf.d/*.conf $ cat /etc/ld.so.conf.d/*.conf /usr/lib/x86_64-linux-gnu/libfakeroot /usr/local/lib /lib/x86_64-linux-gnu /usr/lib/x86_64-linux-gnu ``` When libcurl is compiled, it links to the install path of libopenssl via the `--with-ssl` option: https://github.com/apache/pulsar/blob/1f50366768e76f1a5f7084f7972167f989ddd0af/pulsar-client-cpp/pkg/deb/Dockerfile#L85 i.e. `/usr/local/ssl/lib/libopenssl.so`. However, after the `libcurl.so` is built, it links to `/usr/lib/x86_64-linux-gnu/libssl.so.1.1`, see the following output: ```bash $ ldd /usr/local/lib/libcurl.so /usr/local/lib/libcurl.so: /usr/lib/x86_64-linux-gnu/libssl.so.1.1: version `OPENSSL_1_1_1' not found (required by /usr/local/lib/libcurl.so) ``` In `debian:9`, the default libopenssl version is 1.1.0: ```bash $ strings /usr/lib/x86_64-linux-gnu/libssl.so.1.1 | grep OpenSSL OpenSSL 1.1.0l 10 Sep 2019 ``` The ABI compatibility is not guaranteed between 1.1.0l and 1.1.1n, see https://abi-laboratory.pro/index.php?view=timeline&l=openssl. Set the `LD_LIBRARY_PATH` to `/usr/local/ssl/lib` in the Dockerfile to build deb package. Actually it's not required for other images like `centos:7`, but it's also good to add the `LD_LIBRARY_PATH` to them. So this PR set the environment variable to them as well. * Fix workflow so that cpp-tests isn't skipped * Revisit workflow fix to cover doc only workflows too * Fix multi-line condition Co-authored-by: Lari Hotari <lhotari@apache.org> (cherry picked from commit 0754ea1)
nicoloboschi
pushed a commit
to datastax/pulsar
that referenced
this pull request
Sep 20, 2022
…e#17614) * [fix][cpp] Fix libcurl build failure when building deb package See apache#17538 (comment) The root cause is when libcurl is built from source, it uses [`ld`](https://linux.die.net/man/1/ld) to check if the `libcurl.so` links to the correct dependencies in runtime. In Linux, a dynamic library links to the paths of `/etc/ld.so.conf` by default. However, different from other images like `centos:7` and `alpine`, this file includes `/usr/lib/x86_64-linux-gnu` in `debian:9`. ```bash $ cat /etc/ld.so.conf include /etc/ld.so.conf.d/*.conf $ cat /etc/ld.so.conf.d/*.conf /usr/lib/x86_64-linux-gnu/libfakeroot /usr/local/lib /lib/x86_64-linux-gnu /usr/lib/x86_64-linux-gnu ``` When libcurl is compiled, it links to the install path of libopenssl via the `--with-ssl` option: https://github.com/apache/pulsar/blob/1f50366768e76f1a5f7084f7972167f989ddd0af/pulsar-client-cpp/pkg/deb/Dockerfile#L85 i.e. `/usr/local/ssl/lib/libopenssl.so`. However, after the `libcurl.so` is built, it links to `/usr/lib/x86_64-linux-gnu/libssl.so.1.1`, see the following output: ```bash $ ldd /usr/local/lib/libcurl.so /usr/local/lib/libcurl.so: /usr/lib/x86_64-linux-gnu/libssl.so.1.1: version `OPENSSL_1_1_1' not found (required by /usr/local/lib/libcurl.so) ``` In `debian:9`, the default libopenssl version is 1.1.0: ```bash $ strings /usr/lib/x86_64-linux-gnu/libssl.so.1.1 | grep OpenSSL OpenSSL 1.1.0l 10 Sep 2019 ``` The ABI compatibility is not guaranteed between 1.1.0l and 1.1.1n, see https://abi-laboratory.pro/index.php?view=timeline&l=openssl. Set the `LD_LIBRARY_PATH` to `/usr/local/ssl/lib` in the Dockerfile to build deb package. Actually it's not required for other images like `centos:7`, but it's also good to add the `LD_LIBRARY_PATH` to them. So this PR set the environment variable to them as well. * Fix workflow so that cpp-tests isn't skipped * Revisit workflow fix to cover doc only workflows too * Fix multi-line condition Co-authored-by: Lari Hotari <lhotari@apache.org> (cherry picked from commit 0754ea1) (cherry picked from commit 7162c30)
This was referenced Sep 23, 2022
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
9 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
Currently, there exists two different OpenSSL version in the repo: 1.1.1n and 1.1.0j. This PR upgrades all these OpenSSL dependencies to 1.1.1n.
Modifications
Documentation
Check the box below or label this PR directly.
Need to update docs?
doc-required(Your PR needs to update docs and you will update later)
doc-not-needed(Please explain why)
doc(Your PR contains doc changes)
doc-complete(Docs have been already added)