[improve][authentication] Support for get token from HTTP params

[nodece]

Signed-off-by: Zixuan Liu nodeces@gmail.com

Fixes #16626

Motivation

A scenario that running the standalone Pulsar with WebSocket service, then using the token request param in a WebSocket URL gives an unauthorized response, it seems is not working as intended in the documentation, but it works using a separate WebSocket service.

The root cause is that we did not check whether the token is in the HTTP parameters.

Modifications

• Add HttpServletRequestWrapper to support for get token from HTTP params
• Add a test for this changes

Documentation

• doc-not-needed

[acortes-okode]

Hi, it's first time for me submitting a review so I'm not sure if I'm doing it correctly. I have written only one consideration about checking "Bearer " prefix and I tried to test the code by building it locally but it gave me compilation errors on 'pulsar-common' (I'm unable to make it work sorry)

[nodece]

Hi, it's first time for me submitting a review so I'm not sure if I'm doing it correctly. I have written only one consideration about checking "Bearer " prefix and I tried to test the code by building it locally but it gave me compilation errors on 'pulsar-common' (I'm unable to make it work sorry)

You can use mvn clean install -Pcore-modules,-main -DskipTests -Dspotbugs.skip command to build this project.

[acortes-okode]

You can use mvn clean install -Pcore-modules,-main -DskipTests -Dspotbugs.skip command to build this project.

Thanks @nodece, I was able to pass the compilation problem on 'pulsar-common' but now I have others in 'managed-ledget', I think is something regarding lombok maybe since it says it cannot find some 'get' symbols. I'll try to figure out if I can later.

Anyways, I was trying to build it to test something I had in mind but maybe you could know it firsthand. The thing is, I'm not sure if AuthenticationProviderToken#newHttpAuthState is called for every HTTP request (correct me if I'm wrong) but, if it is, then I think one could manage to authenticate to other requests rather than the WebSocket one by using the token request query param (also not sure if this is intended). E.g.

curl "http://localhost:8080/admin/v2/brokers/configuration?token=ASDFGH...."

Thanks in advance!

[nodece]

You can use mvn clean install -Pcore-modules,-main -DskipTests -Dspotbugs.skip command to build this project.

Thanks @nodece, I was able to pass the compilation problem on 'pulsar-common' but now I have others in 'managed-ledget', I think is something regarding lombok maybe since it says it cannot find some 'get' symbols. I'll try to figure out if I can later.

Build Pulsar requires JDK 17.

Anyways, I was trying to build it to test something I had in mind but maybe you could know it firsthand. The thing is, I'm not sure if AuthenticationProviderToken#newHttpAuthState is called for every HTTP request (correct me if I'm wrong) but, if it is, then I think one could manage to authenticate to other requests rather than the WebSocket one by using the token request query param (also not sure if this is intended). E.g.

curl "http://localhost:8080/admin/v2/brokers/configuration?token=ASDFGH...."

Good idea, I verified that it worked.

[acortes-okode]

Anyways, I was trying to build it to test something I had in mind but maybe you could know it firsthand. The thing is, I'm not sure if AuthenticationProviderToken#newHttpAuthState is called for every HTTP request (correct me if I'm wrong) but, if it is, then I think one could manage to authenticate to other requests rather than the WebSocket one by using the token request query param (also not sure if this is intended). E.g.

curl "http://localhost:8080/admin/v2/brokers/configuration?token=ASDFGH...."

Good idea, I verified that it worked.

Hi @nodece, thanks for checking that!

Sorry since I think that I didn't expose correctly my concerns about that. They are regarding security, I think there is no other option than putting token information in the query param when there is a WebSocket request due to browser API limitations. But I think then that the token query param check should only be applied on WebSocket connection URIs (/ws/ and /ws/v2/) and not the rest.

Do not know if this could be addressed easily, maybe checking in the AuthorizationProviderToken for ws paths? I see references to this paths on pulsar-websocket classes:

• WebSocketConsumerServlet
• WebSocketReaderServlet
• WebSocketProducerServlet
• WebSocketPingPongServlet

But not in a "general" one, so maybe just check for /ws/** and /ws/v2/** in the same AuthorizationProviderToken is valid.

And I'm sure security here depends a lot on how Apache Pulsar is managed and if the users use that query param for other requests rather than the WebSocket ones but maybe it could be a security risk in some situations. I'm referencing an OWASP post about this.

Thank you again and sorry for not exposing clearly all my thoughts before!

[nodece]

@acortes-okode Thanks for your explanation about security!

But I think then that the token query param check should only be applied on WebSocket connection URIs (/ws/ and /ws/v2/) and not the rest.

You are right, we should add a filter to check the WebSocket connection URL.

And I'm sure security here depends a lot on how Apache Pulsar is managed and if the users use that query param for other requests rather than the WebSocket ones but maybe it could be a security risk in some situations. I'm referencing an OWASP post about this.

Hi @michaeljmarshall @lhotari, I think you will be interested in this, could you share your thoughts?

[BewareMyPower]

I found many code changes of this PR relies on #14044. @nodece Could you help verify whether could this PR cherry-picked into branch-2.8?

[nodece]

I found many code changes of this PR relies on #14044. @nodece Could you help verify whether could this PR cherry-picked into branch-2.8?

@BewareMyPower I submitted #16871 for branch-2.8.

[BewareMyPower]

Move the release/2.8.4 label to #16871

[Jason918]

Move release/2.7.5 label to #16892

[codelipenghui]

@nodece I think we don't need to cherry-pick this one to release branches? PR #14044 only happened on the master branch and will release in 2.11.0

[nodece]

@nodece I think we don't need to cherry-pick this one to release branches? PR #14044 only happened on the master branch and will release in 2.11.0

#14044 improves the HTTP authentication that adds a new methods to passing the HTTP data, but this issue is still presents in the old version, so we need to cherry-pick to release branches.

[nodece]

Move release/2.10.2 to #16986.

[nodece]

Move release/2.9.4 label to #16987.