Skip to content

[Proxy] Fix issue when Proxy fails to start and logs about an uncaught exception - #13171

Merged
lhotari merged 2 commits into
apache:masterfrom
lhotari:lh-fix-proxy-fails-to-start
Dec 7, 2021
Merged

[Proxy] Fix issue when Proxy fails to start and logs about an uncaught exception#13171
lhotari merged 2 commits into
apache:masterfrom
lhotari:lh-fix-proxy-fails-to-start

Conversation

@lhotari

@lhotari lhotari commented Dec 7, 2021

Copy link
Copy Markdown
Member

Motivation

The Pulsar proxy fails to start when deploying the master branch version of Pulsar with the Apache Pulsar Helm chart.
The proxy logs an obscure error message:

2021-12-07T12:27:16,052Z [jdk.internal.loader.ClassLoaders$AppClassLoader@5ffd2b27] error Uncaught exception in thread main: Failed to start HTTP server on ports [8080]

The reason why this happens is that the thread pool doesn't have sufficient amount of threads. Jetty reserves the threads from the main executor. The number of threads in the main executor can be configured with the httpNumThreads configuration parameter. (The workaround for the issue is to increase the httpNumThreads value until the proxy can start.)

Modifications

  • add logging of the stack traces for uncaught exceptions in the proxy
  • limit the number of selector threads to 1 for the Proxy http client since it will consume a lot of threads from the main executor (50% of number of CPU cores).

- happens at startup since the proxy client consumes 50% of number of available CPU cores
  for selectors
@lhotari lhotari added area/proxy doc-not-needed Your PR changes do not impact docs labels Dec 7, 2021
@lhotari lhotari self-assigned this Dec 7, 2021
@lhotari lhotari added this to the 2.10.0 milestone Dec 7, 2021
@lhotari
lhotari merged commit 3986be6 into apache:master Dec 7, 2021
fxbing pushed a commit to fxbing/pulsar that referenced this pull request Dec 19, 2021
…t exception (apache#13171)

* [Proxy] Print stacktrace in uncaught exception handler

* [Proxy] Fix IllegalStateException: Insufficient configured threads

- happens at startup since the proxy client consumes 50% of number of available CPU cores
  for selectors
codelipenghui pushed a commit that referenced this pull request Dec 21, 2021
…t exception (#13171)

* [Proxy] Print stacktrace in uncaught exception handler

* [Proxy] Fix IllegalStateException: Insufficient configured threads

- happens at startup since the proxy client consumes 50% of number of available CPU cores
  for selectors

(cherry picked from commit 3986be6)
@codelipenghui codelipenghui added the cherry-picked/branch-2.9 Archived: 2.9 is end of life label Dec 21, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/proxy cherry-picked/branch-2.9 Archived: 2.9 is end of life doc-not-needed Your PR changes do not impact docs release/2.9.2

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants