Skip to content

Fix thread leak in hive-metastore caused by thread pool per HiveTableOperations#6605

Closed
ghost wants to merge 1 commit into
masterfrom
unknown repository
Closed

Fix thread leak in hive-metastore caused by thread pool per HiveTableOperations#6605
ghost wants to merge 1 commit into
masterfrom
unknown repository

Conversation

@ghost
Copy link
Copy Markdown

@ghost ghost commented Jan 17, 2023

Fix thread leak in hive-metastore caused by new thread pool instance created for each HiveTableOperations instance

…created for each HiveTableOperations instance
@pvary
Copy link
Copy Markdown
Contributor

pvary commented Jan 17, 2023

Funny thing that I have found the same issue when running the spark tests against my PR: #6570.
#6570 would fix this too by recreating the hiveLockHeartbeatExecutorService for every Lock (the same way that we were doing for the HiveTableOperations before #6570).

I have some concerns about the proposed solution:

  • The size of the threadpool is depends on the configuration of the first executor to call the initHiveLockHeartbeatScheduledExecutorService - this would create hard to reproduce situations and it would be hard to understand why the new iceberg.hive.lock-heartbeat-threadpool-size is "not used" in some cases
  • This creates a JVM wide threadpool, and if the pool is exhausted then the heartbeating of the Lock will not happen, and it is done silently, which can cause issues later

On the plus side, we do not recreate the pool every time when there is a new HiveTableOperations/Lock

@ghost
Copy link
Copy Markdown
Author

ghost commented Jan 17, 2023

Yes, I have completely missed scenario with multiple hive configurations.
Great that issue is already resolved by #6570. Closing PR.

@ghost ghost closed this Jan 17, 2023
@pvary
Copy link
Copy Markdown
Contributor

pvary commented Jan 17, 2023

For the record #6570 has not been merged yet.
I was not clear enough, sorry for that

@ghost ghost deleted the hive-metastore_fix_thread_leak branch February 20, 2023 19:46
@ghost ghost restored the hive-metastore_fix_thread_leak branch February 20, 2023 20:13
This pull request was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant