Skip to content

Allow function rebalance to be run periodically - #7449

Merged
jerrypeng merged 4 commits into
apache:masterfrom
jerrypeng:rebalance_scheduled
Jul 7, 2020
Merged

Allow function rebalance to be run periodically#7449
jerrypeng merged 4 commits into
apache:masterfrom
jerrypeng:rebalance_scheduled

Conversation

@jerrypeng

@jerrypeng jerrypeng commented Jul 4, 2020

Copy link
Copy Markdown
Contributor

Motivation

After a rebalance mechanism was implement for pulsar functions in:

#7388

it is only natural to add the functionality for the rebalance to be triggered periodically.

Modifications

Allow rebalance to be triggered on a user defined frequency

@jerrypeng jerrypeng added this to the 2.7.0 milestone Jul 4, 2020
@jerrypeng
jerrypeng requested a review from srkukarni July 4, 2020 02:21
@jerrypeng jerrypeng self-assigned this Jul 4, 2020
@jerrypeng
jerrypeng requested review from merlimat and sijie July 4, 2020 02:21
Comment thread conf/functions_worker.yml Outdated
failureCheckFreqMs: 30000
rescheduleTimeoutMs: 60000
# frequency at which to check if cluster needs rebalancing (15min default interval)
rebalanceCheckFreqSec: 900

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we want to turn it on by default? Also is 15 min too frequent

@jerrypeng jerrypeng Jul 6, 2020

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Set it 1 hour? Is there any harm to enable by default?

@wolfstudy

Copy link
Copy Markdown
Member

@jerrypeng can we update the Motivation and Modifications?

@jerrypeng
jerrypeng requested a review from srkukarni July 6, 2020 18:19
Comment thread conf/functions_worker.yml Outdated
@jerrypeng
jerrypeng merged commit 304685b into apache:master Jul 7, 2020
huangdx0726 pushed a commit to huangdx0726/pulsar that referenced this pull request Aug 24, 2020
Co-authored-by: Jerry Peng <jerryp@splunk.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants