Skip to content

HBASE-29933: update_all_config hangs indefinitely when balancing even…#7932

Open
hingu-8103 wants to merge 3 commits intoapache:branch-2from
hingu-8103:HBASE-29933-branch-2
Open

HBASE-29933: update_all_config hangs indefinitely when balancing even…#7932
hingu-8103 wants to merge 3 commits intoapache:branch-2from
hingu-8103:HBASE-29933-branch-2

Conversation

@hingu-8103
Copy link
Contributor

…t is in progress.

This solves the problem of update config client call hangs indefinitely when balancing event is in progress.
If update config call comes during balance run then it will store the configuration as pending configuration and apply it post balance run.

I've manually tested the change and also added one integration test that verifies that

  1. Update config client call doesn't hang during balance run.
  2. Old config of balancer is used through out the balance run(No config update during balance run).
  3. New config of balancer is applied after balance run is finished.

Copy link
Contributor

@vaijosh vaijosh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

*/
@Override
public void onBalancingStart() {
LOG.debug("setting isBalancing to true as balance is starting");
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: Please keep first character of sentence capital.

*/
@Override
public void onBalancingComplete() {
LOG.debug("setting isBalancing to false as balance is completed");
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: Please keep first character of sentence capital.

throw new RuntimeException(e);
synchronized (this) {
try {
wait(sleepTime);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment line here to explain its giving up monitor.

for (Map<ServerName, List<RegionInfo>> serverMap : assignments.values()) {
serverMap.keySet().removeAll(this.serverManager.getDrainingServersList());
}
Map<TableName, Map<ServerName, List<RegionInfo>>> assignments =
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can see space character here in diff? Did you fix formatting here and next few lines?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess this is because of introduction of try block. I ran "mvn spotless:apply" to fix formatting.

Copy link
Contributor

@vaijosh vaijosh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Contributor

@wchevreuil wchevreuil left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The UT failures look unrelated, but I think we need to review some "synchronized" blocks within BaseLoadBalancer/CacheAwareLoadBalancer.


@Override
public synchronized void onConfigurationChange(Configuration conf) {
public void onConfigurationChange(Configuration conf) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removing "synchronized" from here will make other Balancer implementations (StochasticLoadBalancer, SimpleBalancer) non thread-safe. And it looks that CacheAwareLoadBalancer.loadConf logic is inside a synchronized block altogether, so better to keep the synchronized here, rather than defining a syncrhonized block in CacheAwareLoadBalancer.loadConf.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@wchevreuil My bad, I missed that part. I've done the changes and executed the test related to CacheAwareLoadBalancer as well.
Thanks.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants