Skip to content

HDFS-17934. RBF: Add router web address to router web UI#8540

Open
kokonguyen191 wants to merge 2 commits into
apache:trunkfrom
kokonguyen191:HDFS-17934
Open

HDFS-17934. RBF: Add router web address to router web UI#8540
kokonguyen191 wants to merge 2 commits into
apache:trunkfrom
kokonguyen191:HDFS-17934

Conversation

@kokonguyen191

@kokonguyen191 kokonguyen191 commented Jun 9, 2026

Copy link
Copy Markdown
Contributor

Description of PR

In router web interface, there are clickable URLs for namenodes and datanodes, but none for routers.

This patch uses a hacky method to guess the web URL from configuration and admin address, which works for cluster that runs the same port for all routers, instead of 100% accurately getting it using some other method that requires more intrusive changes to what's stored in the router state store. I think it's a fair compromise for a small improvement that works for most deployments.

How was this patch tested?

Local deployment and test deployment with a few nodes.

image

No unit tests since it's a minor front end change.

@hadoop-yetus

Copy link
Copy Markdown

💔 -1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 0m 23s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+0 🆗 codespell 0m 1s codespell was not available.
+0 🆗 detsecrets 0m 1s detect-secrets was not available.
+0 🆗 xmllint 0m 1s xmllint was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
-1 ❌ test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ trunk Compile Tests _
+1 💚 mvninstall 26m 4s trunk passed
+1 💚 compile 0m 41s trunk passed with JDK Ubuntu-21.0.11+10-1-24.04.2-Ubuntu
+1 💚 compile 0m 52s trunk passed with JDK Ubuntu-17.0.19+10-1-24.04.2-Ubuntu
+1 💚 checkstyle 1m 9s trunk passed
+1 💚 mvnsite 0m 55s trunk passed
+1 💚 javadoc 0m 57s trunk passed with JDK Ubuntu-21.0.11+10-1-24.04.2-Ubuntu
+1 💚 javadoc 0m 41s trunk passed with JDK Ubuntu-17.0.19+10-1-24.04.2-Ubuntu
+1 💚 spotbugs 1m 21s trunk passed
+1 💚 shadedclient 16m 25s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+1 💚 mvninstall 0m 32s the patch passed
+1 💚 compile 0m 24s the patch passed with JDK Ubuntu-21.0.11+10-1-24.04.2-Ubuntu
+1 💚 javac 0m 24s the patch passed
+1 💚 compile 0m 31s the patch passed with JDK Ubuntu-17.0.19+10-1-24.04.2-Ubuntu
+1 💚 javac 0m 31s the patch passed
+1 💚 blanks 0m 0s The patch has no blanks issues.
+1 💚 checkstyle 0m 16s the patch passed
+1 💚 mvnsite 0m 35s the patch passed
+1 💚 javadoc 0m 21s the patch passed with JDK Ubuntu-21.0.11+10-1-24.04.2-Ubuntu
+1 💚 javadoc 0m 21s the patch passed with JDK Ubuntu-17.0.19+10-1-24.04.2-Ubuntu
+1 💚 spotbugs 1m 12s the patch passed
+1 💚 shadedclient 15m 51s patch has no errors when building and testing our client artifacts.
_ Other Tests _
+1 💚 unit 42m 54s hadoop-hdfs-rbf in the patch passed.
+1 💚 asflicense 0m 25s The patch does not generate ASF License warnings.
114m 5s
Subsystem Report/Notes
Docker ClientAPI=1.54 ServerAPI=1.54 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8540/1/artifact/out/Dockerfile
GITHUB PR #8540
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets xmllint
uname Linux 0e03ef8879e7 5.15.0-181-generic #191-Ubuntu SMP Fri May 22 19:09:02 UTC 2026 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / dc55daa
Default Java Ubuntu-17.0.19+10-1-24.04.2-Ubuntu
Multi-JDK versions /usr/lib/jvm/java-21-openjdk-amd64:Ubuntu-21.0.11+10-1-24.04.2-Ubuntu /usr/lib/jvm/java-17-openjdk-amd64:Ubuntu-17.0.19+10-1-24.04.2-Ubuntu
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8540/1/testReport/
Max. process+thread count 3034 (vs. ulimit of 10000)
modules C: hadoop-hdfs-project/hadoop-hdfs-rbf U: hadoop-hdfs-project/hadoop-hdfs-rbf
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8540/1/console
versions git=2.43.0 maven=3.9.15 spotbugs=4.9.7
Powered by Apache Yetus 0.14.1 https://yetus.apache.org

This message was automatically generated.

@ZanderXu ZanderXu left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Make Sense. LGTM + 1

Comment on lines +395 to +396
String host = adminAddress.split(":")[0];
return "http://" + host + ":" + webPort;

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

how do u know it is http only, we have DFS_ROUTER_HTTPS_ADDRESS_KEY as well

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fair point, updated with HTTPS

return JSON.toString(info);
}

private static String guessRouterWebAddress(String adminAddress, int webPort) {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

guess isn't a very nice word here & moreover we can't rely on guesswork either

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated methods to remove guess naming. If we want fully correct per router web address, router state store has to store those data (which currently it does not). I think for a simple web UI improvement and for most deployments with a consistent config for all routers in a cluster, this is a fair compromise to arrive at

@hadoop-yetus

Copy link
Copy Markdown

💔 -1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 0m 22s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+0 🆗 codespell 0m 1s codespell was not available.
+0 🆗 detsecrets 0m 1s detect-secrets was not available.
+0 🆗 xmllint 0m 1s xmllint was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
-1 ❌ test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ trunk Compile Tests _
+1 💚 mvninstall 24m 41s trunk passed
+1 💚 compile 0m 40s trunk passed with JDK Ubuntu-21.0.11+10-1-24.04.2-Ubuntu
+1 💚 compile 0m 51s trunk passed with JDK Ubuntu-17.0.19+10-1-24.04.2-Ubuntu
+1 💚 checkstyle 0m 38s trunk passed
+1 💚 mvnsite 0m 56s trunk passed
+1 💚 javadoc 0m 42s trunk passed with JDK Ubuntu-21.0.11+10-1-24.04.2-Ubuntu
+1 💚 javadoc 0m 40s trunk passed with JDK Ubuntu-17.0.19+10-1-24.04.2-Ubuntu
+1 💚 spotbugs 1m 23s trunk passed
+1 💚 shadedclient 16m 41s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+1 💚 mvninstall 0m 33s the patch passed
+1 💚 compile 0m 23s the patch passed with JDK Ubuntu-21.0.11+10-1-24.04.2-Ubuntu
+1 💚 javac 0m 23s the patch passed
+1 💚 compile 0m 32s the patch passed with JDK Ubuntu-17.0.19+10-1-24.04.2-Ubuntu
+1 💚 javac 0m 32s the patch passed
+1 💚 blanks 0m 0s The patch has no blanks issues.
+1 💚 checkstyle 0m 15s the patch passed
+1 💚 mvnsite 0m 36s the patch passed
+1 💚 javadoc 0m 21s the patch passed with JDK Ubuntu-21.0.11+10-1-24.04.2-Ubuntu
+1 💚 javadoc 0m 23s the patch passed with JDK Ubuntu-17.0.19+10-1-24.04.2-Ubuntu
+1 💚 spotbugs 1m 7s the patch passed
+1 💚 shadedclient 15m 30s patch has no errors when building and testing our client artifacts.
_ Other Tests _
+1 💚 unit 42m 35s hadoop-hdfs-rbf in the patch passed.
+1 💚 asflicense 0m 25s The patch does not generate ASF License warnings.
111m 16s
Subsystem Report/Notes
Docker ClientAPI=1.54 ServerAPI=1.54 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8540/2/artifact/out/Dockerfile
GITHUB PR #8540
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets xmllint
uname Linux f9e4e3f0718d 5.15.0-181-generic #191-Ubuntu SMP Fri May 22 19:09:02 UTC 2026 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / 79e498d
Default Java Ubuntu-17.0.19+10-1-24.04.2-Ubuntu
Multi-JDK versions /usr/lib/jvm/java-21-openjdk-amd64:Ubuntu-21.0.11+10-1-24.04.2-Ubuntu /usr/lib/jvm/java-17-openjdk-amd64:Ubuntu-17.0.19+10-1-24.04.2-Ubuntu
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8540/2/testReport/
Max. process+thread count 3382 (vs. ulimit of 10000)
modules C: hadoop-hdfs-project/hadoop-hdfs-rbf U: hadoop-hdfs-project/hadoop-hdfs-rbf
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8540/2/console
versions git=2.43.0 maven=3.9.15 spotbugs=4.9.7
Powered by Apache Yetus 0.14.1 https://yetus.apache.org

This message was automatically generated.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a clickable web URL for each Router entry in the RBF Router web UI by deriving the Router web address from the Router’s admin address plus locally configured HTTP/HTTPS port/scheme.

Changes:

  • Render Router “Address” as a link when a derived routerWebAddress is available.
  • Extend RBFMetrics#getRouters() JSON output to include a routerWebAddress field computed from config + router admin address.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

File Description
hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/webapps/router/federationhealth.html Conditionally wraps router address in an <a> pointing to routerWebAddress.
hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/metrics/RBFMetrics.java Adds derivation of routerWebAddress and injects it into the Routers JSON payload.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +388 to +402
private static String getRouterWebAddress(Configuration conf, String adminAddress) {
try {
if (isNullOrEmpty(adminAddress)) {
return "";
}
String scheme = DFSUtil.getHttpClientScheme(conf);
int webPort = getRouterWebAddressPort(conf, scheme);
InetSocketAddress adminSocketAddress = NetUtils.createSocketAddr(adminAddress.trim());
return new URI(scheme, null, adminSocketAddress.getHostString(), webPort, null, null,
null).toString();
} catch (Exception e) {
LOG.error("Cannot get router web address", e);
return "";
}
}
long dateModified = record.getDateModified();
long lastHeartbeat = getSecondsSince(dateModified);
innerInfo.put("lastHeartbeat", lastHeartbeat);
innerInfo.put("routerWebAddress", getRouterWebAddress(conf, record.getAdminAddress()));
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants