[fix] PIP-457: Remove support for V1 topic names and V1 Admin API (implementation)#25304
Open
merlimat wants to merge 42 commits intoapache:masterfrom
Open
[fix] PIP-457: Remove support for V1 topic names and V1 Admin API (implementation)#25304merlimat wants to merge 42 commits intoapache:masterfrom
merlimat wants to merge 42 commits intoapache:masterfrom
Conversation
merlimat
added
type/cleanup
github-actions
bot
added
the
doc-required
…plementation)
This commit implements PIP-457 by removing all V1 topic name support
and V1 Admin API endpoints from Apache Pulsar.
Topic naming changes:
- Remove V1 topic name format (persistent://tenant/cluster/namespace/topic)
Only V2 format is now supported (persistent://tenant/namespace/topic)
- Remove V1 namespace name format (tenant/cluster/namespace)
Only V2 format is now supported (tenant/namespace)
- TopicName and NamespaceName constructors now reject V1 format names
- fromPersistenceNamingEncoding() still handles 5-part managed ledger
names for backward compatibility with existing stored data
V1 Admin API removal:
- Delete entire org.apache.pulsar.broker.admin.v1 package
(Namespaces, PersistentTopics, NonPersistentTopics, Properties,
Brokers, BrokerStats, Clusters, Functions, ResourceQuotas,
SchemasResource)
- Delete V1 topic lookup endpoint (broker.lookup.v1.TopicLookup)
- Delete V1 WebSocket stats endpoint (websocket.admin.v1)
- Remove V1 servlet registrations from PulsarWebResource,
ProxyServiceStarter, WebSocketServiceStarter
- Remove /admin/v1/ path prefix handling from all client admin impls
(BrokerStatsImpl, BrokersImpl, LookupImpl, NamespacesImpl,
NonPersistentTopicsImpl, ResourceQuotasImpl, SchemasImpl,
TopicPoliciesImpl, TopicsImpl)
- Remove V1 cluster-aware namespace handling from NamespacesBase
Deprecated Properties API removal:
- Delete Properties.java interface (V1 term for Tenants)
- Remove Properties compat bridge methods from TenantsImpl
- Remove properties() accessor from PulsarAdmin/PulsarAdminImpl
TopicVersion enum removal:
- Delete TopicVersion.java (V1/V2 enum)
- Remove TopicVersion parameter from healthcheck API
(Brokers interface, BrokersImpl, BrokersBase, CmdBrokers)
- Remove topicVersion HTTP query parameter from healthcheck requests
- Simplify PulsarService.runHealthCheck() and BrokerService
V1 heartbeat and SLA namespace cleanup:
- Remove V1 heartbeat namespace format (pulsar/cluster/brokerId)
Rename V2 format methods to be the standard (pulsar/brokerId)
- Update SLA namespace to V2 format (sla-monitor/brokerId)
- Simplify HealthChecker to use single heartbeat topic
- Remove V1 heartbeat registration from ServiceUnitStateTableViewBase
Property to tenant terminology rename:
- Rename parameter/variable names from "property" to "tenant"
in AdminResource, TransactionsBase, and 7 V2 Namespaces endpoints
- Fix @PathParam("property") to @PathParam("tenant") in V2 Namespaces
- Rename in test utilities and test variables
Configuration cleanup:
- Remove allowAutoTopicCreationType config (V1 only)
- Remove V1 auto-topic-creation settings from broker.conf/standalone.conf
Test cleanup:
- Delete V1-specific test suites (V1AdminApiTest, V1AdminApi2Test,
V1ProducerConsumerTest, V1ProducerConsumerBase,
V1ProxyAuthenticationTest)
- Remove V1 test cases from TopicNameTest, NamespaceNameTest
- Update all remaining tests to use V2 format exclusively
- Remove TopicVersion test parameters from AdminApiHealthCheckTest
- Update SLAMonitoringTest for 2-part SLA namespace format
- Fix import order in AdminResource (HashSet before Map) - Remove unused ArrayList and List imports from PulsarWebResource
Migrate all test files from V1 topic name format (persistent://tenant/cluster/namespace/topic) to V2 format (persistent://tenant/namespace/topic). Key changes: - Replace V1 topic names with V2 in 82 test files across broker, proxy, websocket, client, and compaction tests - Update MockBrokerService HTTP handler to match V2 admin URLs (/admin/v2/persistent and /lookup/v2/topic/persistent) - Fix test setup cluster references from "use" to "test" to match the broker's configured cluster name - Add my-property/my-ns tenant+namespace creation in BrokerTestBase for tests that previously relied on V1 implicit namespace resolution - Add namespace creation in ManagedCursorMetricsTest and StrategicCompactionTest setups - Fix ResendRequestTest and PerMessageUnAcknowledgedRedeliveryTest to call baseSetup() instead of internalSetup() - Remove V1-specific test methods (namespacesCreateV1, namespacesCreateV1WithBundlesAndClusters) - Remove V1-specific test assertions (non-local cluster topic rejection in ServerCnxTest, V1 managed ledger path listener in TopicResourcesTest) - Update PulsarAdminToolTest createNamespace verifications to match V2 API call signature
…ckstyle - Convert 4-part V1 topic names to 3-part V2 format in proxy tests, BrokerServiceLookupTest, and ConnectionPoolTest - Fix V1 3-part namespace creation in ModularLoadManagerImplTest and AdminApiSchemaTest - Remove V1-specific test methods from SchemaUpdateStrategyTest - Remove duplicate tenant creation in ResendRequestTest - Update V1 format references in comments and CLI parameter descriptions - Fix checkstyle import order in StrategicCompactionTest and ManagedCursorMetricsTest
…pace Use public/default namespace (created by setupDefaultTenantAndNamespace) instead of sample/local which was never created by these tests.
- NamespaceIsolationPolicyImplTest: Change V1 3-part namespaces (pulsar/use/testns-1) to V2 format (pulsar/testns-1) in namespace regex patterns and NamespaceName.get() calls - NamespaceIsolationPoliciesTest: Fix testBrokerAssignment to use a namespace that doesn't match the policy for the shared broker test, and fix case-sensitivity issue in testGetNamespaceIsolationPolicyByNamespace
Change /ws/consumer/ and /ws/producer/ (V1 4-part format with cluster) to /ws/v2/consumer/ and /ws/v2/producer/ (V2 3-part format).
…Test
Replace getNamespaces("prop-xyz", "test") (V1 2-arg cluster-scoped API)
with getNamespaces("prop-xyz") (V2 tenant-scoped API).
- mockBundleName: change 3-part V1 format to 2-part V2 format - testBrokerAffinity: remove cluster from namespace path - testLoadSheddingWithNamespaceIsolationPolicies: remove cluster from namespace string - testNamespaceIsolationPoliciesForPrimaryAndSecondaryBrokers: fix misaligned String.format args after template change from 3-part to 2-part namespace format
Remove V1 3-part namespace formats (tenant/cluster/namespace) and V1 lookup URLs (/lookup/v2/destination/) from tests. Add missing tenant/namespace creation and replication_clusters configuration where V2 topic operations now require them. Files fixed: AdminTest, NamespacesV2Test, PersistentTopicsTest, TopicPoliciesTest, AntiAffinityNamespaceGroupExtensionTest, HttpTopicLookupv2Test, PeerReplicatorTest, ReplicatorGlobalNSTest, WebServiceTest.
Change topic names from V1 format (sample/local/*) to V2 (public/default/*) and add setupDefaultTenantAndNamespace() call.
…tem topic names ProxyProtocolTest: Add internalSetUpForNamespace() call so the my-property/my-ns namespace exists before producing. CompactionRetentionTest: Use getLocalName() when constructing system topic names to avoid concatenating full TopicName objects with the topic prefix, which created malformed topic names.
Replace V1 topic namespace sample/local with public/default across all proxy test classes and add setupDefaultTenantAndNamespace() to ensure the namespace exists before tests run.
After V1 topic name removal, many tests used V1 3-part namespace patterns (tenant/cluster/namespace) or referenced namespaces that no longer exist. This fixes all remaining V1 namespace issues found in CI test failures: - BrokerBookieIsolationTest: Convert V1 namespace format to V2 - AntiAffinityNamespaceGroupTest: Remove cluster from namespace - ModularLoadManagerImplTest: Fix cluster name for testBrokerAffinity - HttpTopicLookupv2Test: Add cross-colo policies and peer clusters - AdminTest: Remove pre-policy-creation assertions in resourceQuotas - AdminApi2Test: Include "global" cluster in getClusters results - NamespacesTest: Fix replication cluster expectations and redirects - PrometheusMetricsTest: Sort metrics by topic for stable ordering - WebServiceTest: Adjust rate limit counters for tenant/ns creation - TlsProducerConsumerTest/TlsSniTest: Add namespace setup - BrokerClientIntegrationTest: Fix namespace setup after broker restart - TopicsConsumerImplTest: Add namespace creation for prop/ns-abc - PerMessageUnAcknowledgedRedeliveryTest: Remove redundant tenant creation - ZeroQueueSizeTest: Use prop/ns-abc (created by baseSetup) - PulsarClientToolTest/WsTest: Fix V1 topic names and setup - NamespacesImpl: Rename query param "property" to "tenant"
The subclass overrides setup() but was missing the setupDefaultTenantAndNamespace() call that the parent class has.
TopicsConsumerImplTest: Remove redundant createCluster("test") calls
in testDefaultBacklogTTL and multiTopicsInDifferentNameSpace since the
cluster is already created by producerBaseSetup() in @BeforeMethod.
CompactionRetentionTest: Remove checks for TRANSACTION_COORDINATOR_ASSIGN
and TRANSACTION_COORDINATOR_LOG in testRetentionPolicesForSystemTopic.
These topics are only recognized as system topics in the pulsar/system
namespace (via startsWith check), not in user namespaces.
The setup() overrides added to call internalSetUpForNamespace() were missing @BeforeMethod annotations. TestNG does not inherit configuration annotations on overridden methods, so setup() was never being called, leaving this.pulsar as null.
…ertion The failover active consumer is selected via consistent hashing of the topic name. With V1 removal, topic names changed from 4-part to 3-part format, altering the hash and changing which consumer is selected as active. Make the test dynamically detect the active vs standby consumer instead of hardcoding the expectation.
The broker's internal PulsarClient (created lazily by SystemTopicBasedTopicPoliciesService during namespace bundle ownership) adds extra binary connections during test setup. Use >= assertions for cumulative connection counts that include setup connections, while keeping exact assertions for failure count and zero-value checks.
- Remove "global" from mocked clusters set (V1-specific) - Remove test block for V1-specific "global not allowed in replication clusters" behavior - Add peer cluster configuration for "usc" since NamespaceName.isGlobal() now always returns true, causing all namespaces to go through the peer-cluster redirect path in checkLocalOrGetPeerReplicationCluster - Update deleteNamespace redirect URL expectation to include ?authoritative=false query param added by validateNamespacePoliciesAsync
After V1 removal, managed ledger names use the format tenant/namespace/domain/topic. The parseNamespaceFromLedgerName method captured the first 3 path components (tenant/namespace/domain) as the namespace, which was correct for V1 format (tenant/cluster/namespace/domain/topic) but incorrect for V2. Fix by extracting only groups 2 and 3 (tenant/namespace) from the regex, excluding the domain component.
After V1 removal, NamespaceName.isGlobal() always returns true, causing checkLocalOrGetPeerReplicationCluster to run for all namespaces. The guard at line 875 excluded heartbeat namespaces but not SLA namespaces, so SLA namespace lookups failed with "Namespace not found" when no policies existed in the metadata store. Fix: use isSLAOrHeartbeatNamespace() to skip peer cluster validation for both system namespace types. Also fix DispatcherBlockConsumerTest.testBlockDispatcherStats which referenced namespace prop/ns-abc that is not created by ProducerConsumerBase setup.
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## master #25304 +/- ##
=============================================
+ Coverage 37.31% 72.69% +35.37%
- Complexity 13330 34204 +20874
=============================================
Files 1910 1954 +44
Lines 151980 154703 +2723
Branches 17303 17701 +398
=============================================
+ Hits 56711 112454 +55743
+ Misses 87546 33172 -54374
- Partials 7723 9077 +1354
Flags with carried forward coverage won't be shown. Click here to find out more.
🚀 New features to boost your workflow:
|
# Conflicts: # pulsar-broker/src/main/java/org/apache/pulsar/broker/admin/v1/Namespaces.java # pulsar-broker/src/test/java/org/apache/pulsar/broker/admin/AdminApi2Test.java # pulsar-broker/src/test/java/org/apache/pulsar/broker/admin/NamespacesTest.java # pulsar-broker/src/test/java/org/apache/pulsar/broker/admin/v1/V1AdminApi2Test.java # pulsar-broker/src/test/java/org/apache/pulsar/broker/service/BrokerTestBase.java # pulsar-broker/src/test/java/org/apache/pulsar/broker/service/ReplicatorTestBase.java # pulsar-broker/src/test/java/org/apache/pulsar/client/api/NonPersistentTopicTest.java # pulsar-client-admin/src/main/java/org/apache/pulsar/client/admin/internal/NamespacesImpl.java # pulsar-client-tools-test/src/test/java/org/apache/pulsar/admin/cli/PulsarAdminToolTest.java # pulsar-client-tools/src/main/java/org/apache/pulsar/admin/cli/CmdNamespaces.java # tests/pulsar-client-admin-shade-test/src/test/java/org/apache/pulsar/tests/integration/SimpleProducerConsumerTest.java # tests/pulsar-client-all-shade-test/src/test/java/org/apache/pulsar/tests/integration/SimpleProducerConsumerTest.java # tests/pulsar-client-shade-test/src/test/java/org/apache/pulsar/tests/integration/SimpleProducerConsumerTest.java
pulsar-common/src/main/java/org/apache/pulsar/common/naming/TopicName.java
Show resolved
Hide resolved
pulsar-broker/src/main/java/org/apache/pulsar/broker/stats/metrics/AbstractMetrics.java
Outdated
Show resolved
Hide resolved
pulsar-broker/src/main/java/org/apache/pulsar/broker/admin/impl/TenantsBase.java
Outdated
Show resolved
Hide resolved
Changed TopicName constructor to use limit(4) split and reject 4-part paths (tenant/cluster/namespace/topic) instead of silently misinterpreting them as V2 names. Also updated toFullTopicName() with the same check. Removed support for slashes in V2 topic local names since V2 format does not allow them. Updated websocket handler to no longer join slash-separated URI path segments into topic names.
Restored the check that prevents removing clusters from a tenant's allowed list when namespaces still reference those clusters. The implementation is adapted for V2: instead of checking the V1 cluster path hierarchy, it now lists all namespaces under the tenant and checks their replication_clusters policy field.
Keep the existing namespace dimension format (tenant/namespace/domain) to avoid breaking existing dashboards and alerts. The cleanup can be done separately.
- AbstractWebSocketHandlerTest: remove slashes from test topic names - BrokerBkEnsemblesTest: remove slashes from wildcard char topic name - ResourceGroupUsageAggregationTest: fix topic name that was accidentally constructing a V1 4-part path (pulsar-test/test/test/topic) - NamespacesTest: remove usw from namespace replication clusters before removing it from tenant allowed clusters, matching canUpdateCluster validation semantics
- AbstractWebSocketHandlerTest: remove slashes from test topic names - BrokerBkEnsemblesTest: remove slashes from wildcard char topic name - ResourceGroupUsageAggregationTest: fix topic name that was accidentally constructing a V1 4-part path (pulsar-test/test/test/topic) - NamespacesTest: remove usw from namespace replication clusters before removing it from tenant allowed clusters, matching canUpdateCluster validation semantics
shibd
reviewed
Mar 12, 2026
pulsar-websocket/src/main/java/org/apache/pulsar/websocket/AbstractWebSocketHandler.java
Show resolved
Hide resolved
shibd
reviewed
Mar 12, 2026
pulsar-client-admin/src/main/java/org/apache/pulsar/client/admin/internal/NamespacesImpl.java
Outdated
Show resolved
Hide resolved
The V1 cluster-scoped namespace listing is no longer relevant. Removed the method from Namespaces interface, NamespacesImpl, the list-cluster CLI command, and the validatePropertyCluster helper.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This commit implements PIP-457 by removing all V1 topic name support and V1 Admin API endpoints from Apache Pulsar.
Topic naming changes:
V1 Admin API removal:
Deprecated Properties API removal:
TopicVersion enum removal:
V1 heartbeat and SLA namespace cleanup:
Property to tenant terminology rename:
Configuration cleanup:
Test cleanup:
Fixes #xyz
Main Issue: #xyz
PIP: #xyz
Motivation
Modifications
Verifying this change
(Please pick either of the following options)
This change is a trivial rework / code cleanup without any test coverage.
(or)
This change is already covered by existing tests, such as (please describe tests).
(or)
This change added tests and can be verified as follows:
(example:)
Does this pull request potentially affect one of the following parts:
If the box was checked, please highlight the changes
Documentation
docdoc-requireddoc-not-neededdoc-completeMatching PR in forked repository
PR in forked repository: