Skip to content

Conversation

@jcassanji-southworks
Copy link
Contributor

Related command
az cosmosdb restore

Description
This PR fixes a known issue (GitHub Issue #28434) where az cosmosdb restore command fails with a (Forbidden) Database Account <account-name>-<location> does not exist error.

The Azure backend service occasionally appends the location name to the account name during the polling operation of a restore action. This causes the CLI to poll a non-existent account name (e.g., myaccount-westeurope instead of myaccount), resulting in a 403 Forbidden error.

Changes:

  • Implemented a workaround in custom.py to catch and suppress the specific HttpResponseError (403 Forbidden with "does not exist") during the polling of client.begin_create_or_update.
  • Added a fallback check using client.get to verify if the account was successfully created/restored when this error is encountered.
  • Added comprehensive unit tests in test_cosmosdb_backuprestore_scenario.py to ensure the fix handles the specific error condition correctly without masking other legitimate errors.

Testing Guide
Run the newly added unit tests to verify the fix:

python -m unittest azure.cli.command_modules.cosmosdb.tests.latest.test_cosmosdb_backuprestore_scenario.CosmosDBRestoreUnitTests

History Notes
[CosmosDB] az cosmosdb restore: Fix bug where restore operation fails with "Database Account does not exist" error due to incorrect location appending.


This checklist is used to make sure that common guidelines for a pull request are followed.

Copilot AI review requested due to automatic review settings February 5, 2026 22:05
@azure-client-tools-bot-prd
Copy link

azure-client-tools-bot-prd bot commented Feb 5, 2026

❌AzureCLI-FullTest
️✔️acr
️✔️latest
️✔️3.12
️✔️3.13
️✔️acs
️✔️latest
️✔️3.12
️✔️3.13
️✔️advisor
️✔️latest
️✔️3.12
️✔️3.13
️✔️ams
️✔️latest
️✔️3.12
️✔️3.13
️✔️apim
️✔️latest
️✔️3.12
️✔️3.13
️✔️appconfig
️✔️latest
️✔️3.12
️✔️3.13
️✔️appservice
️✔️latest
️✔️3.12
️✔️3.13
️✔️aro
️✔️latest
️✔️3.12
️✔️3.13
️✔️backup
️✔️latest
️✔️3.12
️✔️3.13
️✔️batch
️✔️latest
️✔️3.12
️✔️3.13
️✔️batchai
️✔️latest
️✔️3.12
️✔️3.13
️✔️billing
️✔️latest
️✔️3.12
️✔️3.13
️✔️botservice
️✔️latest
️✔️3.12
️✔️3.13
️✔️cdn
️✔️latest
️✔️3.12
️✔️3.13
️✔️cloud
️✔️latest
️✔️3.12
️✔️3.13
️✔️cognitiveservices
️✔️latest
️✔️3.12
️✔️3.13
️✔️compute_recommender
️✔️latest
️✔️3.12
️✔️3.13
️✔️computefleet
️✔️latest
️✔️3.12
️✔️3.13
️✔️config
️✔️latest
️✔️3.12
️✔️3.13
️✔️configure
️✔️latest
️✔️3.12
️✔️3.13
️✔️consumption
️✔️latest
️✔️3.12
️✔️3.13
️✔️container
️✔️latest
️✔️3.12
️✔️3.13
️✔️containerapp
️✔️latest
️✔️3.12
️✔️3.13
️✔️core
️✔️latest
️✔️3.12
️✔️3.13
❌cosmosdb
❌latest
❌3.12
Type Test Case Error Message Line
Failed test_restore_handles_forbidden_error self = <azure.cli.command_modules.cosmosdb.tests.latest.test_cosmosdb_backuprestore_scenario.CosmosDBRestoreUnitTests testMethod=test_restore_handles_forbidden_error>

    def test_restore_handles_forbidden_error(self):
        from azure.core.exceptions import HttpResponseError
        # Lazy import to ensure mocks are applied first
        from azure.cli.command_modules.cosmosdb.custom import create_database_account
    
        # Setup mocks
        client = mock.MagicMock()
    
        # Simulate the LRO poller raising the specific error
        poller = mock.MagicMock()
        error_json = '{"code":"Forbidden","message":"Database Account riks-models-003-acc-westeurope does not exist"}'
        exception = HttpResponseError(message=error_json)
        exception.status_code = 403
    
        # side_effect raises the exception when called
        poller.result.side_effect = exception
        client.begin_create_or_update.return_value = poller
    
        # Simulate client.get returning the account successfully
        mock_account = mock.MagicMock()
        mock_account.provisioning_state = "Succeeded"
        client.get.return_value = mock_account
    
        # Parameters
        resource_group_name = "rg"
        account_name = "myaccount"
    
        # Call the private function directly to verify logic
>       result = create_database_account(
            client=client,
            resource_group_name=resource_group_name,
            account_name=account_name,
            locations=[],
            is_restore_request=True,
            arm_location="westeurope",
            restore_source="/subscriptions/sub/providers/Microsoft.DocumentDB/locations/westeurope/restorableDatabaseAccounts/source-id",
            restore_timestamp="2026-01-01T00:00:00+00:00"
        )

src/azure-cli/azure/cli/command_modules/cosmosdb/tests/latest/test_cosmosdb_backuprestore_scenario.py:585: 
 
                                       

client = <MagicMock id='140581256392256'>, resource_group_name = 'rg'
account_name = 'myaccount'
locations = [<MagicMock name='mock.Location()' id='140581252397728'>]
tags = None
kind = <MagicMock name='mock.DatabaseAccountKind.global_document_db.value' id='140581248923456'>
default_consistency_level = None, max_staleness_prefix = 100, max_interval = 5
ip_range_filter = None, enable_automatic_failover = None, capabilities = None
enable_virtual_network = None, virtual_network_rules = None
enable_multiple_write_locations = None
disable_key_based_metadata_write_access = None, disable_local_auth = None
key_uri = None, public_network_access = None, enable_analytical_storage = None
enable_free_tier = None, server_version = None, network_acl_bypass = None
network_acl_bypass_resource_ids = None, backup_interval = None
backup_retention = None, backup_redundancy = None, assign_identity = None
default_identity = None, backup_policy_type = None, continuous_tier = None
analytical_storage_schema_type = None, databases_to_restore = None
gremlin_databases_to_restore = None, tables_to_restore = None
is_restore_request = True
restore_source = '/subscriptions/sub/providers/Microsoft.DocumentDB/locations/westeurope/restorableDatabaseAccounts/source-id'
restore_timestamp = '2026-01-01T00:00:00+00:00', arm_location = 'westeurope'
enable_partition_merge = None, enable_burst_capacity = None
enable_prpp_autoscale = None, minimal_tls_version = None, disable_ttl = None
enable_pbe = None, default_priority_level = None, source_backup_location = None

    def _create_database_account(client,
                                 resource_group_name,
                                 account_name,
                                 locations=None,
                                 tags=None,
                                 kind=DatabaseAccountKind.global_document_db.value,
                                 default_consistency_level=None,
                                 max_staleness_prefix=100,
                                 max_interval=5,
                                 ip_range_filter=None,
                                 enable_automatic_failover=None,
                                 capabilities=None,
                                 enable_virtual_network=None,
                                 virtual_network_rules=None,
                                 enable_multiple_write_locations=None,
                                 disable_key_based_metadata_write_access=None,
                                 disable_local_auth=None,
                                 key_uri=None,
                                 public_network_access=None,
                                 enable_analytical_storage=None,
                                 enable_free_tier=None,
                                 server_version=None,
                                 network_acl_bypass=None,
                                 network_acl_bypass_resource_ids=None,
                                 backup_interval=None,
                                 backup_retention=None,
                                 backup_redundancy=None,
                                 assign_identity=None,
                                 default_identity=None,
                                 backup_policy_type=None,
                                 continuous_tier=None,
                                 analytical_storage_schema_type=None,
                                 databases_to_restore=None,
                                 gremlin_databases_to_restore=None,
                                 tables_to_restore=None,
                                 is_restore_request=None,
                                 restore_source=None,
                                 restore_timestamp=None,
                                 arm_location=None,
                                 enable_partition_merge=None,
                                 enable_burst_capacity=None,
                                 enable_prpp_autoscale=None,
                                 minimal_tls_version=None,
                                 disable_ttl=None,
                                 enable_pbe=None,
                                 default_priority_level=None,
                                 source_backup_location=None):
    
        consistency_policy = None
        if default_consistency_level is not None:
            consistency_policy = ConsistencyPolicy(default_consistency_level=default_consistency_level,
                                                   max_staleness_prefix=max_staleness_prefix,
                                                   max_interval_in_seconds=max_interval)
    
        if not locations:
            locations = []
            locations.append(Location(location_name=arm_location, failover_priority=0, is_zone_redundant=False))
    
        managed_service_identity = None
        SYSTEM_ID = '[system]'
        enable_system = False
        if assign_identity is not None:
            if assign_identity == [] or (len(assign_identity) == 1 and assign_identity[0] == '[system]'):
                enable_system = True
                managed_service_identity = ManagedServiceIdentity(type=ResourceIdentityType.system_assigned.value)
            else:
                user_identities = {}
                for x in assign_identity:
                    if x != SYSTEM_ID:
                        user_identities[x] = ManagedServiceIdentityUserAssignedIdentity()  # pylint: disable=line-too-long
                    else:
                        enable_system = True
                if enable_system:
                    managed_service_identity = ManagedServiceIdentity(
                        type=ResourceIdentityType.system_assigned_user_assigned.value,
                        user_assigned_identities=user_identities
                    )
                else:
                    managed_service_identity = ManagedServiceIdentity(
                        type=ResourceIdentityType.user_assigned.value,
                        user_assigned_identities=user_identities
                    )
    
        api_properties = {}
        if kind == DatabaseAccountKind.mongo_db.value:
            api_properties['ServerVersion'] = server_version
        elif server_version is not None:
            raise CLIError('server-version is a valid argument only when kind is MongoDB.')
    
        backup_policy = None
        if backup_policy_type is not None:
            if backup_policy_type.lower() == 'periodic':
                if backup_interval is None and backup_retention is None and backup_redundancy is None:
                    raise CLIError(
                        '--backup-interval, --backup-retention or --backup-redundancy ' +
                        'must be specified when Backup Policy Type is Periodic.')
                backup_policy = PeriodicModeBackupPolicy()
                if backup_interval is not None or backup_retention is not None or backup_redundancy is not None:
                    periodic_mode_properties = PeriodicModeProperties(
                        backup_interval_in_minutes=backup_interval,
                        backup_retention_interval_in_hours=backup_retention,
                        backup_storage_redundancy=backup_redundancy
                    )
                backup_policy.periodic_mode_properties = periodic_mode_properties
            elif backup_policy_type.lower() == 'continuous':
                backup_policy = ContinuousModeBackupPolicy()
                if continuous_tier is not None:
                    continuous_mode_properties = ContinuousModeProperties(
                        tier=continuous_tier
                    )
                else:
                    continuous_mode_properties = ContinuousModeProperties(
                        tier='Continuous30Days'
                    )
                backup_policy.continuous_mode_properties = continuous_mode_properties
            else:
                raise CLIError('backup-policy-type argument is invalid.')
        elif backup_interval is not None or backup_retention is not None or backup_redundancy is not None:
            backup_policy = PeriodicModeBackupPolicy()
            periodic_mode_properties = PeriodicModeProperties(
                backup_interval_in_minutes=backup_interval,
                backup_retention_interval_in_hours=backup_retention
            )
            backup_policy.periodic_mode_properties = periodic_mode_properties
    
        analytical_storage_configuration = None
        if analytical_storage_schema_type is not None:
            analytical_storage_configuration = AnalyticalStorageConfiguration()
            analytical_storage_configuration.schema_type = analytical_storage_schema_type
    
        create_mode = CreateMode.restore.value if is_restore_request else CreateMode.default.value
        params = None
        restore_parameters = None
        if create_mode == 'Restore':
            if restore_source is None or restore_timestamp is None:
                raise CLIError('restore-source and restore-timestamp should be provided for a restore request.')
    
            restore_parameters = RestoreParameters(
                restore_mode='PointInTime',
                restore_source=restore_source,
                restore_timestamp_in_utc=restore_timestamp
            )
    
            if databases_to_restore is not None:
                restore_parameters.databases_to_restore = databases_to_restore
    
            if gremlin_databases_to_restore is not None:
                restore_parameters.gremlin_databases_to_restore = gremlin_databases_to_restore
    
            if tables_to_restore is not None:
                restore_parameters.tables_to_restore = tables_to_restore
    
            if disable_ttl is not None:
                restore_parameters.restore_with_ttl_disabled = disable_ttl
    
            if source_backup_location is not None:
                restore_parameters.source_backup_location = source_backup_location
    
        params = DatabaseAccountCreateUpdateParameters(
            location=arm_location,
            locations=locations,
            tags=tags,
            kind=kind,
            consistency_policy=consistency_policy,
            ip_rules=ip_range_filter,
            is_virtual_network_filter_enabled=enable_virtual_network,
            enable_automatic_failover=enable_automatic_failover,
            capabilities=capabilities,
            virtual_network_rules=virtual_network_rules,
            enable_multiple_write_locations=enable_multiple_write_locations,
            disable_key_based_metadata_write_access=disable_key_based_metadata_write_access,
            disable_local_auth=disable_local_auth,
            key_vault_key_uri=key_uri,
            public_network_access=public_network_access,
            api_properties=api_properties,
            enable_analytical_storage=enable_analytical_storage,
            enable_free_tier=enable_free_tier,
            network_acl_bypass=network_acl_bypass,
            network_acl_bypass_resource_ids=network_acl_bypass_resource_ids,
            backup_policy=backup_policy,
            identity=managed_service_identity,
            default_identity=default_identity,
            analytical_storage_configuration=analytical_storage_configuration,
            create_mode=create_mode,
            restore_parameters=restore_parameters,
            enable_partition_merge=enable_partition_merge,
            enable_burst_capacity=enable_burst_capacity,
            enable_per_region_per_partition_autoscale=enable_prpp_autoscale,
            minimal_tls_version=minimal_tls_version,
            enable_pbe=enable_pbe,
            default_priority_level=default_priority_level
        )
    
        async_docdb_create = client.begin_create_or_update(resource_group_name, account_name, params)
        try:
            docdb_account = async_docdb_create.result()
        except HttpResponseError as ex:
            message = str(ex)
            if (is_restore_request
                    and ex.status_code == 403
                    and "does not exist" in message
                    and ("Database Account" in message or "Forbidden" in message)):
                logger.warning(
                    "Encountered known service issue (403 'does not exist') while restoring Cosmos DB account '%s' "
                    "in resource group '%s'. Using client.get() as a workaround. Raw error: %s",
                    account_name, resource_group_name, ex
                )
            else:
                raise ex
>       return docdb_account
               ^^^^^^^^^^^^^
E       UnboundLocalError: cannot access local variable 'docdb_account' where it is not associated with a value

src/azure-cli/azure/cli/command_modules/cosmosdb/custom.py:427: UnboundLocalError
azure/cli/command_modules/cosmosdb/tests/latest/test_cosmosdb_backuprestore_scenario.py:556
Failed test_locations_database_accounts self = <azure.cli.command_modules.cosmosdb.tests.latest.test_cosmosdb_commands.CosmosDBTests testMethod=test_locations_database_accounts>
resource_group = 'cli_test_cosmosdb_account000001'

    @ResourceGroupPreparer(name_prefix='cli_test_cosmosdb_account')
    def test_locations_database_accounts(self, resource_group):
    
        write_location = 'eastus'
        read_location = 'westus'
    
        self.kwargs.update({
            'acc': self.create_random_name(prefix='cli', length=40),
            'write_location': write_location,
            'read_location': read_location
        })
    
        account1 = self.cmd('az cosmosdb create -n {acc} -g {rg} --locations regionName={write_location} failoverPriority=0 --locations regionName={read_location} failoverPriority=1').get_output_in_json()
        assert len(account1['writeLocations']) == 1
        assert len(account1['readLocations']) == 2
        assert account1['writeLocations'][0]['failoverPriority'] == 0
        assert account1['writeLocations'][0]['locationName'] == "East US"
        assert account1['readLocations'][0]['locationName'] == "West US" or account1['readLocations'][1]['locationName'] == "West US"
        assert account1['readLocations'][0]['failoverPriority'] == 1 or account1['readLocations'][1]['failoverPriority'] == 1
    
        self.cmd('az cosmosdb failover-priority-change -n {acc} -g {rg} --failover-policies {read_location}=0 {write_location}=1')
        account2 = self.cmd('az cosmosdb show -n {acc} -g {rg}').get_output_in_json()
        assert len(account2['writeLocations']) == 1
        assert len(account2['readLocations']) == 2
    
        assert account2['writeLocations'][0]['failoverPriority'] == 0
>       assert account2['writeLocations'][0]['locationName'] == "West US"
E       AssertionError: assert 'East US' == 'West US'
E         
E         - West US
E         + East US

src/azure-cli/azure/cli/command_modules/cosmosdb/tests/latest/test_cosmosdb_commands.py:247: AssertionError
azure/cli/command_modules/cosmosdb/tests/latest/test_cosmosdb_commands.py:220
Failed test_locations_database_accounts_offline self = <azure.cli.command_modules.cosmosdb.tests.latest.test_cosmosdb_commands.CosmosDBTests testMethod=test_locations_database_accounts_offline>
resource_group = 'cli_test_cosmosdb_account000001'

    @ResourceGroupPreparer(name_prefix='cli_test_cosmosdb_account')
    def test_locations_database_accounts_offline(self, resource_group):
    
        write_location = 'eastus'
        read_location = 'westus'
    
        self.kwargs.update({
            'acc': self.create_random_name(prefix='cli', length=40),
            'write_location': write_location,
            'read_location': read_location
        })
    
        account_pre_offline = self.cmd('az cosmosdb create -n {acc} -g {rg} --locations regionName={write_location} failoverPriority=0 --locations regionName={read_location} failoverPriority=1').get_output_in_json()
    
        assert account_pre_offline['writeLocations'][0]['locationName'] == "East US"
    
        # Offline write region 'East US'
        self.cmd('az cosmosdb offline-region -n {acc} -g {rg} --region {write_location}')
        account_post_offline = self.cmd('az cosmosdb show -n {acc} -g {rg}').get_output_in_json()
    
        # Assert writeLocations is switched to 'West US' after offlining 'East US' region
        assert len(account_post_offline['writeLocations']) == 1
>       assert account_post_offline['writeLocations'][0]['locationName'] == 'West US'
E       AssertionError: assert 'East US' == 'West US'
E         
E         - West US
E         + East US

src/azure-cli/azure/cli/command_modules/cosmosdb/tests/latest/test_cosmosdb_commands.py:273: AssertionError
azure/cli/command_modules/cosmosdb/tests/latest/test_cosmosdb_commands.py:250
❌3.13
Type Test Case Error Message Line
Failed test_restore_handles_forbidden_error self = <azure.cli.command_modules.cosmosdb.tests.latest.test_cosmosdb_backuprestore_scenario.CosmosDBRestoreUnitTests testMethod=test_restore_handles_forbidden_error>

    def test_restore_handles_forbidden_error(self):
        from azure.core.exceptions import HttpResponseError
        # Lazy import to ensure mocks are applied first
        from azure.cli.command_modules.cosmosdb.custom import create_database_account
    
        # Setup mocks
        client = mock.MagicMock()
    
        # Simulate the LRO poller raising the specific error
        poller = mock.MagicMock()
        error_json = '{"code":"Forbidden","message":"Database Account riks-models-003-acc-westeurope does not exist"}'
        exception = HttpResponseError(message=error_json)
        exception.status_code = 403
    
        # side_effect raises the exception when called
        poller.result.side_effect = exception
        client.begin_create_or_update.return_value = poller
    
        # Simulate client.get returning the account successfully
        mock_account = mock.MagicMock()
        mock_account.provisioning_state = "Succeeded"
        client.get.return_value = mock_account
    
        # Parameters
        resource_group_name = "rg"
        account_name = "myaccount"
    
        # Call the private function directly to verify logic
>       result = create_database_account(
            client=client,
            resource_group_name=resource_group_name,
            account_name=account_name,
            locations=[],
            is_restore_request=True,
            arm_location="westeurope",
            restore_source="/subscriptions/sub/providers/Microsoft.DocumentDB/locations/westeurope/restorableDatabaseAccounts/source-id",
            restore_timestamp="2026-01-01T00:00:00+00:00"
        )

src/azure-cli/azure/cli/command_modules/cosmosdb/tests/latest/test_cosmosdb_backuprestore_scenario.py:585: 
 
                                       

client = <MagicMock id='139751037171568'>, resource_group_name = 'rg'
account_name = 'myaccount'
locations = [<MagicMock name='mock.Location()' id='139751035502672'>]
tags = None
kind = <MagicMock name='mock.DatabaseAccountKind.global_document_db.value' id='139751037170560'>
default_consistency_level = None, max_staleness_prefix = 100, max_interval = 5
ip_range_filter = None, enable_automatic_failover = None, capabilities = None
enable_virtual_network = None, virtual_network_rules = None
enable_multiple_write_locations = None
disable_key_based_metadata_write_access = None, disable_local_auth = None
key_uri = None, public_network_access = None, enable_analytical_storage = None
enable_free_tier = None, server_version = None, network_acl_bypass = None
network_acl_bypass_resource_ids = None, backup_interval = None
backup_retention = None, backup_redundancy = None, assign_identity = None
default_identity = None, backup_policy_type = None, continuous_tier = None
analytical_storage_schema_type = None, databases_to_restore = None
gremlin_databases_to_restore = None, tables_to_restore = None
is_restore_request = True
restore_source = '/subscriptions/sub/providers/Microsoft.DocumentDB/locations/westeurope/restorableDatabaseAccounts/source-id'
restore_timestamp = '2026-01-01T00:00:00+00:00', arm_location = 'westeurope'
enable_partition_merge = None, enable_burst_capacity = None
enable_prpp_autoscale = None, minimal_tls_version = None, disable_ttl = None
enable_pbe = None, default_priority_level = None, source_backup_location = None

    def _create_database_account(client,
                                 resource_group_name,
                                 account_name,
                                 locations=None,
                                 tags=None,
                                 kind=DatabaseAccountKind.global_document_db.value,
                                 default_consistency_level=None,
                                 max_staleness_prefix=100,
                                 max_interval=5,
                                 ip_range_filter=None,
                                 enable_automatic_failover=None,
                                 capabilities=None,
                                 enable_virtual_network=None,
                                 virtual_network_rules=None,
                                 enable_multiple_write_locations=None,
                                 disable_key_based_metadata_write_access=None,
                                 disable_local_auth=None,
                                 key_uri=None,
                                 public_network_access=None,
                                 enable_analytical_storage=None,
                                 enable_free_tier=None,
                                 server_version=None,
                                 network_acl_bypass=None,
                                 network_acl_bypass_resource_ids=None,
                                 backup_interval=None,
                                 backup_retention=None,
                                 backup_redundancy=None,
                                 assign_identity=None,
                                 default_identity=None,
                                 backup_policy_type=None,
                                 continuous_tier=None,
                                 analytical_storage_schema_type=None,
                                 databases_to_restore=None,
                                 gremlin_databases_to_restore=None,
                                 tables_to_restore=None,
                                 is_restore_request=None,
                                 restore_source=None,
                                 restore_timestamp=None,
                                 arm_location=None,
                                 enable_partition_merge=None,
                                 enable_burst_capacity=None,
                                 enable_prpp_autoscale=None,
                                 minimal_tls_version=None,
                                 disable_ttl=None,
                                 enable_pbe=None,
                                 default_priority_level=None,
                                 source_backup_location=None):
    
        consistency_policy = None
        if default_consistency_level is not None:
            consistency_policy = ConsistencyPolicy(default_consistency_level=default_consistency_level,
                                                   max_staleness_prefix=max_staleness_prefix,
                                                   max_interval_in_seconds=max_interval)
    
        if not locations:
            locations = []
            locations.append(Location(location_name=arm_location, failover_priority=0, is_zone_redundant=False))
    
        managed_service_identity = None
        SYSTEM_ID = '[system]'
        enable_system = False
        if assign_identity is not None:
            if assign_identity == [] or (len(assign_identity) == 1 and assign_identity[0] == '[system]'):
                enable_system = True
                managed_service_identity = ManagedServiceIdentity(type=ResourceIdentityType.system_assigned.value)
            else:
                user_identities = {}
                for x in assign_identity:
                    if x != SYSTEM_ID:
                        user_identities[x] = ManagedServiceIdentityUserAssignedIdentity()  # pylint: disable=line-too-long
                    else:
                        enable_system = True
                if enable_system:
                    managed_service_identity = ManagedServiceIdentity(
                        type=ResourceIdentityType.system_assigned_user_assigned.value,
                        user_assigned_identities=user_identities
                    )
                else:
                    managed_service_identity = ManagedServiceIdentity(
                        type=ResourceIdentityType.user_assigned.value,
                        user_assigned_identities=user_identities
                    )
    
        api_properties = {}
        if kind == DatabaseAccountKind.mongo_db.value:
            api_properties['ServerVersion'] = server_version
        elif server_version is not None:
            raise CLIError('server-version is a valid argument only when kind is MongoDB.')
    
        backup_policy = None
        if backup_policy_type is not None:
            if backup_policy_type.lower() == 'periodic':
                if backup_interval is None and backup_retention is None and backup_redundancy is None:
                    raise CLIError(
                        '--backup-interval, --backup-retention or --backup-redundancy ' +
                        'must be specified when Backup Policy Type is Periodic.')
                backup_policy = PeriodicModeBackupPolicy()
                if backup_interval is not None or backup_retention is not None or backup_redundancy is not None:
                    periodic_mode_properties = PeriodicModeProperties(
                        backup_interval_in_minutes=backup_interval,
                        backup_retention_interval_in_hours=backup_retention,
                        backup_storage_redundancy=backup_redundancy
                    )
                backup_policy.periodic_mode_properties = periodic_mode_properties
            elif backup_policy_type.lower() == 'continuous':
                backup_policy = ContinuousModeBackupPolicy()
                if continuous_tier is not None:
                    continuous_mode_properties = ContinuousModeProperties(
                        tier=continuous_tier
                    )
                else:
                    continuous_mode_properties = ContinuousModeProperties(
                        tier='Continuous30Days'
                    )
                backup_policy.continuous_mode_properties = continuous_mode_properties
            else:
                raise CLIError('backup-policy-type argument is invalid.')
        elif backup_interval is not None or backup_retention is not None or backup_redundancy is not None:
            backup_policy = PeriodicModeBackupPolicy()
            periodic_mode_properties = PeriodicModeProperties(
                backup_interval_in_minutes=backup_interval,
                backup_retention_interval_in_hours=backup_retention
            )
            backup_policy.periodic_mode_properties = periodic_mode_properties
    
        analytical_storage_configuration = None
        if analytical_storage_schema_type is not None:
            analytical_storage_configuration = AnalyticalStorageConfiguration()
            analytical_storage_configuration.schema_type = analytical_storage_schema_type
    
        create_mode = CreateMode.restore.value if is_restore_request else CreateMode.default.value
        params = None
        restore_parameters = None
        if create_mode == 'Restore':
            if restore_source is None or restore_timestamp is None:
                raise CLIError('restore-source and restore-timestamp should be provided for a restore request.')
    
            restore_parameters = RestoreParameters(
                restore_mode='PointInTime',
                restore_source=restore_source,
                restore_timestamp_in_utc=restore_timestamp
            )
    
            if databases_to_restore is not None:
                restore_parameters.databases_to_restore = databases_to_restore
    
            if gremlin_databases_to_restore is not None:
                restore_parameters.gremlin_databases_to_restore = gremlin_databases_to_restore
    
            if tables_to_restore is not None:
                restore_parameters.tables_to_restore = tables_to_restore
    
            if disable_ttl is not None:
                restore_parameters.restore_with_ttl_disabled = disable_ttl
    
            if source_backup_location is not None:
                restore_parameters.source_backup_location = source_backup_location
    
        params = DatabaseAccountCreateUpdateParameters(
            location=arm_location,
            locations=locations,
            tags=tags,
            kind=kind,
            consistency_policy=consistency_policy,
            ip_rules=ip_range_filter,
            is_virtual_network_filter_enabled=enable_virtual_network,
            enable_automatic_failover=enable_automatic_failover,
            capabilities=capabilities,
            virtual_network_rules=virtual_network_rules,
            enable_multiple_write_locations=enable_multiple_write_locations,
            disable_key_based_metadata_write_access=disable_key_based_metadata_write_access,
            disable_local_auth=disable_local_auth,
            key_vault_key_uri=key_uri,
            public_network_access=public_network_access,
            api_properties=api_properties,
            enable_analytical_storage=enable_analytical_storage,
            enable_free_tier=enable_free_tier,
            network_acl_bypass=network_acl_bypass,
            network_acl_bypass_resource_ids=network_acl_bypass_resource_ids,
            backup_policy=backup_policy,
            identity=managed_service_identity,
            default_identity=default_identity,
            analytical_storage_configuration=analytical_storage_configuration,
            create_mode=create_mode,
            restore_parameters=restore_parameters,
            enable_partition_merge=enable_partition_merge,
            enable_burst_capacity=enable_burst_capacity,
            enable_per_region_per_partition_autoscale=enable_prpp_autoscale,
            minimal_tls_version=minimal_tls_version,
            enable_pbe=enable_pbe,
            default_priority_level=default_priority_level
        )
    
        async_docdb_create = client.begin_create_or_update(resource_group_name, account_name, params)
        try:
            docdb_account = async_docdb_create.result()
        except HttpResponseError as ex:
            message = str(ex)
            if (is_restore_request
                    and ex.status_code == 403
                    and "does not exist" in message
                    and ("Database Account" in message or "Forbidden" in message)):
                logger.warning(
                    "Encountered known service issue (403 'does not exist') while restoring Cosmos DB account '%s' "
                    "in resource group '%s'. Using client.get() as a workaround. Raw error: %s",
                    account_name, resource_group_name, ex
                )
            else:
                raise ex
>       return docdb_account
               ^^^^^^^^^^^^^
E       UnboundLocalError: cannot access local variable 'docdb_account' where it is not associated with a value

src/azure-cli/azure/cli/command_modules/cosmosdb/custom.py:427: UnboundLocalError
azure/cli/command_modules/cosmosdb/tests/latest/test_cosmosdb_backuprestore_scenario.py:556
Failed test_locations_database_accounts self = <azure.cli.command_modules.cosmosdb.tests.latest.test_cosmosdb_commands.CosmosDBTests testMethod=test_locations_database_accounts>
resource_group = 'cli_test_cosmosdb_account000001'

    @ResourceGroupPreparer(name_prefix='cli_test_cosmosdb_account')
    def test_locations_database_accounts(self, resource_group):
    
        write_location = 'eastus'
        read_location = 'westus'
    
        self.kwargs.update({
            'acc': self.create_random_name(prefix='cli', length=40),
            'write_location': write_location,
            'read_location': read_location
        })
    
        account1 = self.cmd('az cosmosdb create -n {acc} -g {rg} --locations regionName={write_location} failoverPriority=0 --locations regionName={read_location} failoverPriority=1').get_output_in_json()
        assert len(account1['writeLocations']) == 1
        assert len(account1['readLocations']) == 2
        assert account1['writeLocations'][0]['failoverPriority'] == 0
        assert account1['writeLocations'][0]['locationName'] == "East US"
        assert account1['readLocations'][0]['locationName'] == "West US" or account1['readLocations'][1]['locationName'] == "West US"
        assert account1['readLocations'][0]['failoverPriority'] == 1 or account1['readLocations'][1]['failoverPriority'] == 1
    
        self.cmd('az cosmosdb failover-priority-change -n {acc} -g {rg} --failover-policies {read_location}=0 {write_location}=1')
        account2 = self.cmd('az cosmosdb show -n {acc} -g {rg}').get_output_in_json()
        assert len(account2['writeLocations']) == 1
        assert len(account2['readLocations']) == 2
    
        assert account2['writeLocations'][0]['failoverPriority'] == 0
>       assert account2['writeLocations'][0]['locationName'] == "West US"
E       AssertionError: assert 'East US' == 'West US'
E         
E         - West US
E         + East US

src/azure-cli/azure/cli/command_modules/cosmosdb/tests/latest/test_cosmosdb_commands.py:247: AssertionError
azure/cli/command_modules/cosmosdb/tests/latest/test_cosmosdb_commands.py:220
Failed test_locations_database_accounts_offline self = <azure.cli.command_modules.cosmosdb.tests.latest.test_cosmosdb_commands.CosmosDBTests testMethod=test_locations_database_accounts_offline>
resource_group = 'cli_test_cosmosdb_account000001'

    @ResourceGroupPreparer(name_prefix='cli_test_cosmosdb_account')
    def test_locations_database_accounts_offline(self, resource_group):
    
        write_location = 'eastus'
        read_location = 'westus'
    
        self.kwargs.update({
            'acc': self.create_random_name(prefix='cli', length=40),
            'write_location': write_location,
            'read_location': read_location
        })
    
        account_pre_offline = self.cmd('az cosmosdb create -n {acc} -g {rg} --locations regionName={write_location} failoverPriority=0 --locations regionName={read_location} failoverPriority=1').get_output_in_json()
    
        assert account_pre_offline['writeLocations'][0]['locationName'] == "East US"
    
        # Offline write region 'East US'
        self.cmd('az cosmosdb offline-region -n {acc} -g {rg} --region {write_location}')
        account_post_offline = self.cmd('az cosmosdb show -n {acc} -g {rg}').get_output_in_json()
    
        # Assert writeLocations is switched to 'West US' after offlining 'East US' region
        assert len(account_post_offline['writeLocations']) == 1
>       assert account_post_offline['writeLocations'][0]['locationName'] == 'West US'
E       AssertionError: assert 'East US' == 'West US'
E         
E         - West US
E         + East US

src/azure-cli/azure/cli/command_modules/cosmosdb/tests/latest/test_cosmosdb_commands.py:273: AssertionError
azure/cli/command_modules/cosmosdb/tests/latest/test_cosmosdb_commands.py:250
️✔️databoxedge
️✔️latest
️✔️3.12
️✔️3.13
️✔️dls
️✔️latest
️✔️3.12
️✔️3.13
️✔️dms
️✔️latest
️✔️3.12
️✔️3.13
️✔️eventgrid
️✔️latest
️✔️3.12
️✔️3.13
️✔️eventhubs
️✔️latest
️✔️3.12
️✔️3.13
️✔️feedback
️✔️latest
️✔️3.12
️✔️3.13
️✔️find
️✔️latest
️✔️3.12
️✔️3.13
️✔️hdinsight
️✔️latest
️✔️3.12
️✔️3.13
️✔️identity
️✔️latest
️✔️3.12
️✔️3.13
️✔️iot
️✔️latest
️✔️3.12
️✔️3.13
️✔️keyvault
️✔️latest
️✔️3.12
️✔️3.13
️✔️lab
️✔️latest
️✔️3.12
️✔️3.13
️✔️managedservices
️✔️latest
️✔️3.12
️✔️3.13
️✔️maps
️✔️latest
️✔️3.12
️✔️3.13
️✔️marketplaceordering
️✔️latest
️✔️3.12
️✔️3.13
️✔️monitor
️✔️latest
️✔️3.12
️✔️3.13
️✔️mysql
️✔️latest
️✔️3.12
️✔️3.13
️✔️netappfiles
️✔️latest
️✔️3.12
️✔️3.13
️✔️network
️✔️latest
️✔️3.12
️✔️3.13
️✔️policyinsights
️✔️latest
️✔️3.12
️✔️3.13
️✔️postgresql
️✔️latest
️✔️3.12
️✔️3.13
️✔️privatedns
️✔️latest
️✔️3.12
️✔️3.13
️✔️profile
️✔️latest
️✔️3.12
️✔️3.13
️✔️rdbms
️✔️latest
️✔️3.12
️✔️3.13
️✔️redis
️✔️latest
️✔️3.12
️✔️3.13
️✔️relay
️✔️latest
️✔️3.12
️✔️3.13
️✔️resource
️✔️latest
️✔️3.12
️✔️3.13
️✔️role
️✔️latest
️✔️3.12
️✔️3.13
️✔️search
️✔️latest
️✔️3.12
️✔️3.13
️✔️security
️✔️latest
️✔️3.12
️✔️3.13
️✔️servicebus
️✔️latest
️✔️3.12
️✔️3.13
️✔️serviceconnector
️✔️latest
️✔️3.12
️✔️3.13
️✔️servicefabric
️✔️latest
️✔️3.12
️✔️3.13
️✔️signalr
️✔️latest
️✔️3.12
️✔️3.13
️✔️sql
️✔️latest
️✔️3.12
️✔️3.13
️✔️sqlvm
️✔️latest
️✔️3.12
️✔️3.13
️✔️storage
️✔️latest
️✔️3.12
️✔️3.13
️✔️synapse
️✔️latest
️✔️3.12
️✔️3.13
️✔️telemetry
️✔️latest
️✔️3.12
️✔️3.13
️✔️util
️✔️latest
️✔️3.12
️✔️3.13
️✔️vm
️✔️latest
️✔️3.12
️✔️3.13

@azure-client-tools-bot-prd
Copy link

azure-client-tools-bot-prd bot commented Feb 5, 2026

️✔️AzureCLI-BreakingChangeTest
️✔️Non Breaking Changes

@yonzhan
Copy link
Collaborator

yonzhan commented Feb 5, 2026

Thank you for your contribution! We will review the pull request and get back to you soon.

@github-actions
Copy link

github-actions bot commented Feb 5, 2026

The git hooks are available for azure-cli and azure-cli-extensions repos. They could help you run required checks before creating the PR.

Please sync the latest code with latest dev branch (for azure-cli) or main branch (for azure-cli-extensions).
After that please run the following commands to enable git hooks:

pip install azdev --upgrade
azdev setup -c <your azure-cli repo path> -r <your azure-cli-extensions repo path>

@microsoft-github-policy-service microsoft-github-policy-service bot added the customer-reported Issues that are reported by GitHub users external to the Azure organization. label Feb 5, 2026
@microsoft-github-policy-service
Copy link
Contributor

Thank you for your contribution @jcassanji-southworks! We will review the pull request and get back to you soon.

Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR attempts to fix a known issue where az cosmosdb restore fails with a "Database Account does not exist" error during polling. The Azure backend service occasionally appends the location name to the account name during restore operations, causing a 403 Forbidden error. The fix implements error handling in _create_database_account to catch this specific error and fall back to using client.get() to verify account creation.

Changes:

  • Added exception handling in custom.py to catch and suppress specific 403 "does not exist" errors during restore operations
  • Added comprehensive unit tests in test_cosmosdb_backuprestore_scenario.py to verify the error handling logic

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 8 comments.

File Description
src/azure-cli/azure/cli/command_modules/cosmosdb/custom.py Added try-except block around async_docdb_create.result() to handle the service bug where location is appended to account name, with fallback to client.get()
src/azure-cli/azure/cli/command_modules/cosmosdb/tests/latest/test_cosmosdb_backuprestore_scenario.py Added unit test class CosmosDBRestoreUnitTests with three test cases covering the forbidden error handling, other errors, and normal create operations
Comments suppressed due to low confidence (4)

src/azure-cli/azure/cli/command_modules/cosmosdb/custom.py:422

  • The comment label "# Workaround" on this line is misleading because it applies to all account creation operations, not just the workaround for the specific restore bug. This line should always execute to retrieve the final account state, but the comment suggests it's part of the temporary workaround. Consider clarifying the comment or removing it since this is standard behavior.
    src/azure-cli/azure/cli/command_modules/cosmosdb/custom.py:422
  • Missing error handling: When the specific 403 error is caught and client.get() is called as a fallback, there's no verification that the restore operation actually succeeded. The code should check the provisioning_state or other indicators from the account returned by client.get() to ensure the operation completed successfully. Without this check, a failed restore could be silently treated as successful.
            raise ex
    docdb_account = client.get(resource_group_name, account_name)  # Workaround
    return docdb_account


src/azure-cli/azure/cli/command_modules/cosmosdb/custom.py:422

  • Missing error handling: If the workaround triggers but the account genuinely doesn't exist yet (e.g., the operation is still in progress), client.get() could raise a ResourceNotFoundError. This exception should be caught and handled appropriately, potentially with retry logic or a clearer error message explaining that the restore operation status is unclear.
            raise ex
    docdb_account = client.get(resource_group_name, account_name)  # Workaround
    return docdb_account


src/azure-cli/azure/cli/command_modules/cosmosdb/custom.py:422

  • Critical bug: Line 422 client.get() is now always executed, even when the operation succeeds normally. This changes the behavior for all database account creation operations, not just restore operations with the specific error.

The logic should be:

  1. If no exception occurs, use the result from async_docdb_create.result() and return it
  2. If the specific 403 error occurs during restore, fall back to client.get()

Current implementation causes an unnecessary additional API call for every successful account creation, and more importantly, the result from the successful poller.result() is discarded. The variable docdb_account is assigned on line 413 but then immediately overwritten on line 422 regardless of whether an exception occurred.

Fix: Move line 422 inside the exception handler (after line 419) so it only executes when the specific error is caught.

    try:
        docdb_account = async_docdb_create.result()
    except HttpResponseError as ex:
        if is_restore_request and ex.status_code == 403 and "does not exist" in str(ex):
            pass
        else:
            raise ex
    docdb_account = client.get(resource_group_name, account_name)  # Workaround
    return docdb_account



💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

jcassanji-southworks and others added 6 commits February 5, 2026 19:16
…test_cosmosdb_backuprestore_scenario.py

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
…test_cosmosdb_backuprestore_scenario.py

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
…o-name-causing-it-to-fail-fix' of https://github.com/jcassanji-southworks/azure-cli into 32608-az-cosmosdb-restore-wrongfully-appends-location-to-name-causing-it-to-fail-fix
@yonzhan yonzhan assigned calvinhzy and unassigned evelyn-ys Feb 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Auto-Assign Auto assign by bot CosmosDB az cosmosdb customer-reported Issues that are reported by GitHub users external to the Azure organization.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants