Skip to content

[Compute] az vm cp: Add command for file transfer via storage bridge#32757

Open
Ashutosh0x wants to merge 11 commits intoAzure:devfrom
Ashutosh0x:feature/vm-cp
Open

[Compute] az vm cp: Add command for file transfer via storage bridge#32757
Ashutosh0x wants to merge 11 commits intoAzure:devfrom
Ashutosh0x:feature/vm-cp

Conversation

@Ashutosh0x
Copy link

Description

This PR implements a new 'az vm cp' command to facilitate file transfers to and from Azure Virtual Machines.

Problem

Azure CLI currently lacks a native, easy-to-use command for copying files to/from VMs, especially in private networks where direct SSH/SCP or RDP access is not available. This has been a long-standing feature request (Issue #5275).

Solution

The proposed 'az vm cp' command implements a 'client-side bridge' solution:

  1. Upload/Download: Uses an Azure Storage blob container as an intermediary bridge.
  2. Security: Generates short-lived SAS tokens for temporary access.
  3. Execution: Leverages 'az vm run-command' to execute 'curl' (Linux) or PowerShell (Windows) scripts inside the VM to perform the transfer.
  4. Cleanup: Automatically removes the temporary blobs after the transfer is complete.

Examples

  • Upload: 'az vm cp --source ./local_file.txt --destination my-vm:/tmp/remote_file.txt'
  • Download: 'az vm cp --source my-rg:my-vm:/var/log/app.log --destination ./app.log'

This approach works through the Azure management plane, making it highly compatible with VMs behind firewalls or in private virtual networks.

Fixes #5275

Implements a client-side bridge solution using Azure Storage to facilitate file transfers to and from Azure VMs, particularly useful for private network environments. Leverages existing 'run-command' capabilities.
Copilot AI review requested due to automatic review settings February 6, 2026 11:11
@azure-client-tools-bot-prd
Copy link

azure-client-tools-bot-prd bot commented Feb 6, 2026

❌AzureCLI-FullTest
️✔️acr
️✔️latest
️✔️3.12
️✔️3.13
️✔️acs
️✔️latest
️✔️3.12
️✔️3.13
️✔️advisor
️✔️latest
️✔️3.12
️✔️3.13
️✔️ams
️✔️latest
️✔️3.12
️✔️3.13
️✔️apim
️✔️latest
️✔️3.12
️✔️3.13
️✔️appconfig
️✔️latest
️✔️3.12
️✔️3.13
️✔️appservice
️✔️latest
️✔️3.12
️✔️3.13
️✔️aro
️✔️latest
️✔️3.12
️✔️3.13
️✔️backup
️✔️latest
️✔️3.12
️✔️3.13
️✔️batch
️✔️latest
️✔️3.12
️✔️3.13
️✔️batchai
️✔️latest
️✔️3.12
️✔️3.13
️✔️billing
️✔️latest
️✔️3.12
️✔️3.13
️✔️botservice
️✔️latest
️✔️3.12
️✔️3.13
️✔️cdn
️✔️latest
️✔️3.12
️✔️3.13
️✔️cloud
️✔️latest
️✔️3.12
️✔️3.13
️✔️cognitiveservices
️✔️latest
️✔️3.12
️✔️3.13
️✔️compute_recommender
️✔️latest
️✔️3.12
️✔️3.13
️✔️computefleet
️✔️latest
️✔️3.12
️✔️3.13
️✔️config
️✔️latest
️✔️3.12
️✔️3.13
️✔️configure
️✔️latest
️✔️3.12
️✔️3.13
️✔️consumption
️✔️latest
️✔️3.12
️✔️3.13
️✔️container
️✔️latest
️✔️3.12
️✔️3.13
️✔️containerapp
️✔️latest
️✔️3.12
️✔️3.13
️✔️core
️✔️latest
️✔️3.12
️✔️3.13
️✔️cosmosdb
️✔️latest
️✔️3.12
️✔️3.13
️✔️databoxedge
️✔️latest
️✔️3.12
️✔️3.13
️✔️dls
️✔️latest
️✔️3.12
️✔️3.13
️✔️dms
️✔️latest
️✔️3.12
️✔️3.13
️✔️eventgrid
️✔️latest
️✔️3.12
️✔️3.13
️✔️eventhubs
️✔️latest
️✔️3.12
️✔️3.13
️✔️feedback
️✔️latest
️✔️3.12
️✔️3.13
️✔️find
️✔️latest
️✔️3.12
️✔️3.13
️✔️hdinsight
️✔️latest
️✔️3.12
️✔️3.13
️✔️identity
️✔️latest
️✔️3.12
️✔️3.13
️✔️iot
️✔️latest
️✔️3.12
️✔️3.13
️✔️keyvault
️✔️latest
️✔️3.12
️✔️3.13
️✔️lab
️✔️latest
️✔️3.12
️✔️3.13
️✔️managedservices
️✔️latest
️✔️3.12
️✔️3.13
️✔️maps
️✔️latest
️✔️3.12
️✔️3.13
️✔️marketplaceordering
️✔️latest
️✔️3.12
️✔️3.13
️✔️monitor
️✔️latest
️✔️3.12
️✔️3.13
️✔️mysql
️✔️latest
️✔️3.12
️✔️3.13
️✔️netappfiles
️✔️latest
️✔️3.12
️✔️3.13
️✔️network
️✔️latest
️✔️3.12
️✔️3.13
️✔️policyinsights
️✔️latest
️✔️3.12
️✔️3.13
️✔️postgresql
️✔️latest
️✔️3.12
️✔️3.13
️✔️privatedns
️✔️latest
️✔️3.12
️✔️3.13
️✔️profile
️✔️latest
️✔️3.12
️✔️3.13
️✔️rdbms
️✔️latest
️✔️3.12
️✔️3.13
️✔️redis
️✔️latest
️✔️3.12
️✔️3.13
️✔️relay
️✔️latest
️✔️3.12
️✔️3.13
️✔️resource
️✔️latest
️✔️3.12
️✔️3.13
️✔️role
️✔️latest
️✔️3.12
️✔️3.13
️✔️search
️✔️latest
️✔️3.12
️✔️3.13
️✔️security
️✔️latest
️✔️3.12
️✔️3.13
️✔️servicebus
️✔️latest
️✔️3.12
️✔️3.13
️✔️serviceconnector
️✔️latest
️✔️3.12
️✔️3.13
️✔️servicefabric
️✔️latest
️✔️3.12
️✔️3.13
️✔️signalr
️✔️latest
️✔️3.12
️✔️3.13
️✔️sql
️✔️latest
️✔️3.12
️✔️3.13
️✔️sqlvm
️✔️latest
️✔️3.12
️✔️3.13
️✔️storage
️✔️latest
️✔️3.12
️✔️3.13
️✔️synapse
️✔️latest
️✔️3.12
️✔️3.13
️✔️telemetry
️✔️latest
️✔️3.12
️✔️3.13
️✔️util
️✔️latest
️✔️3.12
️✔️3.13
❌vm
❌latest
❌3.12
Type Test Case Error Message Line
Failed test_parse_vm_file_path self = <latest.test_vm_cp_unit.TestVmCp testMethod=test_parse_vm_file_path>

    def test_parse_vm_file_path(self):
        # Local paths (non-VM)
        self.assertIsNone(_parse_vm_file_path("/path/to/file"))
        self.assertIsNone(_parse_vm_file_path("C:\path\to\file"))
        self.assertIsNone(_parse_vm_file_path("D:/path/to/file"))
        self.assertIsNone(_parse_vm_file_path("justfile"))
    
        # VM path: vm:path
        self.assertEqual(_parse_vm_file_path("myvm:/tmp/file"), (None, "myvm", "/tmp/file"))
    
        # VM path: rg:vm:path
        self.assertEqual(_parse_vm_file_path("myrg:myvm:/tmp/file"), ("myrg", "myvm", "/tmp/file"))
    
        # VM path with colons in the path component
>       self.assertEqual(_parse_vm_file_path("vm:C:\remote\path"), (None, "vm", "C:\remote\path"))
E       AssertionError: Tuples differ: ('vm', 'C', '\remote\path') != (None, 'vm', 'C:\remote\path')
E       
E       First differing element 0:
E       'vm'
E       None
E       
E       - ('vm', 'C', '\remote\path')
E       ?          ^^^^
E       
E       + (None, 'vm', 'C:\remote\path')
E       ?  ++++++        ^

src/azure-cli/azure/cli/command_modules/vm/tests/latest/test_vm_cp_unit.py:26: AssertionError
azure/cli/command_modules/vm/tests/latest/test_vm_cp_unit.py:11
Failed test_vm_cp_upload_basic /opt/hostedtoolcache/Python/3.12.12/x64/lib/python3.12/unittest/mock.py:1393: in patched
    with self.decoration_helper(patched,
/opt/hostedtoolcache/Python/3.12.12/x64/lib/python3.12/contextlib.py:137: in enter
    return next(self.gen)
           ^^^^^^^^^^^^^^
/opt/hostedtoolcache/Python/3.12.12/x64/lib/python3.12/unittest/mock.py:1375: in decoration_helper
    arg = exit_stack.enter_context(patching)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
/opt/hostedtoolcache/Python/3.12.12/x64/lib/python3.12/contextlib.py:526: in enter_context
    result = enter(cm)
             ^^^^^^^^^^
/opt/hostedtoolcache/Python/3.12.12/x64/lib/python3.12/unittest/mock.py:1467: in enter
    original, local = self.get_original()
                      ^^^^^^^^^^^^^^^^^^^
                                       _ 

self = <unittest.mock._patch object at 0x7f6a645aacc0>

    def get_original(self):
        target = self.getter()
        name = self.attribute
    
        original = DEFAULT
        local = False
    
        try:
            original = target.dict[name]
        except (AttributeError, KeyError):
            original = getattr(target, name, DEFAULT)
        else:
            local = True
    
        if name in _builtins and isinstance(target, ModuleType):
            self.create = True
    
        if not self.create and original is DEFAULT:
>           raise AttributeError(
                "%s does not have the attribute %r" % (target, name)
            )
E           AttributeError: <module 'azure.cli.command_modules.vm.custom' from '/mnt/vss/_work/1/s/src/azure-cli/azure/cli/command_modules/vm/custom.py'> does not have the attribute 'cf_blob_service'

/opt/hostedtoolcache/Python/3.12.12/x64/lib/python3.12/unittest/mock.py:1437: AttributeError
azure/cli/command_modules/vm/tests/latest/test_vm_cp_unit.py:28
❌3.13
Type Test Case Error Message Line
Failed test_parse_vm_file_path self = <latest.test_vm_cp_unit.TestVmCp testMethod=test_parse_vm_file_path>

    def test_parse_vm_file_path(self):
        # Local paths (non-VM)
        self.assertIsNone(_parse_vm_file_path("/path/to/file"))
        self.assertIsNone(_parse_vm_file_path("C:\path\to\file"))
        self.assertIsNone(_parse_vm_file_path("D:/path/to/file"))
        self.assertIsNone(_parse_vm_file_path("justfile"))
    
        # VM path: vm:path
        self.assertEqual(_parse_vm_file_path("myvm:/tmp/file"), (None, "myvm", "/tmp/file"))
    
        # VM path: rg:vm:path
        self.assertEqual(_parse_vm_file_path("myrg:myvm:/tmp/file"), ("myrg", "myvm", "/tmp/file"))
    
        # VM path with colons in the path component
>       self.assertEqual(_parse_vm_file_path("vm:C:\remote\path"), (None, "vm", "C:\remote\path"))
E       AssertionError: Tuples differ: ('vm', 'C', '\remote\path') != (None, 'vm', 'C:\remote\path')
E       
E       First differing element 0:
E       'vm'
E       None
E       
E       - ('vm', 'C', '\remote\path')
E       ?          ^^^^
E       
E       + (None, 'vm', 'C:\remote\path')
E       ?  ++++++        ^

src/azure-cli/azure/cli/command_modules/vm/tests/latest/test_vm_cp_unit.py:26: AssertionError
azure/cli/command_modules/vm/tests/latest/test_vm_cp_unit.py:11
Failed test_vm_cp_upload_basic /opt/hostedtoolcache/Python/3.13.11/x64/lib/python3.13/unittest/mock.py:1423: in patched
    with self.decoration_helper(patched,
/opt/hostedtoolcache/Python/3.13.11/x64/lib/python3.13/contextlib.py:141: in enter
    return next(self.gen)
           ^^^^^^^^^^^^^^
/opt/hostedtoolcache/Python/3.13.11/x64/lib/python3.13/unittest/mock.py:1405: in decoration_helper
    arg = exit_stack.enter_context(patching)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
/opt/hostedtoolcache/Python/3.13.11/x64/lib/python3.13/contextlib.py:530: in enter_context
    result = enter(cm)
             ^^^^^^^^^^
/opt/hostedtoolcache/Python/3.13.11/x64/lib/python3.13/unittest/mock.py:1497: in enter
    original, local = self.get_original()
                      ^^^^^^^^^^^^^^^^^^^
                                       _ 

self = <unittest.mock._patch object at 0x7f5faea0b490>

    def get_original(self):
        target = self.getter()
        name = self.attribute
    
        original = DEFAULT
        local = False
    
        try:
            original = target.dict[name]
        except (AttributeError, KeyError):
            original = getattr(target, name, DEFAULT)
        else:
            local = True
    
        if name in _builtins and isinstance(target, ModuleType):
            self.create = True
    
        if not self.create and original is DEFAULT:
>           raise AttributeError(
                "%s does not have the attribute %r" % (target, name)
            )
E           AttributeError: <module 'azure.cli.command_modules.vm.custom' from '/mnt/vss/_work/1/s/src/azure-cli/azure/cli/command_modules/vm/custom.py'> does not have the attribute 'cf_blob_service'

/opt/hostedtoolcache/Python/3.13.11/x64/lib/python3.13/unittest/mock.py:1467: AttributeError
azure/cli/command_modules/vm/tests/latest/test_vm_cp_unit.py:28

@azure-client-tools-bot-prd
Copy link

azure-client-tools-bot-prd bot commented Feb 6, 2026

⚠️AzureCLI-BreakingChangeTest
⚠️vm
rule cmd_name rule_message suggest_message
⚠️ 1001 - CmdAdd vm cp cmd vm cp added

@yonzhan
Copy link
Collaborator

yonzhan commented Feb 6, 2026

Thank you for your contribution! We will review the pull request and get back to you soon.

@github-actions
Copy link

github-actions bot commented Feb 6, 2026

The git hooks are available for azure-cli and azure-cli-extensions repos. They could help you run required checks before creating the PR.

Please sync the latest code with latest dev branch (for azure-cli) or main branch (for azure-cli-extensions).
After that please run the following commands to enable git hooks:

pip install azdev --upgrade
azdev setup -c <your azure-cli repo path> -r <your azure-cli-extensions repo path>

@microsoft-github-policy-service microsoft-github-policy-service bot added customer-reported Issues that are reported by GitHub users external to the Azure organization. Auto-Assign Auto assign by bot labels Feb 6, 2026
@microsoft-github-policy-service microsoft-github-policy-service bot added the Compute az vm/vmss/image/disk/snapshot label Feb 6, 2026
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR implements a new az vm cp command to facilitate file transfers to and from Azure Virtual Machines using an Azure Storage blob container as an intermediary bridge. The solution addresses issue #5275 by providing a native Azure CLI command for file transfers that works through the Azure management plane, making it compatible with VMs in private networks.

Changes:

  • Added vm_cp function implementing client-side bridge file transfer using storage blobs and az vm run-command
  • Registered new cp command in the VM command group
  • Added parameter definitions for source, destination, storage_account, and container_name
  • Added help documentation with usage examples

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 22 comments.

File Description
src/azure-cli/azure/cli/command_modules/vm/custom.py Core implementation of vm_cp function with path parsing, storage setup, and VM run-command execution
src/azure-cli/azure/cli/command_modules/vm/commands.py Command registration for 'az vm cp'
src/azure-cli/azure/cli/command_modules/vm/_params.py Parameter definitions for the new command
src/azure-cli/azure/cli/command_modules/vm/_help.py Help documentation and usage examples

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +6618 to +6621
blob_service_client = cf_blob_service(cmd.cli_ctx, {'account_name': sa_name, 'account_key': account_key})
container_client = blob_service_client.get_container_client(container_name)
try:
container_client.create_container()
Copy link

Copilot AI Feb 6, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The bare except clause catches all exceptions silently, which can hide important errors like authentication failures, network issues, or permission problems. This makes debugging difficult for users. Change this to catch specific exceptions like ResourceExistsError and let other exceptions propagate with helpful error messages.

Copilot uses AI. Check for mistakes.
if is_linux:
script = "curl -L -o '{}' '{}'".format(vm_path, blob_url)
command_id = 'RunShellScript'
else:
Copy link

Copilot AI Feb 6, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Security vulnerability: The PowerShell script includes user-provided paths without proper escaping. The vm_path variable is directly interpolated into the Invoke-WebRequest command. If vm_path contains single quotes or other PowerShell special characters, it could enable command injection. Use proper PowerShell escaping before constructing the script.

Suggested change
else:
escaped_blob_url = blob_url.replace("'", "''")
escaped_vm_path = vm_path.replace("'", "''")
script = "Invoke-WebRequest -Uri '{}' -OutFile '{}'".format(escaped_blob_url, escaped_vm_path)

Copilot uses AI. Check for mistakes.
else:
# DOWNLOAD: VM -> Local
rg, vm_name, vm_path = source_vm
if not rg:
Copy link

Copilot AI Feb 6, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Inconsistent whitespace: There's a leading space before the comment on line 6674 that breaks the indentation pattern. This should be removed to maintain consistent code formatting.

Suggested change
if not rg:
# find VM RG

Copilot uses AI. Check for mistakes.
if is_linux:
script = "curl -X PUT -T '{}' -H 'x-ms-blob-type: BlockBlob' '{}'".format(vm_path, blob_url)
command_id = 'RunShellScript'
else:
Copy link

Copilot AI Feb 6, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Security vulnerability: The PowerShell script includes user-provided vm_path without proper escaping. If vm_path contains single quotes or other PowerShell special characters, it could enable command injection. Use proper PowerShell escaping before constructing the script.

Suggested change
else:
# Escape single quotes in vm_path for safe use inside a single-quoted PowerShell string
escaped_vm_path = vm_path.replace("'", "''") if vm_path is not None else vm_path
script = "$body = Get-Content -Path '{}' -Encoding Byte; Invoke-RestMethod -Uri '{}' -Method Put -Headers @{{'x-ms-blob-type'='BlockBlob'}} -Body $body".format(escaped_vm_path, blob_url)

Copilot uses AI. Check for mistakes.
Comment on lines 6657 to 6668

logger.info("Executing download script in VM...")
from .aaz.latest.vm.run_command import Invoke
Invoke(cli_ctx=cmd.cli_ctx)(command_args={
'resource_group': rg,
'vm_name': vm_name,
'command_id': command_id,
'script': [script]
})

# Cleanup
logger.info("Cleaning up bridge storage...")
Copy link

Copilot AI Feb 6, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Operational reliability issue: If an error occurs during VM script execution (lines 6659-6664 or 6703-6708), the cleanup code at lines 6668 or 6715 will not execute, leaving orphaned blobs in the storage account. This will accumulate storage costs over time. Consider wrapping the operation in a try-finally block to ensure cleanup always occurs, or implement a timeout-based cleanup mechanism.

Copilot uses AI. Check for mistakes.
short-summary: Copy files to and from a virtual machine.
long-summary: >
This command uses an Azure Storage blob container as an intermediary bridge to transfer files.
It requires 'az vm run-command' capability on the target VM.
Copy link

Copilot AI Feb 6, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Documentation issue: The long-summary should explain the requirements more clearly, specifically that curl must be available on Linux VMs and that the VM must be running and accessible via run-command. It should also mention that the command requires appropriate permissions on both the VM and the storage account.

Suggested change
It requires 'az vm run-command' capability on the target VM.
It requires that the target VM is running and accessible via 'az vm run-command'. On Linux VMs, 'curl'
must be installed and available on the PATH. The identity used to run this command must have sufficient
permissions both on the VM (to execute run-command) and on the storage account/container (to read and write blobs).

Copilot uses AI. Check for mistakes.
Comment on lines 6557 to 6717


def vm_cp(cmd, source, destination, storage_account=None, container_name='azvmcp'):
source_vm = _parse_vm_file_path(source)
dest_vm = _parse_vm_file_path(destination)

if source_vm and dest_vm:
raise ValidationError("Both source and destination cannot be VM paths.")
if not source_vm and not dest_vm:
raise ValidationError("Either source or destination must be a VM path (format: [rg:]vm:path).")

# 1. Prepare Storage Account
if not storage_account:
# Try to find a storage account in the VM's resource group
rg = (source_vm[0] if source_vm else dest_vm[0])
vm_name = (source_vm[1] if source_vm else dest_vm[1])

if not rg:
# Get RG of the VM
client = _compute_client_factory(cmd.cli_ctx)
vms = client.virtual_machines.list_all()
vm = next((v for v in vms if v.name.lower() == vm_name.lower()), None)
if not vm:
raise ResourceNotFoundError("VM '{}' not found.".format(vm_name))
# parse RG from ID
rg = vm.id.split('/')[4]

from azure.cli.command_modules.storage._client_factory import cf_sa
sa_client = cf_sa(cmd.cli_ctx, None)
accounts = list(sa_client.list())
# Filter by RG if possible
rg_accounts = [a for a in accounts if a.id.split('/')[4].lower() == rg.lower()]
if rg_accounts:
storage_account = rg_accounts[0].name
elif accounts:
storage_account = accounts[0].name
else:
raise RequiredArgumentMissingError("No storage account found in the subscription. Please provide one with --storage-account.")

# Get account key
from azure.cli.command_modules.storage._client_factory import cf_sa_for_keys
sa_keys_client = cf_sa_for_keys(cmd.cli_ctx, None)
# Check if storage_account is name or ID
if '/' in storage_account:
sa_rg = storage_account.split('/')[4]
sa_name = storage_account.split('/')[-1]
else:
# Search for it
sa_client = cf_sa(cmd.cli_ctx, None)
accounts = list(sa_client.list())
account = next((a for a in accounts if a.name.lower() == storage_account.lower()), None)
if not account:
raise ResourceNotFoundError("Storage account '{}' not found.".format(storage_account))
sa_rg = account.id.split('/')[4]
sa_name = account.name

keys = sa_keys_client.list_keys(sa_rg, sa_name).keys
account_key = keys[0].value

# Ensure container exists
from azure.cli.command_modules.storage._client_factory import cf_blob_service
blob_service_client = cf_blob_service(cmd.cli_ctx, {'account_name': sa_name, 'account_key': account_key})
container_client = blob_service_client.get_container_client(container_name)
try:
container_client.create_container()
except Exception: # Already exists or other error # pylint: disable=broad-except
pass

blob_name = str(uuid.uuid4())

blob_client = container_client.get_blob_client(blob_name)

if dest_vm:
# UPLOAD: Local -> VM
rg, vm_name, vm_path = dest_vm
if not rg:
# find VM RG
client = _compute_client_factory(cmd.cli_ctx)
vms = client.virtual_machines.list_all()
vm = next((v for v in vms if v.name.lower() == vm_name.lower()), None)
rg = vm.id.split('/')[4]

logger.info("Uploading local file to bridge storage...")
upload_blob(cmd, blob_client, file_path=source)

# Get SAS for VM to download
sas_token = create_short_lived_blob_sas_v2(cmd, sa_name, container_name, blob_name, account_key=account_key)
blob_url = "https://{}.blob.{}/{}/{}?{}".format(sa_name, cmd.cli_ctx.cloud.suffixes.storage_endpoint, container_name, blob_name, sas_token)

# VM run-command to download
# Check OS type
vm_obj = _compute_client_factory(cmd.cli_ctx).virtual_machines.get(rg, vm_name)
is_linux = vm_obj.storage_profile.os_disk.os_type.lower() == 'linux'

if is_linux:
script = "curl -L -o '{}' '{}'".format(vm_path, blob_url)
command_id = 'RunShellScript'
else:
script = "Invoke-WebRequest -Uri '{}' -OutFile '{}'".format(blob_url, vm_path)
command_id = 'RunPowerShellScript'

logger.info("Executing download script in VM...")
from .aaz.latest.vm.run_command import Invoke
Invoke(cli_ctx=cmd.cli_ctx)(command_args={
'resource_group': rg,
'vm_name': vm_name,
'command_id': command_id,
'script': [script]
})

# Cleanup
logger.info("Cleaning up bridge storage...")
blob_client.delete_blob()

else:
# DOWNLOAD: VM -> Local
rg, vm_name, vm_path = source_vm
if not rg:
# find VM RG
client = _compute_client_factory(cmd.cli_ctx)
vms = client.virtual_machines.list_all()
vm = next((v for v in vms if v.name.lower() == vm_name.lower()), None)
rg = vm.id.split('/')[4]

# Get SAS with WRITE permission
t_sas = cmd.get_models('_shared_access_signature#BlobSharedAccessSignature',
resource_type=ResourceType.DATA_STORAGE_BLOB)
t_blob_permissions = cmd.get_models('_models#BlobSasPermissions', resource_type=ResourceType.DATA_STORAGE_BLOB)
expiry = (datetime.utcnow() + timedelta(days=1)).strftime('%Y-%m-%dT%H:%M:%SZ')
sas = t_sas(sa_name, account_key=account_key)
sas_token = sas.generate_blob(container_name, blob_name,
permission=t_blob_permissions(write=True),
expiry=expiry, protocol='https')
blob_url = "https://{}.blob.{}/{}/{}?{}".format(sa_name, cmd.cli_ctx.cloud.suffixes.storage_endpoint, container_name, blob_name, sas_token)

vm_obj = _compute_client_factory(cmd.cli_ctx).virtual_machines.get(rg, vm_name)
is_linux = vm_obj.storage_profile.os_disk.os_type.lower() == 'linux'

if is_linux:
script = "curl -X PUT -T '{}' -H 'x-ms-blob-type: BlockBlob' '{}'".format(vm_path, blob_url)
command_id = 'RunShellScript'
else:
script = "$body = Get-Content -Path '{}' -Encoding Byte; Invoke-RestMethod -Uri '{}' -Method Put -Headers @{{'x-ms-blob-type'='BlockBlob'}} -Body $body".format(vm_path, blob_url)
command_id = 'RunPowerShellScript'

logger.info("Executing upload script in VM...")
from .aaz.latest.vm.run_command import Invoke
Invoke(cli_ctx=cmd.cli_ctx)(command_args={
'resource_group': rg,
'vm_name': vm_name,
'command_id': command_id,
'script': [script]
})

logger.info("Downloading from bridge storage to local...")
download_blob(blob_client, file_path=destination)

# Cleanup
logger.info("Cleaning up bridge storage...")
blob_client.delete_blob()

Copy link

Copilot AI Feb 6, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Missing test coverage: There are no tests for the new vm_cp command. The vm module has extensive test files (test_vm_commands.py has 13,881 lines), indicating that comprehensive testing is a convention in this codebase. The vm_cp command involves complex logic with multiple error paths and should have unit tests covering various scenarios including: path parsing edge cases, VM/storage account discovery, upload/download flows, error handling, and cleanup behavior.

Copilot uses AI. Check for mistakes.
Comment on lines 6574 to 6678
if not rg:
# Get RG of the VM
client = _compute_client_factory(cmd.cli_ctx)
vms = client.virtual_machines.list_all()
vm = next((v for v in vms if v.name.lower() == vm_name.lower()), None)
if not vm:
raise ResourceNotFoundError("VM '{}' not found.".format(vm_name))
# parse RG from ID
rg = vm.id.split('/')[4]

from azure.cli.command_modules.storage._client_factory import cf_sa
sa_client = cf_sa(cmd.cli_ctx, None)
accounts = list(sa_client.list())
# Filter by RG if possible
rg_accounts = [a for a in accounts if a.id.split('/')[4].lower() == rg.lower()]
if rg_accounts:
storage_account = rg_accounts[0].name
elif accounts:
storage_account = accounts[0].name
else:
raise RequiredArgumentMissingError("No storage account found in the subscription. Please provide one with --storage-account.")

# Get account key
from azure.cli.command_modules.storage._client_factory import cf_sa_for_keys
sa_keys_client = cf_sa_for_keys(cmd.cli_ctx, None)
# Check if storage_account is name or ID
if '/' in storage_account:
sa_rg = storage_account.split('/')[4]
sa_name = storage_account.split('/')[-1]
else:
# Search for it
sa_client = cf_sa(cmd.cli_ctx, None)
accounts = list(sa_client.list())
account = next((a for a in accounts if a.name.lower() == storage_account.lower()), None)
if not account:
raise ResourceNotFoundError("Storage account '{}' not found.".format(storage_account))
sa_rg = account.id.split('/')[4]
sa_name = account.name

keys = sa_keys_client.list_keys(sa_rg, sa_name).keys
account_key = keys[0].value

# Ensure container exists
from azure.cli.command_modules.storage._client_factory import cf_blob_service
blob_service_client = cf_blob_service(cmd.cli_ctx, {'account_name': sa_name, 'account_key': account_key})
container_client = blob_service_client.get_container_client(container_name)
try:
container_client.create_container()
except Exception: # Already exists or other error # pylint: disable=broad-except
pass

blob_name = str(uuid.uuid4())

blob_client = container_client.get_blob_client(blob_name)

if dest_vm:
# UPLOAD: Local -> VM
rg, vm_name, vm_path = dest_vm
if not rg:
# find VM RG
client = _compute_client_factory(cmd.cli_ctx)
vms = client.virtual_machines.list_all()
vm = next((v for v in vms if v.name.lower() == vm_name.lower()), None)
rg = vm.id.split('/')[4]

logger.info("Uploading local file to bridge storage...")
upload_blob(cmd, blob_client, file_path=source)

# Get SAS for VM to download
sas_token = create_short_lived_blob_sas_v2(cmd, sa_name, container_name, blob_name, account_key=account_key)
blob_url = "https://{}.blob.{}/{}/{}?{}".format(sa_name, cmd.cli_ctx.cloud.suffixes.storage_endpoint, container_name, blob_name, sas_token)

# VM run-command to download
# Check OS type
vm_obj = _compute_client_factory(cmd.cli_ctx).virtual_machines.get(rg, vm_name)
is_linux = vm_obj.storage_profile.os_disk.os_type.lower() == 'linux'

if is_linux:
script = "curl -L -o '{}' '{}'".format(vm_path, blob_url)
command_id = 'RunShellScript'
else:
script = "Invoke-WebRequest -Uri '{}' -OutFile '{}'".format(blob_url, vm_path)
command_id = 'RunPowerShellScript'

logger.info("Executing download script in VM...")
from .aaz.latest.vm.run_command import Invoke
Invoke(cli_ctx=cmd.cli_ctx)(command_args={
'resource_group': rg,
'vm_name': vm_name,
'command_id': command_id,
'script': [script]
})

# Cleanup
logger.info("Cleaning up bridge storage...")
blob_client.delete_blob()

else:
# DOWNLOAD: VM -> Local
rg, vm_name, vm_path = source_vm
if not rg:
# find VM RG
client = _compute_client_factory(cmd.cli_ctx)
vms = client.virtual_machines.list_all()
vm = next((v for v in vms if v.name.lower() == vm_name.lower()), None)
Copy link

Copilot AI Feb 6, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code duplication: The VM lookup logic (lines 6574-6580, 6633-6636, 6675-6678) is repeated three times with nearly identical code. This violates the DRY principle and makes maintenance harder. Extract this logic into a helper function that takes the VM name and optional resource group, and returns the VM object and resource group.

Copilot uses AI. Check for mistakes.
upload_blob(cmd, blob_client, file_path=source)

# Get SAS for VM to download
sas_token = create_short_lived_blob_sas_v2(cmd, sa_name, container_name, blob_name, account_key=account_key)
Copy link

Copilot AI Feb 6, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Potential bug with cloud endpoints: The blob URL construction assumes the storage_endpoint suffix will not have a leading slash, but it's concatenated directly. If cmd.cli_ctx.cloud.suffixes.storage_endpoint includes unexpected formatting or is None, this will fail. Add validation or use a more robust URL construction method to handle different cloud environments correctly.

Suggested change
sas_token = create_short_lived_blob_sas_v2(cmd, sa_name, container_name, blob_name, account_key=account_key)
storage_suffix = getattr(cmd.cli_ctx.cloud.suffixes, 'storage_endpoint', None)
if not storage_suffix:
raise CLIError("The storage endpoint suffix for the current cloud is not configured.")
# Normalize to avoid leading dots or slashes that would break the URL host
storage_suffix = str(storage_suffix).lstrip("./").strip()
blob_url = "https://{}.blob.{}/{}/{}?{}".format(sa_name, storage_suffix, container_name, blob_name, sas_token)

Copilot uses AI. Check for mistakes.
resource_type=ResourceType.DATA_STORAGE_BLOB)
t_blob_permissions = cmd.get_models('_models#BlobSasPermissions', resource_type=ResourceType.DATA_STORAGE_BLOB)
expiry = (datetime.utcnow() + timedelta(days=1)).strftime('%Y-%m-%dT%H:%M:%SZ')
sas = t_sas(sa_name, account_key=account_key)
Copy link

Copilot AI Feb 6, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Security concern: The SAS token expiry time is set to 1 day (lines 6686, and via create_short_lived_blob_sas_v2 at line 6642), which is excessive for a file transfer operation that typically completes in minutes. If the token is intercepted, it provides 24 hours of access to the blob. Reduce the expiry to a shorter duration (e.g., 1-2 hours) to minimize the security risk window while still providing enough time for the transfer to complete.

Suggested change
sas = t_sas(sa_name, account_key=account_key)
expiry = (datetime.utcnow() + timedelta(hours=2)).strftime('%Y-%m-%dT%H:%M:%SZ')

Copilot uses AI. Check for mistakes.
@Ashutosh0x
Copy link
Author

Hi @zhoxing-ms @evelyn-ys @yanzhudd @yonzhan, I've just pushed an update addressing the feedback regarding security and reliability.

Highlights of the latest changes:

  • Security: Added proper shell/PowerShell escaping to prevent command injection.
  • Path Parsing: Fixed a bug where Windows local absolute paths were being misidentified as VM paths.
  • Reliability: Implemented a try-finally block for bridge storage cleanup and shortened SAS token durations.
  • Compatibility: Added a PowerShell version check to handle binary downloads correctly across different VM images.

Looking forward to your thoughts!

@yonzhan
Copy link
Collaborator

yonzhan commented Feb 6, 2026

Please fix CI issues

@Ashutosh0x
Copy link
Author

Hi @yonzhan, I've addressed the CI failure (duplicate string formatting) and incorporated the Copilot feedback:

  • Extracted VM lookup logic into a helper function (DRY).
  • Updated the command help documentation with clearer requirements (curl and run-command accessibility).
  • Fixed a path parsing bug for Windows local paths.
  • Verified all linters and unit tests locally.

The PR should be ready for review now. Thank you!

@yanzhudd
Copy link
Contributor

yanzhudd commented Feb 8, 2026

/azp run

@azure-pipelines
Copy link

Azure Pipelines successfully started running 3 pipeline(s).

@yanzhudd yanzhudd changed the title feat(vm): add 'az vm cp' command for file transfer via storage bridge [Compute] az vm cp: Add command for file transfer via storage bridge Feb 8, 2026
@yanzhudd
Copy link
Contributor

yanzhudd commented Feb 9, 2026

please fix the CI issues

@Ashutosh0x
Copy link
Author

Refinement and Fixes for az vm cp

I have completed a comprehensive set of refinements to address the review feedback and CI failures. Key changes include:

  • Security & Reliability:
    • Implemented robust shell/PowerShell escaping to prevent command injection.
    • Shortened SAS token expiry to 2 hours.
    • Added a try...finally block to ensure bridge storage cleanup occurs even if transfers fail.
    • Improved PowerShell compatibility by handling binary downloads across different versions (-AsByteStream vs -Encoding Byte).
  • Robust Path Parsing: Fixed the bug where Windows local absolute paths were misidentified as VM paths. It now correctly parses formats like myvm:C:\path and rg:myvm:/path.
  • Documentation & Style:
    • Fixed azdev-style indentation issues in _help.py.
    • Detailed the requirements for curl on Linux VMs in the help documentation.
    • Refactored custom.py for better de-duplication and moved all imports to the module level per project standards.
  • Test Coverage: Added new unit tests in test_vm_cp_unit.py covering path parsing logic and end-to-end mock validation for both Upload and Download flows.

All changes have been committed and verified for syntactic correctness. The command is now ready for a final review.

@Ashutosh0x
Copy link
Author

Final CI Fixes and Refinements for az vm cp

I have pushed another set of refinements to address CI failures and incorporate remaining feedback:

  • Fixed Module Loading Issues: Moved all storage module imports from the top level to the function level (vm_cp). This fixes potential loading failures in environments where command modules are tested in isolation (like Test Extensions Loading Python313).
  • Improved URL Robustness: Added host normalization and validation for the storage endpoint suffix. This ensures correct blob URL construction across different Azure cloud environments (Public, Government, etc.), handling potential variations in suffix formatting.
  • Updated Unit Tests: Adjusted all unit test mocks to account for the function-level lazy imports, ensuring full test coverage remains intact.
  • Verified Standards: Confirmed that all cross-module dependencies in custom.py now follow the established secondary-import pattern used elsewhere in the codebase.

The previous CI failures (Build #20260208.2) should be resolved by these changes. The PR is ready for the latest CI run.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Auto-Assign Auto assign by bot Compute az vm/vmss/image/disk/snapshot customer-reported Issues that are reported by GitHub users external to the Azure organization. Storage az storage

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[FEATURE] New command to copy files from/to VMs

5 participants