Initialize CSI client before checking publish capability

[toughlex-donatas]

Summary

Initialize the CSI plugin client before checking whether the plugin supports PUBLISH_UNPUBLISH_VOLUME.

Problem

The publisher field is populated during CSI client initialization, when SwarmKit calls ControllerGetCapabilities.

After a Swarm manager restart or leader change, a new plugin wrapper starts with:

publisher = false

However, PublishVolume checks this field before calling Client():

if !p.publisher {
    return nil, nil
}

At this point, false may mean that the plugin has not been initialized yet, rather than that it does not support controller publish/unpublish operations.

As a result, SwarmKit can incorrectly treat volume publication as successful without calling ControllerPublishVolume and without receiving a publish context.

For the AWS EBS CSI driver, this produces the following failure sequence:

1. SwarmKit marks the volume as published.
2. The EBS volume remains available because no attach request was made.
3. The node receives an empty publish context.
4. NodeStageVolume fails with:

rpc error: code = InvalidArgument desc = Device path not provided

The volume then appears as in use in Docker, while it is not attached in AWS.

Fix

Call Client() before checking p.publisher:

c, err := p.Client(ctx)
if err != nil {
    return nil, err
}

if !p.publisher {
    return nil, nil
}

This ensures that plugin capabilities are loaded before SwarmKit decides whether controller-side publish operations are supported.

Reproduction

1. Create an AWS EBS cluster volume using the AWS EBS CSI driver.
2. Restart the current Swarm leader or trigger a leader election.
3. Schedule a service using an existing cluster volume.
4. Observe that Docker marks the volume as in use, while the EBS volume remains available.
5. Observe repeated node-side errors:

publishing volume failed: rpc error: code = InvalidArgument desc = Device path not provided

Creating a new temporary CSI volume before scheduling the service also works around the issue because the create path initializes the CSI client and loads controller capabilities.

Expected behavior

SwarmKit should initialize the CSI plugin before checking its controller capabilities.

When the driver supports PUBLISH_UNPUBLISH_VOLUME, ControllerPublishVolume should be called and its publish context should be passed to the node-side CSI operations.

Testing

Tested with:

• Docker Engine 29.7.1
• Docker Swarm
• AWS EBS CSI driver
• Existing cluster volumes after a manager restart or leader change

Before this change, existing EBS volumes could become stuck in an inconsistent state:

Docker: in use
AWS: available

After this change, the EBS volume is attached normally and the service starts successfully.

[thaJeztah]

Thank you for contributing! It looks like your commit message is missing a DCO sign-off,
causing the DCO check to fail.

We require all commit messages to have a Signed-off-by line with your name
and e-mail (see "Sign your work"
in the CONTRIBUTING.md in this repository), which looks something like:

Signed-off-by: YourFirstName YourLastName <yourname@example.org>

There is no need to open a new pull request, but to fix this (and make CI pass),
you need to amend the commit(s) in this pull request, and "force push" the amended
commit.

Unfortunately, it's not possible to do so through GitHub's web UI, so this needs
to be done through the git commandline.

You can find some instructions in the output of the DCO check (which can be found
in the "checks" tab on this pull request), as well as in the Moby contributing guide.

Steps to do so "roughly" come down to:

1. Set your name and e-mail in git's configuration:

   git config --global user.name "YourFirstName YourLastName"
   git config --global user.email "yourname@example.org"

   (Make sure to use your real name (not your GitHub username/handle) and e-mail)

2. Clone your fork locally

3. Check out the branch associated with this pull request

4. Sign-off and amend the existing commit(s)

   git commit --amend --no-edit --signoff

   If your pull request contains multiple commits, either squash the commits (if
   needed) or sign-off each individual commit.

5. Force push your branch to GitHub (using the --force or --force-with-lease flags) to update the pull request.

Let me know if you need help or more detailed instructions!

[Copilot]

🟡 Not ready to approve

The new blocks introduce non-gofmt indentation/trailing whitespace that may fail formatting/lint checks and should be corrected before approval.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

This review doesn't count toward merge requirements. Sign up for the private preview to control whether Copilot approvals count.

Pull request overview

This PR fixes a CSI controller capability detection edge case where PublishVolume/UnpublishVolume could incorrectly no-op after a Swarm manager restart/leader change because the plugin client (and thus ControllerGetCapabilities) hadn’t been initialized yet.

Changes:

• Initialize the CSI controller client (Client()) before checking p.publisher in PublishVolume.
• Initialize the CSI controller client (Client()) before checking p.publisher in UnpublishVolume.

File summaries

File	Description
manager/csi/plugin.go	Ensures CSI controller capabilities are loaded before deciding whether to call controller-side publish/unpublish RPCs.

Review details

Suppressed comments (1)

manager/csi/plugin.go:242

• The new error-handling block is indented with spaces (and the following blank line contains trailing whitespace), which deviates from gofmt formatting used throughout this file; this can cause gofmt/lint checks to fail and makes the code inconsistent.

	c, err := p.Client(ctx)
    if err != nil {
        return err
    }
	

• Files reviewed: 1/1 changed files
• Comments generated: 2
• Review effort level: Lite

We're testing this review assessment. Please use 👍 or 👎 to tell us if it's correct.

[thaJeztah]

I see there's minor lining issue as well;

0.054 + golangci-lint run --config /go/src/github.com/docker/swarmkit/.golangci.yml ./...
39.98 manager/csi/plugin.go:212:1: File is not properly formatted (gofmt)
39.98     if err != nil {
39.98 ^
39.98 1 issues:
39.98 * gofmt: 1

[codecov-commenter]

Codecov Report

❌ Patch coverage is 33.33333% with 4 lines in your changes missing coverage. Please review.
✅ Project coverage is 14.72%. Comparing base (6e9e7b8) to head (82675ab).
⚠️ Report is 60 commits behind head on master.

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #3276      +/-   ##
==========================================
- Coverage   14.73%   14.72%   -0.01%     
==========================================
  Files         200      200              
  Lines       93077    93027      -50     
==========================================
- Hits        13712    13699      -13     
+ Misses      78019    77991      -28     
+ Partials     1346     1337       -9     

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[toughlex-donatas]

Thank you, I fixed sign-off and whitespace