Repository navigation
Conversation
| // A volume not exposed to the OS yet is listed without paths: it is | ||
| // not settled, the caller rescans and retries. | ||
| if lv.PermanentPath == "" { | ||
| return nil, errors.Errorf("new logical volume %s has no device yet", lv.ID) |
There was a problem hiding this comment.
This returns on the first volume that shares a drive slot and has no paths, and skips the rest of the list. hasMatchingPDs matches on any one shared slot. So if the offline volume of a failed drive still lists that slot after the drive is replaced, it can show up before the new volume, and CreateLV keeps retrying until it times out even though the new volume is already exposed. Use continue so the loop still checks the other volumes. It still ends in an error, and a retry, when no matching volume has paths.
| return nil, errors.Errorf("new logical volume %s has no device yet", lv.ID) | |
| continue |
697914a to
4ee0286
Compare
74a8271 to
eb4261b
Compare
| DevicePath: strings.TrimSpace(vd.Properties.OSDriveName), | ||
| PermanentPath: permanentPath(vd.Properties.SCSINAAID), | ||
| DevicePath: devicePath(vd.Properties), | ||
| PermanentPath: permanentPath(vd.Properties), |
There was a problem hiding this comment.
This also changes storcli2 CreateLV. logicalvolumemanager.StorCLI2.findNewLogicalVolume returns the first volume with a matching drive set, without checking its paths. The PR says a new volume can be listed before it is exposed to the OS. In that case CreateLV now returns the volume with empty DevicePath/PermanentPath, where before it got the NAA-based path. Add the same wait as in megaraid: treat an empty PermanentPath as not settled and retry, or make the creator re-query until the volume is exposed.
| blockDevice, err := s.getBlockDevice(value) | ||
| if err != nil { | ||
| return errors.Wrapf(err, "failed to get block device for %s", value) | ||
| facts.blockDeviceUnknown = true |
There was a problem hiding this comment.
This drops every getBlockDevice error, not just a missing device node. If lsblk is missing, fails for another reason, or its JSON format changes, every drive silently loses its DevicePath and every free drive becomes Unknown, and nothing reports why. That is the "all disks blanked" symptom this PR fixes, now with no error at all. Only degrade when the node is actually gone (for example, check that value exists before calling lsblk), and keep returning the wrapped error otherwise.
4ee0286 to
594d87d
Compare
| } | ||
|
|
||
| if lv.PermanentPath == "" { | ||
| return nil, errors.Wrapf(errNewVolumeNotExposed, "volume %s", lv.ID) |
There was a problem hiding this comment.
This has the same early-return issue that was fixed in megaraid's selectNewLogicalVolume. If the offline RAID0 volume of a failed drive still lists the replaced slot, it matches the single-drive set exactly. When it is listed before the new volume, this returns errNewVolumeNotExposed without checking the rest of the list, so CreateLV waits 60 s and fails even though the new volume is already exposed. Remember the unexposed match and continue, then return errNewVolumeNotExposed after the loop only if no matching volume has paths.
594d87d to
a1f75a3
Compare
| continue | ||
| } | ||
|
|
||
| if lv.PermanentPath == "" { |
There was a problem hiding this comment.
An empty PermanentPath does not only mean "not exposed". The getter also returns it when SCSI NAA Id is empty (see the empty case in TestStorCLI2PermanentPath). In that case an exposed new volume that has a DevicePath is treated as unsettled, so CreateLV waits 60 s and fails after the volume was already created. An unexposed volume has both paths empty, so check both.
| if lv.PermanentPath == "" { | |
| if lv.PermanentPath == "" && lv.DevicePath == "" { |
dc6a89b to
da9ef41
Compare
eb4261b to
ae2f0cd
Compare
When the drive of a single-drive RAID0 volume fails, the volume goes offline and is no longer a block device of the host: it has no by-id link, and the paths of its failed drive cannot be computed. getPaths returned an error, which failed LogicalVolumes for the whole controller, so the healthy drives of that controller lost their paths too (RD-2154). - A volume whose "Exposed to OS" is "No" gets empty paths without error. Only this explicit value, spaces trimmed, skips the resolution, as older storcli versions may not report the field. - An offline volume (OfLn) is reported Failed instead of Unknown. - Volume creation still waits for the new volume: a volume listed before it is exposed to the OS has no paths yet, so it is not settled and creation keeps rescanning and retrying. Such a volume no longer stops the search: the offline volume of a failed drive still listing a requested slot must not hide the new, exposed volume. DESIGN.md states that getter read paths degrade gracefully while the create path stays strict. The failed drive scenario of megaraid-disk-failure.feature now reports its expected table: its known bug step is removed. Fixes: ARTESCA-17960
A failed or pulled drive whose device node has disappeared made the lsblk lookup of its Disk Name fail, which aborted the physical drive inventory of the whole controller. The lookup only refines the status, so when it fails and the device node is gone, it no longer aborts the inventory. Instead: - the usage of the device is unknown, and a free drive whose usage is unknown is reported Unknown, never UnassignedGood, so a drive that may carry data is never offered for a new volume; - the device path is dropped, as the kernel can reuse a vanished /dev/sdX name for another disk. Array members and failed drives keep the status ssacli reports. A lookup failure on a device that exists, such as lsblk missing or its output changing, is still an error: ignoring it would silently blank every drive. Refs: ARTESCA-17960
… resolved A JBOD drive whose by-id link cannot be resolved, for instance because udev has not settled yet, failed the physical drive inventory of the whole controller. Keep it with empty paths instead, when its by-id link or device node is missing (ComputePaths now returns ErrNoPermanentPath for a missing link). Any other failure, such as a permission error, is still an error: ignoring it would silently blank the paths of every JBOD drive. Also cover a degraded RAID volume, whose paths come from the controller output only. Refs: ARTESCA-17960
The storcli2 volume getter read the device path of a volume from its OS drive name and built its permanent path from its NAA Id. The offline RAID0 volume of a failed drive keeps its NAA Id, and may keep a stale OS drive name, but it is not exposed to the OS: it has no block device and no by-id link, so both paths named nothing, or another disk. A volume whose "Exposed to OS" is "No" now has no paths; only this explicit value drops them, as older storcli2 versions may not report the field. Volume creation used to get the NAA-based path of a new volume listed before it is exposed to the OS. It now gets no paths, so CreateLV waits, like megaraid, until the new volume is exposed or NewVolumeSettleTimeout elapses (60s when zero). DESIGN.md says which failures degrade and which stay errors. The failed drive scenario of storcli2-disk-failure.feature now reports its expected table: its known bug step is removed. Fixes: ARTESCA-17960
ae2f0cd to
3791dcd
Compare
da9ef41 to
f943de8
Compare
Commits, one per adapter fix: megaraid, ssacli, storcli2 drives, storcli2 volumes.
Fixes ARTESCA-17960 (RD-2154). Stacked on #105: it targets #105's branch, so the diff only shows the fix. Retarget it to
mainonce #105 is merged. It replaces #103 (stacked on the closed YAML PR #102) and supersedes #91.Problem
One unhealthy drive could fail the inventory of its whole controller, so every healthy drive on that controller lost its device paths. On the RD-2154 node this blanked all disks in the UI.
Fixes
OfLn,Exposed to OS: No) and is no longer a block device.getPathsstill tried to resolve its paths, failed, and failedLogicalVolumesfor the whole controller.Exposed to OS: Nonow gets empty paths without error. Only this explicit value (spaces trimmed) skips resolution, since older storcli versions may not report the field.OfLnis now reportedFailedinstead ofUnknown./devnode is gone made thelsblklookup fail, which aborted the whole inventory. When the device node is gone, a lookup failure now marks the device's usage as unknown instead. Any other lookup failure, such aslsblkmissing or failing on a device that exists, is still an error, so it cannot silently blank every drive. A free drive whose usage is unknown is reportedUnknown, neverUnassignedGood, so a drive that may carry data is never offered for a new volume.UsedJBOD drive whose by-id link or device node is missing is kept with empty paths instead of failing the whole inventory. Any other resolution failure, such as a permission error, is still an error.Exposed to OS: Nonow has no paths: neither the permanent path built from its NAA Id, nor a device path from a possibly stale OS drive name.CreateLVused to return a new volume with an NAA-based path even before it was exposed. It now waits, like megaraid, until the volume is exposed orNewVolumeSettleTimeout(60 s, also when left at zero) elapses. As in megaraid, an offline volume with the same drives cannot hide the new one. A volume counts as not exposed only when it has no path at all, so an exposed volume without an NAA Id (device path only) is returned at once.DESIGN.md. Documents the contract that getter read paths degrade gracefully, which failures still are errors, and that the megaraid and storcli2 create paths are deliberately stricter.#91's change for multi-drive megaraid volumes is not carried over, since ARTESCA does not use them.
Tests
megaraid-disk-failure.featureandstorcli2-disk-failure.feature, the scenario "A failed drive does not take the paths of the others" now reports its expected table, so bothknown bug ARTESCA-17960 reports instead:steps are removed. Until they were removed, those steps failed with "the bug looks fixed". All 12 scenarios pass (4 per controller), for example on MegaRAID:getPathswithExposed to OSset toNo(empty paths, no error),Yes, or empty (paths still resolved);OfLn→Failed;lsblklookup does not abort the inventory when the device node is gone, and is still an error when it exists. A free drive whose device could not be inspected is reportedUnknown.permanentPathcoversExposed to OSset toNo,Yesand missing.CreateLVwaits for a volume exposed on the third listing, gives up on one never exposed, is not hidden by an offline volume with the same drives listed first, and returns at once a volume exposed without an NAA Id.Delivery
disk-management-agent picks this up once raidmgmt is released and the agent bumps it.