Skip to content

test: describe hardware scenarios in Gherkin, play them on the adapters - #105

Draft
g-carre wants to merge 3 commits into
mainfrom
test/ARTESCA-17960-disk-failure-scenario-gherkin
Draft

g-carre wants to merge 3 commits into
mainfrom
test/ARTESCA-17960-disk-failure-scenario-gherkin

Conversation

@g-carre

@g-carre g-carre commented Oct 12, 2026 •

Copy link
Copy Markdown
Contributor

Gherkin counterpart of #102, to compare both formats. It follows the conventions of the org's Gherkin suites (metalk8s, artesca, Zenko, scality-tui). There is one commit per adapter: MegaRAID (with the framework), storcli2, then ssacli. It uses the same captures and expects the same tables as #102.

@megaraid
Feature: MegaRAID disk failure and replacement

  Background:
    Given the controller captured as "megaraid9560-4hdd"

  Scenario: A replacement drive gets the paths of its new volume
    Given drive 251:3 has failed
    And drive 251:3 has been replaced by a drive with serial "SIMD0013000000000000" and WWN "5000C50000000013"
    When a volume is created on drive 251:3, exposed as "/dev/sdm" with WWN "600062b2000000000000000000001237"
    Then the drives are reported as:
      | slot  | status | serial               | devicePath | permanentPath |
      ...
#102 (YAML) This PR (Gherkin)
scenarios/*.yaml, controller: key features/*.feature, @megaraid / @storcli2 / @ssacli tag, capture in Background
Cumulative phases, each a subtest One self-contained Scenario per phase (its prerequisites as Given steps), each a subtest
changes: in each controller's vocabulary The same domain steps for every controller (drive X fails, is replaced by…, a volume is created on…); each simulator applies them the way its controller shows them
knownBug: {ticket, reported} But known bug ARTESCA-17960 reports instead: table, right after the Then
sims: New(s *Scenario, phase int) sims: New(capture string, events ...scenario.Event)
scenario_test.go (Go API) steps_internal_test.go on testdata/*.feature fixtures: checks each failure case fails, and for the right reason

Behaviour is the same:

Trade-offs:

  • Scenarios read the same for every controller. The controller-specific details (MegaRAID cuts the serial of a failed drive to 8 characters, storcli2 has a separate drive status, ssacli moves the drive to Unassigned) live in the simulators and are documented there.
  • A state that no event can produce can't be written without adding a step and its handling in every simulator. test: describe hardware scenarios in YAML, play them on the adapters #102 can describe any state with changes.
  • go.mod uses github.com/cucumber/godog v0.16.0 instead of gopkg.in/yaml.v3.
  • disk-management-agent (test(storcli2): on-hardware e2e harness + sibling layout #74) would need rework. scenario.Run takes a *testing.T, so it can't run inside the agent's Ginkgo envtest suite as is.

Not meant to merge alongside #102: keep one of the two.

Refs: ARTESCA-17960

@g-carre
g-carre changed the base branch from main to test/ARTESCA-17960-megaraid-disk-failure-scenario October 12, 2026 09:50
@g-carre
g-carre changed the base branch from test/ARTESCA-17960-megaraid-disk-failure-scenario to main October 12, 2026 09:51
Add github.com/scality/raidmgmt/scenario, a separate module so that
consumers can play the scenarios without changing the raidmgmt version they
test:

- features/*.feature: Gherkin scenarios tagged with their controller
  family. The Background names the capture of a real controller; each
  scenario plays events on its drives (a drive fails, is replaced, gets a
  new volume), the ones it depends on written as Given steps, and gives the
  table of disks expected. When a bug makes it differ today, a known bug
  step gives the table reported because of it.
- scenario: the step definitions, run by godog strictly (undefined steps
  fail, empty tables and duplicate slots are rejected), and the comparison
  of what is reported with a table. A known bug passes only while its table
  is reported; it fails once the expected table is (the bug is fixed) and
  when neither is (the behaviour changed). A drive reported twice is a
  difference. Run plays the features of a controller family through a
  player; Report collects the disks an adapter reports, listing the
  controllers first as disk-management-agent does.
- steps_internal_test.go: checks on the features of testdata/ that the
  steps fail when they should, for the right reason.
- megaraidsim: answers storcli64 commands from a capture with events
  applied as storcli showed them on RD-2154, and shows the
  /dev/disk/by-id links of exposed volumes. Drive groups are written as
  storcli prints them: a number, or "-" for an unconfigured drive.
- megaraidsim/adapter_test.go: plays every @MegaRAID scenario through the
  megaraid adapter of this repository.

The first feature, megaraid-disk-failure, replays real storcli output of a
lab node, kept to its first 4 single-drive RAID0 volumes and with fake
serials and WWNs: all drives healthy, then drive 251:3 failing, being
replaced and getting a new volume. The failure scenario documents
ARTESCA-17960: the offline volume of the failed drive makes LogicalVolumes
fail for the whole controller, so no drive gets a path.

CI and the release gate now test and lint every module.

Refs: ARTESCA-17960
Add storcli2sim, the backend of storcli2/perccli2 controllers: it answers the
commands of the raidmgmt storcli2 adapter (controller listing, drives and
volumes) from a capture with events applied. storcli2 reports a drive
status apart from its state: a failed drive keeps state Conf with status
Failed, a replacement drive is UConf with status Good. Drive groups are
written as numbers, or "-" for an unconfigured drive. The capture is the
storcli2 test data of this repository, kept to 4 RAID0 drives, with fake
serials, WWNs, SAS addresses, volume and controller identifiers.

The storcli2-disk-failure feature plays the megaraid-disk-failure scenarios
with the same steps. The healthy drives keep their paths when 306:2 fails,
but the offline volume of the failed drive is still reported with a
permanent path built from its NAA Id, although it has no block device:
documented as a known bug of ARTESCA-17960.

Refs: ARTESCA-17960
…ntrollers

Add ssaclisim, the backend of ssacli controllers: it answers the commands of
the raidmgmt ssacli getters (controller listing, physical and logical drive
details, show config) from a capture with events applied. ssacli prints
text, so the backend edits the lines of the captured sections: drive and
logical drive status, serials, disk names, and moves a replacement drive to
the Unassigned section once its array is gone. The capture is the ssacli
test data of this repository, kept to 4 single-drive RAID0 arrays, with fake
serials, WWIDs and identifiers.

The ssacli-disk-failure feature plays the megaraid-disk-failure scenarios
with the same steps. The ssacli getters already report them as expected:
the healthy drives keep their paths, the failed one is Failed, and the
replacement drive is UnassignedGood.

Refs: ARTESCA-17960
@g-carre
g-carre force-pushed the test/ARTESCA-17960-disk-failure-scenario-gherkin branch from 92d09e9 to 74a8271 Compare October 12, 2026 14:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant