Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions src/SUMMARY.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,3 +22,4 @@
- [Ampere study](case_studies/Ampere_study.md)
- [Google study](case_studies/Google_study.md)
- [OCP TiogaPass](case_studies/TiogaPass.md)
- [Dasharo BenchRack ASRock TURIND8UD-2T/X550](case_studies/DasharoBenchRack_study.md)
300 changes: 300 additions & 0 deletions src/case_studies/DasharoBenchRack_study.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,300 @@
# Dasharo BenchRack ASRock TURIND8UD-2T/X550

BenchRack is [3mdeb](https://3mdeb.com/)'s modular test-bench reference design
for open-source firmware development and validation.

This case study covers the
[Dasharo BenchRack ASRock TURIND8UD-2T/X550](https://docs.dasharo.com/variants/asrock_turind8ud/overview/) configuration,
in which the DUT boots with [coreboot](https://www.coreboot.org/) (as
distributed by 3mdeb) performing silicon initialization and LinuxBoot serving
as the payload, following the same LinuxBoot adoption approach described in the
[Ampere](Ampere_study.md) and [Google](Google_study.md) case studies.

It contains the following sections:

* [Reference design and goals](#reference-design-and-goals)
* [Platform overview](#platform-overview)
* [Firmware architecture](#firmware-architecture)
* [Build process](#build-process)
* [Prerequisites](#prerequisites)
* [Build the firmware image](#build-the-firmware-image)
* [Flashing and operation](#flashing-and-operation)
* [Booting with LinuxBoot](#booting-with-linuxboot)
* [Support](#support)
* [Hardware support](#hardware-support)
* [Community support](#community-support)
* [Professional support](#professional-support)
* [See also](#see-also)

## Reference design and goals

BenchRack is intended as a reference design, not a one-off test rig. By
standardizing how a DUT is powered, flashed, and observed, it serves several
business and engineering goals:

* **Compliance optimization showcasing** — demonstrating how an open firmware
stack can be tuned and evidenced to meet compliance and certification
requirements.
* **Firmware due diligence and patching loop** — providing a repeatable
environment to audit firmware, reproduce issues, and validate patches in a
tight build-flash-test loop.
* **Decomposition of complex rack systems** — breaking a complex rack system
into individually manageable components, enabling a divide-and-conquer
strategy for reasoning about and validating large systems.

Because BenchRack is defined by this control-and-validation harness rather than
by a specific board, future BenchRack configurations may host entirely
different motherboards. This case study documents the ASRock TURIND8UD-2T/X550
configuration.

## Platform overview

The device under test (DUT) in this configuration is the
[ASRock Rack TURIND8UD-2T/X550](https://docs.dasharo.com/variants/asrock_turind8ud/overview/),
a single-socket AMD EPYC server board:

* **CPU**: single socket SP5 (LGA 6096), supporting AMD EPYC 9005 series
processors.
* **Memory**: 8 DDR5 RDIMM slots (1 DIMM per channel).
* **Expansion**: 4 PCIe 5.0 slots and 2 M.2 (PCIe 5.0 NVMe) slots.
* **Networking**: onboard 10 GbE.

The DUT is mounted in a BenchRack, which supplies remote power control, firmware
flashing, and serial capture so the board can be validated automatically. Both
silicon initialization (coreboot) and the payload (LinuxBoot) are open source,
though — as on most modern x86 platforms — a few binary blobs are still required
for silicon initialization (for example AMD PSP firmware and microcode). This

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Said twice. Nitpick

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed the redundant comment: d75461c

places the platform at the most advanced practical stage of the LinuxBoot
adoption model described in the [Ampere](Ampere_study.md) case study.

## Firmware architecture

The boot firmware for the DUT is largely open source, with a few binary blobs
required for silicon initialization:

* **coreboot**: performs early hardware and silicon initialization. Source is
available at [Dasharo/coreboot](https://github.com/Dasharo/coreboot.git).
* **LinuxBoot**: built as the coreboot payload (u-root initramfs +
[flashkernel](../glossary.md)), responsible for boot device selection and
`kexec` into the target OS.

## Build process

The firmware image is a single `coreboot.rom` containing coreboot plus the
LinuxBoot payload.

### Prerequisites

The recommended, reproducible build path uses the Dasharo/coreboot SDK
container, which pins the toolchain and build dependencies.

```bash
git clone https://github.com/Dasharo/coreboot.git
cd coreboot
git checkout asrock_turind8ud_linuxboot_v0.9.0
git submodule update --init --checkout
```

You must also obtain and extract blobs necessary for memory and silicon
initialization. Please familiarize yourself with the [Dasharo terms of service](https://www.dasharo.com/pages/terms/),
then download and extract them:

```bash
wget https://dl.3mdeb.com/open-source-firmware/Dasharo/gigabyte_mz33_ar1/uefi/v0.9.0/Turin.zip
unzip Turin.zip -d 3rdparty/blobs/soc/amd/
```

Enter the Dasharo SDK container:

```bash
# Enter the pinned Dasharo SDK container
docker run --rm -it \
-v "$PWD:/home/coreboot/coreboot" \
-w /home/coreboot/coreboot \
ghcr.io/dasharo/dasharo-sdk:v1.9.2 \
/bin/bash
```

### Build the firmware image

From inside the SDK container:

```bash
# Select the BenchRack DUT board target and the LinuxBoot payload
./build.sh asrock_turind8ud_linuxboot
```

The resulting flashable image is at `asrock_turind8ud_linuxboot_<version>.rom`.

## Flashing and operation

BenchRack provides remote flashing, power control, and serial capture for each
DUT, enabling automated (CI) firmware deployment and testing. These are driven
by [benchctl](https://github.com/zarhus/benchctl), a single command-line tool
that controls the bench from a workstation or from the bench itself. For
BenchRack it operates the [Remote Test Environment (RTE)](https://docs.dasharo.com/transparent-validation/rte/v1.1.0/specification/),
driving the GPIO lines, the SPI mux, and host power through the RTE REST API and
running `flashrom` over SSH — so the low-level SPI sequencing no longer has to be
performed by hand.

Select the BenchRack platform and DUT through flags (or the matching
`BENCHCTL_*` environment variables). The `benchrack` platform defaults to the
`asrock-turin` board profile, so `--board` can be omitted for this DUT. When run
from a workstation, `--host` points at the RTE by IP or hostname; run directly on
the RTE, the host is implied and the flag is dropped.

To flash the host boot flash with the image built above:

```bash
# From a workstation, targeting the RTE:
benchctl --host <RTE-IP> \
flash host ./asrock_turind8ud_linuxboot_<version>.rom
```

`benchctl` checks the firmware size, copies it to the RTE with `scp`, takes
control of the SPI bus, runs the blocking flash, confirms it succeeded, and
cleans up the temporary file. Power and serial console are managed through the
same tool:

```bash
benchctl --host <RTE-IP> power reset # power-cycle the DUT
benchctl --host <RTE-IP> console # attach to serial
```

## Booting with LinuxBoot

On power-on, the DUT runs coreboot, hands off to the LinuxBoot payload, boots
into u-root, and `kexec`s into the target OS.

```text
Welcome to LinuxBoot's Menu

Enter a number to boot a kernel:

01. Ubuntu

02. Ubuntu, with Linux 7.0.0-27-generic

03. Ubuntu, with Linux 7.0.0-27-generic (recovery mode)

04. Ubuntu, with Linux 7.0.0-22-generic

05. Ubuntu, with Linux 7.0.0-22-generic (recovery mode)

06. Memory test (mt86+x64)

07. Memory test (mt86+x64)

08. Memory test (mt86+x64, serial console)

09. Memory test (mt86+x64, serial console)

10. Memory test (mt86+ia32)

11. Memory test (mt86+ia32)

12. Memory test (mt86+ia32, serial console)

13. Memory test (mt86+ia32, serial console)

14. Reboot

15. Enter a LinuxBoot shell


Enter an option ('01' is the default, 'e' to edit kernel cmdline):
>

Attempting to boot LinuxImage(
Name: Ubuntu
Kernel: file:///tmp/u-root-mounts1460215710/nvme0n1p2/boot/vmlinuz-7.0.0-27-generic
Initrd: file:///tmp/u-root-mounts1460215710/nvme0n1p2/boot/initrd.img-7.0.0-27-generic
Cmdline: root=UUID=d6f381c6-1549-4560-87e9-f26d9e317ab1 ro quiet splash crashkernel=2G-4G:320M,4G-32G:512M,32G-64G:1024M,64G-128G:2048M,128G-:4096M console=ttyS0,115200
DTB: <nil>
)
```

## Benchmarks

### Boot Time

coreboot timestamps may be obtained using `cbmem -t`:

```text
ubuntu@3mdeb:~$ sudo cbmem -t
36 entries total:

0:1st timestamp 29,106,318 (0)
11:start of bootblock 29,106,553 (235)
12:end of bootblock 29,109,232 (2,678)
13:starting to load romstage 29,109,232 (0)
17:starting LZ4 decompress (ignore for x86) 29,174,661 (65,428)
18:finished LZ4 decompress (ignore for x86) 29,174,726 (65)
14:finished loading romstage 29,174,730 (3)
1:start of romstage 29,174,734 (3)
4:end of romstage 29,174,885 (151)
8:starting to load ramstage 29,174,885 (0)
15:starting LZMA decompress (ignore for x86) 29,536,943 (362,058)
16:finished LZMA decompress (ignore for x86) 29,583,677 (46,733)
9:finished loading ramstage 29,586,162 (2,485)
10:start of ramstage 29,587,320 (1,158)
30:device enumeration 29,587,383 (62)
112:started reading uCode 29,587,766 (383)
113:finished reading uCode 29,617,222 (29,455)
31:<unknown> 31,365,077 (1,747,855)
40:device configuration 31,371,820 (6,742)
50:device enable 31,379,595 (7,775)
60:device initialization 31,381,611 (2,015)
70:device setup done 31,884,811 (503,199)
920:starting APOB read 31,884,813 (2)
921:starting APOB erase 31,954,244 (69,431)
922:starting APOB write 39,357,767 (7,403,523)
923:finished APOB 40,321,934 (964,166)
75:cbmem post 40,343,709 (21,775)
80:write tables 40,343,713 (3)
85:finalize chips 40,352,487 (8,774)
90:starting to load payload 40,352,494 (6)
15:starting LZMA decompress (ignore for x86) 54,335,177 (13,982,683)
16:finished LZMA decompress (ignore for x86) 54,335,213 (35)
15:starting LZMA decompress (ignore for x86) 56,242,188 (1,906,975)
16:finished LZMA decompress (ignore for x86) 56,242,324 (136)
15:starting LZMA decompress (ignore for x86) 56,242,327 (2)
16:finished LZMA decompress (ignore for x86) 56,242,350 (23)
99:selfboot jump 57,809,271 (1,566,921)
```

Stock ASRock UEFI can be easily measured with systemd-analyze, though that's
excluding PSP time, which should be the same as in coreboot:

```text
ubuntu@3mdeb:~$ sudo systemd-analyze
[sudo] password for ubuntu:
Startup finished in 2min 58.748s (firmware) + 1.126s (loader) + 539ms (kernel) + 10.353s (initrd) + 10.407s (userspace) = 3min 21.174s
graphical.target reached after 10.402s in userspace.
```

Including time spent in PSP firmware, coreboot + LinuxBoot reduces the boot time
from 3:27 to 57 seconds.

## Support

### Hardware support

Hardware support is provided by [3mdeb](https://3mdeb.com/), and BenchRack units
are available from the [3mdeb shop](https://shop.3mdeb.com/product/benchrack-asrock-turind8ud-2t-x550-with-dasharo-pro-package-for-servers/).

### Community support

* [Dasharo community](https://docs.dasharo.com/#community) — documentation and
matrix chat channels.
* [LinuxBoot open source community](https://www.linuxboot.org/) — Slack, IRC,
mailing list, and regular meetings for technical questions.

### Professional support

Professional support services are provided by [3mdeb](https://3mdeb.com/).

## See also

* [Dasharo documentation](https://docs.dasharo.com/)
* [Dasharo RTE specification](https://docs.dasharo.com/transparent-validation/rte/v1.1.0/specification/)
1 change: 1 addition & 0 deletions src/case_studies/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,3 +7,4 @@ This chapter contains case studies for various solutions.
1. [Ampere study](Ampere_study.md)
2. [Google study](Google_study.md)
3. [OCP TiogaPass](TiogaPass.md)
4. [Dasharo BenchRack ASRock TURIND8UD-2T/X550](DasharoBenchRack_study.md)