OSUOSL decommissioning POWER8

[richardlau]

https://lists.osuosl.org/pipermail/openpower/Week-of-Mon-20260720/000122.html

We are decommissioning our POWER8 hardware.
...
On the migration itself: all instances are moving to POWER9. I migrated VMs
off POWER8 yesterday afternoon so that several machines could be powered
down immediately, and I am moving any remaining instances today. Your
instances will continue to run on the new hardware, so no action is
required on your part. I expect POWER8 to be fully decommissioned over the
next few weeks.

The text of the email refers to Linux but we might need to check if this also applies to our AIX machines.

I don't know yet if this is going to risk releases on Node.js 22 and 24 which still support POWER8 hardware (e.g. would compiling on POWER9 result in instructions being used that are not present in POWER8?). On IBM Cloud we can run VMs in POWER8 compatibility mode -- I don't know yet if that is possible in OpenStack at OSUOSL.

Node.js 26 dropped support for POWER8 so will not be affected.

[richardlau]

Following on from today's Build WG call:

We currently only set -mcpu=power9 on AIX. We're not setting -mcpu (at all, to any value) for Linux ppc64le.

I have confirmed all of our OSUOSL ppc64le VMs are now POWER9:
e.g.

$  ssh test-osuosl-rhel8-ppc64_le-3 lscpu
Architecture:        ppc64le
Byte Order:          Little Endian
CPU(s):              4
On-line CPU(s) list: 0-3
Thread(s) per core:  1
Core(s) per socket:  1
Socket(s):           4
NUMA node(s):        1
Model:               2.2 (pvr 004e 1202)
Model name:          POWER9 (architected), altivec supported
Hypervisor vendor:   KVM
Virtualization type: para
L1d cache:           32K
L1i cache:           32K
NUMA node0 CPU(s):   0-3
$

For an IBM Cloud VM in POWER8 compatibility mode (not yet connected to Jenkins):

[root@ilon04-asr4 ~]#  lscpu
Architecture:        ppc64le
Byte Order:          Little Endian
CPU(s):              32
On-line CPU(s) list: 0-31
Thread(s) per core:  8
Core(s) per socket:  4
Socket(s):           1
NUMA node(s):        1
Model:               2.2 (pvr 004e 0202)
Model name:          POWER8 (architected), altivec supported
Hypervisor vendor:   pHyp
Virtualization type: para
L1d cache:           32K
L1i cache:           32K
NUMA node0 CPU(s):   0-31
Physical sockets:    2
Physical chips:      1
Physical cores/chip: 10
[root@ilon04-asr4 ~]#

In OSUOSL OpenStack these VMs are now p9.large (previously p8.large). I don't see anything obvious in OpenStack to set a processor compatibility mode.

[richardlau]

I've built a test build on the release CI (machine in now POWER9): https://ci-release.nodejs.org/job/iojs+release/11949/nodes=rhel8-ppc64le-release/
Test builds also available in: https://nodejs.org/download/test/v24.18.0-test20260723b881107b8f/

Basic node --version, node -p process.versions work on the VM in POWER8 compatibility mode. I'd need to do more configuration on the machine (i.e. connect it to our RHEL subscription) to set up for running the Node.js test suite.

We might be saved in that it looks like Red Hat have configured gcc (10 for Node.js 20 and 12 for Node.js 22) on RHEL 8 for POWER8 (--with-cpu-64=power8 --with-tune-64=power8) so that if -mcpu is not specified it is defaulting to POWER8:

[cloud-user@release-osuosl-rhel8-ppc64--le-1 ~]$  . /opt/rh/gcc-toolset-12/enable
[cloud-user@release-osuosl-rhel8-ppc64--le-1 ~]$  gcc -v
Using built-in specs.
COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=/opt/rh/gcc-toolset-12/root/usr/libexec/gcc/ppc64le-redhat-linux/12/lto-wrapper
OFFLOAD_TARGET_NAMES=nvptx-none
OFFLOAD_TARGET_DEFAULT=1
Target: ppc64le-redhat-linux
Configured with: ../configure --enable-bootstrap --enable-languages=c,c++,fortran,lto --prefix=/opt/rh/gcc-toolset-12/root/usr --mandir=/opt/rh/gcc-toolset-12/root/usr/share/man --infodir=/opt/rh/gcc-toolset-12/root/usr/share/info --with-bugurl=http://bugzilla.redhat.com/bugzilla --enable-shared --enable-threads=posix --enable-checking=release --enable-targets=powerpcle-linux --disable-multilib --with-system-zlib --enable-__cxa_atexit --disable-libunwind-exceptions --enable-gnu-unique-object --enable-linker-build-id --with-gcc-major-version-only --enable-libstdcxx-backtrace --with-linker-hash-style=gnu --enable-plugin --enable-initfini-array --with-isl=/builddir/build/BUILD/gcc-12.2.1-20221121/obj-ppc64le-redhat-linux/isl-install --enable-offload-targets=nvptx-none --without-cuda-driver --enable-offload-defaulted --enable-gnu-indirect-function --enable-secureplt --with-long-double-128 --with-long-double-format=ibm --with-cpu-32=power8 --with-tune-32=power8 --with-cpu-64=power8 --with-tune-64=power8 --build=ppc64le-redhat-linux
Thread model: posix
Supported LTO compression algorithms: zlib zstd
gcc version 12.2.1 20221121 (Red Hat 12.2.1-7) (GCC)
[cloud-user@release-osuosl-rhel8-ppc64--le-1 ~]$  gcc -Q --help=target 2>&1 | grep power
  -mcpu=                                power8
  -mpower10                             [disabled]
  -mpower10-fusion                      [disabled]
  -mpower8-fusion                       [enabled]
  -mpower8-fusion-sign                  [disabled]
  -mpower8-vector                       [enabled]
  -mpower9-minmax                       [disabled]
  -mpower9-misc                         [disabled]
  -mpower9-vector                       [disabled]
  -mpowerpc                             [ignored]
  -mpowerpc-gfxopt                      [enabled]
  -mpowerpc-gpopt                       [enabled]
  -mpowerpc64                           [enabled]
  -mtune=                               power8
    natural power
    native power10 power3 power4 power5 power5+ power6 power6x power7 power8 power9 powerpc powerpc64 powerpc64le rs64 titan
[cloud-user@release-osuosl-rhel8-ppc64--le-1 ~]$  . /opt/rh/gcc-toolset-10/enable
[cloud-user@release-osuosl-rhel8-ppc64--le-1 ~]$  gcc -v
Using built-in specs.
COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=/opt/rh/gcc-toolset-10/root/usr/libexec/gcc/ppc64le-redhat-linux/10/lto-wrapper
Target: ppc64le-redhat-linux
Configured with: ../configure --enable-bootstrap --enable-languages=c,c++,fortran,lto --prefix=/opt/rh/gcc-toolset-10/root/usr --mandir=/opt/rh/gcc-toolset-10/root/usr/share/man --infodir=/opt/rh/gcc-toolset-10/root/usr/share/info --with-bugurl=http://bugzilla.redhat.com/bugzilla --enable-shared --enable-threads=posix --enable-checking=release --enable-targets=powerpcle-linux --disable-multilib --with-system-zlib --enable-__cxa_atexit --disable-libunwind-exceptions --enable-gnu-unique-object --enable-linker-build-id --with-gcc-major-version-only --with-linker-hash-style=gnu --enable-plugin --enable-initfini-array --with-isl=/builddir/build/BUILD/gcc-10.3.1-20210422/obj-ppc64le-redhat-linux/isl-install --disable-libmpx --enable-gnu-indirect-function --enable-secureplt --with-long-double-128 --with-cpu-32=power8 --with-tune-32=power8 --with-cpu-64=power8 --with-tune-64=power8 --build=ppc64le-redhat-linux
Thread model: posix
Supported LTO compression algorithms: zlib zstd
gcc version 10.3.1 20210422 (Red Hat 10.3.1-1) (GCC)
[cloud-user@release-osuosl-rhel8-ppc64--le-1 ~]$  gcc -Q --help=target 2>&1 | grep power
  -mcpu=                                power8
  -mpower10                             [disabled]
  -mpower8-fusion                       [enabled]
  -mpower8-fusion-sign                  [disabled]
  -mpower8-vector                       [enabled]
  -mpower9-minmax                       [disabled]
  -mpower9-misc                         [disabled]
  -mpower9-vector                       [disabled]
  -mpowerpc                             [ignored]
  -mpowerpc-gfxopt                      [enabled]
  -mpowerpc-gpopt                       [enabled]
  -mpowerpc64                           [enabled]
  -mtune=                               power8
    natural power
    native power10 power3 power4 power5 power5+ power6 power6x power7 power8 power9 powerpc powerpc64 powerpc64le rs64 titan
[cloud-user@release-osuosl-rhel8-ppc64--le-1 ~]$

[richardlau]

This table is what we're supposed to have set up (after #4236):

Intended arch	IBM	OSUOSL
RHEL 8 POWER8	-	release-osuosl-rhel8-ppc64_le-1 test-osuosl-rhel8-ppc64_le-1 test-osuosl-rhel8-ppc64_le-2 test-osuosl-rhel8-ppc64_le-3
RHEL 8 POWER9	test-ibm-rhel8-ppc64_le-1	release-osuosl-rhel8-ppc64_le-2 test-osuosl-rhel8-ppc64_le-4 test-osuosl-rhel8-ppc64_le-5
RHEL 9 POWER9	test-ibm-rhel9-ppc64_le-1	test-osuosl-rhel9-ppc64_le-1 test-osuosl-rhel9-ppc64_le-2

All of those OSUOSL VMs are now POWER9.

Short term problem (prior to next week's security releases)

We currently have no POWER8 VMs. Not having any POWER8 testing is risky because Node.js 24 and 22 support POWER8 and we have had noticeable differences in behaviour on POWER8 vs newer (e.g. POWER9) due to e.g. SIMD only being available from POWER9.

The quickest way to get POWER8 testing back is to have two (minimum number of test runners per platform) VMs on IBM Cloud in POWER8 compatibility mode. This would either be two new VMs, or we could repurpose one (and one new VM) or both of the existing POWER9 VMs hosted on IBM Cloud.

Release machine?

Based on how RHEL 8 configures gcc it looks like we can leave the release machines alone for now. Longer term we may need to move a release machine to IBM Cloud for POWER8, or take a decision that we do not need a POWER8 release machine (we do not run tests on the release machines) and consolidate to a single RHEL 8 POWER9 release machine.

RHEL 8 is unlikely to remove support for POWER8, and the gcc 10 and 12 toolsets no longer get updates, which means it is highly unlikely that the default target/tuning will change for those compilers from power8.

We could also explicitly set -mcpu=power8 in the gyp configuration, but that would potentially affect other people building Node.js from source on other toolchains.

[sxa]

Based on #4405 (comment) it looks like we can leave the release machines alone for now.

Agreed.

We could also explicitly set -mcpu=power8 in the gyp configuration, but that would potentially affect other people building Node.js from source on other toolchains.

I'm not sure if we have any precedent for it but in theory we could set -mcpu=power8 at the end of the CC variable in select-compiler.sh if we didn't want to affect the build for others, but like you I'm inclined to believe it's not necessary.

I'll also note that the risk here is if the release builds on POWER9 fail when executed on POWER8 so

• Adding in POWER8 (in compat mode) should give us protection against any changes in V8's codegen (low risk I'd suggest unless someone tries to backport a POWER9 instruction)
• It does not give us any protection in the case that gcc starts outputting POWER9 code, although for the reasons you suggest that's unlikely to be an issue as long as we build on RHEL8 and the toolsets aren't likely to be updated to do that. If we wanted to protect against that then we should look at putting -mcpu=power8 /somewhere/ but that doesn't sound critical right now.

[richardlau]

I've added two new IBM Cloud hosted RHEL 8 POWER 8 compatibility mode VMs

• test-ibm-rhel8-ppc64_le-2
• test-ibm-rhel8-ppc64_le-3

I've taken the rhel8-ppc64le off the OSUOSL hosted VMs that are now POWER9.

Refs: #4407