GROMACS 2026.3 Port and Benchmark on Hygon Z100

1. Summary

2. Performance results (idle node; three runs per case)

Method: gmx_bench.slurm ran each backend/system combination three times. The options -notunepme -resetstep N/2 keep the timed region consistent. Results are reported as mean ± standard deviation; every case had a standard deviation below 2%. Smaller systems showed larger speedups. Their GPUs are less fully utilized, so the lower startup and scheduling overhead of the native HIP kernels has a greater effect.

3. Why the waterbox8 result changed

The previous report measured 15.31 vs 41.22 ns/day for waterbox8 NPT, making HIP appear 2.7× slower. That result was the only evidence then suggesting lower HIP performance. The idle-node retest produced the following results:

Cause: CPU contention on the node substantially affected the HIP backend. On the same contended node, HIP throughput fell from 148 to 9.4 ns/day (15× slower), while the CUDA-translated build remained near 66.8 ns/day. The earlier waterbox8 HIP result was likewise measured under contention. The resulting hypotheses about slow waterbox8 NPT performance—including virial rollback, DPP reduction, and register occupancy—were therefore based on contaminated measurements. Section 4 reports the retests of those optimization hypotheses.

3. Correctness verification

For each case, the principal energy terms in the Energies section of run.log (LJ SR, Coulomb SR, potential, and total energy) agreed within approximately 0.01–0.05%. Temperatures stayed near the 300 K target, pressure remained within the expected ensemble fluctuations, and no NaN or Inf values appeared. The small differences are consistent with trajectory divergence caused by floating-point operations being accumulated in a different order; they do not indicate a physical error.

100 ps waterbox8 NPT run (50,000 steps): total energy −5.718 × 10⁵ kJ/mol, temperature 299.88 K, pressure −1.8 bar, no drift or abnormal termination, and throughput of 67.5 ns/day, consistent with the benchmark.

4. Build issues and performance analysis

4.1 Compiler configuration failure and its effect

Architecture-probe mismatch (gfx90a vs gfx906): gmxManageHipccConfig.cmake used --offload-arch=gfx90a to test whether hipcc supported several optimization flags. DTK 26.04 does not provide a gfx90a device library, so every probe failed and all optimization flags were silently discarded. Performance fell to about 17 ns/day, with no reported error. The fix was to probe gfx906, add -mcpu=gfx906, and remove the unsupported -fno-gpu-rdc and -fno-slp-vectorize checks. This was the build change with the largest performance impact.

4.2 Performance optimization assumptions and verification results

5. Benchmark preconditions

6. Build procedure and reproduction steps

Toolchain: DTK 26.04 (HIP 6.3.26113 / Clang 17), GCC 12.2.0, CMake 3.28.6, and OpenMPI 4.1.5.

# 0. Restore cmake_minimum_required to 3.28 (revert any downgrade to 3.25)
cd $SRC
sed -i 's/cmake_minimum_required(VERSION 3\.25)/cmake_minimum_required(VERSION 3.28)/g' CMakeLists.txt
find . -name CMakeLists.txt -exec sed -i 's/cmake_minimum_required *(VERSION 3\.25)/cmake_minimum_required(VERSION 3.28)/g' {} \;

# 1. gfx906 architecture-probe patch (critical; see §4.1)
cd $SRC/cmake
sed -i 's/--offload-arch=gfx90a/--offload-arch=gfx906/g' gmxManageHipccConfig.cmake
sed -i 's/gmx_hip_check_single_flag("-fno-gpu-rdc")/# REMOVED for gfx906: gmx_hip_check_single_flag("-fno-gpu-rdc")/' gmxManageHipccConfig.cmake
sed -i 's/gmx_hip_check_single_flag("-fno-slp-vectorize")/# REMOVED for gfx906: gmx_hip_check_single_flag("-fno-slp-vectorize")/' gmxManageHipccConfig.cmake
sed -i '/gmx_hip_check_single_flag("-ffast-math")/a\    gmx_hip_check_single_flag("-mcpu=gfx906")' gmxManageHipccConfig.cmake

# 2. Environment
module purge
module load mpi/openmpi/gcc-9.3.0/4.1.5
source <DTK_ROOT>/env.sh
export PATH=<GCC_ROOT>/bin:$PATH
export LD_LIBRARY_PATH=<GCC_ROOT>/lib64:$LD_LIBRARY_PATH
export amd_comgr_DIR=<DTK_ROOT>/lib64/cmake/amd_comgr

# 3. HIP Clang wrapper (adds AMD GPU optimization flags)
cat > $BUILD/hip_wrapper.sh << 'EOF'
#!/bin/bash
exec <DTK_ROOT>/llvm/bin/clang++ \
  --gcc-toolchain=<GCC_ROOT> \
  -mllvm -amdgpu-early-inline-all=true \
  -mllvm -amdgpu-function-calls=false \
  "$@"
EOF
chmod +x $BUILD/hip_wrapper.sh

# 4. Configure and build with CMake
$CMAKE $SRC \
  -DGMX_BUILD_OWN_FFTW=ON -DGMX_GPU=HIP -DGMX_MPI=ON \
  -DCMAKE_C_COMPILER=$GCC_HOME/bin/gcc -DCMAKE_CXX_COMPILER=$GCC_HOME/bin/g++ \
  -DCMAKE_HIP_COMPILER=$BUILD/hip_wrapper.sh \
  -DCMAKE_PREFIX_PATH=<DTK_ROOT> \
  -Damd_comgr_DIR=<DTK_ROOT>/lib64/cmake/amd_comgr \
  -DGMX_SIMD=AVX2_256 -DGMX_GPU_FFT_LIBRARY=rocFFT \
  -DGMX_GPU_NB_DISABLE_CLUSTER_PAIR_SPLIT=ON \
  -DGMX_GPU_NB_NUM_CLUSTER_PER_CELL_X=1 -DGMX_GPU_NB_NUM_CLUSTER_PER_CELL_Y=1 -DGMX_GPU_NB_NUM_CLUSTER_PER_CELL_Z=1 \
  -DGMX_HIP_TARGET_ARCH=gfx906 -DAMDGPU_TARGETS=gfx906 -DGPU_TARGETS=gfx906 \
  -DCMAKE_BUILD_TYPE=Release
make -j8

Full build script: <USER_HOME>/scripts/gromacs_cmake28_build.slurm. Submit it with sbatch to reproduce the build.

7. Build artifacts

The validated build is located at <USER_HOME>/softwares/gromacs/build_cmake28/:

Run the build with a DTK runtime compatible with the one used to compile it:

module purge
module load mpi/openmpi/gcc-9.3.0/4.1.5
source <DTK_ROOT>/env.sh
export LD_LIBRARY_PATH=<GCC_ROOT>/lib64:$LD_LIBRARY_PATH

/path/to/build_cmake28/bin/gmx_mpi mdrun -s case.tpr -nb gpu -pme gpu ...

GROMACS 2026.3 writes version 138 .tpr files, which the cluster's prebuilt 2023.2 cannot read. Version 2026.3 can read older .tpr files. For cross-version comparisons, use input files generated by the corresponding GROMACS version.

8. Feature changes (GROMACS 2023.2—2026.3)

• Native HIP backend: Version 2026.3 adds native AMD/HIP support, allowing GPU kernels to use the upstream HIP path instead of a translation layer or SYCL.
• Force fields and water models: AMBER with OPC/OPC3 water models (2026), and per-residue CMAP types (2025).
• Machine-learning potentials: PyTorch interface, link atoms, pair-list input, and EMLE electrostatic embedding.
• Enhanced sampling: Native Colvars (2024) and the PLUMED interface (MDModules).
• Performance features: HMR (hydrogen-mass repartitioning), AWH improvements, H5MD trajectory output, and NVSHMEM halo exchange.