A single parameterized test is failing for the csingle and cdouble cases:

     @pytest.mark.parametrize('dtype', [single, double, csingle, cdouble])
      def test_types(self, dtype):
          x = np.array([[1, 0.5], [0.5, 1]], dtype=dtype)
  >       assert_equal(np.linalg.det(x).dtype, dtype)
...
RuntimeWarning: divide by zero encountered in det

Changing to draft to indicate that this is stuck until we can figure out what is going wrong

In case the logs go away before this gets revisited: the (failing) meson build here reports

clang 15.0.0 "Apple clang version 15.0.0 (clang-1500.3.9.4)"

while the (successful) cirrus CI one reports clang 16.0.0 "Apple clang version 16.0.0 (clang-1600.0.26.4)"

I hacked the call to zgetrf to reset the FPE registers on macos-arm64, since I couldn't actually fix the problem. In my defense most people with macos-arm64 machines will be using the Accelerate LAPACK wheel and not the OpenBLAS one, since they will have macos-14+ (released Sep 2023).

I wonder why scipy does not see this

Edit: scipy, if I am not mistaken, does not check FPE registers?

The goal of this PR is to build macos-arm64 wheels on github, and with the hacky fix it is passing. The pypy failure is unrelated.

If you can tell me what to look out for, I may find time eventually, but please don't expect me to wade through numpy logs. I take it you are seeing fpe exception registers getting set without an exception actually being raised, i.e. something like an underflow or a quiet NaN that depends on xcode/AppleClang version (so might be an Apple compiler bug) ?

depends on xcode/AppleClang

IIRC, it is using SME that has that problem, apparently it is part of the spec. See #29223.

@martin-frbg sorry for the noise, I will try to whittle it down to a minimal reproducer in C. The general idea is that this sets FPE error registers in the call to zgetrf when I use a NumPy built locally or with github CI on macOS-arm64, but does not when build on cirrusCI.

x = np.array([[1, 0.5], [0.5, 1]], dtype=np.complex128)
d = np.linalg.det(x)

I don't know if I should open an issue on OpenBLAS, I am not sure they commit to correct handling of FPE registers. Here is the minimum reproducer I could create, starting in the OpenBLAS repo HEAD on macos-arm64

• Compile OpenBLAS with

  CFLAGS=' -arch arm64 -fvisibility=protected -Wno-uninitialized' \
  make BUFFERSIZE=20 DYNAMIC_ARCH=1 QUIET_MAKE=1 USE_OPENMP=0 NUM_THREADS=64 BINARY=64 INTERFACE64=1 TARGET=VORTEX
  

• Compile this test.c with clang test.c -L. -lopenblas -lm

  test.c

  #include <stdio.h>
  #include <fenv.h>
  #include <stdint.h>
  
  typedef struct { double r, i; } f2c_doublecomplex;
  typedef int64_t fortran_int;
  
  fortran_int
  zgetrf_(fortran_int *m, fortran_int *n,
              f2c_doublecomplex a[], fortran_int *lda,
              fortran_int ipiv[],
              fortran_int *info);
  
  int main(int argv, const char *argc[]) {
  
    f2c_doublecomplex a[] = {{1, 0}, {0.5, 0}, {0.5, 0}, {1, 0}};
    fortran_int m = 2, n=2, lda=2, info=1, ipiv[] = {0, 0};
    fortran_int ret = zgetrf_(&m, &n, a, &lda, ipiv, &info);
    int fpstatus = fetestexcept(FE_DIVBYZERO | FE_OVERFLOW | FE_UNDERFLOW | FE_INVALID);
    printf("zgetrf returned %lld, fpstatus=%d, info=%lld", ret, fpstatus, info);
    return 0;
  }
  

• Run the test with OPENBLAS_VERBOSE=2 ./a.out. On my macbook m2 it prints

  Core: neoversen1
  zgetrf returned 0, fpstatus=3, info=0
  

  where on x86_64 it prints fpstatus=0. I did try OPENBLAS_CORETYPE=ARMV8 but still got fpstatus=3.

@seiko2plus pointed out we need to add -ftrapping-math when using clang. Indeed, the reproducer above does not emit FP errors when using that flag 👍

rebased and updated to use OpenBLAS 0.3.30 compiled with -ftrapping-math

wheel building on macos is passing, now that OpenBLAS is built with -ftrapping-math. Apparently this is known behaviour, see #19479, #24060.

The PyPy failures are fixed on PyPy-HEAD, the wheel building uses the last release.

Would be nice if this ended up fixing the mac-os matmul warning heisenbug :).

Ping

I've marked this for backport, we may want it for the 3.14 build. Updating OpenBLAS is open for discussion.

I've marked this for backport, we may want it for the 3.14 build. Updating OpenBLAS is open for discussion.

I don't think we can do one without the other, because this PR was blocked on failure that was fixed by rebuilding a new version of OpenBLAS with -ftrapping-math.

I think it'd be fine to backport this PR in whole to 2.3.x. Or not and stay with Cirrus CI for 2.3.x, either way should work so I think you should make the call on that @charris.