Skip to content

Tests always have zero exit status #497

Closed

Description

@christoph-conrads

The Makefile-based build runs all tests and then shows a test summary. In CMake, all tests are run, the test summary script is called but with the default CMake settings, the testsuite will never fail and never show the test summary because

  • failing tests are detected by non-zero exit status and
  • test output is suppressed by default.

As far as I can see, all tests always exit with a zero status even if there was a problem. This breaks testing and this behavior makes it impossible to do any sane scripting on UNIX (think of distribution packers that run make test && make install; this will always succeed). This behavior makes it also impossible to tell apart critical test failures (e.g., not recognizing negative matrix dimensions) from spurious test failures (e.g., convergence failures for numerically hard test problems).

Activity

  1. ACSimon33 commented on Oct 19, 2026

    @ACSimon33
    Collaborator

    This was fixed by #1371, which makes test failures fatal in the CMake build: the LAPACK_Test_Summary test now runs the new lapack_testing.py with --fail-on-error, so any failure or error reported by the drivers makes ctest (and make test) fail, and their fatal error paths now exit with STOP 1. The fix will be part of the next release (3.13.0). Closing as completed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions