Repository navigation
gh-156049: Add support for building with HWASAN #156721
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
7b1be6e
78926b4
f63d3cc
baba1c0
006662c
5f3b65e
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -1022,6 +1022,19 @@ Debug options | |
|
|
||
| .. versionadded:: 3.6 | ||
|
|
||
| .. option:: --with-hwaddress-sanitizer | ||
|
|
||
| Enable HWAddressSanitizer memory error detector, ``hwasan`` (default is no). | ||
| Note that on x86-64 this uses `page aliasing | ||
| <https://clang.llvm.org/docs/HardwareAssistedAddressSanitizerDesign.html#supported-architectures>`_, | ||
| which only tags heap allocations and is unsafe for programs that ``fork()``, | ||
| including much of the test suite. | ||
|
StanFromIreland marked this conversation as resolved.
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Do you know which GCC and clang versions added support for this sanitizer?
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. LLVM (the reference implementation) supports it on AArch64, and additionally on x86-64 since LLVM 13 with page aliasing. GCC supports it only on AArch64, since GCC 11. |
||
| See the `LLVM HWASan design documentation | ||
| <https://clang.llvm.org/docs/HardwareAssistedAddressSanitizerDesign.html>`_ | ||
| for more information. | ||
|
|
||
| .. versionadded:: next | ||
|
|
||
| .. option:: --with-memory-sanitizer | ||
|
|
||
| Enable MemorySanitizer allocation error detector, ``msan`` (default is no). | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -448,11 +448,12 @@ def skip_if_buildbot(reason=None): | |
| isbuildbot = False | ||
| return unittest.skipIf(isbuildbot, reason) | ||
|
|
||
| def check_sanitizer(*, address=False, memory=False, ub=False, thread=False, | ||
| function=True): | ||
| def check_sanitizer(*, address=False, hwaddress=False, memory=False, ub=False, | ||
| thread=False, function=True): | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. That's a long list of sanitizers! 😃 It's cool that we supported all of them! |
||
| """Returns True if Python is compiled with sanitizer support""" | ||
| if not (address or memory or ub or thread): | ||
| raise ValueError('At least one of address, memory, ub or thread must be True') | ||
| if not (address or hwaddress or memory or ub or thread): | ||
| raise ValueError('At least one of address, hwaddress, memory, ub or ' | ||
| 'thread must be True') | ||
|
|
||
|
|
||
| cflags = sysconfig.get_config_var('CFLAGS') or '' | ||
|
|
@@ -461,9 +462,14 @@ def check_sanitizer(*, address=False, memory=False, ub=False, thread=False, | |
| '-fsanitize=memory' in cflags or | ||
| '--with-memory-sanitizer' in config_args | ||
| ) | ||
| hwaddress_sanitizer = ( | ||
| '-fsanitize=hwaddress' in cflags or | ||
| '--with-hwaddress-sanitizer' in config_args | ||
| ) | ||
| address_sanitizer = ( | ||
| '-fsanitize=address' in cflags or | ||
| '--with-address-sanitizer' in config_args | ||
| '--with-address-sanitizer' in config_args or | ||
| hwaddress_sanitizer | ||
| ) | ||
| ub_sanitizer = ( | ||
| '-fsanitize=undefined' in cflags or | ||
|
|
@@ -479,6 +485,7 @@ def check_sanitizer(*, address=False, memory=False, ub=False, thread=False, | |
| return ( | ||
| (memory and memory_sanitizer) or | ||
| (address and address_sanitizer) or | ||
| (hwaddress and hwaddress_sanitizer) or | ||
| (ub and ub_sanitizer) or | ||
| (thread and thread_sanitizer) or | ||
| (function and function_sanitizer) | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,2 @@ | ||
| Add :option:`--with-hwaddress-sanitizer` to build with `HWAddressSanitizer | ||
| <https://clang.llvm.org/docs/HardwareAssistedAddressSanitizerDesign.html>`_. |
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -3531,6 +3531,34 @@ with_pymalloc="no" | |
| ], | ||
| [AC_MSG_RESULT([no])]) | ||
|
|
||
| AC_MSG_CHECKING([for --with-hwaddress-sanitizer]) | ||
| AC_ARG_WITH( | ||
| [hwaddress_sanitizer], | ||
| [AS_HELP_STRING( | ||
| [--with-hwaddress-sanitizer], | ||
| [enable HWAddressSanitizer memory error detector, 'hwasan' (default is no)] | ||
| )], | ||
| [ | ||
| AC_MSG_RESULT([$withval]) | ||
| hwasan_flags="-fsanitize=hwaddress" | ||
| # x86-64 lacks address tagging, so HWASan needs the page aliasing mode there. | ||
| # See gh-156049 and | ||
| # https://clang.llvm.org/docs/HardwareAssistedAddressSanitizerDesign.html#supported-architectures | ||
| AS_CASE([$host_cpu], | ||
| [x86_64|amd64], [hwasan_flags="$hwasan_flags -fsanitize-hwaddress-experimental-aliasing"] | ||
| ) | ||
| AX_CHECK_COMPILE_FLAG([$hwasan_flags],[ | ||
| BASECFLAGS="$hwasan_flags -fno-omit-frame-pointer $BASECFLAGS" | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. What's the rationale for adding
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. It isn't required, it's just there for usable stack traces in the reports as the sanitizer runtime uses its frame-pointer-based unwinder for the allocation/free stacks. The
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Oh ok, I didn't notice that it's also used by other sanitizers. Since PEP 831 in Python 3.15, Python is built with |
||
| LDFLAGS="$hwasan_flags $LDFLAGS" | ||
| ],[AC_MSG_ERROR([The selected compiler doesn't support hardware address sanitizer])]) | ||
| # HWASan works by controlling memory allocation, our own malloc interferes, | ||
| # so disable it by default, but allow --with-pymalloc to override. | ||
| if test -z "$with_pymalloc"; then | ||
| with_pymalloc="no" | ||
| fi | ||
| ], | ||
| [AC_MSG_RESULT([no])]) | ||
|
|
||
| AC_MSG_CHECKING([for --with-memory-sanitizer]) | ||
| AC_ARG_WITH( | ||
| [memory_sanitizer], | ||
|
|
||
Uh oh!
There was an error while loading. Please reload this page.