e2e: fix pod resolution, NUMA kernel restore and swap detection - #791
Conversation
klihub
changed the title
askervin
left a comment
askervin
left a comment
There was a problem hiding this comment.
We have some other tests, too, that change system configuration in such a way that it may lead failures in next tests (modify kernel command line, for instance).
Therefore I wonder if we should actually have some more generic options to control:
- whether or not we want a test to try recovering the node after failed test execution, and/or
- whether or not we want execution of multiple tests stop on first error
In that perspective, some sort of global debug_first_fail=1 could perhaps imply keep_numa_disabled=1 and possibly other "don't restore on failure" variables.
But this doesn't need to be part of this PR. I'm just thinking how we could manage keep_numa_disabled-like variables...
b8d62fc to
e9e8929
Compare
The num-disabled tests boot the node with a kernel without NUMA
support, run the tests, then boot back to a NUMA-enabled kernel.
A failure in the tests leaves the node running with the special
test kernel, causing subsequent tests which expect NUMA support
to fail. So restore the kernel from an EXIT trap instead.
To debug the NUMA-less tests themselves use keep_numa_disabled=1
like this
keep_numa_disabled=1 ./run_tests.sh \
policies.test-suite/balloons/n4c16/test30-numa-disabled
Assisted-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Krisztian Litkey <krisztian.litkey@intel.com>
Print a single pod in vm-wait-pod-regexp, and never one which is
terminating. Otherwise port forwarding got two names while one pod was
terminating and another starting, and kubectl refused:
error: TYPE/NAME and list of ports are required for port-forward
Assisted-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Krisztian Litkey <krisztian.litkey@intel.com>
Use ansible_facts to disable swap during provisioning. Signed-off-by: Krisztian Litkey <krisztian.litkey@intel.com>
e9e8929 to
01d7fcd
Compare
@askervin That's a good point. Let's take a closer look at that and then extend/clean up the limited concept this PR introduced to be more generally usable, but let's do it in a separate dedicated PR. |
2 participants
Three fixes to the test harness which the rest of this stack builds on.
terminating. Port forwarding to the plugin then got two names and kubectl
refused with "TYPE/NAME and list of ports are required for port-forward".
Print a single pod, and never a terminating one.
compiled out and boot it back by calling enable-numa as their last statement,
so a failure anywhere in between left the node without NUMA and every later
test saw a single node and failed too. Restore the kernel from an EXIT trap
instead, the way the other tests with state to put back already do, and let
enable-numa return without doing anything when NUMA is already enabled.
keep_numa_disabled=1 leaves the node NUMA-less for looking into a failure.
ansible_swaptotal_mb fact, which is not there with injected top-level facts
turned off. Take it from ansible_facts instead.