Skip to content

fix(kvflash): tests that share a KvFlashPager with luce_common get its CUDA define - #1027

Open
dusterbloom wants to merge 2 commits into
Luce-Org:mainfrom
dusterbloom:fix/qwen4exp-test-pager-define
Open

dusterbloom wants to merge 2 commits into
Luce-Org:mainfrom
dusterbloom:fix/qwen4exp-test-pager-define

Conversation

@dusterbloom

@dusterbloom dusterbloom commented Nov 17, 2026 •

Copy link
Copy Markdown
Collaborator

What

On CUDA builds, test_qwen4exp_prefix_cache segfaults on main in its KVFlash case. The crash comes right after the case's expected refusal, [qwen4exp][kvflash] a 385-row forward exceeds the 384 rows a 512-slot pool keeps resident:

#4  cuEventRecord () from libcuda.so.1
#6  cudaEventRecord () from libcudart.so.12
#7  luce::common::qwen4exp_kvflash_reserve(Qwen4ExpCache&, int, int)
#8  kvflash(ggml_backend*)

Cause: the two sides build KvFlashPager differently.

  • KvFlashPager (src/common/kvflash_pager.h) adds its async-DMA members only when LUCE_BACKEND_CUDA, LUCE_BACKEND_HIP or GGML_USE_HIP is defined.
  • luce_common gets LUCE_BACKEND_CUDA=1 as a PRIVATE definition; the test target does not. So the test compiles a smaller pager than the one luce_common code reads.
  • The test constructs the pager (std::make_unique<KvFlashPager>(), attach). qwen4exp_kvflash_reserve, compiled in luce_common, then reads compute_stream_ past the end of the test's object, gets garbage, and records an event on it.

HIP builds are not affected: ggml propagates GGML_USE_HIP to every consumer, so both sides build the DMA layout.

Fix: on CUDA, give the define to the two test targets that construct a KvFlashPager and pass it to luce_common code:

  • test_qwen4exp_prefix_cache
  • test_kvflash

The change is build config only.

Evidence (RTX 3090, sm_86, CUDA 12.0)

tree test_qwen4exp_prefix_cache
main 4af911b74 segfault (rc 139) in kvflash() after the 385-row refusal
this PR rc 0, all 9 sections PASS, including the KVFlash case

Not run:

  • test_kvflash, which needs a qwen35 GGUF; it is built only.
  • HIP, where the change does nothing because it is CUDA-only.

Considered: making kvflash_pager.h also check GGML_USE_CUDA, which ggml already propagates, would fix every consumer at once. But it would also move the header-only pager tests onto the DMA path, so it belongs in a separate change.

View guided diff

…s CUDA define

KvFlashPager adds its async-DMA members only under LUCE_BACKEND_CUDA,
LUCE_BACKEND_HIP or GGML_USE_HIP. luce_common sets LUCE_BACKEND_CUDA=1
PRIVATE, so on CUDA test_qwen4exp_prefix_cache built a smaller pager than
the one luce_common code reads: qwen4exp_kvflash_reserve read
compute_stream_ past the test's object and segfaulted in cudaEventRecord.
test_kvflash has the same pattern. HIP builds already match through
ggml's GGML_USE_HIP.

@davide221 davide221 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code review

No issues found. Checked for bugs and logic errors in the changed code.

Reviewed 10098386 · 0 candidate issues checked, kept at ≥70% confidence · add the code-review label to review again

@dusterbloom

Copy link
Copy Markdown
Collaborator Author

The two AMD jobs never ran: the Strix Halo job timed out after 24 h in the queue and the R9700 job is still queued. The lucebox3 runner has taken no jobs since 11-17.

Device evidence in the meantime. #1028's head 6940669 contains this commit, and on it test_qwen4exp_prefix_cache built for gfx1151;gfx1201 and passed:

  • lucebox2, gfx1201 target + gfx1151 experts with Qwen3.8-Flash UD-Q4_K_XL: 14/14 PASS, rc 0.
  • luceboxcina2, device sections on gfx1201 alone and on gfx1151 alone: 13 PASS each. Given a model path that does not exist, so each run ends on the expected b.init() failure.

Once the runner is back, the two AMD jobs can be re-run; or this can merge on the runs above.

server/CMakeLists.txt: main's Luce-Org#1031 added a comment above add_test(qwen4exp_prefix_cache) where this branch adds the
LUCE_BACKEND_CUDA define; keep both.
dusterbloom added a commit to dusterbloom/lucebox-hub that referenced this pull request Nov 18, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants