Conversation
A prompt that is a slash command and nothing else (`/simplify`) gives the decision model only the command's name, never the skill or command it runs. Measured on TypeSafe, three calls each: /simplify, /run and /github came back fast with effort 0.1 to 0.5, so a multi-step skill ran at low effort, while /code-review and /security-review came back balanced and deep. Such a prompt is now not sent; its turn keeps the session's model and effort, and a null decision is put so an earlier prompt's cannot reach it. With text after the name the prompt is classified as before.
👋 Thanks for contributing, @meesp123!This PR touches What happens next
|
|
| expect(routing.model).toBeNull() | ||
| }) | ||
|
|
||
| test('a slash command alone is not a task; with text after it, it is', () => { |
There was a problem hiding this comment.
Hook behavior remains untested
This test checks only whether bareCommand() matches a string. It does not exercise the new prompt.submit branch or the later turn.step that reads its pending decision. A regression that still classifies a bare command, or applies a decision to the wrong turn, could therefore pass the checked-in tests. Please add a hook-level test for that flow.
Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!
There was a problem hiding this comment.
Written as hook-level tests: a bare /simplify sends nothing and its turn keeps the effort it came with (high), then /simplify the retry loop in payments.ts is sent and routed (low); and a bare command right after a routed prompt keeps the session's effort, so the earlier decision does not reach its turn. 2 of 2 pass on this branch's head and both fail on main. They run locally with claude plugin test (the engine's claude-code/testing kit) and are not in this PR: the kit loads a plugin with its manifest defaults only, so the backend path needs a copy of the mod with a key patched into plugin.json, and this suite still runs under bun test until #958 lands. I can add them once #958 is in, if you want them in the repo.
There was a problem hiding this comment.
Thanks—that addresses the behavioral gap: the hook-level checks cover suppression of a bare command, prevention of a prior decision leaking into its turn, and routing commands with trailing text. Given the current test-kit limitation and dependency on #958, I’m fine with keeping them out of this PR. Please add them to the repository once #958 lands; no further change is needed here.
Before: a prompt that is only a slash command (
/simplify) was sent to the decision model like any other. The model sees just the command's name, never the skill or command it runs, so it judged the name. On TypeSafe, three calls each:/simplify/run/github/code-review/security-reviewOn Opus 5.5 a fast answer with effort 0.1 moves the main loop to
low, so a multi-step skill started with a bare command ran at low effort.After: a prompt that is a slash command and nothing else is not classified. The log says
a command with nothing after it; leaving the turn alone, its turn keeps the session's model and effort, and a null decision is put so an earlier prompt's cannot reach that turn. With text after the name (/code-review high,/simplify the retry loop) the prompt is classified as before. The check is/^\/[^\s/]+$/on the trimmed text, so a path like/tmp/log.txtis still classified.Live (
claude -p, Claude Code 2.1.280, TypeSafe backend, Opus 5.5):main/about-userjev: tier fast (0.81) · effort 0.1 → low, thenmain loop → effort low(twice)a command with nothing after it; leaving the turn alone, thenmain loop: no decision/about-user who am Ijev: tier fast (0.95),main loop → effort low)Checks
cd cli-tool/components/mods && npx -y -p typescript@5 tsc -p tsconfig.json: cleanbun test productivity/jev-model-router/tests: 42 pass (1 new: bare commands, and prompts that are not)claude plugin validate cli-tool/components/mods/productivity/jev-model-router: passedclaude plugin test(kept locally until jev-model-router: run the tests on the built-in testing kit instead of bun:test #958 lands): a bare command sends nothing and its turn keepshigh; with text after it, it is sent and routed; a bare command after a routed prompt keeps the session's effort. Both pass on this branch and fail onmain./etc) matches too.Version not bumped: several open PRs touch this mod. Catalog not regenerated (external PR).
Related
maincommit. fix(mods): jev-model-router applies a prompt typed mid-turn and skips text nobody typed #968 restructuresprompt.submit; whichever lands second, I'll rebase.