Skip to content

fix(mods): jev-model-router leaves a bare slash command unclassified - #983

Open
meesp123 wants to merge 1 commit into
davila7:mainfrom
meesp123:jev-router-bare-command
Open

meesp123 wants to merge 1 commit into
davila7:mainfrom
meesp123:jev-router-bare-command

Conversation

@meesp123

@meesp123 meesp123 commented Sep 24, 2026

Copy link
Copy Markdown

Before: a prompt that is only a slash command (/simplify) was sent to the decision model like any other. The model sees just the command's name, never the skill or command it runs, so it judged the name. On TypeSafe, three calls each:

Prompt Jev's answer
/simplify fast 0.80, effort 0.5
/run fast 0.88, effort 0.1
/github fast 0.67, effort 0.1
/code-review balanced 0.96, effort 1.7
/security-review deep 0.91, effort 1.5

On Opus 5.5 a fast answer with effort 0.1 moves the main loop to low, so a multi-step skill started with a bare command ran at low effort.

After: a prompt that is a slash command and nothing else is not classified. The log says a command with nothing after it; leaving the turn alone, its turn keeps the session's model and effort, and a null decision is put so an earlier prompt's cannot reach that turn. With text after the name (/code-review high, /simplify the retry loop) the prompt is classified as before. The check is /^\/[^\s/]+$/ on the trimmed text, so a path like /tmp/log.txt is still classified.

Live (claude -p, Claude Code 2.1.280, TypeSafe backend, Opus 5.5):

Prompt main this branch
/about-user jev: tier fast (0.81) · effort 0.1 → low, then main loop → effort low (twice) a command with nothing after it; leaving the turn alone, then main loop: no decision
/about-user who am I classified classified (jev: tier fast (0.95), main loop → effort low)

Checks

  • cd cli-tool/components/mods && npx -y -p typescript@5 tsc -p tsconfig.json: clean
  • bun test productivity/jev-model-router/tests: 42 pass (1 new: bare commands, and prompts that are not)
  • claude plugin validate cli-tool/components/mods/productivity/jev-model-router: passed
  • Hook-level tests under claude plugin test (kept locally until jev-model-router: run the tests on the built-in testing kit instead of bun:test #958 lands): a bare command sends nothing and its turn keeps high; with text after it, it is sent and routed; a bare command after a routed prompt keeps the session's effort. Both pass on this branch and fail on main.
  • component-reviewer: no blocking issues. Taken from it: the README paragraph moved after the failure list (it is a policy, not a failure), and the doc comment notes that a one-segment path alone (/etc) matches too.

Version not bumped: several open PRs touch this mod. Catalog not regenerated (external PR).

Related

A prompt that is a slash command and nothing else (`/simplify`) gives the
decision model only the command's name, never the skill or command it
runs. Measured on TypeSafe, three calls each: /simplify, /run and /github
came back fast with effort 0.1 to 0.5, so a multi-step skill ran at low
effort, while /code-review and /security-review came back balanced and
deep. Such a prompt is now not sent; its turn keeps the session's model
and effort, and a null decision is put so an earlier prompt's cannot
reach it. With text after the name the prompt is classified as before.
@github-actions github-actions Bot added the review-pending Component PR awaiting maintainer review label Sep 24, 2026
@github-actions

Copy link
Copy Markdown
Contributor

👋 Thanks for contributing, @meesp123!

This PR touches cli-tool/components/** and has been marked review-pending.

What happens next

  1. 🤖 Automated security audit runs and posts results on this PR.
  2. 👀 Maintainer review — a human reviewer validates the component with the component-reviewer agent (format, naming, security, clarity).
  3. Merge — once approved, your PR is merged to main.
  4. 📦 Catalog regeneration — the component catalog is rebuilt automatically.
  5. 🚀 Live on aitmpl.com — your component appears on the website after deploy.

⚠️ Do not commit generated catalog files

Please make sure your PR does not include docs/components.json or anything under dashboard/public/
(components.json, counts.json, search-index.json, components/*.json, component-content/**).
Those files are regenerated automatically by scripts/generate_components_json.py after merge, and committing them causes merge conflicts.
If you already ran the script, revert them with: git checkout origin/main -- docs/components.json dashboard/public/

While you wait

  • Check the Security Audit comment below for any issues to fix.
  • Make sure your component follows the contribution guide.

This is an automated message. No action is required from you right now — a maintainer will review soon.

@greptile-apps

greptile-apps Bot commented Sep 24, 2026

Copy link
Copy Markdown
Contributor

RetriggerConfidence Score: 4/5

The PR appears safe to merge, though a hook-level regression test would better protect its central behavior.

Findings

  1. P2 Hook behavior remains untested

Summary

The router now leaves bare slash commands at the session’s model and effort while continuing to classify commands with arguments.

  • Adds a predicate and an early hook path that records a null pending decision.
  • Documents the policy and adds predicate tests; the hook-level behavior remains untested in the checked-in suite.
Diagram
%%{init: {'theme': 'neutral'}}%%
flowchart LR
  A[prompt.submit] --> B{Bare slash command?}
  B -- Yes --> C[Put null decision]
  B -- No --> D[Classify and put decision]
  C --> E[Continue prompt]
  D --> E
  E --> F[First turn.step takes pending decision]
Loading

Reviews (1) · Last reviewed commit: "fix(mods): jev-model-router leaves a bar..."

expect(routing.model).toBeNull()
})

test('a slash command alone is not a task; with text after it, it is', () => {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Hook behavior remains untested

This test checks only whether bareCommand() matches a string. It does not exercise the new prompt.submit branch or the later turn.step that reads its pending decision. A regression that still classifies a bare command, or applies a decision to the wrong turn, could therefore pass the checked-in tests. Please add a hook-level test for that flow.

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Written as hook-level tests: a bare /simplify sends nothing and its turn keeps the effort it came with (high), then /simplify the retry loop in payments.ts is sent and routed (low); and a bare command right after a routed prompt keeps the session's effort, so the earlier decision does not reach its turn. 2 of 2 pass on this branch's head and both fail on main. They run locally with claude plugin test (the engine's claude-code/testing kit) and are not in this PR: the kit loads a plugin with its manifest defaults only, so the backend path needs a copy of the mod with a key patched into plugin.json, and this suite still runs under bun test until #958 lands. I can add them once #958 is in, if you want them in the repo.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks—that addresses the behavioral gap: the hook-level checks cover suppression of a bare command, prevention of a prior decision leaking into its turn, and routing commands with trailing text. Given the current test-kit limitation and dependency on #958, I’m fine with keeping them out of this PR. Please add them to the repository once #958 lands; no further change is needed here.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

review-pending Component PR awaiting maintainer review

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant