Repository navigation
test(evals): accept handleAuthkitProxy and app-owned AuthKitProvider modules - #262
Conversation
…modules handleAuthkitHeaders is a deprecated alias of handleAuthkitProxy in authkit-nextjs 4.4. AuthKitProvider can live in a provider module, not only src/main.tsx or src/index.tsx.
|
| checks.push( | ||
| await this.fileGrader.checkFileWithPattern( | ||
| 'src/{main,index}.tsx', | ||
| 'src/**/*.tsx', |
There was a problem hiding this comment.
Unused provider passes grading
If src/auth/Providers.tsx imports and renders AuthKitProvider and reads the environment config, this search accepts it even when the entry file never mounts it. A component using useAuth and a successful build can satisfy the remaining checks, so an app without the provider can receive a passing grade. Check that the matched provider is connected to the entry file.
Prompt To Fix With AI
This is a comment left during a code review.
Path: tests/evals/graders/react.grader.ts
Line: 32
Comment:
**Unused provider passes grading**
If `src/auth/Providers.tsx` imports and renders `AuthKitProvider` and reads the environment config, this search accepts it even when the entry file never mounts it. A component using `useAuth` and a successful build can satisfy the remaining checks, so an app without the provider can receive a passing grade. Check that the matched provider is connected to the entry file.
---
For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.- react: AuthKitProvider must be in the entry file or a module imported elsewhere in src/; client ID env may be read in any src file - nextjs: match handleAuthkitProxy/handleAuthkitHeaders by name so aliased imports and spaced calls still pass
What
Two eval graders were rejecting valid integrations. Both false negatives came from the Opus 5.5 eval run in #261:
handleAuthkitProxy(as well ashandleAuthkitHeaders(. In@workos-inc/authkit-nextjs4.4.0,handleAuthkitHeadersis a@deprecatedalias ofhandleAuthkitProxy.AuthKitProvider(and the env-based config next to it) anywhere insrc/**/*.tsx, not only insrc/{main,index}.tsx. Wrapping the provider in an app-owned provider module is valid, and the React skill doesn't require a specific file.The companion skills change (workos/skills#53) updates the Next.js reference to recommend
handleAuthkitProxy, so this grader change is needed before that lands.Verification
bunx vitest run tests/evals: 101 passedclaude-opus-4-5-20251101andclaude-opus-5-5(requires chore(deps): bump claude-agent-sdk to 0.3.295 for Opus 5.5 support #261 for the Opus 5.5 SDK).