Repository navigation
feat(store): model consumers, model keys and ctx_model_auth (Migration 161, W01-1) - #85
Conversation
…n 161) Wave W01-1 of the OpenAI-compatible API layer (RC-B, first migration of the chain; design/01 §3.3-§3.5 with the masterplan amendments K16, K20, K24, design/03 §3.1, design/04 M-F3, W06-7 F1). Schema only: nothing in ctx writes the new tables yet, no behaviour changes outside the database. - context_model_consumers (identity + policy: tenant, tenant_scope, kind, all_models, prompt_sensitivity, fair_weight, admission_lane, CIDR allowlist, allow_plaintext, active, peer binding) and context_model_keys (SHA-256 + hint only, no policy column); context_tenants.max_model_consumers = 64 for every tenant; model_cidrs_private() for the plaintext CHECK. - ctx_model_auth(key_hash, client_ip, secure): STABLE, read-only, six gates, one sentinel row for every refusal; result columns fixed, admission_lane and peer_instance_id last. - Deactivating a consumer revokes its live keys on every path; a consumer's tenant_id and a key's consumer_id are immutable (23514). - Whitelist audit of both tables into context_settings_audit (model_consumer / model_key) via audit_actor_meta(), metadata.tenant_id and metadata.consumer_id on every row; one trigger set per table. - idx_settings_audit_entity_tenant serves the tenant shapes of GET /api/audit (1 000 000 rows, tenant without rows: 1.47 s -> 0.33 ms). Gates, red first, then green: - TestModelAuthGateMatrix (13 rows): red against 160 (42P01 at the fixture; ctx_model_auth 42883 in TestModelAuthContract); overlays gate1..gate6 turn exactly their rows red (3/1/2/1/2/1); green 13/13 + reads_only (n_tup_upd unchanged, provolatile s). - I-K8 red with no-deactivate, identity pins red with no-identity, audit bookkeeping red with no-of-list (2 subtests), whitelist red with blacklist, FK/policy census red with extra-fk / key-policy. - Classification pin + per-column probe: red against 160 (36 mismatches), red with the migration but without the entries (new completeness check: 2 mismatches); green. - TestAuditListPlan: tenant shapes on the new index under custom and generic plans; tenant without rows removed 2400 rows by filter (160), 0 now. - Handler: tenant-admin reads its own model_consumer/model_key rows only; red against 160 (42P01) and against 161 without its audit triggers (rows: [] for both readers). - T07 gates red with test.sh at 59 tables, green at 61; idempotent second run with identical catalog; documented way back verified. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…61, W01-1 review 2 F-1) Main-context review 2 of W01-1, F-1 (Medium, executed): a key INSERT and a consumer deactivation did not wait for each other. The foreign key takes FOR KEY SHARE on the consumer row, the deactivating UPDATE FOR NO KEY UPDATE; the revoking trigger did not see the uncommitted key. Under READ COMMITTED a key minted during a deactivation stayed live at the inactive consumer and authenticated again after active = true; a key inserted for an already inactive consumer did the same. 161 is not landed yet, so the migration itself changes (no 162): - BEFORE INSERT trigger trg_model_keys_consumer_active / model_keys_consumer_active(): locks the consumer row FOR SHARE and refuses a missing or inactive consumer with 23514. A mint holding the lock makes the deactivation wait (its revoke then sees the key); a mint behind a running deactivation waits and fails. - Fixture TestModelAuthGateMatrix/consumer_inactive: key minted while the consumer is active, then deactivated with the revoking trigger disabled (gate 2 stays testable as defence in depth). - Header, docs/operations.md and docs/security.md: "revokes and refuses"; known limit named (a deactivation under explicit REPEATABLE READ does not revoke a key committed after its snapshot; ctx writes under READ COMMITTED). Way back and manifest carry the new function and trigger. Gates: - TestModelKeyInsertRacesDeactivation (READ COMMITTED, insert_first, deactivate_first, sequential insert into an inactive consumer): red against 161 without the guard (3/3: "consumer active=false with 1 live key(s)", "valid after reactivation", "err <nil>, want 23514"), red again with CTX_M161_OVERLAY=no-insert-guard, green with the guard (deactivate_first: insert fails 23514). - All 161 tests green; every overlay still red as before. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
|
No actionable comments were generated in the recent review. Walkthrough
Merge Risk:
|
There was a problem hiding this comment.
Note
Copilot was unable to complete this review at the selected tier due to model policy or plan restrictions.
🟡 Changes recommended
The change cannot be approved because the audit integration test’s EXPLAIN (ANALYZE) index/plan verification currently ignores a potential auCheckPlan error, allowing performance regressions (e.g., falling back to a sequential scan) to pass undetected.
1 open issue
What changed in this PR
This pull request introduces Migration 161, adding database primitives for authenticating tenant-scoped “model consumers” and their per-consumer “model keys,” alongside a stable ctx_model_auth(hash, ip, secure) function, lifecycle/immutability constraints, and audited change history. It also updates schema contract artifacts, integration tests, and documentation (security/ops/multi-tenancy/API) to reflect the new surfaces and expected behavior.
Changes:
- Add Migration 161 schema:
context_model_consumers,context_model_keys,ctx_model_auth(...), related triggers/guards, and tenant audit performance indexing. - Expand store/handler integration tests and audit classification/whitelist coverage for the new entities and invariants.
- Regenerate schema manifest and update operational + security documentation for the new auth/audit model.
| File | Summary |
|---|---|
test.sh |
Updates schema integrity fallback expectations for migration 161 table counts. |
go/migrations/161_model_consumers.sql |
Defines model consumer/key tables, ctx_model_auth(...), triggers/guards, audit projections, and a tenant-audit index. |
go/internal/store/model_consumers_migration_integration_test.go |
Adds integration gates for migration 161 behavior (auth semantics, triggers, audit whitelist, idempotence). |
go/internal/store/audit.go |
Aligns/document audit read-path behavior for tenant vs server-admin readers (including tenant index usage). |
go/internal/store/audit_integration_test.go |
Updates audit plan checks for the new tenant index and adds an EXPLAIN ANALYZE-based guard; moderate: plan check currently discards auCheckPlan error—test should fail on plan-check error to prevent silent index regressions. |
go/internal/store/audit_classification_test.go |
Extends audit classification/pinning to the new model tables and enforces audited-table classification completeness. |
go/internal/store/api_keys_audit_integration_test.go |
Expands the “one audit trigger set per table” gate to include model consumer/key audit triggers. |
go/internal/schemacontract/manifest.json |
Regenerates schema contract manifest to include migration 161 objects and updated counters. |
go/internal/handler/audit.go |
Updates/notes handler-level audit visibility/redaction considerations for new model entities. |
go/internal/handler/audit_integration_test.go |
Adds handler-level integration coverage for tenant filtering/redaction of model_consumer/model_key audit rows. |
docs/security.md |
Documents audit trail semantics for model consumers/keys (actions, whitelist contents, filtering/redaction behavior). |
docs/operations.md |
Adds migration 161 operational notes (locks/behavior, downgrade/rollback considerations). |
docs/multi-tenancy.md |
Documents max_model_consumers and immutable tenant binding for consumers/keys. |
docs/api.md |
Updates audit/API documentation to reflect model_consumer/model_key audit behavior introduced by the migration. |
Review effort: Balanced
| summary, _ := auCheckPlan(raw, "") | ||
| t.Logf("tenant %s: %d row(s) returned, Rows Removed by Filter %d — %s", tenant, len(rows), removed, summary) |

Summary
W01-1 (RC-B, first migration of the credential class): Migration 161 —
context_model_consumers(tenant-bound caller identity + policy:tenant_scope,all_models,prompt_sensitivity,fair_weight,admission_lane,allowed_cidrs,allow_plaintext, peer binding),context_model_keys(credential of exactly one consumer, stored as SHA-256 + hint),ctx_model_auth(hash, ip, secure)STABLE with six gates and one byte-identical sentinel row, deactivation trigger (revokes live keys), identity pins (tenant_id/consumer_idimmutable, 23514), whitelist audit of both tables throughaudit_actor_meta()withmetadata.tenant_id/consumer_id,model_cidrs_private,context_tenants.max_model_consumers, and the tenant expression indexidx_settings_audit_entity_tenantoncontext_settings_audit(W06-7 review F1). Nothing writes these tables yet (store primitives W01-3, admin actions W01-5, the/v1door W01-6).BEFORE INSERTguard oncontext_model_keyslocks the consumerFOR SHAREand refuses an inactive consumer with 23514, so a mint and a deactivation serialize on the consumer row under READ COMMITTED (the FK alone takesFOR KEY SHARE, which the deactivating UPDATE does not wait for — a mint overlapping a deactivation committed a live key at an inactive consumer). Known limit documented: a deactivation under explicit REPEATABLE READ does not revoke a key committed after its snapshot.test.shT07 59 → 61 tables, docs (operations.mdMigration 161,security.mdaudit trail,multi-tenancy.md).Gates (red against 6773d5b / migration 160, then green)
ctx_model_auth42883, classification pin 36 deviations, plan test 2 400 / 1 407 rows removed by filter → 0. Gate matrix 13/13 +reads_only(n_tup_updunchanged,provolatile = s).CTX_M161_OVERLAY=gate1…gate6make exactly their rows red (3/1/2/1/2/1);no-deactivate,no-identity,no-of-list,blacklist,extra-fk,key-policy,cidr-null,no-insert-guardeach red. Main context re-rangate3(2 rows) andno-insert-guard(3/3 subtests red)._migrationsway-back confirmed.-short -race, integration store (full) / handlerAudit/ auth / schemacontract / ctxd — green on lead and main context.TestClusterAnnotationCostAtCeiling(p95 latency budget) red once under host load ≈ 60, isolated 10.3 ms PASS — outside this wave.Hand-offs
GREATEST, refuse mint into an inactive consumer on the Go side too (lock duty), setrevoked_by_principal. W01-5: peer binding visibility (design/05 vs design/04 disagree). W01-6: normalize client IP withUnmap(). W01-8: NOTIFY also onadmission_lane/fair_weight. W01-9: grant table with its own whitelist. Offboarding wave: pincontext_api_keys.tenant_idincl. fixture rebuild. Open: F4 timing index, extension-floor decision.Wave report
.project/plan-openai-api-2026-11-03/reports/bau/W01-1.md, reviewreports/review/W01-1-fremdreview-2.md.🤖 Generated with Claude Code
Summary by CodeRabbit