Repository navigation
Add the file_sizes table for SQL benchmark file sizes - #221
Merged
Merged
Conversation
The monorepo's PR benchmark comments compare against baselines from this database, and SQL benchmark file sizes were never stored, so the comments lost their file-size section. Migration 014 adds `file_sizes`, keyed by (commit_sha, benchmark, scale_factor, format, file), with an index on the dims the CI baseline lookup filters on. The default privileges from 004, 005 and 011 grant the ingest, read and CI-read roles their access. Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UeniB9quRiv5fCx8AhzH9s Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
🔦 Lighthouse audit
Run |
Review of the CI reader: the baseline commit is chosen from the query measurements, and file sizes are then read for that commit alone, so the index ends in `commit_sha` rather than `file`. Also document the sixth kind in the bench README, the provenance of the `file_sizes` golden vectors, why a new kind needs no `SCHEMA_VERSION` bump, and the table in the restore check. Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UeniB9quRiv5fCx8AhzH9s Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UeniB9quRiv5fCx8AhzH9s Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changes
bench (database schema and docs only; the site's pages and charts don't change).
The PR benchmark comments in
vortex-data/vortexcompare against baselines read from this database. SQL benchmark file sizes were never stored here, so when vortex-data/vortex#10987 moved the baseline from S3 to Postgres, the comments lost their file-size section. This adds a table for those sizes.db/migrations/014_file_sizes.sqlcreatesfile_sizes, keyed by(commit_sha, benchmark, scale_factor, format, file)withvalue_bytes.(benchmark, scale_factor, format, commit_sha). CI chooses the baseline commit from the query measurements, then reads that commit's files for each scope, and this index serves that read as a range scan.ALTER DEFAULT PRIVILEGESrules in 004, 005 and 011 givebench_ingest,bench_readandbench_ci_readtheir access whenmigratorcreates the table.requires-superusermarker either, sobench-schema-deploy.ymlcan apply it.docs/bench/CONTRACT.md,docs/bench/README.mdanddocs/bench/architecture/data-pipeline.mdadd thefile_sizekind. CONTRACT.md also records where the new golden vectors came from, and why a new kind with its own table needs noSCHEMA_VERSIONbump.infra/RESTORE.mdadds the table to the restore data check, anddb/migrations/README.mdlists 014.The writer and the reader are in vortex-data/vortex#11443. Apply this migration in production before that PR merges. Develop's ingest is required and writes each file in one transaction, so
file_sizerecords sent before the table exists would fail every develop SQL benchmark job. The PR-side reader now skips a missing table, but develop's ingest does not.How it was checked
Applied the migrations to a local PostgreSQL 16 and loaded 1,500 commits × 100 files. With the PR 100 commits behind develop, the vortex PR's baseline fetch took 5 ms, and the file read used
idx_file_sizes_baseline(an index scan of 100 rows). Under the earlier index and commit search it took 5.3 s.An independent review applied 001–013 as a bootstrap role and 014 as
migratorthroughdb/migrate-schema.py. It foundfile_sizesowned bymigrator, with SELECT/INSERT/UPDATE forbench_ingestand SELECT for both read roles.bun run check:cianddb/testswere not run locally. CI ran them on the first commit and passed.Every commit is signed off (
git commit -s), for the DCO check.bun run check:cipasses.Each new image or other asset has an annotation in
REUSE.toml. (No new assets.)🤖 Generated with Claude Code
https://claude.ai/code/session_01UeniB9quRiv5fCx8AhzH9s