Skip to content

Store every format's SQL benchmark file sizes - #11447

Merged
joseph-isaacs merged 1 commit into
developfrom
ji/friendly-noether-hk4glm
Nov 13, 2026
Merged

joseph-isaacs merged 1 commit into
developfrom
ji/friendly-noether-hk4glm

Conversation

@joseph-isaacs

@joseph-isaacs joseph-isaacs commented Nov 13, 2026 •

Copy link
Copy Markdown
Contributor

Summary

#11443 started storing SQL benchmark file sizes in the benchmarks database, but only for vortex-file-compressed and vortex-compact. This PR stores every on-disk format, so bench can chart Parquet, DuckDB, Lance and the rest beside Vortex (vortex-data/vortex.dev#222).

`capture-file-sizes.py` recorded only the two Vortex formats. It now records
every on-disk format directory `Format::name()` names (Parquet, DuckDB, Lance,
Arrow IPC, CSV, LitData and the Vortex formats), so bench can chart them side
by side. It is a few more rows per develop commit.

DuckDB keeps a catalog of views in each format directory it queries
(`duckdb.db` and its WAL). That is not the format's data, so it is skipped
everywhere except the `duckdb` format's directory, where the database is the
data. This also drops the catalog from the Vortex totals.

The PR comment's "File Size Changes" table still compares only the `vortex*`
formats: the other formats' files come from the data generators and do not
change with a Vortex PR.

Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UeniB9quRiv5fCx8AhzH9s
Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>
@joseph-isaacs joseph-isaacs added the changelog/chore A trivial change label Nov 13, 2026 — with Claude
@joseph-isaacs
joseph-isaacs enabled auto-merge (squash) November 13, 2026 14:48
@joseph-isaacs
joseph-isaacs merged commit bcbe4c2 into develop Nov 13, 2026
100 of 102 checks passed
@joseph-isaacs
joseph-isaacs deleted the ji/friendly-noether-hk4glm branch November 13, 2026 14:53
@codspeed

codspeed Bot commented Nov 13, 2026

Copy link
Copy Markdown

Merging this PR will improve performance by 12.21%

2 warnings

⚠️ Walltime results may be unreliable

Some benchmarks used the Walltime instrument outside CodSpeed Macro Runners, which can make the results inconsistent.

Learn more about Macro Runners

See affected benchmarks

⚠️ 1 benchmark is dominated by system calls

System calls cannot be simulated consistently and are left out of the measure. The reported value understates the real cost. Switch to the Walltime instrument to measure them.

Learn more about system calls

See affected benchmarks

⚡ 1 improved benchmark
✅ 1935 untouched benchmarks
⏩ 50 skipped benchmarks1

Performance Changes

Mode Benchmark BASE HEAD Efficiency
⚡ CPU Simulation test_repeated_scan_scalar_at[int_float] 317.2 µs 282.7 µs +12.21%

Tip

Curious why performance improved? Comment @codspeedbot explain why performance improved on this PR, or directly use the CodSpeed MCP with your agent.


Comparing ji/friendly-noether-hk4glm (4e82b5a) with develop (eacb353)

Open in CodSpeed

Footnotes

  1. 50 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

changelog/chore A trivial change

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants