borg2 check: overview / tracking ticket #9998

This ticket collects all cmd: check tickets into one place: a summary of how borg2 check
works, plus a checkbox list of the concrete work items. Ticked items are in master (PR or commit
linked); unticked items are still to do (open PR linked where one exists).

Status as of master ab0e8e9ce (2026-10-05).

Open tickets: #9825 #2397 #10479 #10410 #10397, #6584 (b27).
Closed tickets: #9696 #9898 #9901 #9218 #9443 #8009 #7893 #8466 #8476 #8478 #10026 #10042 #10434 #10364
#10122 #9897 #9411 #8559 #8297 #1931 #7062 #8517 #8518 #2398.

How borg2 check works

What changed vs. borg 1.x

borg 1.x check had a repository part (crc32-check all segment entries, always rebuild the
repository index, run server-side for ssh repos) and an archives part (read all archives, check
chunk presence, optionally --verify-data). Much of that design no longer applies:

• A borg2 repository is a borgstore store. There is no borg process on the server side
  (except for ssh:///rest: served by borg serve --rest), so "run the check on the server"
  is replaced by content-addressed verification.
• Pack files (packs/<id>) and chunk index fragments (index/<id>) are named by the store hash
  (blake3, name content-addressed objects by blake3 instead of sha256 #10363) of their content. So "is this object intact?" is simply
  Store.hash(name) == name. The REST backend computes the hash server-side, so a full
  repository check of a remote repo transfers no pack data. The store hash detects accidental
  corruption, not tampering — that is what the AEAD authentication (key needed) is for.
• The chunks index (chunk id → pack id, offset, size) lives in the repository and is the
  authoritative record of which chunks exist and where.
• There is no chunks_healthy anymore (borg2: reconsider chunks_healthy approach #8559): an item's chunks list always holds the
  correct chunk ids, even when chunks are missing. Missing chunks are handled dynamically at
  read time (zeros or error), and if a later borg create re-produces a missing chunk, all
  old archives referencing it are healed instantly — no repeated check --repair runs.

Phase 1: repository check (cheap, read-only)

1. Verify all index/ fragments via Store.hash (small, always checked in full).
2. If (and only if) the index is intact, cross-check that every pack referenced by the index is
   present in packs/ (borg2 check: vanished packs? #9898), then verify all packs/ via Store.hash.
3. Pack results (intact with a timestamp, or corrupt) are persisted in cache/checked-packs
   (borg2 check: move from marker to set-of-checked-packs #9897, check: keep pack check results, add --max-age to reuse them #9925). Partial checks (--max-duration) resume from that set, with periodic
   checkpoints; --max-age skips packs with a fresh intact result. Packs recorded as corrupt
   are always re-verified, never skipped, and keep failing the check until repaired or gone.
4. The read-only check never rebuilds the index: reading every pack is far too slow and
   expensive for a routine check. A corrupt index is reported and left for repair.

cache/checked-packs and the index fragments are stored in the key's envelope, so borg check
always needs the key (#10385); the passphrase is asked up front (#1931). This phase is designed
to be a routine, cron-able job: bounded time, meaningful exit code, and its findings (corrupt
pack ids) are consumed by a later repair instead of re-scanning everything.

Phase 2: archives check (client-side)

1. Check the manifest (rebuild if missing/corrupt), the archives directory, each selected
   archive's metadata object and item metadata stream (robust resync on damage), and each
   item's file-content chunk presence against the (verified) chunks index.
2. --verify-data: additionally read, decrypt (AEAD-authenticate) and decompress every
   object — full cryptographic verification, detects malicious tampering too (for encrypted
   repos, against attackers without the key). Reads pack by pack, each pack's chunks in offset
   order (ChunkIndex.iter_packs()), so each pack is fetched once.
3. --find-lost-archives: scan for archive metadata without an archives directory entry; with
   --repair, recreate the entry (possible until borg compact removes the data). Combined
   with --verify-data, it reuses the archive metadata ids found by the verify pass.

A normal (non-repair) archives check trusts the hash-verified stored index and shares one
chunks index with the repository phase; it does not rebuild it (#8476, #10368).

Repair (borg check --repair)

The command split was decided in #10026: repair stays under borg check --repair.

• Only repair what is known to be broken. Repair consumes the corrupt pack list in
  cache/checked-packs and reuses intact results within --max-age (check: --repair salvages the packs recorded corrupt, refs #10026 #10455, check: --repair reuses the intact pack check results within --max-age, refs #10026 #10464).
• Pack repair salvages a corrupt pack: every object that still authenticates goes into a
  new pack, the corrupt pack is dropped, the index updated (Repository.salvage_pack(),
  repository: add salvage_pack, keeping only the authenticated objects of a corrupt pack, refs #10026 #10419, check: --repair salvages the packs recorded corrupt, refs #10026 #10455). If a second load of the pack differs from the first (storage/transfer
  fault), nothing is dropped and the check fails.
• Index repair rebuilds the index from the packs' object headers and validates every object
  with the key (header + metadata), so a tampered header does not get into the index (borg2: index rebuild trusts pack headers #9901).
  A full --repair rebuilds the index exactly once per run (in the archives phase, check: a full --repair rebuilds a corrupt index once, in the archives phase, fixes #10434 #10445);
  finish() re-reads only the packs the repair wrote and writes the index back to the repo, so
  the next command does not rebuild it (borg2: check [--repair] improvements #8466, check --repair: re-read only the packs the repair wrote, refs #8466 #10379). Index entries of missing packs are
  removed (check --repair: remove the index entries of missing packs, refs #8572, #9898 #10411).
• Defect chunks (found by --verify-data) are only deleted after a second read attempt
  also fails (transient errors — network, RAM — must not destroy data). If another copy of a
  removed chunk is intact, it gets indexed (check: --verify-data --repair indexes an intact other copy of a defect chunk, fixes #10491 #10493).

Design goals

• Data safety: never delete data on a single failed read; never let a rebuilt index claim
  objects that do not exist (fatal for borg create dedup, borg2: avoid doing an unnecessary rebuild of the index #8476); verify before trusting
  (borg2: index rebuild trusts pack headers #9901); an interrupted check/repair must leave a consistent state (Ctrl-C at safe
  boundaries, Ctrl-C does not interrupt borg check #7893).
• Usability for admins: routine checks need no prompts (other than the passphrase, unless
  given via env); progress indication everywhere (Add progress indicator to repository check loop #9443, Progress of Remote: checking segment file /whatever/data/17/17078... #8009); a summary of findings and
  repairs at the end (check: summarize findings, --repair: summarize repairs #2397); reports organized around the question the admin actually has —
  "which files/archives are affected?" (Improve borg2 check reporting of missing chunks #9218); a documented "what do I do now?" workflow after
  corruption (borg2: improve dealing with repo corruption #9825); customizable output (borg2: check does not support custom output formatting #9411).
• Performance: server-side hashing (no transfer); no unnecessary index rebuilds (borg2: avoid doing an unnecessary rebuild of the index #8476);
  partial/time-boxed checks; reuse of still-fresh check results (--max-age); headers-only
  scans where possible; pack-ordered reads; combine full-repo passes instead of reading
  everything twice (borg2: check --verify-data --find-lost-archives can be made faster #8478).

Work items

Repository check (read-only)

• Content-addressed read-only check: verify index/ and packs/ objects via
  Store.hash, server-side hashing for rest: repos; index first, packs only if the
  index is intact; no index rebuild in the read-only path (borg2: improve borg check #9696, borg2: avoid doing an unnecessary rebuild of the index #8476, borg2 check redesign? #8518) —
  27d693c2 (PR repository: read-only check by hashing pack/index objects #9794, with PRs repository: make check and delete work at N=1 with sha256 pack ids #9783/repository: always name packs by sha256(pack) #9789 for content-addressed pack naming; blake3 since name content-addressed objects by blake3 instead of sha256 #10363)
• Progress indicators (percent) for the index and pack verification loops (Add progress indicator to repository check loop #9443, Progress of Remote: checking segment file /whatever/data/17/17078... #8009) —
  27d693c2 (PR repository: read-only check by hashing pack/index objects #9794)
• Partial checks: persist a set of checked packs instead of a "last pack" marker, so
  packs added mid-cycle cannot be skipped (borg2 check: move from marker to set-of-checked-packs #9897) —
  bc8b3c07
• Re-verify packs recorded as corrupt instead of skipping them on resume (borg2 check: move from marker to set-of-checked-packs #9897) —
  9543c141
• Checkpoint the checked-packs set periodically (30 min) during long runs (borg2 check: move from marker to set-of-checked-packs #9897) —
  a1b88e2d
• Keep pack check results after a completed check and add --max-age to skip packs with
  a fresh intact result — makes cheap periodic checks feed later repairs and avoids
  re-hashing everything on slow/cloud storage (borg2: improve borg check #9696, borg2 check: vanished packs? #9898) — PR check: keep pack check results, add --max-age to reuse them #9925
• Detect vanished packs: cross-check that every pack referenced from the chunks index is
  present in packs/ (borg2 check: vanished packs? #9898) — PR check: detect missing packs referenced by the index (#9898) #10069. Presence only, no size check (the index has no
  pack sizes).
• Cheap check mode for cloud repos: make full pack hash verification optional; default
  (or an option) could be listing packs and checking presence + size against the index,
  avoiding even the server-side hashing (was part of borg2 check: vanished packs? #9898, which is closed; no ticket yet)
• Fix: borg compact makes a corrupt pack pass borg check (compact makes a corrupt pack pass borg check #10410; compact no longer
  rewrites or merges packs recorded corrupt since PR compact: do not rewrite or merge packs recorded corrupt, refs #10410 #10412)
• Fix: a corrupt store cache entry is treated as repository corruption, check --repair
  then deletes the healthy chunk (store cache: a corrupt cache entry is treated as repository corruption, check --repair deletes the healthy chunk #10397) — PR check, repo-compress: read packs from the repository, not from the store cache, fixes #10397 #10467 open
• Survive I/O errors while reading repository objects — PR check: survive I/O errors while reading repository objects #10194 open

Archives check

• Ask for the key passphrase early, before the (potentially very long) repository phase
  (ux: borg check asks for passphrase very late #1931) — 2340e605
• Archive selection filters --first/--last/-a/--sort-by (adapt borg check like in PR 1554 #1663) and
  --older/--newer/--oldest/--newest (borg check --archives-only --last-n-seconds #7062) —
  b2654bc1
• --format support for the "Analyzing archive ..." output, validated up front,
  BORG_CHECK_FORMAT env var (borg2: check does not support custom output formatting #9411) —
  d7ffe748
• Non-repair archives check trusts the hash-verified in-repo chunks index — no slow index
  rebuild for a routine check (borg2: avoid doing an unnecessary rebuild of the index #8476, borg2: check [--repair] improvements #8466) —
  27d693c2
  (earlier step: reuse the repo-part ChunkIndex, 7288f4f9, Check improvements #8468;
  one chunks index for the checker and the repository: PR check: use one chunk index for the checker and the repository, refs #10364 #10368, borg2: avoid having the chunks index loaded multiple times #10364)
• Remove chunks_healthy; self-healing archives, defined read-time behaviour for missing
  chunks, no all-zero patch chunks (borg2: reconsider chunks_healthy approach #8559) —
  6357f2eb
• --find-lost-archives (né --undelete-archives): scan for archive metadata lacking an
  archives directory entry, recreate entries with --repair —
  682aedba,
  renamed in a48a8d2b
• Speed up --verify-data combined with --find-lost-archives: remember archive-metadata
  object ids during the verify pass and reuse them instead of a second full repo scan
  (borg2: check --verify-data --find-lost-archives can be made faster #8478, ex-check: optimization potential with --verify-data AND --undelete-archives #8517) — PR check: --find-lost-archives reuses the archive metadata ids found by --verify-data, fixes #8478 #10447
• --verify-data reads pack by pack, each pack's chunks in offset order, so each pack is
  fetched once, sequentially ("read data only once", idea from borg2 check redesign? #8518) — PR verify_data: read the repo pack by pack #10408
  (ChunkIndex.iter_packs(), partitioned by pack id within a memory budget)
• --archives-only no longer crashes on a missing pack (borg2 check --archives-only crashes if a pack referenced by the index is missing #10466) — PR check: --archives-only reports missing packs instead of crashing, fixes #10466 #10473
• Robust resync inside huge damaged items no longer hangs (borg check: RobustUnpacker resync is quadratic/cubic in the amount of data it has to skip #10459) — PR RobustUnpacker: resync in linear time, fixes #10459 #10463

Repair

• Repository-level repair (borg2 check: implement repository repair (index and pack rebuild) #10026, borg2: improve borg check #9696 "to be continued", borg2: improve dealing with repo corruption #9825):
  • Index repair: rebuild the index from pack object headers (range reads), validating
    every object, when index/ objects are corrupt (borg2: improve borg check #9696, borg2: avoid doing an unnecessary rebuild of the index #8476) — PRs check --repair: rebuild a corrupt repository index from the packs, #10026 #10048, validate object headers when walking a pack (PackReader.iter_headers) #10083,
    check --repair: resync past corrupt object headers when rebuilding the chunks index #10094, index rebuild: abort cleanly on a corrupt object header, #10122 #10168
  • Pack repair: salvage all AEAD-valid objects of a corrupt pack into a new pack, drop the
    corrupt pack, update the index — without a full scan of all other packs (borg2: avoid doing an unnecessary rebuild of the index #8476, borg2: improve borg check #9696)
    — PRs repository: add salvage_pack, keeping only the authenticated objects of a corrupt pack, refs #10026 #10419 (salvage_pack), check: --repair salvages the packs recorded corrupt, refs #10026 #10455
  • Let repair consume the persisted corrupt-pack list from previous read-only checks, so
    an admin's --repair doesn't redo the cron job's scanning work (borg2: improve borg check #9696) — PR check: --repair salvages the packs recorded corrupt, refs #10026 #10455;
    --repair reuses intact results within --max-age — PR check: --repair reuses the intact pack check results within --max-age, refs #10026 #10464
  • Decide the command split: keep everything under borg check --repair (decided in
    borg2 check: implement repository repair (index and pack rebuild) #10026)
  • Remove the index entries of missing packs — PR check --repair: remove the index entries of missing packs, refs #8572, #9898 #10411
• Index rebuild must not blindly trust unauthenticated pack headers: validate every object
  with the key during the rebuild (borg2: index rebuild trusts pack headers #9901) — PRs check --repair: validate the repository index rebuild with the key, #9901 #10369, remove the drop_corrupt_tail mechanism, refs #10369 #10378, check: always require the key #10385
• After check --repair, write the freshly built chunks index back to the repository
  instead of deleting it and forcing the next command into a slow rebuild (borg2: check [--repair] improvements #8466) — PR
  check --repair: rebuild a corrupt repository index from the packs, #10026 #10048; finish() re-reads only the packs the repair wrote — PR check --repair: re-read only the packs the repair wrote, refs #8466 #10379
• A full check --repair rebuilds a corrupt index only once, not twice (check --repair: a full check rebuilds a corrupt chunks index from all packs twice #10434) — PR
  check: a full --repair rebuilds a corrupt index once, in the archives phase, fixes #10434 #10445
• Repair progress indication: removing defect chunks, re-reading the written packs, loading
  / rebuilding the chunk index (Add progress indicator to repository check loop #9443) — PRs check: show progress while --repair removes defect chunks and re-reads the written packs, fixes #9443 #10458, chunk index: show a spinner or percentage while loading / rebuilding it, refs #10082 #10478
• --verify-data --repair indexes an intact other copy of a removed defect chunk (check --repair --verify-data: salvaged gap copy of a defect chunk is not indexed #10491)
  — PR check: --verify-data --repair indexes an intact other copy of a defect chunk, fixes #10491 #10493
• Fix: a corrupt pack in which no object authenticates is never removed, so check and
  compact keep failing (check --repair: a corrupt pack in which no object authenticates is never removed, check and compact keep failing #10479)
• Repair using redundant related repos: fetch chunks that are missing/corrupt here from a
  related repo (same key material / same chunk ids), e.g.
  borg check --repair --other-repo=...; borgstore cache vs. primary self-healing is a
  variant of the same idea (borg check: doing a repair using redundant related repos #6584 — milestone b27)

Accepted for now: check --repair rebuilds the index from all packs once per run, even if the
stored index is intact — the full walk is what finds archives that reference chunks in missing
packs (#8476).

Reporting & UX

• Summarize findings at the end of check, and repairs actually done with --repair
  ("N problems found, M repaired") (check: summarize findings, --repair: summarize repairs #2397, dup [1.2.7] borg check -repair lacks a summarizing information at the end when it actually repaired something(s) #8192) — PR check: summarize findings and repairs #9891 open
• Report missing chunks grouped as chunk → files → archives instead of one log line per
  (archive, file, chunk); avoids the hardlink-caused flood of repeated lines and answers
  "which files are affected?" directly (Improve borg2 check reporting of missing chunks #9218) — PR check: report missing chunks grouped as chunk -> files -> archives, #9218 #9965
• Ctrl-C handling: first Ctrl-C stops soon but cleanly at a safe boundary (saving
  checkpoint state, running finish()), second Ctrl-C aborts immediately (Ctrl-C does not interrupt borg check #7893) —
  PR check: handle Ctrl-C at safe boundaries (#7893) #9966; pre-check index/key rebuild phases interruptible (check: make the pre-check index/key rebuild phases interruptible (remaining work for #7893) #10042) — PR check: make the chunk index rebuild interruptible, fixes #10042 #10382
• Document the post-corruption workflow (borg2: improve dealing with repo corruption #9825): what --repair does and does not do,
  that re-running borg create heals reappearing chunks automatically (no second
  check --repair needed), how to get rid of permanently lost data
  (borg recreate --exclude ... the affected files), "considerations to fix your repo
  corruption" guide
• A way to acknowledge known-lost chunks so subsequent check/compact runs stop returning
  exit code 2 / error mails for damage the admin has already accepted (borg2: improve dealing with repo corruption #9825)

Done / resolved by the borg2 design (no action needed)

• No more borg check requirement before prune/compact, so the --repair YES prompt no
  longer haunts cron jobs ([Feature Request] Disable the YES prompt when BORG_CHECK_I_KNOW_WHAT_I_AM_DOING is enabled #8297; BORG_CHECK_I_KNOW_WHAT_I_AM_DOING=YES still skips the
  prompt for actual repair runs)
• Segment-era tickets are obsolete: there are no segments, transactions or commits in a
  borgstore repo — Review check operation if no commit exists #2454 (check with no commit), Corrupted segment reference count - corrupted index or hints #8535/Frequent "Index object count mismatch." errors on borg (1.4.0) check runs #8580 (segment refcount / index
  object count mismatch), Possibility to run borg check for specific segment(s) #8070 (check a single segment; nearest borg2 equivalent is the
  per-pack salvage above)
• "check --repair commit prompt" (check --repair commit prompt #2398) — obsolete: borg2 has no repo transaction to
  commit; the equivalent admin need (see what check finds, then repair exactly that) is
  covered by the persisted check results + repair consuming them
• borg extract does not abort on corrupted chunks anymore, it writes all-zero data and
  warns ( borg extract: check behaviour on integrity (or other) errors #840) — PR extract: do not abort on corrupted chunks, replace them by all-zero data with a warning, fixes #840 #10357

Ticket overview

made by Claude Fable 5, status updated 2026-10-05 by Claude Opus 5.5