Repository navigation
fix: Step 1 fails loudly instead of leaving empty marker files; readers keep .graphify_root exactly (#4346) - #4353
Conversation
|
Thanks for the pull request, @Deepjyoti-Sarmah. A maintainer will review it soon. Want to talk it through while it is in review? Come join us on our Discord server. For longer-form discussion there is also GitHub Discussions. A couple of things that speed up review: make sure the test suite passes on Python 3.10 and 3.13, and that the change keeps extraction deterministic. |
There was a problem hiding this comment.
Graphify reviewed this change.
No issues found by static checks — no coupling regressions or blocking issues in the code graph. That is not the same as safe to merge: see what was not checked.
Not checked: tests were not run; formal verification proved 0 of 4 changed function(s) (2 sampled, not proven, 2 not verified).
Formal verification. PR-changed functions: 2/4 verified (0 proven, 2 may-equivalent, 0 distinguished) · 2 not verified (1 vacuous, 1 unsupported).
Not verified on this run: dispatch\_command (vacuous: never exercised), \_\_init\_\_ (unsupported).
Graphify review — findings
Fixes .graphify_root handling so a scan root with trailing spaces survives: the new read_root_marker drops only a UTF-8 BOM and one trailing newline instead of calling .strip(), which turned proj into its sibling proj, and every marker reader, including the generated hook scripts, now uses it. graphify watch with no path now exits 1 on an empty marker instead of guessing a path. The skill templates write .graphify_python and .graphify_root in one Python call that deletes last run's files, writes each through a temp file and rename, and exits 1 naming the file if a write fails (e.g. a full disk), so no empty or stale marker is left behind.
Review partial — this diff was larger than one review pass covers, so later files were not reviewed; some findings may be missing.
Analysis details — impact, health, verification
Impact & health
Graphify review
Impact — 5016 functions depend on the 1591 functions this change touches.
Health — this change adds coupling hotspots:
- worse:
dispatch_command()— 2 callers, 131 callees
Verification — 5016 functions in the blast radius were not formally verified this run (proofs are advisory here).
Gate & verification
graphify gate
PASS — objectively clean (no health regressions, tests not run — proofs not run this pass (advisory)). Grounded, not self-assessed.
Advisory (not blocking):
- verification_scope: 4244 function(s) in the blast radius were not formally verified this run
Test selection
Test selection
377 of 377 test file(s) selected (100%) via static blast radius.
Escalated to a full run for safety — the selection is not trustworthy on its own (see below). CI should run the whole suite.
tests/test_affected_cli.py— impact, full-run-safetytests/test_affected_defines.py— full-run-safetytests/test_affected_member_seed.py— full-run-safetytests/test_affected_overrides.py— full-run-safetytests/test_agents_platform.py— impact, full-run-safetytests/test_analyze.py— impact, full-run-safetytests/test_anthropic_custom_endpoint.py— full-run-safetytests/test_antigravity_install.py— full-run-safetytests/test_apm_fallback_version.py— full-run-safetytests/test_architecture_doc.py— full-run-safetytests/test_astro_extraction.py— full-run-safetytests/test_astro_import_ids.py— full-run-safetytests/test_atomic_canvas_export.py— impact, full-run-safetytests/test_atomic_version_stamp.py— impact, full-run-safetytests/test_atomic_writes.py— impact, full-run-safetytests/test_backend_env_isolation.py— full-run-safetytests/test_backend_extras.py— full-run-safetytests/test_benchmark.py— impact, full-run-safetytests/test_benchmark_raw_graph.py— impact, full-run-safetytests/test_blade_extractor.py— full-run-safetytests/test_build.py— impact, full-run-safetytests/test_build_located_semantic_identity.py— impact, full-run-safetytests/test_build_merge_dedup_scope.py— impact, full-run-safetytests/test_build_merge_hyperedges_and_prune.py— impact, full-run-safetytests/test_build_merge_shrink_guard.py— impact, full-run-safetytests/test_builtin_global_type_refs.py— full-run-safetytests/test_cache.py— impact, full-run-safetytests/test_cache_stale_import_target.py— impact, full-run-safetytests/test_callflow_html.py— impact, full-run-safetytests/test_cargo_introspect.py— full-run-safetytests/test_cargo_missing_manifest.py— full-run-safetytests/test_carried_hyperedge_remap.py— impact, full-run-safetytests/test_case_sensitive_resolution.py— full-run-safetytests/test_charmap_encoding.py— impact, full-run-safetytests/test_chunking.py— impact, full-run-safetytests/test_cjs_module_extension.py— full-run-safetytests/test_claude_cli_backend.py— impact, full-run-safetytests/test_claude_md.py— full-run-safetytests/test_cli_broken_pipe.py— full-run-safetytests/test_cli_export.py— impact, full-run-safetytests/test_cli_help.py— full-run-safetytests/test_cloud_cta.py— impact, full-run-safetytests/test_cluster.py— impact, full-run-safetytests/test_cluster_ambiguous_scale.py— full-run-safetytests/test_cluster_exclude_hubs.py— full-run-safetytests/test_cobol_extractor.py— full-run-safetytests/test_codebuddy.py— impact, full-run-safetytests/test_community_hub_labels.py— full-run-safetytests/test_community_labels_skill.py— impact, full-run-safetytests/test_compact_graph_json.py— impact, full-run-safety- … and 327 more
non-code file(s) changed (
graphify/skill-agents.md,graphify/skill-aider.md,graphify/skill-amp.md,graphify/skill-claw.md,graphify/skill-codex.md…) → running the full suite for safety (a code graph can't see config/fixture/data deps)
changed code file(s) with no mapped test (
graphify/skill-agents.md,graphify/skill-aider.md,graphify/skill-amp.md,graphify/skill-claw.md,graphify/skill-codex.md…) — a coverage gap or a missing link — running the full suite rather than only the selected tests
Selection is safe under the controlled-regression assumption; always-run tests + a periodic full run are the backstops. Advisory — it never changes the check verdict.
Formal verification
No difference found (not proven): No behavior difference found in \_infer\_merge\_root (not a proof).
The verifier ran both versions of \_infer\_merge\_root on many inputs and saw identical behavior every time. Strong evidence the change is safe, but evidence, not a proof.
Guarantee: Empirical: differential testing (both versions run on many generated inputs). A divergence on an untested input remains possible, so this is 'no counterexample found', not 'proven equivalent'.
Note: An input the sampler did not try could still differ.
Could not verify: Could not verify dispatch\_command.
The verifier did not have enough to check dispatch\_command, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: no capturable inputs from the test suite; property tier: not verifiable: all 40 sampled inputs raised on both versions — the function never executed, so 'no divergence' would be vacuous (mostly SystemExit — names the real obstacle, not a sampling gap)
No difference found (not proven): No behavior difference found in \_resolve\_source\_path (not a proof).
The verifier ran both versions of \_resolve\_source\_path on many inputs and saw identical behavior every time. Strong evidence the change is safe, but evidence, not a proof.
Guarantee: Empirical: differential testing (both versions run on many generated inputs). A divergence on an untested input remains possible, so this is 'no counterexample found', not 'proven equivalent'.
Note: An input the sampler did not try could still differ.
Could not verify: Could not verify \_\_init\_\_.
The verifier did not have enough to check \_\_init\_\_, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: no capturable inputs from the test suite; property tier: method — first parameter is `self`, which needs a constructed instance (not synthesizable)
· 1 more finding(s) on lines outside this diff (see the check run).
655c2e0 to
378d4b2
Compare
378d4b2 to
2212b9a
Compare
2212b9a to
7afe3fa
Compare
…DICT (Graphify-Labs#4309) The skill has the agent paste these two values into python -c scripts. A label like `x $(touch PWNED)` then runs in the shell, and `x'}; ...` runs as Python. Step 1 and --watch were fixed earlier; the other steps and the labels were not. The tests render every skill file and check that no script pastes them, that the interpreter path is quoted, and that Step 5 reads labels from JSON. They also pin read_scan_root(), which doesn't exist yet, so they fail until the fix lands.
…aphify-Labs#4309) Skill steps paste INPUT_PATH into python -c scripts, so a folder name such as `proj$(touch X)` can run as shell code. The root is already saved by Step 1 in graphify-out/.graphify_root; this reads it back as plain data. The root may sit outside the cwd, so it only has to be a non-empty path to an existing directory.
…code (Graphify-Labs#4309) Step 5 had the agent paste a Python dict over LABELS_DICT and the scan path over INPUT_PATH inside a double-quoted python -c script. A label such as `x $(touch PWNED)` ran in the shell, and `x'}; ...` ran as Python. The agent now writes the names as JSON through a quoted heredoc and the script loads them with json.loads. The scan root comes from read_scan_root(). The generator turns the heredoc into a PowerShell here-string for Windows, and the monolith audit allows these lines. INPUT_PATH in the other steps and the interpreter quoting follow in separate commits.
…tep (Graphify-Labs#4309) The skill had the agent paste the scan path over INPUT_PATH inside the double-quoted python -c scripts of Steps 2, 3, 4, 4.5 and 9 and the --update runbook. A folder name such as `proj$(touch X)` then ran as shell code. Those scripts now call read_scan_root(). The monolith audit allows the seven lines that changed, and the persisted-scan-options test writes graphify-out/.graphify_root as the skill does. Step 1 still takes the path through a quoted heredoc, which is safe. A step run by hand without Step 1 now fails with FileNotFoundError.
…y-Labs#4309) Step 1 saves the interpreter to graphify-out/.graphify_python and every later step runs it as $(cat graphify-out/.graphify_python). Unquoted, an interpreter under a folder with a space (/Users/First Last/...) splits into two words and the step fails. Every invocation is now "$(cat graphify-out/.graphify_python)", including the guidance that tells the agent how to call Python. The Windows translator recognizes script openers by exact text, so _PY_INVOKE_POSIX changes to match. The monolith round-trip unquotes the rendered side before diffing, so a quote-only change cancels out while any other edit on those lines is still flagged.
Graphify-Labs#4309) Steps B0 and B3 had the agent paste the spec path over SPEC_PATH inside prompt_file='SPEC_PATH' in a double-quoted python -c body, so a path with a quote or $( ran as Python or shell. Step B0 now writes the path to graphify-out/.graphify_spec_path through a quoted heredoc, and both cache calls read it back, so the read and the write still name the same prompt (Graphify-Labs#1939). The Windows translator handled only the labels heredoc; it now takes a table of the known data heredocs, and any other heredoc still raises.
…Labs#4309) A scan path with a line break could end the quoted heredoc (or the PowerShell here-string) early and run the lines after it as shell. The skill now tells the agent to refuse such a path and ask for another one. The POSIX Python side reads all of stdin and exits non-zero on more than one line; the PowerShell block does the same check. The block comments now say only what is true.
…ath is quoted (Graphify-Labs#4309) test_query_fallback_edges.py located the script by the unquoted opener $(cat graphify-out/.graphify_python) -c ". This branch quotes the interpreter path, so the lookup failed. It now matches on -c " plus the import lines that follow, which only the fallback block has.
… seed .graphify_root in the code-only update test (Graphify-Labs#4309) The aider and devin code-only blocks that landed with Graphify-Labs#4447 still called $(cat graphify-out/.graphify_python) unquoted. The code-only update test pasted INPUT_PATH itself and never wrote the marker Step 1 records, which the update blocks now read.
…#4346) watch.py and reflect.py also read the marker with .strip(), so a scan root named `proj ` was read as its sibling `proj`. In watch that re-anchors stored paths on the wrong folder, which can judge unchanged files deleted (Graphify-Labs#2603). Both now use read_root_marker(), and a guard test fails if any module decodes the marker and strips it again.
7afe3fa to
9e1037a
Compare
Stacked on #4344. Please review that one first. The first 8 commits here belong to #4344. This PR adds the last 4.
With both applied, Step 1 does two things: it refuses a scan path with a line break (#4344), and it fails loudly when a write fails (this PR).
read_scan_root()from #4344 now callsread_root_marker()from this PR, so one function decides how the marker is read.What does this PR do?
Fixes #4346.
Step 1 of the
/graphifyskill saves two files that every later step reads:.graphify_python(which Python to run) and.graphify_root(which folder to scan).1. Step 1 said "done" even when it couldn't save them. On a full disk it left both files empty and exited 0. The next step failed with
bash: : command not found, andgraphify updaterebuilt the current folder instead of the project. Now Step 1:So after Step 1, both files are complete and from this run, or Step 1 failed and said so. The Windows Step 1 does the same.
2. Six readers used
.strip()on the saved folder. A folder namedproj(legal on Linux and macOS) came back asproj. A newread_root_marker()removes only a byte-order mark (#3028) and a final newline, and every reader uses it:graphify update,build.py,watch.py,reflect.pyand the two git hooks.graphify updatenow also stops on an empty.graphify_rootinstead of guessing.How to review
tools/skillgen/fragments/shell/posix.mdtools/skillgen/fragments/core/aider.md,core/devin.mdtools/skillgen/fragments/shell/powershell.mdgraphify/paths.pyread_root_marker()(11 lines)graphify/cli.py,build.py,watch.py,reflect.py,hooks.pytools/skillgen/gen.pytests/test_graphify_root_marker.py,tests/test_skillgen_input_path_injection.pySkip
graphify/skill*.md,graphify/skills/**andtools/skillgen/expected/**. They're generated.Evidence: before and after
Step 1 with
ulimit -f 0, so every write fails the way it does on a full disk:graphify updatewith the saved folderprojand a siblingproj:Reproduce bug 1
From a checkout of
v8:Commits
Type of change
Verification & Invariants
The rule: if
.graphify_pythonor.graphify_rootexists and isn't empty, it holds what this run of Step 1 found, and every reader gets that exact value.Enforced by:
ulimit -f 0with old files present. Each must exit non-zero, name the file and leave nothing behind. The Windows Step 1 gets the same test underpwsh, with a directory blocking its temp file (ulimitstops pwsh itself from starting on CI). That test skips wherepwshisn't installed.projandprojstay apart inread_root_marker(),build.py, the hooks andgraphify update. A guard test fails if any module reads a file asutf-8-sigand then calls.strip().All of these fail on
v8and pass on this branch.What changes for existing setups:
graphify hook installis run again.graphify updatewith an empty.graphify_rootnow exits 1. A missing one still falls back to the current folder.Limitations: not tested on Windows PowerShell 5.1. The Windows Step 1 ran under PowerShell 7.4.6 on Linux.
How was this tested?
Linux, Python 3.13, rebased on v8 fe085ce (0.9.90), on top of #4344. No pwsh on this run.
The Step 1, reader and skillgen tests also passed on Python 3.10 and 3.13.
Merge Danger
Door: two-way. A revert restores the old Step 1 and readers. The files keep the same format, so nothing is migrated.
Blast Radius: every skill's Step 1, plus six small reader changes. Extraction and the graph format are untouched.
graphify/paths.pyandtools/skillgen/gen.py. I'll rebase whichever lands second.Graphify-specific checklist
uv run python -m tools.skillgen --bless) when changing their source fragments.AI disclosure: I made these changes with Claude Code (Claude Sonnet and Claude Opus 5.5). Claude Code ran the tests and commands listed above in my local checkout, and I reviewed every change and the results.