Skip to content

perf(cognitive): read function depth off the ancestor chain - #1095

Merged
dekobon merged 5 commits into
mainfrom
fix/issue-1062
Jul 28, 2026
Merged

perf(cognitive): read function depth off the ancestor chain#1095
dekobon merged 5 commits into
mainfrom
fix/issue-1062

Conversation

@dekobon

@dekobon dekobon commented Jul 28, 2026

Copy link
Copy Markdown
Owner

Summary

Closes the last of #1062's four Node::parent sites.
increment_function_depth asked every function node whether a function
encloses it by climbing with Node::parent, which tree_sitter
resolves by descending from the root. That is O(depth) per step, so
Cognitive stayed O(depth²) on nested function definitions
everywhere it is called.

The scan now reads the ancestor chain the walker hands down (Ancestors,
from #1084), so a step is a slice index. It was deferred out of #1084 to
bound that change's blast radius; all 19 call sites (22 languages,
counting the four the JS-family macro expands to) already had an
ancestors value in scope, so this is one helper signature plus one
argument per call — no per-language logic moves.

Code review then turned up two more per-node climbs inside the same
metric that the issue never listed: Kotlin's when-default check and
Ruby's case-default check each called node.parent() once per else
node, so a deeply nested if/else chain stayed quadratic in the very
metric this PR makes linear on nested definitions. Both read the same
chain now, and cognitive has no Node::parent call left in
production.

The in_function-flag mechanism sketched on the issue was not needed,
and neither was the per-language proof it would have required that each
match arm's kind set equals its stops set.

Measured

cargo bench -p big-code-analysis-bench --bench scaling, new
cognitive/nested-fn probe (Rust, Metric::Cognitive, depths
1000 / 2000 / 4000), fitted time ~ depth^k:

build k verdict
Node::parent climb (pre-fix body, new probe) 2.04 OVER BOUND
ancestor chain 1.21 ok (bound 1.50)

At depth 4000 the walk drops from ~150 ms to ~16 ms. No other probe
moved; nom/nested-fn on the same shape fits 1.13 and is the metric
control.

The probe is new because nom/nested-fn never reached this function —
it selects Metric::Nom. Its shape gains one if per level so
cognitive scores n(n+1)/2 on it, which moves if function depth stops
being counted, so the value column guards the walk as well as the
timing.

Correctness

Cognitive values are unchanged in every language. Two tests pin that:

  • cognitive_function_depth_is_inherited_at_depth — the closed form
    n(n+1)/2 at depth 1000 for nested Rust fns.
  • function_depth_surcharge_holds_across_languages — an if costs 1 in
    a top-level function and 2 one function deeper, for C, Objective-C,
    Mozcpp, JavaScript, MozJS, TypeScript, TSX, Ruby, Lua, Bash and
    iRules. Those were the languages whose function-depth arm had no test
    of its own; C++, C#, Groovy, Java, Kotlin, Perl, PHP, Python, Rust and
    Tcl already had one. Go is deliberately absent — its stop set is
    function_declaration / method_declaration, neither of which the
    grammar admits inside a function body, so the surcharge is unreachable
    there and a nested Go function takes the lambda path.

Both verified by perturbation: with *depth += 1 turned into
*depth += 0, the cross-language test reports all 11 rows and the
whole-suite run fails exactly 10 tests out of 3,110 — precise rather
than coarse.

The second commit is a test-audit fix: the surcharge table asserted only
the two costs, so a row whose snippet stopped parsing would keep
reporting 1 and 2 off an error-recovery tree (verified — appending
garbage to the C source left the test green). Each row now rejects an
ERROR-node parse first, which matters for the rows leaning on GNU nested
functions, Lua's nested function statement and an iRules proc inside
a proc.

Baseline

RubyCode::compute's halstead.effort crosses its recorded
.bca-baseline.toml value twice here — 52,423 → 53,271 on the added
argument, then → 53,802 on the case-default fix. Refreshed with
make self-scan-write-baseline-headroom in each of those commits, per
the baseline-refresh discipline in AGENTS.md; skipping it turns
make self-scan red on a clean checkout for everyone, not just the
author.

Still open elsewhere

The Node::parent climbs that remain are all outside cognitive and
outside every probed walk: the JS/TS is_func / is_closure walk,
Elixir's Npa / Npm / get_func_space_name / suppression-marker
lookups, the Halstead get_op_type getters, and per-node lookups in
several loc, npa / npm, cyclomatic and checker arms. That
last group was missing from the inventories in the CHANGELOG and
benchmarking.md, which read as exhaustive; both now say otherwise.
All stay tracked in #1088, which has been updated to record that its
deferred increment_function_depth item is done.

Validation

  • make pre-commit green on the final tree.
  • cargo test --workspace --all-features: 3,110 + 75 passing, 0 failed.
  • Reviewed with the review, rust-optimize, audit-tests and
    code-review passes; every finding is folded into the four commits.
    The Kotlin / Ruby fix is covered by five existing tests, confirmed by
    perturbation (swapping in the grandparent fails exactly those five
    and nothing else).

Fixes #1062

dekobon added 4 commits July 27, 2026 17:46
increment_function_depth asked every function node whether a function
encloses it by climbing with Node::parent, which tree-sitter resolves
by descending from the root. That is O(depth) per step, so Cognitive
stayed O(depth^2) on nested definitions across the 19 languages that
call it -- the last of this issue's four parent lookups, deferred out
of #1084 because the ancestor-chain mechanism landed there.

The scan now reads the chain the walker hands down, so a step is a
slice index. All 19 call sites already had an `ancestors` value in
scope, so no per-language logic moves and values are unchanged.

The new cognitive/nested-fn scaling probe fits time ~ depth^k at 2.04
against the climb and 1.21 against the chain; at depth 4000 the walk
drops from ~150 ms to ~16 ms. The existing nom/nested-fn probe never
reached this function -- it selects Metric::Nom -- and its shape gains
one `if` per level so cognitive scores n(n+1)/2 on it, which moves if
function depth stops being counted.

Adds the cross-language surcharge table for the eleven languages whose
function-depth arm had no test of its own, and refreshes the self-scan
baseline for RubyCode::compute, whose halstead.effort crosses its
recorded value on the extra argument.

Fixes #1062
The cross-language function-depth table asserted only the two costs, so
a row whose snippet stopped parsing would keep reporting 1 and 2 off an
error-recovery tree. Verified: appending garbage to the C source leaves
the test green. Several rows lean on grammar corners -- GNU nested
functions, a Lua function statement inside another, an iRules proc
inside a proc -- which is exactly where a grammar bump lands.

Also renames the loop's shadowed cost bindings and corrects a CHANGELOG
sentence that described the old climb as reading the ancestor chain,
which is what the fix does, not what it replaced.
Kotlin's `when`-default check and Ruby's `case`-default check each
called `node.parent()` once per `else` node while the walker's ancestor
chain was already in scope. `Node::parent` is `O(depth)` -- tree-sitter
resolves it by descending from the root -- so a deeply nested
`if`/`else` chain stayed quadratic in the very metric the previous
commit makes linear on nested definitions.

Both now read `ancestors.parent(node)`, which is the same answer as a
slice index. Found by code review of the function-depth change; these
were the last two `Node::parent` calls left anywhere in `cognitive`.

Refreshes the baseline: the Ruby edit moves `RubyCode::compute`'s
recorded `halstead.effort` from 53271 to 53802.
Review follow-ups to the #1062 change, none behavioural:

- The CHANGELOG claimed `cognitive`'s parent lookups were gone while
  the Kotlin and Ruby `else` climbs remained. True as of the previous
  commit; reworded to say so accurately.
- The "remaining climbs" lists in the CHANGELOG and in
  benchmarking.md read as exhaustive but omitted per-node
  `Node::parent` calls in several `loc`, `npa` / `npm`, `cyclomatic`
  and `checker` arms. Marked illustrative and extended.
- "19 languages" is the call-site count; the JS-family macro expands
  to four more, so it is 19 sites across 22 languages.
- `PROBES`' doc comment still named one metric control after
  `nom/nested-fn` became a second.
- The surcharge table's `Objc` row reused the C source, duplicating
  the row above and leaving `MethodDefinition` -- the one stop
  `objc.rs` adds over `c.rs` -- untested. It now uses a real
  `@implementation`.
@codecov

codecov Bot commented Jul 28, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 98.10%. Comparing base (415ab6a) to head (07c4ac1).

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #1095      +/-   ##
==========================================
+ Coverage   98.09%   98.10%   +0.01%     
==========================================
  Files         272      272              
  Lines       67426    67527     +101     
  Branches    66996    67097     +101     
==========================================
+ Hits        66143    66249     +106     
+ Misses        873      868       -5     
  Partials      410      410              
Flag Coverage Δ
rust 98.09% <100.00%> (+0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
src/metrics/cognitive.rs 99.85% <100.00%> (+<0.01%) ⬆️
src/metrics/cognitive/bash.rs 100.00% <100.00%> (ø)
src/metrics/cognitive/c.rs 100.00% <100.00%> (ø)
src/metrics/cognitive/cpp.rs 100.00% <100.00%> (ø)
src/metrics/cognitive/csharp.rs 100.00% <100.00%> (ø)
src/metrics/cognitive/go.rs 97.91% <100.00%> (+0.04%) ⬆️
src/metrics/cognitive/groovy.rs 100.00% <100.00%> (ø)
src/metrics/cognitive/irules.rs 94.11% <100.00%> (+0.11%) ⬆️
src/metrics/cognitive/java.rs 100.00% <100.00%> (ø)
src/metrics/cognitive/kotlin.rs 100.00% <100.00%> (ø)
... and 11 more

... and 5 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Two coverage gaps, one visible to Codecov and one not.

The visible one: the cross-language surcharge table accumulated
failures into a `wrong` vector and asserted it empty, so the `push`
arm and the `join` in the assert message were a branch no passing run
ever takes -- six of the eight uncovered lines in the PR, and
unreachable by construction. Comparing the measured column against the
expected one as whole vectors keeps the "report every language at
once" property, drops the hand-rolled formatting, and shows both
columns on failure.

The invisible one: `a_known_chain_answers_exactly_what_climbing_
answers` pins the equivalence this change rests on, but its C, Java,
Python and Elixir fixtures contain no function nested inside a
function, no Kotlin `when`/`else`, and no Ruby `case`/`else` -- the
three shapes #1062 added as consumers. Parity over the machinery is
not parity over the shape a caller asks about. Adds a fixture for
each, verified against their AST dumps.

`assert_parity` now also takes the kinds that must appear inside
another node of the same kind. `visited > 20` does not keep a fixture
honest: a grammar bump that flattened the nesting would leave a large,
clean-parsing tree exercising nothing while every assertion still
passed. Verified by flattening the Rust fixture to two sibling `fn`s,
which trips the new guard.
@dekobon

dekobon commented Jul 28, 2026

Copy link
Copy Markdown
Owner Author

Test coverage follow-up (07c4ac15)

Patch coverage was 93.88% — 92 hits, 4 misses, 2 partials — and all six
sat in one place: the failure-reporting branch of
function_depth_surcharge_holds_across_languages.

L9840 partial: if (flat_cost, nested_cost) != (1, 2) {
L9841 miss:        wrong.push(format!(
L9842 miss:            "{lang:?}: expected 1 then 2, got …"
L9843 miss:        ));
L9844 partial: }
L9850 miss:    wrong.join("\n"),

A hand-rolled diagnostic that runs only when the test fails is
unreachable on a green run by construction, so the fix is to not write
it. The table now builds a measured column and compares it against
expected with one assert_eq!, which keeps the "show every language
at once" property that the accumulator existed for and improves the
failure output — both columns, so you can see which rows are right:

left:  [(C, 1, 1), (Objc, 1, 1), (Mozcpp, 1, 1), …]
right: [(C, 1, 2), (Objc, 1, 2), (Mozcpp, 1, 2), …]

The gap the report could not see

Chasing those lines sent me back to what actually proves this change
correct. a_known_chain_answers_exactly_what_climbing_answers pins the
equivalence the whole PR rests on — a known chain answers exactly what
Node::parent climbing answers — over C, Java, Python and Elixir
fixtures. None of them contains a function nested inside a function,
Kotlin's when/else, or Ruby's case/else: the three shapes this
PR added as consumers. Parity over the machinery is not parity over the
shape a caller asks about. Added a fixture for each, checked against
their AST dumps rather than assumed.

Fixture rot

visited > 20 does not keep a fixture honest. A grammar bump that
flattened the nesting a row was added for would leave a large,
clean-parsing tree that exercises nothing, while every parity assertion
kept passing. assert_parity now takes the kinds that must appear
inside another node of the same kind; flattening the Rust fixture to
two sibling fns trips it:

rust: no `function_item` sits inside another `function_item`, so the
fixture no longer exercises the nesting it was added for

make pre-commit green.

@dekobon
dekobon merged commit 0e630cd into main Jul 28, 2026
39 checks passed
@dekobon
dekobon deleted the fix/issue-1062 branch July 28, 2026 02:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

perf(metrics/cognitive): O(nodes x depth) parent() lookup per node

1 participant