perf(metrics): retire the remaining Node::parent climbs (#1088) - #1097
Open
dekobon wants to merge 2 commits into
Open
perf(metrics): retire the remaining Node::parent climbs (#1088)#1097dekobon wants to merge 2 commits into
dekobon wants to merge 2 commits into
Conversation
`Node::wraps_any` — reached from `is_child` and `has_sibling`, and so from most languages' checkers and getters — walked a node's children with `child(0)` + `next_sibling()`. #217 chose that form over the cursor iterator to avoid a `TreeCursor` heap allocation per call, on the stated premise that a sibling step is O(1). It is not. `ts_node_next_sibling` resolves the parent before it can find a sibling, and `tree_sitter` stores no parent pointer — it descends from the root. Each step therefore cost O(depth) and the scan O(children x depth), which is the same defect #1084 removed from the predicates that ask for an ancestor outright, one level further down. The cursor iterator is O(children) after one allocation and measures faster on ordinary input as well as pathological input: a metric walk over the 384-file pdf.js corpus drops from ~443 ms to ~370 ms. The equivalence test keeps its node-by-node comparison but flips which side is the reference: the retired sibling chain is now the reference implementation, so what it pins is that the swap did not change the set of children enumerated.
#1084 gave the metric walk an ancestor chain and threaded it through three hot predicates; #1062 took the fourth. Four call sites stayed on `Ancestors::unknown()`, each correct but paying `Node::parent`'s O(depth) per step, and each therefore a latent quadratic on a deeply nested file. Three are threaded here: - the JS-family name-binding walk behind `Checker::is_func` / `is_closure`, plus the JS-family space-name getters, which climbed to read the `pair` or `variable_declarator` that names an anonymous function; - Elixir's `Getter::get_func_space_name`; - the suppression-marker scan. `is_func`, `is_closure`, `get_func_name`, `get_func_space_name`, and `NArgs::compute` gain an `Ancestors` parameter. All are `pub(crate)`, so the published API is unchanged. Ruby's `is_closure` reads it too: its `{ … }` block is a closure only when its parent is not the `Lambda` that already counted it, which makes it the one non-JS predicate of that pair that asks for an ancestor. The three walks that reached these predicates without a chain — `ops_inner`, `act_on_node` (behind `bca function`), and `suppression_markers` — now maintain one by the same truncate/push rule `metrics_inner` uses, constructed through `Ancestors::checked` so a bookkeeping slip fails a debug build rather than feeding a predicate a wrong ancestor in silence. `act_on_node` seeds its chain with the subtree root's own ancestry rather than empty, so it stays correct if it is ever called below the tree root. The fourth site, Elixir's `Npm` / `Npa`, wanted deleting rather than threading. Both `compute` bodies opened with `is_func_space_with_code`, which cost a source-text keyword scan per node and, for `def`-shaped calls, the `quote`-template ancestor walk. Its answer was always discarded: `elixir_is_class_macro` is exactly `defmodule`, so the keyword check immediately below admits precisely the nodes the classifier would have and rejects every node the walk was consulted for. Removing the call removes the work instead of making it cheaper, and leaves `Npm::compute` / `Npa::compute` with the signature they had. Guarded by a new probe pair. `nom/nested-arrow` fits `time ~ depth^k` at 1.97 before and 1.03 after, and at depth 4000 `nom` over nested arrow functions drops from ~17.6 s to ~6.3 ms; `nom/nested-declared-function` is the same nesting written with `function` declarations, which `is_func` answers from the node's own kind, and holds at 1.08 either side. Most of that win came from the child scan retired in the previous commit: threading the chain alone left the probe at 1.97, because the walk is two steps long on that shape and the O(depth) term had moved into `Node::wraps_any`. A chain-fed predicate is only as linear as the primitives it calls. Metric values are unchanged for every language. The remaining per-node `node.parent()` calls in the Halstead getters, `src/metrics/abc/`, and the `loc` / `npm` / `npa` / `cyclomatic` bodies are tracked in #1096. Fixes #1088
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #1097 +/- ##
==========================================
- Coverage 98.10% 98.08% -0.03%
==========================================
Files 272 272
Lines 67527 67776 +249
Branches 67097 67346 +249
==========================================
+ Hits 66249 66477 +228
- Misses 868 888 +20
- Partials 410 411 +1
Flags with carried forward coverage won't be shown. Click here to find out more.
🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #1088.
Retires the four
Node::parentclimbs #1084 deferred — and one causethe issue did not name, which turned out to be the one that mattered.
What changed
perf(node): scan a node's children with a cursor—Node::wraps_any,reached from
is_childandhas_siblingand so from most languages'checkers and getters, walked a node's children with
child(0)+next_sibling(). #217 chose that form over the cursor iterator to avoida
TreeCursorallocation, on the stated premise that a sibling step isO(1). It is not:ts_node_next_siblingresolves the parent first, andtree_sitterstores no parent pointer — it descends from the root. Thescan was
O(children × depth), making every callerO(depth)per node.perf(metrics): thread the chain through the last climbs— three ofthe four listed sites take an
Ancestorsparameter (JS-familyis_func/is_closureand the JS space-name getters, Elixir'sget_func_space_name, the suppression scan), plus Ruby'sis_closure,which the first pass missed.
ops_inner,act_on_node, andsuppression_markersmaintain chains of their own. All widened itemsare
pub(crate); the published API is unchanged.The fourth site wanted deleting, not threading. Elixir's
Npm/Npacomputeopened withis_func_space_with_code, whose answer thedefmodulekeyword check immediately below always discarded —elixir_is_class_macrois exactlydefmodule, so the classifieradmits precisely what that check admits, and every node whose
classification consulted the
quote-template ancestor walk is rejectedby it regardless. Removing the call removes the work rather than making
it cheaper, and leaves both signatures untouched.
Measurements
The issue's "worth measuring first" instinct is what caught the real
cause: threading the chain into the JS predicates left the probe at
k = 1.97. The walk is two steps long on that shape, so it should haveflattened — the remaining
O(depth)was one level down, inwraps_any. A chain-fed predicate is only as linear as theprimitives it calls.
nom/nested-arrow(JS, one arrow per level)k = 1.97, 17.6 s @ depth 4000k = 1.03, 6.3 msnom/nested-declared-function(shape control)k = 1.08k = 1.08Ordinary input benefits too: a metric walk over the 384-file
pdf.jscorpus drops from ~443 ms to ~370 ms.
The probe deliberately uses block-bodied arrows, where the walk stops
in two steps. A chain of expression-bodied arrows
(
a => b => c => …) is the one shape no chain can flatten — no stopkind intervenes, so the walk itself is
O(depth)long. Documented atthe macro rather than silently left.
Metric values
Unchanged for every language. Full workspace suite green with no
.snap.newunder thebig-code-analysis-outputsubmodule, so nosubmodule bump is needed.
Tests
Chain-vs-climb parity for every newly threaded consumer, each verified
by perturbation to fail against a wrong ancestor read:
is_func/is_closureacross all four JS grammars plus Ruby,get_func_space_nameacross all four JS grammars plus Elixir,
has_sibling, a JS fixtureadded to the existing
a_known_chain_answers_exactly_what_climbing_answers,the
bca functionspan walk, andact_on_nodeon a subtree.wraps_any's equivalence test keeps its node-by-node comparison with thereference flipped — the retired sibling chain is now the reference, so
what it pins is that the swap did not change the set of children
enumerated.
Two
Npm/Npaparity tests written during the first pass werevacuous and were replaced: they asserted that a known chain and a
climb agree, over a fixture where the ancestor answer never reached the
output — which is the redundancy described above, showing up as a test
that could not fail. The replacements pin counts over a
defmodulenested inside a
quote.Follow-up
#1096 tracks the remaining per-node
node.parent()calls (Halsteadget_op_typegetters,src/metrics/abc/, theloc/npm/npa/cyclomaticbodies).docs/development/benchmarking.mdis updated topoint there, and the
Ancestors::unknown()call sites that remain aredeliberate and documented at each.
Validation
make pre-commitgreen, including the self-scan gate at both tiers..bca-baseline.tomlnet shrank — removing the redundant Elixirclassifier and reverting the unused
Npm/Npaparameter took severalfunctions back below their recorded values.