Skip to content

fix(phi-scan): corroborate every v2 field number against a published HL7 v2.5.1 (PHI-SCAN-RESIDUALS) - #37

Merged
NSchatz merged 3 commits into
mainfrom
phi-scan-residuals-field-numbers
Aug 8, 2026
Merged

fix(phi-scan): corroborate every v2 field number against a published HL7 v2.5.1 (PHI-SCAN-RESIDUALS)#37
NSchatz merged 3 commits into
mainfrom
phi-scan-residuals-field-numbers

Conversation

@NSchatz

@NSchatz NSchatz commented Aug 8, 2026

Copy link
Copy Markdown
Contributor

What and why

The segment field list is the detector in this package. The cross-cutting SSN/email floor finds zero in the tracked files carrying inline PID| literals, so a wrong field number is not cosmetic: it is either a missed leak (a PHI-bearing field never read) or a false positive on a clinical field. IN1-17 shipped once as a telephone field and is Insured's Relationship To Patient, so a coded relationship was reported as a phone number.

Fifteen numbers had never been checked against any published source: the whole GT1 row, plus PID-6/9/19/20, NK1-30/33 and IN1-18/19.

What was grounded, against what

All fifteen (and every other row) are checked against the HL7 v2.5.1 segment attribute tables: Chapter 3, Patient Administration (PID 3.4.2, NK1 3.4.5, PV1 3.4.3) and Chapter 6, Financial Management (GT1 6.5.5, IN1 6.5.6), parsed out of the published chapter HTML rather than read through a summarizer. Every row was then cross-checked field by field against a second, independently published, version-pinned artifact: the HAPI HL7 v2 generated structures for v2.5.1. The two agree on the number, the name and the data type of every row, and on the segment lengths (PID 39, NK1 39, GT1 57, IN1 53).

  • None of the fifteen was wrong. A null result is the report: the risk was real either way, and which direction it fell in was not knowable until it was checked.
  • One citation was wrong, and only reading the standard could have shown it: GT1 was cited as clause 6.5.4 and is 6.5.5. A wrong clause is how the next reader confirms a number against the wrong table and comes away more confident.
  • The version is load-bearing, measured: PID-9, PID-19 and PID-20 are withdrawn from v2.7 onward, so grounding against a later version's table yields a confident wrong answer rather than an error.

Every row now carries its v2.5.1 item number, the standard's own stable identifier for an element.

What it bought as detection

Seven fields a review had measured as reported-clean are read now, because the same tables ground them: NK1-26, NK1-31, NK1-32, NK1-37, GT1-2, GT1-4, IN1-49.

A union, never a replacement, and the superset is proved cell by cell rather than asserted: every one of the 28 cells that reported before still reports, and every deliberate non-cell (IN1-17, IN1-7, PID-10, PID-18, NK1-3, GT1-11, PV1-7, PV1-19) still reports nothing. pnpm phi-scan exits 0 over the tracked corpus before and after, so nothing was traded for it. NK1-37 is the sharpest of the seven: an undashed contact SSN the floor is structurally blind to, pinned in both polarities in one case.

PHI_SEGMENTS is derived from the union of all five field tables instead of one of them. Read from one table, a segment added to another alone would never be located and nothing would report: no error, no warning. Both expressions answer identically today, so it guards the next edit.

Both polarities, everywhere

The coverage case is 35 positives and 9 negative controls in the same case, so a wholesale detector failure cannot produce the zeros. The positive locator assertion is the exact segment=<SEG>-<N> token, because PID-3 alone is a prefix of PID-33 and would let a renumbering pass. Bite was checked by mutation, not assumed: dropping NK1-37 reds 2 cases, renumbering PID-6 to PID-8 reds the coverage case, and re-adding IN1-17 as a phone reds it with the original defect's own message.

Reviews

Two conformance-refuter passes. Pass 1 returned REFUTED on an INTRODUCED major of exactly the class this change is about: the nine negative-control rows had been written from recall rather than extracted, and PV1-7 was given as item 00147 (it is 00137; 00147 is PV1-17). Eight of nine were right, which is the finding, because being right was luck and being wrong was invisible. Fixed in a follow-up commit, with all 44 rows extracted and verified. Pass 2 re-verified 44/44 independently and returned NOT REFUTED; its one remaining minor (a control sentence that did not name its comparand) is corrected in the last commit.

Two PRE-EXISTING minors are for the backlog and are not touched here: the coverage case is durable in the narrowing direction only, and the test file's assemble-at-runtime banner is falsified by literals already present at base.

Verification

scripts/verify.sh transform green. Steps that actually ran: audit, licenses, typecheck, lint, format:check, phi-scan, check:no-emdash, check:no-internal-refs, test:coverage (26 files, 428 tests, coverage 97.63/92.09/100/98.69), build, attw. check:agent-notes is gate-shaped and not on the umbrella's ladder, so it was run by hand: exit 0.

CLAUDE.md is untouched at 21,891 bytes. The narrative went to documentation/agent-notes.md under the existing PHI-scanner section, as a ### subsection, so the two-file agent-instruction contract still holds and no trap was deleted or relocated away.

NSchatz added 3 commits August 8, 2026 18:39
…HL7 v2.5.1, and read seven fields it disclosed as unread (PHI-SCAN-RESIDUALS)

The segment field list IS the detector in this package: the cross-cutting
SSN/email floor finds zero in the tracked files carrying inline `PID|`
literals, so a wrong field number is a MISSED LEAK or a FALSE POSITIVE ON A
CLINICAL FIELD. Fifteen numbers had never been checked against any published
source: the whole GT1 row, plus PID-6/9/19/20, NK1-30/33 and IN1-18/19.

All fifteen are now checked against the v2.5.1 SEGMENT ATTRIBUTE TABLES in
Chapter 3 (PID 3.4.2, NK1 3.4.5) and Chapter 6 (GT1 6.5.5, IN1 6.5.6), and
cross-checked field by field against a second version-pinned publication.
NONE OF THE FIFTEEN WAS WRONG. One citation was: the source said GT1 was
clause 6.5.4. Three of these rows read differently in a later v2 (PID-9,
PID-19 and PID-20 are withdrawn from v2.7 onward), so the version is
load-bearing rather than decorative. Every row now carries its v2.5.1 ITEM
NUMBER, the standard's own stable identifier for an element.

Seven fields a refuter had measured as reported-clean are read now, because
the same published tables ground them: NK1-26, NK1-31, NK1-32, NK1-37, GT1-2,
GT1-4 and IN1-49. A UNION, never a replacement, and the superset is pinned
cell by cell: every field that reported before still reports, every deliberate
non-field (IN1-17, IN1-7, PID-10, PID-18, NK1-3, GT1-11, PV1-19) still reports
nothing, and the tracked corpus stays clean on `pnpm phi-scan`.

`PHI_SEGMENTS` is derived from the union of all five field tables instead of
one of them. Read from one table, a segment added to another alone would never
be located and nothing would report: no error, no warning. Both expressions
answer identically today, so it guards the next edit rather than changing this
one.

Grounding, sources, the version trap and what is still NOT ground truth here:
documentation/agent-notes.md, under the PHI scanner section.
…ative-control rows the first draft asserted from recall (PHI-SCAN-RESIDUALS)

A refuter measured `PV1-7` written into the suite's negative-control list as
v2.5.1 item `00147`. PV1-7 is `00137` (Attending Doctor); `00147` is PV1-17
(Admitting Doctor).

The 35 READ rows were extracted from the published attribute tables
mechanically. The nine negative-control rows in the same case were not: they
were written from recall. Eight of the nine happened to be right, which is the
finding rather than a mitigation, because being right was luck and being wrong
was invisible: PV1 is read by nothing and the case asserts a clean result, so
no detection changed either way. A wrong citation costs nothing at all until
the next reader re-checks a row against it, lands on a DIFFERENT element, and
comes away confirmed. That is the failure the item numbers were added to
prevent, reproduced by the change that added them.

All 44 rows are extracted and verified now, with a negative control that
re-checks every row against the WRONG segment's table and mismatches 43 of 44.
The rule is recorded where the table lives: extract an item number or do not
write one.

Also corrected: a sentence in the notes said no tracked file carries a GT1
segment literal, and wrote the literal to say so. This file is inside the
scan's own corpus, so the sentence became the only occurrence of the thing it
said did not occur. It parsed to an empty segment and reported nothing, so it
was false rather than dangerous, but a note about a detector, inside the corpus
that detector reads, is itself an input.
…rrect two loose sentences about it (PHI-SCAN-RESIDUALS)

Prose only. No code, no test, no detection change.

A second refuter pass could not reproduce "44 of 44 against the published
tables, with a negative control that re-checks every row against the WRONG
segment's table and mismatches 43 of 44", and it was right that it could not.
43 was the count taken BEFORE `PV1-7` was corrected, when the wrong row
mismatched its own segment too, and the sentence never said WHICH table the
rows were re-checked against, so neither number was reproducible. Re-measured
and stated with its pairing: 44 of 44 rows match their own segment's table, and
re-running the same comparison with every row looked up in the PV1 table
mismatches 42 of 44, the only two matches being PV1-7 and PV1-19, which
genuinely are PV1 rows.

A control whose comparand is unstated is not a control. It is also a scratch
measurement rather than a test, and is now recorded as a citation to repeat
rather than as a gate that fires.

Also corrected: the note said the stray GT1 literal "parsed to an empty
segment". It opened a segment whose only field is GT1-1, which no table reads,
so it reported nothing. The conclusion was right and the mechanism was not.
@NSchatz
NSchatz merged commit 990be26 into main Aug 8, 2026
9 checks passed
@NSchatz
NSchatz deleted the phi-scan-residuals-field-numbers branch August 8, 2026 19:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant