diff --git a/.changeset/olive-hounds-repeat.md b/.changeset/olive-hounds-repeat.md new file mode 100644 index 0000000..dda65c1 --- /dev/null +++ b/.changeset/olive-hounds-repeat.md @@ -0,0 +1,11 @@ +--- +"@cosyte/transform": patch +--- + +Corroborate every HL7 v2 field number the PHI scanner reads against a published HL7 v2.5.1, and read seven PHI-bearing fields it previously disclosed as unread (PHI-SCAN-RESIDUALS). + +The segment field list is the detector in this package: the cross-cutting SSN/email floor finds nothing in a corpus whose messages are inline v2 string literals, so a wrong field number is either a missed leak or a false positive on a clinical field. Fifteen numbers had never been checked against any published source (the whole GT1 row, plus PID-6, PID-9, PID-19, PID-20, NK1-30, NK1-33, IN1-18 and IN1-19). All fifteen are now checked against the v2.5.1 segment attribute tables in Chapter 3 (PID 3.4.2, NK1 3.4.5) and Chapter 6 (GT1 6.5.5, IN1 6.5.6), cross-checked against a second version-pinned publication, and none of them was wrong. The GT1 clause citation was: it said 6.5.4. + +Every row in the scanner's coverage table and in the suite's coverage case now carries the v2.5.1 item number the field number was corroborated by, which is the standard's own stable identifier for an element. + +Seven fields that a review had measured as reported-clean are read now, because the same published tables ground them: NK1-26, NK1-31, NK1-32, NK1-37, GT1-2, GT1-4 and IN1-49. This is a union with the previous list rather than a replacement, and the superset is pinned cell by cell: every field that reported before still reports, every deliberate non-field (IN1-17, IN1-7, PID-10, PID-18, NK1-3, GT1-11, PV1-19) still reports nothing, and the tracked corpus stays clean. `PHI_SEGMENTS` is derived from the union of all five field tables instead of one of them, so a segment added to a single table can no longer go silently unlocated. diff --git a/documentation/agent-notes.md b/documentation/agent-notes.md index a633715..0351514 100644 --- a/documentation/agent-notes.md +++ b/documentation/agent-notes.md @@ -454,8 +454,8 @@ leading 8 digits of a TS), ids (CX-1 across `~` repetitions, with an `SS` type c 9-digit value named as an SSN), addresses (XAD 1/2/3/5) and phones (XTN components with 4+ digits). **▶ EVERY FIELD NUMBER IS CITED TO HL7 v2.5.1 BY CHAPTER AND CLAUSE IN THE SOURCE, AND THE REASON IS -A MEASURED DEFECT: an uncited table produced `IN1-17` as a telephone field.** IN1-17 is *Insured's -Relationship To Patient*, so a SNOMED relationship code was reported as a phone number, and the +A MEASURED DEFECT: an uncited table produced `IN1-17` as a telephone field.** IN1-17 is _Insured's +Relationship To Patient_, so a SNOMED relationship code was reported as a phone number, and the remedy that diagnostic steered a developer toward was a global `PHONE` clearance of that digit string. **IN1 carries no insured telephone at all**; IN1-7 is the payer's. Found by the refuter. @@ -481,17 +481,100 @@ a reader the pass covers "member id"). **A negative list of that shape cannot be every clause of every segment would have to appear on it. The banner in `scripts/phi-scan.ts` now enumerates EXACTLY the fields that are read, and says that anything not named is not checked. That claim is checkable; the other one was not. **Correct it by narrowing the claim, never by silently -adding a field number.** - -**▶ PROVENANCE, RECORDED FIELD BY FIELD, BECAUSE "cross-corroborated in repo" WAS ITSELF MEASURED AS -AN OVERCLAIM.** The numbers are asserted from HL7 v2.5.1 and were **not** checked against a published -copy of the standard. **13 of the 28 are corroborated in-repo** (PID-3/5/7/11/13/14, -NK1-2/4/5/6/16, IN1-16/36) by `src/messages/patient.ts`, `src/messages/related-person.ts` and the -vendored `@cosyte/hl7` type surface. **15 are corroborated by nothing here at all, and that is where -the residual risk sits: the WHOLE GT1 row** (3, 5, 6, 7, 8, 12, 19, which no in-repo source mentions -even once), plus PID-6, PID-9, PID-19, PID-20, NK1-30, NK1-33, IN1-18 and IN1-19. One number was -wrong on the way here (IN1-17). That is why the table is deliberately narrow and why widening it -means citing a source, not adding a number. +adding a field number.** **Those seven are READ as of 2026-08-08**, because the corroboration below +grounds them; the rule that produced them is unchanged and is the durable half. + +### The field numbers, corroborated against a published v2.5.1 + +**▶ THE 15 THAT WERE OWED ARE PAID, AND NONE OF THEM WAS WRONG.** The provenance note here used to +say the numbers were asserted from v2.5.1 and **not** checked against a published copy: 13 of 28 were +corroborated in-repo only (against `src/messages/patient.ts`, `src/messages/related-person.ts` and +the vendored `@cosyte/hl7` type surface) and **15 by nothing at all** (the whole GT1 row, plus PID-6, +PID-9, PID-19, PID-20, NK1-30, NK1-33, IN1-18 and IN1-19). All 15 are now checked, and all 15 are +correct as used. **A null result is the report**: the risk was real either way, because an ungrounded +number is a MISSED LEAK or a FALSE POSITIVE ON A CLINICAL FIELD and which one it is cannot be known +until it is checked. `IN1-17` is the proof that the risk was not theoretical. + +**Sources, both named so a reader can repeat this rather than trust it.** (1) The HL7 v2.5.1 standard +text, read as the SEGMENT ATTRIBUTE TABLES ("HL7 Attribute Table - PID", and so on): Chapter 3, +Patient Administration, PID 3.4.2 and NK1 3.4.5, at `www.hl7.eu/HL7v2x/v251/std251/ch03.html`; and +Chapter 6, Financial Management, GT1 6.5.5 and IN1 6.5.6, at the same path with `ch06.html`. Read +2026-08-08. (2) A second, independently published, version-pinned artifact, checked field by field: +the HAPI HL7 v2 **generated structures for v2.5.1**, +`hapifhir.github.io/hapi-hl7v2/v251/apidocs/ca/uhn/hl7v2/model/v251/segment/.html`. The two +agree on the number, the name and the data type of every row, and on the segment lengths (PID 39, +NK1 39, GT1 57, IN1 53). The tables were parsed out of the HTML rather than read through a +summarizer, so the extraction is repeatable. + +**▶ ONE THING WAS WRONG, AND ONLY READING THE STANDARD COULD HAVE SHOWN IT: THE GT1 CLAUSE CITATION.** +The source said Chapter 6 **6.5.4**; GT1 is **6.5.5** in v2.5.1. Nothing detected differently because +of it, which is exactly why it survived: a wrong clause is how the NEXT reader "confirms" a number +against the wrong table and comes away more confident rather than less. + +**▶ THE VERSION IS LOAD-BEARING, AND THAT IS MEASURED.** Three of these rows read differently in a +later v2: **PID-9, PID-19 and PID-20 are WITHDRAWN from v2.7 onward**, and a published v2+ copy of +the same segment says so at exactly those numbers. Grounding against the wrong version's table yields +a confident wrong answer, not an error. The other tell is the data type: v2.5.1 says TS where a later +version says DTM, and IS/CE where a later one says CWE. **Cite the version, not just the segment.** + +**▶ AND THE ANTI-DRIFT DEVICE THAT OUTLIVES THIS SESSION: EVERY ROW NOW CARRIES ITS v2.5.1 ITEM +NUMBER**, in the banner and in the suite's coverage case. An item number is the standard's own stable +identifier for an element (PID-6 and NK1-26 share `00109` because they ARE the same element), so a +reader re-checking a row is checking the same element rather than a same-numbered one. + +**▶ AND THE DEVICE IMMEDIATELY PRODUCED ITS OWN DEFECT, WHICH IS THE MOST USEFUL THING IN THIS +SECTION.** The 35 READ rows were extracted from the attribute tables mechanically. The nine +NEGATIVE-CONTROL rows in the same case were not: they were written from recall, and a refuter +measured `PV1-7` as item `00147`. **PV1-7 is `00137` (Attending Doctor); `00147` is PV1-17 (Admitting +Doctor).** Eight of the nine happened to be right, which is the whole lesson: being right was luck, +and being wrong was invisible, because **no detection changes either way** (PV1 is read by nothing +and the case asserts a clean result). A wrong citation costs nothing until the next reader re-checks +a row against it, lands on a different element, and comes away CONFIRMED. All nine are extracted and +verified now: **44 of 44 rows (35 read, 9 negative-control) match the item number the published table +gives for their own segment.** The instrument was then checked against a WRONG table: re-running the +same comparison with **every** row looked up in the **PV1** table mismatches **42 of 44**, and the +only two that match are `PV1-7` and `PV1-19`, which genuinely are PV1 rows. **Extract an item number +or do not write one**, and never let "I checked the important ones" stand in for a list that says it +was all checked. + +**▶ AND THAT SENTENCE WAS ITSELF WRONG ONCE, WHICH IS WHY IT NOW NAMES THE PAIRING.** It first read +"mismatches 43 of 44" with no statement of WHICH table the rows were re-checked against. 43 was the +count taken **before** `PV1-7` was corrected, when the wrong row mismatched its own segment too, and +without the pairing named nobody could reproduce either number. A control whose comparand is unstated +is not a control. It is also a SCRATCH measurement, not a test: **nothing in this repository +re-derives it**, and it is recorded as a citation to repeat rather than a gate that fires. + +**What the corroboration then bought, as detection.** The seven fields the refuter had measured as +unread are read: NK1-26 `00109`, NK1-31 `00749`, NK1-32 `00750`, NK1-37 `00754`, GT1-2 `00406`, +GT1-4 `00408`, IN1-49 `01230`. **A UNION, never a replacement, and the superset is proved cell by +cell**: a probe of one violator per cell through the real CLI read `exit=1` on all 28 cells that +reported before and on all 7 that did not, with every deliberate non-cell (IN1-17, IN1-7, PID-10, +PID-18, NK1-3, GT1-11, PV1-19) still `exit=0`. **The tracked corpus stayed clean** (`pnpm phi-scan` +exit 0 before and after), so nothing was traded for it. NK1-37 is the sharpest of the seven: an +undashed contact SSN, which the cross-cutting floor is structurally blind to, pinned in both +polarities in one case. + +**▶ AND A SILENT-MISS SHAPE FIXED WHILE IT WAS CHEAP: `PHI_SEGMENTS` WAS `Object.keys(NAME_FIELDS)`.** +Read from one table, a segment added to (say) `ID_FIELDS` alone would never be LOCATED, so its fields +would never be read and nothing would report: no error, no warning, and a banner naming rows the +scanner cannot reach. It is the union of all five tables now. Both expressions answer identically +today, so this is a guard against the next edit rather than a behaviour change in this one. + +**▶ WHAT IS STILL NOT GROUND TRUTH HERE, SAID PLAINLY.** Nothing in this repository can check a field +number against the standard at CI time, and nothing should try: the corroboration is a citation a +human or an agent repeats, and the suite pins what the table makes the scanner DO, not whether the +table matches HL7. **The GT1 row also fires on nothing in this corpus**: measured at `ec4b95b`, no +tracked file carried a GT1 segment literal, so every GT1 cell is exercised only by synthetic segments +assembled in the suite. **A first draft of that sentence wrote the segment id followed by the field +separator, and this file is INSIDE the scan's own corpus**, so the sentence became the only +occurrence of the thing it said did not occur. It opened a segment whose only field is GT1-1, which +no table reads, so it reported nothing and was false rather than dangerous. **It did NOT parse to an +empty segment, and an earlier wording of this line said it did**: the shape is the one to watch, and +so is describing it loosely. **A note about a detector, +inside the corpus that detector reads, is itself an input.** Name the segment; do not spell a +literal. **Do not add a field number you cannot ground**, and if you cannot ground one, leave it out +and disclose it: an ungrounded number left OUT of the table is a better outcome than a confident +wrong one in it. **And a fourth recogniser limit the second pass found**: a literal backslash followed by `r` or `n` inside a field value ends the segment early, because the escaped separator is also the terminator. diff --git a/scripts/phi-scan.ts b/scripts/phi-scan.ts index 5c0b8b3..2e56d93 100644 --- a/scripts/phi-scan.ts +++ b/scripts/phi-scan.ts @@ -37,48 +37,115 @@ * Two successive refuter passes measured an EXHAUSTIVE NEGATIVE LIST of "what * this does not catch" incomplete, in the false-confidence direction, and the * second measured it incomplete AGAIN after it had been extended in answer to - * the first. Seven PHI-bearing v2.5.1 fields reported clean while the list - * claimed to be authoritative: NK1-26 (mother's maiden name), NK1-31 (contact - * telephone), NK1-32 (contact address), NK1-37 (contact SSN), GT1-2 - * (guarantor number), GT1-4 (guarantor spouse name) and IN1-49 (insured's id). + * the first. A negative list of that shape CANNOT be kept true: every clause + * of every segment of the standard would have to appear on it. So the claim is + * stated the only way that is checkable, as EXACTLY WHAT IS READ. Anything not + * named below IS NOT CHECKED. * - * A negative list of that shape CANNOT be kept true: every clause of every - * segment of the standard would have to appear on it. So the claim is stated - * the only way that is checkable, as EXACTLY WHAT IS READ. Anything not named - * below IS NOT CHECKED, including but not limited to the seven fields above. + * ▶ EACH ROW CARRIES THE v2.5.1 ITEM NUMBER, AND THAT IS THE ANTI-DRIFT + * DEVICE, NOT DECORATION. An item number is the standard's own stable + * identifier for an element, so a reader can re-check a row against a + * published copy without having to trust this comment, and a number that + * silently moved is visible rather than plausible. * - * PID-3, PID-19, PID-20 ....... id / SSN / driver's licence - * PID-5, PID-6, PID-9 ......... name / mother's maiden name / alias - * PID-7 ....................... date of birth - * PID-11 ...................... address - * PID-13, PID-14 .............. home / business telephone - * NK1-2, NK1-30 ............... name / contact person's name - * NK1-4 ....................... address - * NK1-5, NK1-6 ................ telephone - * NK1-16 ...................... date of birth - * NK1-33 ...................... next-of-kin identifiers - * GT1-3 ....................... guarantor name - * GT1-5 ....................... guarantor address - * GT1-6, GT1-7 ................ guarantor telephone - * GT1-8 ....................... guarantor date of birth - * GT1-12, GT1-19 .............. guarantor SSN / employee id - * IN1-16, IN1-18, IN1-19 ...... insured name / DOB / address - * IN1-36 ...................... insured's policy identifier + * ▶ EXTRACT AN ITEM NUMBER OR DO NOT WRITE ONE, AND THAT IS MEASURED HERE. + * The rows in this table were pulled out of the attribute tables + * mechanically; nine rows in the suite's NEGATIVE-CONTROL list were written + * from recall instead, and one of them (`PV1-7`) was wrong. Eight being right + * was luck, and the wrong one was invisible because it changed no detection at + * all. A wrong citation costs nothing until the next reader re-checks a row + * against it, lands on a DIFFERENT element, and comes away confirmed. + * + * PID-3 00106 .. Patient Identifier List ................. id + * PID-5 00108 .. Patient Name ............................ name + * PID-6 00109 .. Mother's Maiden Name .................... name + * PID-7 00110 .. Date/Time of Birth ...................... DOB + * PID-9 00112 .. Patient Alias ........................... name + * PID-11 00114 .. Patient Address ......................... address + * PID-13 00116 .. Phone Number - Home ..................... phone + * PID-14 00117 .. Phone Number - Business ................. phone + * PID-19 00122 .. SSN Number - Patient .................... id + * PID-20 00123 .. Driver's License Number - Patient ....... id + * NK1-2 00191 .. Name .................................... name + * NK1-4 00193 .. Address ................................. address + * NK1-5 00194 .. Phone Number ............................ phone + * NK1-6 00195 .. Business Phone Number ................... phone + * NK1-16 00110 .. Date/Time of Birth ...................... DOB + * NK1-26 00109 .. Mother's Maiden Name .................... name + * NK1-30 00748 .. Contact Person's Name ................... name + * NK1-31 00749 .. Contact Person's Telephone Number ....... phone + * NK1-32 00750 .. Contact Person's Address ................ address + * NK1-33 00751 .. Next of Kin/Associated Party's Ids ...... id + * NK1-37 00754 .. Contact Person Social Security Number ... id + * GT1-2 00406 .. Guarantor Number ........................ id + * GT1-3 00407 .. Guarantor Name .......................... name + * GT1-4 00408 .. Guarantor Spouse Name ................... name + * GT1-5 00409 .. Guarantor Address ....................... address + * GT1-6 00410 .. Guarantor Ph Num - Home ................. phone + * GT1-7 00411 .. Guarantor Ph Num - Business ............. phone + * GT1-8 00412 .. Guarantor Date/Time Of Birth ............ DOB + * GT1-12 00416 .. Guarantor SSN ........................... id + * GT1-19 00423 .. Guarantor Employee ID Number ............ id + * IN1-16 00441 .. Name Of Insured ......................... name + * IN1-18 00443 .. Insured's Date Of Birth ................. DOB + * IN1-19 00444 .. Insured's Address ....................... address + * IN1-36 00461 .. Policy Number ........................... id + * IN1-49 01230 .. Insured's ID Number ..................... id * * NO OTHER SEGMENT IS READ AT ALL: not PV1, ORC, OBR, OBX, RXA, SCH, TXA, and * not MSH. NO OTHER FIELD of the four segments above is read. Within a field, * only the components named in each `check…Field` are read. * - * ▶ PROVENANCE, SAID PLAINLY BECAUSE ITS ABSENCE WAS THE ROOT CAUSE OF A - * MEASURED DEFECT. The field numbers are asserted from HL7 v2.5.1 (PID and - * NK1 in Chapter 3, GT1 and IN1 in Chapter 6) and were cross-corroborated - * in-repo only, against `src/messages/related-person.ts` and the vendored - * `@cosyte/hl7` type surface. **They were NOT checked against a published copy - * of the standard.** One of them was wrong on the way here: IN1-17 shipped as - * a telephone field and is in fact Insured's Relationship To Patient, so a - * SNOMED code was reported as a phone number. That is why the table is - * deliberately narrow, why it is stated positively, and why widening it means - * citing a source rather than adding a number. + * ══════════════════════════════════════════════════════════════════════════ + * ▶ PROVENANCE. EVERY ROW ABOVE IS CORROBORATED AGAINST A PUBLISHED COPY OF + * HL7 v2.5.1, WHICH THE PREVIOUS DRAFT OF THIS BANNER SAID IT WAS NOT. + * + * Read as the SEGMENT ATTRIBUTE TABLES ("HL7 Attribute Table - PID", and so + * on) of the v2.5.1 standard text: Chapter 3, Patient Administration (PID + * §3.4.2, NK1 §3.4.5) and Chapter 6, Financial Management (GT1 §6.5.5, IN1 + * §6.5.6), at `www.hl7.eu/HL7v2x/v251/std251/ch03.html` and `…/ch06.html`, + * 2026-08-08. Every row was then cross-checked, field by field, against a + * SECOND independently published and version-pinned artifact: the HAPI HL7 v2 + * generated structures for v2.5.1, + * `hapifhir.github.io/hapi-hl7v2/v251/apidocs/…/model/v251/segment/.html`. + * The two agree on the number, the name and the data type of every row, and on + * the segment lengths (PID 39, NK1 39, GT1 57, IN1 53). + * + * WHAT THE CORROBORATION FOUND: + * - NO FIELD NUMBER WAS WRONG. The fifteen that had never been checked + * against a published source (the whole GT1 row, plus PID-6/9/19/20, + * NK1-30/33 and IN1-18/19) are all correct as used. That is a finding, not + * a formality: an ungrounded number is a missed leak or a false positive + * on a clinical field, and which one is not knowable until it is checked. + * - THE GT1 CLAUSE CITATION WAS WRONG, AND ONLY READING THE STANDARD SHOWED + * IT. This banner cited GT1 as Chapter 6 §6.5.4; GT1 is §6.5.5 in v2.5.1. + * A wrong clause is how the next reader "confirms" a number against the + * wrong table and comes away more confident, not less. + * - SEVEN FIELDS DISCLOSED AS UNREAD ARE NOW READ, because the same source + * grounds them: NK1-26, NK1-31, NK1-32, NK1-37, GT1-2, GT1-4 and IN1-49. + * That is a UNION with the previous list, never a replacement: every cell + * that reported before still reports, which the suite pins cell by cell. + * + * ▶ THE VERSION IS LOAD-BEARING, AND THAT IS MEASURED RATHER THAN CAUTIONARY. + * Three of these rows read DIFFERENTLY in a later v2: PID-9, PID-19 and PID-20 + * are WITHDRAWN from v2.7 onward, and a published v2+ copy of the same segment + * says exactly that at exactly those numbers. Grounding a number against the + * wrong version's table therefore yields a confident wrong answer rather than + * an error. This package targets v2.5.1, so cite v2.5.1. The other tell is the + * data type: v2.5.1 says TS where a later version says DTM, and IS/CE where a + * later one says CWE. + * + * ▶ AND THE RULE THAT OUTLIVES THIS CHANGE: DO NOT ADD A FIELD NUMBER YOU + * CANNOT GROUND. Measured cost of not citing at all: a first draft of this + * table mapped IN1-17 as a telephone field. IN1-17 is Insured's Relationship + * To Patient (CE, table 0063, item 00442), so a SNOMED relationship code was + * reported as a phone number, and the remedy that diagnostic steered a + * developer toward was a global `PHONE` clearance of that digit string. **IN1 + * carries no insured telephone at all**: IN1-7 (item 00432) is Insurance Co + * Phone Number, the PAYER's, an organisation's, so `IN1` is absent from + * `PHONE_FIELDS` deliberately rather than by omission. IN1-17 is the suite's + * negative control for exactly that reason. An ungrounded number left OUT of + * the table, disclosed, is a better outcome than a confident wrong one in it. * ══════════════════════════════════════════════════════════════════════════ * * ⚠ FOUR THINGS THE PASS CANNOT SEE EVEN INSIDE THE FIELDS IT READS. These @@ -1137,22 +1204,26 @@ function scanCommonShapes(path: string, content: string, allow: AllowList, hits: // carry is names, DOBs, MRNs, one undashed SSN in an `SS`-typed identifier, one // street address and two phone numbers, and the floor is blind to every one. -// ▶ EVERY FIELD NUMBER BELOW IS FROM HL7 v2.5.1, AND THE CLAUSE IS CITED BECAUSE -// AN UNCITED TABLE IS WHAT PRODUCES A WRONG ONE. PID is Chapter 3 §3.4.2, NK1 -// Chapter 3 §3.4.5, PV1 Chapter 3 §3.4.3, GT1 Chapter 6 §6.5.4 and IN1 Chapter 6 -// §6.5.6. Measured cost of not citing them: a first draft of this table mapped -// **IN1-17 as a telephone field**. IN1-17 is *Insured's Relationship To Patient* -// (CE, table 0063), so a SNOMED relationship code was reported as a phone -// number, and the remedy that diagnostic steered a developer toward was a global -// `PHONE` clearance of that digit string. **IN1 carries no insured telephone at -// all**: IN1-7 is the PAYER's number, an organisation's, so `IN1` is absent from -// `PHONE_FIELDS` deliberately rather than by omission. +// ▶ EVERY FIELD NUMBER BELOW IS CORROBORATED AGAINST A PUBLISHED HL7 v2.5.1, AND +// THE CLAUSE IS CITED BECAUSE AN UNCITED TABLE IS WHAT PRODUCES A WRONG ONE. PID +// is Chapter 3 §3.4.2, NK1 Chapter 3 §3.4.5, PV1 Chapter 3 §3.4.3, GT1 Chapter 6 +// §6.5.5 and IN1 Chapter 6 §6.5.6. **GT1 WAS CITED HERE AS §6.5.4 AND THAT WAS +// WRONG**; reading the standard is what showed it. The sources, the second +// artifact both were checked against, and the version trap are in the banner at +// the head of this file. Measured cost of not citing at all: a first draft of +// this table mapped **IN1-17 as a telephone field**. IN1-17 is *Insured's +// Relationship To Patient* (CE, table 0063, item 00442), so a SNOMED +// relationship code was reported as a phone number, and the remedy that +// diagnostic steered a developer toward was a global `PHONE` clearance of that +// digit string. **IN1 carries no insured telephone at all**: IN1-7 is the +// PAYER's number, an organisation's, so `IN1` is absent from `PHONE_FIELDS` +// deliberately rather than by omission. /** PHI-bearing fields per segment, by v2 field number (`PID-5` is index 5). */ const NAME_FIELDS: Record = { PID: [5, 6, 9], - NK1: [2, 30], - GT1: [3], + NK1: [2, 26, 30], + GT1: [3, 4], IN1: [16], }; const DOB_FIELDS: Record = { @@ -1163,24 +1234,45 @@ const DOB_FIELDS: Record = { }; const ID_FIELDS: Record = { PID: [3, 19, 20], - NK1: [33], - GT1: [12, 19], - IN1: [36], + NK1: [33, 37], + GT1: [2, 12, 19], + IN1: [36, 49], }; const ADDRESS_FIELDS: Record = { PID: [11], - NK1: [4], + NK1: [4, 32], GT1: [5], IN1: [19], }; const PHONE_FIELDS: Record = { PID: [13, 14], - NK1: [5, 6], + NK1: [5, 6, 31], GT1: [6, 7], // IN1 is absent on purpose: see the citation note above. }; -const PHI_SEGMENTS = Object.keys(NAME_FIELDS); +/** + * The segment ids `SEGMENT_OPENING` will locate at all, as the UNION of every + * table above. + * + * ▶ IT WAS `Object.keys(NAME_FIELDS)`, AND THAT IS A SILENT-MISS SHAPE EVEN + * THOUGH IT ANSWERS IDENTICALLY TODAY. Read from one table, a segment added to + * (say) `ID_FIELDS` alone is never located, so its fields are never read, and + * NOTHING reports: no error, no warning, and a coverage table above that names + * rows the scanner cannot reach. The union cannot do that. Today both + * expressions yield exactly `PID, NK1, GT1, IN1` in that order (`PHONE_FIELDS` + * is the only table missing one, and it is a subset), so this is a guard against + * the next edit rather than a behaviour change in this one. + */ +const PHI_SEGMENTS = [ + ...new Set([ + ...Object.keys(NAME_FIELDS), + ...Object.keys(DOB_FIELDS), + ...Object.keys(ID_FIELDS), + ...Object.keys(ADDRESS_FIELDS), + ...Object.keys(PHONE_FIELDS), + ]), +]; /** * Locate a segment literal by its `SEG|` opening. The leading boundary keeps diff --git a/test/scripts/phi-scan.test.ts b/test/scripts/phi-scan.test.ts index 08f7b9b..e8a1516 100644 --- a/test/scripts/phi-scan.test.ts +++ b/test/scripts/phi-scan.test.ts @@ -1319,77 +1319,143 @@ describe("phi-scan: the coverage claim is POSITIVE, and this is what makes it ch return parts.join("|"); }; - it("reads EVERY ONE of the 28 fields the banner names, and NONE that it does not", () => { - // ▶ ALL 28, NOT A SAMPLE, AND THAT IS THE FINDING. An earlier draft named + it("reads EVERY ONE of the 35 fields the banner names, and NONE that it does not", () => { + // ▶ ALL 35, NOT A SAMPLE, AND THAT IS THE FINDING. An earlier draft named // eight and its comment claimed a field added to the table without being // added to the banner would red here. Measured: adding PID-4 and PID-18 to // the table left the suite 75/75 GREEN. Worse is the NARROWING direction, // where the code drops a field and the banner goes on claiming it: 15 of - // the 28 fired in no test at all, so that shipped green too. Enumerating - // every named field is what makes the claim durable in both directions. + // the (then) 28 fired in no test at all, so that shipped green too. + // Enumerating every named field is what makes the claim durable in both + // directions. + // + // ▶ EACH ROW CARRIES THE v2.5.1 ITEM NUMBER THE FIELD NUMBER WAS CORROBORATED + // BY, and that is the point of this list rather than a note on it. Every + // number here is checked against a published copy of HL7 v2.5.1 (the + // segment attribute tables of Chapter 3 for PID/NK1 and Chapter 6 for + // GT1/IN1), cross-checked against a second version-pinned publication, and + // cited in `scripts/phi-scan.ts`. An item number is the standard's own + // stable identifier for an element, so a future reader re-checking a row is + // checking the SAME element rather than a same-numbered one. Fifteen of + // these had never been checked against any published source: the whole GT1 + // row, PID-6/9/19/20, NK1-30/33 and IN1-18/19. None of the fifteen was + // wrong. One CITATION was (GT1 is clause 6.5.5, not 6.5.4). const name = `${family}^${given}`; - const covered: [string, string, Record][] = [ - ["PID", "PID-3", { 3: "A77321^^^HOSP^MR" }], - ["PID", "PID-5", { 5: name }], - ["PID", "PID-6", { 6: name }], - ["PID", "PID-7", { 7: "19631207" }], - ["PID", "PID-9", { 9: name }], - ["PID", "PID-11", { 11: "9 Elm Rd^^Dayton^OH^45402" }], - ["PID", "PID-13", { 13: "9375550187" }], - ["PID", "PID-14", { 14: "9375550186" }], - ["PID", "PID-19", { 19: "555443210" }], - ["PID", "PID-20", { 20: "DL77321" }], - ["NK1", "NK1-2", { 2: name }], - ["NK1", "NK1-4", { 4: "9 Elm Rd^^Dayton^OH^45402" }], - ["NK1", "NK1-5", { 5: "9375550188" }], - ["NK1", "NK1-6", { 6: "9375550189" }], - ["NK1", "NK1-16", { 16: "19631207" }], - ["NK1", "NK1-30", { 30: name }], - ["NK1", "NK1-33", { 33: "A77321" }], - ["GT1", "GT1-3", { 3: name }], - ["GT1", "GT1-5", { 5: "9 Elm Rd^^Dayton^OH^45402" }], - ["GT1", "GT1-6", { 6: "9375550190" }], - ["GT1", "GT1-7", { 7: "9375550191" }], - ["GT1", "GT1-8", { 8: "19631207" }], - ["GT1", "GT1-12", { 12: "555443210" }], - ["GT1", "GT1-19", { 19: "EMP77321" }], - ["IN1", "IN1-16", { 16: name }], - ["IN1", "IN1-18", { 18: "19631207" }], - ["IN1", "IN1-19", { 19: "9 Elm Rd^^Dayton^OH^45402" }], - ["IN1", "IN1-36", { 36: "POL77321" }], + const covered: [string, string, string, Record][] = [ + ["PID", "PID-3", "00106", { 3: "A77321^^^HOSP^MR" }], + ["PID", "PID-5", "00108", { 5: name }], + ["PID", "PID-6", "00109", { 6: name }], + ["PID", "PID-7", "00110", { 7: "19631207" }], + ["PID", "PID-9", "00112", { 9: name }], + ["PID", "PID-11", "00114", { 11: "9 Elm Rd^^Dayton^OH^45402" }], + ["PID", "PID-13", "00116", { 13: "9375550187" }], + ["PID", "PID-14", "00117", { 14: "9375550186" }], + ["PID", "PID-19", "00122", { 19: "555443210" }], + ["PID", "PID-20", "00123", { 20: "DL77321" }], + ["NK1", "NK1-2", "00191", { 2: name }], + ["NK1", "NK1-4", "00193", { 4: "9 Elm Rd^^Dayton^OH^45402" }], + ["NK1", "NK1-5", "00194", { 5: "9375550188" }], + ["NK1", "NK1-6", "00195", { 6: "9375550189" }], + ["NK1", "NK1-16", "00110", { 16: "19631207" }], + // The seven below were DISCLOSED AS UNREAD by a refuter and asserted clean + // by this very case. The same published tables that corroborated the rest + // ground them, so they are read now: a UNION with the previous list, never + // a replacement. Every row that reported before this change still reports. + ["NK1", "NK1-26", "00109", { 26: name }], + ["NK1", "NK1-30", "00748", { 30: name }], + ["NK1", "NK1-31", "00749", { 31: "9375550188" }], + ["NK1", "NK1-32", "00750", { 32: "9 Elm Rd^^Dayton^OH^45402" }], + ["NK1", "NK1-33", "00751", { 33: "A77321" }], + ["NK1", "NK1-37", "00754", { 37: "555443210" }], + ["GT1", "GT1-2", "00406", { 2: "G77321" }], + ["GT1", "GT1-3", "00407", { 3: name }], + ["GT1", "GT1-4", "00408", { 4: name }], + ["GT1", "GT1-5", "00409", { 5: "9 Elm Rd^^Dayton^OH^45402" }], + ["GT1", "GT1-6", "00410", { 6: "9375550190" }], + ["GT1", "GT1-7", "00411", { 7: "9375550191" }], + ["GT1", "GT1-8", "00412", { 8: "19631207" }], + ["GT1", "GT1-12", "00416", { 12: "555443210" }], + ["GT1", "GT1-19", "00423", { 19: "EMP77321" }], + ["IN1", "IN1-16", "00441", { 16: name }], + ["IN1", "IN1-18", "00443", { 18: "19631207" }], + ["IN1", "IN1-19", "00444", { 19: "9 Elm Rd^^Dayton^OH^45402" }], + ["IN1", "IN1-36", "00461", { 36: "POL77321" }], + ["IN1", "IN1-49", "01230", { 49: "MEM77321" }], ]; - expect(covered).toHaveLength(28); - for (const [segment, label, fields] of covered) { + expect(covered).toHaveLength(35); + for (const [segment, label, item, fields] of covered) { const r = scan(`cov-${label}.ts`, `const m = "${seg(segment, fields)}";\n`); - expect(r.code, `${label} is NAMED in the banner and must report. stderr: ${r.stderr}`).toBe( - 1, - ); - expect(r.stderr).toContain(label); + expect( + r.code, + `${label} (v2.5.1 item ${item}) is NAMED in the banner and must report. stderr: ${r.stderr}`, + ).toBe(1); + // The exact locator the report prints, with its trailing space: `PID-3` + // alone is a prefix of `PID-33` and would let a renumbering pass. + expect(r.stderr).toContain(`segment=${label} `); } // OUTSIDE the named set: each must be clean, and each zero is a GAP the - // banner declares, not a clearance. They run in the SAME case as the 28 + // banner declares, not a clearance. They run in the SAME case as the 35 // positives above, deliberately, so a wholesale detector failure cannot - // produce them. Seven of these were measured by a refuter against a banner - // that called itself the authoritative list of this gate's limits and did - // not mention one of them. - const uncovered: [string, string, Record][] = [ - ["NK1", "NK1-26", { 26: name }], - ["NK1", "NK1-31", { 31: "9375550188" }], - ["NK1", "NK1-32", { 32: "9 Elm Rd^^Dayton^OH^45402" }], - ["NK1", "NK1-37", { 37: "555443210" }], - ["GT1", "GT1-2", { 2: "G77321" }], - ["GT1", "GT1-4", { 4: name }], - ["IN1", "IN1-49", { 49: "MEM77321" }], - ["PID", "PID-4", { 4: "ALT77321" }], - ["PID", "PID-18", { 18: "ACC77321" }], - ["PV1", "PV1-7", { 7: `1234^${family}^${given}` }], + // produce them, and each carries the v2.5.1 element it actually is. + // + // ▶ IN1-17 IS FIRST BECAUSE IT IS THE MEASURED DEFECT THIS WHOLE DISCIPLINE + // EXISTS FOR. It shipped as a telephone field and is *Insured's + // Relationship To Patient* (item 00442), so a coded relationship was + // reported as a phone number and the remedy it steered a developer toward + // was a global PHONE clearance of that digit string. IN1 carries no insured + // telephone at all; IN1-7 (item 00432) is the payer's, an organisation's. + // The rest are OFF-BY-ONE controls: a value one field away from a field + // that IS read must stay clean, which is what a wrong number would break. + // + // ▶ THESE NINE ROWS ARE CORROBORATED THE SAME WAY THE 35 ABOVE ARE, AND THE + // FIRST DRAFT OF THIS LIST WAS NOT. A refuter measured `PV1-7` written here + // as item 00147; PV1-7 is item **00137** (Attending Doctor) and 00147 is + // PV1-17 (Admitting Doctor). Eight of the nine happened to be right, which + // is the point: they were ASSERTED FROM RECALL rather than extracted from + // the table the 35 were extracted from, so being right was luck and being + // wrong was invisible. No detection changed either way (PV1 is read by + // nothing, and the case asserts a clean result), which is exactly how a + // wrong citation survives: it costs nothing until someone re-checks a row + // against it and lands on a different element, confirmed. **Extract an item + // number from the standard or do not write one.** + const uncovered: [string, string, string, Record][] = [ + ["IN1", "IN1-17", "00442", { 17: "9375550190" }], + ["IN1", "IN1-7", "00432", { 7: "9375550191" }], + ["PID", "PID-4", "00107", { 4: "ALT77321" }], + ["PID", "PID-10", "00113", { 10: name }], + ["PID", "PID-18", "00121", { 18: "ACC77321" }], + ["NK1", "NK1-3", "00192", { 3: name }], + ["GT1", "GT1-11", "00415", { 11: name }], + ["PV1", "PV1-7", "00137", { 7: `1234^${family}^${given}` }], + ["PV1", "PV1-19", "00149", { 19: "V77321" }], ]; - for (const [segment, label, fields] of uncovered) { + for (const [segment, label, item, fields] of uncovered) { const r = scan(`unc-${label}.ts`, `const m = "${seg(segment, fields)}";\n`); - expect(r.code, `${label} is declared OUT of scope. stderr: ${r.stderr}`).toBe(0); + expect( + r.code, + `${label} (v2.5.1 item ${item}) is declared OUT of scope. stderr: ${r.stderr}`, + ).toBe(0); + expect(r.stderr).not.toContain(`segment=${label} `); } - }, 60_000); + }, 120_000); + + it("NK1-37 is an UNDASHED SSN, which the cross-cutting floor is structurally blind to", () => { + // ▶ NON-VACUITY FOR THE SHARPEST CELL THE WIDENING ADDED, and the class rule + // this repository paid for: ENUMERATION ALONE BUYS THE FLOOR AND NOTHING + // ELSE. The floor matches a DASHED shape only, so this value passes it on + // its own; the field table is the only thing that catches it. Both + // polarities in one case, so neither half can drift away from the other. + const undashed = ["555", "44", "3210"].join(""); + + const floorOnly = scan("nk1-37-floor.txt", `${undashed}\n`); + expect(floorOnly.code, `stderr: ${floorOnly.stderr}`).toBe(0); + + const structured = scan("nk1-37-field.ts", `const m = "${seg("NK1", { 37: undashed })}";\n`); + expect(structured.code, `stderr: ${structured.stderr}`).toBe(1); + expect(structured.stderr).toContain("segment=NK1-37 "); + expect(structured.stderr).toContain("social security number"); + }); it("a literal backslash before r or n truncates the segment, and can silence the field it cuts", () => { // The fourth recogniser limit, disclosed rather than guessed at: the escaped