Skip to content

fix(governance): audit independent approval requirement - #87

Closed
seonghobae wants to merge 21 commits into
mainfrom
fix/governance-independent-approval-audit
Closed

fix(governance): audit independent approval requirement#87
seonghobae wants to merge 21 commits into
mainfrom
fix/governance-independent-approval-audit

Conversation

@seonghobae

@seonghobae seonghobae commented Aug 9, 2026

Copy link
Copy Markdown
Contributor

Purpose

Make issue #27's generic independent-approval requirement machine-checkable in the repository-owned main governance audit without re-enabling CODEOWNERS, and correct Noema's repository-consumed Security Scan guidance where it had drifted from the live central workflow.

Current dependency identity

Freshly revalidated immediately before this metadata write:

The branch was rebuilt onto the exact #76 predecessor after the earlier direct-main experiment showed that retaining divergent common-main history did not provide dependency-ordered current-base evidence. No repair workflow, self-modifying Action, encoded patch workflow, or new credential authority is used.

Governance audit implementation

The evaluator requires at least one active pull-request rule whose required_approving_review_count is positive while preserving require_code_owner_review: false. This keeps the current solo-maintainer CODEOWNERS hold separate from Noema's generic independent-review governance contract. Stale-review dismissal, review-thread resolution, status-source pinning, current-base policy, squash policy, force-push prevention, and deletion prevention remain independently checked.

Formal GitHub review submissions, authenticated Noema App review decisions, check runs, commit statuses, scanner evidence and model judgments remain distinct evidence classes. No model/status/comment is promoted to independent approval.

Security Scan guidance RCA and test-first repair

A read-only comparison against the live central .github workflow reproduced two Noema-owned documentation defects:

  1. AGENTS.md stated that central Security Scan runs on every PR base, including stacked feature-base PRs. The live central workflow selects protected bases (main, master, develop), so a feature-base stack can be event-ineligible.
  2. AGENTS.md summarized the Trivy hard gate as CRITICAL/HIGH; the live workflow fails on fixable MEDIUM/HIGH/CRITICAL findings.

This mismatch could make an agent treat feature-base Security Scan absence as an unexpected central outage or understate the vulnerability threshold.

Test-first lineage:

  • cf96f3d115d810fa72d642ed3cd7727000fee0b5 — RED guidance contract rejecting the false “every stacked PR” claim and requiring explicit feature-base absence semantics.
  • 9c59a8bb3dc36835a0bb380bb9a7dea0bfad1986 — RED threshold contract requiring MEDIUM/HIGH/CRITICAL and rejecting stale CRITICAL/HIGH text.
  • e68e0a4bb319cd52c52b6641ae5f1fdd3312811b — GREEN Noema-owned AGENTS.md correction: eligible protected-base PRs require Security Scan; feature-base absence is non-passing defer_until_trigger and requires dependency-ordered refresh/retarget onto an eligible protected base before merge.

The central .github repository remains read-only; this PR changes only Noema-owned guidance and deterministic regressions.

Newly proven sibling-overlap boundary with #80

Fresh comparison against Draft PR #80 proves both branches share merge base exactly at #76 head and are diverged siblings, not a valid stack. Both modify AGENTS.md:

Therefore #87 must not be treated as a safe successor to an independently integrated #80 snapshot. Current convergence plan after #76: integrate/stabilize #87 (or an equivalent protected-main governance/guidance successor), then rebuild/refresh only #80's unique scheduler/publisher/NIM delta onto that exact protected lineage and reacquire all checks/reviews. Do not manufacture ancestry by retargeting a diverged head. Old sibling evidence does not transfer.

PR #89 does not modify AGENTS.md, so it does not create this same direct documentation-source overwrite risk.

Independent-approval test-first lineage

  • d0dead4cd0f41743f3dac340184fda2c7043eb38 — RED contract requiring a compliant pull-request rule to specify required_approving_review_count: 1 while retaining require_code_owner_review: false, plus a zero-approval regression.
  • 9fb72e4eb5917c7cd0990777301d34de8a2e3a0c — GREEN evaluator implementation requiring a positive generic approval count.

Historical direct-main commits and their runs are retained only as RCA evidence, not current branch acceptance evidence.

Current exact-head verification

For unchanged exact head e68e0a4bb319cd52c52b6641ae5f1fdd3312811b:

  • ci run 31336097264: terminal success. Exact source checkout/assertion passed; Node 24.19.0 and npm 11.17.0 matched the branch's inherited toolchain contract; frozen install succeeded; 62 test files / 648 tests passed; configured statements/branches/functions/lines are 100%; npm audit --audit-level=high found 0 vulnerabilities; acquisition manifest generation passed.
  • reviewer-ci run 31336097263: terminal success, including exact checkout verification, reviewer coverage/docstring gates and sandbox verification.
  • central Security Scan: absent by feature-base event selection; this is non-passing defer_until_trigger, never scanner success.
  • formal GitHub reviews: none.
  • inline review threads: none.

The acquisition manifest continues to identify missing production KPI/provenance/security-validation/release/deployment/environment/revenue/transfer final-gate evidence; those omissions are not promoted to release/acquisition readiness.

Governance and merge boundary

This PR closes repository-owned detection/documentation gaps only. It cannot create the live main ruleset, provision an eligible independent Reviewer App, prove direct-push/break-glass rejection, or make a feature-base central Security Scan eligible. Issue #27 owns enforceable live main governance and #29 owns Reviewer/Maintainer App provisioning/activation evidence.

Keep this PR Draft while #76 is unintegrated. After #76 integrates, refresh only #87's unique delta onto the then-current protected base, require fresh exact-head ci, reviewer-ci, and central Security Scan terminal success under eligible event semantics, re-review current findings/threads, satisfy the actual live ruleset and qualifying current-head review requirements, and only then consider merge. After #87 is protected/stable, #80 must be rebuilt/refreshed onto that lineage before its own integration.

No version bump, release, deployment, gate waiver, protection weakening, synthetic approval, or repair workflow is introduced.

Related: #27, #29, #30, #75, #76, #77, #79, #80, #89

Summary by CodeRabbit

  • 개선 사항

    • CI에서 PR의 정확한 커밋을 검증하고, Node.js 및 npm 버전을 고정해 빌드 재현성을 강화했습니다.
    • 의존성 설치 및 보안 검사 기준을 개선하고, 관련 보안 수정 사항을 기록했습니다.
    • 저장소 거버넌스 규칙에 독립적인 승인 검증을 추가했습니다.
  • 문서

    • nanoid 보안 취약점의 수정 근거와 검증 절차를 문서화했습니다.
  • 테스트

    • CI 체크아웃, 런타임 버전, 거버넌스 및 보안 검사 규칙에 대한 자동 검증을 추가했습니다.

@coderabbitai

coderabbitai Bot commented Aug 9, 2026

Copy link
Copy Markdown

Review Change Stack

📝 Walkthrough

Walkthrough

CI가 정확한 PR head를 checkout하고 고정된 Node/npm 도구 체인을 검증하도록 변경되었습니다. 보안 스캔과 독립 승인 규칙이 갱신되었습니다. nanoid 3.3.17 remediation 근거와 관련 검증 계약이 문서화되었습니다.

Changes

CI checkout 및 도구 체인

Layer / File(s) Summary
정확한 checkout 및 도구 체인 검증
.github/workflows/ci.yml, .github/workflows/reviewer-ci.yml, test/ci-exact-head-contract.test.ts, test/workflow-readiness.test.ts
워크플로가 정확한 SHA를 checkout하고 실제 commit을 검증합니다. Node.js 24.19.0과 npm 11.17.0을 고정하고 설치 옵션을 검증합니다.

거버넌스 및 remediation

Layer / File(s) Summary
보안 스캔 및 승인 거버넌스 규칙
AGENTS.md, scripts/lib/main-governance-audit.mjs, test/main-governance-audit.test.ts
보안 스캔 대상 base branch와 MEDIUM/HIGH/CRITICAL 기준을 갱신합니다. 양의 승인 리뷰 수를 요구하는 감사를 추가합니다.
nanoid remediation 기록 및 검증 계약
CHANGELOG.md, docs/doctoring/nanoid-cve-2026-67213.md
lockfile의 nanoid 해상도를 3.3.17로 기록하고, 고정된 실행 환경과 exact-head CI 검증 계약을 문서화합니다.

Estimated code review effort: 3 (Moderate) | ~25 minutes

Possibly related issues

  • ContextualWisdomLab/noema 이슈 27: 독립 승인 요구와 보호 base branch 보안 스캔 규칙을 함께 다룹니다.
  • ContextualWisdomLab/.github 이슈 772: independent_approval_not_required 감사와 독립 승인 검증이 직접 연결됩니다.
  • ContextualWisdomLab/four-pillars 이슈 28: exact-head 재검증과 거버넌스 제어를 지원하는 변경입니다.

Possibly related PRs

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 60.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed 제목은 활성 풀 리퀘스트 규칙의 독립 승인 요구사항을 감사하는 핵심 변경을 정확하고 간결하게 설명합니다.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches 💡 1
📝 Generate docstrings 💡
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/governance-independent-approval-audit

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@seonghobae
seonghobae changed the base branch from main to fix/nanoid-cve-2026-67213 August 9, 2026 18:15
@seonghobae seonghobae closed this Aug 9, 2026
@seonghobae
seonghobae force-pushed the fix/governance-independent-approval-audit branch from 7b6701a to e0106ce Compare August 9, 2026 18:18

Copy link
Copy Markdown
Contributor Author

@coderabbitai review

Please review the unchanged exact head e68e0a4bb319cd52c52b6641ae5f1fdd3312811b relative to its live predecessor e0106ce16b7b8b493f46bf075ec5baf58762bd95. Focus on the governance evaluator's independent-approval semantics, evidence-class separation, Security Scan event-eligibility guidance, and whether the three-file delta can false-pass a zero-approval or stale/ineligible ruleset. This review is diagnostic evidence only and does not substitute for qualifying independent merge approval.

@coderabbitai

coderabbitai Bot commented Aug 10, 2026

Copy link
Copy Markdown
🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

head_sha='e68e0a4bb319cd52c52b6641ae5f1fdd3312811b'
base_sha='e0106ce16b7b8b493f46bf075ec5baf58762bd95'
repo='ContextualWisdomLab/noema'

echo '== Exact commits and ancestry =='
git show -s --format='HEAD %H%nParents %P%nSubject %s' "$head_sha"
git show -s --format='BASE %H%nParents %P%nSubject %s' "$base_sha"
git merge-base --is-ancestor "$base_sha" "$head_sha" && echo 'base_is_ancestor=true'
git rev-list --left-right --count "${base_sha}...${head_sha}"
git diff --name-status "$base_sha" "$head_sha"

echo '== Compact source maps =='
ast-grep outline scripts/lib/main-governance-audit.mjs --items all
ast-grep outline test/main-governance-audit.test.ts --items all

echo '== Exact delta =='
git diff --no-ext-diff --unified=80 "$base_sha" "$head_sha" -- \
  AGENTS.md scripts/lib/main-governance-audit.mjs test/main-governance-audit.test.ts

echo '== Relevant evaluator references =='
rg -n -C 5 \
  'required_approving_review_count|require_code_owner_review|pull_request|ruleset|Security Scan|security scan|approval|review' \
  scripts/lib/main-governance-audit.mjs test/main-governance-audit.test.ts AGENTS.md

echo '== Live protected-branch governance metadata =='
gh api "repos/${repo}/branches/main/protection" 2>&1 || true
gh api "repos/${repo}/rules/branches/main" 2>&1 || true
gh api "repos/${repo}/rulesets" 2>&1 || true

echo '== PR state and current commits =='
gh pr view 87 --repo "$repo" --json number,state,isDraft,headRefOid,baseRefName,baseRefOid,reviews,reviewThreads 2>&1 || true

Length of output: 43213


🏁 Script executed:

#!/bin/bash
set -euo pipefail

repo='ContextualWisdomLab/noema'
central_repo='ContextualWisdomLab/.github'
central_sha='6eb06cdd08c79a06f7b390069d4ffa49e2eb7dba'

echo '== Governance evaluator call sites and data acquisition =='
rg -n -C 8 \
  'evaluateMainGovernanceRules\s*\(|main-governance-audit|/rules/branches/main|rules/branches|rulesets|rules_response_invalid' \
  -g '*.mjs' -g '*.ts' -g '*.json' .

echo '== Central Security Scan workflow at cited exact ref =='
gh api \
  -H 'Accept: application/vnd.github.raw+json' \
  "repos/${central_repo}/contents/workflows/security-scan.yml?ref=${central_sha}" 2>&1 || true

echo '== Central repository ref identity =='
gh api "repos/${central_repo}/git/commits/${central_sha}" \
  --jq '{sha: .sha, tree: .tree.sha, message: .message}' 2>&1 || true

echo '== PR 87 current metadata and formal review submissions =='
gh pr view 87 --repo "$repo" \
  --json number,state,isDraft,headRefName,headRefOid,baseRefName,baseRefOid,reviewDecision,reviews,statusCheckRollup,url 2>&1 || true
gh api "repos/${repo}/pulls/87/reviews" 2>&1 || true

echo '== PR 87 review-thread count via GraphQL =='
gh api graphql -F owner='ContextualWisdomLab' -F name='noema' -F number=87 -f query='
query($owner: String!, $name: String!, $number: Int!) {
  repository(owner: $owner, name: $name) {
    pullRequest(number: $number) {
      reviewThreads(first: 100) {
        totalCount
        nodes { isResolved }
      }
    }
  }
}' 2>&1 || true

Length of output: 36800


🏁 Script executed:

#!/bin/bash
set -euo pipefail

central_repo='ContextualWisdomLab/.github'
central_sha='6eb06cdd08c79a06f7b390069d4ffa49e2eb7dba'

echo '== Candidate Security Scan workflow paths at the cited central commit =='
gh api "repos/${central_repo}/git/trees/${central_sha}?recursive=1" \
  --jq '.tree[] | select(.type == "blob" and (.path | test("security.*scan|scan.*security"; "i"))) | [.path, .sha] | `@tsv`' \
  2>&1 || true

echo '== Matching workflow content =='
while IFS=$'\t' read -r path sha; do
  [ -n "${path:-}" ] || continue
  echo "--- ${path} (${sha}) ---"
  gh api -H 'Accept: application/vnd.github.raw+json' \
    "repos/${central_repo}/git/blobs/${sha}" 2>&1 || true
done < <(
  gh api "repos/${central_repo}/git/trees/${central_sha}?recursive=1" \
    --jq '.tree[] | select(.type == "blob" and (.path | test("security.*scan|scan.*security"; "i"))) | [.path, .sha] | `@tsv`' \
    2>/dev/null || true
)

Length of output: 1542


🤖 web_search executed:

GitHub documentation required approving review count pull request author own approval count branch protection ruleset

Length of output: 18252


@seonghobae No blocking finding in the reviewed three-file delta.

  • The requested base is an ancestor of e68e0a4bb319cd52c52b6641ae5f1fdd3312811b. The delta is 0 behind / 5 ahead and changes only the stated three files.
  • evaluateMainGovernanceRules() now fails when all active pull_request rules have zero or invalid required_approving_review_count.
  • The new regression test covers the zero-approval case. It changes the only applicable pull-request rule from 1 to 0 and requires independent_approval_not_required.
  • The audit collects rules from GitHub’s live rules/branches/main endpoint at runtime. It fails closed if collection fails. It does not accept a stored rules response as current evidence.
  • The evaluator does not use check runs, commit statuses, scanner output, CodeRabbit output, comments, or model judgments as approval evidence. PR #87 currently has no formal reviews and zero review threads.
  • A positive required approval count is policy evidence only. It is not an approval for this PR. GitHub documents that a pull-request author cannot approve their own pull request, so the GitHub approval mechanism provides the author-independent reviewer boundary: https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/reviewing-changes-in-pull-requests/approving-a-pull-request-with-required-reviews.
  • The audit cannot prove that main rules remain unchanged after the audit runs. Fresh governance and merge-time enforcement remain necessary.
  • The updated Security Scan guidance correctly treats absence on a feature-base PR as non-passing evidence. I verified the cited central commit identity. Direct retrieval of the central workflow body failed in the review environment, so independent verification of its trigger filter and Trivy threshold is inconclusive.

This diagnostic review is not a formal approval. It does not satisfy the required independent merge approval.

⚠️ Action not completed

Review rate limited.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 4

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@docs/doctoring/nanoid-cve-2026-67213.md`:
- Line 21: Update the paragraph beginning with “#77/#78 remain” to begin with
“PR `#77/`#78 remain,” preserving the rest of the sentence unchanged.

In `@test/ci-exact-head-contract.test.ts`:
- Around line 19-22: Update the contract test around the existing
NOEMA_EXPECTED_HEAD_SHA and HEAD assertions to also require the SHA-format
validation condition `^[0-9a-f]{40}$` in the workflow content. Ensure the test
covers both workflows so removing either validation causes the contract test to
fail.

In `@test/main-governance-audit.test.ts`:
- Around line 103-108: Update the governance audit test around
evaluateMainGovernanceRules to also assert that result.status equals "FAIL",
while preserving the existing failures entry assertion.
- Around line 208-221: Extend the test in “repository governance guidance”
beyond AGENTS.md text checks to inspect the canonical workflow configuration for
the pull_request base-branch trigger and the osv-scan, dependency-review, and
trivy-fs jobs. Assert that the workflow contract explicitly covers the main
branch, while preserving the existing documentation assertions.
🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: c9c19b7a-ea06-42f5-80f6-2ad1ad4cdaf5

📥 Commits

Reviewing files that changed from the base of the PR and between c85d710 and e68e0a4.

⛔ Files ignored due to path filters (1)
  • package-lock.json is excluded by !**/package-lock.json
📒 Files selected for processing (9)
  • .github/workflows/ci.yml
  • .github/workflows/reviewer-ci.yml
  • AGENTS.md
  • CHANGELOG.md
  • docs/doctoring/nanoid-cve-2026-67213.md
  • scripts/lib/main-governance-audit.mjs
  • test/ci-exact-head-contract.test.ts
  • test/main-governance-audit.test.ts
  • test/workflow-readiness.test.ts


The baseline `main` CI did not provide a deterministic package-manager identity: earlier terminal-success PR-associated CI used Node `24.18.0`, npm `11.16.0`, and plain `npm ci`. PR #76 now narrows that gap for this security remediation by selecting Node.js `24.19.0` through immutable `actions/setup-node` commit `48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e` (v6.4.0), verifying `node --version` equals `v24.19.0`, verifying `npm --version` equals `11.17.0`, and installing with exactly `npm ci --legacy-peer-deps=false --install-links=false`. Checkout is supplied by immutable `actions/checkout` commit `de0fac2e4500dabe0009e67214ff5f5447ce83dd` (v6.0.2). Any mismatch fails before dependency installation. These are execution-time reproduction constraints for PR #76; they do not claim that repository-wide package-manager metadata and install-script policy are complete.

#77/#78 remain the broader test-first acquisition-readiness workstream for repository-level `packageManager`/`devEngines` identity, strict dependency install-script policy, immutable Node-24-native Actions, exact PR-head checkout, and lockfile change control. PR #76 deliberately does not duplicate #78's full lockfile-policy machinery merely to remediate one transitive vulnerability.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win

Markdown 문법 오류를 수정하세요.

Line 21의 #77/#78 remain...은 해시 뒤 공백이 없어 MD018 경고를 발생시킵니다. PR 참조임을 명확히 하도록 PR #77/#78 remain...으로 시작하세요.

수정 예시
-#77/#78 remain the broader test-first acquisition-readiness workstream for repository-level `packageManager`/`devEngines` identity, strict dependency install-script policy, immutable Node-24-native Actions, exact PR-head checkout, and lockfile change control. PR `#76` deliberately does not duplicate `#78`'s full lockfile-policy machinery merely to remediate one transitive vulnerability.
+PR `#77/`#78 remain the broader test-first acquisition-readiness workstream for repository-level `packageManager`/`devEngines` identity, strict dependency install-script policy, immutable Node-24-native Actions, exact PR-head checkout, and lockfile change control. PR `#76` deliberately does not duplicate `#78`'s full lockfile-policy machinery merely to remediate one transitive vulnerability.
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
#77/#78 remain the broader test-first acquisition-readiness workstream for repository-level `packageManager`/`devEngines` identity, strict dependency install-script policy, immutable Node-24-native Actions, exact PR-head checkout, and lockfile change control. PR #76 deliberately does not duplicate #78's full lockfile-policy machinery merely to remediate one transitive vulnerability.
PR `#77/`#78 remain the broader test-first acquisition-readiness workstream for repository-level `packageManager`/`devEngines` identity, strict dependency install-script policy, immutable Node-24-native Actions, exact PR-head checkout, and lockfile change control. PR `#76` deliberately does not duplicate `#78`'s full lockfile-policy machinery merely to remediate one transitive vulnerability.
🧰 Tools
🪛 markdownlint-cli2 (0.23.2)

[warning] 21-21: No space after hash on atx style heading

(MD018, no-missing-space-atx)

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@docs/doctoring/nanoid-cve-2026-67213.md` at line 21, Update the paragraph
beginning with “#77/#78 remain” to begin with “PR `#77/`#78 remain,” preserving
the rest of the sentence unchanged.

Source: Linters/SAST tools

Comment on lines +19 to +22
expect(workflow).toContain(
"NOEMA_EXPECTED_HEAD_SHA: ${{ github.event.pull_request.head.sha || github.sha }}",
);
expect(workflow).toContain('test "$(git rev-parse HEAD)" = "$NOEMA_EXPECTED_HEAD_SHA"');

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔒 Security & Privacy | 🟡 Minor | ⚡ Quick win

SHA 형식 검증도 계약 테스트에 고정하세요.

현재 테스트는 NOEMA_EXPECTED_HEAD_SHA 설정과 HEAD 비교만 확인합니다. 두 워크플로에서 ^[0-9a-f]{40}$ 검증을 제거해도 테스트가 통과합니다. 해당 조건문을 assertion으로 추가하세요.

수정 예시
   expect(workflow).toContain(
     "NOEMA_EXPECTED_HEAD_SHA: ${{ github.event.pull_request.head.sha || github.sha }}",
   );
+  expect(workflow).toContain(
+    'if [[ ! "$NOEMA_EXPECTED_HEAD_SHA" =~ ^[0-9a-f]{40}$ ]]; then',
+  );
   expect(workflow).toContain('test "$(git rev-parse HEAD)" = "$NOEMA_EXPECTED_HEAD_SHA"');

코딩 가이드라인의 “Add or update regression tests for security and API behavior changes” 요구사항에 따른 것입니다.

📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
expect(workflow).toContain(
"NOEMA_EXPECTED_HEAD_SHA: ${{ github.event.pull_request.head.sha || github.sha }}",
);
expect(workflow).toContain('test "$(git rev-parse HEAD)" = "$NOEMA_EXPECTED_HEAD_SHA"');
expect(workflow).toContain(
"NOEMA_EXPECTED_HEAD_SHA: ${{ github.event.pull_request.head.sha || github.sha }}",
);
expect(workflow).toContain(
'if [[ ! "$NOEMA_EXPECTED_HEAD_SHA" =~ ^[0-9a-f]{40}$ ]]; then',
);
expect(workflow).toContain('test "$(git rev-parse HEAD)" = "$NOEMA_EXPECTED_HEAD_SHA"');
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@test/ci-exact-head-contract.test.ts` around lines 19 - 22, Update the
contract test around the existing NOEMA_EXPECTED_HEAD_SHA and HEAD assertions to
also require the SHA-format validation condition `^[0-9a-f]{40}$` in the
workflow content. Ensure the test covers both workflows so removing either
validation causes the contract test to fail.

Source: Coding guidelines

Comment on lines +103 to +108
const result = evaluateMainGovernanceRules(rules);

expect(result.failures).toContainEqual({
code: "independent_approval_not_required",
detail: "Active pull-request rules do not require at least one approving review.",
});

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

최종 status도 검증하세요.

현재 테스트는 result.failures 배열에 특정 실패 항목이 포함되는지만 검사합니다. 구현이 실패 항목을 기록하고도 status: "PASS"를 반환하면 이 테스트는 통과합니다.

result.status"FAIL"인지도 단언하세요.

수정 예시
     const result = evaluateMainGovernanceRules(rules);
 
+    expect(result.status).toBe("FAIL");
     expect(result.failures).toContainEqual({

PR 목표가 거버넌스 감사를 기계적으로 판정하도록 요구하므로 최종 상태를 검증해야 합니다.

📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
const result = evaluateMainGovernanceRules(rules);
expect(result.failures).toContainEqual({
code: "independent_approval_not_required",
detail: "Active pull-request rules do not require at least one approving review.",
});
const result = evaluateMainGovernanceRules(rules);
expect(result.status).toBe("FAIL");
expect(result.failures).toContainEqual({
code: "independent_approval_not_required",
detail: "Active pull-request rules do not require at least one approving review.",
});
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@test/main-governance-audit.test.ts` around lines 103 - 108, Update the
governance audit test around evaluateMainGovernanceRules to also assert that
result.status equals "FAIL", while preserving the existing failures entry
assertion.

Comment on lines +208 to +221
describe("repository governance guidance", () => {
it("documents the live central Security Scan trigger and severity boundary", () => {
const agents = readFileSync(new URL("../AGENTS.md", import.meta.url), "utf8");

expect(agents).not.toContain("It runs on every PR base, **including stacked PRs**.");
expect(agents).toContain(
"The central workflow currently selects pull requests whose base branch is `main`, `master`, or `develop`.",
);
expect(agents).toContain(
"A feature-base stacked PR can therefore have no Security Scan run; absence is non-passing evidence",
);
expect(agents).toContain("MEDIUM/HIGH/CRITICAL");
expect(agents).not.toContain("CRITICAL/HIGH, fixable only");
});

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔒 Security & Privacy | 🟡 Minor | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

workflow_files="$(fd --hidden --type f --extension yml --extension yaml .github 2>/dev/null || true)"
if [ -z "$workflow_files" ]; then
  echo "No workflow files found under .github" >&2
  exit 1
fi

while IFS= read -r file; do
  rg -n -C 12 \
    'Security Scan|pull_request|branches|trivy-fs|severity|ignore-unfixed' \
    "$file" || true
done <<< "$workflow_files"

Repository: ContextualWisdomLab/noema

Length of output: 201


🏁 Script executed:

#!/bin/bash
set -u

echo "Root files:"
git ls-files | sed -n '1,120p'

echo
echo "AGENTS.md references:"
if [ -f AGENTS.md ]; then
  rg -n -C 8 'Security Scan|trivy|ignore-unfixed|Main governance|approval|stacked|pull requests|severity|MEDIUM|HIGH|CRITICAL' AGENTS.md || true
else
  echo "AGENTS.md not found"
fi

echo
echo "Workflow-like files:"
git ls-files | rg '(^|/) workflow |\.ya?ml$|github/action|\.gitlab|sonar|trivy|gosec|codeql|security' | sed -n '1,200p'

echo
echo "Test file section:"
if [ -f test/main-governance-audit.test.ts ]; then
  sed -n '1,260p' test/main-governance-audit.test.ts | cat -n
fi

Repository: ContextualWisdomLab/noema

Length of output: 17365


🏁 Script executed:

#!/bin/bash
set -u

echo "Search Security Scan-related terms:"
rg -n -C 4 'Security Scan|trivy-fs|trivy|dependency-review|osv-scan|pull_request|branches|trivyignore' .github/test scripts/main-governance-audit.mjs scripts/lib/main-governance-audit.mjs docs AGENTS.md test/main-governance-audit.test.ts || true

echo
echo "Workflows:"
for f in .github/workflows/*.yml; do
  echo "---- $f ----"
  sed -n '1,220p' "$f" | sed -n '/security\|trivy\|dependency\|osv\|Security Scan/p; 1,220p'
done

echo
echo "Audit source:"
wc -l scripts/lib/main-governance-audit.mjs
sed -n '1,260p' scripts/lib/main-governance-audit.mjs | cat -n

Repository: ContextualWisdomLab/noema

Length of output: 50384


🏁 Script executed:

#!/bin/bash
set -u

echo "Workflow files with exact Security Scan/Scanner references:"
for f in .github/workflows/*.yml; do
  hits="$(rg -n 'Security Scan|trivy-fs|trivy|dependency-review|osv-scan|pull_request' "$f" 2>/dev/null || true)"
  if [ -n "$hits" ]; then
    echo "---- $f ----"
    sed -n '1,220p' "$f" | rg -n -C 8 'Security Scan|trivy-fs|trivy|dependency-review|osv-scan|pull_request|branches:'
  fi
done

echo
echo "Workflow occurrences:"
rg -n -C 3 'Security Scan|trivy-fs|trivy|dependency-review|osv-scan|pull_request:' .github/workflows .github/reusable 2>/dev/null || true

echo
echo "Tests referencing scanner triggers/workflows:"
rg -n -C 4 'Security Scan|pull_request|trivy-fs|dependency-review|osv-scan|getSecurityScan|scanner|readme|AGENTS' test scripts docs docs/superpowers specs scripts/lib/main-governance-audit.mjs 2>/dev/null || true

Repository: ContextualWisdomLab/noema

Length of output: 50383


Security Scan 문서 회귀 검사를 실제 워크플로 기준과 맞춰보세요.

현재 생성된 테스트는 AGENTS.md 문구만 검증합니다. 저장소에 pull_request branches: main 트리거의 osv-scan / dependency-review / trivy-fs 실행 계약이 있는지 분리하지 못합니다. 이 지침을 유지하려면 테스트도 같은 pull_request job 기준을 읽도록 확장하거나, 문서가 워크플로 설정을 정확히 반영한다고 명시된 다른 소스를 계약으로 두세요.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@test/main-governance-audit.test.ts` around lines 208 - 221, Extend the test
in “repository governance guidance” beyond AGENTS.md text checks to inspect the
canonical workflow configuration for the pull_request base-branch trigger and
the osv-scan, dependency-review, and trivy-fs jobs. Assert that the workflow
contract explicitly covers the main branch, while preserving the existing
documentation assertions.

Copy link
Copy Markdown
Contributor Author

#76 has now integrated into protected main. Clean replacement PR #90 was created directly from that protected tip and preserves #87's unique delta exactly: AGENTS.md, scripts/lib/main-governance-audit.mjs, and test/main-governance-audit.test.ts match #87's current blobs, while #76's already-integrated files are not replayed. #90 also recorded a fresh exact-head RED with the two intended failures before the GREEN implementation. Closing this diverged/noisy predecessor so review/check evidence is reacquired only on the clean protected-main successor; no predecessor evidence transfers.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant