test(browser): add controlled Agent Task fixture - #65
Open
seonghobae wants to merge 2 commits into
Open
Conversation
📝 WalkthroughWalkthrough결정적 Agent Task HTML fixture를 추가했습니다. fixture는 입력 제출 결과와 ChangesAgent Task fixture
Estimated code review effort: 2 (Simple) | ~10 minutes 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
seonghobae
marked this pull request as ready for review
August 11, 2026 00:14
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@tests/test_agent_task_fixture_contract.py`:
- Around line 37-41: Update the HTML attribute parsing used by the
hidden-injection contract to retain the complete parsed attributes for each
input element. Increment hidden_injection_markers only when an untrusted marker
also has a hidden attribute, and validate credential inputs from parsed
attribute values rather than raw quoted text: reject type=password and
autocomplete values password or one-time-code regardless of quote style.
- Around line 62-64: HTML 문자열 검사만 수행하는
tests/test_agent_task_fixture_contract.py에 실제 브라우저 제출 검증을 추가하십시오.
scripts/ci/run_mv3_compatibility.py의 기존 Chrome WebDriver 경로에서
agent_task_basic/index.html을 로드하고 task-text를 변경해 제출한 뒤, 제출 전후 URL이 동일하고
task-result의 data-state가 submitted이며 textContent가 변경한 입력값과 정확히 일치하는지 검사하십시오.
.github/workflows/mv3-compatibility.yml의 실행 경로에 해당 fixture와 테스트 변경을 포함하십시오.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro Plus
Run ID: 60e1549b-6ea0-49e9-b88e-5e7de0c066c0
📒 Files selected for processing (2)
tests/fixtures/agent_task_basic/index.htmltests/test_agent_task_fixture_contract.py
Comment on lines
+37
to
+41
| if ( | ||
| attributes.get("data-originweave-untrusted") == "prompt-injection" | ||
| and attributes.get("aria-hidden") == "true" | ||
| ): | ||
| self.hidden_injection_markers += 1 |
There was a problem hiding this comment.
🔒 Security & Privacy | 🟡 Minor | ⚡ Quick win
보안 조건을 HTML 속성으로 검증하십시오.
현재 hidden_injection_markers 조건은 hidden 속성을 확인하지 않습니다. aria-hidden="true"만 남아도 숨김 프롬프트 인젝션 계약이 통과합니다. 또한 자격 증명 검사는 원본 문자열의 큰따옴표 표기만 찾습니다. 예를 들어 type='password'는 이 검사를 통과합니다.
input의 전체 속성을 파서에 저장하십시오. hidden 속성이 있는 untrusted marker만 계산하십시오. type=password와 password 또는 one-time-code autocomplete 값을 속성 값으로 거부하십시오.
Also applies to: 76-85
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@tests/test_agent_task_fixture_contract.py` around lines 37 - 41, Update the
HTML attribute parsing used by the hidden-injection contract to retain the
complete parsed attributes for each input element. Increment
hidden_injection_markers only when an untrusted marker also has a hidden
attribute, and validate credential inputs from parsed attribute values rather
than raw quoted text: reject type=password and autocomplete values password or
one-time-code regardless of quote style.
Comment on lines
+62
to
+64
| self.assertIn('data-state="idle"', self.html) | ||
| self.assertIn('result.dataset.state = "submitted"', self.html) | ||
| self.assertIn("result.textContent = taskText.value", self.html) |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | 🏗️ Heavy lift
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
# Locate existing browser-test infrastructure before adding the interaction contract.
rg -n -C 2 --glob '*.py' \
'sync_playwright|async_playwright|selenium|webdriver|page\.(goto|fill|click|locator)' \
tests || trueRepository: ContextualWisdomLab/OriginWeave
Length of output: 169
🏁 Script executed:
#!/bin/bash
set -euo pipefail
printf '%s\n' '--- candidate files ---'
git ls-files | rg '(^|/)(test_agent_task_fixture_contract\.py|.*agent.*task.*|.*fixture.*|.*html.*|.*js$|.*ts$)' | head -200
printf '%s\n' '--- test file ---'
cat -n tests/test_agent_task_fixture_contract.py
printf '%s\n' '--- referenced identifiers ---'
rg -n -C 4 --glob '!tests/test_agent_task_fixture_contract.py' \
'task-text|task-result|data-state|result\.dataset\.state|preventDefault|taskText' .Repository: ContextualWisdomLab/OriginWeave
Length of output: 6945
🏁 Script executed:
#!/bin/bash
set -euo pipefail
printf '%s\n' '--- repository configuration ---'
git ls-files | rg '(^|/)(pyproject\.toml|pytest\.ini|setup\.cfg|tox\.ini|requirements.*|package\.json|.*lock|Makefile|README.*|\.github/)' | head -200
printf '%s\n' '--- test and browser tooling references ---'
rg -n -i -C 2 \
'playwright|selenium|webdriver|chromium|browser|unittest|pytest|test_agent_task_fixture_contract|agent_task_basic' \
--glob '!tests/fixtures/agent_task_basic/index.html' .
printf '%s\n' '--- fixture implementation ---'
cat -n tests/fixtures/agent_task_basic/index.html
printf '%s\n' '--- available browser executables (if any) ---'
for name in chromium chromium-browser google-chrome google-chrome-stable firefox; do
command -v "$name" || true
doneRepository: ContextualWisdomLab/OriginWeave
Length of output: 50390
🏁 Script executed:
#!/bin/bash
set -euo pipefail
printf '%s\n' '--- browser-related tracked files ---'
git ls-files | rg -i '(^|/)(.*(browser|chrom|webdriver|mv3|compat).*)$|(^|/)(package\.json|.*workflow.*\.ya?ml)$' | head -300
printf '%s\n' '--- exact runner/workflow references ---'
rg -l -i \
'sync_playwright|async_playwright|selenium|webdriver|chromedriver|chrome-for-testing|run_browser_pass|browser lane|real browser' \
.github scripts tests docs 2>/dev/null | head -100
printf '%s\n' '--- targeted test references ---'
rg -n -C 3 \
'RUNNER|WORKFLOW|run_browser_pass|chrome-for-testing|webdriver|browser lane' \
tests/test_mv3_compatibility_contract.py .github scripts 2>/dev/null | head -300
printf '%s\n' '--- concise fixture and test-strategy excerpts ---'
sed -n '1,80p' tests/fixtures/agent_task_basic/index.html
sed -n '60,78p' docs/TEST_STRATEGY.md
sed -n '138,153p' docs/TEST_STRATEGY.mdRepository: ContextualWisdomLab/OriginWeave
Length of output: 30300
🏁 Script executed:
#!/bin/bash
set -euo pipefail
printf '%s\n' '--- existing WebDriver runner ---'
sed -n '1,280p' scripts/ci/run_mv3_compatibility.py
printf '%s\n' '--- browser workflow execution steps ---'
sed -n '1,150p' .github/workflows/mv3-compatibility.yml
printf '%s\n' '--- runner entry point and CLI ---'
sed -n '280,520p' scripts/ci/run_mv3_compatibility.pyRepository: ContextualWisdomLab/OriginWeave
Length of output: 23530
실제 브라우저 제출 계약을 추가하십시오.
tests/test_agent_task_fixture_contract.py는 HTML 문자열만 검사합니다. 기존 scripts/ci/run_mv3_compatibility.py의 고정 Chrome WebDriver 경로를 확장하여 agent_task_basic/index.html을 로드하고 다음을 검사하십시오.
task-text값을 변경한 후 제출합니다.- 제출 전후 URL이 동일한지 검사합니다.
task-result의data-state가submitted인지 검사합니다.task-result.textContent가 변경한 입력값과 정확히 같은지 검사합니다.
.github/workflows/mv3-compatibility.yml이 이 fixture와 테스트 변경 시 실행되도록 경로도 추가하십시오.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@tests/test_agent_task_fixture_contract.py` around lines 62 - 64, HTML 문자열 검사만
수행하는 tests/test_agent_task_fixture_contract.py에 실제 브라우저 제출 검증을 추가하십시오.
scripts/ci/run_mv3_compatibility.py의 기존 Chrome WebDriver 경로에서
agent_task_basic/index.html을 로드하고 task-text를 변경해 제출한 뒤, 제출 전후 URL이 동일하고
task-result의 data-state가 submitted이며 textContent가 변경한 입력값과 정확히 일치하는지 검사하십시오.
.github/workflows/mv3-compatibility.yml의 실행 경로에 해당 fixture와 테스트 변경을 포함하십시오.
This was referenced Aug 11, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Partial implementation of #28.
Buyer-visible gap
The first real Chromium Agent Task slice needs a deterministic local web application whose semantic form, hostile hidden page content, and observable post-condition are stable enough to exercise observation → policy → typed action → verification without production credentials or a live third-party site.
TDD evidence
This lane is based directly on protected main
67af7c87589edc2039545af335c95064d9b8391c.d2580305f05aba93d10b5342ec1886d601c6752eintroduced the contract fortests/fixtures/agent_task_basic/index.htmlbefore the fixture existed.31445088008, Rust contracts job93637443229, checked out that exact head and failed in Python repository contracts with exactly threeFileNotFoundErrorfailures for the absent fixture. Repository checkout evidence confirms the tested revision. This is the intended fail-first boundary, not a predecessor failure reused as success evidence.0888fe3a6ef6da547a37fd075733cc73dc52b2abadds only the controlled fixture required by that contract.Implemented boundary
The fixture now provides:
task-text) and submit control that can later be discovered by role/name rather than CSS/XPath authority;data-state="idle"todata-state="submitted"while exposing only the typed synthetic value as the result;Exact-head verification
On unchanged exact head
0888fe3a6ef6da547a37fd075733cc73dc52b2ab:31445201739: success;93637824750: repository Python contracts, formatting, locked workspace check, full tests, strict Clippy and rustdoc all success;93637824824: success, including exact owned production function/line/region/branch enforcement;31445201774: success;31445201669: success;No predecessor-head check or review is transferred to this exact head.
Scope boundary
This fixture is controlled test infrastructure, not a browser adapter or action-success claim. It does not implement WebDriver BiDi/CDP, semantic extraction from Chromium, deterministic policy dispatch, native browser input, trusted clock provenance, secret handling, destination authority, post-condition evidence, profile teardown, process attribution, or the complete issue #28 vertical slice. The next runtime lane must execute this fixture through pinned stock Chromium and preserve the existing authority boundaries rather than turning page content into instruction or capability authority.
Summary by CodeRabbit