Skip to content

deps: bump modernc.org/sqlite from 1.40.1 to 1.53.0#36

Open
dependabot[bot] wants to merge 319 commits into
masterfrom
dependabot/go_modules/modernc.org/sqlite-1.53.0
Open

deps: bump modernc.org/sqlite from 1.40.1 to 1.53.0#36
dependabot[bot] wants to merge 319 commits into
masterfrom
dependabot/go_modules/modernc.org/sqlite-1.53.0

Conversation

@dependabot

@dependabot dependabot Bot commented on behalf of github Jun 21, 2026

Copy link
Copy Markdown
Contributor

Bumps modernc.org/sqlite from 1.40.1 to 1.53.0.

Changelog

Sourced from modernc.org/sqlite's changelog.

Changelog

  • 2026-06-21 v1.53.0:

    • Add experimental netbsd/amd64 support, resolving the long-standing build break in [GitLab issue #246](https://gitlab.com/cznic/sqlite/-/issues/246). This target is intentionally not yet listed among the supported platforms in the package documentation: the port had been broken for years and is only now revived, and there is as yet no real-world experience running it under production workloads. Green CI is not the same as battle-tested — so while the full test suite (including the pcache and vec packages and the -race concurrency test) passes on NetBSD 10.1 / Go 1.26.3, and the entire upstream toolchain (libc, cc, ccgo, libz, libtcl8.6, libsqlite3, libsqlite_vec) is green on the NetBSD CI builder, the target is offered for evaluation only. If you run NetBSD, please exercise it with your own workloads and report back via #246; the intent is to promote it to a fully supported platform after a period of broader real-world testing (on the order of a month) elapses without surprises.
    • Implementation notes: the previously shipped lib/sqlite_netbsd_amd64.go was a stale old-generator transpile that no longer compiled (the mu.enter/mu.leave break in #246); it is replaced by a fresh new-generator transpile consistent with every other platform, and modernc.org/sqlite/vec (sqlite-vec) is vendored and auto-registers on netbsd. Correct operation requires the matching pinned modernc.org/libc, which carries two NetBSD-specific fixes found during this work: the mmap(2) PAD-argument ABI (without it, concurrent WAL access faults with SIGBUS in the WAL-index shared memory) and a working abort(3) (the prior stub left SQLite's crash-recovery writecrash test unable to terminate by signal). As usual, downstream modules must pin the exact modernc.org/libc version this module's go.mod pins.
    • See [GitLab merge request #82](https://gitlab.com/cznic/sqlite/-/merge_requests/82), thanks Leonardo Taccari (@​iamleot) and Thomas Klausner (@wiz)!
    • Add experimental freebsd/386 and freebsd/arm support. As with the netbsd/amd64 target above, these two 32-bit FreeBSD ports are intentionally not yet listed among the supported platforms in the package documentation: freebsd/386 previously shipped a stale, effectively untested SQLite 3.41 transpile, and freebsd/arm is entirely new, so neither has real-world production mileage yet. Both are now freshly transpiled at SQLite 3.53.2 consistent with every other platform, build cleanly, and pass the full test suite (core, WAL/concurrency, and the vec package) on the FreeBSD CI builders; they are offered for evaluation only. If you run 32-bit FreeBSD, please exercise these targets with your own workloads and report back — the intent is to promote freebsd/386, freebsd/arm, and netbsd/amd64 to fully supported platforms in a future release cycle, once a period of broader real-world testing elapses without surprises.
    • Implementation notes: correct operation on freebsd/arm requires the matching pinned modernc.org/libc (v1.73.4), which fixes the per-arch mmap(2) off_t encoding for 32-bit FreeBSD; without it the WAL shared-memory mapping faults with SIGBUS under concurrent access, the same class of bug found on the netbsd port. As usual, downstream modules must pin the exact modernc.org/libc version this module's go.mod pins.
    • See [GitLab merge request #119](https://gitlab.com/cznic/sqlite/-/merge_requests/119), thanks Olivier Cochard-Labbé (@​ocochard)!
    • Add a Go-facing wrapper for SQLITE_CONFIG_PCACHE2. PageCache is the factory and Cache the per-database instance, both idiomatic Go interfaces; Page exposes the raw Buf and Extra pointers that SQLite reads through the C pcache contract. RegisterPageCache and MustRegisterPageCache install the module process-globally before the first sql.Open; subsequent Open calls are gated through a one-shot Xsqlite3_config(SQLITE_CONFIG_PCACHE2) so a too-late Register returns ErrPageCacheTooLate rather than silently falling through to the built-in pcache1. The binding owns the sqlite3_pcache_page stub and re-consults the implementation on every Fetch, reusing the stub only when the returned Page value is unchanged, which keeps a bounded/evicting purgeable cache safe by construction.
    • See [GitLab merge request #126](https://gitlab.com/cznic/sqlite/-/merge_requests/126), thanks Ian Chechin!
    • Add modernc.org/sqlite/pcache, the reference page-cache implementation that accompanies the #126 SQLITE_CONFIG_PCACHE2 wrapper. pcache.New returns a *Pool satisfying the PageCache interface; register it once with sqlite.MustRegisterPageCache(pcache.New()) and every connection opened afterwards draws its pages from it. Each Pool.Create mints a fresh per-database Cache: a bounded, LRU-evicting page store that honours the PRAGMA cache_size soft cap and releases the least-recently-unpinned page when it must make room. Page memory — the Buf and Extra buffers SQLite reads through — is allocated with libc.Xmalloc/libc.Xcalloc and therefore lives off the Go heap, which keeps SQLite's interior pointer arithmetic on the page extras from tripping the race detector's checkptr enforcement. Pool.Stats reports aggregate lifetime counters (hits, misses, allocs, evictions, rekeys, truncates, caches) across every cache a Pool has created, so hit/miss/eviction behaviour is observable without instrumenting individual caches. Cross-connection page sharing is out of scope for now; each Create returns an independent per-database cache.
    • Validated end-to-end against the #126 stress workload (cache_size=16, 4000 BLOB rows with DELETE and incremental_vacuum, integrity_check clean under -race) and benchmarked for the memory-utilization goal tracked in [GitLab issue #204](https://gitlab.com/cznic/sqlite/-/issues/204).
    • See [GitLab merge request #127](https://gitlab.com/cznic/sqlite/-/merge_requests/127), thanks Ian Chechin!
    • Tighten the modernc.org/sqlite/pcache reference implementation per cznic's !127 review follow-ups. Adds Stats.EasyRefusals, a per-Pool counter for the cases where FetchCreateEasy returns nil at cap; SQLite reacts to a refusal by spilling dirty pages and retrying with FetchCreateForce, so the new field is a direct proxy for the I/O pressure the strict Easy contract imposes vs pcache1's recycle-without-spill behavior. BenchmarkPoolEvictionChurn was reworked to drive a rotating-residue DELETE (k % 3 = i % 3) and re-insert a matching batch each cycle so the spill pressure recurs and easy-refusals/op scales with b.N instead of capping at the seed's one-time first-cycle cost; both existing benchmarks now report easy-refusals/op alongside the page-allocs/evictions metrics. Stats.Evictions documentation was tightened to match the actual behavior (counts LRU eviction, Unpin(discard=true), Shrink releases, and Unpin(discard=false) trimming back to target after a FetchCreateForce overcommit; bulk frees from Truncate, Rekey collisions, and Destroy are not counted). The TestPoolRoundTripIntegrity comment claiming the workload exercises xRekey ~15 times has been corrected; the SQL surface does not reliably emit xRekey here, and that codepath is covered by the unit tests instead.
    • See [GitLab merge request #130](https://gitlab.com/cznic/sqlite/-/merge_requests/130), thanks Ian Chechin!
    • Make modernc.org/sqlite/pcache -race-clean under SQLite's cache=shared mode. The pool already runs correctly under shared-cache because every callback into a given Cache is serialised internally by SQLite's sqlite3BtreeEnter on the BtShared mutex; verified empirically with a lock-free in-flight probe (max-in-flight = 1 on the canonical two-connection workload, 4 on a positive control with goroutines hitting the cache directly). However the Go race detector does not recognise SQLite's libc mutex as a happens-before edge and reports false-positive races on Fetch vs Unpin reads/writes of the per-cache state, which surfaces as DATA RACE failures for any user who registers the pool and runs their suite under -race. A sync.Mutex on the cache type is now taken on every public method (SetSize, PageCount, Fetch, Unpin, Rekey, Truncate, Destroy, Shrink), always. On the common non-shared-cache path the lock is uncontended (one atomic CAS per Lock/Unlock pair, negligible next to the SQLite work it bookends); on the shared-cache path it just rubber-stamps the order SQLite's BtShared mutex already established. A new e2e_test.go TestSharedCacheTwoConns_Integrity drives two sql.Conn against the same cache=shared URI with concurrent writers and asserts PRAGMA integrity_check = ok under -race; passes cleanly with the lock, would surface the false-positive without it. Design notes live in pcache/sharing.go.
    • See [GitLab merge request #131](https://gitlab.com/cznic/sqlite/-/merge_requests/131), thanks Ian Chechin!
    • Add a Go wrapper for sqlite3_db_status, the per-connection runtime counters (cache hit/miss/write/spill rates, schema and prepared-statement memory, lookaside usage, deferred foreign keys). DBStatus is an interface implemented by the driver connection and reached through the database/sql escape hatch (*sql.Conn).Raw(), mirroring the existing FileControl surface; DBStatusOp is a distinct typed enum of the SQLITE_DBSTATUS_* verbs so a counter from a different op family will not compile in its place. Status(op, reset) returns the (current, high) pair and optionally resets the counter. This also lets modernc.org/sqlite/pcache measure real I/O instead of the EasyRefusals proxy: the new BenchmarkPoolSpillIO reads the pager-level SQLITE_DBSTATUS_CACHE_SPILL/_CACHE_WRITE counters, which the pager maintains identically for pcache1 and the pool, making the pcache1-vs-pool comparison cznic raised on the !127 review a genuine apples-to-apples measurement. On the rotating-residue eviction-churn workload at cache_size=16 the pool spills ~3.5x more than pcache1 (cache-spill/op 31.96 vs 8.96) for ~3% more page writes (cache-write/op 450 vs 436) at identical hit/miss, quantifying the I/O cost of the strict Easy contract that EasyRefusals only proxied.
    • See [GitLab merge request #132](https://gitlab.com/cznic/sqlite/-/merge_requests/132), thanks Ian Chechin!
    • Add an opt-in _dqs DSN query parameter that disables SQLite's double-quoted string literal compatibility quirk on a per-connection basis. When _dqs=0 (or any strconv.ParseBool false value) is supplied, the driver calls sqlite3_db_config with SQLITE_DBCONFIG_DQS_DDL and SQLITE_DBCONFIG_DQS_DML set to off before any statement is prepared, so a double-quoted identifier that fails to resolve raises a parse error instead of silently falling back to a string literal. Absence of the parameter, or _dqs=1, leaves SQLite's default behavior unchanged; existing DSNs continue to work byte-for-byte. Resolves [GitLab issue #61](https://gitlab.com/cznic/sqlite/-/issues/61).
    • See [GitLab merge request #128](https://gitlab.com/cznic/sqlite/-/merge_requests/128), thanks Ian Chechin!
    • Add an opt-in _error_rc DSN query parameter for clearer error reporting on open-time failures. When _error_rc=1 (or any strconv.ParseBool true value) is supplied, error strings synthesised from a (rc, db) pair only append sqlite3_errmsg(db) when sqlite3_extended_errcode(db) is consistent with the operation rc (full match first, primary code &0xff as fallback). On mismatch the canonical sqlite3_errstr(rc) is used alone, so an open-time SQLITE_CANTOPEN no longer carries the temporary handle's stale "out of memory" errmsg. Absence of the parameter, or _error_rc=0, preserves the legacy "errstr: errmsg" form byte-for-byte; existing callers that parse error strings are unaffected. The driver's *Error.Code() returns the same SQLite result code in both modes. Parsed before sqlite3_open_v2 so open-time errors are covered. Resolves [GitLab issue #230](https://gitlab.com/cznic/sqlite/-/issues/230).
    • See [GitLab merge request #129](https://gitlab.com/cznic/sqlite/-/merge_requests/129), thanks Ian Chechin!
  • 2026-06-06 v1.52.0:

    • Upgrade to SQLite 3.53.2.
    • Add Backup.Remaining and Backup.PageCount, thin wrappers around the existing sqlite3_backup_remaining and sqlite3_backup_pagecount C symbols. Together they expose the per-Step progress counters that the underlying backup object already maintains, enabling progress reporting during online backups without dropping to modernc.org/sqlite/lib directly.
    • See [GitLab merge request #122](https://gitlab.com/cznic/sqlite/-/merge_requests/122), thanks Ian Chechin!
    • Drop the redundant second copy in (*conn).columnText, the path that backs every Rows.Scan into a Go string for a TEXT column. The value's bytes are still copied once out of SQLite-owned memory into a fresh Go buffer; that buffer is then reinterpreted as the result string with unsafe.String rather than copied a second time by the implicit string([]byte) conversion. This removes one allocation per TEXT value per row and roughly halves the bytes allocated on that path; on the new BenchmarkColumnTextScan cases it is ~13–20% faster for payloads of 256 B and larger, with no measurable change for very short strings. Purely internal: no API or behavioral change, and the returned string never aliases SQLite's buffer.
    • See [GitLab merge request #123](https://gitlab.com/cznic/sqlite/-/merge_requests/123), thanks Ian Chechin!
    • Cache each result column's declared type once per result set in newRows instead of recomputing it on every row. The TEXT branch of Rows.Next calls ColumnTypeDatabaseTypeName for every TEXT column on every row (independent of any DSN flag), which previously did a libc.GoString + strings.ToUpper each time; that lookup is now a single index into a cached, pre-uppercased []string, and ColumnTypeScanType reads the same cache and drops its per-call strings.ToLower. The declared type is fixed for the lifetime of a prepared statement, so the C round-trip is paid once per column rather than once per column per row, removing exactly 1 alloc + 8 B per TEXT column per row from the Next hot path. The new BenchmarkTextToTimeScan cases show ~7% faster on a 1000-row DATETIME SELECT under _texttotime=1. Purely internal: ColumnTypeDatabaseTypeName and ColumnTypeScanType return identical values, no API or behavioral change.
    • See [GitLab merge request #124](https://gitlab.com/cznic/sqlite/-/merge_requests/124), thanks Ian Chechin!
    • Cache, per result column, the parseTimeFormats index that first parsed a TEXT-stored DATE/DATETIME/TIMESTAMP value, and try that format first on later rows instead of re-walking the list from the top. (*conn).parseTime previously ran time.Parse down the format list on every such row; for the canonical SQLite TEXT datetime format every row paid two failed time.Parse attempts — each allocating a *time.ParseError — before the match. On a 1000-row DATETIME TEXT SELECT this cuts ~50% of allocs/op and ~57% of B/op and is ~37% faster. The fall-through chain is preserved exactly: the seven formats are mutually exclusive, so the cached hint can never select a different match than the in-order scan, and the parsed driver.Value is identical to before. Purely internal: no API or behavioral change.
    • See [GitLab merge request #125](https://gitlab.com/cznic/sqlite/-/merge_requests/125), thanks Ian Chechin!
  • 2026-05-28 v1.51.0:

    • Pool the []driver.Value slice passed to scalar/aggregate UDF callbacks and to vtab Filter/Insert/Update callbacks, eliminating the dominant per-row allocation on UDF-heavy queries. Benchmarks on a 1000-row, 3-arg noop scalar UDF show ~40% fewer bytes/op and ~15% fewer allocs/op.
    • Document the matching "arguments are not valid past return" contract on vtab.Cursor.Filter and vtab.Updater.Insert/Update, consistent with the existing rule for FunctionImpl.Scalar / AggregateFunction.Step / WindowInverse.
    • Resolves [GitLab issue #226](https://gitlab.com/cznic/sqlite/-/issues/226). See [GitLab merge request #114](https://gitlab.com/cznic/sqlite/-/merge_requests/114), thanks Ian Chechin!
    • Add FileControl.FileControlDataVersion, a wrapper around SQLITE_FCNTL_DATA_VERSION for observing pager-cache data-version changes, including those made on the same connection. Useful as a primitive for application-level cache invalidation.
    • Exposed via the idiomatic database/sql escape hatch (*sql.Conn).Raw(), consistent with the existing FileControlPersistWAL.
    • See [GitLab merge request #115](https://gitlab.com/cznic/sqlite/-/merge_requests/115), thanks Ian Chechin!
    • Fix a regression where in-memory connections (:memory:, file::memory:, shared-cache memory URIs) were discarded by database/sql after a context-cancelled query, taking the entire in-memory store with them. The fix for #198 had added an sqlite3_is_interrupted check to the connection validator that mistakenly applied to in-memory connections too, re-introducing the bug originally fixed by !74. File-backed connections keep the existing behaviour and are still discarded after an interrupt.
    • Resolves [GitLab issue #196](https://gitlab.com/cznic/sqlite/-/issues/196). See [GitLab merge request #116](https://gitlab.com/cznic/sqlite/-/merge_requests/116), thanks Ian Chechin!
    • Add an opt-in FunctionImpl.VolatileArgs flag that hands TEXT and BLOB arguments to scalar and aggregate UDF callbacks as zero-copy views (unsafe.String/unsafe.Slice) over SQLite's own value buffers, eliminating the per-argument libc.GoString/make([]byte) copy that the #226 slice-pooling left as the remaining per-row allocation. On the same 1000-row, 3-arg (INTEGER/TEXT/BLOB) noop scalar UDF this removes a further ~35% of allocs/op and ~11% of bytes/op on top of #226.
    • The views are valid only for the duration of the callback and must not be retained past return or across rows; a callback that needs to keep a value must copy it. With VolatileArgs unset (the default) arguments keep the existing copied, caller-owned semantics, so the flag is fully backward compatible; it has no effect on integer, float, time, or NULL arguments.
    • See [GitLab merge request #120](https://gitlab.com/cznic/sqlite/-/merge_requests/120), thanks Ian Chechin!
    • Extend the opt-in VolatileArgs zero-copy TEXT/BLOB argument access from #120 to the virtual-table Cursor.Filter (xFilter) and Updater.Insert/Update (xUpdate) callbacks. A vtab.Module opts in by implementing the new optional vtab.VolatileArgsOpter interface (VolatileArgs() bool); the flag is read once at module registration and shared by every table created from it. On a vtab call carrying one TEXT and one BLOB argument this removes 2 allocs/op (one libc.GoString, one make([]byte)) on each of the Filter and Update paths.
    • The same safety contract as #120 applies: the views are valid only for the duration of the callback and must not be retained past return or across rows; a callback that needs to keep a value must copy it. Modules that do not implement VolatileArgsOpter (the default for all existing modules) are byte-for-byte unchanged, and the flag has no effect on integer, float, time, or NULL arguments.

... (truncated)

Commits
  • 6b32d1e CHANGELOG.md: document experimental freebsd/386 + freebsd/arm (#119)
  • 697300f Merge branch 'dbstatus-binding' into 'master'
  • 759639f sqlite: review fixes for !132 — restore #131 CHANGELOG link, correct DBStatus...
  • 40ff027 sqlite: add DBStatus wrapper for sqlite3_db_status + pcache spill-I/O benchmark
  • 6a28fe7 HACKING.md: document CHANGELOG versioning + MR integration flow
  • adff4b1 Merge branch 'pcache-shared-cache-draft' into 'master'
  • 14e5790 vendor: regenerate freebsd/arm vec at SQLite 3.53.2
  • 8725c22 Add freebsd/386 + freebsd/arm targets
  • 73050dc Merge branch 'pcache-pool-polish' into 'master'
  • 1897fdd CHANGELOG.md: consolidate untagged v1.53.0/v1.54.0 into one v1.53.0 section
  • Additional commits viewable in compare view

M09Ic and others added 30 commits May 25, 2026 08:01
New judge.go:
- Judge struct wraps an OpenAI-compatible LLM API for evaluation
- Evaluate(intent, criteria, result) sends execution trace to LLM,
  returns structured Verdict{Pass, Score, Reason, Issues}
- buildTrace renders tool call sequence, args, results, and output
  into a human-readable format for the judge
- Judge prompt enforces strict evaluation: 'ran without errors' is
  not the same as 'fulfilled the intent'

Integration:
- Verifier.JudgeWith(judge, intent, criteria) for imperative use
- Intent.JudgeCriteria field for declarative use — when set, the
  judge runs automatically after structural checks pass
- Harness.Judge() creates a judge using the same LLM credentials

New test TestAgentLoopLifecycleJudged demonstrates three-layer
verification: structural (ExpectInOrder) + constraint (MaxTurns,
NoErrors) + semantic (LLM judge confirms create/list/delete intent).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Extract ShellCommand() into pkg/agent/task/shell.go, shared by
both bash.go and task/manager.go. Searches /bin/bash, /usr/bin/bash,
/bin/sh, /usr/bin/sh before falling back to PATH lookup.

Fixes 'sh: executable file not found in $PATH' in environments
where PATH is restricted but /bin/bash exists.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
All agent and subagent tests rewritten as Intent{} declarations with:
- Structural: ExpectInOrder tool patterns with arg/result matching
- Constraints: MaxTurns, NoErrors
- Semantic: JudgeCriteria for LLM-as-judge evaluation

Judge improvements:
- Retry up to 3 times on API failures (empty responses, TLS errors)
- Degrade to warning when judge API is unavailable (test still passes
  on structural checks alone)

Results: 12/14 tests scored 100 from LLM judge, 2 degraded due to
DeepSeek API instability (structural checks still passed).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
13 tests across 7 layers of agentic scanning:

Layer 1 - Direct scanner (no AI):
  TestRealScanDirectGogo, TestRealScanDirectSpray, TestRealScanDirectPipeline

Layer 2 - Scanner + AI post-analysis (--ai):
  TestRealScanGogoAI, TestRealScanPipelineAI

Layer 3 - Agent mode (LLM decides how to scan):
  TestRealAgentGogoScan, TestRealAgentSprayScan, TestRealAgentFullPipeline

Layer 4 - Agent + verification skills:
  TestRealAgentScanWithVerify, TestRealAgentScanReport

Layer 5 - Agent + subagent fan-out (parallel):
  TestRealAgentParallelScan

Layer 6 - Agent + loop tool (recurring scan):
  TestRealAgentLoopScan

Layer 7 - IOA loop mode (swarm worker):
  TestRealIOALoopScanTask

Each test uses Intent{} with structural checks (ExpectInOrder, Steps)
and LLM judge semantic verification (JudgeCriteria).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
New monitor.go: tails the events JSONL file during test execution,
renders a compact live view of agent behavior:
  - Turn boundaries (── turn N ──)
  - Tool calls with args (🔧 bash {"command": ...})
  - Tool results with size (✓ bash → 10 bytes: hello)
  - Tool errors (❌ bash error: ...)
  - Assistant messages (💬 text preview)
  - Agent completion (── agent done ──)

Integration:
  - h.WithMonitor() enables for specific test
  - AISCAN_MONITOR=1 env var enables globally
  - Runs as background goroutine, non-blocking
  - Tails fsync'd JSONL with 200ms poll interval

Usage: AISCAN_MONITOR=1 go test -tags e2e -run TestName -v ./pkg/harness/

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…ne lifecycle cleanup

Track per-turn token usage (prompt/completion/total) and current context
window size in the transcript, surfacing them through EventTurnEnd events,
JSONL event logs, and the Result struct. This lays the groundwork for
future context compaction strategies by making token growth observable.

Also clean up Pipeline dedup maps after Run() completes, giving the
pipeline a clear lifecycle boundary for GC.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…king tests

- Eliminate double YAML parse in config loading by mapping scan defaults
  into the Option struct directly (remove loadScanDefaults)
- Add 32-slot buffer to OpenAI streaming channel to prevent SSE reader
  blocking when consumer is briefly delayed
- Reduce collector lock contention: merge double lock acquisition into
  one, move format computation outside the critical section
- Add tests for per-turn token usage tracking and EventTurnEnd usage
  propagation

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…esult.Details

ImageBlock() had zero callers; MimeType/Data fields on ContentBlock were
only assigned by ImageBlock; ToolResult.Details was never read or written.

These will be re-introduced with a cleaner design as part of the
multimodal provider integration.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Replace the standalone vision tool with native multimodal support in the
provider layer, aligning with pi-mono's architecture where image content
flows naturally through the read tool and message system.

Core changes:
- Add ContentPart/ImageURL types and ChatMessage.MarshalJSON for
  multimodal message serialization (text or content-part array)
- Anthropic provider: contentPartsToAnthropicBlocks converts image_url
  parts to Anthropic's native base64 source format
- OpenAI provider: MarshalJSON handles serialization automatically
- Read tool: detect image files by magic bytes (JPEG/PNG/GIF/WEBP),
  return ImageBlock content instead of "[binary file]"
- Agent loop: preserve ToolResult structure, convert image tool results
  to multimodal ChatMessages via toolResultToMessage
- Event serialization: ContentParts rendered as "[image: mime]" in JSONL

Deleted:
- pkg/tools/vision/ (entire package — 447 lines)
- VisionOptions struct, --vision flag, --vision-* flags (6 CLI options)
- VisionEnabled from Deps/ToolConfig/App/Config (4 layers)
- Vision provider resolution and mergeVisionOptions
- System prompt "Vision Analysis" section

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…ol support

New `proxy` pseudo-command for the AI agent:
- `proxy auto <url>` — one-command setup: subscribe + adaptive load balancing
  with --type, --name, --country filters
- `proxy subscribe/list/switch/test/current/clear` for manual control
- Auto mode activates when config proxy is clash:// URL

Integration with scanner tools:
- BashTool.SetScannerProxy() for shell env vars (ALL_PROXY, HTTP_PROXY)
- InstallGlobalProxy() for gogo/neutron/zombie in-process transports
- SetProxy() on gogo/spray/zombie/neutron for CLI --proxy injection
- All extra protocols registered: trojan, vmess/vless, anytls, hysteria2

Config support:
- --proxy and cyberhub.proxy now accept clash:// URLs
- clash:// auto-activates adaptive load balancing at startup

Depends on proxyclient v1.0.4-0.20260527160727-36cf133952c3
and proxyclient/extra v0.0.0-20260527160727-36cf133952c3

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- New() 接受 proxy 参数,优先于环境变量 HTTP_PROXY/HTTPS_PROXY
- Deps 新增 WebSearchProxy 字段,register.go 透传到 web_search
- DefaultWebSearchProxy 编译时变量 + build.sh --websearch-proxy 支持
- 统一通过 gookit/config 直接读取运行时默认值
- 支持 websearch.tavily_keys 和 websearch.proxy 配置项
- config 模板新增 websearch 段
- 新增 TestLoadConfigWebSearchOptions 测试
- appConfig 传递 EnableAllAISkills / VerifyMode / WebSearchProxy
- verify skill 改为 Agent 模式,支持 bash 工具主动探测目标
- sniper 发现定制验证 prompt,区分 CVE 情报与已确认漏洞
- 新增 WithAgentFunc 用于 verify agent 调度
- scanVerifyBlocksCommand 防止 verify 递归调用扫描器
- collector 输出标注验证状态 (verified/not_confirmed/inconclusive)
- 新增 EnableAllAISkills / VerifyMode 配置传递
- makeMaxTurnStop 控制 verify agent 最大轮次
- 新增 session 管理: open/close、TTL 自动回收、最多 8 并发
- 新增 discover 子命令: katana 注入发现表单/按钮/onclick 元素
- 新增 autofill 子命令: 智能表单填充 + --data 覆盖
- 新增交互子命令: click/fill/select/wait
- 新增 dialog 子命令: XSS alert/confirm/prompt 捕获验证
- 统一 URL/session 派发: navigate/content/eval/screenshot/network 自动识别
- SKILL.md 增加漏洞验证工作流示例 (XSS/SQLi/CAPTCHA/cookie bypass)
- ioa_send 描述改为 --content JSON 格式说明,避免传字符串
- 新增 ioa_read 描述,强调 --space_id 必填
- ioa/swarm SKILL.md 中 ioa_read 示例统一使用 --space_id 参数
- report: 新增验证标注语义表,区分 verified/not_confirmed/inconclusive
- report: 新增 Potential Risks 和 Dismissed Findings 段
- verify: 重写为主动探测协议,含 4 步验证流程
- verify: 新增 browser 交互式验证工作流 (XSS/SQLi/CAPTCHA)
- verify: 明确 status 判定标准和 JSON 输出格式
Remove all 5 global singleton save/restore proxy patterns:
- proxy/install.go (InstallGlobalProxy) — deleted entirely
- scan/command.go (installProxy) — deleted
- gogo/gogo.go (installProxyDialer) — deleted
- neutron/neutron.go (installProxy) — deleted
- zombie/zombie.go (installProxy) — deleted

New proxy injection:
- gogo: RunnerOption.ProxyDialContext/ProxyDialTimeout per-instance
- gogo/zombie scan pipeline: SDK Context.SetProxy()
- neutron: engine.ApplyNeutronProxy() set-once at init/switch
- zombie standalone: RunOptions.ProxyDial per-execution
- spray: unchanged (already per-instance via --proxy arg)
- engine init: Config.WithProxy() for engine-level defaults

Add proxy passthrough (proxychains-style):
- proxy <url> <command> [args...] — one-shot proxy wrapper
- proxy <node-index|keyword> <command> [args...] — use subscribed node
- findNode supports index, exact name, and substring match

Migrate to new SDK APIs (remove compat shims):
- sdk/pkg.Stats → sdk/pkg/types.Stats
- association.FingerPOCIndex → association.Index
- Config.WithCyberhub() → Config.WithProvider(cyberhub.NewProvider())
- NewWeakpassTask → NewBruteTask

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- README.md: project overview, features, quick start
- docs/quickstart.md: installation and first scan guide
- docs/configuration.md: config file and CLI options reference
- docs/ioa.md: IOA (Indicator of Attack) server documentation

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Simplify bash.go: remove platform-specific files (bash_other.go, bash_unix.go)
- Delete task_tool.go: inline task management into command registry
- Add tmux.go: dedicated tmux session command
- Refactor task manager: simplify lifecycle, reduce code by ~200 lines
- Add PTY support (pty_linux.go, pty_darwin.go, pty_unix.go, pty_other.go)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- browser: reduce session.go boilerplate, improve error handling
- websearch: clean up register.go and proxy configuration in web_search.go
- browser SKILL.md: minor documentation updates

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- cmd: simplify session.go, config.go, provider_defaults.go
- build.sh: minor flag adjustments
- tests: update agent scanner/inbox test fixtures for new APIs

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- errcheck: explicitly discard return values (buffer.Write, fmt.Sscanf, etc.)
- bodyclose: ensure resp.Body.Close on all paths in anthropic stream
- nilerr: return errors instead of nil in glob.go and anthropic.go
- misspell: cancelled → canceled
- unused: remove colorGreen, colorWhite, floatOption, inboxRefsFromPeer
- staticcheck QF1008: remove embedded field from selector
- staticcheck SA9003: remove empty if branch in read.go
- staticcheck QF1002: use tagged switch in gogo arg parser
- gosec G703: filepath.Clean on write path
- S1039: remove unnecessary fmt.Sprintf in proxy command
- gitignore: exclude scan_results.jsonl

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
go1.26 is too new for the latest golangci-lint (built with go1.25).
Minimum required by dependency chain is 1.25.7 (katana).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Ensures the clash:// scheme is always registered with proxyclient,
regardless of import order. Without this, clash:// URLs would fail
with "unsupported proxy client" if command.go/state.go imports
were reorganized.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add comprehensive test coverage for the proxy tool (command + state)
and tmux interactive PTY sessions (unit, agent integration, and live
LLM). Add a dedicated `tool-tests` CI job that runs these explicitly
with verbose output alongside the existing `./...` coverage run.
Also fix pre-existing app_test assertion for removed `task` tool.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…AI protocols

Introduce a CacheRetention type ("short"/"long") that flows from agent Config
through ChatCompletionRequest into each provider's request marshaling:

- Anthropic: inject cache_control markers on system prompt, last tool definition,
  and last user message; preserve CacheReadTokens/CacheWriteTokens from API response
- OpenAI: add prompt_cache_key and prompt_cache_retention; parse
  prompt_tokens_details.cached_tokens and DeepSeek's prompt_cache_hit_tokens
- Agent layer: auto-generate SessionID, accumulate cache metrics in TurnUsage,
  display cache_read/cache_write/hit_ratio in turn output
- Default to CacheShort for all sessions; subagent DeriveChild inherits cache config

Verified with httptest mock servers (both protocols × multi-turn/streaming/tool-call/fork)
and live DeepSeek v4-pro tests showing 78-96% cache hit ratios.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…tool remnants

- Rename package directory and all internal `package task` → `package tmux`
- Update all imports (cmd/, pkg/command/, pkg/agent/ tests)
- Remove dead `case "task":` from agent output summarizer
- Update e2e test prompts to use tmux commands instead of old task tool
- Update SKILL.md documentation to reference tmux pseudo-command
- No functional changes — same PTY session manager, new package name

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
gomod (grouped golang.org/x/* and chainreactors/*) plus github-actions where present. No ignore on chainreactors/* so inter-repo deps are auto-updated.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Bumps [goreleaser/goreleaser-action](https://github.com/goreleaser/goreleaser-action) from 6 to 7.
- [Release notes](https://github.com/goreleaser/goreleaser-action/releases)
- [Commits](goreleaser/goreleaser-action@v6...v7)

---
updated-dependencies:
- dependency-name: goreleaser/goreleaser-action
  dependency-version: '7'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
M09Ic and others added 28 commits June 23, 2026 00:55
The hand-written config.yaml was stale — missing agent section, cyberhub
proxy, recon fields (hunter_token/proxy/limit), data_dir, and had wrong
ioa.db (now ioa.token). Regenerated via `aiscan --init` so it matches
the current Option struct.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Provider description now lists all supported providers (openai,
  anthropic, deepseek, openrouter, ollama, groq, moonshot)
- LLMProviderEntry gains Images field to match ProviderConfig
- FallbackProviderConfigs forwards Images to agent.ProviderConfig
- Regenerated config.yaml with full build tag

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- DefaultConfigName changed from "config.yaml" to "aiscan.yaml"
- mergeOption now merges Providers list from config file
- Updated all references in code, tests, build.sh, docs, and READMEs
- Regenerated aiscan.yaml via `aiscan --init`

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
When single-provider fields (provider/api_key/model/base_url) are empty
but llm.providers list is set, providers[0] becomes the primary and
providers[1:] become fallbacks. Both formats are backward-compatible:
single fields still work, and mixing both uses single fields as primary
with the providers list as fallback.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Register LoopTool in agent runtime for scheduled recurring tasks
- Add /loop console command (create/list/stop loop tasks)
- Rename /reset to /clear for consistency
- Add /goal as alias for /eval
- Remove unused /continue command

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Add FindingsSummary component for aggregated vulnerability overview
- Add FindingsPanel component for detailed findings display
- Show AI verification status badges (Verified/CVE Intel/Deep Test)
- Highlight AI-sourced items with colored left border
- Extend scan-result.ts with serviceAIStatus and assetItemContent helpers

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Add LoopTool (pkg/agent/loop_tool.go) implementing AgentTool interface
  for create/list/remove operations on the LoopScheduler
- Add Ctrl+O keybinding to cycle verbosity levels (quiet/default/tools/thinking)
- Adjust tool output display thresholds based on verbosity level
- Show keyboard shortcuts (Esc, Ctrl+O) in welcome banner
- Add e2e tests for loop tool create and full lifecycle

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…ion, and spinner help hints

- Show tool execution headers at default verbosity (v=0), reserve result
  preview for -v and full output for -vv
- Add Ctrl+O keybinding to cycle verbosity (default → tools → thinking)
- Bind Ctrl+C in readline to interrupt running tasks or confirm exit,
  replacing unreliable SIGINT-based signal handling in raw mode
- Fix Esc interrupt to call InterruptCurrentRun directly instead of
  routing through readline error propagation
- Wire SIGINT to InterruptCurrentRun for non-readline code paths
- Add ctx.Err() checks in executeToolCalls between parallel/sequential
  tool phases and before provider fallback to ensure prompt cancellation
- Replace blocking `for range events` with select on ctx.Done in LLM
  stream consumer for immediate cancellation response
- Register @ prefix file completion via carapace on the root command
- Show keyboard shortcut hints below spinner during agent execution
- Upgrade spinner to bubbletea-based renderer with help hint support

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…eline)

Pseudo-commands now support single pipe (|) to shell pipelines. The
pseudo-command runs in-process with output captured to a buffer, then
the buffer is piped as stdin to the shell pipeline via sh -c — all
within a single CreateFunc session.

Examples: `found -i . | grep critical`, `scan -i target -j | head -20`

Double pipe (||), redirections (>), and chaining (&&, ;) remain rejected.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Implement sensitive information scanner as a pseudo-command based on
proton, with 21 builtin rules (AWS/GitHub/Stripe/DB/JWT/private keys
etc). Add shell-to-pseudo pipe support so `echo data | found` and
`cat file | found` work via StdinReceiver interface and temp file
bridging. Includes 24 e2e tests covering all pipe directions.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Rename pkg/tools/found → pkg/tools/proton, align with neutron naming
- Replace hardcoded rules.go (21 rules) with SDK proton.Engine loading
  embedded found templates via ResourceProvider (156 rules)
- Generate found_keys/found_spray/found_filter .bin files into
  core/resources/data/ via templates_gen.go
- JSON output uses proton's native file.Finding type directly
- Simplify RunCommand: single splitPipeline() call for both pipe directions
- Extract createPseudo() helper in tmux manager
- Clean stale "found" references in pipe_test.go and SKILL.md
- Consolidate tests: delete found_test.go, rename to proton_test.go
- Update test assertions to match real template IDs/severities

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
All tool calls in a batch now share a single semaphore queue
(default 16 slots). Removes the ExecParallel/ExecSequential
distinction — LLM-batched calls are independent by definition.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Add proxy support (WithProxy/SetProxy, wired from deps.ScannerProxy)
- Add -l/--list for multi-target input from file
- Add --stats/--no-stats/--silent output control (replace --quiet)
- Add detailed --template-list output (ID [severity] name, matching neutron)
- Expand normalizeShortFlags to full known-flags normalization
- Wire ResourceProvider from resources.Set instance method (not package func)
- Unify error wrapping to %w throughout
- Add readInputs() for multi-source input (aligned with readNeutronTargets)
- Add -l/--list to resolveRelativePaths file flags

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Resources:
- Replace 4 separate config maps + 4 default functions + 4 accessor
  methods with unified configs map + Config(engine, name) method
- Delete embeddedOrDefault() — call loadEmbeddedConfig() directly
- All engines now fall back to embedded data on nil/missing (consistent)

Deps:
- Add Deps.GetLogger() helper — replaces 5x repeated logger extraction
- Remove unused Deps.Model field
- Remove telemetry import from 5 register.go files

toolargs:
- Extract ResolveRelativePaths + ResolvePath to toolargs/resolve.go
  (was duplicated across 6 tools: proton, neutron, scan, spray, zombie, katana)
- Extract NormalizeFlags + CommonAliases to toolargs/normalize.go
  (shared nuclei-style flag normalization for proton and neutron)

Net: -180 lines across 14 files.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
FOFA simplified API auth (2023): only the API key is required and the
email never appears in the request URL. The engine required both FofaEmail
and FofaKey to register fofa, so accounts holding only a single API key
could not use `passive -s fofa`.

- NewUncoverEngine: register fofa on FofaKey presence (email optional);
  legacy "email:key" still accepted.
- Backfill FofaKey after uncover's GetKeys (which only populates the pair
  for two-segment "email:key" credentials).
- detectSources / richFofaAgent: gate on FofaKey only.
- Add key-only and legacy regression tests.

Refs #41

Co-Authored-By: Claude <noreply@anthropic.com>
Master did not compile, masked by the CI tidy gate blocking test/build.

1. commands.Deps.Model was removed in b59e32e (refactor: simplify scanner
   tool infrastructure) but two call sites still set it:
   - core/runner/app.go
   - cmd/aiscan/setup.go
   Nothing reads deps.Model, so drop the dead assignments.

2. telemetry.SDKRecover / SDKCapRecover were removed in 5821bee (unified
   panic recovery, replaced by SafeGo/SafeRun) but pkg/tools/scan/engine/
   sdk_e2e_test.go still exercised them. These functions are gone and have
   zero production callers, so remove the stale tests (keep the
   TestSDKGoRecoverDoesNotCrash test, which targets the still-present
   SDKGoRecover).

Co-Authored-By: Claude <noreply@anthropic.com>
go.mod was not tidy under the Go version mandated by CI
(go-version-file: go.mod = go 1.25.7, actions/go-versions build,
GOTOOLCHAIN=local): sigs.k8s.io/yaml was listed in both the direct and
indirect require blocks. Regenerate go.mod/go.sum so the tidy gate passes.

Note: local Go toolchains (go.dev/dl and golang.org/toolchain builds of
1.25.7) resolve the go.yaml.in/yaml graph differently than the
actions/go-versions 1.25.7 build, so this tidy output was generated inside
CI to match the project's authoritative toolchain.

Co-Authored-By: Claude <noreply@anthropic.com>
fix(recon): support FOFA key-only auth in passive recon
- Create skills/proton/SKILL.md with full usage docs (scan, pipe,
  template filtering, expressions, output, multi-target)
- Add proton to scanner list in skills/aiscan/SKILL.md
- Register proton in embed_expectations_test.go skill list

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Add proton to scanner list and edition table in both READMEs
- Add proton section in docs/reference.md with flag table and examples

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- errcheck: explicitly discard filepath.WalkDir return in proton
- govet: replace deprecated reflect.Ptr with reflect.Pointer
- staticcheck: fix unused result variable in agent_test.go
- race: use syncedBuffer in tui tests to prevent concurrent buffer access
- build: remove stale Model field from commands.Deps
- tests: remove tests for deleted SDKRecover/SDKCapRecover functions
- deps: bump sdk and utils/pty to pick up upstream race fixes

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Bumps [modernc.org/sqlite](https://gitlab.com/cznic/sqlite) from 1.40.1 to 1.53.0.
- [Changelog](https://gitlab.com/cznic/sqlite/blob/master/CHANGELOG.md)
- [Commits](https://gitlab.com/cznic/sqlite/compare/v1.40.1...v1.53.0)

---
updated-dependencies:
- dependency-name: modernc.org/sqlite
  dependency-version: 1.53.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
@dependabot dependabot Bot force-pushed the dependabot/go_modules/modernc.org/sqlite-1.53.0 branch from c374c27 to 23c6e24 Compare June 24, 2026 03:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants