Skip to content

Regex Table Names - #168

Open
GALHP wants to merge 15 commits into
clonio-dev:mainfrom
GALHP:feat/regex-table-names
Open

Regex Table Names#168
GALHP wants to merge 15 commits into
clonio-dev:mainfrom
GALHP:feat/regex-table-names

Conversation

@GALHP

@GALHP GALHP commented Aug 13, 2026

Copy link
Copy Markdown

Summary

Lets a tables: key in a .cloning.yaml be a slash-delimited regex instead
of only an exact name; at run time each regex expands against the real source
tables, so one entry configures a whole family.

Stacked PR — base: feat/improve-chunked-insert-performance · 4th of 6 ·
Part of #164. No code dependency on #165#167 (stacked only)

Motivation

Our schemas have many similarly-named tables — shared prefixes/suffixes, dynamic
datetime suffixes (application_logs_archive_02_2026, …_03_2026, …). Listing
each is verbose and needs re-editing as tables appear. One regex configures them
all and makes configs reusable across stages.

tables:
  '/^application_logs_archive_\d{2}_\d{4}$/':
    rows: { strategy: last, limit: 1, clear: delete }

What's changed

  • App\Services\Cloning\TableConfigResolver (new) — after schema inspection,
    expands table keys against the real source tables and returns a config whose
    tables all carry concrete names, so the orchestrator, getTable(),
    dependency sort, and audit code work unchanged.
  • RunCommand — invokes the resolver right after SchemaInspector::inspect().
  • CloningYamlValidator — rejects a non-compiling regex key in Phase-1,
    before any DB work.
  • Docs: docs/cloning-yaml.md, specs/PRD-cloning-yaml-schema.md.

Semantics

  • /…/ (trailing flags allowed) = regex; anything else = literal. No glob/*
    (regex covers it via .*).
  • Case-insensitive, reusing the PII column matcher's /…/-vs-literal
    convention (PiiMatcherSetData::matchesPattern()).
  • Last matching entry wins — broad regex first, then override specific tables
    with a narrower regex/literal.
  • A literal key for an absent table is still not found; a regex matching
    nothing is skipped.
  • Regex keys are not expanded for key remapping or --skip-tables /
    --only-tables (literal-only) — documented.
  • Single-quote regex keys in YAML so \d, \w, … aren't eaten as escapes.

Testing

TableConfigResolverTest (expansion, last-match-wins, literal-not-found
preserved, skip expansion, no-match ignored, all-literal backward-compat,
case-insensitivity) + validator coverage for an invalid regex key. Verified
end-to-end against SQLite.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant