JITSU-160 fix(console): allow removing entries from free-form config maps - #1443
JITSU-160 fix(console): allow removing entries from free-form config maps#1443absorbb wants to merge 1 commit into
Conversation
Destination and service updates deepMerge the incoming patch into the
stored object. deepMerge iterates source keys only, so it can add or
overwrite but never remove — which makes a warehouse's `parameters` map
grow-only: adding and changing entries works, deleting one silently
keeps it.
Merge semantics can't just be dropped: they're what makes partial
updates work (write-key auto-save, `jitsu-cli config update --field`,
the MCP `update_resource` tool) and what preserves masked secrets the
client never sends back.
So merge everything as before, except fields whose schema leaves the key
set open — where the key set is itself user data and must be replaced:
- destinations (zod): ZodRecord, .passthrough(), or a ZodObject with a
non-ZodNever catchall. The catchall is the signal, not an empty
shape — Mysql's `parameters` declares `tls` alongside its catchall.
- services (declarative): `type: "object"` carrying
additionalProperties in the connector's connectionSpecification,
walking nested objects and oneOf/anyOf like getServiceSecretPaths.
Replacement is keyed off the patch, so a map the caller didn't send is
left alone and partial updates keep working. A map the caller did send —
including an empty {} — replaces what's stored.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
There was a problem hiding this comment.
Reviewed the schema-merge changes in config-objects.ts and the new free-form-maps.ts + unit tests. The overall direction makes sense and the regression test coverage is helpful. I left two inline comments for edge cases that may still cause incorrect behavior in partial updates / connector-spec variants.
| */ | ||
| export function replaceFreeFormMaps<T>(merged: T, patch: any, freeFormPaths: string[]): T { | ||
| for (const path of freeFormPaths) { | ||
| if (has(patch, path)) { |
There was a problem hiding this comment.
Potential regression question: this replaces the entire map whenever the patch contains the map path at all. For CLI/MCP-style field updates like --field.parameters.TIMEZONE=UTC (which typically serializes as {"parameters":{"TIMEZONE":"UTC"}}), this would drop sibling keys that weren’t part of that one-key patch. Is that destructive behavior intentional for field-level updates?
| } | ||
| const branches = schema?.oneOf || schema?.anyOf; | ||
| if (branches) { | ||
| branches.forEach((branch: any) => branch?.properties && walk(branch.properties, path, depth + 1)); |
There was a problem hiding this comment.
Could we also handle branches where the oneOf/anyOf branch itself is an open map (type: "object" + additionalProperties) instead of having nested properties? Right now we only recurse into branch.properties, so those branch shapes would be missed and map deletions would still merge incorrectly.
There was a problem hiding this comment.
Reviewed config-objects merge behavior plus the new free-form map discovery helpers/tests. Nice fix for map-key removal overall, but I found two correctness edge cases worth addressing (left as inline comments):
- Nested field updates into a free-form map currently replace the whole map and can drop sibling keys unexpectedly.
- JSON Schema branch traversal misses oneOf/anyOf branches where the branch itself is an open map (
additionalProperties) without nestedproperties.
Everything else in this change looks consistent with the intended behavior.
| */ | ||
| export function replaceFreeFormMaps<T>(merged: T, patch: any, freeFormPaths: string[]): T { | ||
| for (const path of freeFormPaths) { | ||
| if (has(patch, path)) { |
There was a problem hiding this comment.
Possible regression: has(patch, path) is true even for nested field patches like {"parameters":{"TIMEZONE":"UTC"}} produced by CLI --field.parameters.TIMEZONE=UTC. That means this line replaces the whole parameters object with only TIMEZONE, dropping existing sibling keys the caller did not touch. Is that destructive behavior intended for field-level updates?
| } | ||
| const branches = schema?.oneOf || schema?.anyOf; | ||
| if (branches) { | ||
| branches.forEach((branch: any) => branch?.properties && walk(branch.properties, path, depth + 1)); |
There was a problem hiding this comment.
This only recurses into branch.properties. If a oneOf/anyOf branch itself is an open map (e.g. { "type": "object", "additionalProperties": { ... } } without nested properties), we won't mark it as free-form, so deletions under that branch will still be merge-only and keys can get stuck. Could we handle branch-level additionalProperties too?
Problem
Removing an entry from a data-warehouse destination's
parametersdoesn't stick. Adding and changing entries works; deleting one silently keeps it.Config updates deepMerge the incoming patch into the stored object (
config-objects.ts→deepMerge), anddeepMergereduces over source keys only, so keys present in the stored object but absent from the patch are never touched. Invisible for fixed-key fields — the form always submits every key — butparametersis a map whose key set is the data, so it can only ever grow.Reproduced against the real
deepMergewith a stored Snowflakeparameters:TIMEZONEQUERY_TAGparameters: {}Why not just make updates replace
mergerunsremoveMaskedValuesfirst, so the UI's masked secrets are stripped from the patch anddeepMergeis the only reason the storedpassword/privateKeysurvives a save. Merge semantics are also load-bearing for genuine partial updates:ApiKeyEditor.tsxauto-save on adding a write key →{ privateKeys: [...] }onlyProfileBuilderPage.tsxsave/rollback →{ draft, type }onlyjitsu-cli config update --field.path=valueupdate_resourcetool, documented as "merged into the existing object"The CLI is published to npm, so old versions would keep sending partial PUTs indefinitely — flipping PUT to replace would silently destroy config for anyone who hasn't upgraded.
Fix
Merge everything as before, except fields whose schema leaves the key set open — those get replaced.
ZodRecord,.passthrough(), or aZodObjectwith a non-ZodNevercatchall. The catchall is the signal, not an empty shape: Mysql'sparametersdeclarestlsalongside its catchall, so an "isobject({})" test would miss it. Wrappers (.optional()/.default()/.nullable()) are unwrapped and declared sub-objects are recursed into.type: "object"withadditionalProperties. Walks nested objects andoneOf/anyOfbranches, mirroringgetServiceSecretPaths.Replacement is keyed off the patch (
has(patch, path)): a map the caller didn't send is left alone, so every partial-update caller above keeps working and secrets stay put. A map the caller did send — including an empty{}— replaces what's stored.Affects
parameterson ClickHouse, Snowflake and Mysql today, and any future field declared the same way, without a hardcoded list.Testing
__tests__/unit/free-form-maps.test.ts— 13 cases: detection across the three warehouses plus a closed-schema and unknown destination; add / change / remove-one / remove-all; other fields still merging; a partial patch leaving both the map and the stored secret intact; and the JSON-Schema walk over nesting,oneOf, closed objects and empty specs.vitest run --project unit— 43 passedtsc --noEmit— cleanNote
There is one behaviour change: a partial patch that touches a map now replaces it rather than merging into it, so
parameters: { tls: "true" }drops the other entries. That's inherent to treating the map as the unit of edit and is what makes removal work.Related:
config-objects.ts:198-201carries a// TODO: dirty workaround for not be able to clear authorizedJavaScriptDomains— same root cause, left alone here.🤖 Generated with Claude Code
JITSU-160