Skip to content

JITSU-160 fix(console): allow removing entries from free-form config maps - #1443

Open
absorbb wants to merge 1 commit into
newjitsufrom
fix/config-free-form-map-removal
Open

JITSU-160 fix(console): allow removing entries from free-form config maps#1443
absorbb wants to merge 1 commit into
newjitsufrom
fix/config-free-form-map-removal

Conversation

@absorbb

@absorbb absorbb commented Aug 5, 2026

Copy link
Copy Markdown
Contributor

Problem

Removing an entry from a data-warehouse destination's parameters doesn't stick. Adding and changing entries works; deleting one silently keeps it.

Config updates deepMerge the incoming patch into the stored object (config-objects.tsdeepMerge), and deepMerge reduces over source keys only, so keys present in the stored object but absent from the patch are never touched. Invisible for fixed-key fields — the form always submits every key — but parameters is a map whose key set is the data, so it can only ever grow.

Reproduced against the real deepMerge with a stored Snowflake parameters:

patch before
add TIMEZONE
change QUERY_TAG
drop one key ❌ key survives
parameters: {} ❌ all keys survive

Why not just make updates replace

merge runs removeMaskedValues first, so the UI's masked secrets are stripped from the patch and deepMerge is the only reason the stored password / privateKey survives a save. Merge semantics are also load-bearing for genuine partial updates:

  • ApiKeyEditor.tsx auto-save on adding a write key → { privateKeys: [...] } only
  • ProfileBuilderPage.tsx save/rollback → { draft, type } only
  • jitsu-cli config update --field.path=value
  • the MCP update_resource tool, documented as "merged into the existing object"

The CLI is published to npm, so old versions would keep sending partial PUTs indefinitely — flipping PUT to replace would silently destroy config for anyone who hasn't upgraded.

Fix

Merge everything as before, except fields whose schema leaves the key set open — those get replaced.

  • Destinations (zod)ZodRecord, .passthrough(), or a ZodObject with a non-ZodNever catchall. The catchall is the signal, not an empty shape: Mysql's parameters declares tls alongside its catchall, so an "is object({})" test would miss it. Wrappers (.optional() / .default() / .nullable()) are unwrapped and declared sub-objects are recursed into.
  • Services (declarative) — connectors carry Airbyte JSON Schema, not zod, so the equivalent signal is type: "object" with additionalProperties. Walks nested objects and oneOf/anyOf branches, mirroring getServiceSecretPaths.

Replacement is keyed off the patch (has(patch, path)): a map the caller didn't send is left alone, so every partial-update caller above keeps working and secrets stay put. A map the caller did send — including an empty {} — replaces what's stored.

Affects parameters on ClickHouse, Snowflake and Mysql today, and any future field declared the same way, without a hardcoded list.

Testing

__tests__/unit/free-form-maps.test.ts — 13 cases: detection across the three warehouses plus a closed-schema and unknown destination; add / change / remove-one / remove-all; other fields still merging; a partial patch leaving both the map and the stored secret intact; and the JSON-Schema walk over nesting, oneOf, closed objects and empty specs.

  • vitest run --project unit — 43 passed
  • tsc --noEmit — clean

Note

There is one behaviour change: a partial patch that touches a map now replaces it rather than merging into it, so parameters: { tls: "true" } drops the other entries. That's inherent to treating the map as the unit of edit and is what makes removal work.

Related: config-objects.ts:198-201 carries a // TODO: dirty workaround for not be able to clear authorizedJavaScriptDomains — same root cause, left alone here.

🤖 Generated with Claude Code

JITSU-160

Destination and service updates deepMerge the incoming patch into the
stored object. deepMerge iterates source keys only, so it can add or
overwrite but never remove — which makes a warehouse's `parameters` map
grow-only: adding and changing entries works, deleting one silently
keeps it.

Merge semantics can't just be dropped: they're what makes partial
updates work (write-key auto-save, `jitsu-cli config update --field`,
the MCP `update_resource` tool) and what preserves masked secrets the
client never sends back.

So merge everything as before, except fields whose schema leaves the key
set open — where the key set is itself user data and must be replaced:

  - destinations (zod): ZodRecord, .passthrough(), or a ZodObject with a
    non-ZodNever catchall. The catchall is the signal, not an empty
    shape — Mysql's `parameters` declares `tls` alongside its catchall.
  - services (declarative): `type: "object"` carrying
    additionalProperties in the connector's connectionSpecification,
    walking nested objects and oneOf/anyOf like getServiceSecretPaths.

Replacement is keyed off the patch, so a map the caller didn't send is
left alone and partial updates keep working. A map the caller did send —
including an empty {} — replaces what's stored.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@absorbb absorbb added the deploy:console Auto-deploy console to beta when this PR merges (JITSU-68) label Aug 5, 2026
@absorbb absorbb changed the title fix(console): allow removing entries from free-form config maps JITSU-160 fix(console): allow removing entries from free-form config maps Aug 5, 2026

@jitsu-code-review jitsu-code-review Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed the schema-merge changes in config-objects.ts and the new free-form-maps.ts + unit tests. The overall direction makes sense and the regression test coverage is helpful. I left two inline comments for edge cases that may still cause incorrect behavior in partial updates / connector-spec variants.

*/
export function replaceFreeFormMaps<T>(merged: T, patch: any, freeFormPaths: string[]): T {
for (const path of freeFormPaths) {
if (has(patch, path)) {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Potential regression question: this replaces the entire map whenever the patch contains the map path at all. For CLI/MCP-style field updates like --field.parameters.TIMEZONE=UTC (which typically serializes as {"parameters":{"TIMEZONE":"UTC"}}), this would drop sibling keys that weren’t part of that one-key patch. Is that destructive behavior intentional for field-level updates?

}
const branches = schema?.oneOf || schema?.anyOf;
if (branches) {
branches.forEach((branch: any) => branch?.properties && walk(branch.properties, path, depth + 1));

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we also handle branches where the oneOf/anyOf branch itself is an open map (type: "object" + additionalProperties) instead of having nested properties? Right now we only recurse into branch.properties, so those branch shapes would be missed and map deletions would still merge incorrectly.

@jitsu-code-review jitsu-code-review Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed config-objects merge behavior plus the new free-form map discovery helpers/tests. Nice fix for map-key removal overall, but I found two correctness edge cases worth addressing (left as inline comments):

  1. Nested field updates into a free-form map currently replace the whole map and can drop sibling keys unexpectedly.
  2. JSON Schema branch traversal misses oneOf/anyOf branches where the branch itself is an open map (additionalProperties) without nested properties.

Everything else in this change looks consistent with the intended behavior.

*/
export function replaceFreeFormMaps<T>(merged: T, patch: any, freeFormPaths: string[]): T {
for (const path of freeFormPaths) {
if (has(patch, path)) {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Possible regression: has(patch, path) is true even for nested field patches like {"parameters":{"TIMEZONE":"UTC"}} produced by CLI --field.parameters.TIMEZONE=UTC. That means this line replaces the whole parameters object with only TIMEZONE, dropping existing sibling keys the caller did not touch. Is that destructive behavior intended for field-level updates?

}
const branches = schema?.oneOf || schema?.anyOf;
if (branches) {
branches.forEach((branch: any) => branch?.properties && walk(branch.properties, path, depth + 1));

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This only recurses into branch.properties. If a oneOf/anyOf branch itself is an open map (e.g. { "type": "object", "additionalProperties": { ... } } without nested properties), we won't mark it as free-form, so deletions under that branch will still be merge-only and keys can get stuck. Could we handle branch-level additionalProperties too?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

deploy:console Auto-deploy console to beta when this PR merges (JITSU-68)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant