fix(encoding/json): honor omitempty and unwrap boxed interface{} on marshal#144
fix(encoding/json): honor omitempty and unwrap boxed interface{} on marshal#144mck wants to merge 4 commits into
Conversation
…arshal
Two independent marshal bugs, both fixed in marshalValue/assignStructFields:
1. `,omitempty` struct tags were ignored: fields were emitted
unconditionally even when holding a Go zero value (zero number, empty
string, false, empty slice/map, nil pointer/interface). Added
jsonOmitEmpty (parses the tag's option list) and isEmptyValue (mirrors
Go's zero-value rules) and skip the field when both hold.
2. A boxed interface{} value leaked its wrapper into the output. The
runtime boxes a named/defined type's value stored in an interface as
`{ __goType, __goValue, ... }` so it can carry methods
(namedValueInterfaceValue in builtin/type.ts). marshalValue serialized
that wrapper object directly instead of the value it holds:
type Status int
var v any = Status(2) // boxed as {__goType, __goValue: 2, ...}
b, _ := json.Marshal(v)
// got: {"__goType":"...","__goValue":2,...}
// want: 2
Fixed by unwrapping `__goValue` (recursively, in case of nesting)
before falling through to the primitive/slice/map/struct handling,
guarded by !isStructValue so a real struct value's own fields are
never mistaken for a wrapper.
Extends gs/encoding/json/index.test.ts with both cases (zero vs
populated omitempty fields; a boxed primitive at the top level and
nested inside a struct field). All four new assertions fail on master
and pass with this fix.
Signed-off-by: Marco Christian Krenn <marco.krenn@gmail.com>
There was a problem hiding this comment.
Pull request overview
This PR fixes two correctness issues in the GoScript gs/encoding/json runtime’s Marshal path: it now honors ,omitempty on struct tags and unwraps GoScript’s boxed interface{} wrapper ({ __goType, __goValue, ... }) so the underlying value is serialized instead of the wrapper object.
Changes:
- Add boxed-
interface{}unwrapping inmarshalValueto serialize__goValue(recursively) rather than the wrapper. - Implement
,omitemptyhandling for struct fields viajsonOmitEmpty+isEmptyValue. - Extend JSON runtime tests to cover omitempty and boxed interface marshaling (top-level and nested).
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
| gs/encoding/json/index.ts | Adds boxed interface unwrapping and omitempty skipping logic during struct marshaling. |
| gs/encoding/json/index.test.ts | Adds regression tests for omitempty and boxed interface{} marshaling behavior. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| if (Array.isArray(t)) { | ||
| return t.length === 0 | ||
| } | ||
| if (t instanceof Map) { | ||
| return t.size === 0 | ||
| } | ||
| return false |
There was a problem hiding this comment.
Good catch, fixed in df3de9d: isEmptyValue now treats a zero-length Uint8Array as empty, and the omitempty test struct has a []byte field covering both the omit-when-empty and include-when-populated cases.
| {}, | ||
| ) | ||
| const holder = new FieldAlias() | ||
| holder._fields.Name.value = namedString as unknown as string |
There was a problem hiding this comment.
Fixed in df3de9d: replaced the double-cast with a cast on the VarRef itself, keeping the boxed wrapper value intact.
There was a problem hiding this comment.
The direction is right, but I found two marshal correctness cases that should be fixed before merge. I verified the current head is still 88ebd2f before re-checking this.
First, the boxed-interface unwrap is too broad. The new code treats any non-struct object with a __goValue property as a GoScript box:
if (
typeof v === 'object' &&
v !== null &&
'__goValue' in v &&
!isStructValue(v)
) {
return marshalValue((v as { __goValue: unknown }).__goValue)
}The runtime box created by namedValueInterfaceValue also carries a string __goType. __goValue alone is not enough evidence that the object is a box. Because Marshal otherwise accepts ordinary JS objects and lets them serialize as JSON objects, this currently changes accepted input like this:
const [data, err] = Marshal({ __goValue: 5 })
expect(err).toBeNull()
expect($.bytesToString(data)).toBe('{"__goValue":5}')Current result on this PR head:
Received: "5"
Suggested fix: gate this path on the actual boxed shape, at least typeof __goType === 'string' && '__goValue' in v, or put a shared boxed-value predicate near the boxed-value owner and call that from JSON.
Second, omitempty still misses empty byte slices. marshalValue serializes Uint8Array as the runtime byte-slice representation, but isEmptyValue only treats JS arrays and Map as empty collections. A json:",omitempty" byte-slice field still emits an empty base64 string:
class ByteOmitEmptyStruct {
public _fields = {
Data: $.varRef(new Uint8Array(0)),
}
static __typeInfo = $.registerStructType(
'test.ByteOmitEmptyStruct',
new ByteOmitEmptyStruct(),
[],
ByteOmitEmptyStruct,
[
{
name: 'Data',
key: 'Data',
type: {
kind: $.TypeKind.Slice,
elemType: { kind: $.TypeKind.Basic, name: 'uint8' },
},
tag: 'json:"data,omitempty"',
},
],
)
}
const [data, err] = Marshal(new ByteOmitEmptyStruct())
expect(err).toBeNull()
expect($.bytesToString(data)).toBe('{}')Current result on this PR head:
Received: "{\"data\":\"\"}"
Suggested fix: include t instanceof Uint8Array && t.length === 0 in isEmptyValue. If complex slice proxies can reach this path, cover those through the slice helper rather than adding another ad hoc shape check.
One smaller cleanup: the new JSON test uses namedString as unknown as string. Please avoid the double-cast. It hides the runtime boundary the test is trying to prove. A fixture field typed as unknown/any, or a test that asserts the boxed value at the interface boundary directly, would keep the contract clearer.
Checks I ran:
bun run vitest run gs/encoding/json/index.test.ts
# 20 passed
bun run vitest run gs/encoding/json/pr-review.test.ts
# 2 failed: the two cases above
Addresses review feedback on s4wave#144: - isEmptyValue did not treat a zero-length Uint8Array as empty. []byte marshals through the Uint8Array branch in marshalValue, so a json:",omitempty" tag on a []byte field never omitted an empty byte slice. Extends the existing Array.isArray check to also cover Uint8Array, and adds a Data field to the omitempty test struct to cover it (both the omit-when-empty and include-when-populated cases). - Replaced an `as unknown as string` double-cast in the boxed-interface test with a cast on the VarRef itself, avoiding the type-system escape hatch flagged in review while keeping the boxed wrapper value intact. Signed-off-by: Marco Christian Krenn <marco.krenn@gmail.com>
Addresses review feedback on s4wave#144: - isEmptyValue did not treat a zero-length Uint8Array as empty. []byte marshals through the Uint8Array branch in marshalValue, so a json:",omitempty" tag on a []byte field never omitted an empty byte slice. Extends the existing Array.isArray check to also cover Uint8Array, and adds a Data field to the omitempty test struct to cover it (both the omit-when-empty and include-when-populated cases). - Replaced an `as unknown as string` double-cast in the boxed-interface test with a cast on the VarRef itself, avoiding the type-system escape hatch flagged in review while keeping the boxed wrapper value intact. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Signed-off-by: Marco Christian Krenn <marco.krenn@gmail.com>
The boxed-interface unwrap in marshalValue treated any non-struct object
with a __goValue property as a runtime box, which changed accepted input:
Marshal({ __goValue: 5 }) serialized as 5 instead of {"__goValue":5}.
Add an isNamedValueBox predicate next to the box creator
(namedValueInterfaceValue in gs/builtin/type.ts) requiring both markers,
a string __goType plus __goValue, and use it from the JSON marshal path
so the notion of a box cannot drift from its owner. Regression test
covers the plain-object case.
Signed-off-by: Marco Christian Krenn <marco.krenn@gmail.com>
mck
left a comment
There was a problem hiding this comment.
Thanks for the thorough review, all three points are addressed.
The boxed-shape gate: fixed in f0043e4 using your second suggestion, a shared predicate next to the box creator. isNamedValueBox (gs/builtin/type.ts, right after namedValueInterfaceValue) requires both markers, a string __goType plus __goValue, and the JSON marshal path now calls that instead of checking __goValue alone. Your Marshal({ __goValue: 5 }) case is added as a regression test and passes ({"__goValue":5}).
The empty byte-slice omitempty and the double-cast: both were fixed in df3de9d, which crossed with your review (it landed for Copilot's overlapping comments a few minutes after you reviewed 88ebd2f). isEmptyValue treats a zero-length Uint8Array as empty, with omit-when-empty and include-when-populated cases on a []byte field, and the double-cast is gone.
Verify:
bun run vitest run gs/encoding/json/index.test.ts
21 passed. Full suite: typecheck clean, test:js 1274 passed / 4 skipped / 0 failed, go test ./... all packages ok, lint clean.
paralin
left a comment
There was a problem hiding this comment.
Requesting changes: the direction is right, but the current omitempty implementation still has two Go semantics regressions.
- Non-nil pointer/interface fields are omitted when their contents are zero.
At gs/encoding/json/index.ts:929, the skip decision calls isEmptyValue(ref.value) without looking at field.type. isEmptyValue unwraps VarRefs before testing zero-ness, so a non-nil *int pointing at 0 is treated as empty. Interface fields have the same issue: any(0), any(""), any(false), and any([]T{}) are non-nil interfaces and should still be emitted under omitempty.
Focused repro on this head:
expected {"ptr":0,"any":0}
received {}
Fix direction: make the empty check type-aware. For TypeKind.Pointer and TypeKind.Interface, omit only when the pointer/interface value itself is nil/null/undefined. Do not unwrap a non-nil pointer VarRef or inspect an interface’s dynamic value for the omitempty decision.
- Zero
bigintvalues are not treated as empty.
At gs/encoding/json/index.ts:960, isEmptyValue handles numeric zero only for typeof t === "number". The runtime already emits bigint JSON values, and GoScript uses bigint for int64/uint64, so int64 with json:",omitempty" and value 0n currently emits {"count":0} instead of {}.
Focused repro on this head:
expected {}
received {"count":0}
Fix direction: add a typeof t === "bigint" branch returning t === 0n, while preserving the pointer/interface rule above so any(0n) remains non-empty when the interface itself is non-nil.
…d bigint Addresses the CHANGES_REQUESTED review on s4wave#144. isEmptyValue unwrapped VarRefs before testing zero-ness without consulting the field type, which broke two Go semantics for omitempty: 1. Non-nil pointer and interface fields were omitted when their contents were zero. A non-nil *int pointing at 0, or any(0)/any("")/any(false)/ any([]T{}), is a non-nil pointer/interface and must still be emitted. isEmptyValue now takes the field type and, for TypeKind.Pointer and TypeKind.Interface, is empty only when the pointer/interface itself is nil (null/undefined). Non-nil pointer VarRefs and interface dynamic values are no longer unwrapped for the omitempty decision. 2. Zero bigint values were never treated as empty. GoScript uses bigint for int64/uint64, so int64 with json:",omitempty" and value 0n emitted {"count":0}. Added a typeof t === "bigint" branch returning t === 0n, ordered after the pointer/interface rule so any(0n) stays non-empty while the interface is non-nil. Regression tests cover a nil vs non-nil pointer/interface holding a zero value, and a zero vs non-zero int64, plus a non-nil interface holding a zero bigint. All fail on the prior head and pass with this change. Signed-off-by: Marco Christian Krenn <marco.krenn@gmail.com>
f0043e4 to
7756eb9
Compare
|
Thanks for the careful review. Both omitempty regressions are fixed in 7756eb9. isEmptyValue now takes the field type. For TypeKind.Pointer and TypeKind.Interface it is empty only when the pointer or interface itself is nil (null/undefined), so a non-nil *int pointing at 0, and any(0), any(""), any(false), any([]T{}), are all still emitted. Non-nil pointer VarRefs and interface dynamic values are no longer unwrapped for the omitempty decision, exactly per your fix direction. Marshal({"ptr":0,"any":0}) now comes out as {"ptr":0,"any":0} instead of {}. For zero bigint, added a typeof t === "bigint" branch returning t === 0n, ordered after the pointer/interface rule so int64 with omitempty and value 0n now yields {} while any(0n) stays non-empty because the interface is non-nil. New regression tests in gs/encoding/json/index.test.ts cover nil vs non-nil pointer/interface holding a zero value, zero vs non-zero int64, and a non-nil interface holding a zero bigint. All fail on the prior head and pass with this change. For completeness on the earlier round: the too-broad boxed-interface unwrap is gated on the real box shape via isNamedValueBox in gs/builtin/type.ts (requires both a string __goType and __goValue), the empty []byte case is handled in isEmptyValue via the Uint8Array length check, and the test double-cast was removed. Verify: |
| function isEmptyValue(v: unknown, fieldType?: unknown): boolean { | ||
| const t = $.isVarRef(v) ? (v as $.VarRef<unknown>).value : v | ||
| if (isPointerOrInterfaceFieldType(fieldType)) { | ||
| return t === null || t === undefined | ||
| } | ||
| if (t === null || t === undefined) { | ||
| return true | ||
| } |
Part of #142 (items 1 and 2). Independent of #143 (different area of the codebase). Touches
gs/encoding/json/index.tslike #145 and #146, but a different area (Marshal's encode path vs theirUnmarshaldecode path);index.tsitself merges cleanly, only the sharedindex.test.tshas a trivial fixture-insertion conflict (see #145's note).Two separate
Marshalbugs, grouped into one PR because both live inmarshalValue/assignStructFieldsand are small on their own.First,
,omitemptystruct tags were ignored: fields tagged,omitemptywere emitted unconditionally, even when holding a Go zero value (zero number, empty string,false, empty slice/map, nil pointer/interface).Second, a boxed
interface{}value leaked its wrapper into the output. The runtime boxes a named/defined type's value stored in an interface as{ __goType, __goValue, ... }so it can carry methods (namedValueInterfaceValueings/builtin/type.ts), andmarshalValueserialized that wrapper object directly instead of the value it holds.The fix adds
jsonOmitEmpty(parses the tag's option list) andisEmptyValue(mirrors Go's zero-value rules for numbers, strings, bools, slices, maps,Uint8Arrayas[]byte, and nil); a field is skipped when both hold.marshalValuenow unwraps__goValue(recursively, in case of nesting) before falling through to primitive/slice/map/struct handling, guarded by!isStructValue(v)so a real struct value is never mistaken for a wrapper.Tests extend
gs/encoding/json/index.test.tswith zero-valued vs populatedomitemptyfields (string/int/slice/[]byte) and a boxed primitive at the top level and nested inside a struct field. All new assertions fail onmasterand pass with this fix.Verify:
20 passed. Full suite:
bun run typecheckclean,bun run test:js1273 passed / 4 skipped / 0 failed,go test ./...all packages green,bun run lint:jsclean.Found while transpiling and running a real ~100k-LOC Go interpreter/parser package through goscript.