Skip to content

Add cross-language round-trip contract tests — catch serialization mismatches before production #87

Description

@haasonsaas

Problem

The existing contract test (contract_test.go) validates that Go's protojson uses UseProtoNames: true (snake_case field names). This catches one class of issue, but misses cross-language mismatches:

  • evalops/chat#948: @bufbuild/protobuf v1 generates different TypeScript types than v2
  • evalops/fermata#9239: Python protobuf version produces incompatible serialization

These bugs surface at runtime in production when a Go service sends a proto-JSON payload that a TypeScript or Python consumer can't deserialize — or deserializes with silently wrong values.

Proposed solution

1. Cross-language round-trip tests

For each message type with a contract fixture:

Go: serialize fixture → JSON
TypeScript: deserialize JSON → re-serialize → JSON₂
Python: deserialize JSON → re-serialize → JSON₃
Assert: JSON == JSON₂ == JSON₃

This catches:

  • Field name casing mismatches (camelCase vs snake_case)
  • Enum serialization differences (string vs int)
  • Timestamp format differences
  • oneof handling differences
  • Default value presence/absence

2. Fixture generation from proto

Generate random valid messages using proto reflection:

func FuzzRoundTrip(f *testing.F) {
    f.Fuzz(func(t *testing.T, data []byte) {
        msg := &governancev1.EvaluateActionRequest{}
        if err := proto.Unmarshal(data, msg); err != nil {
            return // invalid proto, skip
        }
        json, _ := protojson.MarshalOptions{UseProtoNames: true}.Marshal(msg)
        // ... deserialize in TS and Python, compare
    })
}

3. CI integration

Add a test job that:

  1. Installs Go, Node.js, and Python
  2. Runs the round-trip tests against gen/go, gen/ts, gen/python
  3. Fails the proto PR if any language produces different serialization

Relationship to existing work

  • Extends the existing contract_test.go pattern
  • Complements the contractfixtures/ directory (fixtures become the test corpus)
  • Catches the class of bugs from chat#948 and fermata#9239 at PR time instead of production

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions