Skip to content

feat(base): add typed NDJSON workflows for professional data analysis #2196

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
zgz2048 wants to merge 15 commits into
base: main
Choose a base branch
from
Open

feat(base): add typed NDJSON workflows for professional data analysis #2196

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
15 commits
Select commit Hold shift + click to select a range
d76686d
feat(base): add typed NDJSON data analysis workflow
zgz2048 Aug 5, 2026
99d6a7d
docs(base): simplify cloud pagination guidance
zgz2048 Aug 5, 2026
1e63459
docs(base): clarify Link relation IDs
zgz2048 Aug 5, 2026
c476b25
feat(base): query NDJSON records with jq
zgz2048 Aug 5, 2026
81ad082
docs(base): route analysis through built-in jq
zgz2048 Aug 5, 2026
81336a4
docs: clarify per-table local analysis limit
zgz2048 Aug 5, 2026
3a9edb6
docs: simplify base analysis guidance
zgz2048 Aug 5, 2026
86fe3db
docs: centralize base analysis routing
zgz2048 Aug 6, 2026
b9fcde5
docs(base): refine cell value and datetime guidance
zgz2048 Aug 6, 2026
1c82d24
test(base): align ignored field fixtures
zgz2048 Aug 6, 2026
0dcf39b
docs(base): add semantic analysis routing
zgz2048 Aug 6, 2026
9ea07db
feat(base): improve NDJSON analysis workflow
zgz2048 Aug 7, 2026
bca0723
docs(base): centralize filter predicate examples
zgz2048 Aug 7, 2026
3d6ab8d
feat(base): clarify record get export scope
zgz2048 Aug 7, 2026
ebd4a36
Merge remote-tracking branch 'upstream/main' into codex/base-record-n…
zgz2048 Aug 7, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 14 additions & 6 deletions shortcuts/base/base_execute_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2028,7 +2028,7 @@ func TestBaseRecordExecuteReadCreateDelete(t *testing.T) {
factory, stdout, reg := newExecuteFactory(t)
reg.Register(&httpmock.Stub{
Method: "GET",
URL: "field_id=Name&field_id=Age&limit=2&offset=0",
URL: "field_id=Name&field_id=Age&field_id=Formula&limit=2&offset=0",
Body: map[string]interface{}{
"code": 0,
"data": map[string]interface{}{
Expand All @@ -2044,11 +2044,15 @@ func TestBaseRecordExecuteReadCreateDelete(t *testing.T) {
"record_scope": "all_records",
"field_scope": "selected_fields",
},
"ignored_fields": []interface{}{"Formula"},
"ignored_fields": []interface{}{map[string]interface{}{
"id": "fld_formula",
"name": "Formula",
"reason": "UNSUPPORTED: formula field cannot be read through OpenAPI because this base uses an old schema version without backend formula computation.",
}},
},
},
})
if err := runShortcut(t, BaseRecordList, []string{"+record-list", "--base-token", "app_x", "--table-id", "tbl_x", "--limit", "2", "--field-id", "Name", "--field-id", "Age"}, factory, stdout); err != nil {
if err := runShortcut(t, BaseRecordList, []string{"+record-list", "--base-token", "app_x", "--table-id", "tbl_x", "--limit", "2", "--field-id", "Name", "--field-id", "Age", "--field-id", "Formula"}, factory, stdout); err != nil {
t.Fatalf("err=%v", err)
}
got := stdout.String()
Expand All @@ -2057,7 +2061,7 @@ func TestBaseRecordExecuteReadCreateDelete(t *testing.T) {
"| _record_id | Name | Age |",
"| rec_1 | Alice | 18 |",
"Meta: count=2; has_more=false; record_scope=all_records; field_scope=selected_fields; ignored_fields=1",
"Ignored fields: Formula",
`Ignored fields: {"id":"fld_formula","name":"Formula","reason":"UNSUPPORTED: formula field cannot be read through OpenAPI because this base uses an old schema version without backend formula computation."}`,
} {
if !strings.Contains(got, want) {
t.Fatalf("stdout missing %q:\n%s", want, got)
Expand Down Expand Up @@ -2814,11 +2818,15 @@ func TestBaseRecordExecuteReadCreateDelete(t *testing.T) {
Body: map[string]interface{}{
"code": 0,
"data": map[string]interface{}{
"ignored_fields": []interface{}{"Formula"},
"ignored_fields": []interface{}{map[string]interface{}{
"id": "fld_formula",
"name": "Formula",
"reason": "READONLY: formula field cannot be written through OpenAPI.",
}},
},
},
})
if err := runShortcut(t, BaseRecordBatchUpdate, []string{"+record-batch-update", "--base-token", "app_x", "--table-id", "tbl_x", "--json", `{"update_records":{"rec_1":{"Status":["Done"]}}}`}, factory, stdout); err != nil {
if err := runShortcut(t, BaseRecordBatchUpdate, []string{"+record-batch-update", "--base-token", "app_x", "--table-id", "tbl_x", "--json", `{"update_records":{"rec_1":{"Status":["Done"],"Formula":"ignored"}}}`}, factory, stdout); err != nil {
Comment on lines +2821 to +2829

@coderabbitai coderabbitai Bot Aug 7, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

Assert the structured ignored-field contract.

The test only checks that stdout contains "ignored_fields" and "Formula". It does not verify id or reason. A regression to name-only ignored fields would pass.

Assert the complete structured value, as the markdown test does.

As per coding guidelines, “Every behavior change must have an accompanying test, and contract tests must assert the changed field or behavior directly so reverting the implementation causes failure.”

🤖 Prompt for AI Agents 
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@shortcuts/base/base_execute_test.go` around lines 2821 - 2829, The
BaseRecordBatchUpdate test should assert the complete ignored_fields structure
rather than only checking that stdout contains “ignored_fields” and “Formula”.
Validate the ignored field’s id, name, and reason values, matching the
structured contract asserted by the corresponding markdown test.

Source: Coding guidelines

t.Fatalf("err=%v", err)
}
if got := stdout.String(); !strings.Contains(got, `"ignored_fields"`) || !strings.Contains(got, `"Formula"`) {
Expand Down
33 changes: 20 additions & 13 deletions shortcuts/base/base_shortcuts_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -360,16 +360,18 @@ func TestBaseRecordReadHelpGuidesAgents(t *testing.T) {
"view ID or name; omit for reading all table records, or set to read a user-specified or temporary filtered/sorted view",
`filter JSON object or @file`,
`sort JSON array or @file`,
"pagination size, range 1-200",
"output format: markdown (default) | json",
"maximum records to return; range 1-200, or 1-2000 for ndjson",
"ndjson typed artifact (preferred for analysis)",
"preferred analysis output: relative .ndjson output path",
},
wantTips: []string{
"lark-cli base +record-list --base-token <base_token> --table-id <table_id> --limit 50",
"lark-cli base +record-list --base-token <base_token> --table-id <table_id> --field-id Name --field-id Status --limit 50",
"Text equality filter",
"Option intersection filter",
"Query priority",
"Default output is markdown",
"Example for analysis",
"prefer --output ./records.ndjson --minimal-stdout",
"Use --field-id repeatedly to keep output small",
},
},
Expand All @@ -382,15 +384,17 @@ func TestBaseRecordReadHelpGuidesAgents(t *testing.T) {
"field ID or name to search",
`filter JSON object or @file`,
`sort JSON array or @file`,
"output format: markdown (default) | json",
"ndjson typed artifact (preferred for analysis)",
"preferred analysis output: relative .ndjson output path",
},
wantTips: []string{
"Example: lark-cli base +record-search",
"Example with filter/sort JSON",
"Text equality filter",
"Query priority",
"Use --json only when you need to pass the full search body directly",
"Default output is markdown",
"Example for analysis",
"prefer --output ./records.ndjson --minimal-stdout",
},
},
{
Expand All @@ -399,15 +403,16 @@ func TestBaseRecordReadHelpGuidesAgents(t *testing.T) {
wantHelp: []string{
"record ID (repeatable)",
"field ID or name to project; repeat to keep only needed columns",
"output format: markdown (default) | json",
"ndjson typed artifact (preferred for analysis)",
"preferred analysis output: relative .ndjson output path",
},
wantTips: []string{
"lark-cli base +record-get --base-token <base_token> --table-id <table_id> --record-id <record_id>",
"lark-cli base +record-get --base-token <base_token> --table-id <table_id> --record-id rec_001 --record-id rec_002 --field-id Name --field-id Status",
"Default output is markdown",
"Example for analysis input",
"prefer --output ./records.ndjson --minimal-stdout",
"projection boundary",
"record_id is already known",
"lark-base record read SOP",
},
},
}
Expand Down Expand Up @@ -448,8 +453,8 @@ func TestBasePaginationHelpShowsDefaults(t *testing.T) {
{name: "table list", shortcut: BaseTableList, flag: "limit", defaultVal: "50", help: "pagination size, range 1-100"},
{name: "field list", shortcut: BaseFieldList, flag: "limit", defaultVal: "100", help: "pagination size, range 1-200"},
{name: "field search options", shortcut: BaseFieldSearchOptions, flag: "limit", defaultVal: "30", help: "pagination size, range 1-200"},
{name: "record list", shortcut: BaseRecordList, flag: "limit", defaultVal: "100", help: "pagination size, range 1-200"},
{name: "record search", shortcut: BaseRecordSearch, flag: "limit", defaultVal: "10", help: "pagination size, range 1-200"},
{name: "record list", shortcut: BaseRecordList, flag: "limit", defaultVal: "100", help: "maximum records to return; range 1-200, or 1-2000 for ndjson"},
{name: "record search", shortcut: BaseRecordSearch, flag: "limit", defaultVal: "10", help: "maximum records to return; range 1-200, or 1-2000 for ndjson"},
{name: "view list", shortcut: BaseViewList, flag: "limit", defaultVal: "100", help: "pagination size, range 1-200"},
{name: "form list", shortcut: BaseFormsList, flag: "page-size", defaultVal: "100", help: "page size per request, range 1-100"},
{name: "workflow list", shortcut: BaseWorkflowList, flag: "page-size", defaultVal: "100", help: "page size per request, range 1-100"},
Expand Down Expand Up @@ -932,11 +937,13 @@ func TestBaseRecordWriteHelpGuidesAgents(t *testing.T) {
`{"Parent Link":[{"id":"rec_xxx"}]}`,
"do not look for parent_record_id or a separate child-record API",
"CellValue happy path: text/phone/url",
"select (multiple=false) -> \"Todo\"",
"select (multiple=true) -> [\"Tag A\",\"Tag B\"]",
"datetime -> \"2026-03-24 10:00:00\"",
"select -> [\"Todo\"] or [\"Tag A\",\"Tag B\"]",
"when multiple=false, the array can contain only one option",
"datetime -> \"2026-03-24 10:00\"",
"checkbox -> true/false",
`ID-based CellValue: user/group/link fields use arrays like [{"id":"ou_xxx"}]`,
"User and group fields always use arrays",
"when multiple=false, the array can contain only one item",
`location uses {"lng":116.397428,"lat":39.90923}`,
"Do not guess user/chat/linked-record IDs or location coordinates",
"lark-base-cell-value.md",
Expand Down
Loading
Loading