Add company-data-copier command#264
Conversation
Trigger the platform Data Copier to copy a source company's data into a brand-new company in a destination (development) firm, so BSO developers can reproduce a client's situation in a dev firm without touching the production firm. - sfApi.runCompanyDataCopier: POST /api/v4/f/:firm_id/company_data_copier/run - index.copyCompanyData: validates input and reports the enqueued job - bin/cli.js: 'company-data-copier' command (-c source company, -l source period ids (variadic), -f destination firm) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A 422 is not specifically a permissions problem, so the hardcoded "You don't have the rights to update the previous parameters" line was misleading (e.g. it appeared on a Data Copier "source_company_id does not exist" error). Surface only the real API error message. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Covers the sfApi HTTP call (axios-mock-adapter), the E2E command test in tests/bin/cli/, and Commander wiring in tests/bin/cli.test.js, plus the TESTS.md catalogue entries. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
WalkthroughAdds a new ChangesCompany Data Copier Feature
Estimated code review effort: 3 (Moderate) | ~25 minutes 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Comment |
There was a problem hiding this comment.
🧹 Nitpick comments (2)
tests/bin/cli/company-data-copier.test.js (2)
77-84: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low valueTest only covers
undefinedresponse, not a "truthy response without.data" case.The guard in
copyCompanyDatais!response || !response.data; only the!responsebranch is exercised here. Consider adding a case likeSF.runCompanyDataCopier.mockResolvedValue({})to also cover the "response exists but has nodata" path.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@tests/bin/cli/company-data-copier.test.js` around lines 77 - 84, The current test for copyCompanyData only covers the undefined response path, but not the case where SF.runCompanyDataCopier returns an object without data. Update the company-data-copier test suite to add a separate case that mock-resolves a truthy response with no data (for example via the copyCompanyData flow) and asserts the same error/false behavior, so both branches of the !response || !response.data guard are exercised.
21-44: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low valueUnused e2e-style scaffolding in a pure unit test suite.
tempDir/chdir/process.exitmocking (lines 24-30, 41-43) are never exercised —copyCompanyDatadoesn't touch the filesystem or callprocess.exit. This looks copy-pasted from an e2e CLI test file and just adds noise.♻️ Suggested simplification
-const fsPromises = require("fs").promises; -const path = require("path"); -const os = require("os"); - jest.mock("consola"); jest.mock("../../../lib/api/sfApi"); const SF = require("../../../lib/api/sfApi"); const consola = require("consola"); const toolkit = require("../../../index"); describe("company-data-copier", () => { - let tempDir; - let originalCwd; - let originalExit; - const destinationFirmId = 13692; const sourceCompanyId = 1224550; const sourceLedgerIds = [33417839, 32116688]; beforeEach(async () => { jest.clearAllMocks(); - - tempDir = await fsPromises.mkdtemp(path.join(os.tmpdir(), "sf-cli-test-")); - - originalCwd = process.cwd(); - process.chdir(tempDir); - - originalExit = process.exit; - process.exit = jest.fn(); consola.success = jest.fn(); consola.error = jest.fn(); consola.info = jest.fn(); consola.log = jest.fn(); consola.warn = jest.fn(); consola.debug = jest.fn(); }); - - afterEach(async () => { - process.chdir(originalCwd); - process.exit = originalExit; - await fsPromises.rm(tempDir, { recursive: true, force: true }); - });🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@tests/bin/cli/company-data-copier.test.js` around lines 21 - 44, The test setup in company-data-copier.test.js includes unused e2e-style scaffolding that `copyCompanyData` never exercises. Remove the unnecessary `tempDir` creation, `process.chdir`/`originalCwd` handling, and `process.exit` mocking from `beforeEach`/`afterEach`, and keep only the mocks actually needed for the unit tests such as the `consola` methods and `jest.clearAllMocks()`.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Nitpick comments:
In `@tests/bin/cli/company-data-copier.test.js`:
- Around line 77-84: The current test for copyCompanyData only covers the
undefined response path, but not the case where SF.runCompanyDataCopier returns
an object without data. Update the company-data-copier test suite to add a
separate case that mock-resolves a truthy response with no data (for example via
the copyCompanyData flow) and asserts the same error/false behavior, so both
branches of the !response || !response.data guard are exercised.
- Around line 21-44: The test setup in company-data-copier.test.js includes
unused e2e-style scaffolding that `copyCompanyData` never exercises. Remove the
unnecessary `tempDir` creation, `process.chdir`/`originalCwd` handling, and
`process.exit` mocking from `beforeEach`/`afterEach`, and keep only the mocks
actually needed for the unit tests such as the `consola` methods and
`jest.clearAllMocks()`.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: c6576f3c-3c1f-4198-8542-fb631ebcb724
⛔ Files ignored due to path filters (1)
package-lock.jsonis excluded by!**/package-lock.json
📒 Files selected for processing (11)
CHANGELOG.mdREADME.mdbin/cli.jsindex.jslib/api/sfApi.jslib/utils/apiUtils.jspackage.jsontests/TESTS.mdtests/bin/cli.test.jstests/bin/cli/company-data-copier.test.jstests/lib/api/sfApi.test.js
Based on a review of the merged backend (GitLab MRs !26606 / !26472 and doc/bso_company_data_copier.md): - Route: POST the public, non-firm-scoped `/api/public/v3/company_data_copier/run` (destination firm derived from the OAuth token) instead of the firm-scoped `/api/v4/f/:id/...` route, which the doc says was renamed away. Uses an absolute URL to bypass the firm instance's baseURL while keeping the destination firm's token/auth/refresh. - Copied-company name: the backend names it `Company_<source_id>_<hex>` (with `_FAILED` appended on failure), not `SF_COPY_<id>_<timestamp>`. Updated the success message and README. - Honest fire-and-forget messaging: `202` means enqueued, not copied. There is no status endpoint; verify in the destination firm after a few minutes and treat a `_FAILED` company name as a failed (non-retried) copy. - README: documented backend prerequisites (demo-firm-only, source != dest, ledgers must belong to the source company, templates matched by handle / name_nl are silently skipped if absent, caller-rights) and the per-firm concurrency mutex (silent no-op while a copy is in flight). - Tests updated to assert the public v3 URL and the fire-and-forget messaging. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The command POSTed to /api/public/v3/company_data_copier/run, which
returns 404 on live. Probing production confirmed the copier is served
at the firm-scoped /api/v4/f/:id/company_data_copier/run route:
- POST /api/public/v3/company_data_copier/run -> 404 page not found
- POST /api/v4/f/:id/company_data_copier/run -> 401 invalid_token
(route exists, only auth missing; returns proper 422 once authed)
Switch runCompanyDataCopier to POST the relative "company_data_copier/run"
path so it inherits the firm axios instance's /api/v4/f/:id baseURL along
with its token, staging Basic-auth and refresh interceptor. Drop the now
unused firmCredentials import and update the sfApi test to assert the
firm-scoped route.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
What
Adds a new
silverfin company-data-copierCLI command that triggers the platform's Data Copier to copy a source company's data into a brand-new company in a destination (development) firm. This lets BSO developers reproduce a client's situation in a dev firm to debug templates against realistic data, without ever touching the production firm.-c, --source-company-id— company to copy data from (in the source/client firm)-l, --source-ledger-ids— one or more period ids, space-separated (variadic, matchingrun-test's--handle)-f, --firm— destination firm whose token authenticates the request and where the copy is created (defaults to the configured default firm)It copies data only (account values incl. adjustments, text properties, people/company drop, configuration) — not template code. The destination firm is taken from the URL/token; the source is identified purely by company + period ids in the body (no source-firm parameter). The platform enforces source-firm access rights on the authenticating user (see the access note below).
How it wires up
Follows the existing layering exactly:
lib/api/sfApi.js→runCompanyDataCopier, reusingAxiosFactory.createInstance(so staging Basic-auth,access_token, and token refresh all work unchanged).index.js→copyCompanyData(input validation + reporting).bin/cli.js→ thecompany-data-copiercommand.✅ Route verified on live — it is the firm-scoped v4 route
An earlier revision of this command POSTed to the public, non-firm-scoped
/api/public/v3/company_data_copier/run, on the basis thatdoc/bso_company_data_copier.mdsaid the firm-scoped route had been renamed away (commit24e2547bc9). Live probing shows that is not the case — the copier is served at the firm-scoped route:live.getsilverfin.comPOST /api/public/v3/company_data_copier/run404 page not found— route does not existPOST /api/v4/f/:id/company_data_copier/run401 invalid_token— route exists, only auth missing; returns a proper422business-validation once authenticatedSo this PR now POSTs the relative
company_data_copier/run, inheriting the firm axios instance's/api/v4/f/:idbaseURL (and its token, staging Basic-auth and refresh interceptor) — the "revert this one line" the previous note anticipated.Two other alignments with the merged backend (unchanged from the previous revision):
Company_<source_id>_<hex>(e.g.Company_42_a3f1b9d4), notSF_COPY_<id>_<timestamp>. Success message + README reflect this.202 {"status":"enqueued"}means enqueued, not copied — the copy runs in a Sidekiq job and there is no status endpoint. Messaging says the copy was requested, that it runs async, to verify in the destination firm after a few minutes, and that a company name ending in_FAILEDmeans the (non-retried) copy failed.The feature's purpose is reproducing client data in a dev firm, but doing so runs into an access boundary worth flagging explicitly:
The command authenticates as the destination-firm user, and the backend resolves the source company in that user's context. Reading a company that lives in a different (client) firm therefore requires the authenticating user to be either:
silverfin_support?), which can read any firm's companies, orA normal BSO-developer account cannot reach client firms — the OAuth authorize page only lists the internal dev/demo/testing firms the user belongs to — so attempting a client source yields
422 {"error":"source_company_id <id> does not exist"}(the backend reports cross-firm invisibility as "does not exist"). This is almost certainly intentional given the data-protection implications of copying production client data into a dev firm.Open question for the team: how are BSO developers meant to authenticate for real client-data copies? Options: a scoped support credential for this workflow, support-initiated copies, or a consent/approval flow. Note that copies within firms the developer already belongs to (e.g. functional-testing → demo) work today and are sufficient to validate the command itself end-to-end (
404 → 401 → 422 validation → 202 enqueued).Backend prerequisites / gotchas (now in the README)
These cause
422s or silent skips that would otherwise confuse users:422).422).source_ledger_idsmust belong tosource_company_id(422).handle(reconciliations) / Dutch namename_nl(account detail templates). If the template doesn't already exist in the destination firm, its data is silently skipped — so the template you're debugging must already exist there.Also included
apiUtils.js) — a 422 isn't specifically a permissions issue, so"You don't have the rights to update the previous parameters"was confusing (it showed on a Data Copier "source_company_id does not exist" error). Now only the real API error is surfaced. Applies to all commands.1.57.0.Testing
axios-mock-adapter) asserts the request hits the firm-scopedcompany_data_copier/runroute (relative to the firm/api/v4/f/:idbaseURL).tests/bin/cli/) asserts the success message reports requested/enqueued and the follow-up guidance flags the async nature +_FAILEDsignal.tests/bin/cli.test.js, plus TESTS.md catalogue entries.🤖 Generated with Claude Code