feat(attachments): add "match by mapping file" support

[grantfitzsimmons]

Fixes #8302

This PR adds a "Match by mapping file" mode to the Bulk Attachment Uploader. Users can upload a CSV mapping file that explicitly pairs filenames with record identifiers (catalog numbers, etc.), enabling multi-attachment-per-record uploads without requiring filename structure conventions.

filename	catalogNumber
specimen1.jpg	12345
specimen1_2.jpg	12345
specimen2.jpg	67890

This lets you attach multiple files to the same record, use files with names that don't follow a pattern, and see exactly what's expected before you start uploading.

When you first open the uploader, a dialog asks which matching method you want. If you choose mapping file, you upload your CSV and pick which columns contain the filename and the record identifier. The table then shows all your expected rows — rows waiting for a file show "Awaiting file", and any files you add are paired with the correct row automatically.

Warning

This PR affects database migrations. See migration testing instructions.

Migration changes

• Added matchingmode field to Spattachmentdataset model with choices [('mappingFile', 'Mapping File')] (null = filename mode). New migration 0002_add_matchingmode.py in attachment_gw Django app.

Checklist

• Self-review the PR after opening it to make sure the changes look good and
  self-explanatory (or properly documented)
• Add relevant issue to release milestone
• Add pr to documentation list
• Add automated tests
• Add a reverse migration if a migration is present in the PR
• Add migration function to

  specify7/specifyweb/specify/management/commands/run_key_migration_functions.py

  Line 50 in ea04665

  def fix_schema_config(stdout: WriteToStdOut | None = None):

Testing instructions

Test sets

You can use this set of test-files.zip, tested with the KU Entomology database. Please also test a variety of other cases!

1. catalog-number/: Collection Object → Catalog Number

Test both the standard mapping and multi-mapping.

Steps:

1. Create new dataset → Match by mapping file
2. Upload catalog-number/mapping.csv
3. Verify columns auto-detect: fileName + catalogNumber
4. Click Proceed → 6 rows show "Awaiting file"
5. Drop all 6 .jpg files → status should clear
6. Validate → Upload → Rollback
7. Repeat with mapping-multi.csv

2. taxon/: Taxon → Full Name

Steps:

1. Create new dataset → Match by mapping file
   (must have a Taxon table with matching records in your collection)
2. Upload taxon/mapping.csv
3. Verify columns auto-detect: File Name + Full Name (based on schema captions)
4. Click Proceed → 6 rows
5. Drop all 6 .jpg files
6. Validate, Upload, Rollback

3. reference-work/: Reference Work → Title

Steps:

1. Create new dataset → Match by mapping file
   (must have a Reference Work table with matching records)
2. Upload reference-work/mapping.csv
3. Verify columns auto-detect: fileName + Title
4. Click Proceed
5. Drop all 5 images
6. Validate, Upload, Rollback

Special Cases

• CSV row without a matching file — shows "Awaiting file" forever; Validate warns
  

• Make sure that files not in the CSV are handled (they will not be matched or uploaded)
  

• Make sure that duplicate filenames in CSV are handled, showing only one row after import

• Verify that when re-opening a saved dataset the mapping CSV data and column selections are preserved

• Test rollback and random interruptions (cancel during actions), make sure behavior is consistent with our current bulk attachment uploader

• Make sure the matching mode dialog shows on first visit (and subsequent visits if the data set is empty), but stays hidden after choosing (even on refresh/re-open)

Cases that are not handled

• Currently, you cannot upload a single attachment to multiple objects. This means you can't say 8.jpg and list it multiple times in the mapping for different linked objects.

---

Summary by CodeRabbit

Summary by CodeRabbit

• New Features
  • Added bulk attachment “match by mapping file” mode with CSV/TSV input.
  • Introduced a matching-mode dialog and mapping-file setup to select the match-value and filename columns.
• Bug Fixes
  • Added server-side validation for matchingmode and preserved the existing value when omitted.
• UI Improvements
  • Enhanced dataset row notes and mapping-mode specific warnings for missing/unmatched/duplicate placeholders.
• Tests
  • Expanded Jest coverage for mapping-file selection, deduplication, and placeholder handling.
• Style/Documentation
  • Minor formatting updates to link and documentation URL strings.

[github-actions]

Warning

One or more dependencies are approaching or past End-of-Life.
Please plan upgrades accordingly.

STATUS=WARNING
NODE_VERSION=20
NODE_CYCLE=20
EOL_DATE=2026-04-30
DAYS_REMAINING=-85

--- Node.js ---
Version: 20
EOL: 2026-04-30
Status: WARNING

STATUS=OK
PYTHON_VERSION=3.12
PYTHON_CYCLE=3.12
EOL_DATE=2028-10-31
DAYS_REMAINING=830

--- Python ---
Version: 3.12
EOL: 2028-10-31
Status: OK

STATUS=WARNING
DJANGO_VERSION=4.2
DJANGO_CYCLE=4.2
EOL_DATE=2026-04-07
DAYS_REMAINING=-108

--- Django ---
Version: 4.2
EOL: 2026-04-07
Status: WARNING

[grantfitzsimmons]

Waiting a bit on the way we finalize creating schema captions for fields. I imagine we may want to have a schema config entry for this field? No real substantive benefit for it, but the sync fields command would create one, so maybe better to add here.

[coderabbitai]

🧹 Nitpick comments (1)

specifyweb/frontend/js_src/lib/components/AttachmentsBulkImport/Import.tsx (1)

293-346: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Eliminate duplicate logic and fix missing dependencies.

The record column definition is identical in both branches of isMapping. We can extract it to make the logic DRY. Additionally, isMappingMode and currentBaseTable should be included in the useMemo dependency array to ensure the headers update reliably if those values change.

Here is a cleaner approach that condenses the conditionally included matchValue column and resolves the dependencies:

♻️ Proposed refactor

-  const mainHeaders = React.useMemo(() => {
-    const isMapping = isMappingMode;
-    const baseHeaders: IR<JSX.Element | LocalizedString> = isMapping
-      ? {
-          selectedFileName: commonText.selectedFileName(),
-          matchValue: attachmentsText.matchValue(),
-          fileSize: attachmentsText.fileSize(),
-          record: (
-            <div className="flex min-w-fit items-center gap-2">
-              {currentBaseTable === undefined ? (
-                userText.resource()
-              ) : (
-                <>
-                  <TableIcon label name={currentBaseTable} />
-                  {eagerDataSet.uploadplan.staticPathKey === undefined
-                    ? ''
-                    : strictGetTable(currentBaseTable).strictGetField(
-                        staticAttachmentImportPaths[
-                          eagerDataSet.uploadplan.staticPathKey
-                        ].path
-                      ).label}
-                </>
-              )}
-            </div>
-          ),
-          progress: attachmentsText.progress(),
-        }
-      : {
-          selectedFileName: commonText.selectedFileName(),
-          fileSize: attachmentsText.fileSize(),
-          record: (
-            <div className="flex min-w-fit items-center gap-2">
-              {currentBaseTable === undefined ? (
-                userText.resource()
-              ) : (
-                <>
-                  <TableIcon label name={currentBaseTable} />
-                  {eagerDataSet.uploadplan.staticPathKey === undefined
-                    ? ''
-                    : strictGetTable(currentBaseTable).strictGetField(
-                        staticAttachmentImportPaths[
-                          eagerDataSet.uploadplan.staticPathKey
-                        ].path
-                      ).label}
-                </>
-              )}
-            </div>
-          ),
-          progress: attachmentsText.progress(),
-        };
-    let headers = baseHeaders;
-    if (process.env.NODE_ENV === 'development')
-      headers = { ...headers, attachmentId: attachmentsText.attachmentId() };
-    return headers;
-  }, [eagerDataSet.uploadplan.staticPathKey, eagerDataSet.uploadplan.matchingMode]);
+  const mainHeaders = React.useMemo(() => {
+    const recordHeader = (
+      <div className="flex min-w-fit items-center gap-2">
+        {currentBaseTable === undefined ? (
+          userText.resource()
+        ) : (
+          <>
+            <TableIcon label name={currentBaseTable} />
+            {eagerDataSet.uploadplan.staticPathKey === undefined
+              ? ''
+              : strictGetTable(currentBaseTable).strictGetField(
+                  staticAttachmentImportPaths[
+                    eagerDataSet.uploadplan.staticPathKey
+                  ].path
+                ).label}
+          </>
+        )}
+      </div>
+    );
+
+    let headers: IR<JSX.Element | LocalizedString> = {
+      selectedFileName: commonText.selectedFileName(),
+      ...(isMappingMode ? { matchValue: attachmentsText.matchValue() } : {}),
+      fileSize: attachmentsText.fileSize(),
+      record: recordHeader,
+      progress: attachmentsText.progress(),
+    };
+
+    if (process.env.NODE_ENV === 'development')
+      headers = { ...headers, attachmentId: attachmentsText.attachmentId() };
+    return headers;
+  }, [eagerDataSet.uploadplan.staticPathKey, currentBaseTable, isMappingMode]);

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@specifyweb/frontend/js_src/lib/components/AttachmentsBulkImport/Import.tsx`
around lines 293 - 346, Refactor the headers construction around baseHeaders so
the shared record column is defined once and reused in both mapping and
non-mapping modes, while conditionally adding only matchValue for mapping mode.
Update the useMemo dependency array to include isMappingMode and
currentBaseTable alongside the existing dependencies so headers recompute when
any referenced value changes.

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@specifyweb/frontend/js_src/lib/components/AttachmentsBulkImport/Import.tsx`:
- Around line 293-346: Refactor the headers construction around baseHeaders so
the shared record column is defined once and reused in both mapping and
non-mapping modes, while conditionally adding only matchValue for mapping mode.
Update the useMemo dependency array to include isMappingMode and
currentBaseTable alongside the existing dependencies so headers recompute when
any referenced value changes.

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: e692a47f-54e5-47d4-9f13-f6871f3782eb

📥 Commits

Reviewing files that changed from the base of the PR and between dc2a98b and dcd25e6.

📒 Files selected for processing (5)

• specifyweb/backend/attachment_gw/models.py
• specifyweb/frontend/js_src/lib/components/AttachmentsBulkImport/Import.tsx
• specifyweb/frontend/js_src/lib/components/AttachmentsBulkImport/MappingFileSetup.tsx
• specifyweb/frontend/js_src/lib/components/AttachmentsBulkImport/ViewAttachmentFiles.tsx
• specifyweb/frontend/js_src/lib/localization/attachments.ts

🚧 Files skipped from review as they are similar to previous changes (3)

• specifyweb/frontend/js_src/lib/localization/attachments.ts
• specifyweb/frontend/js_src/lib/components/AttachmentsBulkImport/ViewAttachmentFiles.tsx
• specifyweb/frontend/js_src/lib/components/AttachmentsBulkImport/MappingFileSetup.tsx

[emenslin]

1. catalog-number/: Collection Object → Catalog Number

Steps:

• Validate → Upload → Rollback
• Repeat with mapping-multi.csv

2. taxon/: Taxon → Full Name

Steps:

• Validate, Upload, Rollback

3. reference-work/: Reference Work → Title

Steps:

• Validate, Upload, Rollback

Special Cases

• CSV row without a matching file — shows "Awaiting file" forever; Validate warns

• Make sure that files not in the CSV are handled (they will not be matched or uploaded)

• Make sure that duplicate filenames in CSV are handled, showing only one row after import

• Verify that when re-opening a saved dataset the mapping CSV data and column selections are preserved

• Test rollback and random interruptions (cancel during actions), make sure behavior is consistent with our current bulk attachment uploader

• Make sure the matching mode dialog hows on first visit (and subsequent visits if the data set is empty), but stays hidden after choosing (even on refresh/re-open)

---

Overall it looks good! I did notice that it always auto selects match to filename even if you last uploaded a mapping file which doesn't match up with the last point on the checklist.

07-21_08.15.mp4

Additionally if you have a row without a matching file, upload is disabled. I don't think this is necessarily a problem since it works similarly in bulk attachment upload, but I wanted to make note of it because I didn't realize I missed a file and was briefly confused why I couldn't upload.

Lastly, another one that's not necessarily an issue but when you upload a mapping file, it'll correctly match the file name and value columns in the dialog but then when you go to the data set itself you have to reselect the path. I imagine this is the intended behavior, but we'll want to make sure to document it since I could see it being assumed that the path would automatically be selected.

[grantfitzsimmons]

@emenslin I think you can approve this PR if I understand you correctly! Thanks for taking the time to review. 😁

Overall it looks good! I did notice that it always auto selects match to filename even if you last uploaded a mapping file which doesn't match up with the last point on the checklist.

That bullet point is meant to communicate that the "Matching Mode" dialog should appear when you create a new attachment data set in only two cases:

• The first time the attachment data set is created
• If an existing attachment data set is opened, but no rows were added (e.g., no attachments or mapping files were selected)

It is not intended to remember the last choice you made, as the choice is unique for each attachment data set. It defaults to matching by filename, as that is the standard workflow at the moment and requires no additional input from the user.

Additionally if you have a row without a matching file, upload is disabled. I don't think this is necessarily a problem since it works similarly in bulk attachment upload, but I wanted to make note of it because I didn't realize I missed a file and was briefly confused why I couldn't upload.

This is meant to replicate the current handling, as it prevents unexpected behavior, but I might adjust this to ignore rows that aren't part of the mapping instead.

Lastly, another one that's not necessarily an issue but when you upload a mapping file, it'll correctly match the file name and value columns in the dialog but then when you go to the data set itself you have to reselect the path. I imagine this is the intended behavior, but we'll want to make sure to document it since I could see it being assumed that the path would automatically be selected.

This is the intended behavior, but I think there are two things happening here:

1. The user needs to map their spreadsheet columns to the "Match Value Column" and the "File Name Column":
   

2. The user then needs to choose which field the "Match Value Column" is matching to.
   

I think it would be more intuitive from a UX perspective, perhaps in this PR or in the future, to have the path choice during the "matching mode" dialog. In a sense, you are not currently reselecting anything, but you are having to set things at two different points.

[emenslin]

That bullet point is meant to communicate that the "Matching Mode" dialog should appear when you create a new attachment data set in only two cases:

• The first time the attachment data set is created
• If an existing attachment data set is opened, but no rows were added (e.g., no attachments or mapping files were selected)

It is not intended to remember the last choice you made, as the choice is unique for each attachment data set. It defaults to matching by filename, as that is the standard workflow at the moment and requires no additional input from the user.

This makes a lot more sense, in this case it works as expected!

This is meant to replicate the current handling, as it prevents unexpected behavior, but I might adjust this to ignore rows that aren't part of the mapping instead.

Yeah, I could see it going either way. I think this behavior works as long as we document it, but I think either a dialog or something at the top of the screen (e.g. the warning when you have to reselect files) that informs the user why they can't upload since if you have a larger data set the only way to find out would be scrolling through and seeing that a file was missed. Either way I don't think this is necessarily an issue for this PR

I think it would be more intuitive from a UX perspective, perhaps in this PR or in the future, to have the path choice during the "matching mode" dialog. In a sense, you are not currently reselecting anything, but you are having to set things at two different points.

I agree, I think having the path choice on the matching mode dialog would be much more intuitive. I think since it automatically detects the identifier in the matching mode dialog a user might think that they wouldn't have to select the path on the next screen and then might be confused when they can't upload. I don't think it is necessarily a problem, just not the most intuitive and should be documented.

Overall, I don't think any of my issues need to be fixed in this PR as everything I tested worked correctly, just things we should either fix in the future or make sure is explained in documentation!

[bronwyncombs]

• 1. catalog-number/: Collection Object → Catalog Number

• 2. taxon/: Taxon → Full Name

• 3. reference-work/: Reference Work → Title

Special Cases

• CSV row without a matching file — shows "Awaiting file" forever; Validate warns
  

Note: Upload is disabled if/when a row has Awaiting File status, even if other rows are otherwise ready for uploading

• Make sure that files not in the CSV are handled (they will not be matched or uploaded)

Note: Upload is enabled for other rows even if/when a row has Not In Mapping File status

• Make sure that duplicate filenames in CSV are handled, showing only one row after import
• Verify that when re-opening a saved dataset the mapping CSV data and column selections are preserved
• Test rollback and random interruptions (cancel during actions), make sure behavior is consistent with our current bulk attachment uploader
• Make sure the matching mode dialog shows on first visit (and subsequent visits if the data set is empty), but stays hidden after choosing (even on refresh/re-open)