feat(plugins): remember row import column mappings per table and add Match by Name and Match by Position

[datlechin]

Summary

Row imports (CSV, JSON, Excel) now remember the field-to-column choices made for each table. The mapping sheet also gets a Match Columns pull-down with Match by Name, Match by Position and Use Saved Mapping. When saved choices are applied, the sheet says so: "Restored the mapping saved for table."

Fixes #3172

Root cause

RowImportSheet kept the mapping in view @State only. loadExistingContext rebuilt it from a case-insensitive name match every time a table was picked or the file was read again. So a header like E-mail for an email column had to be mapped by hand on every import, and there was no way to map by position.

Design

• ImportColumnMatcher is a pure type that resolves the mapping in tiers:

  • choices made in this sheet, then choices saved for the table, then exact name matches, then case-insensitive ones;
  • each tier is settled for every field before the next, so a column goes to one field at most;
  • a saved column the table no longer has falls back to the name match.

  The name rule deliberately differs from TableColumnMatcher (Table Transfer), which is left as it is.

• ImportColumnMappingStore is a TableScopedSettingsStore modelled on ForeignKeyLabelColumnStore: UserDefaults, stored only on this Mac, no CloudKit.

  • It stores only the choices that differ from Match by Name, as column or skip. An automatic "no match" is never frozen, so a column added to the table later is still found by name.
  • Choices for fields the current file doesn't have are kept, so two file layouts into one table both work.
  • It is registered in TableScopedSettingsRegistry, so a sidebar rename or drop, or deleting the connection, carries the mapping along or forgets it.

• Saved when Import starts, whether the import then succeeds or fails. It is saved from the same plan the import runs, not from what the rows show. A new table saves only the columns that were renamed or left out.

• RowImportMapping: the sheet's mapping state moved into a @MainActor ObservableObject (the CreateTableDraft shape). That makes the rules unit-testable and keeps RowImportSheet under the type-body limit.

• Reads of the file now go through one .task(id:) keyed on destination, database scope, table, detection options and a per-destination retry count.

  • A stale read is cancelled and dropped.
  • Import is enabled only for a finished, error-free read that matches the current options, table and scope.
  • A generic observer watches the plugin itself, so an option change re-reads the file straight away. A re-read keeps the choices already made in the sheet.

• Match by Position is offered only when the plugin says its fields follow the file's order.

  • JSON lists fields alphabetically, because JSONSerialization drops key order. That includes the .jsonl snapshot the Data Files window hands over.
  • This needed a new PluginKit static, ImportFormatPlugin.sourceFieldsFollowFileOrder. It defaults to false; CSV and XLSX return true.
  • ImportFieldNaming.uniqueNames is also new in PluginKit. CSV and XLSX now name their header fields with it.
  • Both are additive. scripts/check-pluginkit-abi.sh shows only additions. currentPluginKitVersion goes from 33 to 34 (the first bump since v0.76.1), along with every plugin Info.plist; minimumCompatiblePluginKitVersion is unchanged.

Defects fixed on the way

The new commands and the one-field-per-column rule hit each of these directly:

• An option change went unseen until the next unrelated redraw. The sheet observed only PluginManager, so the re-read ran late, usually right after the user's next manual pick, and then threw that pick away.
• A slow load for one table could land over the table picked after it.
• Two headers differing only by case, like Email and email. The sink folded field names, so the field the user skipped was imported onto its twin's column and every INSERT named that column twice. The sink now matches the exact name first. It folds case only for a field the sheet never listed, and only when that spelling points at a single listed field, a single mapping key and a single field of the row.
• Header naming in CSV and Excel. Excel never de-duplicated headers, so the second Notes column overwrote the first. In both formats, a blank header's placeholder could take a name a later header spelled out, so the import read the wrong column under that name.
• Try Again in one destination re-read the other, discarding the new-table column edits.

Reviews

• Codex review, adversarial-review and a second review. The second one found one more case-folding path, now fixed.
• A four-lens review workflow, with each finding checked by a separate agent trying to refute it.
• Every confirmed finding is fixed here. The rest are listed below as follow-ups.

Before / After

Not captured yet. The change adds a row above the mapping list: the "Restored the mapping saved for …" caption on the left and a Match Columns pull-down on the right. Screenshots need a driven Debug build, and during this session the Mac was first locked and then in use. A SwiftUI Menu also takes no synthetic click, so the open pull-down needs XCUITest or a hand capture. To do before merge.

Tests

• ImportColumnMatcherTests (23), ImportColumnMappingStoreTests (8), RowImportMappingTests (14), ImportDataSinkAdapterMappingTests (13), XLSXSheetParserTests (12), CSVImportPluginTests (24)
• Neighbours: SQLServerImportBatchTests, TableScopedSettingsRegistryTests, PreferenceKeysGuardTests, PluginKitABIResilienceTests, PluginSignatureSweepTests, TransferAlertWindowOwnershipTests, ValueDisplayFormatStorageTests, AppSettingsStorageResetTests
• App build, AllPlugins (all 41) and the ABI check all pass locally.
• UI: RowImportMappingMemoryUITests imports a CSV with GenreId,Title into the sample database's Genre table and maps Title to Name by hand. It then opens the sheet again and checks that Title comes back on Name with the restored caption. It has not run locally: the first attempt timed out enabling UI automation while the Mac was locked, and afterwards the machine was in use. CI's UI job is its first run.

Not in this PR (found while investigating)

• The new-table retry path keys createdTables by bare table name. If the browse database changes while the sheet is open, a retry can clear rows from a same-named table in the other database. Pre-existing.
• JSON field detection samples only the first 200 rows, so a key that first appears later is dropped without a report. Pre-existing.
• NDJSON detection decodes a 256 KB prefix as one UTF-8 string. A cut mid-character leaves the sheet with no fields. Pre-existing.
• Table Transfer's mapping editor lets two source columns target one destination column. Pre-existing.
• New-table column edits are lost when a parsing option changes. Pre-existing.
• A DROP TABLE run in the SQL editor does not clear any table-scoped settings (filters, value formats, FK labels, and now import mappings). Only a sidebar drop does. Pre-existing; the docs say so.

CI fixes

The failed package test was SyncRecordMapperTests.unknownWireValueFailsClosed. The mapper's duplicate Safe Mode decoder turned unknown values into Off. It now uses the model's SafeModeLevel(wireValue:isReadOnly:) policy. Added regressions for unknown values on read-only connections and renames that preserve unknown wire values.

SurrealDB's JSON helper had become a property while its callers still supplied the requested text length. Restored the length-taking helper, keeping valid display truncation and complete export text.

Integrated main through 3cf419ce1, keeping both sets of changelog entries. The storage conflicts retain both preference key families and register both ImportColumnMappingStore and TableFolderStorage, so rename, drop and connection deletion reach both.

Verification for these fixes:

• Full TableProCore package tests after the final integration: passed, including 1,239 Swift Testing tests plus XCTest.
• Isolated SwiftPM build of every SurrealDB plugin source with the actual driver test file: 50 tests in 8 suites passed.
• AllPlugins Xcode build with PluginKit 34: passed. Plugin sources and plugin build configuration remain identical after the final integration.
• Project regeneration: passed after each integration.
• SwiftLint on the shared fixes and both storage conflict resolutions: zero violations. The helper still flags four existing stale documentation paths (TableProApp/rust-dameng, Contents/MacOS, Contents/Helpers, /Applications/Xcode-beta.app).
• The full unit-test target compiled on feat(plugins): remember row import column mappings per table and add Match by Name and Match by Position #3183. All 87 selected tests passed: PreferenceKeysGuardTests, TableScopedSettingsRegistryTests, ImportColumnMappingStoreTests, TableFolderStorageTests, AppSettingsStorageResetTests, TypesenseStatementGeneratorTests, and ElasticsearchStatementGeneratorTests.

Pushing these commits triggers fresh GitHub Actions runs.

Unit-test compile correction

The unit-test build exposed three fixtures still calling the old one-argument cell API: two in Typesense and one in Elasticsearch. They now explicitly pass length: .display, retaining their expected display truncation and write-refusal behavior.

Both fixture files are identical on all four updated branches. The complete test target built on #3183, with all 87 selected storage and write-refusal tests passing. Separately, an isolated SwiftPM build of all Elasticsearch and Typesense driver sources and the actual driver test files passed 175 tests in 14 suites on this branch. SwiftLint found no violations in the two fixture files.

[mintlify]

Preview deployment for your docs. Learn more about Mintlify Previews.

Project	Status	Preview	Updated
TablePro	🟢 Ready	View Preview	Sep 30, 2026, 12:50 PM

💡 Tip: Enable Automations to automatically generate PRs for you.