fix(schema_loader): quote case-sensitive column names in external models - #6059
devtechedge wants to merge 1 commit into
Conversation
create_external_models wrote column names from the adapter verbatim, so on case-folding dialects like postgres a quoted column such as "ID" was serialized as the unquoted key ID. On load, normalize_identifiers folded it to id and the generated external_models.yaml no longer described the table, breaking plan and lint with ambiguousorinvalidcolumn errors. Only quote identifiers whose unquoted form would be rewritten by the dialect's normalization; already-quoted names and identifiers that normalize to themselves (including pseudo columns like _sync_row_hash) are written unchanged. Signed-off-by: devtechedge <devtechedge@gmail.com>
|
Hi - polite check-in on the external-models quoting fix whenever you have bandwidth.
This adds Quoting on the write side stays lossless because
Happy to tweak the helper, adjust which dialects quote, or add another fixture if that would help review. No rush at all - still ready from my end whenever it fits. |
sqlmesh create_external_modelsserializes column names frominformation_schemaverbatim, while the tablenameis already dialect-serialized. On case-folding dialects like postgres, a column created as"ID"was written as the unquoted keyID. On load,normalize_identifiersfolds that key toid, so the generatedexternal_models.yamlno longer describes the table andplan/lintfail withambiguousorinvalidcolumn.This adds a
_serialize_column_namehelper insqlmesh/core/schema_loader.pythat quotes a column name only when the dialect's identifier normalization would rewrite its unquoted form. Identifiers that normalize to themselves (for examplebillnoor the_sync_row_hashpseudo column) and names the adapter already returned quoted are written unchanged, so existing projects' generated files are not churned.Quoting on the write side is lossless:
_columns_validatorinsqlmesh/core/model/meta.pypasses the quoted key throughnormalize_identifiers, which preserves the quoted form, so the round trip restores the original case.Tests:
test_create_external_models_quotes_case_sensitive_columnscovers uppercase columns (ID,ORGANID,CDATE) emitted quoted, a lowercase column and the_sync_row_hashpseudo column left unquoted, and asserts the quoted keys round-trip throughcreate_external_modelwith their original case.tests/core/test_schema_loader.pysuite passes (9 tests).Fixes #6058