fix: stop Enum columns raising TypeError on _enums - #76
Open
TangoEnSkai wants to merge 1 commit into
Open
Conversation
Any operation touching an Enum column failed with
TypeError: String.__init__() got an unexpected keyword argument '_enums'
Enum subclasses String, so with no colspecs entry of its own it resolved
through the String entry and SQLAlchemy adapted it to
DatabricksStringType. That class is a TypeDecorator whose __init__ hands
its arguments straight to its impl (String), while Enum.adapt() forwards
Enum's internal keyword arguments — so String received _enums and
rejected it. metadata.create_all() on a model with an Enum column could
not run.
Two approaches were tried and rejected before this one:
Filtering the offending keyword out is whack-a-mole. Dropping _enums
just moves the failure to _disable_warnings, and the set of forwarded
keywords is SQLAlchemy's to change.
Letting Enum fall back to SQLAlchemy's own implementation (mapping it to
sqlalchemy.types.Enum) fixes the crash but renders literals with doubled
single-quotes — 'O''Brien' rather than 'O\'Brien' — which is exactly the
breakage DatabricksStringType was written to prevent.
Subclassing Enum keeps every behaviour of the generic type (value
validation under validate_strings, length inference from the longest
value, native Python enum.Enum support) and overrides only
literal_processor, so Enum literals are escaped exactly as plain strings
are. Verified against the SQLite dialect as a reference: validation and
default behaviour now match it.
The existing camel_case_type_map covers Enum but did not catch this,
because it asserts on Enum(...).compile(dialect), which renders the type
name (STRING) without going through colspecs adaptation. Only
dialect_impl() — the path every real query and DDL run takes — hit the
crash. The new tests exercise that path.
Resolves databricks#61
Signed-off-by: TangoEnSkai <21152231+TangoEnSkai@users.noreply.github.com>
TangoEnSkai
requested review from
deeksha-db,
gopalldb,
jackyhu-db,
jayantsing-db,
jprakash-db,
madhav-db,
samikshya-db,
shivam2680 and
vikrantpuppala
as code owners
August 23, 2026 05:33
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Context
Any operation touching an
Enumcolumn fails:EnumsubclassesString, so with no colspecs entry of its own it resolvesthrough the
Stringentry and SQLAlchemy adapts it toDatabricksStringType.That class is a
TypeDecorator, andTypeDecorator.__init__passes itsarguments straight to its
impl(String) — whileEnum.adapt()forwardsEnum's internal keyword arguments.
Stringrejects them.metadata.create_all()on a model with an Enum column cannot run, as reported.Worth noting where the crash actually lives:
Enum("A", "B").compile(dialect)succeeds and renders
STRING. Onlydialect_impl()— the path every realquery and DDL run takes — raises. That is why the existing
camel_case_type_mapentry for
Enumpasses today without catching this.What
DatabricksEnumType, a subclass ofsqlalchemy.types.Enumthat overridesonly
literal_processor()to escape literals the wayDatabricksStringTypedoes.
colspecssoEnumstops resolving throughString.TestDatabricksEnumcovering the adaptation that used to crash, DDLcompilation, native Python
enum.Enumcolumns, literal escaping parity withString, length inference,validate_stringsenforcement, and that ordinaryStringcolumns are unaffected.Why
Two other approaches were tried first and rejected on evidence:
Filtering the offending keyword out is whack-a-mole. Dropping
_enumsjustmoves the failure to
_disable_warnings, and the set of forwarded keywordsbelongs to SQLAlchemy, not to us.
Letting Enum fall back to SQLAlchemy's own implementation (mapping
Enum→sqlalchemy.types.Enum) fixes the crash but changes literal rendering:O'Bri\enrenders asStringtoday (the target behaviour)'O\'Bri\\en'Enum'O''Bri\en'❌'O\'Bri\\en'✅The middle row is precisely the single-quote doubling that
DatabricksStringTypeexists to prevent — fixing one bug by reintroducing another.
Subclassing
Enumkeeps every behaviour the generic type provides and changesonly the literal rendering. Checked against the SQLite dialect as a reference:
validate_strings=True, bad valueLookupErrorLookupErrorA
TypeDecorator-based variant was also prototyped and discarded: it silentlylost
validate_stringsenforcement, letting"ZZZ"through where SQLite raises.Completion Criteria
Enumcolumns adapt withoutTypeError, including the reportedcreate_all()pathStringliteralsenum.Enumsupport preserved,matching the SQLite dialect
Stringcolumns unaffecteddepend on it, and nothing else
pytest tests/test_local(offline modules) — 302 passedblack --checkclean on all three changed files# UnreleasedsectionNote
This touches
base.pyandCHANGELOG.md, which #74 also touches. Thebase.pyedits are far apart and merge cleanly; the CHANGELOG entries both open an
# Unreleasedsection, so whichever lands second needs a one-line rebase. Happyto do that whenever you'd like.
close #61