Release: 1.2.0b3 pre release | Release Date: October 13, 2017

SQLAlchemy 1.2 Documentation

1.2 Changelog

1.2.0b4

no release date

1.2.0b3

Released: October 13, 2017

orm

  • [orm] [bug] Fixed bug where correlated select used against single-table inheritance entity would fail to render correctly in the outer query, due to adjustment for single inheritance discriminator criteria inappropriately re-applying the criteria to the outer query.

    This change is also backported to: 1.1.15

    References: #4103

  • [orm] [bug] Fixed bug where ORM relationship would warn against conflicting sync targets (e.g. two relationships would both write to the same column) for sibling classes in an inheritance hierarchy, where the two relationships would never actually conflict during writes.

    This change is also backported to: 1.1.15

    References: #4078

  • [orm] [bug] Fixed bug in Session.merge() following along similar lines as that of #4030, where an internal check for a target object in the identity map could lead to an error if it were to be garbage collected immediately before the merge routine actually retrieves the object.

    This change is also backported to: 1.1.14

    References: #4069

  • [orm] [bug] Fixed bug where an undefer_group() option would not be recognized if it extended from a relationship that was loading using joined eager loading. Additionally, as the bug led to excess work being performed, Python function call counts are also improved by 20% within the initial calculation of result set columns, complementing the joined eager load improvements of #3915.

    This change is also backported to: 1.1.14

    References: #4048

  • [orm] [bug] Fixed bug in Session.merge() where objects in a collection that had the primary key attribute set to None for a key that is typically autoincrementing would be considered to be a database-persisted key for part of the internal deduplication process, causing only one object to actually be inserted in the database.

    This change is also backported to: 1.1.14

    References: #4056

  • [orm] [bug] An InvalidRequestError is raised when a synonym() is used against an attribute that is not against a MapperProperty, such as an association proxy. Previously, a recursion overflow would occur trying to locate non-existent attributes.

    This change is also backported to: 1.1.14

    References: #4067

  • [orm] [bug] Fixed regression introduced in 1.2.0b1 due to #3934 where the Session would fail to “deactivate” the transaction, if a rollback failed (the target issue is when MySQL loses track of a SAVEPOINT). This would cause a subsequent call to Session.rollback() to raise an error a second time, rather than completing and bringing the Session back to ACTIVE.

    References: #4050

  • [orm] [bug] Fixed issue where the make_transient_to_detached() function would expire all attributes on the target object, including “deferred” attributes, which has the effect of the attribute being undeferred for the next refesh, causing an unexpected load of the attribute.

    References: #4084

  • [orm] [bug] Fixed bug in “selectin” polymorphic loading, loads subclasses using separate IN queries which prevented “selectin” and “inline” settings in a multi-level class hierarchy from interacting together as expected. A new example is added to the documentation.

    References: #4026

  • [orm] [bug] Removed the warnings that are emitted when the LRU caches employed by the mapper as well as loader srtategies reach their threshold; the purpose of this warning was at first a guard against excess cache keys being generated but became basically a check on the “creating many engines” antipattern. While this is still an antipattern, the presense of test suites which both create an engine per test as well as raise on all warnings will be an inconvenience; it should not be critical that such test suites change their architecture just for this warning (though engine-per-test suite is always better).

    References: #4071

  • [orm] [bug] Fixed regression where the use of a undefer_group() option in conjunction with a lazy loaded relationship option would cause an attribute error, due to a bug in the SQL cache key generation added in 1.2 as part of #3954.

    References: #4049

  • [orm] [bug] Modified the change made to the ORM update/delete evaluator in #3366 such that if an unmapped column expression is present in the update or delete, if the evaluator can match its name to the mapped columns of the target class, a warning is emitted, rather than raising UnevaluatableError. This is essentially the pre-1.2 behavior, and is to allow migration for applications that are currently relying upon this pattern. However, if the given attribute name cannot be matched to the columns of the mapper, the UnevaluatableError is still raised, which is what was fixed in #3366.

    References: #4073

orm declarative

  • [bug] [orm] [declarative] A warning is emitted if a subclass attempts to override an attribute that was declared on a superclass using @declared_attr.cascading that the overridden attribute will be ignored. This use case cannot be fully supported down to further subclasses without more complex development efforts, so for consistency the “cascading” is honored all the way down regardless of overriding attributes.

    References: #4091

  • [bug] [orm] [declarative] A warning is emitted if the @declared_attr.cascading attribute is used with a special declarative name such as __tablename__, as this has no effect.

    References: #4092

engine

  • [engine] [feature] Added __next__() and next() methods to ResultProxy, so that the next() builtin function works on the object directly. ResultProxy has long had an __iter__() method which already allows it to respond to the iter() builtin. The implementation for __iter__() is unchanged, as performance testing has indicated that iteration using a __next__() method with StopIteration is about 20% slower in both Python 2.7 and 3.6.

    References: #4077

  • [engine] [bug] Made some adjustments to Pool and Connection such that recovery logic is not run underneath exception catches for pool.Empty, AttributeError, since when the recovery operation itself fails, Python 3 creates a misleading stack trace referring to the Empty / AttributeError as the cause, when in fact these exception catches are part of control flow.

    References: #4028

sql

  • [sql] [bug] Fixed bug where the recently added ColumnOperators.any_() and ColumnOperators.all_() methods didn’t work when called as methods, as opposed to using the standalone functions any_() and all_(). Also added documentation examples for these relatively unintuitive SQL operators.

    This change is also backported to: 1.1.15

    References: #4093

  • [sql] [bug] Added a new method DefaultExecutionContext.get_current_parameters() which is used within a function-based default value generator in order to retrieve the current parameters being passed to the statement. The new function differs from the DefaultExecutionContext.current_parameters attribute in that it also provides for optional grouping of parameters that correspond to a multi-valued “insert” construct. Previously it was not possible to identify the subset of parameters that were relevant to the function call.

    References: #4075

  • [sql] [bug] Fixed bug in new SQL comments feature where table and column comment would not be copied when using Table.tometadata().

    References: #4087

  • [sql] [bug] In release 1.1, the Boolean type was broken in that boolean coercion via bool() would occur for backends that did not feature “native boolean”, but would not occur for native boolean backends, meaning the string "0" now behaved inconsistently. After a poll, a consensus was reached that non-boolean values should be raising an error, especially in the ambiguous case of string "0"; so the Boolean datatype will now raise ValueError if an incoming value is not within the range None, True, False, 1, 0.

    References: #4102

  • [sql] [bug] Refined the behavior of Operators.op() such that in all cases, if the Operators.op.is_comparison flag is set to True, the return type of the resulting expression will be Boolean, and if the flag is False, the return type of the resulting expression will be the same type as that of the left-hand expression, which is the typical default behavior of other operators. Also added a new parameter Operators.op.return_type as well as a helper method Operators.bool_op().

    References: #4063

  • [sql] [bug] Internal refinements to the Enum, Interval, and Boolean types, which now extend a common mixin Emulated that indicates a type that provides Python-side emulation of a DB native type, switching out to the DB native type when a supporting backend is in use. The Postgresql INTERVAL type when used directly will now include the correct type coercion rules for SQL expressions that also take effect for sqltypes.Interval (such as adding a date to an interval yields a datetime).

    References: #4088

postgresql

  • [postgresql] [bug] Fixed bug in Postgresql postgresql.dml.Insert.on_conflict_do_update() which would prevent the insert statement from being used as a CTE, e.g. via Insert.cte(), within another statement.

    This change is also backported to: 1.1.15

    References: #4074

  • [postgresql] [bug] Fixed bug in array_agg function where passing an argument that is already of type ARRAY, such as a Postgresql postgresql.array construct, would produce a ValueError, due to the function attempting to nest the arrays.

    This change is also backported to: 1.1.15

    References: #4107

  • [postgresql] [bug] Made further fixes to the ARRAY class in conjunction with COLLATE, as the fix made in #4006 failed to accommodate for a multidimentional array.

    This change is also backported to: 1.1.15

    References: #4006

  • [postgresql] [bug] Fixed bug where the pg8000 driver would fail if using MetaData.reflect() with a schema name, since the schema name would be sent as a “quoted_name” object that’s a string subclass, which pg8000 doesn’t recognize. The quoted_name type is added to pg8000’s py_types collection on connect.

    References: #4041

  • [postgresql] [bug] Enabled UUID support for the pg8000 driver, which supports native Python uuid round trips for this datatype. Arrays of UUID are still not supported, however.

    References: #4016

mysql

  • [mysql] [bug] Changed the name of the .values attribute of the new MySQL INSERT..ON DUPLICATE KEY UPDATE construct to .inserted, as Insert already has a method called Insert.values(). The .inserted attribute ultimately renders the MySQL VALUES() function.

    References: #4072

mssql

  • [mssql] [feature] Added a new mssql.TIMESTAMP datatype, that correctly acts like a binary datatype for SQL Server rather than a datetime type, as SQL Server breaks the SQL standard here. Also added mssql.ROWVERSION, as the “TIMESTAMP” type in SQL Server is deprecated in favor of ROWVERSION.

    References: #4086

  • [mssql] [feature] Added support for “AUTOCOMMIT” isolation level, as established via Connection.execution_options(), to the PyODBC and pymssql dialects. This isolation level sets the appropriate DBAPI-specific flags on the underlying connection object.

    References: #4058

  • [mssql] [bug] Added a full range of “connection closed” exception codes to the PyODBC dialect for SQL Server, including ‘08S01’, ‘01002’, ‘08003’, ‘08007’, ‘08S02’, ‘08001’, ‘HYT00’, ‘HY010’. Previously, only ‘08S01’ was covered.

    This change is also backported to: 1.1.15

    References: #4095

  • [mssql] [bug] SQL Server supports what SQLAlchemy calls “native boolean” with its BIT type, as this type only accepts 0 or 1 and the DBAPIs return its value as True/False. So the SQL Server dialects now enable “native boolean” support, in that a CHECK constraint is not generated for a Boolean datatype. The only difference vs. other native boolean is that there are no “true” / “false” constants so “1” and “0” are still rendered here.

    References: #4061

  • [mssql] [bug] Fixed the pymssql dialect so that percent signs in SQL text, such as used in modulus expressions or literal textual values, are not doubled up, as seems to be what pymssql expects. This is despite the fact that the pymssql DBAPI uses the “pyformat” parameter style which itself considers the percent sign to be significant.

    References: #4057

  • [mssql] [bug] Fixed bug where the SQL Server dialect could pull columns from multiple schemas when reflecting a self-referential foreign key constraint, if multiple schemas contained a constraint of the same name against a table of the same name.

    References: #4060

  • [mssql] [bug] [orm] Added a new class of “rowcount support” for dialects that is specific to when “RETURNING”, which on SQL Server looks like “OUTPUT inserted”, is in use, as the PyODBC backend isn’t able to give us rowcount on an UPDATE or DELETE statement when OUTPUT is in effect. This primarily affects the ORM when a flush is updating a row that contains server-calcluated values, raising an error if the backend does not return the expected row count. PyODBC now states that it supports rowcount except if OUTPUT.inserted is present, which is taken into account by the ORM during a flush as to whether it will look for a rowcount.

    References: #4062

  • [mssql] [bug] [orm] Enabled the “sane_rowcount” flag for the pymssql dialect, indicating that the DBAPI now reports the correct number of rows affected from an UPDATE or DELETE statement. This impacts mostly the ORM versioning feature in that it now can verify the number of rows affected on a target version.

  • [mssql] [bug] Added a rule to SQL Server index reflection to ignore the so-called “heap” index that is implicitly present on a table that does not specify a clustered index.

    References: #4059

oracle

  • [oracle] [bug] [py2k] [performance] Fixed performance regression caused by the fix for #3937 where cx_Oracle as of version 5.3 dropped the .UNICODE symbol from its namespace, which was interpreted as cx_Oracle’s “WITH_UNICODE” mode being turned on unconditionally, which invokes functions on the SQLAlchemy side which convert all strings to unicode unconditionally and causing a performance impact. In fact, per cx_Oracle’s author the “WITH_UNICODE” mode has been removed entirely as of 5.1, so the expensive unicode conversion functions are no longer necessary and are disabled if cx_Oracle 5.1 or greater is detected under Python 2. The warning against “WITH_UNICODE” mode that was removed under #3937 is also restored.

    This change is also backported to: 1.1.13, 1.0.19

    References: #4035

  • [oracle] [bug] Partial support for persisting and retrieving the Oracle value “infinity” is implemented with cx_Oracle, using Python float values only, e.g. float("inf"). Decimal support is not yet fulfilled by the cx_Oracle DBAPI driver.

    References: #4064

  • [oracle] [bug] The cx_Oracle dialect has been reworked and modernized to take advantage of new patterns that weren’t present in the old 4.x series of cx_Oracle. This includes that the minimum cx_Oracle version is the 5.x series and that cx_Oracle 6.x is now fully tested. The most significant change involves type conversions, primarily regarding the numeric / floating point and LOB datatypes, making more effective use of cx_Oracle type handling hooks to simplify how bind parameter and result data is processed.

  • [oracle] [bug] two phase support for cx_Oracle has been completely removed for all versions of cx_Oracle, whereas in 1.2.0b1 this change only took effect for the 6.x series of cx_Oracle. This feature never worked correctly in any version of cx_Oracle and in cx_Oracle 6.x, the API which SQLAlchemy relied upon was removed.

    References: #3997

  • [oracle] [bug] The column keys present in a result set when using Insert.returning() with the cx_Oracle backend now use the correct column / label names like that of all other dialects. Previously, these came out as ret_nnn.

  • [oracle] [bug] Several parameters to the cx_Oracle dialect are now deprecated and will have no effect: auto_setinputsizes, exclude_setinputsizes, allow_twophase.

  • [oracle] [bug] Fixed bug where an index reflected under Oracle with an expression like “column DESC” would not be returned, if the table also had no primary key, as a result of logic that attempts to filter out the index implicitly added by Oracle onto the primary key columns.

    References: #4042

  • [oracle] [bug] Fixed more regressions caused by cx_Oracle 6.0; at the moment, the only behavioral change for users is disconnect detection now detects for cx_Oracle.DatabaseError in addition to cx_Oracle.InterfaceError, as this behavior seems to have changed. Other issues regarding numeric precision and uncloseable connections are pending with the upstream cx_Oracle issue tracker.

    References: #4045

  • [oracle] [bug] Fixed bug where Oracle 8 “non ansi” join mode would not add the (+) operator to expressions that used an operator other than the = operator. The (+) needs to be on all columns that are part of the right-hand side.

    References: #4076

1.2.0b2

Released: July 24, 2017

orm

  • [orm] [bug] Fixed regression from 1.1.11 where adding additional non-entity columns to a query that includes an entity with subqueryload relationships would fail, due to an inspection added in 1.1.11 as a result of #4011.

    This change is also backported to: 1.1.12

    References: #4033

  • [orm] [bug] Fixed bug involving JSON NULL evaluation logic added in 1.1 as part of #3514 where the logic would not accommodate ORM mapped attributes named differently from the Column that was mapped.

    This change is also backported to: 1.1.12

    References: #4031

  • [orm] [bug] Added KeyError checks to all methods within WeakInstanceDict where a check for key in dict is followed by indexed access to that key, to guard against a race against garbage collection that under load can remove the key from the dict after the code assumes its present, leading to very infrequent KeyError raises.

    This change is also backported to: 1.1.12

    References: #4030

misc

  • [bug] [tests] [py3k] Fixed issue in testing fixtures which was incompatible with a change made as of Python 3.6.2 involving context managers.

    This change is also backported to: 1.1.12, 1.0.18

    References: #4034

1.2.0b1

Released: July 10, 2017

orm

  • [orm] [feature] An aliased() construct can now be passed to the Query.select_entity_from() method. Entities will be pulled from the selectable represented by the aliased() construct. This allows special options for aliased() such as aliased.adapt_on_names to be used in conjunction with Query.select_entity_from().

    This change is also backported to: 1.1.7

    References: #3933

  • [orm] [feature] Added .autocommit attribute to scoped_session, proxying the .autocommit attribute of the underling Session currently assigned to the thread. Pull request courtesy Ben Fagin.

  • [orm] [feature] Added a new feature orm.with_expression() that allows an ad-hoc SQL expression to be added to a specific entity in a query at result time. This is an alternative to the SQL expression being delivered as a separate element in the result tuple.

    References: #3058

  • [orm] [feature] Added a new style of mapper-level inheritance loading “polymorphic selectin”. This style of loading emits queries for each subclass in an inheritance hierarchy subsequent to the load of the base object type, using IN to specify the desired primary key values.

    References: #3948

  • [orm] [feature] Added a new kind of eager loading called “selectin” loading. This style of loading is very similar to “subquery” eager loading, except that it uses an IN expression given a list of primary key values from the loaded parent objects, rather than re-stating the original query. This produces a more efficient query that is “baked” (e.g. the SQL string is cached) and also works in the context of Query.yield_per().

    References: #3944

  • [orm] [feature] The lazy="select" loader strategy now makes used of the BakedQuery query caching system in all cases. This removes most overhead of generating a Query object and running it into a select() and then string SQL statement from the process of lazy-loading related collections and objects. The “baked” lazy loader has also been improved such that it can now cache in most cases where query load options are used.

    References: #3954

  • [orm] [feature] [ext] The Query.update() method can now accommodate both hybrid attributes as well as composite attributes as a source of the key to be placed in the SET clause. For hybrids, an additional decorator hybrid_property.update_expression() is supplied for which the user supplies a tuple-returning function.

    References: #3229

  • [orm] [feature] Added new attribute event AttributeEvents.bulk_replace(). This event is triggered when a collection is assigned to a relationship, before the incoming collection is compared with the existing one. This early event allows for conversion of incoming non-ORM objects as well. The event is integrated with the @validates decorator.

    References: #3896

  • [orm] [feature] Added new event handler AttributeEvents.modified() which is triggered when the func:.attributes.flag_modified function is invoked, which is common when using the sqlalchemy.ext.mutable extension module.

    References: #3303

  • [orm] [bug] Fixed issue with subquery eagerloading which continues on from the series of issues fixed in #2699, #3106, #3893 involving that the “subquery” contains the correct FROM clause when beginning from a joined inheritance subclass and then subquery eager loading onto a relationship from the base class, while the query also includes criteria against the subclass. The fix in the previous tickets did not accommodate for additional subqueryload operations loading more deeply from the first level, so the fix has been further generalized.

    This change is also backported to: 1.1.11

    References: #4011

  • [orm] [bug] Fixed bug where a cascade such as “delete-orphan” (but others as well) would fail to locate an object linked to a relationship that itself is local to a subclass in an inheritance relationship, thus causing the operation to not take place.

    This change is also backported to: 1.1.10

    References: #3986

  • [orm] [bug] Fixed a race condition which could occur under threaded environments as a result of the caching added via #3915. An internal collection of Column objects could be regenerated on an alias object inappropriately, confusing a joined eager loader when it attempts to render SQL and collect results and resulting in an attribute error. The collection is now generated up front before the alias object is cached and shared among threads.

    This change is also backported to: 1.1.7

    References: #3947

  • [orm] [bug] An UPDATE emitted as a result of the relationship.post_update feature will now integrate with the versioning feature to both bump the version id of the row as well as assert that the existing version number was matched.

    References: #3496

  • [orm] [bug] Repaired several use cases involving the relationship.post_update feature when used in conjunction with a column that has an “onupdate” value. When the UPDATE emits, the corresponding object attribute is now expired or refreshed so that the newly generated “onupdate” value can populate on the object; previously the stale value would remain. Additionally, if the target attribute is set in Python for the INSERT of the object, the value is now re-sent during the UPDATE so that the “onupdate” does not overwrite it (note this works just as well for server-generated onupdates). Finally, the SessionEvents.refresh_flush() event is now emitted for these attributes when refreshed within the flush.

    References: #3471, #3472

  • [orm] [bug] Fixed bug where programmatic version_id counter in conjunction with joined table inheritance would fail if the version_id counter were not actually incremented and no other values on the base table were modified, as the UPDATE would have an empty SET clause. Since programmatic version_id where version counter is not incremented is a documented use case, this specific condition is now detected and the UPDATE now sets the version_id value to itself, so that concurrency checks still take place.

    References: #3996

  • [orm] [bug] The versioning feature does not support NULL for the version counter. An exception is now raised if the version id is programmatic and was set to NULL for an UPDATE. Pull request courtesy Diana Clarke.

    References: #3673

  • [orm] [bug] Removed a very old keyword argument from scoped_session called scope. This keyword was never documented and was an early attempt at allowing for variable scopes.

    References: #3796

  • [orm] [bug] Fixed bug where combining a “with_polymorphic” load in conjunction with subclass-linked relationships that specify joinedload with innerjoin=True, would fail to demote those “innerjoins” to “outerjoins” to suit the other polymorphic classes that don’t support that relationship. This applies to both a single and a joined inheritance polymorphic load.

    References: #3988

  • [orm] [bug] Added new argument with_for_update to the Session.refresh() method. When the Query.with_lockmode() method were deprecated in favor of Query.with_for_update(), the Session.refresh() method was never updated to reflect the new option.

    References: #3991

  • [orm] [bug] Fixed bug where a column_property() that is also marked as “deferred” would be marked as “expired” during a flush, causing it to be loaded along with the unexpiry of regular attributes even though this attribute was never accessed.

    References: #3984

  • [orm] [bug] Fixed bug in subquery eager loading where the “join_depth” parameter for self-referential relationships would not be correctly honored, loading all available levels deep rather than correctly counting the specified number of levels for eager loading.

    References: #3967

  • [orm] [bug] Added warnings to the LRU “compiled cache” used by the Mapper (and ultimately will be for other ORM-based LRU caches) such that when the cache starts hitting its size limits, the application will emit a warning that this is a performance-degrading situation that may require attention. The LRU caches can reach their size limits primarily if an application is making use of an unbounded number of Engine objects, which is an antipattern. Otherwise, this may suggest an issue that should be brought to the SQLAlchemy developer’s attention.

  • [orm] [bug] Fixed bug to improve upon the specificity of loader options that take effect subsequent to the lazy load of a related entity, so that the loader options will match to an aliased or non-aliased entity more specifically if those options include entity information.

    References: #3963

  • [orm] [bug] The attributes.flag_modified() function now raises InvalidRequestError if the named attribute key is not present within the object, as this is assumed to be present in the flush process. To mark an object “dirty” for a flush without referring to any specific attribute, the attributes.flag_dirty() function may be used.

    References: #3753

  • [orm] [bug] The “evaluate” strategy used by Query.update() and Query.delete() can now accommodate a simple object comparison from a many-to-one relationship to an instance, when the attribute names of the primary key / foreign key columns don’t match the actual names of the columns. Previously this would do a simple name-based match and fail with an AttributeError.

    References: #3366

  • [orm] [bug] The @validates decorator now allows the decorated method to receive objects from a “bulk collection set” operation that have not yet been compared to the existing collection. This allows incoming values to be converted to compatible ORM objects as is already allowed from an “append” event. Note that this means that the @validates method is called for all values during a collection assignment, rather than just the ones that are new.

    References: #3896

  • [orm] [bug] Fixed bug in single-table inheritance where the select_from() argument would not be taken into account when limiting rows to a subclass. Previously, only expressions in the columns requested would be taken into account.

    References: #3891

  • [orm] [bug] When assigning a collection to an attribute mapped by a relationship, the previous collection is no longer mutated. Previously, the old collection would be emptied out in conjunction with the “item remove” events that fire off; the events now fire off without affecting the old collection.

    References: #3913

  • [orm] [bug] The state of the Session is now present when the SessionEvents.after_rollback() event is emitted, that is, the attribute state of objects prior to their being expired. This is now consistent with the behavior of the SessionEvents.after_commit() event which also emits before the attribute state of objects is expired.

    References: #3934

  • [orm] [bug] Fixed bug where Query.with_parent() would not work if the Query were against an aliased() construct rather than a regular mapped class. Also adds a new parameter util.with_parent.from_entity to the standalone util.with_parent() function as well as Query.with_parent().

    References: #3607

orm declarative

  • [bug] [orm] [declarative] Fixed bug where using declared_attr on an AbstractConcreteBase where a particular return value were some non-mapped symbol, including None, would cause the attribute to hard-evaluate just once and store the value to the object dictionary, not allowing it to invoke for subclasses. This behavior is normal when declared_attr is on a mapped class, and does not occur on a mixin or abstract class. Since AbstractConcreteBase is both “abstract” and actually “mapped”, a special exception case is made here so that the “abstract” behavior takes precedence for declared_attr.

    References: #3848

engine

  • [engine] [feature] Added native “pessimistic disconnection” handling to the Pool object. The new parameter Pool.pre_ping, available from the engine as create_engine.pool_pre_ping, applies an efficient form of the “pre-ping” recipe featured in the pooling documentation, which upon each connection check out, emits a simple statement, typically “SELECT 1”, to test the connection for liveness. If the existing connection is no longer able to respond to commands, the connection is transparently recycled, and all other connections made prior to the current timestamp are invalidated.

    References: #3919

  • [engine] [bug] Added an exception handler that will warn for the “cause” exception on Py2K when the “autorollback” feature of Connection itself raises an exception. In Py3K, the two exceptions are naturally reported by the interpreter as one occurring during the handling of the other. This is continuing with the series of changes for rollback failure handling that were last visited as part of #2696 in 1.0.12.

    This change is also backported to: 1.1.7

    References: #3946

  • [engine] [bug] Fixed bug where in the unusual case of passing a Compiled object directly to Connection.execute(), the dialect with which the Compiled object were generated was not consulted for the paramstyle of the string statement, instead assuming it would match the dialect-level paramstyle, causing mismatches to occur.

    References: #3938

sql

  • [sql] [feature] Added a new kind of bindparam() called “expanding”. This is for use in IN expressions where the list of elements is rendered into individual bound parameters at statement execution time, rather than at statement compilation time. This allows both a single bound parameter name to be linked to an IN expression of multiple elements, as well as allows query caching to be used with IN expressions. The new feature allows the related features of “select in” loading and “polymorphic in” loading to make use of the baked query extension to reduce call overhead. This feature should be considered to be experimental for 1.2.

    References: #3953

  • [sql] [feature] [postgresql] [mysql] [oracle] Added support for SQL comments on Table and Column objects, via the new Table.comment and Column.comment arguments. The comments are included as part of DDL on table creation, either inline or via an appropriate ALTER statement, and are also reflected back within table reflection, as well as via the Inspector. Supported backends currently include MySQL, Postgresql, and Oracle. Many thanks to Frazer McLean for a large amount of effort on this.

    References: #1546

  • [sql] [feature] The longstanding behavior of the ColumnOperators.in_() and ColumnOperators.notin_() operators emitting a warning when the right-hand condition is an empty sequence has been revised; a simple “static” expression of “1 != 1” or “1 = 1” is now rendered by default, rather than pulling in the original left-hand expression. This causes the result for a NULL column comparison against an empty set to change from NULL to true/false. The behavior is configurable, and the old behavior can be enabled using the create_engine.empty_in_strategy parameter to create_engine().

    References: #3907

  • [sql] [feature] Added a new option autoescape to the “startswith” and “endswith” classes of comparators; this supplies an escape character also applies it to all occurrences of the wildcard characters “%” and “_” automatically. Pull request courtesy Diana Clarke.

    References: #2694

  • [sql] [bug] Fixed AttributeError which would occur in WithinGroup construct during an iteration of the structure.

    This change is also backported to: 1.1.11

    References: #4012

  • [sql] [bug] Fixed regression released in 1.1.5 due to #3859 where adjustments to the “right-hand-side” evaluation of an expression based on Variant to honor the underlying type’s “right-hand-side” rules caused the Variant type to be inappropriately lost, in those cases when we do want the left-hand side type to be transferred directly to the right hand side so that bind-level rules can be applied to the expression’s argument.

    This change is also backported to: 1.1.9

    References: #3952

  • [sql] [bug] [postgresql] Changed the mechanics of ResultProxy to unconditionally delay the “autoclose” step until the Connection is done with the object; in the case where Postgresql ON CONFLICT with RETURNING returns no rows, autoclose was occurring in this previously non-existent use case, causing the usual autocommit behavior that occurs unconditionally upon INSERT/UPDATE/DELETE to fail.

    This change is also backported to: 1.1.9

    References: #3955

  • [sql] [bug] The rules for type coercion between Numeric, Integer, and date-related types now include additional logic that will attempt to preserve the settings of the incoming type on the “resolved” type. Currently the target for this is the asdecimal flag, so that a math operation between Numeric or Float and Integer will preserve the “asdecimal” flag as well as if the type should be the Float subclass.

    References: #4018

  • [sql] [bug] [mysql] The result processor for the Float type now unconditionally runs values through the float() processor if the dialect specifies that it also supports “native decimal” mode. While most backends will deliver Python float objects for a floating point datatype, the MySQL backends in some cases lack the typing information in order to provide this and return Decimal unless the float conversion is done.

    References: #4020

  • [sql] [bug] Added some extra strictness to the handling of Python “float” values passed to SQL statements. A “float” value will be associated with the Float datatype and not the Decimal-coercing Numeric datatype as was the case before, eliminating a confusing warning emitted on SQLite as well as unecessary coercion to Decimal.

    References: #4017

  • [sql] [bug] The operator precedence for all comparison operators such as LIKE, IS, IN, MATCH, equals, greater than, less than, etc. has all been merged into one level, so that expressions which make use of these against each other will produce parentheses between them. This suits the stated operator precedence of databases like Oracle, MySQL and others which place all of these operators as equal precedence, as well as Postgresql as of 9.5 which has also flattened its operator precendence.

    References: #3999

  • [sql] [bug] Repaired issue where the type of an expression that used ColumnOperators.is_() or similar would not be a “boolean” type, instead the type would be “nulltype”, as well as when using custom comparison operators against an untyped expression. This typing can impact how the expression behaves in larger contexts as well as in result-row-handling.

    References: #3873

  • [sql] [bug] Fixed the negation of a Label construct so that the inner element is negated correctly, when the not_() modifier is applied to the labeled expression.

    References: #3969

  • [sql] [bug] The system by which percent signs in SQL statements are “doubled” for escaping purposes has been refined. The “doubling” of percent signs mostly associated with the literal_column construct as well as operators like ColumnOperators.contains() now occurs based on the stated paramstyle of the DBAPI in use; for percent-sensitive paramstyles as are common with the Postgresql and MySQL drivers the doubling will occur, for others like that of SQLite it will not. This allows more database-agnostic use of the literal_column construct to be possible.

    References: #3740

  • [sql] [bug] Fixed bug where a column-level CheckConstraint would fail to compile the SQL expression using the underlying dialect compiler as well as apply proper flags to generate literal values as inline, in the case that the sqltext is a Core expression and not just a plain string. This was long-ago fixed for table-level check constraints in 0.9 as part of #2742, which more commonly feature Core SQL expressions as opposed to plain string expressions.

    References: #3957

  • [sql] [bug] Fixed bug where a SQL-oriented Python-side column default could fail to be executed properly upon INSERT in the “pre-execute” codepath, if the SQL itself were an untyped expression, such as plain text. The “pre- execute” codepath is fairly uncommon however can apply to non-integer primary key columns with SQL defaults when RETURNING is not used.

    References: #3923

  • [sql] [bug] The expression used for COLLATE as rendered by the column-level expression.collate() and ColumnOperators.collate() is now quoted as an identifier when the name is case sensitive, e.g. has uppercase characters. Note that this does not impact type-level collation, which is already quoted.

    References: #3785

  • [sql] [bug] Fixed bug where the use of an Alias object in a column context would raise an argument error when it tried to group itself into a parenthesized expression. Using Alias in this way is not yet a fully supported API, however it applies to some end-user recipes and may have a more prominent role in support of some future Postgresql features.

    References: #3939

schema

  • [schema] [bug] An ArgumentError is now raised if a ForeignKeyConstraint object is created with a mismatched number of “local” and “remote” columns, which otherwise causes the internal state of the constraint to be incorrect. Note that this also impacts the condition where a dialect’s reflection process produces a mismatched set of columns for a foreign key constraint.

    This change is also backported to: 1.1.10

    References: #3949

postgresql

  • [postgresql] [bug] Continuing with the fix that correctly handles Postgresql version string “10devel” released in 1.1.8, an additional regexp bump to handle version strings of the form “10beta1”. While Postgresql now offers better ways to get this information, we are sticking w/ the regexp at least through 1.1.x for the least amount of risk to compatibility w/ older or alternate Postgresql databases.

    This change is also backported to: 1.1.11

    References: #4005

  • [postgresql] [bug] Fixed bug where using ARRAY with a string type that features a collation would fail to produce the correct syntax within CREATE TABLE.

    This change is also backported to: 1.1.11

    References: #4006

  • [postgresql] [bug] Added “autocommit” support for GRANT, REVOKE keywords. Pull request courtesy Jacob Hayes.

    This change is also backported to: 1.1.10

  • [postgresql] [bug] Added support for parsing the Postgresql version string for a development version like “PostgreSQL 10devel”. Pull request courtesy Sean McCully.

    This change is also backported to: 1.1.8

  • [postgresql] [bug] Fixed bug where the base sqltypes.ARRAY datatype would not invoke the bind/result processors of postgresql.ARRAY.

    References: #3964

  • [postgresql] [bug] Added support for all possible “fields” identifiers when reflecting the Postgresql INTERVAL datatype, e.g. “YEAR”, “MONTH”, “DAY TO MINUTE”, etc.. In addition, the postgresql.INTERVAL datatype itself now includes a new parameter postgresql.INTERVAL.fields where these qualifiers can be specified; the qualifier is also reflected back into the resulting datatype upon reflection / inspection.

    References: #3959

mysql

  • [mysql] [feature] Added support for MySQL’s ON DUPLICATE KEY UPDATE MySQL-specific mysql.dml.Insert object. Pull request courtesy Michael Doronin.

    References: #4009

  • [mysql] [bug] MySQL 5.7 has introduced permission limiting for the “SHOW VARIABLES” command; the MySQL dialect will now handle when SHOW returns no row, in particular for the initial fetch of SQL_MODE, and will emit a warning that user permissions should be modified to allow the row to be present.

    This change is also backported to: 1.1.11

    References: #4007

  • [mysql] [bug] Removed an ancient and unnecessary intercept of the UTC_TIMESTAMP MySQL function, which was getting in the way of using it with a parameter.

    This change is also backported to: 1.1.10

    References: #3966

  • [mysql] [bug] Fixed bug in MySQL dialect regarding rendering of table options in conjunction with PARTITION options when rendering CREATE TABLE. The PARTITION related options need to follow the table options, whereas previously this ordering was not enforced.

    This change is also backported to: 1.1.10

    References: #3961

  • [mysql] [bug] Added support for views that are unreflectable due to stale table definitions, when calling MetaData.reflect(); a warning is emitted for the table that cannot respond to DESCRIBE, but the operation succeeds.

    References: #3871

mssql

  • [mssql] [bug] Fixed bug where SQL Server transaction isolation must be fetched from a different view when using Azure data warehouse, the query is now attempted against both views and then a NotImplemented is raised unconditionally if failure continues to provide the best resiliency against future arbitrary API changes in new SQL Server versions.

    This change is also backported to: 1.1.11

    References: #3994

  • [mssql] [bug] Added a placeholder type mssql.XML to the SQL Server dialect, so that a reflected table which includes this type can be re-rendered as a CREATE TABLE. The type has no special round-trip behavior nor does it currently support additional qualifying arguments.

    This change is also backported to: 1.1.11

    References: #3973

  • [mssql] [bug] The SQL Server dialect now allows for a database and/or owner name with a dot inside of it, using brackets explicitly in the string around the owner and optionally the database name as well. In addition, sending the quoted_name construct for the schema name will not split on the dot and will deliver the full string as the “owner”. quoted_name is also now available from the sqlalchemy.sql import space.

    References: #2626

oracle

  • [oracle] [feature] [posgresql] Added new keywords Sequence.cache and Sequence.order to Sequence, to allow rendering of the CACHE parameter understood by Oracle and PostgreSQL, and the ORDER parameter understood by Oracle. Pull request courtesy David Moore.

    This change is also backported to: 1.1.12

  • [oracle] [feature] The Oracle dialect now inspects unique and check constraints when using Inspector.get_unique_constraints(), Inspector.get_check_constraints(). As Oracle does not have unique constraints that are separate from a unique Index, a Table that’s reflected will still continue to not have UniqueConstraint objects associated with it. Pull requests courtesy Eloy Felix.

    References: #4003

  • [oracle] [bug] Support for two-phase transactions has been removed entirely for cx_Oracle when version 6.0b1 or later of the DBAPI is in use. The two- phase feature historically has never been usable under cx_Oracle 5.x in any case, and cx_Oracle 6.x has removed the connection-level “twophase” flag upon which this feature relied.

    This change is also backported to: 1.1.11

    References: #3997

  • [oracle] [bug] Fixed bug in cx_Oracle dialect where version string parsing would fail for cx_Oracle version 6.0b1 due to the “b” character. Version string parsing is now via a regexp rather than a simple split.

    This change is also backported to: 1.1.10

    References: #3975

  • [oracle] [bug] The cx_Oracle dialect now supports “sane multi rowcount”, that is, when a series of parameter sets are executed via DBAPI cursor.executemany(), we can make use of cursor.rowcount to verify the number of rows matched. This has an impact within the ORM when detecting concurrent modification scenarios, in that some simple conditions can now be detected even when the ORM is batching statements, as well as when the more strict versioning feature is used, the ORM can still use statement batching. The flag is enabled for cx_Oracle assuming at least version 5.0, which is now commonplace.

    References: #3932

  • [oracle] [bug] Oracle reflection now “normalizes” the name given to a foreign key constraint, that is, returns it as all lower case for a case insensitive name. This was already the behavior for indexes and primary key constraints as well as all table and column names. This will allow Alembic autogenerate scripts to compare and render foreign key constraint names correctly when initially specified as case insensitive.

    References: #3276

misc

  • [feature] [ext] Added new flag Session.enable_baked_queries to the Session to allow baked queries to be disabled session-wide, reducing memory use. Also added new Bakery wrapper so that the bakery returned by BakedQuery.bakery can be inspected.

  • [bug] [ext] Protected against testing “None” as a class in the case where declarative classes are being garbage collected and new automap prepare() operations are taking place concurrently, very infrequently hitting a weakref that has not been fully acted upon after gc.

    This change is also backported to: 1.1.10

    References: #3980

  • [bug] [ext] Fixed bug in sqlalchemy.ext.mutable where the Mutable.as_mutable() method would not track a type that had been copied using TypeEngine.copy(). This became more of a regression in 1.1 compared to 1.0 because the TypeDecorator class is now a subclass of SchemaEventTarget, which among other things indicates to the parent Column that the type should be copied when the Column is. These copies are common when using declarative with mixins or abstract classes.

    This change is also backported to: 1.1.8

    References: #3950

  • [bug] [ext] Added support for bound parameters, e.g. those normally set up via Query.params(), to the baked.Result.count() method. Previously, support for parameters were omitted. Pull request courtesy Pat Deegan.

    This change is also backported to: 1.1.8

  • [bug] [ext] The AssociationProxy.any(), AssociationProxy.has() and AssociationProxy.contains() comparison methods now support linkage to an attribute that is itself also an AssociationProxy, recursively.

    References: #3769

  • [bug] [ext] Implemented in-place mutation operators __ior__, __iand__, __ixor__ and __isub__ for mutable.MutableSet and __iadd__ for mutable.MutableList so that change events are fired off when these mutator methods are used to alter the collection.

    References: #3853

  • [bug] [declarative] A warning is emitted if the declared_attr.cascading modifier is used with a declarative attribute that is itself declared on a class that is to be mapped, as opposed to a declarative mixin class or __abstract__ class. The declared_attr.cascading modifier currently only applies to mixin/abstract classes.

    References: #3847

  • [bug] [ext] Improved the association proxy list collection so that premature autoflush against a newly created association object can be prevented in the case where list.append() is being used, and a lazy load would be invoked when the association proxy accesses the endpoint collection. The endpoint collection is now accessed first before the creator is invoked to produce the association object.

    References: #3941

  • [bug] [ext] The sqlalchemy.ext.hybrid.hybrid_property class now supports calling mutators like @setter, @expression etc. multiple times across subclasses, and now provides a @getter mutator, so that a particular hybrid can be repurposed across subclasses or other classes. This now matches the behavior of @property in standard Python.

    References: #3912, #3911

  • [bug] [ext] Fixed a bug in the sqlalchemy.ext.serializer extension whereby an “annotated” SQL element (as produced by the ORM for many types of SQL expressions) could not be reliably serialized. Also bumped the default pickle level for the serializer to “HIGHEST_PROTOCOL”.

    References: #3918

Previous: What’s New in SQLAlchemy 1.2? Next: 1.1 Changelog