bug: NVARCHAR(MAX) column fails on mssql connector #11191
Comments
|
@akanz1 can you provide a more complete reproducer? The MSSSQL datatype converter already has the right logic, it just is the base converter type that has the bad implementation (as you point out). I am wondering if this bug is already fixed on For example, this does NOT raise an error on main: import ibis
from ibis.expr import datatypes as dt
from ibis.backends.sql.datatypes import MSSQLType
a = dt.String(length=None)
sgt = MSSQLType.from_ibis(a)
b = MSSQLType.to_ibis(sgt)
assert a == b |
|
I don't think we need the entire schema, I think if you can print out the string argument passed to then that would be sufficent. I am trying to do from ibis.backends.mssql import Backend as MSSQLBackend
MSSQLBackend.compiler.type_mapper.from_string("NVARCHAR(MAX)")and that isn't giving me the error that you show. |
|
@NickCrews thank you for looking into this. Not sure i understand correctly but heres the context at this location on main ( query = "\n SELECT\n name,\n is_nullable,\n system_type_name,\n precision,\n scale,\n error_number,\n error_message\n FROM sys.dm_exec_describe_first_result_set(N'WITH [sample_data] AS (SELECT TOP 1000000 * FROM [dbo].[my_table]) SELECT COUNT_BIG(*) FROM [sample_data]', NULL, 0)\n ORDER BY column_ordinal\n "
system_type_name = 'nvarchar(max)'
nullable = True
tsql = "'WITH [sample_data] AS (SELECT TOP 1000000 * FROM [dbo].[my_table]) SELECT COUNT_BIG(*) FROM [sample_data]'"the problem still happens in the same place from what i can see where
|
|
Like what does your user code look like? You are calling my_msssql_connection.table("some_table_with_a_nvarcharmax_column")? |
|
Yes, just like that but using sql_statement = 'SELECT TOP 1000000 * FROM [dbo].[my_table]' # my_table has a column of type nvarchar(max), other nvarchar(some number) columns work
self.con = self.connect() # is of type: ibis.backends.mssql.Backend
self.con.sql(sql_statement)yields |
|
Ah, I see. In SqlGlotType, we do @classmethod
def _from_sqlglot_VARCHAR(
cls, length: sge.DataTypeParam | None = None, nullable: bool | None = None
) -> dt.String:
return dt.String(
length=int(length.this.this) if length is not None else None,
nullable=nullable,
)
_from_sqlglot_NVARCHAR = _from_sqlglot_NCHAR = _from_sqlglot_CHAR = (
_from_sqlglot_FIXEDSTRING
) = _from_sqlglot_VARCHARwhich does too early of binding of the NVARCHAR converter. Even though the |
|
Any updates on this? |
Uh oh!
There was an error while loading. Please reload this page.
What happened?
After upgrading from 10.2.0 to 10.5.0 the dtype
NVARCHAR(MAX)fails on mssql (likely also others are affected). previously this got resolved to aString()type.The recently added handling of this type leads to a failure since in
sql/datatypes.py:219length.this.thisyields 'MAX' which can not be places into anint()statement.to make it work i've forked and fixed it like so:
i guess the following is even a bit nicer as it avoids the redundancy around None checks
What version of ibis are you using?
Ibis 10.5.0
python 3.11.11
What backend(s) are you using, if any?
No response
Relevant log output
Code of Conduct
The text was updated successfully, but these errors were encountered: