14 SPARK

============================================================================== Simba Apache Spark ODBC Data Connector Release Notes ==============================================================================

The release notes provide details of enhancements, features, known issues, and workflow changes in Simba Apache Spark ODBC Connector 2.12.1, as well as the version history.

This version of the driver supports the following data source versions and platforms:

  • Apache Spark versions 3.1.2 to 4.0.0

  • Windows 11, Windows Server 2025, 2022, 2019, 2016, 2012

  • macOS 14, 15, 26

  • Debian 11

  • RHEL 8, 8ARM, 9, 9ARM, 10, 10ARM, AL2023 ARM

  • SLES 15, 16

  • Ubuntu 22.04, 24.04

2.12.1 =======================================================================

Released 2026-7-17

  • [SPARKO-1847] Updated third-party libraries

    The connector now uses the following third-party libraries:

    • Expat 2.8.2 (previously 2.8.1)
    • libcURL 8.21.0 (previously 8.20.0)

Known Issues The following are known issues that you may encounter due to limitations in the data source, the connector, or an application.

  • [SPARKO-1404] When querying tables that contain VOID columns, the server returns an error.

  • [SPARKO-1101] When the Auth_AccessToken line length is longer than the maximum limit of 1000, the connector returns an authentication error. For more information, see the Installation and Configuration Guide.

  • [SPARKO-879] When connecting to a server that supports multiple catalogs, the connector no longer reports the catalog for schemas and tables as SPARK.

    The Spark server now reports the catalog.

  • [SPARKO-670] In some cases, when retrieving timestamp data, the connector returns an error.

    In some cases, when connecting to certain distributions of Apache Spark, the connector returns the following error: “Conversion from number to string failed due to undersized character buffer”. This issue affects versions 2.6.12 to 2.6.14 of the Spark ODBC connector.

    As a workaround, set EnableArrow=0 in the connection string or DSN.

  • [SPARKO-620] Issue with date and timestamp before the beginning of the Gregorian calendar when connecting to Spark 2.4.4 or later, or versions previous to 3.0, with Arrow result set serialization.

    When using Spark 2.4.4 or later, or versions previous to Spark 3.0, DATE and TIMESTAMP data before October 15, 1582 may be returned incorrectly if the server supports serializing query results using Apache Arrow. This issue should not impact most distributions of Apache Spark.

    To confirm if your distribution of Spark 2.4.4 or later has been impacted by this issue, you can execute the following query:

    SELECT DATE ‘1581-10-14’

    If the result returned by the connector is 1581-10-24, then you are impacted by the issue. In this case, if your data set contains date and/or timestamp data earlier than October 15, 1582, you can work around this issue by adding EnableArrow=0 in your DSN or connection string to disable the Arrow result set serialization feature.

  • When retrieving data from a BINARY column, a ClassCastException error occurs.

    In Spark 1.6.3 or earlier, the server sometimes returns a ClassCastException error when attempting to retrieve data from a BINARY column.

    This issue is fixed as of Spark 2.0.0.

    For more information, see the JIRA issue posted by Apache named “When column type is binary, select occurs ClassCastException in Beeline” at https://issues.apache.org/jira/browse/SPARK-12143.

Workflow Changes =============================================================

The following changes may disrupt established workflows for the connector.

2.12.0 ———————————————————————–

  • [SPARKO-1760] SQL transactions limited to native query mode only.

    If the user attempts to use SQL transactions when UseNativeQuery=0, the connector will throw an error.

2.11.0 ———————————————————————–

  • [SPARKO-1673] Removed Windows 10 support.

    Beginning with this release, the connector no longer support Windows 10.

2.10.0 ———————————————————————–

  • [SPARKO-1669] Removed macOS 12 and macOS 13 support

    Beginning with this release, the connector no longer supports macOS 12 or

2.9.4 ————————————————————————

  • [SPARKO-1545] Removed macOS 11 support

    Beginning with this release, the connector no longer supports macOS 11. For information about the supported versions, see the Installation and Configuration Guide.

2.9.2 ————————————————————————

  • [SPARKO-1426] Updated default value

    The default values of max catalog name length and max schema name length have been changed from 128 to 1024.

  • [SPARKO-1389] Removed support for Ubuntu 20.04

    As early as May 2025, the connector will no longer support Ubuntu 20.04. For information about the supported versions, see the Installation and Configuration Guide.

2.9.1 ————————————————————————

  • [SPARKO-1401] Dropped support for earlier versions of Linux distributions

    The connector no longer supports the following versions of Linux distributions:

    • CentOS 7
    • RHEL 7
    • SLES 12

    For a list of supported Linux versions, see the Installation and Configuration Guide.

2.8.0 ————————————————————————

  • [SPARKO-1167] Updated OAuth client ID value

    For the OAuth browser based authentication, the default value of the Client ID property is now “96eecda7-19ea-49cc-abb5-240097d554f5”. Previously, the default value was “5e5b7d2b-32b1-4ac0-be5d-ad3ad710fa6e”.

  • [SPARKO-1155] Removed libcURL folder

    Beginning with this release, on Windows, lib64.DllA (for the 64-bit driver) and lib32.DllA (for the 32-bit driver) folders have been removed.

Version History ==============================================================

2.12.0 =======================================================================

Released 2026-6-12

Enhancements & New Features

  • [SPARKO-1756] SQLProcedures and SQLProcedureColumns support.

    The connector now supports SQLProcedures and SQLProcedureColumns catalog functions for retrieving stored procedure metadata when connected to Databricks 17.0 and later.

  • [SPARKO-1691] Case-insensitive metadata filters.

    The connector now supports case-insensitive metadata filters for catalog functions.

    To enable this feature, set CaseInsensitiveMetadataFilter to 1. For more information, see the Installation and Configuration Guide.

  • [SPARKO-1661] Use the Arrow data type for decimals.

    The connector now enables decimalAsArrow for decimal data types. This can be turned off by setting the connection flag DecimalAsArrow=0.

  • [SPARKO-1721] Arrow data performance improvements.

    This release improves performance when using Arrow data.

  • [SPARKO-1728] Bulk fetch performance improvements.

    This release improves bulk fetch performance.

  • [SPARKO-1674] TLS handshake failure logging improvements.

    Server certificates are reported in the log file when there is an error during SSL handshakes.

  • [SPARKO-1775] SLES 16 support

    The connector now supports SLES 16.

  • [SPARKO-1798] [SPARKO-1810] Updated third-party libraries

    The connector now uses the following third-party libraries:

    • Expat 2.8.1 (previously 2.7.4)
    • libcURL 8.20.0 (previously 8.18.0)
    • OpenSSL 3.5.7 (previously 3.0.19)
    • Thrift 0.23.0 (previously 0.17.0)
    • zlib 1.3.2 (previously 1.3.1)

Resolved Issues The following issues have been resolved in Spark ODBC Connector 2.12.0.

  • [SPARKO-1623] A translation issue for column names when a FROM clause subquery is referenced in a correlated WHERE clause subquery has been resolved.

  • [SPARKO-1812] Previously, the connector may throw a “Failed to deserialize the arrowSchema” error during query execution; this has been fixed.

2.11.0 =======================================================================

Released 2026-3-20

Enhancements & New Features

  • [SPARKO-1658] Stored Procedures

    The connector now supports SQL stored procedures when using Databricks Unity Catalog.

  • [SPARKO-1696] Bulk fetch performance improvement

    The connector has improved bulk fetch performance.

  • [SPARKO-1677] Geospatial datatype metadata improvement

    The connector now returns “GEOMETRY” or “GEOGRAPHY” for SQL_DESC_TYPE_NAME on geospatial data types.

  • [SPARKO-1735] macOS 26 support

    The connector now supports macOS 26.

  • [SPARKO-1571] [SPARKO-1737] Updated third-party libraries

    The connector now uses the following third-party libraries:

    • Expat 2.7.4 (previously 2.7.1)
    • OpenSSL 3.0.19 (previously 3.0.18)
    • libcURL 8.18.0 (previously 8.16.0)

2.10.0 =======================================================================

Released 2025-12-19

Enhancements & New Features

  • [SPARKO-1506][SPARKO-1643] Multi-statement transaction support

    The connector now supports multi-statement transactions when using compatible Databricks warehouses/tables.

    Use either SQLSetConnectAttr(SQL_AUTOCOMMIT_OFF=0) with SQLEndTran(COMMIT) / SQLEndTran(ROLLBACK), or SQL statements (BEGIN TRANSACTION / COMMIT / ROLLBACK). Note that mixing the ODBC API calls and SQL statements to use transactions is not supported.

  • [SPARKO-1562] Metric views support

    The connector now supports Databricks metric views. When the connection property EnableMetricViewMetadata is set to 1, the connector sets the session configuration spark.sql.thriftserver.metadata.metricview.enabled to true and will identify metric views as distinct from tables, and metric view measure columns as distinct from regular columns.

  • [SPARKO-1619] Databricks 17.3 support

    The connector now support Databricks version 17.3 LTS.

  • [SPARKO-1668] Debian 13 support

    The connector now supports Debian 13 and Debian 13 on ARM.

2.9.4 ========================================================================

Released 2025-10-10

Enhancements & New Features

  • [SPARKO-1499] File DSN in Windows support

    The connector now saves the minimal properties in the FileDSN to work around the limitation of the Windows Driver Manager. For more information, see the Installation and Configuration Guide.

  • [SPARKO-1525][SPARKO-1549] Telemetry milestone support

    The connector now supports sending telemetry to databricks servers. For more information, see the Installation and Configuration Guide.

  • [SPARKO-1535][SPARKO-1552][SPARKO-1553][SPARKO-1565] Linux ARM support

    The connector now supports Linux ARM architecture. For more information, see the Installation and Configuration Guide.

  • [SPARKO-1545] macOS15 ARM support

    The connector now supports macOS15 ARM. For more information, see the Installation and Configuration Guide.

  • [SPARKO-1558] Logging Download Speeds with Cloud Fetch

    With Cloud Fetch enabled, the connector logs average download speed at INFO level, in-progress speeds at TRACE level, and slow downloads at WARNING level.

  • [SPARKO-1561] Kerberos auth support

    The connector now supports Kerberos authentication for Proxy. For more information, see the Installation and Configuration Guide.

  • [SPARKO-1564] SQL Scripting support

    The connector now supports SQL scripting syntax where scripts are bookended by the BEGIN and END keywords.

  • [SPARKO-1599] Updated RHEL support

    The connector now supports RHEL 10 operating system. For a list of supported Linux versions, see the Installation and Configuration Guide.

  • [SPARKO-1607] Updated third-party libraries

    The connector now uses the following third-party libraries:

    • OpenSSL 3.0.18 (previously 3.0.16)
    • libcURL 8.16.0 (previously 8.12.1)

Resolved Issues The following issues have been resolved in Simba Apache Spark ODBC Connector 2.9.4.

  • [SPARKO-1516][SPARKO-1518] The connector causes queries to time out after 20 minutes.

  • [SPARKO-1537] The connector uses the ‘username’ and ‘password’ properties specified in the connection string for authentication in place of -u -p.

2.9.2 ————————————————————————

Released 2025-04-25

Enhancements & New Features

  • [SPARKO-1325][SPARKO-1435] Timestamp_NTZ support

    The driver now recognizes timestamp_ntz columns in the following data source tables:

    • SQLGetTypeInfo (SQL_ALL_TYPES) now lists timestamp_ntz as an available type.
    • SQLGetTypeInfo (SQL_TIMESTAMP) now lists timestamp_ntz as one of the types that map to SQL timestamps.
    • SQLColumns now identifies timestamp_ntz columns correctly.
    • If UseNativeQuery is set to 0, SQLColAttribute (SQL_DESC_TYPE_NAME) identifies timestamp_ntz columns correctly.

  • [SPARKO-1384][SPARKO-1405][SPARKO-1411] CRL cache support

    On Windows, the driver supports the CRL cache when UseSystemTruststore is enabled (set to 1).

  • [SPARKO-1399][SPARKO-1417] Updated third-party libraries

    The connector now uses the following third-party libraries:

    • OpenSSL 3.0.16 (previously 3.0.15)
    • libcURL 8.12.1 (previously 8.11.0)
    • Expat 2.7.1 (previously 2.6.3)

  • [SPARKO-1404] VOID type support

    The driver now supports VOID columns and lists them correctly in SQLGetColumns calls.

  • [SPARKO-1408] Process Name as Default UserAgentEntry

    The driver will now use the process name as the UserAgentEntry if the UserAgentEntry is not set.

  • [SPARKO-1419][SPARKO-1420] OAuth Token exchange support

    The driver now supports OAuth Token exchange feature for IDP different from the host. In these cases, OAuth access token (including BYOT) will be exchanged for a Databricks in-house access token. For more information, see the Installation and Configuration Guide.

  • [SPARKO-1474] Upgraded Windows Server support

    The connector now supports Windows Server 2025. For a list of supported versions, see the Installation and Configuration Guide.

2.9.1 ————————————————————————

Released 2024-11-15

Enhancements & New Features

  • [SPARKO-1383] Updated Thrift Version for HTTP User-Agent Header

    The HTTP User-Agent used by the driver has been updated from thrift/0.9.0 to thrift/0.17.0.

  • [SPARKO-1005] Driver-Side Timeouts for Idle Statements

    Added support for configuring the fetch result idle timeout. For more information, see the Installation and Configuration Guide.

  • [SPARKO-1266] Upgrade Arrow

    Apache Arrow has been upgraded to version 15.0.0 across all non-Windows platforms.

  • [SPARKO-1351][SPARKO-1387] Updated third-party libraries

    The connector now uses the following third-party libraries:

    • OpenSSL 3.0.15 (previously 3.0.13)
    • libcURL 8.11.0 (previously 8.7.1)
    • ICU 74.2 (previously 71.1)
    • Arrow 15.0.0 (previously 9.0.0) for OSX & Linux (Windows was on 15.0.0)
    • lz4 1.10.0. (previously 1.9.4)
    • Expat 2.6.3 (previously 2.6.2)
    • Kerberos 1.21.3 (previously 1.21.2) for OSX & Linux

  • [SPARKO-1360] Support for Token Passthrough OAuth

    The driver now supports token renewal for the Token Passthrough OAuth flow.

  • [SPARKO-1369] Use ‘hive_metastore’ as default for Empty Initial Catalog

    If the server sends an empty string as the initial catalog, the driver will default to using the “hive_metastore” catalog.

  • [SPARKO-1378] Buffer length based on the DM’s encoding.

    The driver now returns the buffer length based on the DM’s encoding.

Resolved Issues The following issues have been resolved in Simba Apache Spark ODBC Connector 2.9.1.

  • [SPARKO-1149] When the driver encounters an error while sending heartbeat, it now re-attempts the heartbeat once before closing the heartbeat thread.

  • [SPARKO-1376] HttpPath property missing issue when setting up DSN through Add-OdbcDsn in PowerShell has been fixed.

2.8.2 ————————————————————————

Released 2024-05-31

Enhancements & New Features

  • [SPARKO-1204][SPARKO-1243][SPARKO-1279] Updated authentication support

    The connector now supports Browser Based (U2M) and Client Credentials (M2) authentication on GCP cloud.

    On AWS, Azure, and GCP these are the new default values:

    • Authorization endpoint: hostname + “/oidc/oauth2/v2.0/authorize”
    • Token endpoint: hostname+“/oidc/oauth2/v2.0/token”
    • Client ID (U2M): databricks-sql-odbc
    • Scope (U2M): sql offline_access
    • Scope (M2M): sql

    On Azure, if the application specifies a client ID that is different from the default value, the default scope is:

    • For U2M: 2ff814a6-3304-4ab8-85cb-cd0e6f879c1d/user_impersonation, offline_access
    • For M2M: 2ff814a6-3304-4ab8-85cb-cd0e6f879c1d/.default

    You can now configure the OAuth redirect port. To do this, set the OAuth2RedirectUrlPort property to your port. For more information, see the Installation and Configuration Guide.

  • [SPARKO-1302] Updated support for Linux distributions

    The connector now supports the following operating systems:

    • Debian 11
    • Ubuntu 24.04

    For a list of supported Linux versions, see the Installation and Configuration Guide.

2.8.1 ————————————————————————

Released 2024-05-31

Enhancements & New Features

  • [SPARKO-1136] UC Volume ingestion

    The connector now supports UC Volume ingestion operations. To do this, set the StagingAllowedLocalPaths property to the local path. For more information, see the Installation and Configuration Guide.

  • [SPARKO-1264] Updated third-party libraries

    The connector now uses the following third-party libraries:

    • Expat 2.6.2 (previously 2.6.0)
    • libcURL 8.7.1 (previously 8.6.0)

Resolved Issues The following issues have been resolved in Simba Apache Spark ODBC Connector 2.8.1.

  • [SPARKO-988] When the comments’ length is greater than MaxCommentLen, the connector returns an error.

  • [SPARKO-1275] When using a private signed CA on Windows Truststore, the connector fails to verify the OIDC endpoint’s certificate.

  • [SPARKO-1267] On Windows, when using Kerbeos authentication and KrbRealm is empty, the connection fails.

  • [SPARKO-1262] On Windows, when using SASL authentication and the SASL plugins folder path exceeds 128 characters, the connector returns a “Failed to initialize SASL client library: generic failure.” error message.

  • [SPARKO-1254] The connector returns an error for SQLCancel.

2.8.0 ————————————————————————

Released 2024-03-11

Enhancements & New Features

  • [SPARKO-1137] Enhanced client credentials

    The connector now supports JWT assertion as the client credentials for OAuth. For more information, see the Installation and Configuration Guide.

  • [SPARKO-1146] Token renew support

    For token passthrough authentication, you can now renew your token.

  • [SPARKO-1147][SPARKO-1171][SPARKO-1236] Updated third-party libraries

    The connector now uses the following third-party libraries:

    • Arrow 15.0.0 (previously 9.0.0)(on Windows)
    • libcURL 8.6.0 (previously 8.4.0)
    • Zlib 1.3.1 (previously 1.2.13)

  • [SPARKO-1147] Upgraded compiler support

    The connector now uses Windows Visual Studio 2022. For supported versions, see the Installation and Configuration Guide.

  • [SPARKO-1147] Updated security flags

    The following security flags have been used to improve security checks in the connector:

    • /CETCOMPAT
    • /guard:cf
    • /Qspectre
    • /SDL

  • [SPARKO-1165] Accept Undetermined Revocation support

    The connector can now access the SSL connection when the certificate’s revocation status is undetermined. To do this, select the Accept Undetermined Revocation checkbox (set the AcceptUndeterminedRevocation property to 1). For more information, see the Installation and Configuration Guide.

  • [SPARKO-1165] ProxyIgnoreList support

    The connector can now allow the hosts or domains that do not use proxy. To do this, set the ProxyIgnoreList property to a comma separated hostname. For more information, see the Installation and Configuration Guide.

  • [SPARKO-1249] OAuth enhancement

    The connector can now connect to the OIDC endpoint when ProxyUID is unset.

  • [SPARKO-1155] Updated libcURL location

    On Windows, the libcURL DLL and manifest files are now located in the lib folder.

Resolved Issues The following issue has been resolved in Simba Apache Spark ODBC Connector 2.8.0.

  • [SPARKO-1252] When UseNativeQuery is set to 1 for a cluster later than DBR 11, the connector returns an incorrect column number after SQLPrepare.

==============================================================================

Installation and Configuration Guide