Release 436 (11 Jan 2024)#

General#

Delta Lake connector#

  • Add support for querying files with corrupt or incorrect statistics, which can be enabled with the parquet_ignore_statistics catalog session property. (#20228)

  • Improve performance of queries with selective joins on partition columns. (#20261)

  • Reduce the number of requests made to AWS Glue when listing tables, schemas, or functions. (#20189)

  • Fix incorrect results when querying Parquet files containing column indexes when the query has filters on multiple columns. (#20267)

ElasticSearch connector#

Hive connector#

  • Reduce the number of requests made to AWS Glue when listing tables, schemas, or functions. (#20189)

  • Fix failure when reading certain Avro data with Union data types. (#20233)

  • Fix incorrect results when querying Parquet files containing column indexes when the query has filters on multiple columns. (#20267)

Hudi connector#

  • Add support for enforcing that a filter on a partition key must be present in the query. This can be enabled by with the hudi.query-partition-filter-required configuration property or the query_partition_filter_required catalog session property. (#19906)

  • Fix incorrect results when querying Parquet files containing column indexes when the query has filters on multiple columns. (#20267)

Iceberg connector#

  • Add support for querying files with corrupt or incorrect statistics, which can be enabled with the parquet_ignore_statistics catalog session property. (#20228)

  • Improve performance of queries with selective joins on partition columns. (#20212)

  • Reduce the number of requests made to AWS Glue when listing tables, schemas, or functions. (#20189)

  • Fix potential loss of data when running multiple INSERT queries at the same time. (#20092)

  • Fix incorrect results when providing a nonexistent namespace while listing namespaces. (#19980)

  • Fix predicate pushdown not running for Parquet files when columns have been renamed. (#18855)

SQL Server connector#

  • Fix incorrect results for DATETIMEOFFSET values before the year 1400. (#16559)