Polars

⚠️ Deprecations

• Deprecate casts from Categorical to integer dtypes (#28525)
• Deprecate not setting the plan_stage argument in show_graph() (#28391)

🚀 Performance improvements

• Pushdown len() to concat/union inputs (#28570)

✨ Enhancements

• Issue FutureWarning on using from_arrow of ArrowStreamExportable (#28442)
• Add infer_schema_files parameter to scan_csv (#28440)

🐞 Bug fixes

• Properly propagate enum metadata for iceberg column mapping (#28580)
• Correctly write parquet field IDs for enums and categoricals (#28581)
• Fix arrow C interop binview offset (#28623)
• Fix duplicate hive values in hive pre-partition rewrite (#28621)
• Double-counted offset for Arrow export of sliced Series(Array) (#28602)
• Struct/List/Array equality handling in sort and join (#28601)
• More careful slice pushdown into joins (#28578)
• Preserve ordering in sliced unions (#28576)
• Drop input sortedness when casting to a string (#28574)
• Clear sortedness flags in StringChunked substring kernels (#28573)
• Bad mask handling when reading optional parquet column (#28547)
• Any operation on Unknown(Int) and Unknown(Float) should result in Unknown(Float) (#28545)
• Flip nulls_last after Expr.reverse() (#28572)
• Fix high blocking thread use in sink_parquet with async local path (#28543)
• Serialize LazyFrames backed by bytes (#28568)
• Release GIL in SQLContext.execute() (#28549)
• Propagate nulls_last in function_expr_sortedness (#28544)
• Incorrect slicing when a join requires sorting (#28541)

🛠️ Other improvements

• Fix duckdb delta extension install collision (#28607)

Thank you to all our contributors for making this release possible! @BitWeaverDev, @Matt711, @Samoilov2004, @borchero, @dancsi, @dsprenkels, @kdn36, @masumi-ryugo, @nameexhaustion and @ritchie46

🚀 Performance improvements

• Optimize not(bool_f) to not_bool_f (#28474)

✨ Enhancements

• Allow callback sinks on cloud (#28458)

🐞 Bug fixes

• Fix panic on self join of scan_delta / scan_iceberg (#28468)
• Fix UB on first/last_non_null with empty chunk (#28495)
• Do not CSE non-column height expr on streaming engine (#28480)
• Propagate adjusted sortedness through Gather (#28493)
• Deduplicate the expression equality checks (#28483)
• Address 3VL issue with SQL NOT IN interaction with NULL values and joins (#28484)
• Use total comparison in Expr literal values (#28482)
• Ensure SQL SUM and CORR aggregates return NULL for all-null inputs, add TOTAL (#28475)
• Share null_count_dtype helper between Delta and Iceberg, fixing SchemaError (#28479)
• Remove non-output columns from the equi-join and semi/anti-join operators (#28446)
• Fix dropped slice on multiple unions (#28477)
• Fix in-memory engine incorrect slice on maintain order join (#28478)
• Check join schema by position (#28455)
• Block predicate pushdown past overwritten window keys (#28429)

📖 Documentation

• Update and restructure README (#28490)
• Minor fixes to Spark migration guide (#27730)
• Relocate Polars Cloud & On-Prem User Guide (#28462)
• Add notes on k8s operator (#28445)

🛠️ Other improvements

• Make hive_part extraction a function and public (#28507)
• Deduplicate the expression equality checks (#28483)
• Bump ruff and mypy package versions (#28456)
• Fix flaky test (#28454)

Thank you to all our contributors for making this release possible! @AnirudhRahul, @EndPositive, @Jesse-Bakker, @alexander-beedie, @carnarez, @dancsi, @mdroogh, @mroeschke, @nameexhaustion, @nchammas, @orlp, @r-brink and @ritchie46

⚠️ Deprecations

• Deprecate casting numeric types to categoricals (#28349)
• Deprecate cat.get_categories() and cat.to_local() (#28299)
• Deprecate LazyFrame.profile() (#28275)
• Deprecate casts from non-nested dtypes into Lists (#28193)
• Deprecate bitwise ops between integers and booleans (#28186)
• Deprecate list/arr.to_struct() calls that don't pass field names (#28243)
• Rename missing_utf8_is_empty_string to empty_string_is_null (#28173)

🚀 Performance improvements

• Split multiplexers that directly scan from in-memory DataFrame (#28376)
• Make DNS cache global (#28352)
• Do not remove cache if predicates not pushed to all inputs (#28341)
• Environment variable for logging slow DNS lookup (#28211)
• Pre-partition on left, right and semi joins on hive partitioned data (#28374)
• Rewrite inner joins on hive partitioned data to a union of partitions filtered joins (#28327)
• Tighten redundant comparisons and propagate filter constraints (#27999)
• Canonicalize predicates (#28212)
• Remove excess memory copy in streaming IPC sink (#28141)
• Parallelize scan_iceberg resolve (#28168)
• Chunk-aware morsel splitting on single-file IO sink (#27950)
• Don't allocate for full_null() with small lengths (#28181)
• Use monotonic deque for rolling min_by/max_by (O(n)) (#27436)

✨ Enhancements

• Pre-partition group-by on hive keys (#28444)
• Add ewm_sum and ewm_sum_by #28151 (#28215)
• Serve stale records from object_store DNS cache (#28256)
• Make partition_hive an opt flag (#28345)
• Add scan_arrow_c_stream (#28340)
• When displaying IRExprs, only add parentheses when needed for disambiguation (#28319)
• Support Iceberg field initial-default in native scan_iceberg (#28324)
• Show only differing dtypes in assert_frame_equal error (#27816)
• Expose ewm and rolling *_by expressions in Python visitor (#28286)
• More accurate rolling moments (#28309)
• Remove hard block on native scan for Iceberg V3 tables (#28277)
• Add Series.degrees/radians and clarify trig functions use radians (#22961)
• Merge sorted with multikey (#28067)
• Add list expression which consistently packs elements together into new List type (#27990)
• Add read_database Arrow fast-path for "python-oracledb" (#28230)
• Optimize joins with redundant equi-join keys (#28017)
• Implement missing datetime options for CastColumnsPolicy (#28084)
• Add Expr.cat.to and Expr.cat.physical (#28096)
• Add POLARS_OOC_DISK_BUDGET_MB (#28045)
• Add build side preference parameter for join (#28154)

🐞 Bug fixes

• Avoid panic when union slice skips all rows (#28420)
• Solve panic in dt.replace when there were multiple chunks (#28437)
• Propagate is_scalar from the input to the output of .sort() and .sort_by() (#28438)
• Resolve CSV column names overwrite in DSL->IR conversion (#28383)
• Avoid IEJoin rewrite for Categorical comparisons (#28427)
• Do not rewrite sort().reverse() to sort(descending=True) when maintain_order=True (#28403)
• Ensure BinaryView offset+len does not exceed i32::MAX where possible (#28048)
• Invalid offset in strptime (#28388)
• Incorrect schema type for decimal <-> primitive division (#28373)
• Panic in in-memory CSEE handling (#28371)
• Fix offset in arrow ffi export of sliced struct arrays (#28369)
• Fix cross filter not applied with sink and CSE (#28297)
• Fix panic on projection pushdown with caches (#28280)
• Fix write_json() null values in Array columns being written incorrectly as null (#28330)
• Fix EntityTooSmall on sink_ipc to S3 (#28255)
• Raise error instead of silent wrapping for select(len()) (#28355)
• Float16 groupby aggregates (#28361)
• Respect lexical ordering of Categorical in top_k/bottom_k (#28359)
• Support dtype-list schema overrides in scan_csv (#28226)
• Resolve unknown type in fused multiply add (#28321)
• Honor aws_checksum_algorithm when provided (#28328)
• Remove usage of deprecated pyarrow.feather.read_table (#28323)
• Fix regression in sourcing HF_TOKEN (#28317)
• Raise on addition/subtraction between temporal and non-temporal Series (#28266)
• Avoid panic constructing Series from unaligned numpy arrays (#28120)
• Rewrite with_columns on empty df to select during DSL -> IR (#28298)
• Fix sink deadlock regression (#28296)
• Avoid panic when consumer of vertical concat is done (#28272)
• Harden IPC mmap API (#28258)
• Classify S3 region request failures as IO errors (#28225)
• Incorrect inner dtype in ArrayChunked::from_aligned_values (#28249)
• Raise on implode of Object dtype instead of creating invalid List(Object) (#28246)
• Return null not 0 for the first unbiased ewm_var/ewm_std value (#28235)
• Database appends with ADBC shouldn't require "CREATE" privileges unless the table is confirmed not to exist (#28228)
• Avoid slice pushdown panic on shared cache inputs (#28157)
• Incorrect dtype for Unknown right literal in arithmetic (#28184)
• Remove unnecessary unsafe Python object downcast (#28189)
• Normalize non-native byte order in numpy array construction (#28197)
• Use-after-free in numpy ufunc FFI (#28195)
• Unpickling for pl.when().then() (#28165)
• Reduce unknown casts (#28145)
• Consider a rolling_rank position invalid if ost.len() < min_samples (#28142)
• Incorrect broadcast with empty len in pl.concat_list (#28134)

📖 Documentation

• Update comparison page (#28418)
• Update GPU support documentation with the cudf-polars 26.06 release (#27830)
• Fix dev docs being canonical for SEO (#28306)
• Document experimental HDFS support on-premises (#28115)
• Add Series.degrees/radians and clarify trig functions use radians (#22961)
• Add polars-bio to community plugins (#28103)
• Add AWS Marketplace walkthrough to Cloud User Guide (#28214)
• Correct merge_sorted() docs also for DataFrame (#28183)
• Clarify DataFrame.remove combines predicates with AND (#28171)
• Update config reference (#28150)
• Correct that merge_sorted() null inputs should be nulls first (#28177)
• Clarify parts of the Polars Cloud and On-prem user guide (#28170)
• Add On-Premise release 0.6.3 to releases page (#28175)
• Update PolarsCloud snippets for 0.9.0 release (#28166)
• Fix to_titlecase incorrectly claiming alphanumeric separation (#28136)

📦 Build system

• Set the default dev profile to line-tables-only (#28358)
• Reduce binary size by optimizing for size in the SQL transpilation (#28336)

🛠️ Other improvements

• Update AI policy for comments (#28436)
• Fix flaky ordering expectation in top_k_by (#28435)
• Ignore all deprecation warnings in Python doctest (#28398)
• Re-work shuffle parameter for sample() (#27460)
• Ignore deprecation warnings on to_struct() in doctest (#28370)
• Update spin dependency to v0.10.1 (#28360)
• Bloat dispatch reduction (#28337)
• Show only differing dtypes in assert_frame_equal error (#27816)
• Accept IR in optimize (#28318)
• Annotate untyped module constants (#27907)
• Remove unencountered deny entries (#28260)
• Bump crossbeam-epoch dependency (#28259)
• Change object_store custom DNS resolver patch to rev-based (#28241)
• Mark shape tests as passing (#28133)
• Attribute annotations for *GroupBy (#27903)
• Clarify AI policy, explicitly forbid agents from interacting with our repository (#28206)
• Tighten first-time-contributor screenshot requirements (#28207)
• Covariant classinstmethod and attribute annotation (#27904)
• Ignore quickxml DoS rustsec (#28185)
• Update typos to v1.48.0 (#28162)

Thank you to all our contributors for making this release possible! @0guban0v, @AnirudhRahul, @Bharath-970, @CodingSelim, @EndPositive, @Jesse-Bakker, @Kevin-Patyk, @LarryHu0217, @MarcoGorelli, @MatteoPossamai, @TNieuwdorp, @alex-h-sun, @alexander-beedie, @azimafroozeh, @borchero, @dancsi, @dependabot[bot], @dsprenkels, @harrywhalen, @jorenham, @kdn36, @makinzm, @malhotrashivam, @mdavis-xyz, @mikhail5555, @mkzung, @mroeschke, @nameexhaustion, @orlp, @r-brink, @raphaelroshan, @ritchie46, @tylerriccio33, @useredsa, @wence-, @yannbolliger, @zichen0116 and dependabot[bot]

⚠️ Deprecations

• Deprecate strict parameter of pl.concat, replace with new how='horizontal_extend' (#27965)

🚀 Performance improvements

• Add a sampled resolve mode for multi-file parquet metadata (#28111)
• Speed up small dtype series sums with a upcast sum kernel (#27958)
• Don't block on path expansion (#28073)

✨ Enhancements

• Close ResourceWarning leaks in database tests (#28066)

🐞 Bug fixes

• Incorrect sorted fast-path returning nan for grouped max (#28129)
• Fix panic on scan_parquet filter of fixed-size binary column (#28122)
• Fix decimal dynamic float cast (#28126)
• Fix pyiceberg key dot HDFS prefix filter (#28109)
• Fix projection pushdown panic on select(len()) after groupby (#28108)
• Fully resolve remaining ResourceWarning leaks in database/iceberg tests (#28107)
• Expiry time loading for AWS DeferredRefreshableCredentials (#28099)
• Close ResourceWarning leaks in database tests (#28066)
• Avoid panic on temporal extraction for datetime columns with nulls (#28054)
• Prevent panic in replace when old/new contain Expr or object dtype values (#27433)

📖 Documentation

• Fix dataset regression in the examples (#28131)
• Emphasize to generally not ever write Rust tests (#28082)
• Mention Polars skill in User Guide's LLM section (#28113)

🛠️ Other improvements

• Disallow usage of tokio::spawn in clippy (#28123)
• Improve Makefile with respect to venv robustness (#28110)
• Remove legacy prelude glob imports from leaf files (#28114)
• Skip pandas 3.0.4 due to pd.TimeDelta segfault (#28125)
• Update mypy to the new 2.x release (#28116)
• Ensure pyrefly checks run as part of make pre-commit (#28095)
• Fix test_select_explode_height_filter_order_by failure when POLARS_MAX_THREADS=2 (#28090)
• Add Morsel::height (#28060)

Thank you to all our contributors for making this release possible! @0guban0v, @Kevin-Patyk, @TNieuwdorp, @alexander-beedie, @azimafroozeh, @dsprenkels, @kdn36, @nameexhaustion, @orlp, @raphaelroshan, @ritchie46 and @sar-cheng

⚠️ Deprecations

• Deprecate casts from string to temporal dtypes (#28056)

🚀 Performance improvements

• Add bytes-based concurrency control for cloud IO (#27924)
• Stop flushing multiplexer on phase change (#28013)
• Prevent empty chunks from triggering copy in rechunk (#27953)
• Improve performance of struct unnest inside list eval on streaming (#27905)
• Eliminate filters with contradictory predicates (#27775)

✨ Enhancements

• Deprecate casts from string to temporal dtypes (#28056)
• Add naive out-of-core spilling to Polars (#27998)
• Add experimental strict mode (#28023)
• Optimize non-negative len and null_count comparisons (#28002)
• Optimize len and null_count comparisons (#27727)
• Support SQL's implicit JOIN syntax (#27890)
• Add Expr.is_sorted (#26708)
• Add suggestions in wrong api usage or mismatched column names (#27954)
• Add external object_store support for non-native schemes (#27871)
• Skip Delta files and parquet row groups using per-field struct statistics (#27887)
• Improve IR sortedness detection for with_row_index and explode (#27875)
• Add DataFrame.is_sorted() (#27870)
• Expose fixed-size rolling window expressions in Python visitor (#27108)
• Fix SchemaError using lazy HConcat->Sink (#27770)
• Add tiered multi-file parquet metadata resolver (#27720)
• Cache and shuffle DNS for cloud object_store (#27659)

🐞 Bug fixes

• Add a deprecation warning for .explode() without empty_as_null argument (#28040)
• Fix select(len()) incorrectly returned 0 when using scan_iceberg with pyiceberg as reader override (#28044)
• Ensure slice expr always observes input order (#28041)
• Fix projection pushdown with row index out of order (#28043)
• Fix cross-join predicates not applied when using sink_* functions (#28042)
• Grammar corrections (recommend to → recommend, allows to → allows using) (#28038)
• Ensure optimisations aren't omitted by cached IR nodes (#28004)
• Computed GROUP keys in SQL projections (#27989)
• Handle empty DataFrames in dt.truncate and dt.round (#26120)
• Address join alias state leakage in SQL CTEs (#27985)
• Support scanning file:// URIs with percent-encoded paths (#27876)
• Lower SQL [NOT] IN (subquery) to semi/anti join (#27888)
• Check for continuity across morsels in AsOf join node (#27461)
• Add test for projection pushdown with hconcat regression (#27937)
• Traverse caches only once in explain (#27939)
• Preserve null placement when collapsing sort.reverse() into single sort (#27918)
• Incorrect schema order for hstack in projection pushdown optimizer (#27920)
• Fix duplicated map_elements warning on streaming engine (#27919)
• Use block_in_place_on for calls which can come from executor thread (#27855)
• Mismatch in max_threads -> pipeline configuration (#27854)
• Keep maintain_order on sliced unique (#27852)
• Fix SchemaError using lazy HConcat->Sink (#27770)
• Fix incorrect projection height when selecting only literals (#27825)
• Select with expr slice and len gave incorrect len (#27824)
• Prevent import panic when environment variable set to unexpected value (#27831)

📖 Documentation

• New On-Prem release (#27979)
• Add note on OOM behavior on Kubernetes (#27972)
• Properly render backslashes in CSV eol_char docstrings (#27962)
• Add entry for DataFrame.is_sorted() (#27913)
• Clarify compat_level export docs (#27715)
• Add robots.txt disallowing stale docs and added sitemap for better SEO (#27901)
• Update Polars <> Spark comparison (#27873)
• Bump to patched version (#27851)
• Replace Typeform sign-up URL with new enterprise link (#27838)
• Correct wrong head call (#27848)
• Add Polars On-Prem 0.5.0 release (#27849)
• Correct onprem license helm values (#27847)
• Update connecting Polars Cloud to AWS documentation (#27823)
• Correct cost/connectivity sections (#27813)
• Add sections about egress (#27805)
• Tragic typo in On-Prem docs (#27804)

🛠️ Other improvements

• Add AI agent disclsoure (#28064)
• Bump memmap2 version (#28050)
• Update sqlparser dependency (#28021)
• Update pyo3 and rust-numpy to 0.29.0 (#27970)
• Add 2.0 branch as primary branch (#27978)
• Patch jemalloc-sys for Makefile bug (#27945)
• Restore Cargo.lock (#27964)
• Add test for projection pushdown with hconcat regression (#27937)
• Update deny.toml configuration (#27949)
• Doctest skip refactor and add hugging face rate limited tests (#27885)
• Run Pyrefly type-checker on source code (#27722)
• Bump Rust dependencies (#27874)
• Changes needed for Rust 0.54.x (#27853)
• Use Vec instead of PlHashMap for ProjectionInfo.map (#27856)
• Reduce codegen-units (#27835)
• Refactor to avoid unbound-name Pyright/Pyrefly errors (#27827)

Thank you to all our contributors for making this release possible! @0guban0v, @April-Sonnet, @BitWeaverDev, @EndPositive, @JakubValtar, @Kevin-Patyk, @Liyixin95, @MarcoGorelli, @Matt711, @TNieuwdorp, @alexander-beedie, @ankane, @azimafroozeh, @cBournhonesque, @carnarez, @dsprenkels, @gautamvarmadatla, @kdn36, @nameexhaustion, @orlp, @ritchie46, @tolleybot, @toreerdmann, @uurl and @xixixao

🚀 Performance improvements

• Update to new jemalloc (#27797)
• Do not materialize ScalarColumn in Column split_at (#27782)
• Avoid materializing broadcast in array.shift (#27740)
• Avoid materializing broadcast list in list.sample(n) and list.sample(frac) (#27679)

✨ Enhancements

• Update to new jemalloc (#27797)

🐞 Bug fixes

• Broken link to AI Policy corrected (#27793)
• Update to new jemalloc (#27797)

📖 Documentation

• Update Polars On-Prem version stamp (#27799)
• Broken link to AI Policy corrected (#27793)
• Add release dates to the On-Prem releases page (#27787)
• Improve on-prem docs (#27788)
• Add query profiler video to On-Prem user guide (#27786)
• Add EKS/AKS/GKE guides (#27774)
• Add Polars On-Prem 0.4.2 (#27780)

🛠️ Other improvements

• Run Pyrefly on _utils and functions (#27789)
• Harden against async blocking deadlocks (take 2) (#27767)

Thank you to all our contributors for making this release possible! @ButteryPaws, @EndPositive, @Kevin-Patyk, @MarcoGorelli, @TNieuwdorp, @azimafroozeh, @carnarez, @kdn36, @lun3x, @orlp and @ritchie46

🚀 Performance improvements

• Adaptive size dispatch to hashset or radix sort + capacity-aware reset in agg_n_unique (#27719)

✨ Enhancements

• Allow deeper expressions (#27768)

🐞 Bug fixes

• Raise length mismatch in multiple sort_by in group_by (#27772)
• Respect min_samples for rolling_by ops with nulls (#27706)
• Fix memory usage regression affecting TPCH Q22 (#27758)
• Add POLARS_ALLOW_NESTED_CSPE env var and make nested CSPE opt-in (#27765)
• Post-apply residual pyarrow predicates (#27764)
• Fix loss of precision for smaller floating types(#27662) (#27732)
• Filter at scan dropped in CSPE filter pushdown (#27763)
• Fix portstate assertion error on is_in (#27757)
• Fix incorrect when/then after forward fill / reverse in groupby (#27745)
• Accept empty Thrift list encoded as bare 0x00 byte in parquet metadata (#27754)
• Stabilize object store credentialprovider cache key (#27712)
• Add to merge_sorted docs that the input must be nulls first (#27743)

📖 Documentation

• Docs fixes (#27766)
• Sync from Polars Cloud (#27751)
• Add to merge_sorted docs that the input must be nulls first (#27743)

🛠️ Other improvements

• Attribute annotations for CatalogCredentialProvider (#27739)
• Solve type: ignore in _AioDataFrameResult (#27311)
• Remove dead code in _write_utils.py (#27721)
• Remove unnecessary not isinstance(v, DataType) check (#27723)

Thank you to all our contributors for making this release possible! @EndPositive, @JakubValtar, @MarcoGorelli, @NicoOhR, @azimafroozeh, @carnarez, @dsprenkels, @jorenham, @kdn36, @nameexhaustion, @orlp and @ritchie46

🏆 Highlights

• Add LazyFrame.gather (#27501)
• Nested common subplan elimination (#27340)
• Stabilize streaming engine (#27497)
• Speed up parquet metadata decode with hand-written Thrift (#27427)

⚠️ Deprecations

• Deprecate the StringCache (#27580)

🚀 Performance improvements

• Dispatch {list,arr}.{unique,n_unique,reverse} to group_by engine (#27278)
• Improve in-memory grouped non-null count (#27702)
• Factor shared conjuncts out of OR-of-ANDs predicates (#27627)
• Skip downloading IPC batches exceeding slice bounds (#27683)
• Avoid materializing broadcast list in list.shift (#27628)
• Optimise json_decode Datetime string parsing (#27559)
• Speed up to_numpy C-order via cache-blocked transpose (#27522)
• Optimize select(len()) for non-strict horizontal concat (#27516)
• Pushdown slices to inputs on left/right/full join (#27508)
• Don't infer CSV schema if schema is set (#27507)
• Nested common subplan elimination (#27340)
• Make is_in row-group pruning precise on null-containing haystacks (#27495)
• Don't do fused-multiply-add on scalars (#27479)
• List full fast path (#27477)
• Make is_in row-group pruning precise on multi-value lists (#27475)
• Add streaming GatherNode (#27465)
• Lower non-elementwise FunctionExprIR to ColumnarFunctionNode (#27462)
• Speed up parquet metadata decode with hand-written Thrift (#27427)

✨ Enhancements

• Use true division for the / operator in Polars SQL (#27391)
• Add Rust backend for Expr.has_nulls (#27590)
• Stabilize float16 (#27607)
• Add Expr.is_empty (#27583)
• Add support for the SQL FILTER clause for aggregate functions, and STRING_AGG (#27564)
• Make parquet FileMetadata prunable for IR-plan dispatch (#27535)
• Broadcast scalar input for list.slice (#27487)
• Add LazyFrame.gather (#27501)
• Add null_on_oob in {Expr/Series}.gather (#27327)
• Stabilize streaming engine (#27497)
• Process batched arr.eval on overflow boundaries (#27496)
• Process batched list.eval on overflow boundaries (#27483)
• Print SLICED UNION in LazyFrame explain (#27467)

🐞 Bug fixes

• Panic in scan of empty IPC with slice (#27708)
• Persist object_store rebuild state in cache (#27707)
• Sort flag on GroupsType only applies to first element (#27684)
• Invalid unwrap_unchecked when length isn't exact (#27685)
• Don't unwrap channel send in streaming join_asof (#27688)
• Fix merge_sorted panic when List in frame (#27568)
• Put AsOf join buffered Morsels back the front of the deque if we cannot process them rn (#27658)
• Fix skip_batches logic for NaN (#27673)
• Raise TypeError when calling next() directly on GroupBy objects (#27562)
• Data type comparison for extension types (#27632)
• Share last-morsel split budget across files in streaming multi-scan (#27630)
• Bytes scalars were not being broadcast in dataframe constructor (#27621)
• Reset the sort-options in Series::is_sorted() after row-encoding columns (#27614)
• Rayon deadlock with re-entrant io sources (#27600)
• Don't push negative-offset slices through HConcat (#27570)
• Logic error in streaming is_empty (#27602)
• Fix incorrect CSE with large is_in literal (#27575)
• AnonymousFunction can qualify as SQL aggregator (#26986)
• Fix CSPE panic in cloud (#27594)
• Set merge-join streaming node to Finished if its sending port is Done (#27572)
• Widen decimal precision on sum aggregation at runtime (#27579)
• Fix str.to_time was raising unnecessarily when input was all nulls (#27574)
• Prevent panic when switching from one extension dtype to another (#27566)
• Fix DataFrame.write_database(..., if_table_exists="append", engine="adbc") not handling missing tables correctly (#26913)
• Ensure json_decode doesn't fail for Date and Time string deserialization (#27554)
• Incorrect RUSTFLAGS passing in Makefile (#27555)
• Fix panic on reading IPC with 0-row compressed bitmap (#27551)
• Set HEAD_RESPONSE_SIZE_ESTIMATE to 0 (#27548)
• Fix lazy concat horizontal didn't raise on mismatching heights after projection pushdown (#27506)
• Prevent join panic when suffix="" and coalesce=True (#27376)
• Do not make a FastCount for csv if pre_slice is set (#27536)
• Support duplicate names in over (#27544)
• Reassign sequence numbers when distributing input morsels in streaming AsOf join node (#27538)
• Do not reverse dataframes when sorting with all-null key columns (#27517)
• Incorrect length check on streaming zip (#27505)
• Remove invalid type annotation Sequence[int] from DataFrame.\_\_setitem\_\_ key (#27355)
• Respect nulls_last for descending over(order_by) in group_by().agg() (#27486)
• Fix perf regression in scan_csv select(len()) when collected on streaming engine (#27504)
• Harden extend strictness (#27476)
• Prevent deadlock when using to_arrow() in a multithreaded context (#27472)
• Do not flatten sliced union (#27466)
• Prevent deadlock when using to_pandas() in multithreaded context (#27451)
• Struct rechunk bug and add Series::with_validity (#27446)
• Handle column indexing in read_parquet/read_csv with pyarrow reader (#27397)
• Export enum as ordered dictionary to arrow (#27432)
• Ensure sample() respects shuffle=False (#27248)
• Return empty DataFrame from concat_list with lit and empty column (#27305)
• Read parquet MAP columns without LogicalType annotation (#27404)
• Raise DuplicateError on parquet files with duplicate column names (#27399)

📖 Documentation

• Document Expr.list.__getitem__ (#27689)
• Add cloudpickle requirement (#27703)
• Clarify from_arrow schema ordering (#27493)
• Fix a typo in join_asof docstring (#27682)
• Clarify schema column order (#27681)
• Document horizontal string concatenation (#27542)
• Document all valid engine options on LazyFrame collect/sink/explain methods (#27374)
• Orchestration docs check (#27605)
• Drop redundant Pattern 2 from Dagster integration page (#27581)
• Update to remove Dockerhub PAT references (#27582)
• Modernize Dagster integration example for Polars Cloud (#27560)
• Use Polars random seed in sample example (#27537)
• Clarify full join description (#27530)
• Make expressions operations RNG deterministic (#27494)
• Document struct field order (#27492)
• Improve over:order_by description (#27520)
• Clarify join output columns (#27449)
• Document null propagation in pl.format (#27447)
• Document gzip support in read_csv (#27434)
• Add See Also sections for datetime docstrings (#27316)
• Polars On-Prem release (#27439)
• Rename to Polars On-Prem (#27435)
• Clarify null handling in unique operations (#27431)
• Document write_ipc buffer behavior with file=None (#27430)

📦 Build system

• Also split debug info in debug-release (#27609)
• Use split-debuginfo on linux (#27608)
• Bump deltalake to 1.5.1 in CI (#27387)

🛠️ Other improvements

• Remove redundant DSL::AGG::Unique (#27718)
• Harden against async blocking deadlocks (#27653)
• Print Python traceback when POLARS_TIMEOUT_MS is exceeded (#27657)
• Format missed in previous PR (#27700)
• Bump pytest and remove codspeed (#27686)
• Remove client-side allow_local_scans option for prepare_cloud_plan (#27663)
• Remove superfluous test (#27676)
• Cleanup streaming flags (#27671)
• Expose unordered concatenation in python visitor (#27666)
• Bump deltalake and fix CI (#27660)
• Add impl IntoAExprBuilder for ExprIR (#27656)
• Split _expand_selector_dicts into multiple functions so return type is simple and accurate (#27618)
• Update object_store patch repo (#27650)
• Match NumPy signature in DataFrame.__array__ and Series.__array__ (#27634)
• Add ImageVersion to rust-cache key (#27626)
• Run Pyrefly on tests (#27459)
• Fix is_empty test (#27597)
• Fix tz type difference pandas assert, take 2 (#27596)
• Fix CSPE panic in cloud (#27594)
• Fix tz type difference pandas assert (#27593)
• Add contributing note about conventional comments (#27543)
• Nested common subplan elimination (#27340)
• Deduplicate interns (#27470)
• Fix merge conflict in ColumnarFunction (#27464)
• Keep the schema ordered in scan projection pushdown (#27429)
• Remove unused type: ignore statements (#27360)
• Remove redundant PhysNodeKind::AsOfJoin::{left_right}_by fields (#27400)
• Resolve type-ignores in udfs.py (#27341)
• Bump rustls-webpki (#27382)

Thank you to all our contributors for making this release possible! @0guban0v, @EndPositive, @JakubValtar, @Jesse-Bakker, @Kevin-Patyk, @Liyixin95, @MarcoGorelli, @NedJWestern, @Shoeboxam, @SuryaSunil1326, @TNieuwdorp, @alexander-beedie, @aryansri05, @ashler-herrick, @azimafroozeh, @carnarez, @coastalwhite, @dependabot[bot], @dsprenkels, @gab23r, @gautamvarmadatla, @ilya-pevzner, @jonathansergio, @junnythemarksman, @kdn36, @lun3x, @nameexhaustion, @orlp, @pablogsal, @ritchie46, @uurl, @waamm, @wence-, @wmoss, @xronocode and dependabot[bot]

🚀 Performance improvements

• Skip validity mask processing in __array_ufunc__ when no inputs have nulls (#27358)

✨ Enhancements

• Cargo deny (#27363)
• Add maintain_order parameter to merge_sorted (#27263)

🐞 Bug fixes

• Honor having predicate in GroupBy iter (#27370)
• Use the physical dtype for NumUnorderedImplodeReducer arrow ListArray (#27375)
• Address bug in reduce_balanced for certain input length lists affecting pl.concat (#27352)
• Ensure list.sample() allows fraction > 1 when with_replacement=True (#27350)
• Ensure append() errors when upcast=False (#27346)
• Always rechunk sorts, prune sorts even in eager execution (#27356)
• Fix typing for DataFrame.__init__ and Series.__init__ so they don't require all optional dependencies to be installed (#27348)

📖 Documentation

• Split out openlineage docs into guide and configuration (#27371)
• Add explanation on the observatory sqlite db file (#27354)

🛠️ Other improvements

• Disable mypy type checking for pyarrow calls (#27377)
• Disable debug symbols in macos coverage tests (#27361)
• Cargo deny (#27363)

Thank you to all our contributors for making this release possible! @EndPositive, @Kevin-Patyk, @MarcoGorelli, @carnarez, @dsprenkels, @gab23r, @jonathanchang31, @kdn36, @mzjp2 and @ritchie46

🏆 Highlights

• Add streaming support for grouped AsOf join (#27293)

⚠️ Deprecations

• Deprecate support for dataframe interchange protocol (#27214)

🚀 Performance improvements

• Create IR slice from expr slice pushdown (#27200)
• Add streaming support for grouped AsOf join (#27293)
• Avoid unnecessary rechunk when sorting already sorted DataFrame (#27264)
• Lower basic over() to streaming primitives (#27303)
• Lower drop_{nulls,nans} in streaming group_by aggregations (#27296)
• Lower entropy to streaming reductions (#27174)
• Add native streaming interpolate (#27185)
• Streaming strptime with format=None (#27056)
• Lower skew / kurtosis to streaming aggregations (#27176)
• Post apply pyarrow filter in Polars' engine instead of pyarrow (#27192)
• Optimize drop_nulls().{first,last}() to {first,last}(ignore_nulls=True) (#27187)
• Always process pyarrow scan in batches (#27183)
• Make cut output Enum and mark as elementwise (#27173)
• Remove unused expression sorts (#27075)
• Use delta stats for mixed hive and non-hive predicate pushdown (#27102)
• Take into account size per row in join sampling (#27098)
• Streaming is_first_distinct and unique(maintain_order=True) (#27052)
• Streaming cov and corr (#27008)
• Add sorted unique node to streaming engine (#26990)
• Ensure Expr.append is lowered in streaming engine (#27022)
• Collapse consecutive Sort nodes (#26965)
• Drop maintain_order=True requirement in sink_delta (#27007)

✨ Enhancements

• Add ignore_nulls to {list,arr}.{any,all} (#27186)
• Lock-free memory manager with spill-to-disk and fully OOC multiplexer (#26774)
• Add is_unique to list/array dtypes (#27290)
• Streaming pyarrow datasets sources (#27230)
• Add pl.merge_sorted operating on multiple frames (#27014)
• Allow group_by() without key exprs (#27141)
• Change default scan/read_lines column name from "lines" to "line" (#27122)
• Make unnest() effective on all columns by default (#27029)
• Collapse consecutive Sort nodes (#26965)

🐞 Bug fixes

• Update groups to correct length for Implode (#27282)
• Fix scan_csv missing_columns='insert' overwrote existing data with NULLs (#27297)
• Raise on non-numeric inputs in pl.int_ranges (#27294)
• Fix always-true filter conversion to Iceberg filter (#27119)
• Do not skip nulls when enumerating over rows in grouped AsOf join (#27275)
• Fix pivot dropping data for null on values (#27273)
• Resolve multiple files deadlock in CSV async reader (#27073)
• Widen decimal precision on sum aggregation (#27270)
• Correct lf.remote type (#27261)
• Default LazyFrame.map_batches to no optimizations (#27262)
• Extend StructEval schema context in StackOptimizer (#27243)
• Preserve nulls when casting from all-null Series to Struct (#27241)
• Fix scan_delta filter on empty dataframe (#27244)
• Prevent DataFrame creation panic on list[struct] with heterogenous types (#27217)
• Named aggregation __structify was being ignored (#27148)
• Skip null group entries when collecting AsOf-by groups (#27215)
• Fix panic with empty order_by in over expression (#27088)
• Write field ID from sink_parquet (#27196)
• Fix statistics for Null columns in Parquet (#27021)
• Do not prune sort nodes containing slice with dyn predicate (#27140)
• Correct grouped Binary arg_min/arg_max and String single-element arg indices (#27172)
• Resolve multiple files deadlock in NDJSON async reader (#27204)
• Overflow panic in interpolate nearest (#27205)
• Using checked arithmetic in int96_to_i64_ns to prevent overflow panic (#27129)
• Don't trigger csv fast count if predicate is pushed down (#27190)
• Support all integer dtypes for Series index assignment (#27188)
• Streaming sort by-expressions were lowered incorrectly (#27158)
• Replace multiprocessing.dummy.Pool with ThreadPoolExecutor (#27175)
• Reset IO metrics instead of consuming (#27156)
• Output SVG if output_path ends with '.svg' in show_graph (#27144)
• Skip extension types for min/max in describe (#27120)
• Address a potential overflow in from_epoch scaling (#27118)
• Fix incorrect IO metrics on multi-phase streaming execution (#27123)
• Use delta stats for mixed hive and non-hive predicate pushdown (#27102)
• Make the files used in docs available locally (#27121)
• Apply scalar bound in clip when the Series bound contains nulls (#27087)
• Ignore ddof parameter in rolling_corr and deprecate (#27104)
• Preserve casts for horizontal ops with untyped literals (#27011)
• Reject invalid input to sql_expr (#27084)
• Ensure SQL COUNT(<lit>) expressions return the correct value (#27085)
• Regression in replace_strict for enums (#27066)
• Make test_group_by_arg_max_boolean_26978 non-flaky for max_by ties (#27048)
• Null count for aggregated list inside count aggregation (#27032)
• Panic in streaming MergeSortedNode (#27024)
• Prevent panic in transpose() with mixed List and non-List columns (#27038)
• Set sorted flag for Boolean and Time (#27035)
• Missing src/ subdirectory to CI Python docs step (#27025)
• Resolve stack overflow on merge_sorted and union (#27018)
• Make pl.DataFrame.fill_null work on columns with Null dtype (#27020)
• Fix repeated word typos in comments (#26917)
• Covariance with constant is zero, not NaN (#27015)
• Don't remove set_sorted in projection pushdown (#27006)
• Infer nulls when df create from empty-struct (#26991)
• Correct suggestion in multi-expr filter error (#27003)
• Implement agg_arg_min/agg_arg_max for boolean data type (#26997)
• Ensure sample() respects the global set seed (#26992)

📖 Documentation

• Add documentation for openlineage on-premises (#27334)
• Release page (#27335)
• Update uv pip install polars-on-premises cmd (#27330)
• Fix outdated LazyGroupBy.map_groups docstring (#27292)
• Add deny_anonymous_users to scheduler config (#27287)
• Slurm documentation (#27259)
• Add link to concepts in index.md (#27077)
• Add docs entry for merge_sorted (#27224)
• Fix typo (#27212)
• Make the files used in docs available locally (#27121)
• Put first-time contribution requirements in its own linkable section (#27113)
• Add missing docstrings for Expr.struct.__getitem__ and Series.__setitem__ (#27092)
• Normalise Series docstring whitespace indents (#27082)
• Change Polars Cloud API to 0.6.0 (#27005)
• Improve write_parquet docstring for use_pyarrow (#26988)

📦 Build system

• Really do not install pyiceberg-core 0.9.0 (#27017)

🛠️ Other improvements

• Add regression test for instantiating polars DataFrame from pandas Timestamp (#27332)
• Bump Python Polars version (#27315)
• Resolve bad instantiations in test_iceberg (#27314)
• Sink DSL and callback for Iceberg (#27258)
• Wait for morsel consumption in merge_sorted streaming node (#27288)
• Use more precise internal typing (pt. iii) (#27232)
• Mark scan_ipc cache arguments as deprecated (#27216)
• Consolidate reordered compare functions (#27229)
• Fix test_dtype_concat_3735 not actually iterating through numeric dtypes (#27178)
• Remove dead code in test_scan_lines (#27213)
• Move/genericize _balanced_reduce to Python utils (#27100)
• Remove unused attributes (#27191)
• Avoid unnecessary recompilation due to changing env vars (#27166)
• Update nightly Rust compiler version (#27145)
• Simplify pyarrow scan and process in batches (#26982)
• Make internal typing more precise (part ii) (#27117)
• Add None & Dataframe to FrameInitTypes (#27126)
• Remove unused expression sorts (#27075)
• Improve internal typing ahead of using ty / pyrefly (#27050)
• Add explicit ResourceWarning coverage (#27083)
• Add sinked paths callback (#26995)
• Pin maturin due to compile time regression (#27062)
• Missing src/ subdirectory to CI Python docs step (#27025)
• Really do not install pyiceberg-core 0.9.0 (#27017)
• Naming for named scopes (#26999)
• Enable hypothesis tests when POLARS_AUTO_NEW_STREAMING=1 (#26818)
• Fix CI by excluding missing wheel version of pyiceberg (#27001)
• Remove indirection in calling python scans (#26981)
• Polars versions (#26980)

Thank you to all our contributors for making this release possible! @0xRozier, @EndPositive, @HCYT, @Kevin-Patyk, @MarcoGorelli, @NeejWeej, @RedZapdos123, @TNieuwdorp, @abhidotsh, @alexander-beedie, @andyjessen, @azimafroozeh, @borchero, @carnarez, @coastalwhite, @debnathshoham, @dpinol, @dsprenkels, @dydev012, @farouk-01, @gab23r, @gautamvarmadatla, @joaquinhuigomez, @kdn36, @nameexhaustion, @orlp, @ritchie46, @wence-, @xenzh, @yangsong97 and @yonatan-genai

• No changes

Thank you to all our contributors for making this release possible! @ritchie46

• No changes

Thank you to all our contributors for making this release possible! @nameexhaustion and @ritchie46

🐞 Bug fixes

• Handle empty rolling windows in streaming engine (#26903)

📖 Documentation

• Add documentation for on_columns for LazyFrame pivot (#26859)

🛠️ Other improvements

• Bump build deps used in ARM64 Windows release pipeline (#26892)

Thank you to all our contributors for making this release possible! @Kevin-Patyk, @RenzoMXD, @TNieuwdorp, @dsprenkels, @gautamvarmadatla, @nameexhaustion, @nicholaslegrand102 and @ritchie46

🚀 Performance improvements

• Lower arg_{min,max} to streaming engine (#26845)
• Additional IR slice pushdown after filter pushdown (#26815)
• Streaming first/last on Enum through physical (#26783)
• Fast filter for scalar predicates (#26745)
• Allow SimpleProjection in streaming engine to rename (#26709)
• Streaming cloud download for scan_csv (#26637)
• Drop columns only needed for predicates after the predicate is applied (#26703)
• Run projection pushdown after predicate pushdown (#26688)
• Comparison literal downcasting (#26663)
• Add dynamic predicates for TopK (#26495)
• Increase minimum default parquet row group prefetch to 8 (#26632)
• Partial predicate conversion to PyArrow (#26567)
• Streaming cloud download for scan_ndjson / scan_lines (#26563)
• Grab GIL fewer times during Object join materialization (#26587)
• Improve CSV and NDJSON cloud sink performance (#26545)
• Tune cloud writer performance (#26518)
• Allow parallel InMemorySinks in streaming engine (#26501)
• Add streaming AsOf join node (#26398)
• Don't always rechunk on gather of nested types (#26478)

✨ Enhancements

• Support Expr for holidays in business day calculations (#26193)
• Parameter for pivot to always include value column name (#26730)
• Raise error in .collect_schema() when arr.get() is out-of-bounds (#26866)
• Extend Expr.reinterpret to all numeric types of the same size (#26401)
• Add missing_columns parameter to scan_csv (#26787)
• Clear no-op scan projections (#26858)
• Support nested datatypes for {min,max}_by (#26849)
• Support SQL ARRAY init from typed literals (#26622)
• Accept table identifier string in scan_iceberg() (#26826)
• Add a convenience make fresh command to the Makefile (#26809)
• Expose "use_zip64" Workbook option for write_excel (#26699)
• Add unstable LazyFrame.sink_iceberg (#26799)
• Add maintain order argument on implode (#26782)
• Speed up casting primitive to bool by at least 2x (#26823)
• Support ASCII format table input to pl.from_repr (#26806)
• Enable rowgroup skipping for float columns (#26805)
• Add expression context to errors (#26716)
• Add Decimal support for product reduction (#26725)
• Support all Iceberg V2 arrow types in sink_parquet arrow_schema parameter (#26669)
• Re-work behavior of arrow_schema parameter on sink_parquet (#26621)
• Add contains_dtype() method for Schema (#26661)
• Implement truncate as a "to_zero" rounding mode (#26677)
• More generic streaming GroupBy lowering (#26696)
• Create an Alignment TypeAlias (#26668)
• Add basic MemoryManager to track buffered dataframes for out-of-core support later (#26443)
• Add truncate Expression for numeric values (#26666)
• Better error messages for hex literal conversion issues in the SQL interface (#26657)
• Add SQL support for LPAD and RPAD string functions (#26631)
• Support SQL "FROM-first" SELECT query syntax (#26598)
• Improve base_type typing (#26602)
• Bump Chrono to 0.4.24, enabling stricter parsing of %.3f/%.6f/%.9f specifiers (#26075)
• Expose unstable assert_schema_equal in py-polars (#24869)
• Allow parsing of compact ISO 8601 strings (#24629)
• Add optional "label" param to DataFrame corr (#26588)
• Streaming cloud download for scan_ndjson / scan_lines (#26563)
• Configuration to cast integers to floats in cast_options for scan_parquet (#26492)
• Add escaping to quotes and newlines when reading JSON object into string (#26578)
• Standardise on RFC-5545 when doing datetime arithmetic on timezone-aware datetimes (#26425)
• Support sas_token in Azure credential provider (#26565)
• Relax SQL requirement for derived tables and subqueries to have aliases (#26543)
• Add polars-config and pl.Config.reload_env_vars() (#26524)
• Record path for object store error raised from sinks (#26541)
• Use CRC64NVME for checksum in aws sinks (#26522)
• Add get() for binary Series (#26514)
• Add streaming AsOf join node (#26398)
• Add primitive filter -> agg lowering in streaming GroupBy (#26459)
• Support for the SQL FETCH clause (#26449)

🐞 Bug fixes

• Prevent Boolean arithmetic with integer literals producing Unknown type in streaming engine (#26878)
• Fix sink to partitioned S3 from Windows corrupted slashes (#26889)
• Remove outdated warning about List columns in unique() (#26295) (#26890)
• Restore pyarrow predicate conversion for is_in (#26811)
• Release GIL before df.to_ndarray() to avoid deadlock (#26832)
• Fix panic on CSV count_rows with FORCE_ASYNC (#26883)
• Add scalar comparisons for UInt128 series (#26886)
• Fix shape error not raised for 0 width inputs with non-0 height for streaming horizontal concat (#26877)
• Fix streaming zip-broadcast node did not raise shape mismatch on empty recv from ready port (#26871)
• Fix incorrect output list.eval with scalar expr, fix panic on list.agg with nulls (#26868)
• Allow list argument in group_by().map_groups() (#26707)
• Support for ADBC drivers instantiated with dbc in DataFrame.write_database (#26157)
• Incorrect arg_sort with descending+limit (#26839)
• Raise error in .collect_schema() when arr.get() is out-of-bounds (#26866)
• Return ComputeError instead of panicking in map_groups UDF (#26665)
• Issue PerformanceWarning in LazyFrame.__contains__ (#26734)
• Correct type hint for map_columns function parameter (#26487)
• Apply thousands_separator to count/null_count in describe() for non-numeric columns (#26486)
• Ensure proper handling of timedelta when multiplying with a Series (#26830)
• Correct type hint for function parameter in DataFrame.map_columns (#26372)
• Segfault in JoinExec on deep plan (#26796)
• Fix unary expressions on literal in over context (#26827)
• Fix {min,max}_by in streaming engine for Boolean full {min,max} value column (#26848)
• Fix debug panic on clip with nan bound (#26854)
• Support grouped {arg_,}_{min,max} for Categoricals (#26856)
• Throw an error if a string is passed to LazyFrame.pivot on_columns (#26852)
• Preserve input float precision in rolling_cov() and rolling_corr() with mixed input types (#26820)
• Preserve row count when converting zero-column DataFrame via arrow PyCapsule interface (#26835)
• Prevent infinite recursion in streaming group_by fallback (#26801)
• Use RowEncodingContext::Struct when determining D::Struct encoded item len (#26817)
• Incorrectly applied CSE on different map_batches functions (#26822)
• Fix duplicated query execution on todo panic when combining collect(engine='streaming') with POLARS_AUTO_NEW_STREAMING (#26792)
• Prevent predicate pushdown across Sort with baked-in slice (#26804)
• Restore compatibility with pd.Timedelta (#26785)
• Fix panic on lazy sink_parquet created in pipe_with_schema (#26784)
• Support {column_name} and {index} placeholders in pl.format string (#26771)
• Do not use merge-join if nulls_last is unknown (#26778)
• Normalize float zeros in Parquet column statistics (#26776)
• Fix out-of-bounds for positive offset in windowed rolling (#26724)
• Raise error when .get() is out-of-bounds in group by context (#26752)
• Boolean bitwise_xor aggregation inverted when column contains nulls (#26749)
• Parameter nulls_last was ignored in over (#26718)
• Allow missing time in inexact strptime (#26714)
• Respect nulls_last in sort_by within group_by().agg() slow path (#26681)
• Return NaN when using corr() with a literal and expr (#26697)
• Allow strict horizontal concat with empty df (#26345)
• Fix PoisonError panic caused by reentrant usage of file cache (#26627)
• Return null for int values exceeding 128-bit range with strict=False (#26674)
• Incorrect boolean min/max with nulls (#26671)
• Slice-slice pushdown for n_rows (#26673)
• Resolve panic in Enum struct slicing (#26643)
• Fix CSPE for group_by.map_groups (#26640)
• Remove non-existent parameter from SQLContext typing overloads (#26658)
• Address pl.from_epoch losing fractional seconds (#26419)
• Fix to_pandas() on empty enum Series did not preserve enum dictionary (#26610)
• Rounding behaviour for f32 values with "HalfAwayFromZero" mode (#26624)
• Updated Sum Type Hint (#26629)
• Don't allow namespace registration to override standard methods or properties (#26450)
• Correct arg_(min|max) for scalar columns (#26609)
• Use monkeypatch.chdir in test_sink_path_slicing_utf8_boundaries_26324 (#26616)
• Respect SQL semantics for cumulative functions mapped via OVER clause (#26570)
• Fix incorrect multiplexer output ordering on source token stop request (#26561)
• Fix PyIceberg filter on boolean column (#26550)
• Set dictionary_page_offset when dictionary encoding is used and point data_page_offset to the first data page (#26542)
• Move query parameters to request body when retrieving Unity Catalog temporary credentials (#26539)
• Ensure read_csv_batched() prints deprecation warning (#26530)
• Implement PhysicalExpr for MinBy/MaxBy nodes (#26506)
• Refactor row-encoding logic in IR join lowering into separate function (#26512)
• Correctly check for path extensions (#26513)
• Change AsOf join to be based on TotalOrd (#26497)
• Correctly raise error on failing nested strict casts (#26499)
• Prevent invalid type casts in replace_strict() (#26453)
• Return null when dividing literals by 0 (#26343)
• Fix type-hint for Series.quantile (#26422)

📖 Documentation

• Mention ComputeContexts create ephemeral environments by default and hint at re-use (#26692)
• Remove confusing join validation note (#26795)
• Fix formatting in categorical documentation (#26746)
• Fix broken AI policy link (#26728)
• Create Polars Cloud Glossary (#26690)
• Additional SQL documentation (#26662)
• Include invalidate_caches in bisect instructions (#26641)
• Add git bisect guide to contributing docs (#26634)
• Fix Polars Cloud examples (formatting & type hints) (#26625)
• Updated Airflow orchestration documentation (#26585)
• Improve SQL docs for EXTRACT and DATE_PART functions (#26575)
• Fix docstring for bitwise_count_zeros method (#26519)
• Add get() for binary Series (#26514)

🛠️ Other improvements

• Use large linux-arm runner for release (#26898)
• Ensure .gitignore and .typos.toml exclude "_polars_runtime*" directories (#26842)
• Additional IR slice pushdown after filter pushdown (#26815)
• Add private _expand_paths scan function (#26798)
• Change Expr sortedness container to AExprSorted and add nulls_last to PyExpr.set_sorted() (#26781)
• Move stop_and_buffer_pipe_contents into joins/utils.rs (#26810)
• Replace iejoin is_supported_type macro with a closure in predicate_pushdown/join.rs (#26812)
• Fix first-time contributor auto-label (#26794)
• Automatically add first-contribution label (#26780)
• Add tests for functions that operate on pl.all() expansion (#26773)
• Make contributing policy more strict (#26772)
• Add unused argument warning to ruff rules (#26720)
• Move shared streaming CSV/NDJSON code into shared mod (#26742)
• Undo pub removal of to_dyn_object_store (#26722)
• Mark {read, scan}_ndjson cache argument(s) as deprecated (#26711)
• Add test for predicate before join (#26705)
• Remove PlanCallback from sql (#26686)
• Bump Rust nightly compiler version (#26379)
• Remove unused problematic ArrayFromIter (#26639)
• Move more boolean code to polars_compute, reusing kernels (#26636)
• Avoid implicit import from importlib (#26603)
• Cleanup assert_schema_equal (#26596)
• Replace some env var reading by polars-config (#26607)
• Use monkeypatch.chdir in test_sink_path_slicing_utf8_boundaries_26324 (#26616)
• Add __init__.py files and docstrings to testing directories (#26408)
• Add wrapper for clippy so it continues on warnings (#26527)
• Use LazyFrame.clear to clear sql (#26562)
• Update docs (#26560)
• Add backtrace coloring (#26544)
• Evaluate sql process_except_intersect during IR (#26516)
• Reformat LICENSE (#26532)
• Add a pipeline in which we test with POLARS_IDEAL_MORSEL_SIZE=4 (#26420)
• Remove test_file and have tests create test.parquet in tmp_path (#26525)
• Refactor row-encoding logic in IR join lowering into separate function (#26512)
• Fix mypy pyiceberg expression errors (#26523)
• Make nix flake mostly work (#26517)
• Switch to custom cloud writer with IO sink metrics (#26494)
• Update s3fs dev dependency (#26509)
• Remove Default on DataType (#26511)
• Have parameterized series rechunk() if not allow_chunks (#26504)
• Remove dead code (RevMapping) (#26508)
• Upgraded ruff, mypy, typos (#26476)
• More SQL to IR conversion execute_isolated (#26455)

Thank you to all our contributors for making this release possible! @BJohnBraddock, @EndPositive, @Jesse-Bakker, @Kevin-Patyk, @MarcoGorelli, @Matt711, @NathanHu725, @RenzoMXD, @TNieuwdorp, @Voultapher, @WaffleLapkin, @abishop1990, @alexander-beedie, @azimafroozeh, @boris324, @cBournhonesque, @carnarez, @coastalwhite, @daizutabi, @dependabot[bot], @dsprenkels, @erandagan, @etiennebacher, @gautamvarmadatla, @henryharbeck, @hutch3232, @itamarst, @jberg5, @johalnes, @kdn36, @leudz, @lukas-reining, @moktamd, @mqqz, @mroeschke, @nameexhaustion, @orlp, @pragun-ananda, @qxzcode, @ritchie46, @spock-yh, @stakeswky, @tlauli, @toroleapinc, @veeceey and dependabot[bot]

✨ Enhancements

• Add get() to retrieve a byte from binary data (#26454)
• Remove with_context in SQL lowering (#26416)

🐞 Bug fixes

• Do not overwrite used names in cluster_with_columns pushdown (#26467)
• Do not mark output of concat_str on multiple inputs as sorted (#26468)
• Fix CSV schema inference content line duplication bug (#26452)
• Fix InvalidOperationError using scan_delta with filter (#26448)
• Alias giving missing column after streaming GroupBy CSE (#26447)
• Ensure by_name selector selects only names (#26437)
• Restore compatibility of strings written to parquet with pyarrow filter (#26436)
• Update schema in cluster_with_columns optimization (#26430)
• Fix negative slice in groups slicing (#26442)
• Don't run CPU check on aarch64 musl (#26439)
• Fixed annotations shadowed by class methods (#26356)
• Remove the POLARS_IDEAL_MORSEL_SIZE monkeypatching in the parametric merge-join test (#26418)
• Fix selector match patterns for multiline column names (#26320)

📖 Documentation

• Add sink_delta to API reference (#26446)

🛠️ Other improvements

• Cleanup unused attributes in optimizer (#26464)
• Use Expr::Display as catch all for IR - DSL asymmetry (#26471)
• Ignore pytz in mypy (#26441)
• Remove the POLARS_IDEAL_MORSEL_SIZE monkeypatching in the parametric merge-join test (#26418)
• Cleanup the parametric merge-join test (#26413)

Thank you to all our contributors for making this release possible! @Voultapher, @alexander-beedie, @azimafroozeh, @cmdlineluser, @dependabot[bot], @dsprenkels, @hamdanal, @kdn36, @nameexhaustion, @orlp, @ritchie46 and dependabot[bot]

⚠️ Deprecations

• Deprecate retries=n in favor of storage_options={"max_retries": n} (#26155)

🚀 Performance improvements

• Enable zero-copy object_store put upload for IPC sink (#26288)
• Resolve file schema's and metadata concurrently (#26325)
• Run elementwise CSEE for the streaming engine (#26278)
• Disable morsel splitting for fast-count on streaming engine (#26245)
• Implement streaming decompression for scan_ndjson and scan_lines (#26200)
• Improve string slicing performance (#26206)
• Refactor scan_delta to use python dataset interface (#26190)
• Add dedicated kernel for group-by arg_max/arg_min (#26093)
• Add streaming merge-join (#25964)
• Generalize Bitmap::new_zeroed opt for Buffer::zeroed (#26142)
• Reduce fs stat calls in path expansion (#26173)
• Lower streaming group_by n_unique to unique().len() (#26109)

✨ Enhancements

• Avoid OOM for scan_ndjson and scan_lines if input is compressed and negative slice (#26396)
• Support annoymous agg in-mem (#26376)
• Add unstable arrow_schema parameter to sink_parquet (#26323)
• Improve error message formatting for structs (#26349)
• Remove parquet field overwrites (#26236)
• Enable zero-copy object_store put upload for IPC sink (#26288)
• Improved disambiguation for qualified wildcard columns in SQL projections (#26301)
• Expose upload_concurrency through env var (#26263)
• Allow quantile to compute multiple quantiles at once (#25516)
• Allow empty LazyFrame in LazyFrame.group_by(...).map_groups (#26275)
• Use delta file statistics for batch predicate pushdown (#26242)
• Add streaming UnorderedUnion (#26240)
• Implement compression support for sink_ndjson (#26212)
• Add unstable record batch statistics flags to {sink/scan}_ipc (#26254)
• Support CSE for python UDFs on the same address (#26253)
• Cloud retry/backoff configuration via storage_options (#26204)
• Use same sort order for expanded paths across local / cloud / directory / glob (#26191)
• Add streaming merge-join (#25964)
• Serialize optimization flags for cloud plan (#26168)
• Add compression support to write_csv and sink_csv (#26111)
• Add scan_lines (#26112)
• Support regex in str.split (#26060)
• Add unstable IPC Statistics read/write to scan_ipc/sink_ipc (#26079)
• Add unstable height parameter to DataFrame/LazyFrame (#26014)
• Remove old partition sink API (#26100)
• Expose ArrowStreamExportable on python collect batches iterator (#26074)
• Add nulls support for all rolling_by operations (#26081)

🐞 Bug fixes

• Correct off-by-one in RLE row counting for nullable dictionary-encoded columns (#26411)
• Support very large integers in env var limits (#26399)
• Fix PlPath panic from incorrect slicing of UTF8 boundaries (#26389)
• Fix Float dtype for spearman correlation (#26392)
• Fix optimizer panic in right joins with type coercion (#26365)
• Don't serialize retry config from local environment vars (#26289)
• Fix PartitionBy with scalar key expressions and diff() (#26370)
• Add {Float16, Float32} -> Float32 lossless upcast (#26373)
• Fix panic using with_columns and collect_all (#26366)
• Add multi-page support for writing dictionary-encoded Parquet columns (#26360)
• Ensure slice advancement when skipping non-inlinable values in is_in with inlinable needles (#26361)
• Pin xlsx2csv version temporarily (#26352)
• Bugs in ViewArray total_bytes_len (#26328)
• Overflow in i128::abs in Decimal fits check (#26341)
• Make Expr.hash on Categorical mapping-independent (#26340)
• Clone shared GroupBy node before mutation in physical plan creation (#26327)
• Fixed "sheet_name" typing for read_ods and read_excel (#26317)
• Improve Polars dtype inference from Python Union typing (#26303)
• Consider the "current location" of an item when computing rolling_rank_by (#26287)
• Reset is_count_star flag between queries in collect_all (#26256)
• Fix incorrect is_between filter on scan_parquet (#26284)
• Make polars compatible with ty (#26270)
• Lower AnonymousStreamingAgg in group-by as aggregate (#26258)
• Avoid overflow in pl.duration scalar arguments case (#26213)
• Broadcast arr.get on single array with multiple indices (#26219)
• Fix panic on CSPE with sorts (#26231)
• Eager DataFrame.slice with negative offset and length=None (#26215)
• Use correct schema side for streaming merge join lowering (#26218)
• Overflow panic in scan_csv with multiple files and skip_rows + n_rows larger than total row count (#26128)
• Respect allow_object flag after cache (#26196)
• Raise error on non-elementwise PartitionBy keys (#26194)
• Allow ordered categorical dictionary in scan_parquet (#26180)
• Allow excess bytes on IPC bitmap compressed length (#26176)
• Address a macOS-specific compile issue (#26172)
• Fix deadlock on hash_rows() of 0-width DataFrame (#26154)
• Fix NameError filtering pyarrow dataset (#26166)
• Fix concat_arr panic when using categoricals/enums (#26146)
• Fix NDJSON/scan_lines negative slice splitting with extremely long lines (#26132)
• Incorrect group_by min/max fast path (#26139)
• Remove a source of non-determinism from lowering (#26137)
• Error when with_row_index or unpivot create duplicate columns on a LazyFrame (#26107)
• Panics on shift with head (#26099)

📖 Documentation

• Fix Expr.get referencing incorrect dtype for index parameter (#26364)
• Fix Expr.quantile formatting (#26351)
• Drop sphinx-llms-txt extension (#26285)
• Remove deprecated cublet_id (#26260)
• Update for new release (#26255)
• Update MCP server section with new URL (#26241)
• Fix unmatched paren and punctuation in pandas migration guide (#26251)
• Add observatory database_path to docs (#26201)
• Note plugins in Python user-defined functions (#26138)

📦 Build system

• Address remaining Python 3.14 issues with make requirements-all (#26195)
• Address a macOS-specific compile issue (#26172)

🛠️ Other improvements

• Ensure local doctests skip from_torch if module not installed (#26405)
• Change linked timezones in test suite to canonical timezones (#26310)
• Implement various deprecations (#26314)
• Rename Operator::Divide to RustDivide (#26339)
• Properly disable the Pyodide tests (#26382)
• Remove unused field (#26367)
• Fix runtime nesting (#26359)
• Remove xlsx2csv dependency pin (#26355)
• Use outer runtime if exists in to_alp (#26353)
• Make CategoricalMapping::new pub(crate) to avoid misuse (#26308)
• Clarify IPC buffer read limit/length paramter (#26334)
• Add dtype test coverage for delta predicate filter (#26291)
• Add AI policy (#26286)
• Unpin "pandas<3" in dev dependencies (#26249)
• Remove all non CSV fast-count paths (#26233)
• Pin pandas to 2.x for now (#26221)
• Remove unnecessary xfail (#26199)
• Ensure optimization flag modification happens local (#26185)
• Simplify IcebergDataset (#26165)
• Reorganize unit tests into logical subdirectories (#26149)
• Lint leftover fixme (#26122)
• Improve backtrace for POLARS_PANIC_ON_ERR (#26125)
• Fix Python docs build (#26117)
• Disable unused-ignore mypy lint (#26110)
• Ignore mypy warning (#26105)
• Raise error on file://hostname/path (#26061)
• Disable debug info for docs workflow (#26086)
• Update docs for next polars cloud release (#26091)
• Support Python 3.14 in dev environment (#26073)

Thank you to all our contributors for making this release possible! @Atarust, @EndPositive, @Kevin-Patyk, @LeeviLindgren, @MarcoGorelli, @Matt711, @MrAttoAttoAtto, @Voultapher, @WaffleLapkin, @agossard, @alex-gregory-ds, @alexander-beedie, @azimafroozeh, @bayoumi17m, @c-peters, @carnarez, @dependabot[bot], @dsprenkels, @hallmason17, @hamdanal, @ion-elgreco, @kdn36, @lun3x, @mcrumiller, @nameexhaustion, @orlp, @qxzcode, @r-brink, @ritchie46, @sweb and dependabot[bot]

🚀 Performance improvements

• Speed up SQL interface "UNION" clauses (#26039)

🐞 Bug fixes

• Optimize slicing support on compressed IPC (#26071)
• CPU check for musl builds (#26076)
• Propagate C Stream import errors instead of panicking (#26036)
• Fix slicing on compressed IPC (#26066)

📖 Documentation

• Clarify min_by/max_by behavior on ties (#26077)

🛠️ Other improvements

• Mark top slow normal tests as slow (#26080)
• Update breaking deps (#26055)
• Fix for upstream url bug and update deps (#26052)
• Properly pin chrono (#26051)
• Don't run rust doctests (#26046)
• Update deps (#26042)
• Ignore very slow test (#26041)

Thank you to all our contributors for making this release possible! @Voultapher, @alexander-beedie, @kdn36, @nameexhaustion, @orlp, @ritchie46 and @wtn

🚀 Performance improvements

• Speed up SQL interface "ORDER BY" clauses (#26037)
• Add fast kernel for is_nan and use it for numpy NaN->null conversion (#26034)
• Optimize ArrayFromIter implementations for ObjectArray (#25712)
• New streaming NDJSON sink pipeline (#25948)
• New streaming CSV sink pipeline (#25900)
• Dispatch partitioned usage of sink_* functions to new-streaming by default (#25910)
• Replace ryu with faster zmij (#25885)
• Reduce memory usage for .item() count in grouped first/last (#25787)
• Skip schema inference if schema provided for scan_csv/ndjson (#25757)
• Add width-aware chunking to prevent degradation with wide data (#25764)
• Use new sink pipeline for write/sink_ipc (#25746)
• Reduce memory usage when scanning multiple parquet files in streaming (#25747)
• Don't call cluster_with_columns optimization if not needed (#25724)

✨ Enhancements

• Add new pl.PartitionBy API (#26004)
• ArrowStreamExportable and sink_delta (#25994)
• Release musl builds (#25894)
• Implement streaming decompression for CSV COUNT(*) fast path (#25988)
• Add nulls support for rolling_mean_by (#25917)
• Add lazy collect_all (#25991)
• Add streaming decompression for NDJSON schema inference (#25992)
• Improved handling of unqualified SQL JOIN columns that are ambiguous (#25761)
• Drop Python 3.9 support (#25984)
• Expose record batch size in {sink,write}_ipc (#25958)
• Add null_on_oob parameter to expr.get (#25957)
• Suggest correct timezone if timezone validation fails (#25937)
• Support streaming IPC scan from S3 object store (#25868)
• Implement streaming CSV schema inference (#25911)
• Support hashing of meta expressions (#25916)
• Improve SQLContext recognition of possible table objects in the Python globals (#25749)
• Add pl.Expr.(min|max)_by (#25905)
• Improve MemSlice Debug impl (#25913)
• Implement or fix json encode/decode for (U)Int128, Categorical, Enum, Decimal (#25896)
• Expand scatter to more dtypes (#25874)
• Implement streaming CSV decompression (#25842)
• Add Series sql method for API consistency (#25792)
• Mark Polars as safe for free-threading (#25677)
• Support Binary and Decimal in arg_(min|max) (#25839)
• Allow Decimal parsing in str.json_decode (#25797)
• Add shift support for Object data type (#25769)
• Add missing Series.arr.mean (#25774)
• Allow scientific notation when parsing Decimals (#25711)

🐞 Bug fixes

• Release GIL on collect_batches (#26033)
• Missing buffer update in String is_in Parquet pushdown (#26019)
• Make struct.with_fields data model coherent (#25610)
• Incorrect output order for order sensitive operations after join_asof (#25990)
• Use SeriesExport for pyo3-polars FFI (#26000)
• Add pl.Schema to type signature for DataFrame.cast (#25983)
• Don't write Parquet min/max statistics for i128 (#25986)
• Ensure chunk consistency in in-memory join (#25979)
• Fix varying block metadata length in IPC reader (#25975)
• Implement collect_batches properly in Rust (#25918)
• Fix panic on arithmetic with bools in list (#25898)
• Convert to index type with strict cast in some places (#25912)
• Empty dataframe in streaming non-strict hconcat (#25903)
• Infer large u64 in json as i128 (#25904)
• Set http client timeouts to 10 minutes (#25902)
• Correct lexicographic ordering for Parquet BYTE_ARRAY statistics (#25886)
• Raise error on duplicate group_by names in upsample() (#25811)
• Correctly export view buffer sizes nested in Extension types (#25853)
• Fix DataFrame.estimated_size not handling overlapping chunks correctly (#25775)
• Ensure Kahan sum does not introduce NaN from infinities (#25850)
• Trim excess bytes in parquet decode (#25829)
• Fix panic/deadlock sinking parquet with rows larger than 64MB estimated size (#25836)
• Fix quantile midpoint interpolation (#25824)
• Don't use cast when converting from physical in list.get (#25831)
• Invalid null count on int -> categorical cast (#25816)
• Update groups in list.eval (#25826)
• Use downcast before FFI conversion in PythonScan (#25815)
• Double-counting of row metrics (#25810)
• Cast nulls to expected type in streaming union node (#25802)
• Incorrect slice pushdown into map_groups (#25809)
• Fix panic writing parquet with single bool column (#25807)
• Fix upsample with group_by incorrectly introduced NULLs on group key columns (#25794)
• Panic in top_k pruning (#25798)
• Fix incorrect collect_schema for unpivot followed by join (#25782)
• Verify arr namespace is called from array column (#25650)
• Ensure LazyFrame.serialize() unchanged after collect_schema() (#25780)
• Function map_(rows|elements) with return_dtype = pl.Object (#25753)
• Fix incorrect cargo sub-feature (#25738)

📖 Documentation

• Fix display of deprecation warning (#26010)
• Document null behaviour for rank (#25887)
• Add QUALIFY clause and SUBSTRING function to the SQL docs (#25779)
• Update mixed-offset datetime parsing example in user guide (#25915)
• Update bare-metal docs for mounted anonymous results (#25801)
• Fix credential parameter name in cloud-storage.py (#25788)
• Configuration options update (#25756)

🛠️ Other improvements

• Update rust compiler (#26017)
• Improve csv test coverage (#25980)
• Ramp up CSV read size (#25997)
• Mark lazy parameter to collect_all as unstable (#25999)
• Update ruff action and simplify version handling (#25940)
• Run python lint target as part of pre-commit (#25982)
• Disable HTTP timeout for receiving response body (#25970)
• Fix mypy lint (#25963)
• Add AI contribution policy (#25956)
• Fix failing scan delta S3 test (#25932)
• Improve MemSlice Debug impl (#25913)
• Remove and deprecate batched csv reader (#25884)
• Remove unused AnonymousScan functions (#25872)
• Filter DeprecationWarning from pyparsing indirectly through pyiceberg (#25854)
• Various small improvements (#25835)
• Clear venv with appropriate version of Python (#25851)
• Skip schema inference if schema provided for scan_csv/ndjson (#25757)
• Ensure proper async connection cleanup on DB test exit (#25766)
• Ensure we uninstall other Polars runtimes in CI (#25739)
• Make 'make requirements' more robust (#25693)
• Remove duplicate compression level types (#25723)

Thank you to all our contributors for making this release possible! @AndreaBozzo, @EndPositive, @Kevin-Patyk, @MarcoGorelli, @Voultapher, @alexander-beedie, @anosrepenilno, @arlyon, @azimafroozeh, @carnarez, @dependabot[bot], @dsprenkels, @edizeqiri, @eitanf, @gab23r, @henryharbeck, @hutch3232, @ion-elgreco, @jqnatividad, @kdn36, @lun3x, @m1guelperez, @mcrumiller, @nameexhaustion, @orlp, @ritchie46, @sachinn854, @yonikremer and dependabot[bot]

🚀 Performance improvements

• Tune partitioned sink_parquet cloud performance (#25687)

✨ Enhancements

• Allow creation of Object literal (#25690)
• Don't collect schema in SQL union processing (#25675)

🐞 Bug fixes

• Don't invalidate node in cluster-with-columns (#25714)
• Move boto3 extra from s3fs in dev requirements (#25667)
• Add missing type stubs for bin_slice, bin_head, and bin_tail (#25697)
• Binary slice methods missing from Series and docs (#25683)
• Mix-up of variable_name/value_name in unpivot (#25685)
• Invalid usage of drop_first in to_dummies when nulls present (#25435)

📖 Documentation

• Fix typos in Excel and Pandas migration guides (#25709)
• Add "right" to how options in join() docstrings (#25678)

🛠️ Other improvements

• Move Object lit fix earlier in the function (#25713)
• Remove unused decimal file (#25701)
• Move boto3 extra from s3fs in dev requirements (#25667)
• Upgrade to latest version of sqlparser-rs (#25673)
• Update slab to version without RUSTSEC (#25686)
• Fix typo (#25684)

Thank you to all our contributors for making this release possible! @AndreaBozzo, @Kevin-Patyk, @alexander-beedie, @dsprenkels, @jamesfricker, @mcrumiller, @nameexhaustion, @orlp and @ritchie46

🏆 Highlights

• Add Extension types (#25322)

✨ Enhancements

• Add SQL support for the QUALIFY clause (#25652)
• Add bin.slice(), bin.head(), and bin.tail() methods (#25647)
• Add SQL syntax support for CROSS JOIN UNNEST(col) (#25623)
• Add separate env var to log tracked metrics (#25586)
• Expose fields for generating physical plan visualization data (#25562)
• Allow pl.Object in pivot value (#25533)
• Minor improvement for as_struct repr (#25529)
• Temporal quantile in rolling context (#25479)
• Add quantile for missing temporals (#25464)
• Add strict parameter to pl.concat(how='horizontal') (#25452)
• Support decimals in search_sorted (#25450)
• Expose and document pl.Categories (#25443)
• Use reference to Graph pipes when flushing metrics (#25442)
• Extend SQL UNNEST support to handle multiple array expressions (#25418)
• Add SQL support for ROW_NUMBER, RANK, and DENSE_RANK functions (#25409)
• Allow elementwise Expr.over in aggregation context (#25402)
• Add SQL support for named WINDOW references (#25400)
• Add leftmost option to str.replace_many / str.find_many / str.extract_many (#25398)
• Automatically Parquet dictionary encode floats (#25387)
• Support unique_counts for all datatypes (#25379)
• Add maintain_order to Expr.mode (#25377)
• Allow hash for all List dtypes (#25372)
• Add empty_as_null and keep_nulls to {Lazy,Data}Frame.explode (#25369)
• Display function of streaming physical plan map node (#25368)
• Allow slice on scalar in aggregation context (#25358)
• Allow implode and aggregation in aggregation context (#25357)
• Move GraphMetrics into StreamingQuery (#25310)
• Documentation on Polars Cloud manifests (#25295)
• Add empty_as_null and keep_nulls flags to Expr.explode (#25289)
• Allow Expr.unique on List/Array with non-numeric types (#25285)
• Raise suitable error on non-integer "n" value for clear (#25266)
• Allow Expr.rolling in aggregation contexts (#25258)
• Allow bare .row() on a single-row DataFrame, equivalent to .item() on a single-element DataFrame (#25229)
• Support additional forms of SQL CREATE TABLE statements (#25191)
• Add support for Float16 dtype (#25185)
• Support column-positional SQL "UNION" operations (#25183)
• Add unstable Schema.to_arrow() (#25149)
• Make DSL-hash skippable (#25140)
• Improve error message on unsupported SQL subquery comparisons (#25135)
• Support arbitrary expressions in SQL JOIN constraints (#25132)
• Allow arbitrary expressions as the Expr.rolling index_column (#25117)
• Set polars/ user-agent (#25112)
• Support ewm_var/std in streaming engine (#25109)
• Rewrite IR::Scan to IR::DataFrameScan in expand_datasets when applicable (#25106)
• Add ignore_nulls to first / last (#25105)
• Allow arbitrary Expressions in "subset" parameter of unique frame method (#25099)
• Add BIT_NOT support to the SQL interface (#25094)
• Streaming {Expr,LazyFrame}.rolling (#25058)
• Add LazyFrame.pivot (#25016)
• Add SQL support for LEAD and LAG functions (#23956)
• Add having to group_by context (#23550)
• Add show methods for DataFrame and LazyFrame (#19634)

🚀 Performance improvements

• Set parallelization threshold in take_unchecked_impl (#25672)
• New single file IO sink pipeline enabled for sink_parquet (#25670)
• Correct overly eager local predicate insertion for unpivot (#25644)
• New partitioned IO sink pipeline enabled for sink_parquet (#25629)
• Use strong hash instead of traversal for CSPE equality (#25537)
• Reduce HuggingFace API calls (#25521)
• Fix panic in is_between support in streaming Parquet predicate push down (#25476)
• Faster kernels for rle_lengths (#25448)
• Mark output of more non-order-maintaining ops as unordered (#25419)
• Enable predicate expressions on unsigned integers (#25416)
• Allow detecting plan sortedness in more cases (#25408)
• Add parquet prefiltering for string regexes (#25381)
• Fast find start window in group_by_dynamic with large offset (#25376)
• Use fast path for agg_min/agg_max when nulls present (#25374)
• Add streaming native LazyFrame.group_by_dynamic (#25342)
• Fuse positive slice into streaming LazyFrame.rolling (#25338)
• Mark Expr.reshape((-1,)) as row separable (#25326)
• Return references from aexpr_to_leaf_names_iter (#25319)
• Use bitmap instead of Vec in first/last w. skip_nulls (#25318)
• Lazy gather for {forward,backward}_fill in group-by contexts (#25115)
• Add streaming sorted Group-By (#25013)

🐞 Bug fixes

• Rechunk on nested dtypes in take_unchecked_impl parallel path (#25662)
• Fix streaming SchemaMismatch panic on list.drop_nulls (#25661)
• Fix panic on Boolean rolling_sum calculation for list or array eval (#25660)
• Fix "dtype is unknown" panic in cross joins with literals (#25658)
• Fix panic edge-case when scanning hive partitioned data (#25656)
• Fix "unreachable code" panic in UDF dtype inference (#25655)
• Address potential "batch_size" parameter collision in scan_pyarrow_dataset (#25654)
• Fix empty format handling (#25638)
• Improve SQL GROUP BY and ORDER BY expression resolution, handling aliasing edge-cases (#25637)
• Preserve List inner dtype during chunked take operations (#25634)
• Fix lifetime for AmortSeries lazy group iterator (#25620)
• Fix spearman panicking on nulls (#25619)
• Properly resolve HAVING clause during SQL GROUP BY operations (#25615)
• Prevent false positives in is_in for large integers (#25608)
• Differentiate between empty list an no list for unpivot (#25597)
• Bug in boolean unique_counts (#25587)
• Hang in multi-chunk DataFrame .rows() (#25582)
• Correct arr_to_any_value for object arrays (#25581)
• Have PySeries::new_f16 receive pf16s instead of f32s (#25579)
• Set Float16 parquet schema type to Float16 (#25578)
• Fix incorrect .list.eval after slicing operations (#25540)
• Strict conversion AnyValue to Struct (#25536)
• Rolling mean/median for temporals (#25512)
• Add .rolling_rank() support for temporal types and pl.Boolean (#25509)
• Fix occurence of exact matches of .join_asof(strategy="nearest", allow_exact_matches=False, ...) (#25506)
• Always respect return_dtype in map_elements and map_rows (#25504)
• Fix group lengths check in sort_by with AggregatedScalar (#25503)
• Fix dictionary replacement error in write_ipc() (#25497)
• Fix expr slice pushdown causing shape error on literals (#25485)
• Allow empty list in sort_by in list.eval context (#25481)
• Raise error on out-of-range dates in temporal operations (#25471)
• Validate list.slice parameters are not lists (#25458)
• Make sum on strings error in group_by context (#25456)
• Prevent panic when joining sorted LazyFrame with itself (#25453)
• Apply CSV dict overrides by name only (#25436)
• Incorrect result in aggregated first/last with ignore_nulls (#25414)
• Fix off-by-one bug in ColumnPredicates generation for inequalities operating on integer columns (#25412)
• Use Cargo.template.toml to prevent git dependencies from using template (#25392)
• Fix arr.{eval,agg} in aggregation context (#25390)
• Support AggregatedList in list.{eval,agg} context (#25385)
• Nested dtypes in streaming first_non_null/last_non_null (#25375)
• Remove Expr casts in pl.lit invocations (#25373)
• Optimize projection pushdown through HConcat (#25371)
• Revert pl.format behavior with nulls (#25370)
• Correct eq_missing for struct with nulls (#25363)
• Resolve edge-case with SQL aggregates that have the same name as one of the GROUP BY keys (#25362)
• Unique on literal in aggregation context (#25359)
• Aggregation with drop_nulls on literal (#25356)
• SQL NATURAL joins should coalesce the key columns (#25353)
• Mark {forward,backward}_fill as length_preserving (#25352)
• Correct drop_items for scalar input (#25351)
• Schema mismatch with list.agg, unique and scalar (#25348)
• AnyValue::to_physical for categoricals (#25341)
• Bugs in pl.from_repr with signed exponential floats and line wrapping (#25331)
• Remove ClosableFile (#25330)
• Increase precision when constructing float Series (#25323)
• Fix link errors reported by markdown-link-check (#25314)
• Parquet is_in for mixed validity pages (#25313)
• Fix building polars-plan with features lazy,concat_str (but no strings) (#25306)
• Fix building polars-mem-engine with the async feature (#25300)
• Nested dtypes in streaming first/last (#25298)
• Fix length preserving check for eval expressions in streaming engine (#25294)
• Panic exception when calling Expr.rolling in .over (#25283)
• Don't quietly allow unsupported SQL SELECT clauses (#25282)
• Reverse on chunked struct (#25281)
• Correct {first,last}_non_null if there are empty chunks (#25279)
• Incorrect results for aggregated {n_,}unique on bools (#25275)
• Run async DB queries with regular asyncio if not inside a running loop (#25268)
• Fix small bug with PyExpr to PyObject conversion (#25265)
• Fix building polars-expr without timezones feature (#25254)
• Correctly prune projected columns in hints (#25250)
• Address multiple issues with SQL OVER clause behaviour for window functions (#25249)
• Allow Null dtype values in scatter (#25245)
• Make str.json_decode output deterministic with lists (#25240)
• Correct handle requested stops in streaming shift (#25239)
• Use (i64, u64) for VisualizationData (offset, length) slices (#25203)
• Fix serialization of lazyframes containing huge tables (#25190)
• Fix single-column CSV header duplication with leading empty lines (#25186)
• Enhanced column resolution/tracking through multi-way SQL joins (#25181)
• Fix format_str in case of multiple chunks (#25162)
• Handle some unusual pl.col.<colname> edge-cases (#25153)
• Fix incorrect reshape on sliced lists (#25139)
• Support "index" as column name in group_by iterator (#25138)
• Fix panic in dt.truncate for invalid duration strings (#25124)
• DSL_SCHEMA_HASH should not changed by line endings (#25123)
• Don't trigger DeprecationWarning from SQL "IN" constraints that use subqueries (#25111)
• Solve multiple issues relating to arena mutation in SQL subqueries (#25110)
• Return the correct string-case Expr reprs (#25101)
• Fix groups update on slices with different offsets (#25097)
• Unique key names in streaming sort/top_k (#25082)
• Fix CSV select(len()) off by 1 with comment prefix (#25069)
• Raise error for all/any on list instead of panic (#25018)
• Ensure out-of-range integers and other edge case values don't give wrong results for index_of() (#24369)
• Improve SQL UNNEST behaviour (#22546)

📖 Documentation

• Document schema parameter in meta methods (#25543)
• Correct link to datetime_range instead of date_range in resampling page (#25532)
• Remove lzo from parquet write options (#25522)
• Deprecate Categorical functions for lexical ordering and local checks (#25514)
• Update LazyFrame.collect_schema() docstring (#25508)
• Update on-premise documentation (#25489)
• Add LazyFrame.pivot to reference guide (#25482)
• Fix incorrect 'bitwise' in any_horizontal/all_horizontal docstring (#25469)
• Add docstring example showing str.slice taking Expression params (#25461)
• Add Extension and BaseExtension to doc index (#25444)
• Add polars-on-premise documentation (#25431)
• Add having API references (#25428)
• Explain aggregation & sorting of lists (#25260)
• Fix LanceDB URL (#25198)
• Update user guide for QueryProgress rename to QueryProfile (#25195)
• Update LazyFrame.remote signature (#25175)
• Fix source path (#25170)
• Fix non-existent replace_all reference in replace docs (#25161)
• Mention Narwhals in ecosystem page (#25100)
• Clarify bitwise behaviour of and_, or_, and not_ Expressions on integer columns (#25092)

🧪 Tests

• Add reliable test for pl.format on multiple chunks (#25164)

🔧 CI

• Avoid relabelling changes-dsl on every commit (#25216)
• Automatically label pull requests that change the DSL (#25177)

🏗️ Build system

• Fix make fmt and make lint commands (#25200)
• Make building the docs on macOS more reliable (#25095)

🛠️ Other improvements

• Add "panic" and "streaming" tagging to issue-labeler workflow (#25657)
• Use dtype for group_aware evaluation on ApplyExpr (#25639)
• Ensure literal-only SELECT broadcast conforms to SQL semantics (#25633)
• Ensure we hash all attributes and visit all children in traverse_and_hash_aexpr (#25627)
• Rename polars-on-premise to polars-on-premises (#25617)
• Constrain new issue-labeler workflow to the Issue title (#25614)
• Avoid rechunk requirement for Series.iter() (#25603)
• Help categorise Issues by automatically applying labels (using the same patterns used for labelling PRs) (#25599)
• Add disk-cleaning step for Ubuntu runners (#25593)
• Show on streaming engine (#25589)
• Ignore a couple of unexplained typing errors (#25580)
• Skip existing files in pypi upload (#25576)
• Fix template path in release-python workflow (#25565)
• Add asserts and tests for list.eval on multiple chunks with slicing (#25559)
• Skip rust integration tests for coverage in CI (#25558)
• Add Final type-qualifier to module-level constants (#25556)
• Print expected DSL schema hashes if mismatched (#25526)
• Update partitioned sink IR (#25524)
• Fix --uv argument for benchmark-remote (#25513)
• Add proptest AnyValue strategies (#25510)
• Fix rolling kernel dispatch with monotonic group attribute (#25494)
• Fix feature gating TZ_AWARE_RE again (#25493)
• Run maturin with --uv option (#25490)
• Add proptest DataFrame strategy (#25446)
• Add some cleanup (#25445)
• Add assert_sql_matches coverage for SQL DISTINCT and DISTINCT ON syntax (#25440)
• Test for group_by(...).having(...) (#25430)
• Remove aggregation context Context (#25424)
• Remove debug file write from test suite (#25393)
• Remove unused import (#25365)
• Enable more streaming tests (#25364)
• Dispatch Series.set to zip_with_same_dtype (#25327)
• Remove Column::Partitioned (#25324)
• Add toolchain file to runtimes for sdist (#25311)
• Fix typo in CI release workflow (#25309)
• Refactor sink IR (#25308)
• Directly take CloudScheme in parse_cloud_options() (#25304)
• Remove PyPartitioning (#25303)
• Simplify sink parameter passing from Python (#25302)
• Better coverage for group_by aggregations (#25290)
• Use dedicated runtime packages from template (#25284)
• Add test for unique with column subset (#25241)
• Fix Decimal precision annotation (#25227)
• Refactor dt_range functions (#25225)
• Add proptest strategies for Series nested types (#25220)
• Update markdown link checker (#25201)
• Add ElementExpr for _eval expressions (#25199)
• Upgraded ruff and typos and made the necessary lint updates (#25196)
• Make python docs build again (#25165)
• Upgrade to schemars 0.9.0 (#25158)
• Update versions (#25141)
• Silence unused mut warning (#25093)
• Add proptest strategies for Series logical types (#24849)

♻️ Refactoring

• Make polars-plan constants more consistent (#25645)
• Add support for multi-column reductions (#25640)
• Simplify _write_any_value (#25622)
• Add parquet file write pipeline for new IO sinks (#25618)
• Add streaming IO sink components (#25594)
• Add arg_sort() and Writeable::as_buffered() (#25583)
• Take task priority argument in parallelize_first_to_local (#25563)
• Rename URL_ENCODE_CHARSET to HIVE_ENCODE_CHARSET (#25554)
• Remove verbose prints on file opens (#25523)
• Remove some dead argminmax impl code (#25501)
• Take sync parameter in Writeable::close() (#25475)
• Fix unsoundness in ChunkedArray::{first, last} (#25449)
• Take &dyn Any instead of Box<dyn Any> in python object converters (#25421)
• Accept multiple files in pipe_with_schema (#25388)
• Add oneshot channel to polars-stream (#25378)
• Remove incorrect cast in reduce code (#25321)
• Clean up CSPE callsite (#25215)
• Move ewm variance code to polars-compute (#25188)
• Make pipe_with_schema work on Arced schema (#25155)
• Remove lower_ir conversion from Scan to InMemorySource (#25150)
• Add functions for scan_lines (#25136)
• Remove unused optimization_toggle (#25130)
• Support for named/anonymous aggregations (#25118)
• Remove old join projection pushdown logic (#25088)
• Remove unused row-count (#25080)
• Add IR for scan_lines (#25066)
• Add stateful EwmCov kernel (#25065)
• Move EwmMeanState to polars-compute (#25034)
• Move asof tolerance type coercion to IR conversion (#25033)
• Move supertype determination and casting to IR for date_range and related functions (#24084)

Thank you to all our contributors for making this release possible! @AndreaBozzo, @DannyStoll1, @EndPositive, @JakubValtar, @Jesse-Bakker, @Kevin-Patyk, @MarcoGorelli, @TNieuwdorp, @Voultapher, @alexander-beedie, @borchero, @c-peters, @cBournhonesque, @camriddell, @carnarez, @cmdlineluser, @coastalwhite, @cr7pt0gr4ph7, @davanstrien, @davidia, @dsprenkels, @etiennebacher, @feliblo, @guilhem-dvr, @itamarst, @jannickj, @jetuk, @kdn36, @lun3x, @marinegor, @mcrumiller, @nameexhaustion, @orlp, @pomo-mondreganto, @ritchie46, @vyasr, @wtn, and more!

🏆 Highlights

• Add Extension types (#25322)

✨ Enhancements

• Add SQL support for ROW_NUMBER, RANK, and DENSE_RANK functions (#25409)
• Add SQL support for named WINDOW references (#25400)
• Add BIT_NOT support to the SQL interface (#25094)
• Add LazyFrame.pivot (#25016)
• Add allow_empty flag to item (#25048)
• Add empty_as_null and keep_nulls flags to Expr.explode (#25289)
• Add empty_as_null and keep_nulls to {Lazy,Data}Frame.explode (#25369)
• Add having to group_by context (#23550)
• Add ignore_nulls to first / last (#25105)
• Add maintain_order to Expr.mode (#25377)
• Add quantile for missing temporals (#25464)
• Add leftmost option to str.replace_many / str.find_many / str.extract_many (#25398)
• Add strict parameter to pl.concat(how='horizontal') (#25452)
• Add support for Float16 dtype (#25185)
• Add unstable Schema.to_arrow (#25149)
• Allow Expr.rolling in aggregation contexts (#25258)
• Allow Expr.unique on List/Array with non-numeric types (#25285)
• Allow glimpse to return a DataFrame (#24803)
• Allow hash for all List dtypes (#25372)
• Allow implode and aggregation in aggregation context (#25357)
• Allow slice on scalar in aggregation context (#25358)
• Allow arbitrary Expressions in "subset" parameter of unique frame method (#25099)
• Allow arbitrary expressions as the Expr.rolling index_column (#25117)
• Allow bare .row on a single-row DataFrame, equivalent to .item on a single-element DataFrame (#25229)
• Allow elementwise Expr.over in aggregation context (#25402)
• Allow pl.Object in pivot value (#25533)
• Automatically Parquet dictionary encode floats (#25387)
• Display function of streaming physical plan map node (#25368)
• Documentation on Polars Cloud manifests (#25295)
• Expose and document pl.Categories (#25443)
• Expose fields for generating physical plan visualization data (#25562)
• Extend SQL UNNEST support to handle multiple array expressions (#25418)
• Improve SQL UNNEST behaviour (#22546)
• Improve error message on unsupported SQL subquery comparisons (#25135)
• Make DSL-hash skippable (#25140)
• Minor improvement for as_struct repr (#25529)
• Move GraphMetrics into StreamingQuery (#25310)
• Raise suitable error on non-integer "n" value for clear (#25266)
• Rewrite IR::Scan to IR::DataFrameScan in expand_datasets when applicable (#25106)
• Set polars/ user-agent (#25112)
• Streaming {Expr,LazyFrame}.rolling (#25058)
• Support BYTE_ARRAY backed Decimals in Parquet (#25076)
• Support ewm_var/std in streaming engine (#25109)
• Support unique_counts for all datatypes (#25379)
• Support additional forms of SQL "CREATE TABLE" statements (#25191)
• Support arbitrary expressions in SQL JOIN constraints (#25132)
• Support column-positional SQL "UNION" operations (#25183)
• Support decimals in search_sorted (#25450)
• Temporal quantile in rolling context (#25479)
• Use reference to Graph pipes when flushing metrics (#25442)

🚀 Performance improvements

• Add parquet prefiltering for string regexes (#25381)
• Add streaming native LazyFrame.group_by_dynamic (#25342)
• Add streaming sorted Group-By (#25013)
• Allow detecting plan sortedness in more cases (#25408)
• Don't recompute full rolling moment window when NaNs/nulls leave the window (#25078)
• Enable predicate expressions on unsigned integers (#25416)
• Fast find start window in group_by_dynamic with large offset (#25376)
• Faster kernels for rle_lengths (#25448)
• Fuse positive slice into streaming LazyFrame.rolling (#25338)
• Lazy gather for {forward,backward}_fill in group-by contexts (#25115)
• Mark Expr.reshape((-1,)) as row separable (#25326)
• Mark output of more non-order-maintaining ops as unordered (#25419)
• Optimize ipc stream read performance (#24671)
• Reduce HuggingFace API calls (#25521)
• Return references from aexpr_to_leaf_names_iter (#25319)
• Skip filtering scan IR if no paths were filtered (#25037)
• Use bitmap instead of Vec in first/last w. skip_nulls (#25318)
• Use fast path for agg_min/agg_max when nulls present (#25374)
• Use strong hash instead of traversal for CSPE equality (#25537)

🐞 Bug fixes

• Add .rolling_rank support for temporal types and pl.Boolean (#25509)
• Address issues with SQL OVER clause behaviour for window functions (#25249)
• Aggregation with drop_nulls on literal (#25356)
• Allow Null dtype values in scatter (#25245)
• Allow broadcast in group_by for ApplyExpr and BinaryExpr (#25053)
• Allow empty list in sort_by in list.eval context (#25481)
• Allow for negative time in group_by_dynamic iterator (#25041)
• Always respect return_dtype in map_elements and map_rows (#25504)
• AnyValue::to_physical for categoricals (#25341)
• Apply CSV dict overrides by name only (#25436)
• Block predicate pushdown when group_by key values are changed (#25032)
• Bugs in pl.from_repr with signed exponential floats and line wrapping (#25331)
• Correct drop_items for scalar input (#25351)
• Correct eq_missing for struct with nulls (#25363)
• Correct {first,last}_non_null if there are empty chunks (#25279)
• Correct handle requested stops in streaming shift (#25239)
• Correctly prune projected columns in hints (#25250)
• DSL_SCHEMA_HASH should not changed by line endings (#25123)
• Don't push down predicates passed inserted cache nodes (#25042)
• Don't quietly allow unsupported SQL SELECT clauses (#25282)
• Don't trigger DeprecationWarning from SQL "IN" constraints that use subqueries (#25111)
• Enhanced column resolution/tracking through multi-way SQL joins (#25181)
• Ensure SQL table alias resolution checks against CTE aliases on fallback (#25071)
• Ensure out-of-range integers and other edge case values don't give wrong results for index_of (#24369)
• Fix CSV select(len) off by 1 with comment prefix (#25069)
• Fix arr.{eval,agg} in aggregation context (#25390)
• Fix format_str in case of multiple chunks (#25162)
• Fix groups update on slices with different offsets (#25097)
• Fix assertion panic on group_by (#25179)
• Fix building polars-expr without timezones feature (#25254)
• Fix building polars-mem-engine with the async feature (#25300)
• Fix building polars-plan with features lazy,concat_str (but no strings) (#25306)
• Fix dictionary replacement error in write_ipc (#25497)
• Fix expr slice pushdown causing shape error on literals (#25485)
• Fix field metadata for nested categorical PyCapsule export (#25052)
• Fix group lengths check in sort_by with AggregatedScalar (#25503)
• Fix handling Null dtype in ApplyExpr on group_by (#25077)
• Fix incorrect .list.eval after slicing operations (#25540)
• Fix incorrect reshape on sliced lists (#25139)
• Fix length preserving check for eval expressions in streaming engine (#25294)
• Fix occurence of exact matches of .join_asof(strategy="nearest", allow_exact_matches=False, ...) (#25506)
• Fix off-by-one bug in ColumnPredicates generation for inequalities operating on integer columns (#25412)
• Fix panic if scan predicate produces 0 length mask (#25089)
• Fix panic in dt.truncate for invalid duration strings (#25124)
• Fix panic in is_between support in streaming Parquet predicate push down (#25476)
• Fix panic when using struct field as join key (#25059)
• Fix serialization of lazyframes containing huge tables (#25190)
• Fix single-column CSV header duplication with leading empty lines (#25186)
• Fix small bug with PyExpr to PyObject conversion (#25265)
• Group-By aggregation problems caused by AmortSeries (#25043)
• Handle some unusual pl.col.<colname> edge-cases (#25153)
• Incorrect result in aggregated first/last with ignore_nulls (#25414)
• Incorrect results for aggregated {n_,}unique on bools (#25275)
• Invert drop_nans filtering in group-by context (#25146)
• Make str.json_decode output deterministic with lists (#25240)
• Mark {forward,backward}_fill as length_preserving (#25352)
• Minor improvement to internal is_pycapsule utility function (#25073)
• Nested dtypes in streaming first_non_null/last_non_null (#25375)
• Nested dtypes in streaming first/last (#25298)
• Panic exception when calling Expr.rolling in .over (#25283)
• Panic in group_by_dynamic with group_by and multiple chunks (#25075)
• Parquet is_in for mixed validity pages (#25313)
• Prevent panic when joining sorted LazyFrame with itself (#25453)
• Raise error for all/any on list instead of panic (#25018)
• Raise error on out-of-range dates in temporal operations (#25471)
• Remove Expr casts in pl.lit invocations (#25373)
• Resolve edge-case with SQL aggregates that have the same name as one of the "GROUP BY" keys (#25362)
• Return the correct string-case Expr reprs (#25101)
• Reverse on chunked struct (#25281)
• Revert pl.format behavior with nulls (#25370)
• Rolling mean/median for temporals (#25512)
• Run async DB queries with regular asyncio if not inside a running loop (#25268)
• SQL "NATURAL" joins should coalesce the key columns (#25353)
• Schema mismatch with list.agg, unique and scalar (#25348)
• Solve multiple issues relating to arena mutation in SQL subqueries (#25110)
• Strict conversion AnyValue to Struct (#25536)
• Support "index" as column name in group_by iterator (#25138)
• Support AggregatedList in list.{eval,agg} context (#25385)
• The SQL interface should use logical, not bitwise, behaviour for unary "NOT" operator (#25091)
• Unique key names in streaming sort/top_k (#25082)
• Unique on literal in aggregation context (#25359)
• Use (i64, u64) for VisualizationData (offset, length) slices (#25203)
• Use Cargo.template.toml to prevent git dependencies from using template (#25392)
• Validate list.slice parameters are not lists (#25458)
• Wide-table join performance regression (#25222)

📖 Documentation

• Add Extension and BaseExtension to doc index (#25444)
• Add LazyFrame.pivot to reference guide (#25482)
• Add having API references (#25428)
• Add docstring example showing str.slice taking Expression params (#25461)
• Add polars-on-premise documentation (#25431)
• Clarify bitwise behaviour of and_, or_, and not_ Expressions on integer columns (#25092)
• Correct link to datetime_range instead of date_range in resampling page (#25532)
• Deprecate Categorical functions for lexical ordering and local checks (#25514)
• Document schema parameter in meta methods (#25543)
• Explain aggregation & sorting of lists (#25260)
• Fix LanceDB URL (#25198)
• Fix incorrect 'bitwise' in any_horizontal/all_horizontal docstring (#25469)
• Fix link errors reported by markdown-link-check (#25314)
• Fix non-existent replace_all reference in replace docs (#25161)
• Fix source path (#25170)
• Fix typo in public dataset URL (#25044)
• Mention Narwhals in ecosystem page (#25100)
• Remove lzo from parquet write options (#25522)
• Update LazyFrame.collect_schema docstring (#25508)
• Update LazyFrame.remote signature (#25175)
• Update on-premise documentation (#25489)
• Update user guide for QueryProgress rename to QueryProfile (#25195)

🧪 Tests

• Add assert_sql_matches coverage for SQL "DISTINCT" and "DISTINCT ON" syntax (#25440)
• Add reliable test for pl.format on multiple chunks (#25164)
• Add test for unique with column subset (#25241)
• Better coverage for group_by aggregations (#25290)
• Test for group_by(...).having(...) (#25430)

🔧 CI

• Automatically label pull requests that change the DSL (#25177)
• Avoid relabelling changes-dsl on every commit (#25216)
• Print expected DSL schema hashes if mismatched (#25526)
• Skip existing files in pypi upload (#25576)

🏗️ Build system

• Fix make fmt and make lint commands (#25200)
• Make building the docs on macOS more reliable (#25095)

🛠️ Other improvements

• Add Final type-qualifier to module-level constants (#25556)
• Add proptest AnyValue strategies (#25510)
• Add proptest DataFrame strategy (#25446)
• Add proptest strategies for Series logical types (#24849)
• Add proptest strategies for Series nested types (#25220)
• Add some cleanup (#25445)
• Add toolchain file to runtimes for sdist (#25311)
• Enable more streaming tests (#25364)
• Fix --uv argument for benchmark-remote (#25513)
• Fix Decimal precision annotation (#25227)
• Fix feature gating TZ_AWARE_RE again (#25493)
• Fix template path in release-python workflow (#25565)
• Fix typo in CI release workflow (#25309)
• Make python docs build again (#25165)
• Remove Column::Partitioned (#25324)
• Remove debug file write from test suite (#25393)
• Remove unused import (#25365)
• Run maturin with --uv option (#25490)
• Silence unused mut warning (#25093)
• Skip rust integration tests for coverage in CI (#25558)
• Update markdown link checker (#25201)
• Update toolchain (#25007)
• Update versions (#25141)
• Upgrade to schemars 0.9.0 (#25158)
• Upgraded ruff and typos and made the necessary lint updates (#25196)

♻️ Refactoring

• Accept multiple files in pipe_with_schema (#25388)
• Add IR for scan_lines (#25066)
• Add ElementExpr for _eval expressions (#25199)
• Add asserts and tests for list.eval on multiple chunks with slicing (#25559)
• Add functions for scan_lines (#25136)
• Add oneshot channel to polars-stream (#25378)
• Add stateful EwmCov kernel (#25065)
• Change group length mismatch error to ShapeError (#25004)
• Clean up CSPE callsite (#25215)
• Directly take CloudScheme in parse_cloud_options (#25304)
• Disable recursive CSPE for now (#25085)
• Dispatch Series.set to zip_with_same_dtype (#25327)
• Fix unsoundness in ChunkedArray::{first, last} (#25449)
• Make pipe_with_schema work on Arced schema (#25155)
• Move EwmMeanState to polars-compute (#25034)
• Move asof tolerance type coercion to IR conversion (#25033)
• Move ewm variance code to polars-compute (#25188)
• Move supertype determination and casting to IR for date_range and related functions (#24084)
• Refactor dt_range functions (#25225)
• Refactor sink IR (#25308)
• Remove ClosableFile (#25330)
• Remove PyPartitioning (#25303)
• Remove aggregation context Context (#25424)
• Remove incorrect cast in reduce code (#25321)
• Remove lower_ir conversion from Scan to InMemorySource (#25150)
• Remove old join projection pushdown logic (#25088)
• Remove some dead argminmax impl code (#25501)
• Remove unused optimization_toggle (#25130)
• Remove unused row-count (#25080)
• Remove verbose prints on file opens (#25523)
• Rename URL_ENCODE_CHARSET to HIVE_ENCODE_CHARSET (#25554)
• Simplify sink parameter passing from Python (#25302)
• Support for named/anonymous aggregations (#25118)
• Take &dyn Any instead of Box<dyn Any> in python object converters (#25421)
• Take sync parameter in Writeable::close (#25475)
• Update partitioned sink IR (#25524)
• Use dedicated runtime packages from template (#25284)

Thank you to all our contributors for making this release possible! @AndreaBozzo, @DannyStoll1, @EndPositive, @JakubValtar, @Jesse-Bakker, @Kevin-Patyk, @MarcoGorelli, @TNieuwdorp, @alexander-beedie, @borchero, @c-peters, @cBournhonesque, @carnarez, @cmdlineluser, @coastalwhite, @cr7pt0gr4ph7, @davanstrien, @dsprenkels, @etiennebacher, @feliblo, @itamarst, @jannickj, @jetuk, @kdn36, @lun3x, @marinegor, @mcrumiller, @nameexhaustion, @orlp, @ritchie46, @vyasr, @wtn and more!

• Fix incorrect drop_nans() result when used in group_by() / over() (https://github.com/pola-rs/polars/pull/25146)
• Fix handling Null dtype in ApplyExpr on group_by(https://github.com/pola-rs/polars/pull/25077)
• Fix assertion panic on group_by (https://github.com/pola-rs/polars/pull/25179)
• Fix Wide-table join performance regression (https://github.com/pola-rs/polars/pull/25222)

Thank you to all our contributors for making this release possible! @coastalwhite, @kdn36, @nameexhaustion and @ritchie46

🚀 Performance improvements

• Don't recompute full rolling moment window when NaNs/nulls leave the window (#25078)
• Skip filtering scan IR if no paths were filtered (#25037)
• Optimize ipc stream read performance (#24671)

✨ Enhancements

• Support BYTE_ARRAY backed Decimals in Parquet (#25076)
• Allow glimpse to return a DataFrame (#24803)
• Add allow_empty flag to item (#25048)

🐞 Bug fixes

• The SQL interface should use logical, not bitwise, behaviour for unary "NOT" operator (#25091)
• Fix panic if scan predicate produces 0 length mask (#25089)
• Ensure SQL table alias resolution checks against CTE aliases on fallback (#25071)
• Panic in group_by_dynamic with group_by and multiple chunks (#25075)
• Minor improvement to internal is_pycapsule utility function (#25073)
• Fix panic when using struct field as join key (#25059)
• Allow broadcast in group_by for ApplyExpr and BinaryExpr (#25053)
• Fix field metadata for nested categorical PyCapsule export (#25052)
• Block predicate pushdown when group_by key values are changed (#25032)
• Group-By aggregation problems caused by AmortSeries (#25043)
• Don't push down predicates passed inserted cache nodes (#25042)
• Allow for negative time in group_by_dynamic iterator (#25041)

📖 Documentation

• Fix typo in public dataset URL (#25044)

🛠️ Other improvements

• Disable recursive CSPE for now (#25085)
• Change group length mismatch error to ShapeError (#25004)
• Update toolchain (#25007)

Thank you to all our contributors for making this release possible! @Kevin-Patyk, @Liyixin95, @alexander-beedie, @coastalwhite, @kdn36, @nameexhaustion, @orlp, @r-brink, @ritchie46 and @stijnherfst

🏆 Highlights

• Stabilize decimal (#25020)

🚀 Performance improvements

• Bump foldhash to 0.2.0 and hashbrown to 0.16.0 (#25014)
• Lower unique to native group-by and speed up n_unique in group-by context (#24976)
• Better parallelize take{_slice,}_unchecked (#24980)
• Implement native skew and kurtosis in group-by context (#24961)
• Use native group-by aggregations for bitwise_* operations (#24935)
• Address group_by_dynamic slowness in sparse data (#24916)
• Push filters to PyIceberg (#24910)
• Native filter/drop_nulls/drop_nans in group-by context (#24897)
• Implement cumulative_eval using the group-by engine (#24889)
• Prevent generation of copies of Dataframes in DslPlan serialization (#24852)
• Implement native null_count, any and all group-by aggregations (#24859)
• Speed up reverse in group-by context (#24855)
• Prune unused categorical values when exporting to arrow/parquet/IPC/pickle (#24829)
• Don't check duplicates on streaming simple projection in release mode (#24830)
• Lower approx_n_unique to the streaming engine (#24821)
• Duration/interval string parsing optimisation (2-5x faster) (#24771)
• Use native reducer for first/last on Decimals, Categoricals and Enums (#24786)
• Implement indexed method for BitMapIter::nth (#24766)
• Pushdown slices on plans within unions (#24735)

✨ Enhancements

• Stabilize decimal (#25020)
• Support ewm_mean() in streaming engine (#25003)
• Improve row-count estimates (#24996)
• Remove filtered scan paths in IR when possible (#24974)
• Introduce remote Polars MCP server (#24977)
• Allow local scans on polars cloud (configurable) (#24962)
• Add Expr.item to strictly extract a single value from an expression (#24888)
• Add environment variable to roundtrip empty struct in Parquet (#24914)
• Fast-count for scan_iceberg().select(len()) (#24602)
• Add glob parameter to scan_ipc (#24898)
• Prevent generation of copies of Dataframes in DslPlan serialization (#24852)
• Add list.agg and arr.agg (#24790)
• Implement {Expr,Series}.rolling_rank() (#24776)
• Don't require PyArrow for read_database_uri if ADBC engine version supports PyCapsule interface (#24029)
• Make Series init consistent with DataFrame init for string values declared with temporal dtype (#24785)
• Support MergeSorted in CSPE (#24805)
• Duration/interval string parsing optimisation (2-5x faster) (#24771)
• Recursively apply CSPE (#24798)
• Add streaming engine per-node metrics (#24788)
• Add arr.eval (#24472)
• Drop PyArrow requirement for non-batched usage of read_database with the ADBC engine and support iter_batches with the ADBC engine (#24180)
• Improve rolling_(sum|mean) accuracy (#24743)
• Add separator to {Data,Lazy}Frame.unnest (#24716)
• Add union() function for unordered concatenation (#24298)
• Add name.replace to the set of column rename options (#17942)
• Support np.ndarray -> AnyValue conversion (#24748)
• Allow duration strings with leading "+" (#24737)
• Drop now-unnecessary post-init "schema_overrides" cast on DataFrame load from list of dicts (#24739)
• Add support for UInt128 to pyo3-polars (#24731)

🐞 Bug fixes

• Re-enable CPU feature check before import (#25010)
• Implement read_excel workaround for fastexcel/calamine issue loading a column subset from a named table (#25012)
• Correctness any(ignore_nulls) and OOB in all (#25005)
• Streaming any/all with ignore_nulls=False (#25008)
• Fix incorrect join_asof on a casted expression (#25006)
• Optimize memory on rolling groups in ApplyExpr (#24709)
• Fallback Pyarrow scan to in-memory engine (#24991)
• Make Operator::swap_operands return correct operators for Plus, Minus, Multiply and Divide (#24997)
• Capitalize letters after numbers in to_titlecase (#24993)
• Preserve null values in pct_change (#24952)
• Raise length mismatch on over with sliced groups (#24887)
• Check duplicate name in transpose (#24956)
• Follow Kleene logic in any / all for group-by (#24940)
• Do not optimize cross join to iejoin if order maintaining (#24950)
• Fix typing of scan_parquet partially unknown (#24928)
• Properly release the GIL for read_parquet_metadata (#24922)
• Broadcast partition_by columns in over expression (#24874)
• Clear index cache on stacked df.filter expressions (#24870)
• Fix 'explode' mapping strategy on scalar value (#24861)
• Fix repeated with_row_index() after scan() silently ignored (#24866)
• Correctly return min and max for enums in groupby aggregation (#24808)
• Refactor BinaryExpr in group_by dispatch logic (#24548)
• Fix aggstate for gather (#24857)
• Keep scalars for length preserving functions in group_by (#24819)
• Have range feature depend on dtype-array feature (#24853)
• Fix duplicate select panic (#24836)
• Inconsistency of list.sum() result type with None values (#24476)
• Division by zero in Expr.dt.truncate (#24832)
• Potential deadlock in __arrow_c_stream__ (#24831)
• Allow double aggregations in group-by contexts (#24823)
• Series.shrink_dtype for i128/u128 (#24833)
• Fix dtype in EvalExpr (#24650)
• Allow aggregations on AggState::LiteralScalar (#24820)
• Dispatch to group_aware for fallible expressions with masked out elements (#24815)
• Fix error for arr.sum() on small integer Array dtypes containing nulls (#24478)
• Fix regression on write_database() to Snowflake due to unsupported string view type (#24622)
• Fix XOR did not follow kleene when one side is unit-length (#24810)
• Make Series init consistent with DataFrame init for string values declared with temporal dtype (#24785)
• Incorrect precision in Series.str.to_decimal (#24804)
• Use overlapping instead of rolling (#24787)
• Fix iterable on dynamic_group_by and rolling object (#24740)
• Use Kahan summation for in-memory groupby sum/mean (#24774)
• Release GIL in PythonScan predicate evaluation (#24779)
• Type error in bitmask::nth_set_bit_u64 (#24775)
• Add Expr.sign for Decimal datatype (#24717)
• Correct str.replace with missing pattern (#24768)
• Ensure schema_overrides is respected when loading iterable row data (#24721)
• Support decimal_comma on Decimal type in write_csv (#24718)

📖 Documentation

• Introduce remote Polars MCP server (#24977)
• Add {arr,list}.agg API references (#24970)
• Support LLM in docs (#24958)
• Update Cloud docs with correct fn argument order (#24939)
• Update name.replace examples (#24941)
• Add i128 and u128 features to user guide (#24938)
• Add partitioning examples for sink_* methods (#24918)
• Add more {unique,value}_counts examples (#24927)
• Indent the versionchanged (#24783)
• Relax fsspec wording (#24881)
• Add pl.field into the api docs (#24846)
• Fix duplicated article in SECURITY.md (#24762)
• Document output name determination in when/then/otherwise (#24746)
• Specify that precision=None becomes 38 for Decimal (#24742)
• Mention polars[rt64] and polars[rtcompat] instead of u64-idx and lts-cpu (#24749)
• Fix source mapping (#24736)

📦 Build system

• Ensure build_feature_flags.py is included in artifact (#25024)
• Update pyo3 and numpy crates to version 0.26 (#24760)

🛠️ Other improvements

• Fix benchmark ci (#25019)
• Fix non-deterministic test (#25009)
• Fix makefile arch detection (#25011)
• Make LazyFrame.set_sorted into a FunctionIR::Hint (#24981)
• Remove symbolic links (#24982)
• Deprecate Expr.agg_groups() and pl.groups() (#24919)
• Dispatch to no-op rayon thread-pool from streaming (#24957)
• Unpin pydantic (#24955)
• Ensure safety of scan fast-count IR lowering in streaming (#24953)
• Re-use iterators in set_ operations (#24850)
• Remove GroupByPartitioned and dispatch to streaming engine (#24903)
• Turn element() into {A,}Expr::Element (#24885)
• Pass ScanOptions to new_from_ipc (#24893)
• Update tests to be index type agnostic (#24891)
• Unset Context in Window expression (#24875)
• Fix failing delta test (#24867)
• Move FunctionExpr dispatch from plan to expr (#24839)
• Fix SQL test giving wrong error message (#24835)
• Consolidate dtype paths in ApplyExpr (#24825)
• Add days_in_month to documentation (#24822)
• Enable ruff D417 lint (#24814)
• Turn pl.format into proper elementwise expression (#24811)
• Fix remote benchmark by no-longer saving builds (#24812)
• Refactor ApplyExpr in group_by context on multiple inputs (#24520)
• IR text plan graph generator (#24733)
• Temporarily pin pydantic to fix CI (#24797)
• Extend and rename rolling groups to overlapping (#24577)
• Refactor DataType proptest strategies (#24763)
• Add union to documentation (#24769)

Thank you to all our contributors for making this release possible! @EndPositive, @EnricoMi, @JakubValtar, @Kevin-Patyk, @MarcoGorelli, @Object905, @alexander-beedie, @borchero, @carnarez, @cmdlineluser, @coastalwhite, @craigalodon, @dsprenkels, @eitsupi, @etrotta, @henryharbeck, @jordanosborn, @kdn36, @math-hiyoko, @mjanssen, @nameexhaustion, @orlp, @pavelzw, @r-brink, @ritchie46, @thomasjpfan and @williambdean

🚀 Performance improvements

• Address group_by_dynamic slowness in sparse data (#24916)
• Push filters to PyIceberg (#24910)
• Native filter/drop_nulls/drop_nans in group-by context (#24897)
• Implement cumulative_eval using the group-by engine (#24889)
• Prevent generation of copies of Dataframes in DslPlan serialization (#24852)
• Implement native null_count, any and all group-by aggregations (#24859)
• Speed up reverse in group-by context (#24855)
• Prune unused categorical values when exporting to arrow/parquet/IPC/pickle (#24829)
• Don't check duplicates on streaming simple projection in release mode (#24830)
• Lower approx_n_unique to the streaming engine (#24821)
• Duration/interval string parsing optimisation (2-5x faster) (#24771)
• Use native reducer for first/last on Decimals, Categoricals and Enums (#24786)
• Implement indexed method for BitMapIter::nth (#24766)
• Pushdown slices on plans within unions (#24735)

✨ Enhancements

• Add environment variable to roundtrip empty struct in Parquet (#24914)
• Fast-count for scan_iceberg().select(len()) (#24602)
• Add glob parameter to scan_ipc (#24898)
• Prevent generation of copies of Dataframes in DslPlan serialization (#24852)
• Add list.agg and arr.agg (#24790)
• Implement {Expr,Series}.rolling_rank() (#24776)
• Don't require PyArrow for read_database_uri if ADBC engine version supports PyCapsule interface (#24029)
• Make Series init consistent with DataFrame init for string values declared with temporal dtype (#24785)
• Support MergeSorted in CSPE (#24805)
• Duration/interval string parsing optimisation (2-5x faster) (#24771)
• Recursively apply CSPE (#24798)
• Add streaming engine per-node metrics (#24788)
• Add arr.eval (#24472)
• Drop PyArrow requirement for non-batched usage of read_database with the ADBC engine and support iter_batches with the ADBC engine (#24180)
• Improve rolling_(sum|mean) accuracy (#24743)
• Add separator to {Data,Lazy}Frame.unnest (#24716)
• Add union() function for unordered concatenation (#24298)
• Add name.replace to the set of column rename options (#17942)
• Support np.ndarray -> AnyValue conversion (#24748)
• Allow duration strings with leading "+" (#24737)
• Drop now-unnecessary post-init "schema_overrides" cast on DataFrame load from list of dicts (#24739)
• Add support for UInt128 to pyo3-polars (#24731)

🐞 Bug fixes

• Properly release the GIL for read_parquet_metadata (#24922)
• Broadcast partition_by columns in over expression (#24874)
• Clear index cache on stacked df.filter expressions (#24870)
• Fix 'explode' mapping strategy on scalar value (#24861)
• Fix repeated with_row_index() after scan() silently ignored (#24866)
• Correctly return min and max for enums in groupby aggregation (#24808)
• Refactor BinaryExpr in group_by dispatch logic (#24548)
• Fix aggstate for gather (#24857)
• Keep scalars for length preserving functions in group_by (#24819)
• Have range feature depend on dtype-array feature (#24853)
• Fix duplicate select panic (#24836)
• Inconsistency of list.sum() result type with None values (#24476)
• Division by zero in Expr.dt.truncate (#24832)
• Potential deadlock in __arrow_c_stream__ (#24831)
• Allow double aggregations in group-by contexts (#24823)
• Series.shrink_dtype for i128/u128 (#24833)
• Fix dtype in EvalExpr (#24650)
• Allow aggregations on AggState::LiteralScalar (#24820)
• Dispatch to group_aware for fallible expressions with masked out elements (#24815)
• Fix error for arr.sum() on small integer Array dtypes containing nulls (#24478)
• Fix regression on write_database() to Snowflake due to unsupported string view type (#24622)
• Fix XOR did not follow kleene when one side is unit-length (#24810)
• Make Series init consistent with DataFrame init for string values declared with temporal dtype (#24785)
• Incorrect precision in Series.str.to_decimal (#24804)
• Use overlapping instead of rolling (#24787)
• Fix iterable on dynamic_group_by and rolling object (#24740)
• Use Kahan summation for in-memory groupby sum/mean (#24774)
• Release GIL in PythonScan predicate evaluation (#24779)
• Type error in bitmask::nth_set_bit_u64 (#24775)
• Add Expr.sign for Decimal datatype (#24717)
• Correct str.replace with missing pattern (#24768)
• Ensure schema_overrides is respected when loading iterable row data (#24721)
• Support decimal_comma on Decimal type in write_csv (#24718)

📖 Documentation

• Add partitioning examples for sink_* methods (#24918)
• Add more {unique,value}_counts examples (#24927)
• Indent the versionchanged (#24783)
• Relax fsspec wording (#24881)
• Add pl.field into the api docs (#24846)
• Fix duplicated article in SECURITY.md (#24762)
• Document output name determination in when/then/otherwise (#24746)
• Specify that precision=None becomes 38 for Decimal (#24742)
• Mention polars[rt64] and polars[rtcompat] instead of u64-idx and lts-cpu (#24749)
• Fix source mapping (#24736)

📦 Build system

• Update pyo3 and numpy crates to version 0.26 (#24760)

🛠️ Other improvements

• Re-use iterators in set_ operations (#24850)
• Remove GroupByPartitioned and dispatch to streaming engine (#24903)
• Turn element() into {A,}Expr::Element (#24885)
• Pass ScanOptions to new_from_ipc (#24893)
• Update tests to be index type agnostic (#24891)
• Unset Context in Window expression (#24875)
• Fix failing delta test (#24867)
• Move FunctionExpr dispatch from plan to expr (#24839)
• Fix SQL test giving wrong error message (#24835)
• Consolidate dtype paths in ApplyExpr (#24825)
• Add days_in_month to documentation (#24822)
• Enable ruff D417 lint (#24814)
• Turn pl.format into proper elementwise expression (#24811)
• Fix remote benchmark by no-longer saving builds (#24812)
• Refactor ApplyExpr in group_by context on multiple inputs (#24520)
• IR text plan graph generator (#24733)
• Temporarily pin pydantic to fix CI (#24797)
• Extend and rename rolling groups to overlapping (#24577)
• Refactor DataType proptest strategies (#24763)
• Add union to documentation (#24769)

Thank you to all our contributors for making this release possible! @JakubValtar, @Kevin-Patyk, @MarcoGorelli, @Object905, @alexander-beedie, @borchero, @cmdlineluser, @coastalwhite, @craigalodon, @dsprenkels, @eitsupi, @etrotta, @henryharbeck, @jordanosborn, @kdn36, @math-hiyoko, @nameexhaustion, @orlp, @pavelzw, @ritchie46, @thomasjpfan and @williambdean

🏆 Highlights

• Add LazyFrame.{sink,collect}_batches (#23980)
• Deterministic import order for Python Polars package variants (#24531)

🚀 Performance improvements

• Optimize gather_every(n=1) to slice (#24704)
• Lower null count to streaming engine (#24703)
• Native streaming gather_every (#24700)
• Pushdown filter with strptime if input is literal (#24694)
• Avoid copying expanded paths (#24669)
• Relax filter expr ordering (#24662)
• Remove unnecessary groups call in aggregated (#24651)
• Skip files in scan_iceberg with filter based on metadata statistics (#24547)
• Push row_index predicate for all scan types (#24537)
• Perform integer in-filtering for Parquet inequality predicates (#24525)
• Stop caching Parquet metadata after 8 files (#24513)
• Native streaming .mode() expression (#24459)

✨ Enhancements

• Implement maintain_order for cross join (#24665)
• Add support to output dt.total_{}() duration values as fractionals (#24598)
• Avoid forcing a pyarrow dependency in read_excel when using the default "calamine" engine (#24655)
• Support scanning from file:/path URIs (#24603)
• Log which file the schema was sourced from, and which file caused an extra column error (#24621)
• Add LazyFrame.{sink,collect}_batches (#23980)
• Deterministic import order for Python Polars package variants (#24531)
• Add support to display lazy query plan in marimo notebooks without needing to install matplotlib or mermaid (#24540)
• Add unstable hidden_file_prefix parameter to scan_parquet (#24507)
• Use fixed-scale Decimals (#24542)
• Add support for unsigned 128-bit integers (#24346)
• Add unstable pl.Config.set_default_credential_provider (#24434)
• Roundtrip BinaryOffset type through Parquet (#24344)
• Add opt-in unstable functionality to load interval types as Struct (#24320)
• Support reading parquet metadata from cloud storage (#24443)
• Add user guide section on AWS role assumption (#24421)
• Support unique / n_unique / arg_unique for array columns (#24406)

🐞 Bug fixes

• Removing dots after noqa comments (#24722)
• Parse Decimal with comma as decimal separator in CSV (#24685)
• Make Categories pickleable (#24691)
• Shift on array within list (#24678)
• Fix handling of AggregatedScalar in ApplyExpr single input (#24634)
• Support reading of mixed compressed/uncompressed IPC buffers (#24674)
• Overflow in slice-slice optimization (#24658)
• Package discovery for setuptools (#24656)
• Add type assertion to prevent out-of-bounds in GenericFirstLastGroupedReduction (#24590)
• Remove inclusion of polars dir in runtime sdist/wheel (#24654)
• Method dt.month_end was unnecessarily raising when the month-start timestamp was ambiguous (#24647)
• Widen from_dicts to Iterable[Mapping[str, Any]] (#24584)
• Fix unsupported arrow type Dictionary error in scan_iceberg() (#24573)
• Raise Exception instead of panic when unnest on non-struct column (#24471)
• Include missing feature dependency from polars-stream/diff to polars-plan/abs (#24613)
• Newline escaping in streaming show_graph (#24612)
• Do not allow inferring (-1) the dimension on any Expr.reshape dimension except the first (#24591)
• Sink batches early stop on in-memory engine (#24585)
• More precisely model expression ordering requirements (#24437)
• Panic in zero-weight rolling mean/var (#24596)
• Decimal <-> literal arithmetic supertype rules (#24594)
• Match various aggregation return types in the streaming engine with the in-memory engine (#24501)
• Validate list type for list expressions in planner (#24589)
• Fix scan_iceberg() storage options not taking effect (#24574)
• Have log() prioritize the leftmost dtype for its output dtype (#24581)
• CSV pl.len() was incorrect (#24587)
• Add support for float inputs for duration types (#24529)
• Roundtrip empty string through hive partitioning (#24546)
• Fix potential OOB writes in unaligned IPC read (#24550)
• Fix regression error when scanning AWS presigned URL (#24530)
• Make PlPath::join for cloud paths replace on absolute paths (#24514)
• Correct dtype for cum_agg in streaming engine (#24510)
• Restore support for np.datetime64() in pl.lit() (#24527)
• Ignore Iceberg list element ID if missing (#24479)
• Fix panic on streaming full join with coalesce (#23409)
• Fix AggState on all_literal in BinaryExpr (#24461)
• Show IR sort options in explain (#24465)
• Benchmark CI import (#24463)
• Fix schema on ApplyExpr with single row literal in agg context (#24422)
• Fix planner schema for dividing pl.Float32 by int (#24432)
• Fix panic scanning from AWS legacy global endpoint URL (#24450)
• Fix iterable_to_pydf(..., infer_schema_length=None) to scan all data (#23405)
• Do not propagate struct of nulls with null (#24420)
• Be stricter with invalid NDJSON input when ignore_errors=False (#24404)
• Implement approx_n_unique for temporal dtypes and Null (#24417)

📖 Documentation

• Add default parquet compression levels (#24686)
• Fix syntax error in data-types-and-structures.md (#24606)
• Rename avg_birthday -> avg_age in examples aggregation (#23726)
• Update Polars Cloud user guide (#24366)
• Fix typo in set_expr_depth_warning docstring (#24427)

📦 Build system

• Python pre-release 1.34.0b5 (#24699)
• Use cargo-run to call dsl-schema script (#24607)

🛠️ Other improvements

• Removing dots after noqa comments (#24722)
• Make test_multiple_sorting_columns test runnable (#24719)
• Remove {Upper,Lower}Bound expressions in IR (#24701)
• Fix Makefile uv pip option syntax (#24711)
• Add egg-info to gitignore (#24712)
• Restructure python project directories again (#24676)
• Use IR for polars-expr output field resolution (#24661)
• Remove dist/ from release python workflow (#24639)
• Escape sed ampersand in release script (#24631)
• Remove PyOdide from release for now (#24630)
• Fix sed in-place in release script (#24628)
• Release script pyodide wheel (#24627)
• Release script pyodide wheel (#24626)
• Update release script for runtimes (#24610)
• Remove unused UnknownKind::Ufunc (#24614)
• Use cargo-run to call dsl-schema script (#24607)
• Cleanup and prepare to_field for element and struct field context (#24592)
• Resolve nightly clippy hints (#24593)
• Rename pl.dependencies to pl._dependencies (#24595)
• More release scripting (#24582)
• Again a minor fix for the setup script (#24580)
• Minor fix in release script (#24579)
• Correct release python beta version check (#24578)
• Python dependency failure (#24576)
• Always install yq (#24570)
• Deterministic import order for Python Polars package variants (#24531)
• Check Arrow FFI pointers with an assert (#24564)
• Add a couple of missing type definitions in python (#24561)
• Fix quickstart example in Polars Cloud user guide (#24554)
• Add implementations for loading min/max statistics for Iceberg (#24496)
• Update versions (#24508)
• Add additional unit tests for pl.concat (#24487)
• Refactor parametric tests for as_struct on aggstates (#24493)
• Use PlanCallback in name.map_* (#24484)
• Pin xlsvwriter to 3.2.5 or before (#24485)
• Add dataclass to hold resolved iceberg scan data (#24418)
• Fix iceberg test failure in CI (#24456)
• Move CompressionUtils to polars-utils (#24430)
• Update github template to dispatch to cloud client (#24416)

Thank you to all our contributors for making this release possible! @DeflateAwning, @Gusabary, @JakubValtar, @Kevin-Patyk, @MarcoGorelli, @Matt711, @alexander-beedie, @alonsosilvaallende, @andreseje, @borchero, @c-peters, @camriddell, @coastalwhite, @dangotbanned, @deanm0000, @dongchao-1, @dsprenkels, @eitsupi, @itamarst, @jan-krueger, @joshuamarkovic, @juansolm, @kdn36, @moizescbf, @nameexhaustion, @orlp, @r-brink, @ritchie46 and @stijnherfst

🏆 Highlights

• Add LazyFrame.{sink,collect}_batches (#23980)
• Deterministic import order for Python Polars package variants (#24531)

🚀 Performance improvements

• Pushdown filter with strptime if input is literal (#24694)
• Avoid copying expanded paths (#24669)
• Relax filter expr ordering (#24662)
• Remove unnecessary groups call in aggregated (#24651)
• Skip files in scan_iceberg with filter based on metadata statistics (#24547)
• Push row_index predicate for all scan types (#24537)
• Perform integer in-filtering for Parquet inequality predicates (#24525)
• Stop caching Parquet metadata after 8 files (#24513)
• Native streaming .mode() expression (#24459)

✨ Enhancements

• Implement maintain_order for cross join (#24665)
• Add support to output dt.total_{}() duration values as fractionals (#24598)
• Avoid forcing a pyarrow dependency in read_excel when using the default "calamine" engine (#24655)
• Support scanning from file:/path URIs (#24603)
• Log which file the schema was sourced from, and which file caused an extra column error (#24621)
• Add LazyFrame.{sink,collect}_batches (#23980)
• Deterministic import order for Python Polars package variants (#24531)
• Add support to display lazy query plan in marimo notebooks without needing to install matplotlib or mermaid (#24540)
• Add unstable hidden_file_prefix parameter to scan_parquet (#24507)
• Use fixed-scale Decimals (#24542)
• Add support for unsigned 128-bit integers (#24346)
• Add unstable pl.Config.set_default_credential_provider (#24434)
• Roundtrip BinaryOffset type through Parquet (#24344)
• Add opt-in unstable functionality to load interval types as Struct (#24320)
• Support reading parquet metadata from cloud storage (#24443)
• Add user guide section on AWS role assumption (#24421)
• Support unique / n_unique / arg_unique for array columns (#24406)

🐞 Bug fixes

• Make Categories pickleable (#24691)
• Shift on array within list (#24678)
• Fix handling of AggregatedScalar in ApplyExpr single input (#24634)
• Support reading of mixed compressed/uncompressed IPC buffers (#24674)
• Overflow in slice-slice optimization (#24658)
• Package discovery for setuptools (#24656)
• Add type assertion to prevent out-of-bounds in GenericFirstLastGroupedReduction (#24590)
• Remove inclusion of polars dir in runtime sdist/wheel (#24654)
• Method dt.month_end was unnecessarily raising when the month-start timestamp was ambiguous (#24647)
• Widen from_dicts to Iterable[Mapping[str, Any]] (#24584)
• Fix unsupported arrow type Dictionary error in scan_iceberg() (#24573)
• Raise Exception instead of panic when unnest on non-struct column (#24471)
• Include missing feature dependency from polars-stream/diff to polars-plan/abs (#24613)
• Newline escaping in streaming show_graph (#24612)
• Do not allow inferring (-1) the dimension on any Expr.reshape dimension except the first (#24591)
• Sink batches early stop on in-memory engine (#24585)
• More precisely model expression ordering requirements (#24437)
• Panic in zero-weight rolling mean/var (#24596)
• Decimal <-> literal arithmetic supertype rules (#24594)
• Match various aggregation return types in the streaming engine with the in-memory engine (#24501)
• Validate list type for list expressions in planner (#24589)
• Fix scan_iceberg() storage options not taking effect (#24574)
• Have log() prioritize the leftmost dtype for its output dtype (#24581)
• CSV pl.len() was incorrect (#24587)
• Add support for float inputs for duration types (#24529)
• Roundtrip empty string through hive partitioning (#24546)
• Fix potential OOB writes in unaligned IPC read (#24550)
• Fix regression error when scanning AWS presigned URL (#24530)
• Make PlPath::join for cloud paths replace on absolute paths (#24514)
• Correct dtype for cum_agg in streaming engine (#24510)
• Restore support for np.datetime64() in pl.lit() (#24527)
• Ignore Iceberg list element ID if missing (#24479)
• Fix panic on streaming full join with coalesce (#23409)
• Fix AggState on all_literal in BinaryExpr (#24461)
• Show IR sort options in explain (#24465)
• Benchmark CI import (#24463)
• Fix schema on ApplyExpr with single row literal in agg context (#24422)
• Fix planner schema for dividing pl.Float32 by int (#24432)
• Fix panic scanning from AWS legacy global endpoint URL (#24450)
• Fix iterable_to_pydf(..., infer_schema_length=None) to scan all data (#23405)
• Do not propagate struct of nulls with null (#24420)
• Be stricter with invalid NDJSON input when ignore_errors=False (#24404)
• Implement approx_n_unique for temporal dtypes and Null (#24417)

📖 Documentation

• Add default parquet compression levels (#24686)
• Fix syntax error in data-types-and-structures.md (#24606)
• Rename avg_birthday -> avg_age in examples aggregation (#23726)
• Update Polars Cloud user guide (#24366)
• Fix typo in set_expr_depth_warning docstring (#24427)

📦 Build system

• Python pre-release 1.34.0b5 (#24699)
• Use cargo-run to call dsl-schema script (#24607)

🛠️ Other improvements

• Restructure python project directories again (#24676)
• Use IR for polars-expr output field resolution (#24661)
• Remove dist/ from release python workflow (#24639)
• Escape sed ampersand in release script (#24631)
• Remove PyOdide from release for now (#24630)
• Fix sed in-place in release script (#24628)
• Release script pyodide wheel (#24627)
• Release script pyodide wheel (#24626)
• Update release script for runtimes (#24610)
• Remove unused UnknownKind::Ufunc (#24614)
• Use cargo-run to call dsl-schema script (#24607)
• Cleanup and prepare to_field for element and struct field context (#24592)
• Resolve nightly clippy hints (#24593)
• Rename pl.dependencies to pl._dependencies (#24595)
• More release scripting (#24582)
• Again a minor fix for the setup script (#24580)
• Minor fix in release script (#24579)
• Correct release python beta version check (#24578)
• Python dependency failure (#24576)
• Always install yq (#24570)
• Deterministic import order for Python Polars package variants (#24531)
• Check Arrow FFI pointers with an assert (#24564)
• Add a couple of missing type definitions in python (#24561)
• Fix quickstart example in Polars Cloud user guide (#24554)
• Add implementations for loading min/max statistics for Iceberg (#24496)
• Update versions (#24508)
• Add additional unit tests for pl.concat (#24487)
• Refactor parametric tests for as_struct on aggstates (#24493)
• Use PlanCallback in name.map_* (#24484)
• Pin xlsvwriter to 3.2.5 or before (#24485)
• Add dataclass to hold resolved iceberg scan data (#24418)
• Fix iceberg test failure in CI (#24456)
• Move CompressionUtils to polars-utils (#24430)
• Update github template to dispatch to cloud client (#24416)

Thank you to all our contributors for making this release possible! @DeflateAwning, @Gusabary, @JakubValtar, @Kevin-Patyk, @MarcoGorelli, @Matt711, @alexander-beedie, @alonsosilvaallende, @borchero, @c-peters, @camriddell, @coastalwhite, @dangotbanned, @deanm0000, @dongchao-1, @dsprenkels, @eitsupi, @itamarst, @jan-krueger, @joshuamarkovic, @juansolm, @kdn36, @moizescbf, @nameexhaustion, @orlp, @r-brink, @ritchie46 and @stijnherfst

🏆 Highlights

• Add LazyFrame.{sink,collect}_batches (#23980)
• Deterministic import order for Python Polars package variants (#24531)

🚀 Performance improvements

• Skip files in scan_iceberg with filter based on metadata statistics (#24547)
• Push row_index predicate for all scan types (#24537)
• Perform integer in-filtering for Parquet inequality predicates (#24525)
• Stop caching Parquet metadata after 8 files (#24513)
• Native streaming .mode() expression (#24459)

✨ Enhancements

• Support scanning from file:/path URIs (#24603)
• Log which file the schema was sourced from, and which file caused an extra column error (#24621)
• Add LazyFrame.{sink,collect}_batches (#23980)
• Deterministic import order for Python Polars package variants (#24531)
• Add support to display lazy query plan in marimo notebooks without needing to install matplotlib or mermaid (#24540)
• Add unstable hidden_file_prefix parameter to scan_parquet (#24507)
• Use fixed-scale Decimals (#24542)
• Add support for unsigned 128-bit integers (#24346)
• Add unstable pl.Config.set_default_credential_provider (#24434)
• Roundtrip BinaryOffset type through Parquet (#24344)
• Add opt-in unstable functionality to load interval types as Struct (#24320)
• Support reading parquet metadata from cloud storage (#24443)
• Add user guide section on AWS role assumption (#24421)
• Support unique / n_unique / arg_unique for array columns (#24406)

🐞 Bug fixes

• Widen from_dicts to Iterable[Mapping[str, Any]] (#24584)
• Fix unsupported arrow type Dictionary error in scan_iceberg() (#24573)
• Raise Exception instead of panic when unnest on non-struct column (#24471)
• Include missing feature dependency from polars-stream/diff to polars-plan/abs (#24613)
• Newline escaping in streaming show_graph (#24612)
• Do not allow inferring (-1) the dimension on any Expr.reshape dimension except the first (#24591)
• Sink batches early stop on in-memory engine (#24585)
• More precisely model expression ordering requirements (#24437)
• Panic in zero-weight rolling mean/var (#24596)
• Decimal <-> literal arithmetic supertype rules (#24594)
• Match various aggregation return types in the streaming engine with the in-memory engine (#24501)
• Validate list type for list expressions in planner (#24589)
• Fix scan_iceberg() storage options not taking effect (#24574)
• Have log() prioritize the leftmost dtype for its output dtype (#24581)
• CSV pl.len() was incorrect (#24587)
• Add support for float inputs for duration types (#24529)
• Roundtrip empty string through hive partitioning (#24546)
• Fix potential OOB writes in unaligned IPC read (#24550)
• Fix regression error when scanning AWS presigned URL (#24530)
• Make PlPath::join for cloud paths replace on absolute paths (#24514)
• Correct dtype for cum_agg in streaming engine (#24510)
• Restore support for np.datetime64() in pl.lit() (#24527)
• Ignore Iceberg list element ID if missing (#24479)
• Fix panic on streaming full join with coalesce (#23409)
• Fix AggState on all_literal in BinaryExpr (#24461)
• Show IR sort options in explain (#24465)
• Benchmark CI import (#24463)
• Fix schema on ApplyExpr with single row literal in agg context (#24422)
• Fix planner schema for dividing pl.Float32 by int (#24432)
• Fix panic scanning from AWS legacy global endpoint URL (#24450)
• Fix iterable_to_pydf(..., infer_schema_length=None) to scan all data (#23405)
• Do not propagate struct of nulls with null (#24420)
• Be stricter with invalid NDJSON input when ignore_errors=False (#24404)
• Implement approx_n_unique for temporal dtypes and Null (#24417)

📖 Documentation

• Fix syntax error in data-types-and-structures.md (#24606)
• Rename avg_birthday -> avg_age in examples aggregation (#23726)
• Update Polars Cloud user guide (#24366)
• Fix typo in set_expr_depth_warning docstring (#24427)

📦 Build system

• Use cargo-run to call dsl-schema script (#24607)

🛠️ Other improvements

• Remove dist/ from release python workflow (#24639)
• Escape sed ampersand in release script (#24631)
• Remove PyOdide from release for now (#24630)
• Fix sed in-place in release script (#24628)
• Release script pyodide wheel (#24627)
• Release script pyodide wheel (#24626)
• Update release script for runtimes (#24610)
• Remove unused UnknownKind::Ufunc (#24614)
• Use cargo-run to call dsl-schema script (#24607)
• Cleanup and prepare to_field for element and struct field context (#24592)
• Resolve nightly clippy hints (#24593)
• Rename pl.dependencies to pl._dependencies (#24595)
• More release scripting (#24582)
• Again a minor fix for the setup script (#24580)
• Minor fix in release script (#24579)
• Correct release python beta version check (#24578)
• Python dependency failure (#24576)
• Always install yq (#24570)
• Deterministic import order for Python Polars package variants (#24531)
• Check Arrow FFI pointers with an assert (#24564)
• Add a couple of missing type definitions in python (#24561)
• Fix quickstart example in Polars Cloud user guide (#24554)
• Add implementations for loading min/max statistics for Iceberg (#24496)
• Update versions (#24508)
• Add additional unit tests for pl.concat (#24487)
• Refactor parametric tests for as_struct on aggstates (#24493)
• Use PlanCallback in name.map_* (#24484)
• Pin xlsvwriter to 3.2.5 or before (#24485)
• Add dataclass to hold resolved iceberg scan data (#24418)
• Fix iceberg test failure in CI (#24456)
• Move CompressionUtils to polars-utils (#24430)
• Update github template to dispatch to cloud client (#24416)

Thank you to all our contributors for making this release possible! @Gusabary, @Kevin-Patyk, @Matt711, @MoizesCBF, @alonsosilvaallende, @borchero, @c-peters, @camriddell, @coastalwhite, @dangotbanned, @deanm0000, @dongchao-1, @dsprenkels, @itamarst, @jan-krueger, @joshuamarkovic, @juansolm, @kdn36, @nameexhaustion, @orlp, @r-brink, @ritchie46 and @stijnherfst

🏆 Highlights

• Add LazyFrame.{sink,collect}_batches (#23980)
• Deterministic import order for Python Polars package variants (#24531)

🚀 Performance improvements

• Skip files in scan_iceberg with filter based on metadata statistics (#24547)
• Push row_index predicate for all scan types (#24537)
• Perform integer in-filtering for Parquet inequality predicates (#24525)
• Stop caching Parquet metadata after 8 files (#24513)
• Native streaming .mode() expression (#24459)

✨ Enhancements

• Support scanning from file:/path URIs (#24603)
• Log which file the schema was sourced from, and which file caused an extra column error (#24621)
• Add LazyFrame.{sink,collect}_batches (#23980)
• Deterministic import order for Python Polars package variants (#24531)
• Add support to display lazy query plan in marimo notebooks without needing to install matplotlib or mermaid (#24540)
• Add unstable hidden_file_prefix parameter to scan_parquet (#24507)
• Use fixed-scale Decimals (#24542)
• Add support for unsigned 128-bit integers (#24346)
• Add unstable pl.Config.set_default_credential_provider (#24434)
• Roundtrip BinaryOffset type through Parquet (#24344)
• Add opt-in unstable functionality to load interval types as Struct (#24320)
• Support reading parquet metadata from cloud storage (#24443)
• Add user guide section on AWS role assumption (#24421)
• Support unique / n_unique / arg_unique for array columns (#24406)

🐞 Bug fixes

• Widen from_dicts to Iterable[Mapping[str, Any]] (#24584)
• Fix unsupported arrow type Dictionary error in scan_iceberg() (#24573)
• Raise Exception instead of panic when unnest on non-struct column (#24471)
• Include missing feature dependency from polars-stream/diff to polars-plan/abs (#24613)
• Newline escaping in streaming show_graph (#24612)
• Do not allow inferring (-1) the dimension on any Expr.reshape dimension except the first (#24591)
• Sink batches early stop on in-memory engine (#24585)
• More precisely model expression ordering requirements (#24437)
• Panic in zero-weight rolling mean/var (#24596)
• Decimal <-> literal arithmetic supertype rules (#24594)
• Match various aggregation return types in the streaming engine with the in-memory engine (#24501)
• Validate list type for list expressions in planner (#24589)
• Fix scan_iceberg() storage options not taking effect (#24574)
• Have log() prioritize the leftmost dtype for its output dtype (#24581)
• CSV pl.len() was incorrect (#24587)
• Add support for float inputs for duration types (#24529)
• Roundtrip empty string through hive partitioning (#24546)
• Fix potential OOB writes in unaligned IPC read (#24550)
• Fix regression error when scanning AWS presigned URL (#24530)
• Make PlPath::join for cloud paths replace on absolute paths (#24514)
• Correct dtype for cum_agg in streaming engine (#24510)
• Restore support for np.datetime64() in pl.lit() (#24527)
• Ignore Iceberg list element ID if missing (#24479)
• Fix panic on streaming full join with coalesce (#23409)
• Fix AggState on all_literal in BinaryExpr (#24461)
• Show IR sort options in explain (#24465)
• Benchmark CI import (#24463)
• Fix schema on ApplyExpr with single row literal in agg context (#24422)
• Fix planner schema for dividing pl.Float32 by int (#24432)
• Fix panic scanning from AWS legacy global endpoint URL (#24450)
• Fix iterable_to_pydf(..., infer_schema_length=None) to scan all data (#23405)
• Do not propagate struct of nulls with null (#24420)
• Be stricter with invalid NDJSON input when ignore_errors=False (#24404)
• Implement approx_n_unique for temporal dtypes and Null (#24417)

📖 Documentation

• Fix syntax error in data-types-and-structures.md (#24606)
• Rename avg_birthday -> avg_age in examples aggregation (#23726)
• Update Polars Cloud user guide (#24366)
• Fix typo in set_expr_depth_warning docstring (#24427)

📦 Build system

• Use cargo-run to call dsl-schema script (#24607)

🛠️ Other improvements

• Remove dist/ from release python workflow (#24639)
• Escape sed ampersand in release script (#24631)
• Remove PyOdide from release for now (#24630)
• Fix sed in-place in release script (#24628)
• Release script pyodide wheel (#24627)
• Release script pyodide wheel (#24626)
• Update release script for runtimes (#24610)
• Remove unused UnknownKind::Ufunc (#24614)
• Use cargo-run to call dsl-schema script (#24607)
• Cleanup and prepare to_field for element and struct field context (#24592)
• Resolve nightly clippy hints (#24593)
• Rename pl.dependencies to pl._dependencies (#24595)
• More release scripting (#24582)
• Again a minor fix for the setup script (#24580)
• Minor fix in release script (#24579)
• Correct release python beta version check (#24578)
• Python dependency failure (#24576)
• Always install yq (#24570)
• Deterministic import order for Python Polars package variants (#24531)
• Check Arrow FFI pointers with an assert (#24564)
• Add a couple of missing type definitions in python (#24561)
• Fix quickstart example in Polars Cloud user guide (#24554)
• Add implementations for loading min/max statistics for Iceberg (#24496)
• Update versions (#24508)
• Add additional unit tests for pl.concat (#24487)
• Refactor parametric tests for as_struct on aggstates (#24493)
• Use PlanCallback in name.map_* (#24484)
• Pin xlsvwriter to 3.2.5 or before (#24485)
• Add dataclass to hold resolved iceberg scan data (#24418)
• Fix iceberg test failure in CI (#24456)
• Move CompressionUtils to polars-utils (#24430)
• Update github template to dispatch to cloud client (#24416)

Thank you to all our contributors for making this release possible! @Gusabary, @Kevin-Patyk, @Matt711, @MoizesCBF, @alonsosilvaallende, @borchero, @c-peters, @camriddell, @coastalwhite, @dangotbanned, @deanm0000, @dongchao-1, @dsprenkels, @itamarst, @jan-krueger, @joshuamarkovic, @juansolm, @kdn36, @nameexhaustion, @orlp, @r-brink, @ritchie46 and @stijnherfst

🏆 Highlights

• Add LazyFrame.{sink,collect}_batches (#23980)
• Deterministic import order for Python Polars package variants (#24531)

🚀 Performance improvements

• Skip files in scan_iceberg with filter based on metadata statistics (#24547)
• Push row_index predicate for all scan types (#24537)
• Perform integer in-filtering for Parquet inequality predicates (#24525)
• Stop caching Parquet metadata after 8 files (#24513)
• Native streaming .mode() expression (#24459)

✨ Enhancements

• Add LazyFrame.{sink,collect}_batches (#23980)
• Deterministic import order for Python Polars package variants (#24531)
• Add support to display lazy query plan in marimo notebooks without needing to install matplotlib or mermaid (#24540)
• Add unstable hidden_file_prefix parameter to scan_parquet (#24507)
• Use fixed-scale Decimals (#24542)
• Add support for unsigned 128-bit integers (#24346)
• Add unstable pl.Config.set_default_credential_provider (#24434)
• Roundtrip BinaryOffset type through Parquet (#24344)
• Add opt-in unstable functionality to load interval types as Struct (#24320)
• Support reading parquet metadata from cloud storage (#24443)
• Add user guide section on AWS role assumption (#24421)
• Support unique / n_unique / arg_unique for array columns (#24406)

🐞 Bug fixes

• Add support for float inputs for duration types (#24529)
• Roundtrip empty string through hive partitioning (#24546)
• Fix potential OOB writes in unaligned IPC read (#24550)
• Fix regression error when scanning AWS presigned URL (#24530)
• Make PlPath::join for cloud paths replace on absolute paths (#24514)
• Correct dtype for cum_agg in streaming engine (#24510)
• Restore support for np.datetime64() in pl.lit() (#24527)
• Ignore Iceberg list element ID if missing (#24479)
• Fix panic on streaming full join with coalesce (#23409)
• Fix AggState on all_literal in BinaryExpr (#24461)
• Show IR sort options in explain (#24465)
• Benchmark CI import (#24463)
• Fix schema on ApplyExpr with single row literal in agg context (#24422)
• Fix planner schema for dividing pl.Float32 by int (#24432)
• Fix panic scanning from AWS legacy global endpoint URL (#24450)
• Fix iterable_to_pydf(..., infer_schema_length=None) to scan all data (#23405)
• Do not propagate struct of nulls with null (#24420)
• Be stricter with invalid NDJSON input when ignore_errors=False (#24404)
• Implement approx_n_unique for temporal dtypes and Null (#24417)

📖 Documentation

• Rename avg_birthday -> avg_age in examples aggregation (#23726)
• Update Polars Cloud user guide (#24366)
• Fix typo in set_expr_depth_warning docstring (#24427)

🛠️ Other improvements

• More release scripting (#24582)
• Again a minor fix for the setup script (#24580)
• Minor fix in release script (#24579)
• Correct release python beta version check (#24578)
• Python dependency failure (#24576)
• Always install yq (#24570)
• Deterministic import order for Python Polars package variants (#24531)
• Check Arrow FFI pointers with an assert (#24564)
• Add a couple of missing type definitions in python (#24561)
• Fix quickstart example in Polars Cloud user guide (#24554)
• Add implementations for loading min/max statistics for Iceberg (#24496)
• Update versions (#24508)
• Add additional unit tests for pl.concat (#24487)
• Refactor parametric tests for as_struct on aggstates (#24493)
• Use PlanCallback in name.map_* (#24484)
• Pin xlsvwriter to 3.2.5 or before (#24485)
• Add dataclass to hold resolved iceberg scan data (#24418)
• Fix iceberg test failure in CI (#24456)
• Move CompressionUtils to polars-utils (#24430)
• Update github template to dispatch to cloud client (#24416)

Thank you to all our contributors for making this release possible! @Gusabary, @Kevin-Patyk, @Matt711, @alonsosilvaallende, @borchero, @c-peters, @camriddell, @coastalwhite, @dongchao-1, @dsprenkels, @itamarst, @jan-krueger, @joshuamarkovic, @juansolm, @kdn36, @nameexhaustion, @orlp, @r-brink, @ritchie46 and @stijnherfst

🚀 Performance improvements

• Use specialized decoding for all predicates for Parquet dictionary encoding (#24403)
• Allocate only for read items when reading Parquet with predicate (#24401)
• Don't aggregate groups for strict cast if original len (#24381)
• Allocate only for read items when reading Parquet with predicate (#24324)

✨ Enhancements

• Support S3 virtual-hosted–style URI (#24405)
• Remove explicit file create for local async writes (#24358)
• Add PyCapsule __arrow_c_schema__ interface to pl.Schema (#24365)
• Support Partitioning sinks in cloud (#24399)
• User-friendly error message on empty path expansion (#24337)
• Add unstable pre_execution_query parameter to read_database_uri (#23634)
• Add Polars security policy (#24314)

🐞 Bug fixes

• Correct sink_ipc overload for compression (#24398)
• Enable all integer dtypes for by parameter in join_asof (#24384)
• Fix Group-By + filter aggregation performs subsequent operations on all data instead of only filtered data (#24373)
• Wrap deprecated top-level imports in TYPE_CHECKING (#24340)
• Fix incorrect output ordering for row-separable exprs (#24354)
• Fix Series.__arrow_c_stream__ for Decimal and other logical types (#24120)
• Match output type to engine for Struct arithmetic (#23805)
• Make mmap use MAP_PRIVATE rather than MAP_SHARED (#24343)
• Fix cloud iceberg scan DATASET_PROVIDER_VTABLE error (#24338)
• Don't throw away type information for NumPy numeric values when using lit() (#24229)
• Incorrect logic in negative streaming slice (#24326)
• Ensure read_database_uri with ADBC works as expected with DuckDB URIs (#24097)
• Do not error on non-list Sequence for columns parameter in read_excel (#23967)

📖 Documentation

• Document newly added is_pure parameter for register_io_source (#24311)
• Create a module docstring for the public polars module (#24332)
• Update to Polars Cloud user guide (#24187)
• Update distributed page (#24323)
• Add a note and example about exporting unformatted Excel sheet data (#24145)
• Add detail about server-side cursor behaviour for SQLAlchemy in the "iter_batches" parameter of read_database (#24094)
• Add Polars security policy (#24314)

🛠️ Other improvements

• Bump c-api (#24412)
• Add a regression test for #7631 (#24363)
• Update cloud test InteractiveQuery to DirectQuery (#24287)
• Mark some tests as slow (#24327)
• Mark more tests as ready for cloud (#24315)
• Add hint to update PYPOLARS_VERSION on version assert test (#24313)

Thank you to all our contributors for making this release possible! @Kevin-Patyk, @VictorAtIfInsurance, @alexander-beedie, @coastalwhite, @dsprenkels, @itamarst, @kdn36, @kuril, @mcrumiller, @nameexhaustion, @nesb1, @orlp, @r-brink and @ritchie46

💥 Breaking changes

• Remove, deprecate or change eager Exprs to be lazy compatible (#24027)

🚀 Performance improvements

• Native streaming int_range with len or count (#24280)
• Lower arg_unique natively to the streaming engine (#24279)
• Move unordering optimization to end (#24286)
• Do ordering simplification step after common sub-plan elimination (#24269)
• Always simplify order requirements in IR (#24192)
• Basic de-duplication of filter expressions (#24220)
• Cache the IR in pipe_with_schema (#24213)
• Lower arg_where natively to streaming engine (#24088)
• Lower Expr.shift to streaming engine (#24106)
• Lower order-preserving groupby to streaming engine (#24053)

✨ Enhancements

• Add CSE for custom io sources using pointer for hashing (#24297)
• Allow pl.Expr.log to take in an expression (#24226)
• Add caching to user credential providers (#23789)
• Expose mkdir parameter on write_parquet (#24239)
• Implement diff() in streaming engine (#24189)
• Enable Expr.diff(n) for negative n (#24200)
• Allow upcasting null-typed columns to nested column types in scans (#24185)
• Log pyarrow predicate conversion result in sensitive verbose logs (#24186)
• Drop PyArrow requirement for write_database with the ADBC engine (#24136)
• Add a deprecation warning for pl.Series.shift(Null) (#24114)
• Improve Debug formatting of DataType (#24056)
• Add LazyFrame.pipe_with_schema (#24075)
• Catch additional temporal attributes in BytecodeParser function analysis (#24076)
• Add cum_* as native streaming nodes (#23977)
• Add peak_{min,max} support for booleans (#24068)
• Add DataFrame.map_columns for eager evaluation (#23821)

🐞 Bug fixes

• Invalid conversion from non-bit numpy bools (#24312)
• Make dt.epoch('s') serializable (#24302)
• Make Expr.rechunk serializable (#24303)
• Schema mismatch for 'log' operation (#24300)
• Incorrect first/last aggregate in streaming engine (#24289)
• Fix group offsets in sliced groups (#24274)
• Panic in inexact date(time) conversion (#24268)
• Keep DSL cache after serialization and deserialization (#24265)
• Sanitize and warn about eval usage (#24262)
• Correct incorrect default in from_pandas overload for include_index (#24258)
• Unique with keep="none" in new optimization pass (#24261)
• Correct size limits for Decimal cast (#24252)
• Unordered unions in check order observing pass (#24253)
• Fix dtype for slice on Literal in agg context (#24137)
• Fix incorrect filter(lit(True)) when scanning hive (#24237)
• In-memory group_by on 128-bit integers (#24242)
• Fix panic in gather inside groupby with invalid indices (#24182)
• Release the GIL in map_groups (#24225)
• Remove extra explode in LazyGroupBy.{head,tail} (#24221)
• Fix panic in polars cloud CSV scan (#24197)
• Fix panic when loading categorical columns from IO plugin (#24205)
• Fix credential provider did not auto-init on partition sinks (#24188)
• Fix engine type for concat_list on AggScalar implode (#24160)
• Rolling_mean handle centered weights with len(values) < window_size (#24158)
• Reading is_in predicate for Parquet plain strings (#24184)
• Support native DuckDB connection in read_database (#24177)
• Make PyCategories pickleable (#24170)
• Remove unused unsound function to_mutable_slice (#24173)
• PyO3 extension types giving compat_level errors (#24166)
• Allow non-elementwise by in top_k (#24164)
• Fix sort_by for group_by_dynamic context (#24152)
• Input-independent length aggregations in streaming (#24153)
• Release GIL when iterating df in to_arrow (#24151)
• Respect non-elementwise join_where conditions (#24135)
• Fix mismatched pytest test collection error (#24133)
• Resolve schema mismatch for div on Boolean (#24111)
• Fix from_repr parsing of negative durations (#24115)
• Make group_by/partition_by iterator keys tuple[Any, ...] to enable tuple-unpacking (#24113)
• Keep name when doing empty group-aware aggregation (#24098)
• Implode instead of reshape_list (#24078)
• Rolling mean with weights incorrect when min_samples < window_size (#23485)
• Allow merge_sorted for all types (#24077)
• Include datatypes in row_encode expression (#24074)
• Include UDF materialized type in serialization (#24073)
• Correct .rolling() output type for non-aggregations (#24072)
• Correct planner output schema for join_asof (#24071)
• Correct output for fold and reduce (#24069)
• Expr.meta.output_name for struct fields (#24064)
• Ensure upcast operations on pl.Date default to microsecond precision (#23981)
• Add peak_{min,max} support for booleans (#24068)
• Planner output type for mean with strange input type (#24052)
• Remove, deprecate or change eager Exprs to be lazy compatible (#24027)

📖 Documentation

• Fix few typos (#24305)
• Add missing reference to LazyFrame.pipe_with_schema() on the website (#24285)
• Automatically register doctest.ELLIPSIS so we don't have to add the inline directive each time (#24146)
• Update categorical comparison documentation in user guide (#24249)
• Add missing references for Seriers.rolling_*_by methods (#24254)
• Fix formatting of Series.value_counts examples (#24245)
• Add hint to use DataFrame/Series constructors in from_arrow docstring (#22942)
• Update GPU un/supported features (#24195)
• Add DataFrame.map_columns to API (#24128)
• Update multiple pages in the Polars Cloud user guide (#23661)
• Fix str.find_many() docstring example (#24092)

📦 Build system

• Re-enable macos-x86-64 (#24266)
• Drop binary support for macos_x86-64 (#24257)

🛠️ Other improvements

• Remove PDS-H code (#24301)
• Get ready for even more cloud tests (#24292)
• Add tests for slices with caches (#24288)
• Readd ordering tests (#24284)
• Fix Makefile venv path (#24251)
• Remove unnecessary parentheses (#24244)
• Make non-nested shift{,_and_fill} ops generic (#24224)
• Remove unused Wrap (#24214)
• Allow upcasting null-typed columns to nested column types in scans (#24185)
• Automatically label a few more types of PR (#24147)
• Update toolchain (#24156)
• Add order_sensitive property for AExpr (#24116)
• Mark more tests as not possible on cloud (#24103)
• Turn AggExpr::Count from tuple to struct (#24096)
• Mark tests that may fail in cloud (#24067)
• Extend read database tests to capture more ADBC functionality (#24002)
• Make CI perf failures more lenient (#24066)
• Fix hive partition string encoding in CI by upgrading deltalake (#24018)
• Make tests with sinks run on cloud again (#24048)

Thank you to all our contributors for making this release possible! @Kevin-Patyk, @MarcoGorelli, @NeejWeej, @agossard, @alexander-beedie, @aparna2198, @borchero, @coastalwhite, @deanm0000, @dsprenkels, @eitsupi, @etiennebacher, @gab23r, @henryharbeck, @jjurm, @kdn36, @math-hiyoko, @mcrumiller, @mroeschke, @nameexhaustion, @orlp, @r-brink, @ritchie46, @stijnherfst, @vdrn and @wence-

💥 Breaking changes

• Remove, deprecate or change eager Exprs to be lazy compatible (#24027)

🚀 Performance improvements

• Always simplify order requirements in IR (#24192)
• Basic de-duplication of filter expressions (#24220)
• Cache the IR in pipe_with_schema (#24213)
• Lower arg_where natively to streaming engine (#24088)
• Lower Expr.shift to streaming engine (#24106)
• Lower order-preserving groupby to streaming engine (#24053)

✨ Enhancements

• Allow pl.Expr.log to take in an expression (#24226)
• Add caching to user credential providers (#23789)
• Expose mkdir parameter on write_parquet (#24239)
• Implement diff() in streaming engine (#24189)
• Enable Expr.diff(n) for negative n (#24200)
• Allow upcasting null-typed columns to nested column types in scans (#24185)
• Log pyarrow predicate conversion result in sensitive verbose logs (#24186)
• Drop PyArrow requirement for write_database with the ADBC engine (#24136)
• Add a deprecation warning for pl.Series.shift(Null) (#24114)
• Improve Debug formatting of DataType (#24056)
• Add LazyFrame.pipe_with_schema (#24075)
• Catch additional temporal attributes in BytecodeParser function analysis (#24076)
• Add cum_* as native streaming nodes (#23977)
• Add peak_{min,max} support for booleans (#24068)
• Add DataFrame.map_columns for eager evaluation (#23821)

🐞 Bug fixes

• Correct size limits for Decimal cast (#24252)
• Unordered unions in check order observing pass (#24253)
• Fix dtype for slice on Literal in agg context (#24137)
• Fix incorrect filter(lit(True)) when scanning hive (#24237)
• In-memory group_by on 128-bit integers (#24242)
• Fix panic in gather inside groupby with invalid indices (#24182)
• Release the GIL in map_groups (#24225)
• Remove extra explode in LazyGroupBy.{head,tail} (#24221)
• Fix panic in polars cloud CSV scan (#24197)
• Fix panic when loading categorical columns from IO plugin (#24205)
• Fix credential provider did not auto-init on partition sinks (#24188)
• Fix engine type for concat_list on AggScalar implode (#24160)
• Rolling_mean handle centered weights with len(values) < window_size (#24158)
• Reading is_in predicate for Parquet plain strings (#24184)
• Support native DuckDB connection in read_database (#24177)
• Make PyCategories pickleable (#24170)
• Remove unused unsound function to_mutable_slice (#24173)
• PyO3 extension types giving compat_level errors (#24166)
• Allow non-elementwise by in top_k (#24164)
• Fix sort_by for group_by_dynamic context (#24152)
• Input-independent length aggregations in streaming (#24153)
• Release GIL when iterating df in to_arrow (#24151)
• Respect non-elementwise join_where conditions (#24135)
• Fix mismatched pytest test collection error (#24133)
• Resolve schema mismatch for div on Boolean (#24111)
• Fix from_repr parsing of negative durations (#24115)
• Make group_by/partition_by iterator keys tuple[Any, ...] to enable tuple-unpacking (#24113)
• Keep name when doing empty group-aware aggregation (#24098)
• Implode instead of reshape_list (#24078)
• Rolling mean with weights incorrect when min_samples < window_size (#23485)
• Allow merge_sorted for all types (#24077)
• Include datatypes in row_encode expression (#24074)
• Include UDF materialized type in serialization (#24073)
• Correct .rolling() output type for non-aggregations (#24072)
• Correct planner output schema for join_asof (#24071)
• Correct output for fold and reduce (#24069)
• Expr.meta.output_name for struct fields (#24064)
• Ensure upcast operations on pl.Date default to microsecond precision (#23981)
• Add peak_{min,max} support for booleans (#24068)
• Planner output type for mean with strange input type (#24052)
• Remove, deprecate or change eager Exprs to be lazy compatible (#24027)

📖 Documentation

• Fix formatting of Series.value_counts examples (#24245)
• Add hint to use DataFrame/Series constructors in from_arrow docstring (#22942)
• Update GPU un/supported features (#24195)
• Add DataFrame.map_columns to API (#24128)
• Update multiple pages in the Polars Cloud user guide (#23661)
• Fix str.find_many() docstring example (#24092)

📦 Build system

• Drop binary support for macos_x86-64 (#24257)

🛠️ Other improvements

• Remove unnecessary parentheses (#24244)
• Make non-nested shift{,_and_fill} ops generic (#24224)
• Remove unused Wrap (#24214)
• Allow upcasting null-typed columns to nested column types in scans (#24185)
• Automatically label a few more types of PR (#24147)
• Update toolchain (#24156)
• Add order_sensitive property for AExpr (#24116)
• Mark more tests as not possible on cloud (#24103)
• Turn AggExpr::Count from tuple to struct (#24096)
• Mark tests that may fail in cloud (#24067)
• Extend read database tests to capture more ADBC functionality (#24002)
• Make CI perf failures more lenient (#24066)
• Fix hive partition string encoding in CI by upgrading deltalake (#24018)
• Make tests with sinks run on cloud again (#24048)

Thank you to all our contributors for making this release possible! @Kevin-Patyk, @agossard, @alexander-beedie, @aparna2198, @borchero, @coastalwhite, @deanm0000, @dsprenkels, @henryharbeck, @jjurm, @kdn36, @math-hiyoko, @mcrumiller, @mroeschke, @nameexhaustion, @orlp, @r-brink, @ritchie46, @stijnherfst, @vdrn and @wence-

🚀 Performance improvements

• Lower .sort(maintain_order=True).head() to streaming top_k (#24014)
• Lower top-k to streaming engine (#23979)
• Allow order pass through Filters and relax to row-seperable instead of elementwise (#23969)

✨ Enhancements

• Add native streaming for peaks_{min,max} (#24039)
• IR graph arrows, monospace font, box nodes (#24021)
• Add DataTypeExpr.default_value (#23973)
• Lower rle to a native streaming engine node (#23929)
• Add support for Int128 to pyo3-polars (#23959)

🐞 Bug fixes

• Scan of multiple sources with null datatype (#24065)
• Categorical in nested data in row encoding (#24051)
• Missing length update in builder for pl.Array repetition (#24055)
• Race condition in global categories init (#24045)
• Revert "fix: Don't encode entire CategoricalMapping when going to Arrow (#24036)" (#24044)
• Error when using named functions (#24041)
• Don't encode entire CategoricalMapping when going to Arrow (#24036)
• Fix cast on arithmetic with lit (#23941)
• Incorrect slice-slice pushdown (#24032)
• Dedup common cache subplan in IR graph (#24028)
• Allow join on Decimal in in-memory engine (#24026)
• Fix datatypes for eval.list in aggregation context (#23911)
• Allocator capsule fallback panic (#24022)
• Accept another zlib "magic header" file signature (#24013)
• Fix truediv dtypes so cast in list.eval is not dropped (#23936)
• Don't reuse cached return_dtype for expanded map expressions (#24010)
• Cache id is not a valid dot node id (#24005)
• Align map_elements with and without return_dtype (#24007)
• Fix column dtype lifetime for csv_write segfault on Categorical (#23986)
• Allow serializing LazyGroupBy.map_groups (#23964)
• Correct allocator name in PyCapsule (#23968)
• Mismatched types for write function for windows (#23915)
• Fix unpivot panic when index= column not found (#23958)

📖 Documentation

• Fix a typo in "lazy/execution" user-guide page (#23983)

🛠️ Other improvements

• Update pyo3-polars versions (#24031)
• Remove insert_error_function (#24023)
• Remove cache hits, clean up in-mem prefill (#24019)
• Use .venv instead of venv in pyo3-polars examples (#24024)
• Fix test failing mypy (#24017)
• Remove outdated comment (#23998)
• Add a _plr.pyi to remove mypy issues (#23970)
• Don't define CountStar as dyn OptimizationRule (#23976)
• Rename atol and rtol to abs_tol and rel_tol (#23961)
• Introduce Row{Encode,Decode} as FunctionExpr (#23933)
• Dispatch through pl.map_batches and AnonymousColumnsUdf (#23867)

Thank you to all our contributors for making this release possible! @JakubValtar, @Kevin-Patyk, @borchero, @cmdlineluser, @coastalwhite, @iishutov, @jarondl, @kdn36, @orlp, @rawhuul, @ritchie46 and @stijnherfst