1.21.0
Python Polars 1.21.0
Added 6
- Add `linear_space` function
- Add unique operations for Decimal dtype
- Add NDJson sink for the new streaming engine
- Add CSV sink for the new streaming engine
- Add experimental credential provider support for Delta read/scan/write
- Add experimental unity catalog client
Changed 13
- Stabilize methods and functions
- Allow column expressions in DataFrame `unnest`
- Improve string to temporal parsing in `read_excel` and `read_ods`
- Implement df.unique() on new-streaming engine
- Support nested keys in window functions
- Support cumulative aggregations for Decimal dtype
- Auto-initialize Python credential providers in more cases
- Account for SurrealDB Python API updates to handle both `SurrealDB` and `AsyncSurrealDB` classes in `read_database`
- Drop `nest-asyncio` in favor of custom logic
- Improve window function caching strategy
- Support `lakefs://` URI for delta scanner
- Support loading `numpy.float16` values as Float32
- Periodically check Python signals for CTRL-C handling
Fixed 1
- Incorrect scale used in `log` and `exp` for Decimal type
🚀 Performance improvements
- Use BitmapBuilder in yet more places (#20868)
- Make an owned version of append (#20800)
- Use BitmapBuilder in a lot more places (#20776)
✨ Enhancements
- Stabilize methods/functions (#20850)
- Add
linear_space(#20678) - Improve string → temporal parsing in
read_excelandread_ods(#20845) - Implement df.unique() on new-streaming engine (#20875)
- Experimental credential provider support for Delta read/scan/write (#20842)
- Allow column expressions in DataFrame
unnest(#20846) - Auto-initialize Python credential providers in more cases (#20843)
- Add unique operations for Decimal dtype (#20855)
- Add NDJson sink for the new streaming engine (#20805)
- Support nested keys in window functions (#20837)
- Add CSV sink for the new streaming engine (#20804)
- Periodically check python signals ('CTRL-C' handling) (#20826)
- Experimental unity catalog client (#20798)
- Support cumulative aggregations for
Decimaldtype (#20802) - Account for SurrealDB Python API updates (handle both
SurrealDBandAsyncSurrealDBclasses) inread_database(#20799) - Drop
nest-asyncioin favor of custom logic (#20793) - Improve window function caching strategy (#20791)
- Support
lakefs://URI for delta scanner (#20757) - Additional support for loading
numpy.float16values (as Float32) (#20769)
🐞 Bug fixes
- Warn if asof keys not sorted (#20887)
- Ensure explicit values given to
column_widthsoverride autofit inwrite_excel(#20893) - Avoid name collisions and panicking in object conversion (#20890)
- Incorrect scale used in
logandexpfor Decimal type (#20888) - Don't deep clone manuallydrop in GroupsPosition (#20886)
- Fix DuplicateError when selecting columns after
join_whereor cross join + filter (#20865) - Incorrect
Decimalvalue forfill_null(strategy="one")(#20844) - Fix one edge case (out of many) of int128 literals not working (#20830)
- Add height check to frame-level row indexing when key is int (#20778)
- Remove
assertthat panics ongroup_byfollowed byhead(n), wherenis larger then the frame height (#20819) - Selectors should raise on
+between themselves (#20825) - Fix panic
InvalidHeaderValuescanning from S3 on Windows (#20820) - Fix
clipforDecimalreturning wrong values (#20814) - Incorrect height from slicing after projecting only the file path column (#20817)
- Shift mask when skipping Bitpacked values in Parquet (#20810)
- Error instead of truncate if length mismatch for several
strfunctions (#20781) - Support cumulative aggregations for
Decimaldtype (#20802) - Allow
is_invalues to be given as customCollection(#20801) - Propagate null instead of panicking in
pl.repeat_by()(#20787) - Do not print sensitive information to output on
POLARS_VERBOSE(#20797) - Ignore file cache allocation error if
fallocate()is not permitted (#20796) - Incorrect logic in
assert_series_equalfor infinities (#20763)
📖 Documentation
- Update source URL for
legislators-historical.csv(#20858) - Update ML part of ecosystem user guide page (#20596)
🛠️ Other improvements
- Disable 'catalog' in build (#20897)
- Implement negative slice for new streaming IPC (#20866)
- Debloat Series bitops (#20873)
- Reduce python map bloat (#20871)
- Remove todo and test restriction for new-streaming (#20861)
- Dispatch to the in-mem engine for
AExpr::Gather(#20862) - Dispatch to the in-memory engine for multifile sources (#20860)
- Add tests for open issues (#20857)
- Mark 'register_startup' as unsafe (#20841)
- Reduce mode bloat (#20839)
- Rename
ContainsManytoContainsAny(#20785) - Unpin NumPy in type checking workflow (#20792)
- Add various tests (#20768)
- Small drive-by's (#20772)
- Touch the upload probe for the remote benchmark (#20767)
Thank you to all our contributors for making this release possible! @alexander-beedie, @arnabanimesh, @braaannigan, @burakemir, @coastalwhite, @etiennebacher, @ion-elgreco, @itamarst, @lukemanley, @mcrumiller, @nameexhaustion, @orlp, @ritchie46 and @stinodego