Skip to content

perf: Remove repeated keys from by's - #30095

Merged
orlp merged 4 commits into
pola-rs:mainfrom
orlp:rep-key
Oct 22, 2026
Merged

orlp merged 4 commits into
pola-rs:mainfrom
orlp:rep-key

Conversation

@orlp

@orlp orlp commented Oct 20, 2026 •

Copy link
Copy Markdown
Member

Simplifies .sort(["a", "a"]) to just .sort("a").

Also fixes #30081.
Also fixes #30209.

Also includes a simplification pass - we now require the MultipleSortOptions are expanded in the planner.

Generated mostly with AI, verified by me.

@github-actions github-actions Bot added performance Performance issues or improvements python Related to Python Polars rust Related to Rust Polars labels Oct 20, 2026
@dancsi

dancsi commented Oct 22, 2026

Copy link
Copy Markdown
Collaborator

issue (non-blocking): seems like there are some missed optimization opportunities because we run the new rule after sort-collapsing. Here are these cases, as detected by AI

Optimization: Removing a sort the input already satisfies
Example: set_sorted("a").sort(["a", "a"])
Resulting plan: SORT BY [a] is kept, while .sort("a") is removed
OK?: ⚠️ missed
────────────────────────────────────────
Optimization: first/last rewritten to min_by/max_by
Example: sort_by(a, a).first()
Resulting plan: min_by(row_encode(a, a)), while sort_by(a).first() becomes min_by[total](a)
OK?: ⚠️ missed
────────────────────────────────────────
Optimization: Window order_by with a literal key
Example: over(order_by=[lit, c, c])
Resulting plan: row_encode(lit, c)
OK?: ⚠️ already missed on main

@orlp
orlp merged commit d50289b into pola-rs:main Oct 22, 2026
28 of 29 checks passed

@dsprenkels dsprenkels left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: I'd make expand_flags() move-based instead of mutably updating the sort_options in place.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

performance Performance issues or improvements python Related to Python Polars rust Related to Rust Polars

Projects

None yet

3 participants