Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

filter's length: 3 differs from that of the series: 1 in chained filter #20432

Closed
2 tasks done
Liyixin95 opened this issue Dec 24, 2024 · 1 comment
Closed
2 tasks done
Labels
bug Something isn't working needs triage Awaiting prioritization by a maintainer python Related to Python Polars

Comments

@Liyixin95
Copy link

Checks

  • I have checked that this issue has not already been reported.
  • I have confirmed this bug exists on the latest version of Polars.

Reproducible example

import polars as pl
from polars import col as c

df = pl.DataFrame(
    {
        "a": [1, 2, 3, 4, 5, 6, 7, 8, 9, 10],
        "b": ["a", "a", "a", "b", "b", "b", "c", "c", "c", "d"],
    }
).lazy()

print(df.group_by("b").agg(a=c("a").filter(c("a") > 5).filter(c("a") < 8)).collect())

chained filter in groupby will get this error:

Traceback (most recent call last):
  File "/root/rust/sigrun-op-rs/./test.py", line 11, in <module>
    print(df.group_by("b").agg(a=c("a").filter(c("a") > 5).filter(c("a") < 8)).collect())
  File "/root/rust/polars/py-polars/polars/lazyframe/frame.py", line 2031, in collect
    return wrap_df(ldf.collect(callback))
polars.exceptions.ShapeError: filter's length: 3 differs from that of the series: 1

if i put all the predicate to the same filter, this code works:

import polars as pl
from polars import col as c

df = pl.DataFrame(
    {
        "a": [1, 2, 3, 4, 5, 6, 7, 8, 9, 10],
        "b": ["a", "a", "a", "b", "b", "b", "c", "c", "c", "d"],
    }
).lazy()

print(df.group_by("b").agg(a=c("a").filter(c("a") > 5, c("a") < 8)).collect())

Log output

No response

Issue description

as the example says

Expected behavior

chain filter work fine

Installed versions

--------Version info---------
Polars:              1.17.1
Index type:          UInt32
Platform:            Linux-5.10.0-28-amd64-x86_64-with-glibc2.31
Python:              3.10.13 (main, Sep 11 2023, 13:44:35) [GCC 11.2.0]
LTS CPU:             False
@Liyixin95 Liyixin95 added bug Something isn't working needs triage Awaiting prioritization by a maintainer python Related to Python Polars labels Dec 24, 2024
@orlp
Copy link
Collaborator

orlp commented Dec 24, 2024

This is a usage error. After the first filter the lengths no longer match. c("a") in the second filter expression still refers to the original column a, not the column a after the first filter.

@orlp orlp closed this as not planned Won't fix, can't repro, duplicate, stale Dec 24, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working needs triage Awaiting prioritization by a maintainer python Related to Python Polars
Projects
None yet
Development

No branches or pull requests

2 participants