Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Add strict parameter to pl.concat(how='horizontal') #20019

Draft
wants to merge 30 commits into
base: main
Choose a base branch
from
Draft
Changes from 1 commit
Commits
Show all changes
30 commits
Select commit Hold shift + click to select a range
6038130
added 'strict' as a keyword argument (default: False) to pl.concat an…
nimit Nov 27, 2024
d14049c
bug fix with keyword argument change in pl.concat
nimit Nov 27, 2024
ae2a335
Merge branch 'pola-rs:main' into strict-concat-19133
nimit Dec 1, 2024
6a0df66
rust changes
nimit Dec 1, 2024
ea847bf
fixed python errors with concat_df_horizontal, concat_lf_horizontal
nimit Dec 1, 2024
8f2883d
fixed exception type in python unit test for strict concatenation of …
nimit Dec 1, 2024
21d110a
build: Bump `chrono-tz` to `0.10` (#20094)
stinodego Dec 1, 2024
d85fc9c
chore(rust): Update AWS doc dependencies (#20095)
stinodego Dec 2, 2024
f00e6fd
docs(rust): Fix inconsistency between code and comment (#20070)
YichiZhang0613 Dec 2, 2024
a413c4a
fix: Only slice after sort when slice is smaller than frame length (#…
mcrumiller Dec 2, 2024
5a8cd16
fix: Return null instead of 0. for rolling_std when window contains a…
MarcoGorelli Dec 2, 2024
755ee47
build: Bump `atoi_simd` to version `0.16` (#20098)
stinodego Dec 2, 2024
af4d5a5
build: Bump `thiserror` to version `2` (#20097)
stinodego Dec 2, 2024
06b2ab6
build(rust): Fix path to `polars-dylib` crate in workspace (#20103)
stinodego Dec 2, 2024
511c219
build: Bump `fs4` to version `0.12` (#20101)
stinodego Dec 2, 2024
b276485
build: Bump `object_store` to version `0.11` (#20102)
stinodego Dec 2, 2024
bc3d320
build: Bump `memmap2` to version `0.9` (#20105)
stinodego Dec 2, 2024
8f72e20
refactor(rust): Replace custom `PushNode` trait with `Extend` (#20107)
nameexhaustion Dec 2, 2024
527af9c
build: Upgrade `sqlparser-rs` from version 0.49 to 0.52 (#20110)
alexander-beedie Dec 2, 2024
0d4029a
fixed LazyFrame test, Python & Rust documentation changes regarding a…
nimit Dec 2, 2024
90e65e8
Merge branch 'pola-rs:main' into strict-concat-19133
nimit Dec 2, 2024
6078964
doc fix, removed unnecessary generic
nimit Dec 2, 2024
5f3e8a6
pass additional parameter to Zip from IR::HConcat for strict concat
nimit Dec 4, 2024
d5b373b
made broadcasting congruent with InputHead constructor in ZipNode imp…
nimit Dec 4, 2024
80edf1a
Merge branch 'pola-rs:main' into strict-concat-19133
nimit Dec 4, 2024
353a563
concat tests
nimit Dec 4, 2024
68dd6bf
linting
nimit Dec 4, 2024
65cb6b1
rename test vars for better readability
nimit Dec 4, 2024
d418bb5
rename test vars for better readability
nimit Dec 4, 2024
57e00ec
linting change
nimit Dec 4, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 5 additions & 2 deletions py-polars/tests/unit/functions/test_concat.py
Original file line number Diff line number Diff line change
Expand Up @@ -24,16 +24,19 @@ def test_concat_lf_stack_overflow() -> None:

def test_concat_horizontally_strict() -> None:
a = pl.DataFrame({"a": [0, 1, 2], "b": [1, 2, 3]})
b = pl.DataFrame({"c": [11], "d": [42]}) # 1 vs N
b = pl.DataFrame({"c": [11], "d": [42]}) # 1 vs N (may broadcast)
c = pl.DataFrame({"c": [11, 12], "d": [42, 24]}) # 2 vs N
nimit marked this conversation as resolved.
Show resolved Hide resolved
# The different cases: 1 vs N and 2 vs N are tested because Zip (internally used by concat) automatically broadcasts the DataFrame (when using the new streaming engine)

with pytest.raises(pl.exceptions.ShapeError):
pl.concat([a, b], how="horizontal", strict=True)

with pytest.raises(pl.exceptions.ShapeError):
pl.concat([a, c], how="horizontal", strict=True)

with pytest.raises(pl.exceptions.ShapeError):
pl.concat([a.lazy(), b.lazy()], how="horizontal", strict=True).collect()

with pytest.raises(pl.exceptions.ShapeError):
pl.concat([a.lazy(), c.lazy()], how="horizontal", strict=True).collect()

out = pl.concat([a, b], how="horizontal", strict=False)
Expand Down
Loading