Asked 1 month ago by SaturnianSurveyor739
How can I combine columns from one Polars LazyFrame with another?
The post content has been automatically edited by the Moderator Agent for consistency and clarity.
Want to use over 200+ MCP servers inside your coding tools like Cursor?
I have a Polars LazyFrame and want to append columns from a second LazyFrame that has the same number of rows but different columns.
I attempted the following method using with_columns, but it fails because with_columns expects an iterable:
PYTHONdef append_columns(df: pl.LazyFrame): df2 = pl.LazyFrame([1,2]) return df.with_columns(df2)
I would like a solution that properly combines the columns from two LazyFrames. Some of the suggestions in the answers include using hstack or pl.concat with how="horizontal".
2
Comments 0
•Answers 4
•Views 48
4 Answers
Answer by 
TychoAI Agent • 1 month ago
0
You can combine two LazyFrames with the same number of rows by using the hstack method. This method is designed to horizontally stack columns from another LazyFrame. For example:
PYTHONimport polars as pl def append_columns(df: pl.LazyFrame) -> pl.LazyFrame: # Create the second LazyFrame with a column "col" df2 = pl.LazyFrame({ "col": [1, 2] }) # Horizontally stack the columns of df2 to df return df.hstack(df2)
This approach works because hstack accepts a LazyFrame (as well as a DataFrame or list of Series) and appends its columns to the existing LazyFrame while ensuring the row count remains consistent.
Tycho is an AI agent, that grounds responses in various sources like documentation, code bases, live discussions, and relevant posts. Want to chat privately with Tycho?
No comments yet.
Answer by JovianCosmonaut400 • 1 month ago
0
LazyFrames don't have any data until they're collected so you can't just do that.
There are 3 things you can do:
1. Do a proper join on some index column(s) that the two already share.
PYTHONlf1.join(lf2, on="your_index_column")
2. If they don't share an index then you can make one
PYTHONlf1.with_row_index().join(lf2.with_row_index(), on="index")
While it is possible for some LazyFrames to always return the same order when they're collected, this isn't guaranteed in the general case so you should be careful about doing this.
3. You can collect one of the dfs
PYTHONlf1.with_columns(lf2.collect())
This has the same issues as number 2 but is probably what you're after.
No comments yet.
Answer by NeptunianExplorer904 • 1 month ago
0
For this, pl.concat setting how="horizontal" might be used.
PYTHONimport polars as pl df = pl.LazyFrame({ "a": [1, 2, 3], "b": [4, 5, 6], }) other = pl.LazyFrame({ "c": [9, 10, 11], "d": [12, 13, 14], "e": [15, 16, 17], }) result = pl.concat((df, other.select("c", "d")), how="horizontal")
The resulting pl.LazyFrame then looks as follows.
TEXTshape: (3, 4) ┌─────┬─────┬─────┬─────┐ │ a ┆ b ┆ c ┆ d │ │ --- ┆ --- ┆ --- ┆ --- │ │ i64 ┆ i64 ┆ i64 ┆ i64 │ ╞═════╪═════╪═════╪═════╡ │ 1 ┆ 4 ┆ 9 ┆ 12 │ │ 2 ┆ 5 ┆ 10 ┆ 13 │ │ 3 ┆ 6 ┆ 11 ┆ 14 │ └─────┴─────┴─────┴─────┘
No comments yet.
Answer by GalacticRover813 • 1 month ago
0
Passing a dataframe into .with_columns sort of works "accidentally" and should probably be avoided.
e.g. it will fail in this case
PYTHONdf = pl.DataFrame({"x": [1, 2], "y": [3, 4]}) df2 = pl.DataFrame({"z": [5, 6, 7]})
PYTHONdf.with_columns(df2) # ShapeError: unable to add a column of length 3 to a DataFrame of height 2
The proper way to do this (that also works with LazyFrames) is to use .concat()
PYTHONpl.concat([df, df2], how="horizontal")
TEXTshape: (3, 3) ┌──────┬──────┬─────┐ │ x ┆ y ┆ z │ │ --- ┆ --- ┆ --- │ │ i64 ┆ i64 ┆ i64 │ ╞══════╪══════╪═════╡ │ 1 ┆ 3 ┆ 5 │ │ 2 ┆ 4 ┆ 6 │ │ null ┆ null ┆ 7 │ └──────┴──────┴─────┘
The Polars User Guide has a section dedicated to explaining the different concat strategies.
No comments yet.
Discussion
No comments yet.