How can I dynamically determine list lengths from structs in a Polars expression?

Below is an explanation with a sample solution.

Why Your Attempt Doesn’t Work

Your approach

PYTHONCopied
range(pl.col("y").struct["low"].list.len())

does not work because inside an expression the functions (like pl.col("...")) return a lazy series; however, Python’s range needs a concrete integer value. In other words, you can’t use a per-row computed length in a Python range within the same expression context.

Suggested Workaround

If you know that the length of the lists is identical for all rows (or if you simply want to use a representative length from one row), you can precompute that value and then use it in your expression. For example, if you assume the first row represents the length for all rows, you can do the following:

PYTHONCopied
import polars as pl

df = pl.DataFrame(
    {
        "x": [0, 4],
        "y": [
            {"low": [-1, 0, 1], "up": [1, 2, 3]},
            {"low": [-2, -1, 0], "up": [0, 1, 2]},
        ],
    }
)

# Compute the length of the list in the struct in the first row.
n = df.select(pl.col("y").struct["low"].list.len().alias("len")).item()

df_with_check = df.with_columns(
    check=pl.concat_list(
        [
            pl.all_horizontal(
                [
                    pl.col("x").ge(pl.col("y").struct["low"].list.get(i)),
                    pl.col("x").le(pl.col("y").struct["up"].list.get(i)),
                ]
            )
            for i in range(n)
        ]
    ).list.max()
)

print(df_with_check)

Explanation

1. Precompute List Length:

   • The line n = df.select(pl.col("y").struct["low"].list.len().alias("len")).item() selects the length of the list in the "low" field from the "y" struct (using the first row) and extracts it as a concrete integer.

2. Using the Computed Length:

   • You then use this n to build the list comprehension:

     PYTHONCopied
     [
         pl.all_horizontal(
             [
                 pl.col("x").ge(pl.col("y").struct["low"].list.get(i)),
                 pl.col("x").le(pl.col("y").struct["up"].list.get(i)),
             ]
         )
         for i in range(n)
     ]

   • This creates an expression for each valid index.

3. Final Expression:

   • The expressions are concatenated using pl.concat_list(...) and then aggregated via list.max() over the resulting list.

Final Notes

If the lengths can vary and you need to handle different row lengths in a single expression, you’d have to rethink the problem because the current lazy API does not allow per-row varying control over Python's range objects. Often in such cases, a two-step process or an alternative formulation is needed.

This solution should help you create an expression that is dynamic under the condition that all rows have the same length.