support@lune.dev

Want to use over 200+ MCP servers inside your coding tools like Cursor?

Asked 1 month ago by MartianPilot682

Why is np.log() in np.select Triggering a RuntimeWarning Despite Non-negative Data?

The post content has been automatically edited by the Moderator Agent for consistency and clarity.

I'm encountering a RuntimeWarning when executing a function that calculates logarithmic values from my data. Although the data contains no negative or NaN values, the following error appears:

BASH
C:\Users\quoc\anaconda3\envs\uhart\Lib\site-packages\pandas\core\arraylike.py:399: RuntimeWarning: invalid value encountered in log
  result = getattr(ufunc, method)(*inputs, **kwargs)
C:\Users\quoc\anaconda3\envs\uhart\Lib\site-packages\pandas\core\arraylike.py:399: RuntimeWarning: invalid value encountered in log
  result = getattr(ufunc, method)(*inputs, **kwargs)

The dataset is as follows:

pandas
   gvkey  sale  ebit
4   1000  44.8  16.8
5   1000  53.2  11.5
6   1000  42.9   6.2
7   1000  42.4   0.9
8   1000  44.2   5.3
9   1000  51.9   9.7

The function being used is:

PYTHON
def calculate_ln_values(df):
    conditions_ebit = [
        df['ebit'] >= 0.0,
        df['ebit'] <  0.0
    ]
    choices_ebit = [
        np.log(1 + df['ebit']),
        np.log(1 - df['ebit']) * -1
    ]
    df['lnebit'] = np.select(conditions_ebit, choices_ebit, default=np.nan)
    
    conditions_sale = [
        df['sale'] >= 0.0,
        df['sale'] <  0.0
    ]
    choices_sale = [
        np.log(1 + df['sale']),
        np.log(1 - df['sale']) * -1
    ]
    df['lnsale'] = np.select(conditions_sale, choices_sale, default=np.nan)
    return df

I run the function with:

PYTHON
calculate_ln_values(data)

The formula derived from the paper is:

ln(1+EBIT) if EBIT ≥ 0
-ln(1-EBIT) if EBIT < 0

Thus, my code follows the paper by computing:

PYTHON
np.log(1 + df['ebit']),
np.log(1 - df['ebit']) * -1

The problem occurs because both branches of np.select are evaluated, even if one branch is not selected. For example, when EBIT is non-negative, the branch np.log(1 - df['ebit']) is still computed and may produce invalid (negative) input for np.log(), triggering the warning. I would appreciate any guidance on how to avoid this issue so that each logarithm is computed only on valid data.

Comments 0

•

Answers 3

•

3 Answers

Answer by StackOverflow TychoAI Agent • 1 month ago

The warning occurs because both branches in np.select are fully evaluated before the selection is made. Even though all EBIT values are non‐negative (so the second branch isn’t used), the expression

np.log(1 - df["ebit"]) * -1

is still computed on the entire column. For positive EBIT values, (1 - EBIT) becomes negative (e.g. 1 - 16.8 = –15.8) and np.log(negative) produces an invalid value, triggering the warning.

To fix this, compute each branch only for the rows satisfying the corresponding condition. For example, you can use DataFrame indexing to assign values conditionally:

PYTHON
def calculate_ln_values(df):
    # Calculate lnebit only for valid values
    df.loc[df['ebit'] >= 0, 'lnebit'] = np.log(1 + df.loc[df['ebit'] >= 0, 'ebit'])
    df.loc[df['ebit'] < 0,  'lnebit'] = -np.log(1 - df.loc[df['ebit'] < 0, 'ebit'])
    
    # Calculate lnsale similarly
    df.loc[df['sale'] >= 0, 'lnsale'] = np.log(1 + df.loc[df['sale'] >= 0, 'sale'])
    df.loc[df['sale'] < 0,  'lnsale'] = -np.log(1 - df.loc[df['sale'] < 0, 'sale'])
    
    return df

This approach ensures that the logarithm function is only applied when the argument is positive, thereby avoiding the runtime warning.

Tycho is an AI agent, that grounds responses in various sources like documentation, code bases, live discussions, and relevant posts. Want to chat privately with Tycho?

No comments yet.

Answer by InterstellarScholar114 • 2 months ago

You are getting this error as you are passing negative values into np.log() when you do the below:

PYTHON
np.log(1 - df['ebit']) * -1

and

PYTHON
np.log(1 - df['sale']) * -1

I imagine the * -1 part was you trying to avoid passing in a negative, however you are doing this outside of the log function, hence the error. For example, if 1 - df['ebit'] = n, your code is first trying to do log(n) then multiply that by -1. If n is negative (as it often is in your code), this is not possible.

You want to re-write your log calls such that the * -1 is inside the log, like:

PYTHON
np.log((1 - df['sale']) * -1)

Edit thanks to @Quang Hoang

Using:

PYTHON
np.log((1 - df['sale']).abs())

Is a more robust way of achieving what you're after, as using * -1 will still cause issues with negative values if there is a value in df['sale'] that is less than 1. Using .abs() takes the absolute value of a column, so the value regardless of sign, which will avoid any negative values being passed into np.log().

No comments yet.

Answer by StarSurveyor096 • 2 months ago

The problem is in this block of code:

PYTHON
    choices_ebit = [
        np.log(1 + df['ebit']),
        np.log(1 - df['ebit']) * -1
    ]

Here, you are calculating both formulas, for when ebit is positive and when it's negative, and storing them in choices_ebit. However, when ebit>=1, the second one will give you the runtime warning, and when ebit<=-1, the first one will give your the runtime warning.

In order to avoid calculating both formulas, you can factor them out into one with abs() on the one hand, and np.sign() on the other:

PYTHON
    df['lnebit'] = np.log(1 + df['ebit'].abs()) * np.sign(df['ebit'])

This meets your requirements:

when ebit>=0, sign(ebit) == 1 and abs(ebit) == ebit, so that resolves to log(1+ebit)
when ebit<=, sign(ebit) == -1 and abs(ebit) == -ebit, so that resolves to -log(1-ebit)

No comments yet.

Discussion

No comments yet.

Why is np.log() in np.select Triggering a RuntimeWarning Despite Non-negative Data?

3 Answers

Discussion

Similar Posts

Why Does Matplotlib Produce a Distorted Plot in PyCharm with Python 3.12?

Why is my Keras leap year model predicting biased outputs?

How can I Configure Chaquopy in a KMM Shared Module to Access Python (NumPy) Code?