Fix (E712) changing `==`/`!=` to `is`/`is not` is not correct for some types

[zanieb]

Summary

Generally, comparisons to True, False, and None singletons should use obj is True instead of obj == True.

However, it is common for libraries to override the ==/__eq__ operator to create simple APIs for filtering data. In these cases, correcting == to is changes the meaning of the program and breaks the user's code. The same applies for != and is not.

This is a tracking issue for all invalid corrections from this rule.

Types with the issue

• pandas.DataFrame often used with DataFrame.mask, DataFrame.where
• pandas.Series often used with Series.mask, Series.where
• numpy.Array often used with Array.where
• sqlalchemy.Column often used with Query.having, Query.filter, Query.where

If an issue with an unlisted type is encountered please reply and I will edit to add it here.

Resolution

Eventually, ruff is likely to detect these cases by inferring the datatype involved and exclude it from the suggested fix.

In the meantime, you may:

• Disable rule E712
• Use an alternative comparison method that is not ambiguous (e.g. pandas.Series.eq)

Examples

import numpy

numpy.array([True, False]) == False
# array([False,  True])

numpy.array([True, False]) is False
# False

import pandas

pandas.Series([True, False]) == False
# 0    False
# 1    True
# dtype: bool

pandas.Series([True, False]) is False
# False

# Alternative safe syntax
pandas.Series([True, False]).eq(False)

pandas.DataFrame({"x": [True, False]}) == False
#       x
# 0  False
# 1  True

pandas.DataFrame({"x": [True, False]}) is False
# False

# Alternative safe syntax
pandas.DataFrame({"x": [True, False]}).eq(False)

import sqlalchemy
c = sqlalchemy.Column("foo", sqlalchemy.Boolean)

c == True
# <sqlalchemy.sql.elements.BinaryExpression object at 0x12ed532e0>

c is True
# False

# Alternative safe syntax
c.is_(True)
c.is_not(False)

Related issues

• [ruff --fix] (v0.0.239) When working with numpy arrays mask == True is not the same as mask is True #2443
• E712 (Comparison to True should be cond is True) should be ignored when comparison is against pandas.Series #1852
• E712 : Wrong fix suggested in presence of numpy.bool_ #4356

[ndevenish]

I just spent a while tracking down this exact issue, an error introduced by ruff and reduced to the almost identical:

import numpy as np

arr = np.array([False, True, False, True])
print(repr(arr == False))
# array([ True, False,  True, False])
print(repr(arr is False))
# False

Reading the other thread, it sounds like this autofix can't be made safe. I would suggest disabling it completely then, because using truthiness in numpy comparison isn't a rare operation, and expecting everyone to "know" not to do this seems to defeat the point of an autocorrecting linter.

[charliermarsh]

Reading the other thread, it sounds like this autofix can't be made safe. I would suggest disabling it completely then, because using truthiness in numpy comparison isn't a rare operation, and expecting everyone to "know" not to do this seems to defeat the point of an autocorrecting linter.

I think making this a suggested fix (as it is now) will have the same effect, once we introduce --fix and --fix-unsafe (the former of which will only make automatic fixes, while the latter will include suggested fixes, which may include changes in behavior).

The problem with removing the autofix entirely is that it doesn't really reduce the burden or expectation on the user, because this diagnostic will still be raised, and so users will still be required to look at the code and understand whether or not to change it. Performing the fix automatically has the downside of silently breaking the code, but requiring users to opt-in to the change explicitly seems (to me) as safe as pointing them to the diagnostic without including any possible fix.

[nicornk]

Any conclusion on this one? This issue broke a bunch of our pyspark code by converting code similar to:

df = df.where(F.col("colName") == True)

to

df = df.where(F.col("colName") is True)

which leads to TypeError: condition should be string or Column

Thank you

[dstoeckel]

Adding to @nicornk: PEP-8 actually strongly discourages using is with boolean constants:

Don’t compare boolean values to True or False using ==:
# Correct:
if greeting:
# Wrong:
if greeting == True:
Worse:

# Wrong:
if greeting is True:

So while the autofix may be unsafe, I would argue the diagnostics itself is harmful. The correct suggestion would be to drop == True entirely (though PySpark is certainly another special case, as == False would need a conversion to the unary not ~ operator ...)

[zanieb]

@nicornk you can use the unambiguous alternate syntax e.g. df = df.where(F.col("colName").is_(True)) or disable the rule. Additionally, once Ruff has type inference, we will avoid suggesting a change to col is True.

@dstoeckel while I agree that if foo is definitely better than if foo == True, there are entirely valid uses for checking if something is True or False without having a __bool__ cast occur and there are cases outside of if statments where the user must perform the cast. Unfortunately the diagnostic must try to guess the intent of the user when suggesting a fix which puts us in a tough spot. I'm not sure this issue is the correct place to discuss the validity of the rule as a whole though, I'd like to keep this scoped to a discussion of false positives based on type inference.

[conbrad]

Additionally, once Ruff has type inference

Is there an existing ongoing effort for this that's public?

[zanieb]

Note as of v0.1.0 we do not apply unsafe fixes by default — so this fix will not be applied by default.

[NeilGirdhar]

there are entirely valid uses for checking if something is True or False without having a __bool__ cast occur

Of course this is a matter of opinion, but if you want to check this, I think the Pythonic way to do so is to use:

if isinstance(x, bool) and x:
# or
match x:
    case True:

is True is far more often misused in my opinion.