Boolean Filtering

"Players with more than 300 runs." "MI players who scored over 100." "Everyone who is NOT a pure batter." This is filtering — keeping only the rows that satisfy a condition — and it is the pandas equivalent of SQL's WHERE. You'll use it in nearly every analysis, so it's worth understanding how it works underneath: it is just NumPy-style boolean indexing (from the NumPy notes) applied to rows.

Definition

Boolean filtering (boolean indexing) selects rows of a DataFrame using a boolean mask — a Series of True/False values aligned to the rows — keeping exactly the rows where the mask is True. Masks are built with comparison operators and combined with the element-wise logical operators & (and), | (or) and ~ (not).

• Boolean mask — the result of a comparison like players["runs"] > 300: one True/False per row, with the same index as the DataFrame.
• Aligned to the rows — pandas matches the mask to the rows by index label, so the mask must come from the same table.
• Comparison operators — ==, !=, >, >=, <, <=, applied to every value of a column at once.
• Element-wise logical operators — &, |, ~ combine masks row by row. Python's and, or, not do not work on Series.

Syntax

mask = df["col"] > value                    # build a mask
df[mask]                                    # keep rows where True
df[df["col"] == value]                      # usually written inline
df[(cond1) & (cond2)]                       # AND  - both true
df[(cond1) | (cond2)]                       # OR   - at least one true
df[~(cond)]                                 # NOT  - invert
df.loc[cond, ["col1", "col2"]]              # filter + choose columns

Example — a single condition, step by step

mask = players["runs"] > 300
print(mask)
print(players[mask])

0     True
1     True
2    False
3    False
4    False
5    False
6     True
7    False
Name: runs, dtype: bool
         player team    role  runs  wickets  strike_rate
0   Virat Kohli  RCB  Batter   741        0        154.7
1  Rohit Sharma   MI  Batter   417        0        150.0
6  Shubman Gill   GT  Batter   426        0        147.4

How it works: the comparison is applied to all 8 runs at once and returns a bool Series. Passing it inside players[...] keeps rows 0, 1 and 6 — the True ones. Notice the original index labels are kept (0, 1, 6), so the result is not numbered 0, 1, 2. Text comparisons work the same way: players[players["team"] == "MI"].

Example — multiple conditions with & | ~

print(players[(players["team"] == "MI") & (players["runs"] > 100)])
print(players[(players["team"] == "CSK") | (players["wickets"] >= 10)])
print(players[~(players["role"] == "Batter")][["player", "role"]])

          player team         role  runs  wickets  strike_rate
1   Rohit Sharma   MI       Batter   417        0        150.0
4  Hardik Pandya   MI  All-rounder   216       11        143.0
            player team         role  runs  wickets  strike_rate
2         MS Dhoni  CSK       Keeper   161        0        220.5
3   Jasprit Bumrah   MI       Bowler     8       20         80.0
4    Hardik Pandya   MI  All-rounder   216       11        143.0
5  Ravindra Jadeja  CSK  All-rounder   267        8        157.1
7      Rashid Khan   GT       Bowler   102       10        120.0
            player         role
2         MS Dhoni       Keeper
3   Jasprit Bumrah       Bowler
4    Hardik Pandya  All-rounder
5  Ravindra Jadeja  All-rounder
7      Rashid Khan       Bowler

How it works: & keeps MI players who also scored over 100 — Bumrah (8 runs) is dropped. | keeps anyone from CSK or with 10+ wickets. ~ flips the mask, giving everyone who is not a "Batter" (same result as players["role"] != "Batter").

Why the parentheses are compulsory

In Python, & and | have higher precedence than == and >. Without brackets,

players[players["team"] == "MI" & players["runs"] > 100]

is read as players["team"] == ("MI" & players["runs"]) > 100 — pandas tries to compute "MI" & runs and fails:

TypeError: Cannot perform 'rand_' with a dtyped [int64] array and scalar of type [bool]

And using Python's and:

players[(players["team"] == "MI") and (players["runs"] > 100)]

ValueError: The truth value of a Series is ambiguous. Use a.empty, a.bool(), a.item(), a.any() or a.all().

and needs one True/False for the whole Series, which is ambiguous (8 values!). Rule: wrap every condition in ( ) and join them with &, |, ~.

Example — counting and using matches

mask = players["runs"] > 300
print(mask.sum(), mask.mean())
print(players.loc[mask, "player"].tolist())
print(players[players["runs"] > 1000].empty)

3 0.375
['Virat Kohli', 'Rohit Sharma', 'Shubman Gill']
True

How it works: True counts as 1, so mask.sum() = how many rows match (3) and mask.mean() = what fraction (3/8 = 37.5 %). .empty tells you whether a filter matched nothing at all — an empty result is a DataFrame with the columns but zero rows.

Readable filters — name your conditions

Long filters get hard to read. Store each condition in a variable:

is_mi     = players["team"] == "MI"
scored    = players["runs"] > 100
all_round = players["role"] == "All-rounder"
players[is_mi & (scored | all_round)]

Variables need no extra brackets (they are already single masks), and the logic reads like a sentence.

Common mistakes

• ❌ df[df["team"] == "MI" & df["runs"] > 100] → TypeError. ✅ df[(df["team"] == "MI") & (df["runs"] > 100)].
• ❌ and / or / not → "truth value of a Series is ambiguous". ✅ & / | / ~.
• ❌ df[df["team"] = "MI"] (single =) → SyntaxError. ✅ Comparison is ==.
• ❌ df["team"] == "mi" returns nothing. ✅ Text comparison is case-sensitive (see str.contains(case=False) in the next topic).
• ❌ Expecting the filtered result to be numbered 0, 1, 2. ✅ Original labels are kept; add .reset_index(drop=True) if you want a fresh index.

Interview tip

Q: Why does df[(a > 1) and (b < 5)] raise an error in pandas? A: Python's and asks each Series for a single truth value, and a Series of many booleans has no single truth value — so pandas raises "The truth value of a Series is ambiguous". Element-wise logic needs &, with each condition in parentheses because & binds tighter than comparison operators.