Why Vectorization Wins

You now know four ways to compute a strike rate for every player: an iterrows loop, an itertuples loop, apply(axis=1), and the plain column expression df["runs"] / df["balls"] * 100. All four give the same answer. This topic shows how differently fast they are, and why.

Definition

Vectorisation means expressing an operation on whole columns (arrays) at once — df["runs"] / df["balls"] — so that pandas/NumPy run the loop internally in compiled C code, instead of your Python code visiting each value one by one.

• Whole columns — you write the formula once; it applies to every row.
• Compiled loop — the loop still happens, but in C, on a contiguous block of numbers of a single dtype, without Python's per-item overhead (type checks, function calls, object creation).
• Python-level loops — for loops, iterrows, itertuples, apply, map with a function — call Python once per row/value. They are flexible but slow.

Syntax — the speed ladder (fastest first)

df["runs"] / df["balls"] * 100                    # 1. vectorised arithmetic
np.where(cond, a, b) / np.select(...) / pd.cut     # 2. vectorised conditions
s.str.upper() / s.dt.year                          # 3. vectorised accessors (.str, .dt)
s.map(dict)                                        # 4. hash look-up
s.apply(func) / df.apply(func, axis=1)             # 5. Python call per value / row
df.itertuples()                                    # 6. Python loop, namedtuples
df.iterrows()                                      # 7. Python loop, Series per row — slowest

Example — timing four ways to compute strike rate

import time
import pandas as pd
import numpy as np

rng = np.random.default_rng(42)
n = 20_000
df = pd.DataFrame({
    "runs": rng.integers(0, 120, n),
    "balls": rng.integers(1, 80, n),
})

def timed(label, fn):
    start = time.perf_counter()
    result = fn()
    ms = (time.perf_counter() - start) * 1000
    print(f"{label:<12} {ms:8.1f} ms")
    return result

def with_iterrows():
    out = []
    for _, row in df.iterrows():
        out.append(row["runs"] / row["balls"] * 100)
    return pd.Series(out, index=df.index)

def with_itertuples():
    return pd.Series([t.runs / t.balls * 100 for t in df.itertuples()], index=df.index)

def with_apply():
    return df.apply(lambda row: row["runs"] / row["balls"] * 100, axis=1)

def vectorized():
    return df["runs"] / df["balls"] * 100

a = timed("iterrows", with_iterrows)
b = timed("itertuples", with_itertuples)
c = timed("apply", with_apply)
d = timed("vectorized", vectorized)
print("same answer:", np.allclose(a, d) and np.allclose(b, d) and np.allclose(c, d))

One run on a laptop (your numbers will differ — they depend on your machine, and in the browser playground everything is several times slower):

iterrows        366.3 ms
itertuples       12.7 ms
apply           101.6 ms
vectorized        0.4 ms
same answer: True

How it works: the exact milliseconds change on every run, but the ratio is what matters. Here the vectorised version was roughly 900× faster than iterrows and ~250× faster than apply(axis=1). On 20,000 rows that is the difference between instant and a noticeable pause; on 20 million rows it is seconds vs. many minutes.

time.perf_counter() is a high-resolution clock — subtract a start time from an end time to measure how long code took. (In Jupyter you can also use %timeit.)

Example — if/else labels: apply vs np.where

runs = pd.Series(rng.integers(0, 150, 100_000))

slow = runs.apply(lambda r: r + 10 if r >= 50 else r)        # Python call per value
fast = pd.Series(np.where(runs >= 50, runs + 10, runs))     # vectorised

One run (your numbers will differ):

apply + if/else :   26.6 ms
np.where        :    0.9 ms
same answer: True

Why loops are slow — what happens per row

iterrows, for EACH of 20,000 rows:
  build a new Series object  →  look up "runs" by label  →  look up "balls"
  → box numbers as Python objects → do the maths in Python → append to a list

vectorised, ONCE:
  take the runs array + the balls array → one C loop over 20,000 numbers → new array

Rewrite table — loop → vectorised

Common mistakes

• ❌ "apply is vectorised" — it is not; it is a Python loop with a nicer syntax. ✅ It is fine for small data or complex logic, but reach for column expressions first.
• ❌ Timing a single run and trusting the exact number. ✅ Run timings a few times; compare ratios, not absolute milliseconds.
• ❌ Growing a DataFrame row by row in a loop (df.loc[len(df)] = ... or repeated concat). ✅ Collect rows in a Python list of dicts, then build one DataFrame at the end.
• ❌ Converting a column to a list, looping, and converting back. ✅ Keep the data in pandas and use column operations.

Interview tip

Q: Your teammate's df.apply(f, axis=1) takes 10 minutes on 5 million rows. How do you speed it up? A: Look inside f and rewrite it with column operations: arithmetic directly on columns, np.where/np.select/pd.cut for conditions, .str/.dt accessors for text and dates, map for look-ups, and groupby for per-group logic. These run as compiled loops and are typically 10–1000× faster. Only if the logic truly can't be vectorised, fall back to itertuples (still much faster than iterrows).