Modifying Values Safely (Copy-on-Write)

Selecting data is half the job; the other half is changing it: correct Dhoni's runs, set every MI player's status, add bonus runs to a column. Pandas makes some ways of writing this look right while silently doing nothing. In pandas 3 the rules are finally simple and consistent thanks to Copy-on-Write — once you know the one rule ("one step, with loc"), you'll never be surprised again.

Definition

Copy-on-Write (CoW) is the pandas 3 rule that every DataFrame or Series obtained from another one — by indexing, filtering or a method — behaves as an independent copy. Modifying it never changes the original. (Internally pandas shares the memory and only makes a real copy at the moment one of them is written to — hence "copy on write".)

• Every derived object behaves as a copy — df["runs"], df[df["team"] == "MI"], df.head(), df.reset_index()… are all "new" objects as far as writing is concerned.
• Never changes the original — so the only way to modify df is to call the setter on df itself in a single step: df.loc[...] = ..., df.at[...] = ..., df.iloc[...] = ..., or df["col"] = ....
• Chained assignment — two indexing steps in a row followed by =, like df["runs"][0] = 999. The first step makes a (copy-like) object, the second writes into that temporary object, and the original df is untouched. Under CoW this never works.

Syntax

df.loc[row_label, "col"] = value          # one cell
df.loc[mask, "col"] = value               # all rows matching a condition
df.loc[mask, ["c1", "c2"]] = [v1, v2]     # several columns at once
df["col"] = df["col"] + 10                # a whole column (vectorised)
df["new"] = values                        # create a column
df.loc[mask, "new_col"] = value           # create + fill only some rows (others NaN)
df.at[row_label, "col"] = value           # single cell, fast
sub = df[mask].copy()                     # explicit independent copy to work on

Example — chained assignment does nothing

df = players[["player", "team", "runs"]].head(4).copy()
df["runs"][0] = 999
df[df["team"] == "MI"]["runs"] = 0
print(df)

           player team  runs
0     Virat Kohli  RCB   741
1    Rohit Sharma   MI   417
2        MS Dhoni  CSK   161
3  Jasprit Bumrah   MI     8

pandas also prints a warning for each line:

ChainedAssignmentError: A value is being set on a copy of a DataFrame or Series through chained assignment.
Such chained assignment never works to update the original DataFrame or Series, because the intermediate object on which we are setting values always behaves as a copy (due to Copy-on-Write).

Try using '.loc[row_indexer, col_indexer] = value' instead, to perform the assignment in a single step.

How it works: df["runs"] produced a Series that behaves as a copy; [0] = 999 wrote into that temporary Series, which was then thrown away. Same story for df[mask]["runs"] = 0: the filter created a new DataFrame. The table is unchanged — 741 and 417 are still there.

Example — the right way: one step with loc

df.loc[0, "runs"] = 750
df.loc[df["team"] == "MI", "runs"] = 0
print(df)
df.loc[df["team"] == "CSK", ["team", "runs"]] = ["Chennai", 170]
print(df)

           player team  runs
0     Virat Kohli  RCB   750
1    Rohit Sharma   MI     0
2        MS Dhoni  CSK   161
3  Jasprit Bumrah   MI     0
           player     team  runs
0     Virat Kohli      RCB   750
1    Rohit Sharma       MI     0
2        MS Dhoni  Chennai   170
3  Jasprit Bumrah       MI     0

How it works: df.loc[rows, cols] = value is one indexing operation called directly on df, so it modifies df. The row part can be a label (0) or a mask; the column part can be one name or a list — with a list, give one value per column.

Example — whole columns and new columns

df["runs"] = df["runs"] + 10
df.loc[df["runs"] < 100, "status"] = "Out of form"
print(df)
df["status"] = df["status"].fillna("OK")
print(df)

           player     team  runs       status
0     Virat Kohli      RCB   760          NaN
1    Rohit Sharma       MI    10  Out of form
2        MS Dhoni  Chennai   180          NaN
3  Jasprit Bumrah       MI    10  Out of form
           player     team  runs       status
0     Virat Kohli      RCB   760           OK
1    Rohit Sharma       MI    10  Out of form
2        MS Dhoni  Chennai   180           OK
3  Jasprit Bumrah       MI    10  Out of form

How it works: df["runs"] = ... replaces the entire column (vectorised — no loop). df.loc[mask, "status"] = ... on a column that doesn't exist yet creates it, filling non-matching rows with NaN; fillna then fills the rest. For an either/or column in one line, np.where(condition, value_if_true, value_if_false) is another popular option: df["grade"] = np.where(df["runs"] >= 500, "Star", "Regular").

Example — subsets and Series are independent

mi = df[df["team"] == "MI"]
mi.loc[:, "runs"] = 500
print(mi)
print(df)

s = df["runs"]
s.iloc[0] = 1
print(df.loc[0, "runs"], s.iloc[0])

           player team  runs       status
1    Rohit Sharma   MI   500  Out of form
3  Jasprit Bumrah   MI   500  Out of form
           player     team  runs       status
0     Virat Kohli      RCB   760           OK
1    Rohit Sharma       MI    10  Out of form
2        MS Dhoni  Chennai   180           OK
3  Jasprit Bumrah       MI    10  Out of form
760 1

How it works: mi is its own table now — editing it is perfectly fine (and gives no warning) but does not touch df. Likewise the Series s taken from df["runs"] behaves as a copy: changing it leaves df at 760. If you want to change df, write to df with loc. If you want a separate working table, df[mask].copy() makes that intention explicit.

pandas 2 vs pandas 3 — what changed

Note (pandas 3): if an old tutorial tells you to "fix SettingWithCopyWarning", the pandas 3 answer is always the same: do the assignment in one step on the original — df.loc[mask, "col"] = value — or call .copy() if you meant to work on a separate subset.

Common mistakes

• ❌ df["runs"][0] = 999 / df[mask]["runs"] = 0. ✅ df.loc[0, "runs"] = 999 / df.loc[mask, "runs"] = 0.
• ❌ Updating a filtered table and expecting the original to change. ✅ Filtered results are independent; write to the original with loc[mask, ...].
• ❌ Looping row by row (for i in range(len(df)): df.loc[i, ...] = ...). ✅ Use vectorised column maths, loc[mask], or np.where.
• ❌ Assigning a value of the wrong type into a numeric column. ✅ In pandas 3 this raises TypeError; convert the column first (df["runs"] = df["runs"].astype("float64")) or fix the value.
• ❌ df.loc[mask, ["team", "runs"]] = "x", 1, 2 (wrong number of values). ✅ Give exactly one value per selected column.

Interview tip

Q: What is chained assignment, and why does it not work in pandas 3? A: Chained assignment is setting a value through two consecutive indexing operations, like df["col"][row] = v or df[mask]["col"] = v. Under Copy-on-Write (always on in pandas 3), the first indexing step returns an object that behaves as a copy, so the assignment updates that temporary copy and the original DataFrame never changes. The fix is a single indexing call on the original: df.loc[row, "col"] = v.