set_index & reset_index

Every DataFrame has an index — the row labels shown on the left. By default it's just 0, 1, 2… which says nothing about the data. If each row is a player, it's much more useful for the index to be the player name, so you can write df.loc["MS Dhoni"]. And after filtering or sorting, the old 0…n labels get jumbled (0, 6, 1…) and you may want a fresh 0, 1, 2. set_index and reset_index — from Munna's video 05 — move data between the columns and the index.

Definition

DataFrame.set_index(keys) returns a new DataFrame whose row index is replaced by one or more existing columns. DataFrame.reset_index() does the reverse: it moves the index back into the columns and installs a fresh default RangeIndex (0, 1, 2…), or with drop=True simply discards the old index.

• Row index — the labels used by loc, at, alignment, joins and sorting by index.
• One or more columns — a single column gives a normal Index; a list of columns gives a MultiIndex (hierarchical labels like ("MI", "Rohit Sharma")).
• RangeIndex — the memory-light default index 0 … n-1.
• Returns a new DataFrame — the original is unchanged unless you reassign: df = df.set_index("player").

Syntax

df.set_index(keys, drop=True, append=False)
df.reset_index(level=None, drop=False, names=None)

Example — set_index

small = players[["player", "team", "runs"]].head(4)
p = small.set_index("player")
print(p)
print(p.index)

               team  runs
player
Virat Kohli     RCB   741
Rohit Sharma     MI   417
MS Dhoni        CSK   161
Jasprit Bumrah   MI     8
Index(['Virat Kohli', 'Rohit Sharma', 'MS Dhoni', 'Jasprit Bumrah'], dtype='str', name='player')

How it works: the player column became the row labels. The word player printed on its own line above the index is the index name. It's no longer a column (drop=True by default) — p.columns is now just team, runs. Now p.loc["MS Dhoni"] works, and so does p.at["MS Dhoni", "runs"]. With drop=False the column is copied into the index and kept as a column.

Example — reset_index, with and without drop

print(p.reset_index())             # index goes back to being a column
print(p.reset_index(drop=True))    # index is thrown away

           player team  runs
0     Virat Kohli  RCB   741
1    Rohit Sharma   MI   417
2        MS Dhoni  CSK   161
3  Jasprit Bumrah   MI     8
  team  runs
0  RCB   741
1   MI   417
2  CSK   161
3   MI     8

How it works: reset_index() inserts the old index as the first column (named after the index, here player) and gives a new 0, 1, 2, 3 index. With drop=True the player names are lost — use it only when the old index is meaningless, like jumbled numbers after a filter.

Example — the most common use: renumber after filtering

f = players[players["runs"] > 300]
print(f)
print(f.reset_index(drop=True))

         player team    role  runs  wickets  strike_rate
0   Virat Kohli  RCB  Batter   741        0        154.7
1  Rohit Sharma   MI  Batter   417        0        150.0
6  Shubman Gill   GT  Batter   426        0        147.4
         player team    role  runs  wickets  strike_rate
0   Virat Kohli  RCB  Batter   741        0        154.7
1  Rohit Sharma   MI  Batter   417        0        150.0
2  Shubman Gill   GT  Batter   426        0        147.4

How it works: after filtering, the labels are 0, 1, 6, so f.loc[2] would raise KeyError: 2. reset_index(drop=True) renumbers them 0, 1, 2 without adding an index column. (Forgetting drop=True here gives you an unwanted extra column called index holding 0, 1, 6.)

Example — a multi-column index (briefly)

r = small.set_index(["team", "player"])
print(r)
print(r.loc["MI"])
print(r.reset_index(level="player"))

                     runs
team player
RCB  Virat Kohli      741
MI   Rohit Sharma     417
CSK  MS Dhoni         161
MI   Jasprit Bumrah     8
                runs
player
Rohit Sharma     417
Jasprit Bumrah     8
              player  runs
team
RCB      Virat Kohli   741
MI      Rohit Sharma   417
CSK         MS Dhoni   161
MI    Jasprit Bumrah     8

How it works: a list of columns creates a MultiIndex with two levels: team then player. r.loc["MI"] selects every row whose first level is MI; a full key is a tuple: r.loc[("MI", "Jasprit Bumrah"), "runs"] → 8. reset_index(level="player") moves just one level back to the columns. You'll meet MultiIndexes a lot after groupby with several columns — and reset_index() is the usual way to flatten them back into a normal table.

Duplicate index labels

An index doesn't have to be unique. small.set_index("team") works, but loc["MI"] then returns two rows:

t = small.set_index("team")
print(t.loc["MI"])
print(t.index.is_unique)

              player  runs
team
MI      Rohit Sharma   417
MI    Jasprit Bumrah     8
False

Choose a column whose values identify each row (player name, match_id) when you want loc to return single rows. Check with df.index.is_unique.

Note (pandas 3): set_index(..., verify_integrity=True) is deprecated — check result.index.is_unique instead. And inplace=True is discouraged everywhere: write df = df.set_index("player") rather than df.set_index("player", inplace=True).

Common mistakes

• ❌ df.set_index("player") and then df.loc["MS Dhoni"] fails. ✅ set_index returns a new DataFrame — reassign: df = df.set_index("player").
• ❌ A mysterious index (or level_0) column appears. ✅ You called reset_index() on a meaningless index — use reset_index(drop=True).
• ❌ Using drop=True on a meaningful index (player names) and losing them. ✅ Use plain reset_index() to keep them as a column.
• ❌ KeyError: "None of ['captain'] are in the columns". ✅ set_index needs an existing column name — check df.columns.

Interview tip

Q: What does reset_index(drop=True) do, and when do you use it? A: It replaces the current index with a fresh 0…n-1 RangeIndex without inserting the old index as a column. Use it after filtering, sorting or concatenating, when the old labels are just leftover row numbers that no longer mean anything.