Replacing & Fixing Values
Fix wrong or inconsistent values with replace(), recode categories with map(), tidy messy text and rename columns.
Missing values are only one kind of dirt. The IPL data also has inconsistent values: the city appears as both "Bangalore" and "Bengaluru", and franchises were renamed ("Delhi Daredevils" → "Delhi Capitals", "Deccan Chargers" → "Sunrisers Hyderabad" in Munna's analysis). Counting matches per city or per team is wrong until these are unified. This topic covers the tools for fixing values: replace(), map(), the .str cleaning methods and rename().
Definition
replace()swaps specific listed values for new ones and leaves every other value unchanged.map()looks up every value of a Series in a dict (or passes it to a function) and returns the result — values not found becomeNaN.rename()changes column or index labels, not the data.
- Specific listed values —
replaceis targeted: "change these, keep the rest". - Looks up every value —
mapis a full translation; anything missing from the dict has no translation, so it becomesNaN. - Labels, not data —
renameonly touches the header names (or the index).
Syntax
python
df.replace(to_replace, value=None, regex=False) # also on a Series
series.map(arg, na_action=None) # arg = dict, Series or function
df.rename(columns=None, index=None, errors="ignore")
df.columns = [...] # replace all names at once| Parameter | Meaning | Default |
|---|---|---|
to_replace | value, list, or dict of {old: new} (or {column: {old: new}}) | required |
value | new value when to_replace is a value or list | None |
regex | treat to_replace as regular expressions | False |
arg (map) | dict / Series for lookup, or a function applied to each value | required |
na_action (map) | "ignore" = don't pass NaN to the function | None |
columns (rename) | dict {old_name: new_name} or a function like str.lower | None |
errors (rename) | "raise" to error if an old name does not exist | "ignore" |
Example — replace() to unify names
python
ipl = pd.DataFrame({
"city": ["Bangalore", "Bengaluru", "Delhi", "Hyderabad", "Bengaluru"],
"team1": ["RCB", "RCB", "Delhi Daredevils", "Deccan Chargers", "RCB"],
"result": ["runs", "wickets", "runs", "no result", "tie"],
})
print(ipl["city"].value_counts())
ipl2 = ipl.replace({
"Bengaluru": "Bangalore",
"Delhi Daredevils": "Delhi Capitals",
"Deccan Chargers": "Sunrisers Hyderabad",
})
print(ipl2)
print(ipl2["city"].value_counts())text
city
Bengaluru 2
Bangalore 1
Delhi 1
Hyderabad 1
Name: count, dtype: int64
city team1 result
0 Bangalore RCB runs
1 Bangalore RCB wickets
2 Delhi Delhi Capitals runs
3 Hyderabad Sunrisers Hyderabad no result
4 Bangalore RCB tie
city
Bangalore 3
Delhi 1
Hyderabad 1
Name: count, dtype: int64How it works: a flat dict {old: new} is applied to every column. Values not in the dict stay exactly as they were. After the fix, Bangalore correctly shows 3 matches instead of being split 1 + 2.
Example — replace in one column only, or with a list
python
print(ipl.replace({"result": {"no result": "NR", "tie": "Tie"}}))
print(ipl["result"].replace(["runs", "wickets"], "decided").tolist())text
city team1 result
0 Bangalore RCB runs
1 Bengaluru RCB wickets
2 Delhi Delhi Daredevils runs
3 Hyderabad Deccan Chargers NR
4 Bengaluru RCB Tie
['decided', 'decided', 'decided', 'no result', 'tie']How it works: the nested dict {column: {old: new}} limits the change to result. A list of old values with one new value maps all of them to the same thing.
Example — map() for recoding categories
map() is ideal for turning short codes into full names (Munna's team_map example):
python
players = pd.DataFrame({"player": ["Virat", "Rohit", "Dhoni", "Gill"],
"team": ["RCB", "MI", "CSK", "GT"]})
team_map = {"RCB": "Royal Challengers Bangalore", "MI": "Mumbai Indians",
"CSK": "Chennai Super Kings"}
players["team_full"] = players["team"].map(team_map)
players["team_full_r"] = players["team"].replace(team_map)
print(players)text
player team team_full team_full_r
0 Virat RCB Royal Challengers Bangalore Royal Challengers Bangalore
1 Rohit MI Mumbai Indians Mumbai Indians
2 Dhoni CSK Chennai Super Kings Chennai Super Kings
3 Gill GT NaN GTHow it works: GT is not in the dict. map gives NaN for it (no translation found), while replace leaves GT unchanged. That is the key difference.
map also accepts a function — great for grading:
python
runs = pd.Series([120, 45, 75, 10])
print(runs.map(lambda r: "Century" if r >= 100 else ("Fifty" if r >= 50 else "Low")))text
0 Century
1 Low
2 Fifty
3 Low
dtype: strComparison — replace vs map
replace() | map() | |
|---|---|---|
| Works on | DataFrame and Series | Series (DataFrame has its own DataFrame.map for element-wise functions) |
| Values not in the dict | kept unchanged | become NaN |
| Accepts a function | no | yes |
| Best for | fixing a few wrong spellings | translating every value (codes → names, grades) |
Example — fixing messy text
Text often arrives with extra spaces and random capitalisation. The .str accessor (covered fully in Text & Strings) fixes it column-wide:
python
messy = pd.Series([" mumbai", "MUMBAI ", "Mumbai", " chennai "])
print(messy.str.strip().str.title().value_counts())text
Mumbai 3
Chennai 1
Name: count, dtype: int64Example — renaming columns
python
raw = pd.DataFrame({"Player Name": ["Virat"], "Total Runs ": [741], "SR": [154.7]})
print(raw.columns.tolist())
print(raw.rename(columns={"Player Name": "player", "SR": "strike_rate"}).columns.tolist())
clean_cols = raw.columns.str.strip().str.lower().str.replace(" ", "_")
print(clean_cols.tolist())text
['Player Name', 'Total Runs ', 'SR']
['player', 'Total Runs ', 'strike_rate']
['player_name', 'total_runs', 'sr']How it works: rename(columns={...}) changes only the names you list. For a full clean-up of every header, use the .str methods on df.columns (it is an Index of strings) and assign back: raw.columns = clean_cols.
Example — conditional fixes with loc and mask
When the fix depends on a condition (e.g. a negative run count is impossible):
python
df = pd.DataFrame({"player": ["A", "B", "C"], "runs": [45, -5, 300]})
df.loc[df["runs"] < 0, "runs"] = np.nan
print(df)
print(df["runs"].mask(df["runs"] > 200, 200).tolist())text
player runs
0 A 45.0
1 B NaN
2 C 300.0
[45.0, nan, 200.0]df.loc[condition, "col"] = value is the pandas 3-safe way to change selected cells. mask(cond, other) replaces values where the condition is True.
Common mistakes
- ❌ Using
mapto fix a couple of spellings → every other value becomesNaN. ✅ Usereplacefor partial fixes. - ❌
df["city"][df["city"] == "Bengaluru"] = "Bangalore"— chained assignment never changesdfin pandas 3. ✅df.loc[df["city"] == "Bengaluru", "city"] = "Bangalore"orreplace. - ❌
df.rename({"SR": "strike_rate"})— withoutcolumns=it renames the index. ✅df.rename(columns={...}). - ❌ Forgetting that
replacematches whole values by default —"Bengaluru Stadium"is not changed by{"Bengaluru": ...}. ✅ Use.str.replace("Bengaluru", "Bangalore")for substrings.
Interview tip
Q: What happens to values that are missing from the dict in Series.map(dict)?
A: They become NaN. If you want unmatched values to stay unchanged, use replace(dict) (or map(dict).fillna(series)).
Key points to remember
- replace({old: new}) changes only the listed values and keeps everything else unchanged.
- Use a nested dict {column: {old: new}} to replace inside one column only.
- Series.map(dict) translates every value; values not in the dict become NaN.
- map() also accepts a function, e.g. map(lambda r: 'Fifty' if r >= 50 else 'Low').
- rename(columns={...}) changes column labels; df.columns.str.lower() etc. cleans all headers at once.
- Fix values by condition with df.loc[cond, 'col'] = value — chained assignment does nothing in pandas 3.
- Clean text with .str.strip().str.title() before counting or de-duplicating.
Try it yourself
Unify renamed IPL teams with replace()
map() vs replace() with a partial dict
Clean column headers in one line
Practice: recode roles and fix impossible values
Revision Flashcards
Tap a card to flip it and see the answer.