Replacing & Extracting Text

Cleaning text is mostly two jobs: changing parts of each string (remove the "(c)" captain mark, turn "Delhi Daredevils" into "Delhi Capitals", strip non-digits from a phone number) and pulling out a useful piece (the year from "IPL 2020", the runs from "82* (49)", the engine size from a car name). str.replace does the first, str.extract the second, and slice, len and zfill handle fixed positions and lengths.

Definition

str.replace(pat, repl) returns each string with every occurrence of pat (plain text, or a regex when regex=True) replaced by repl — it works on parts of strings. str.extract(pat) applies a regex with capture groups ( … ) to each string and returns the text matched by each group as new columns (or a Series), with NaN where there is no match.

• Every occurrence / parts — unlike Series.replace(), which swaps whole values, str.replace edits inside the text.
• Capture group — the part of a regex inside parentheses; whatever it matches is "captured" and returned.
• New columns — one column per group; a named group (?P<name>…) becomes the column name.

Syntax

s.str.replace(pat, repl, n=-1, case=None, flags=0, regex=False)
s.str.extract(pat, flags=0, expand=True)
s.str.extractall(pat)                      # every match, not just the first
s.str.slice(start=None, stop=None, step=None)   # same as s.str[start:stop:step]
s.str.len()
s.str.pad(width, side="left", fillchar=" ")
s.str.zfill(width)                          # pad with zeros on the left

Note: since pandas 2.0, str.replace uses regex=False by default. Very old tutorials relied on regex being the default — always pass regex=True when you use a pattern.

Example — plain replace

squad = pd.Series(["MS Dhoni (c)", "Ruturaj Gaikwad", "Ravindra Jadeja", "Devon Conway (wk)"])
print(squad.str.replace(" (c)", "").tolist())
print(squad.str.replace(" (c)", "").str.replace(" (wk)", "").tolist())

['MS Dhoni', 'Ruturaj Gaikwad', 'Ravindra Jadeja', 'Devon Conway (wk)']
['MS Dhoni', 'Ruturaj Gaikwad', 'Ravindra Jadeja', 'Devon Conway']

How it works: with regex=False, "(c)" is ordinary text, so the brackets need no escaping. Each call replaces every occurrence inside each string.

Example — regex replace

print(squad.str.replace(r"\s*\(.*\)", "", regex=True).tolist())

phones = pd.Series(["+91 98765-43210", "(022) 2345 6789", "98765 43210"])
print(phones.str.replace(r"\D", "", regex=True).tolist())

['MS Dhoni', 'Ruturaj Gaikwad', 'Ravindra Jadeja', 'Devon Conway']
['919876543210', '02223456789', '9876543210']

How it works:

• \s*\(.*\) = optional spaces, a literal (, anything, a literal ) — removes any bracketed tag in one go.
• \D = "any character that is not a digit"; replacing them with "" keeps only digits.

Example — reorder with groups in repl

names = pd.Series(["Kohli, Virat", "Sharma, Rohit", "Gill, Shubman"])
print(names.str.replace(r"(\w+), (\w+)", r"\2 \1", regex=True).tolist())

['Virat Kohli', 'Rohit Sharma', 'Shubman Gill']

\1 and \2 in the replacement refer to what groups 1 and 2 captured — so "Kohli, Virat" becomes "Virat Kohli".

Example — extract with capture groups

Scorecard text like "82* (49)" hides two numbers — runs and balls:

cards = pd.DataFrame({"player": ["Virat", "Rohit", "Gill", "Pant"],
                      "score": ["82* (49)", "45 (30)", "DNB", "101 (58)"]})
parts = cards["score"].str.extract(r"(\d+)\*? \((\d+)\)")
print(parts)

     0    1
0   82   49
1   45   30
2  NaN  NaN
3  101   58

parts = cards["score"].str.extract(r"(?P<runs>\d+)\*? \((?P<balls>\d+)\)")
cards = cards.join(parts.astype("Int64"))
cards["not_out"] = cards["score"].str.contains("*", regex=False)
print(cards)

  player     score  runs  balls  not_out
0  Virat  82* (49)    82     49     True
1  Rohit   45 (30)    45     30    False
2   Gill       DNB  <NA>   <NA>    False
3   Pant  101 (58)   101     58    False

How it works: each ( … ) is a group, so we get two columns. (?P<runs>…) names the group, which becomes the column name. "DNB" has no match → <NA>. The extracted text is converted to numbers with astype("Int64") (nullable integers).

Example — one group: expand=False

seasons = pd.Series(["IPL 2008", "IPL 2016 Final", "Season-2020"])
print(seasons.str.extract(r"(\d{4})", expand=False))

0    2008
1    2016
2    2020
dtype: str

With one group and expand=False you get a Series instead of a one-column DataFrame. \d{4} means exactly four digits.

Example — slice, len, pad and zfill

codes = pd.Series(["Mumbai Indians", "Chennai Super Kings", "Delhi Capitals"])
print(pd.DataFrame({
    "first3": codes.str.slice(0, 3).str.upper(),
    "last4": codes.str[-4:],
    "length": codes.str.len(),
}))
jersey = pd.Series([7, 18, 45, 333]).astype(str)
print(jersey.str.zfill(3).tolist())
print(jersey.str.pad(5, side="right", fillchar="_").tolist())

  first3 last4  length
0    MUM  ians      14
1    CHE  ings      19
2    DEL  tals      14
['007', '018', '045', '333']
['7____', '18___', '45___', '333__']

How it works: slice(0, 3) (same as .str[:3]) cuts each string by position. len() counts characters including spaces. zfill(3) pads with zeros to length 3 ("7" → "007") and never cuts longer strings ("333" stays). Numbers must become text first (astype(str)).

Comparison — Series.replace vs str.replace

Common mistakes

• ❌ str.replace(r"\d", "") without regex=True — looks for the literal text \d, changes nothing. ✅ Add regex=True.
• ❌ extract with a pattern that has no parentheses → ValueError: pattern contains no capture groups. ✅ Wrap the part you want in ( ).
• ❌ Forgetting that extracted values are text. ✅ Convert with astype(int) / "Int64" / pd.to_numeric.
• ❌ zfill on an int column → AttributeError. ✅ s.astype(str).str.zfill(3).

Interview tip

Q: How would you pull the domain out of an email column? A: df["email"].str.extract(r"@([\w.]+)", expand=False) — the group captures everything after @.