Data Types & astype

Data loaded from CSV files, web pages or Excel often has the wrong types: runs stored as text, dates stored as text, "True"/"False" stored as text. With the wrong type, maths silently goes wrong — adding text joins it instead of summing. Checking and fixing dtypes is one of the first steps of every cleaning job.

Definition

A dtype (data type) is the kind of value a column holds — it decides how values are stored in memory and which operations are allowed. astype(dtype) converts a Series or DataFrame to another dtype and returns a new object. pd.to_numeric converts values to numbers and, with errors="coerce", turns anything it cannot parse into NaN instead of failing.

• Decides storage and operations — int64 can be summed and averaged; str supports .str methods; datetime64 supports date maths and the .dt accessor.
• Returns a new object — assign it back: df["runs"] = df["runs"].astype(int).
• Coerce — "force": bad values become missing instead of raising an error.

The main pandas dtypes

Syntax

df.dtypes                                 # check every column
s.astype(dtype, errors="raise")           # convert one column
df.astype({"col1": "int64", "col2": "float64"})   # several columns at once
pd.to_numeric(arg, errors="raise", downcast=None)
pd.to_datetime(arg, errors="raise", format=None)
df.select_dtypes(include=None, exclude=None)

Example — a table where everything is text

raw = pd.DataFrame({
    "match_id": ["101", "102", "103", "104"],
    "date": ["2023-03-31", "2023-04-01", "2023-04-02", "2023-04-03"],
    "team": ["GT", "PBKS", "LSG", "RR"],
    "runs": ["178", "191", "193", "203"],
    "economy": ["8.9", "9.55", "9.65", "10.15"],
    "won": ["True", "False", "True", "True"],
})
print(raw.dtypes)
print(raw["runs"].sum())

match_id    str
date        str
team        str
runs        str
economy     str
won         str
dtype: object
178191193203

How it works: every column is str, so sum() concatenated the text "178" + "191" + … instead of adding numbers. This is exactly the kind of silent bug that wrong dtypes cause.

Example — convert each column to the right type

df = raw.copy()
df["match_id"] = df["match_id"].astype("int64")
df["runs"] = df["runs"].astype(int)
df["economy"] = df["economy"].astype("float64")
df["date"] = pd.to_datetime(df["date"])
df["team"] = df["team"].astype("category")
df["won"] = df["won"].map({"True": True, "False": False})
print(df.dtypes)
print(df)
print(df["runs"].sum())

match_id             int64
date        datetime64[us]
team              category
runs                 int64
economy            float64
won                   bool
dtype: object
   match_id       date  team  runs  economy    won
0       101 2023-03-31    GT   178     8.90   True
1       102 2023-04-01  PBKS   191     9.55  False
2       103 2023-04-02   LSG   193     9.65   True
3       104 2023-04-03    RR   203    10.15   True
765

How it works:

• astype("int64") / astype(int) — both give int64; astype(float) gives float64.
• pd.to_datetime parses date strings into real dates (datetime64) so you can sort by date, extract the weekday with .dt.day_name(), and so on.
• astype("category") stores each distinct team once and keeps small integer codes — great for columns with few distinct values repeated many times.
• won uses .map with an explicit dict — see the bool trap below.
• Now sum() gives the real total 765.

You can also convert several columns in one call with a dict: raw.astype({"match_id": "int64", "runs": "int64", "economy": "float64"}).

Note (pandas 3): pd.to_datetime now infers the time resolution, so you see datetime64[us] here; old tutorials show datetime64[ns]. The dates are the same.

The bool trap

print(pd.Series(["True", "False"]).astype(bool).tolist())

[True, True]

astype(bool) follows Python's rule: any non-empty string is True — including "False"! Convert text booleans with .map({"True": True, "False": False}) (or compare: s == "True").

Example — messy numbers: astype fails, to_numeric(errors="coerce") works

dirty = pd.Series(["45", "78", "DNB", "101", None])   # DNB = Did Not Bat
dirty.astype(int)

ValueError: invalid literal for int() with base 10: 'DNB'

print(pd.to_numeric(dirty, errors="coerce"))

0     45.0
1     78.0
2      NaN
3    101.0
4      NaN
dtype: float64

How it works: astype is strict — one bad value stops the whole conversion. pd.to_numeric(..., errors="coerce") converts what it can and replaces "DNB" (and the missing value) with NaN. The result is float64 because NaN needs a float. Without errors="coerce" you would get ValueError: Unable to parse string "DNB" at position 2 — which at least tells you where the bad value is.

To keep whole numbers and missing values, convert to the nullable Int64 dtype:

print(pd.to_numeric(dirty, errors="coerce").astype("Int64"))

0      45
1      78
2    <NA>
3     101
4    <NA>
dtype: Int64

Other conversions worth knowing

Comparison — astype vs to_numeric vs to_datetime

Common mistakes

• ❌ df["runs"].astype(int) without assigning → nothing changes. ✅ df["runs"] = df["runs"].astype(int).
• ❌ astype(bool) on "True"/"False" text → everything True. ✅ .map({"True": True, "False": False}).
• ❌ astype(int) on a column with NaN → IntCastingNaNError. ✅ astype("Int64"), or fillna first.
• ❌ astype(int) on text like "DNB" or "1,200". ✅ Clean first (.str.replace(",", "")) or use pd.to_numeric(errors="coerce").
• ❌ Using astype("datetime64[ns]") on messy date strings. ✅ pd.to_datetime(s, format=..., errors="coerce").

Interview tip

Q: A "price" column was read as text because a few rows contain "N/A". How do you convert it to numbers? A: df["price"] = pd.to_numeric(df["price"], errors="coerce") — valid values become numbers and "N/A" becomes NaN. Then decide how to handle the NaNs (drop or fill). astype(float) would fail on the first "N/A".