Converting to Datetime (pd.to_datetime)

When you load the IPL matches CSV, the date column looks like dates — 2008-04-18 — but info() shows it as text (object in Munna's notebook, str in pandas 3). As text you cannot ask "which month?", "how many days between matches?" or "sort by date" reliably. The very first date step in every project is converting the column with pd.to_datetime, exactly as Munna does: ipl2['date'] = pd.to_datetime(ipl2['date']).

Definition

pd.to_datetime() parses strings, numbers or date parts into pandas datetime values. A single value becomes a Timestamp (one moment in time); a list or Series becomes a column of dtype datetime64, on which date arithmetic, the .dt accessor, date filtering and resampling all work. Values that cannot be parsed either raise an error or become NaT ("Not a Time") with errors="coerce".

• Parses — reads the text and understands which part is the year, month, day, hour…
• Timestamp — pandas' version of Python's datetime.datetime, with extra features.
• datetime64 — the column dtype. Internally each value is a number counting time units since 1970-01-01.

Note (pandas 3): the default resolution is now microseconds, so the dtype prints as datetime64[us]. Older tutorials (and Munna's notebook) show datetime64[ns] (nanoseconds). For everyday analysis they behave the same.

Syntax

pd.to_datetime(arg, errors="raise", dayfirst=False, yearfirst=False,
               utc=False, format=None, unit=None, origin="unix")

Example — the basic conversion

matches = pd.DataFrame({
    "match": ["RCB vs KKR", "CSK vs KXIP", "DC vs RR", "MI vs RCB"],
    "date":  ["2008-04-18", "2008-04-19", "2008-04-19", "2008-04-20"],
})
print(matches.dtypes)
matches["date"] = pd.to_datetime(matches["date"])
print(matches)
print(matches.dtypes)
print(type(matches["date"][0]))

match    str
date     str
dtype: object
         match       date
0   RCB vs KKR 2008-04-18
1  CSK vs KXIP 2008-04-19
2     DC vs RR 2008-04-19
3    MI vs RCB 2008-04-20
match               str
date     datetime64[us]
dtype: object
<class 'pandas.Timestamp'>

How it works: the ISO format YYYY-MM-DD is unambiguous, so pandas parses it directly. The column dtype changes from str to datetime64[us], and each element is a Timestamp.

Example — format= for non-standard text

print(pd.to_datetime("22-03-2024 19:30", format="%d-%m-%Y %H:%M"))
print(pd.to_datetime("March 22, 2024", format="%B %d, %Y"))
print(pd.to_datetime(pd.Series(["22/03/24", "29/03/24"]), format="%d/%m/%y"))

2024-03-22 19:30:00
2024-03-22 00:00:00
0   2024-03-22
1   2024-03-29
dtype: datetime64[us]

Giving format makes parsing strict (any row that does not fit raises an error — or becomes NaT with errors="coerce") and much faster on big files.

Example — the day/month trap: dayfirst

Indian dates are usually written day first (05/04/2024 = 5 April). pandas assumes month first unless told otherwise:

dates = pd.Series(["05/04/2024", "13/04/2024"])
print(pd.to_datetime(dates, dayfirst=True))

0   2024-04-05
1   2024-04-13
dtype: datetime64[us]

pd.to_datetime(pd.Series(["05/04/2024", "13/04/2024"]))

ValueError: time data "13/04/2024" doesn't match format "%m/%d/%Y". You might want to try:
    - passing `format` if your strings have a consistent format;
    - passing `format='ISO8601'` if your strings are all ISO8601 but not necessarily in exactly the same format;
    - passing `format='mixed'`, and the format will be inferred for each element individually. You might want to use `dayfirst` alongside this.

How it works: without dayfirst, pandas reads 05/04/2024 as May 4, fixes that month-first format for the column, and then fails on 13/04/2024 (there is no month 13). Worse, if all your days were ≤ 12, there would be no error at all — just silently wrong dates. Always pass dayfirst=True or an explicit format for dd/mm/yyyy data.

Example — bad values: errors="coerce"

raw = pd.Series(["2024-03-22", "TBD", "2024-04-07", None, "2024-02-30"])
print(pd.to_datetime(raw, errors="coerce"))

0   2024-03-22
1          NaT
2   2024-04-07
3          NaT
4          NaT
dtype: datetime64[us]

pd.to_datetime(raw)

ValueError: time data "TBD" doesn't match format "%Y-%m-%d". You might want to try:
    - passing `format` if your strings have a consistent format;
    - passing `format='ISO8601'` if your strings are all ISO8601 but not necessarily in exactly the same format;
    - passing `format='mixed'`, and the format will be inferred for each element individually. You might want to use `dayfirst` alongside this.

How it works: "TBD" and the impossible date 2024-02-30 cannot be parsed. With the default errors="raise" you get an error; with "coerce" they become NaT, which you can then find with isna() and handle like any missing value.

Example — mixed formats in one column

mixed = pd.Series(["2024-03-22", "2024/03/23", "24 March 2024"])
print(pd.to_datetime(mixed, errors="coerce"))
print(pd.to_datetime(mixed, format="mixed"))

0   2024-03-22
1          NaT
2          NaT
dtype: datetime64[us]
0   2024-03-22
1   2024-03-23
2   2024-03-24
dtype: datetime64[us]

How it works: pandas guesses the format from the first value (%Y-%m-%d) and applies it to all rows — so with errors="coerce" the other two silently become NaT! format="mixed" tells pandas to guess the format for each value separately (slower, but handles messy columns). Always check isna().sum() after converting.

Example — numbers and separate columns

print(pd.to_datetime(1711116000, unit="s"))                # Unix time (seconds since 1970)
parts = pd.DataFrame({"year": [2024, 2024], "month": [3, 5], "day": [22, 26]})
print(pd.to_datetime(parts))

2024-03-22 14:00:00
0   2024-03-22
1   2024-05-26
dtype: datetime64[us]

unit="s" reads a number as seconds since 1970-01-01 (UTC) — the format many APIs use. A DataFrame with year, month, day (and optionally hour, minute) columns is assembled into one date.

Parsing while reading a CSV

import io
csv = "match,date\nRCB vs KKR,2008-04-18\nCSK vs KXIP,2008-04-19\n"
df = pd.read_csv(io.StringIO(csv), parse_dates=["date"])
print(df.dtypes)

match               str
date     datetime64[us]
dtype: object

parse_dates=["date"] converts during loading — one less step.

Common mistakes

• ❌ Leaving dates as text and sorting/comparing them. ✅ Convert first; then sort_values("date") is truly chronological.
• ❌ dd/mm/yyyy without dayfirst=True → swapped day and month. ✅ dayfirst=True or format="%d/%m/%Y".
• ❌ Using errors="coerce" and never checking how many NaT were created. ✅ df["date"].isna().sum() right after.
• ❌ Mixed formats parsed with the first value's format. ✅ format="mixed" (or clean the text first).

Interview tip

Q: What does errors="coerce" do in pd.to_datetime? A: Values that cannot be parsed become NaT instead of raising an error, so the conversion finishes and you can deal with the bad rows as missing values.