First Look at a New Dataset

You have just loaded an IPL matches file. Before any analysis — "which team won the most?", "who is the best finisher?" — you must understand the data: How big is it? What columns exist? Are the types right? What is missing? Skipping this step is the #1 reason beginners get wrong answers. Munna's notebooks start every dataset (Euro debt, car prices, IPL) with the same few commands — this topic turns them into a checklist.

All examples use this small matches file (one match has no result, one city is missing):

import pandas as pd
import io

pd.set_option("display.width", 120)

csv_text = """match_id,season,team1,team2,winner,player_of_match,win_by_runs,city
1,2023,GT,CSK,GT,Shubman Gill,0,Ahmedabad
2,2023,PBKS,KKR,PBKS,Arshdeep Singh,7,Mohali
3,2023,LSG,DC,LSG,Mark Wood,50,Lucknow
4,2023,SRH,RR,RR,Jos Buttler,72,
5,2023,RCB,MI,RCB,Faf du Plessis,0,Bengaluru
6,2024,CSK,RCB,CSK,Mustafizur Rahman,0,Chennai
7,2024,PBKS,DC,PBKS,Sam Curran,0,Mohali
8,2024,KKR,SRH,KKR,Andre Russell,4,Kolkata
9,2024,RR,LSG,RR,Sanju Samson,20,Jaipur
10,2024,GT,MI,,,,Ahmedabad
"""
matches = pd.read_csv(io.StringIO(csv_text))

Definition

Exploratory first look (the start of EDA — Exploratory Data Analysis) is the habit of running a fixed set of quick inspection attributes and methods on a new DataFrame to learn its size, structure, data types, value ranges and missing values before analysing it.

• Inspection attributes — shape, columns, dtypes, ndim, size — no brackets, they are stored facts.
• Inspection methods — head(), info(), describe()… — called with brackets; they compute a summary.
• Missing values — NaN cells; they change dtypes, break calculations and must be counted early.

The checklist (syntax)

df.shape                  # (rows, columns)
df.head(n) / df.tail(n)   # first / last n rows (default 5)
df.sample(n)              # n random rows
df.columns                # column labels
df.info()                 # index, columns, non-null counts, dtypes, memory
df.dtypes                 # dtype of each column
df.describe()             # statistics of numeric columns
df.describe(include="str")    # summary of text columns
df.isna().sum()           # missing values per column
df.isna().sum() * 100 / len(df)   # missing percentage
df.nunique()              # number of distinct values per column
df["col"].value_counts()  # frequency of each value in one column
df.duplicated().sum()     # number of fully duplicated rows

Step 1 — size and a peek: shape, head, tail

print(matches.shape)
print(matches.head())
print(matches.tail(3))

(10, 8)
   match_id  season team1 team2 winner player_of_match  win_by_runs       city
0         1    2023    GT   CSK     GT    Shubman Gill          0.0  Ahmedabad
1         2    2023  PBKS   KKR   PBKS  Arshdeep Singh          7.0     Mohali
2         3    2023   LSG    DC    LSG       Mark Wood         50.0    Lucknow
3         4    2023   SRH    RR     RR     Jos Buttler         72.0        NaN
4         5    2023   RCB    MI    RCB  Faf du Plessis          0.0  Bengaluru
   match_id  season team1 team2 winner player_of_match  win_by_runs       city
7         8    2024   KKR   SRH    KKR   Andre Russell          4.0    Kolkata
8         9    2024    RR   LSG     RR    Sanju Samson         20.0     Jaipur
9        10    2024    GT    MI    NaN             NaN          NaN  Ahmedabad

How it works: shape is a tuple (rows, columns) — an attribute, so no brackets. head() shows the top 5 rows, tail(3) the last 3. Already we spot problems: win_by_runs shows 0.0 (a float, though runs are whole numbers), and the last match has no winner. Also useful: len(df) = rows, df.ndim = 2, df.size = rows × columns (here 80).

Step 2 — structure: info()

matches.info()

<class 'pandas.DataFrame'>
RangeIndex: 10 entries, 0 to 9
Data columns (total 8 columns):
 #   Column           Non-Null Count  Dtype
---  ------           --------------  -----
 0   match_id         10 non-null     int64
 1   season           10 non-null     int64
 2   team1            10 non-null     str
 3   team2            10 non-null     str
 4   winner           9 non-null      str
 5   player_of_match  9 non-null      str
 6   win_by_runs      9 non-null      float64
 7   city             9 non-null      str
dtypes: float64(1), int64(2), str(5)
memory usage: 772.0 bytes

How it works: info() is the single most useful command. It shows the index type, every column with its non-null count (compare with 10 entries — any smaller number means missing values) and its dtype. Here four columns have 9 of 10 values. It also explains the float mystery: win_by_runs has a NaN, and NaN can only live in a float column. info() prints directly and returns None — don't wrap it in print(). (Memory usage may show a slightly different number on your machine.)

Step 3 — numbers: describe()

print(matches.describe())

       match_id       season  win_by_runs
count  10.00000    10.000000     9.000000
mean    5.50000  2023.500000    17.000000
std     3.02765     0.527046    26.334388
min     1.00000  2023.000000     0.000000
25%     3.25000  2023.000000     0.000000
50%     5.50000  2023.500000     4.000000
75%     7.75000  2024.000000    20.000000
max    10.00000  2024.000000    72.000000

How it works: for every numeric column: count (non-missing values), mean, std (standard deviation — how spread out), min, the 25% / 50% (median) / 75% percentiles, and max. Read it critically: the mean margin (17) is far above the median (4) — a few big wins (72, 50) pull the mean up. Also notice describe happily averages match_id and season, which is meaningless — they are labels, not quantities.

Step 4 — text columns: describe(include="str")

print(matches.describe(include="str"))

       team1 team2 winner player_of_match       city
count     10    10      9               9          9
unique     8     8      7               9          7
top       GT    DC   PBKS    Shubman Gill  Ahmedabad
freq       2     2      2               1          2

How it works: for text columns you get count, unique (distinct values), top (the most frequent value) and freq (how often top appears). When several values tie, top is just one of them. describe(include="all") shows numeric and text columns together.

Note (pandas 3): older tutorials (and Munna's notebook) write describe(include="object"). In pandas 3 text columns are dtype str, so use include="str". "object" still works for now but prints a deprecation warning.

Step 5 — missing values: isna().sum() and missing %

print(matches.isna().sum())
print((matches.isna().sum() * 100 / len(matches)).round(1))

match_id           0
season             0
team1              0
team2              0
winner             1
player_of_match    1
win_by_runs        1
city               1
dtype: int64
match_id            0.0
season              0.0
team1               0.0
team2               0.0
winner             10.0
player_of_match    10.0
win_by_runs        10.0
city               10.0
dtype: float64

How it works: isna() builds a True/False table (True = missing). .sum() adds each column — True counts as 1 — giving missing values per column. Multiplying by 100 and dividing by the number of rows gives the missing percentage, which decides what to do: a few % → drop or fill; 60 % → maybe drop the whole column. isnull() is an exact alias of isna(). To list only the columns that have gaps: missing = df.isna().sum(); missing[missing > 0].

Step 6 — variety: nunique() and value_counts()

print(matches.nunique())
print(matches["winner"].value_counts())

match_id           10
season              2
team1               8
team2               8
winner              7
player_of_match     9
win_by_runs         6
city                7
dtype: int64
winner
PBKS    2
RR      2
GT      1
LSG     1
RCB     1
CSK     1
KKR     1
Name: count, dtype: int64

How it works: nunique() counts distinct values per column (ignoring NaN). It tells you which columns are identifiers (match_id: 10 unique in 10 rows), which are categories (season: only 2 values) and which are free text. value_counts() then shows the frequency of each value in one column, most common first — normalize=True gives proportions instead of counts. Finally, matches.duplicated().sum() (here 0) tells you whether any row appears twice.

The checklist at a glance

Common mistakes

• ❌ df.shape() → TypeError: 'tuple' object is not callable. ✅ shape, columns, dtypes, size are attributes — no brackets.
• ❌ print(df.info()) prints the summary followed by None. ✅ Just call df.info().
• ❌ Assuming describe() covers text columns. ✅ It shows numeric columns only by default; use include="str" or "all".
• ❌ Using df.isna() alone on a big table (thousands of True/False). ✅ Add .sum() to get counts per column.
• ❌ Trusting mean of ID-like columns. ✅ IDs and years are labels; summarise them with nunique/value_counts instead.

Interview tip

Q: You receive a new CSV. What are the first five things you run? A: df.shape for size, df.head() to see real rows, df.info() for dtypes and non-null counts, df.describe() for numeric ranges and outliers, and df.isna().sum() for missing values per column — then nunique() / value_counts() on key categorical columns and duplicated().sum() before any cleaning.