What is Pandas?

In the NumPy notes you stored scores in arrays — fast, but every value had to be the same type and rows/columns were known only by position (scores[2, 5]). Real cricket data does not look like that. An IPL batting table has a player name (text), a team (text), a season (integer), runs (integer) and an average (decimal) — five different types side by side, and you want to ask for things by name: "give me the runs column", "show me Virat Kohli's row".

Try putting that table into a plain NumPy array and see what happens:

import numpy as np

arr = np.array([["Virat Kohli", "RCB", 973], ["Jos Buttler", "RR", 863]])
print(arr)
print(arr.dtype)

[['Virat Kohli' 'RCB' '973']
 ['Jos Buttler' 'RR' '863']]
<U21

NumPy forced everything into one type — the runs 973 became the string '973' (dtype <U21 = Unicode text up to 21 characters). You can no longer add or average them. This is exactly the problem pandas solves.

Definition

pandas is an open-source Python library for data manipulation and analysis. It provides two labelled data structures — the one-dimensional Series and the two-dimensional DataFrame — built on top of NumPy, designed for working with tabular, heterogeneous data (like spreadsheets or SQL tables) quickly and expressively.

• Data manipulation and analysis — loading, cleaning, filtering, sorting, grouping, joining, reshaping and summarising data. Almost every step of a data-science project before machine learning happens in pandas.
• Labelled — every row has a label (the index) and every column has a name. You can say df["runs"] instead of remembering "column 3".
• Built on top of NumPy — each column is stored internally as a NumPy array (or a pandas extension array), so vectorised maths is just as fast.
• Tabular, heterogeneous — a table with rows and columns where each column can have a different type (text, integers, floats, dates, booleans…).

The name comes from "panel data" (an economics term for multi-dimensional data) and also plays on "Python data analysis". It was created by Wes McKinney in 2008 while working at a finance firm, and is now one of the most downloaded Python packages in the world.

Example — the same data in pandas

import pandas as pd

df = pd.DataFrame({
    "player": ["Virat Kohli", "Jos Buttler", "KL Rahul", "Shubman Gill"],
    "team": ["RCB", "RR", "PBKS", "GT"],
    "season": [2016, 2022, 2020, 2023],
    "runs": [973, 863, 670, 890],
    "average": [81.08, 57.53, 55.83, 59.33],
})
print(df)
print(df.dtypes)

         player  team  season  runs  average
0   Virat Kohli   RCB    2016   973    81.08
1   Jos Buttler    RR    2022   863    57.53
2      KL Rahul  PBKS    2020   670    55.83
3  Shubman Gill    GT    2023   890    59.33
player         str
team           str
season       int64
runs         int64
average    float64
dtype: object

How it works: each column keeps its own type — player and team are text (str), season and runs are 64-bit integers, average is a 64-bit float. The numbers 0, 1, 2, 3 on the left are the index (row labels) that pandas adds automatically. Now analysis is one line each:

print(df[df["runs"] > 800])
print(df["runs"].mean())

         player team  season  runs  average
0   Virat Kohli  RCB    2016   973    81.08
1   Jos Buttler   RR    2022   863    57.53
3  Shubman Gill   GT    2023   890    59.33
849.0

What can pandas do?

NumPy vs pandas — how they relate

pandas does not replace NumPy — it sits on top of it. Think of NumPy as the engine and pandas as the car built around it.

Note: You can always move between the two: series.to_numpy() / df.to_numpy() gives a NumPy array, and NumPy functions like np.sqrt(series) work directly on pandas objects and keep the labels.

Where pandas fits in a data-science workflow

 raw data (CSV / Excel / SQL / API)
          │   pd.read_csv(...)
          ▼
     pandas DataFrame  ──► inspect ──► clean ──► transform ──► summarise
          │
          ├──► charts      (matplotlib / seaborn use DataFrames directly)
          └──► ML models   (scikit-learn accepts DataFrames / .to_numpy())

Common mistakes

• ❌ Thinking pandas is a separate language or a database. ✅ It is a normal Python library — you write ordinary Python and call pandas functions.
• ❌ Storing a mixed table in a NumPy array. ✅ NumPy forces one dtype (numbers become strings); use a DataFrame for mixed columns.
• ❌ Writing Python for loops over rows to compute totals. ✅ Use vectorised column operations like df["runs"].sum() — faster and shorter.
• ❌ Copying outputs from very old tutorials blindly. ✅ This course uses pandas 3; some old behaviour has changed (see "What's New in pandas 3").

Interview tip

Q: Why use pandas when we already have NumPy? A: NumPy is ideal for homogeneous numeric arrays addressed by position. Real datasets are tables with mixed column types, labels, and missing values. pandas adds labelled rows/columns, per-column dtypes, missing-data handling, label alignment, file I/O, and high-level operations like groupby, merge and pivot — while still using NumPy arrays underneath for speed.