Pandas Essentials for Data Analysis

pandas is the core Python library for working with tables of data.

Loading Data

import pandas as pd
df = pd.read_csv("sales.csv", parse_dates=["order_date"])
df = pd.read_parquet("sales.parquet")

First Look

df.shape; df.head(); df.dtypes; df.describe(); df.isna().sum()

Selecting and Filtering

df[["region", "total"]]
df[df["total"] > 100]
df.query("region == 'North' and total > 100")

Creating Columns

df["month"] = df["order_date"].dt.to_period("M")
df["high_value"] = df["total"] > 500

Grouping and Aggregating

df.groupby("region")["total"].agg(["count", "sum", "mean"])

Merging

df = df.merge(customers, on="customer_id", how="left", validate="many_to_one")

Reshaping

df.pivot_table(index="month", columns="region", values="total", aggfunc="sum")
long = wide.melt(id_vars="month", var_name="region", value_name="total")

Sorting and Ranking

df.sort_values("total", ascending=False).head(10)

Performance Tips

• Use vectorised operations, not loops over rows.
• Use appropriate types (category for repeated strings).
• Read only needed columns.
• For very large data, consider Polars or DuckDB.