NumPy and pandas
NumPy
NumPy provides the ndarray: a typed, fixed-size, vectorised array.
import numpy as np
a = np.array([1, 2, 3, 4])
b = np.arange(12).reshape(3, 4)
print(a * 2) # [2 4 6 8] (vectorised, no loop)
print(b.shape, b.mean()) # (3, 4) 5.5
print(b[:, 1]) # second column
print(a[a > 2]) # boolean mask -> [3 4]
Broadcasting
Arrays with compatible shapes are stretched automatically.
matrix = np.ones((3, 4))
row = np.array([1, 2, 3, 4])
print(matrix + row) # row is added to every row
pandas
A DataFrame is a table with labelled columns.
import pandas as pd
df = pd.DataFrame({
"name": ["Asha", "Ravi", "Meera", "Kabir"],
"dept": ["CS", "EE", "CS", "ME"],
"marks": [91, 74, 88, 69],
})
print(df.head())
print(df.describe())
high = df[df["marks"] > 80] # filter rows
df["grade"] = pd.cut(df["marks"], [0, 70, 85, 100], labels=["C", "B", "A"])
Group, aggregate, join
avg = df.groupby("dept")["marks"].mean()
depts = pd.DataFrame({"dept": ["CS", "EE", "ME"], "head": ["Rao", "Iyer", "Shah"]})
merged = df.merge(depts, on="dept", how="left")
Reading and writing
df = pd.read_csv("students.csv")
df.to_csv("clean.csv", index=False)
TIP
Prefer vectorised operations and built-in methods over Python loops (for over rows). They are usually 10 to 100 times faster.