Unsupervised Learning and Neural Networks
Clustering with k-means
Group points into k clusters by repeatedly assigning each point to the nearest centroid and recomputing centroids.
from sklearn.cluster import KMeans
from sklearn.preprocessing import StandardScaler
X_scaled = StandardScaler().fit_transform(X)
labels = KMeans(n_clusters=3, n_init=10, random_state=0).fit_predict(X_scaled)
Choose k with the elbow method or the silhouette score. Other methods: hierarchical clustering, DBSCAN (finds arbitrary shapes and noise), Gaussian mixtures.
Dimensionality reduction
PCA projects data onto the directions of greatest variance, which helps visualisation and removes redundancy.
from sklearn.decomposition import PCA
pca = PCA(n_components=2).fit(X_scaled)
X_2d = pca.transform(X_scaled)
print(pca.explained_variance_ratio_)
t-SNE and UMAP are popular for visualising high-dimensional structure.
Neural networks
A neuron computes activation(w . x + b). Layers of neurons learn increasingly abstract features.
| Component | Role |
|---|---|
| Activation (ReLU, sigmoid, softmax) | Adds non-linearity |
| Loss (cross-entropy, MSE) | Measures error |
| Backpropagation | Computes gradients of the loss |
| Optimiser (SGD, Adam) | Updates weights using the gradients |
| Epoch / batch | One pass over data / a chunk of data per update |
import torch
from torch import nn
model = nn.Sequential(
nn.Linear(4, 16), nn.ReLU(),
nn.Linear(16, 3),
)
loss_fn = nn.CrossEntropyLoss()
opt = torch.optim.Adam(model.parameters(), lr=1e-2)
for epoch in range(50):
opt.zero_grad()
loss = loss_fn(model(X_train_t), y_train_t)
loss.backward()
opt.step()
Architectures at a glance
| Architecture | Typical use |
|---|---|
| CNN | Images |
| RNN / LSTM | Sequences |
| Transformer | Language, and increasingly vision and audio |
TIP
Start with simple baselines (linear models, gradient boosting). Reach for deep learning when you have unstructured data (images, text, audio) or a lot of data.
Loading Comments...