"Building your own AI model" sounds like it requires a research lab and a stack of GPUs. It doesn't, if your goal is to actually understand how these systems work rather than to compete with a frontier lab. I built my first neural network from scratch — no frameworks, just NumPy — specifically because I wanted to demystify what a forward pass, a loss function, and backpropagation actually do under the hood. This post is the walkthrough I wish I'd had.
Start With the Smallest Possible Problem
Skip image classification and language modeling for your first attempt. Pick something you can reason about by hand — I used a network that predicts whether a point falls inside a circle given its x/y coordinates. The problem is trivial, which is exactly the point. When something breaks, you can debug it by inspecting a handful of numbers instead of staring at a loss curve and hoping.
The Forward Pass Is Just Matrix Multiplication
Strip away the mystique and a neural network layer is: multiply the input by a weight matrix, add a bias, apply a nonlinear function. That's it. Stack a few of these and you have a network capable of approximating any continuous function, per the universal approximation theorem. The nonlinearity is what gives the network its power — without it, stacking layers would collapse into a single linear transformation, no smarter than one layer.
import numpy as np
def relu(x):
return np.maximum(0, x)
class TinyNetwork:
def __init__(self, input_dim, hidden_dim, output_dim):
self.W1 = np.random.randn(input_dim, hidden_dim) * 0.1
self.b1 = np.zeros(hidden_dim)
self.W2 = np.random.randn(hidden_dim, output_dim) * 0.1
self.b2 = np.zeros(output_dim)
def forward(self, x):
self.z1 = x @ self.W1 + self.b1
self.a1 = relu(self.z1)
self.z2 = self.a1 @ self.W2 + self.b2
return self.z2 # raw logitsBackpropagation Is Just the Chain Rule
This is the part that intimidates people, and it shouldn't. Backpropagation is repeatedly applying the chain rule from calculus to figure out how much each weight contributed to the final error, then nudging it in the direction that reduces that error. Writing it by hand once — computing gradients for a two-layer network without autograd — made every subsequent framework (PyTorch, JAX) feel like a convenience rather than magic.
def backward(self, x, y, lr=0.01):
m = x.shape[0]
dz2 = self.z2 - y # gradient of MSE loss w.r.t. output
dW2 = self.a1.T @ dz2 / m
db2 = dz2.mean(axis=0)
da1 = dz2 @ self.W2.T
dz1 = da1 * (self.z1 > 0) # relu derivative
dW1 = x.T @ dz1 / m
db1 = dz1.mean(axis=0)
self.W1 -= lr * dW1
self.b1 -= lr * db1
self.W2 -= lr * dW2
self.b2 -= lr * db2Where Frameworks Actually Earn Their Keep
Once you understand the mechanics, drop the from-scratch approach for anything real. PyTorch's autograd, optimized tensor operations, and GPU support aren't hiding complexity you should reinvent — they're solving problems (numerical stability, memory layout, distributed training) that took the field over a decade to get right. The value of building it by hand once is calibration: you'll read PyTorch code and know exactly what torch.nn.Linear and loss.backward() are doing instead of treating them as incantations.
Training a Model vs. Building a Product
It's worth being honest about scope. Training a small model to solve a toy problem takes an afternoon. Training something competitive with a modern LLM requires trillions of tokens, thousands of GPUs, and a research team iterating on architecture and data curation for months. For almost every practical use case today, fine-tuning an existing open-weight model or building well-designed prompts against a frontier model's API will get you further, faster, and cheaper than training from scratch. Know which problem you're actually solving before you commit to a from-scratch build.
Build a tiny model by hand once, not because you'll use it in production, but because it's the fastest way to stop being afraid of the math.
If you want to go deeper after this, Andrej Karpathy's 'neural networks: zero to hero' series covers the same ground with far more rigor, building all the way up to a small GPT from scratch. But do the tiny two-layer network first, by hand, with pen and paper for the gradients before you write the code. That's where the intuition actually forms.