Neural Nets, Simply

Neural Nets, Simply
Published on 14.05.2025
We know this much about a function.

\( f(x) = y \)

Where,
    \( f \) is a function
    \( x \) is the input
    \( y \) is the output

Given an input one must give an output that's in accordance to some constraints.

- This could be a simple function that takes in the house dimensions and returns an approximate price for the house in dollars.

- It could also be an XOR function which takes 2 inputs and returns the XOR-ed output.

- It could be a function that takes multiple inputs such as height, weight, facial features and age of a person and could somehow return the probablity of the person being a biological male or a female.

- It could also be a function that takes in a 28x28 (784 pixels) resolution image of a hand written number and return the probablity of which number it was between 0-9.

- It could also be a function that can takes in hundreds of different kinds of data points from an MRI machine and somehow detect tumor.

- Or it could be a function that takes in a users "text" input and return a next prediction for your texts and keep doing it until you stop it.

- Or ... you get the point.

Humans love to simplify large complex realities into understandable functions. We do it in math, physics, you name it... And these functions don't have to be continuous, but it can be a cumulation of smaller functions - peicewise functions.

Instead of finding these functions individually, we've got ourselves a universal function approximator called Neural Network. But why does it work?

The simplest intuitive explanation would be that, assume theres a bunch of inputs \( X \) and you've connected a bunch of knobs together, each one influencing the next. Imagine you've got a complex machine out of interconnected knobs. Each knob controls a small part of the machine, tweaking the flow of signals though it. Some knobs might influence others directly, while others have more subtle touch of downstream effects.

When you tune a knob, it might slightly shift the entire machine behviour or it might have a more dramatic effect. Depends entirely on the position of the knob in the interconnected mess.

The goal is to tune these knobs so that the output \( Y \) somehow aligns as closely as possible with the designed outcome (assuming that there exists a function). The magic is in the flexibility. If you have enough knobs, and you turn them just right, the machine can approximate virtually any function you throw at it.

"All you need are Knobs" - @pwnfunction (*famous last words)

In a neural net, these "knobs" are primarily the weights associated with each connection and the biases associated with each neuron. Adjusting them changes how signals are processed and what output the network produces.

And the process of "training" a neural network is essentially just finding the right configurations for all these interconnected knobs, often automatically through vast amounts of data. This is done with the magic of math, especially calculus.

Essentially, when the network makes a mistake, calculus (using derivatives) tells us exactly which way to nudge each knob to make the prediction better. Backpropagation is then the clever method that efficiently calculates these nudges for all knobs by working backward from the mistake.

And that's it.

This is a good intuition, and almost anyone can relate to turning a bunch of interconnected knobs. When somebody asks me for an explanation, I almost always hit em with this.

Change Inputs (X1, X2): Click directly on the "X1 (A)" or "X2 (B)" circles on the left to toggle their values between 0 and 1.

Nodes (Circles):
- X1, X2: Your inputs. Green means 1 (active), white/gray means 0 (inactive). The number below is their value.
- H1, H2: Hidden neurons. Their color indicates if they are active (output > 0 after ReLU). The number shows their output value. The (b:...) shows their bias.
- Out: The network's final output neuron. The color indicates the predicted class (0 or 1). The number shows the raw sigmoid output value (before rounding). The (b:...) shows its bias.

Connections (Lines): These represent the "weights" of the network.
- Color: Green lines are positive weights (excitatory), red lines are negative weights (inhibitory), and gray lines are near-zero weights.
- Thickness: Thicker lines mean a stronger weight (larger absolute value), influencing the next neuron more.

Adjust Knobs (Sliders): Below the diagram, use the sliders to change the values of each weight (e.g., w1_11) and bias (e.g., b1_1).

See Real-time Effects: As you change inputs or sliders,
- The diagram updates immediately.
- The "Performance Text" at the top tells you if the network's prediction matches the true XOR result for the current inputs.
- The detailed mathematical equations below the sliders also update, showing all calculations step-by-step.

Goal: Try to adjust the weights and biases using the sliders so the network correctly predicts the XOR output for all four possible input combinations:
  [0,0] -> 0
  [0,1] -> 1
  [1,0] -> 1
  [1,1] -> 0

The Knob Network 
Here's where you get your hands dirty and play around with the inputs and the knobs. This is a simple neural network to act like an XOR function. 

It might look a little daunting, my apologies for making a mid af ui, but it's very simple to play around with. Hope the above instructions help.


Control knobs

    


Got other perspectives? reach out to me on x - @pwnfunction.