mlp

Introduction to neural networks, with the implementation of a multilayer perceptron.

If you want to learn more about the way deep learning framework work.

Install

uv sync

Training

uv run examples/train_mlp.py

this will also save plot about the training data.

Inference

uv run examples/inference_mlp.py

Load weight from training and perform inference on unseen data.

Other commands you can use, run a python server of the directory and view images in your browser for examples.

Clean

rm -rf .venv
rm -rf data_*
rm -rf *.safetensors
rm -rf *.png

Development

uv sync --dev

Mypy

uv run mypy .

Test

uv run pytest

ToDo

• Explain contrastive method in log_loss maybe renamed to BCE / penalise wrong answer, reward good one

• Optimizer base class / proper GD / Proper SGD

  • explain the different optimization: Weight decay, Momentum: see if saving previous values is heavy in compute
  • explain momentum
  • explain weight decay
  • explain information collapse

• Encoder - Decoder base class ?

• Dataclass ? can transpose from json to dataclass directly

• Calculating normalization stats should be perform only on training data one time

• add third program split data

• make some link about the data to the actual images of a breast cancer using some http balise like in roryclearcam

• implement SGD

• maybe reduce the .gitignore

• testing:

  • shape/type missmatch

• Explain Topo_sort with a graphviz would be perfect not sure its easaly done tho

• refacto Inference

• everything is a function, train should be a loop

  • all training variable in one place / no more changing directly in the function call
  • train function for each epoch what do we do ? whats the return ?
  • extract evaluate function -> return tuple[float, float]
  • metrics dictionary that contain the 4 list;
  • finally a fit function that takes epoch and data and train and evaluate epochs

• look into the save_model comment in train_mlp.py

• can load weight differently in Inference ?

  • we are protecting against missmatch
  • problem with inference, when shape missmatch

• think about the dataset split in training and inference

  • we fixed with a check before split

• change get_parameters() to a yield and yield from

  • don't see the interest anymore, simple is better to explain

Contrastive method is the "study" of difference but at the same time of similarities. Contrastive methods are training techniques that teach a model to bring similar data points close together and push dissimilar ones far apart. the Formula we are using is a contrastive method, this mean that we are pushing up the probability of benign when its benign but we are also pushing down the probability of malign when its benign Actually its call a contrastive embedding, the embedding for Malign [1.0, 0.0] and benign [0.0, 1.0] and for a Malign examples we would want to push the first column of our output to 1 and the 2nd to zero

def log_loss(y, p):
    return -((y * (p).log() + (1 - y) * (1 - p).log()).MEAN())