At some point in my life, as perhaps in yours, I had to write a multilayer perceptron code from scratch. It is a useful exercise, and the result posted here is a nice, barebones implementation that I use on occasion to get a peek under the hood of how my networks are working. If you use the code, please cite this page, and please let me know if you found it useful or not.
my-neural-network.zip This is the archive (5.8kB) containing 5 M source code files. Description of each is below, and the files are heavily commented.
The code files are as follows:
Script for generating synthetic data, calling the ANN, plotting and testing the output. This one can be run and it will call all the others.
This function intializes the model, presents the training patterns a given number of times, and returns the trained model. Arguments given are the training dataset, the target model output, the numbers of hidden layer neurons, the number of iterations to present the whole dataset, and the learning rate. Outputs are the trained model, the output of the model to the training set, and the correlation of output to target output.
This function is called a number of times (number of iterations * number of training examples) from train_mlp.m. This function does most of the complicated calculations for the ANN. It first calculates the activations of each layer of neurons, then calculates the error and propagates the error back through the network, and then updates the weights and biases accordingly. This is done in MATLAB so the calculations are done as matrices, which is nice at the end, but took some head scratching to figure out the first time.
A simple function for calculating the output of a trained model on a test dataset. Arguments to the function are the model, the input, and the target output. Returned variables are the output, and the correlation of output to target output.
This is just a helper function for generating some synthetic data that I use in run_mlp.m. It makes a 2D dataset which consists of populations scattered on the plane. There are some colorcoded plots which make it obvious what is going on.