{"id":23504,"date":"2017-04-27T11:13:27","date_gmt":"2017-04-27T09:13:27","guid":{"rendered":"http:\/\/infozonedev.com\/?p=23504"},"modified":"2017-04-27T11:13:27","modified_gmt":"2017-04-27T09:13:27","slug":"machine-learning-using-neural-network-written-c-part-2-implementation","status":"publish","type":"post","link":"https:\/\/www.infozone.se\/en\/latest-news\/machine-learning-using-neural-network-written-c-part-2-implementation\/","title":{"rendered":"Machine Learning using a Neural Network written in C# \u2013 Part 2 \u2013 Implementation"},"content":{"rendered":"

This is the second part of three in a blog series about Machine learning using a Neural Network. In the previous post, I went through the theory needed to write a shiny neural network, from scratch, in C#. If you feel comfortable with the theory or just want some code then keep on reading, otherwise I recommend reading part 1<\/a> before reading any further.<\/strong><\/p>\n

TL;DR
\nSource code can be found here<\/a>.<\/p>\n

Let us start by defining a shell for our network, just to have something to work with. I\u2019ve defined a class named NeuralNetwork that has two methods; Train and Query. These methods are the two core methods you will see in such networks. Query<\/strong> is understandable \u2013 one needs to ask the network for some result given some input. Train<\/strong>, on the other hand is less intuitive, if a network should be able to solve a problem it needs to be trained. In the training scenario, one needs to pass in the input and target values. The target values are those we want<\/em> the network to emit when it\u2019s queried for the given input values.<\/p>\n

public class NeuralNetwork\n{\npublic void Train(double[] inputs, double[] targets)\n{\n    }\n \n    public double[] Query(double[] inputs)\n    {\n    }\n}<\/pre>\n
There it is, the first step towards a neural network.<\/p>\n
We have learned that a network is built using an input, an output and a hidden layer of neurons. The neurons are connected to each other with signals with an associated weight. These weights need to be initialized when the network is created. Remember that the weights need to be small for the Sigmoid activation function to work properly, in this case a random distribution between -0.5 and 0.5 will suffice.<\/p>\n
Let\u2019s add a constructor in our class, one that requires the number of nodes in each layer as well as a learning rate. The learning rate is a parameter that acts as a factor on the weight adjustment function, tweak this one to either increase or decrease the weights learning rate \u2013 it can prove useful to lower this if the network keeps overshooting, for instance.<\/p>\n
private readonly double _learningRate;\nprivate Matrix _weightHiddenOutput;\nprivate Matrix _weightInputHidden;\n \npublic NeuralNetwork(int numberOfInputNodes, int numberOfHiddenNodes, int numberOfOutputNodes, double learningRate)\n{\n_learningRate = learningRate;\n \n_weightInputHidden = Matrix.Create(numberOfHiddenNodes, numberOfInputNodes);\n_weightHiddenOutput = Matrix.Create(numberOfOutputNodes, numberOfHiddenNodes);\n \nRandomizeWeights();\n}\n \nprivate void RandomizeWeights()\n{\nvar rnd = new Random();\n \n\/\/distribute -0.5 to 0.5.\n_weightHiddenOutput.Initialize(() => rnd.NextDouble() - 0.5);\n_weightInputHidden.Initialize(() => rnd.NextDouble() - 0.5);\n}<\/pre>\n
The constructor does two things in particular \u2013 create two matrices and initializes the with random weights between -0.5 and 0.5. Wait, matrices? Yes! Matrices helps to keep the instructions needed to calculate a neural network small and neat. Note that the Matrix class is written from scratch and is therefore not a part of C# or .NET. The full implementation is found @ GitHub<\/a>.<\/p>\n
The use of matrices<\/h2>\n
Imagine writing all the instructions needed to calculate the weights and signals through a network of thousands (or more) neurons \u2013 it will be close to impossible. I\u2019ll show just why matrices works so well in our scenario.<\/p>\n
Remember that the output of a neuron is the weighted sum of all incoming signals<\/em>? Have a look below what happens if we put all weights between the input and hidden layer in the left-side matrix and the signal itself in right-side matrix and then multiply them. The resulting matrix is the weighted sum of all incoming signals<\/em>. In this simple example, the network only consists of two neurons in each layer, but the best part is that this method works for any number of neurons which will make the solution generic and flexible.<\/p>\n
\"\"<\/p>\n
An image can usually clarify any doubts, look below for an image representation of the above equation. See how the output of neuron 1 of the Hidden layer is (W(1,1) * Signal 1) + (W(2,1) * Signal 2)? Just as the above matrix.<\/p>\n
\"\"<\/p>\n
Implementation of Query<\/h2>\n
Our knowledge of how helpful matrices are will aid the implementation of the Query-method. The reason that I begin with Query over Train is that Query is less complicated as it does not need to update any weights. Remember that the network is trained, and ready to be queried, when the weights are set correctly. Before we dig into some code, lets summarize what we need to do here.<\/p>\n