Wednesday, 23 March 2011

Starting with neural network in matlab

The neural networks is a way to model any input to output relations based on some input output data when nothing is known about the model. This example shows you a very simple example and its modelling through neural network using MATLAB.

Actual Model
Let us take that our model has three inputs a,b and c and generates an output y. For data generation purposes, let us take this model as
y=5a+bc+7c;

we are taking this model for data generation. In actual cases, you dont have the mathematical model and you generate the data by running the real system.

Let us first write a small script to generate the data
    a= rand(1,1000);
    b=rand(1,1000);
    c=rand(1,1000);
    n=rand(1,1000)*0.05;

    y=a*5+b.*c+7*c+n;
n is the noise, we added deliberately to make it more like a real data. The magnitude of the noise is 0.1 and is uniform noise.

So our input is set of a,b and c and output is y.
    I=[a; b; c];
    O=y;
Understanding Neural Networks
Neural network is like brain full of nerons and made of different layers.
The first layer which takes input and put into internal layers or hidden layers are known as input layer.
The outer layer which takes the output from inner layers and gives it to outer world is known as output layer.

The internal layers can be any number of layers.

Each layer is a basically a function which takes some variables (in the form of vector u) and transforms it to another variable(another vector v) by multiplying it with coefficients and adding some biases b. These coefficient is known as weight matrix w. Size of the v vector is known as v-size of the layer.
v=sum(w.*u)+b

So we will make a very simple neural network for our case- 1 input and 1 output layer. We will take the input layer v-size as 5. Since we have three input , our input layer will take u with three values and transform it to a vector of size 5. and our output layer now take this 5 element vector as input and transforms it to a vector of size 1 because we have only on output.

Creating a simple Neural FF Network
We will use matlab inbuilt function newff for generation of model.
First we will make a matrix R which is of 3 *2 size. First column will show the minimum of all three inputs and second will show the maximum of three inputs. In our case all three inputs are from 0 to 1 range, So
R=[0 1; 0 1 ; 0 1];
Now We make a Size matrix which has the v-size of all the layers.
S=[5 1];
Now call the newff function as following
  net = newff([0 1;0 1 ;0 1],S,{'tansig','purelin'});
net is the neural model. {'tansig','purelin'} shows the mapping function of the two layers. Let us not waste time on this.

Now as each brain need training, this neural network too need it. We will train this neural network with the data we generated earlier.
net=train(net,I,O);
Now net is trained. You can see the performance curve, as it gets trained.

So now simulate our neural network again on the same data and compare the out.puts.
O1=sim(net,I);
plot(1:1000,O,1:1000,O1);

 You can observe how closely the the two data green and blue follow each other.
Let us try scatter plot between simulated output and actual target output.
scatter(O,O1);

Let us observe the weight matrix of the trained model.

net.IW{1}
 -0.3684    0.0308   -0.5402
    0.4640    0.2340    0.5875
    1.9569   -1.6887    1.5403
    1.1138    1.0841    0.2439
net.LW{2,1}
-11.1990    9.4589   -1.0006   -0.9138
Now test it again on some other data. What about a=1,b=1 and c=1;

So input matrix will be [1 1 1]';
y1=sim(net,[1 1 1]');
you will see 13.0279. which is close to 13 the actual output (5*1+1*1+12*1);


Updates:
1. O and T are same variable. define O=T or just replace T everywhere with O
2. S=[5 1], so while defining newff you should pass S or [5 1]. [4 1] is incorrect

93 comments:

  1. fuzzy example in matlab http://technical-leader.blogspot.com/2011/09/fuzzy-logic-examples-using-matlabfuzzy.html

    ReplyDelete
  2. Time should be still wasted, since on new version newff has new argument list and therefore it does not work.
    Hopefully i'll try to comeback when i have studied how to use it :)
    My Matlab (R2008a) and FL toolbox v 2.2.7

    ReplyDelete
  3. please if you can help me to using ant colony optimization to training neural network in matlab invernoiment?

    ReplyDelete
  4. Overall this seems like a great little tutorial, but I'm confused by one thing. It says "we will train this neural network with the data we generated earlier," and then provides the command "net = train (net, I, O)" and continues to reference O as a variable for the rest of the explanation. O, however, was never defined. Did the author, perhaps, intend to use T as the output target data?

    ReplyDelete
    Replies
    1. Hi Steve

      Yes , you are right. O and T are same. I forgot to define that. thanks for pointing it out :)
      Corrected the mistake
      Abhishek

      Delete
  5. @ahmed
    training can be seen as fitting the model such that output should match target.. this fitting is an optimization problem over neural netwrk's biases and weights. just try to fit optimize weights and biases such that sum(O1-O)^2 is minimized using any optimization nmethod,,

    ReplyDelete
  6. Hi Steve

    Yes , you are right. O and T are same. I forgot to define that. thanks for pointing it out :)

    Abhishek

    ReplyDelete
  7. Neural Network Toolbox provides functions and apps for modeling complex nonlinear systems that are not easily modeled with a closed-form equation. Neural Network Toolbox supports supervised learning with feed forward, radial basis, and dynamic networks. It also supports unsupervised learning with self-organizing maps and competitive layers. With the toolbox you can design, train, visualize, and simulate neural networks. You can use Neural Network Toolbox for applications such as data fitting, pattern recognition, clustering, time-series prediction, and dynamic system modeling and control.

    ReplyDelete
  8. @sumit Kumar
    thanks for the information, yes you are right, it is very useful and can be used in variety of things

    ReplyDelete
  9. thank you very much Gupta and Abhishek...

    ReplyDelete
    Replies
    1. Gupta and Abhishek are the same person here . thanks :)

      Delete
  10. I have 15inputs and want to come up with two outputs which i will scale down to bits for contol of electric motor speed through variable speed drives.how can i do this using Neural networks considering the inputs are input barley specifications,from two batchs of malt.Ultimately i will be using the two inputs for blending purposes

    ReplyDelete
  11. Thank you for this small tutorial.
    But I have a confusion,
    A. in the first line you defined a,b,c as random no. between 1,1000
    and later in matrix as 0,1 max and min values of input. That I didn't get.
    B.can a matrix be given as single input or there's a different way to deal with matrices[two dimension].

    ReplyDelete
    Replies
    1. when you say a=random(1,1000) it doesnot mean the a is between 1 and 1000. it means that a is matrix containing 1000 datapoints. all points will be between 0 and 1. that is why the max and min limit is 1 and 0
      each input is a single dimension matrix. it should be a column vector. then you should combine all the vectors to a big2D matrix I.

      -Abhishek

      Delete
  12. when you say a=random(1,1000) it doesnot mean the a is between 1 and 1000. it means that a is matrix containing 1000 datapoints. all points will be between 0 and 1. that is why the max and min limit is 1 and 0
    each input is a single dimension matrix. it should be a column vector. then you should combine all the vectors to a big2D matrix I.

    -Abhishek

    ReplyDelete
  13. i need help how to train an face image using backpropogation neural network so that output is recognized?

    ReplyDelete
    Replies
    1. You need to extract some features from face. read about feature extraction. then these feature coefficients will work as input. face's name(persons) you can save as numbers and these will be the target of the model. now run a backpropagation model using matlab.

      Delete
  14. You need to extract some features from face. read about feature extraction. then these feature coefficients will work as input. face's name(persons) you can save as numbers and these will be the target of the model. now run a backpropagation model using matlab.

    ReplyDelete
  15. I think that you forgot to take into account the noise n in your last test. The mean value of the noise is 0.025, so the estimate 13.0279 is even closer to the model which in average returns 13.025.

    ReplyDelete
    Replies
    1. @jiri Falta, You are right in saying that we should expect the answer to be 13.025 and we get 13.0275 which is pretty close.

      But our original model does not have noise. We generated this data ourselves so we knew the noise. But in general, when we work on experimental data, we don't know the noise. So in that case, we expect the correct answer because we have no knowledge about noise. That is why i expect 13 should be the answer.

      Delete
    2. But in that case the program did not have any chance to estimate your original model correctly even if given great amount of data. Any other statistical method would also return something close to y*=5a+bc+7c+mean(n).
      However thanks for the tutorial.

      Delete
    3. genrally noise is zero mean, so you don't face this problem. otherwise if this is not zero mean and i know the exact mean, i can incorporate this information to get few more bounds. thanks for the comments :)

      Delete
    4. Yes, generally, but not in your example. That's why I think that the final comparison is not fair to the computer. You might have mocked him for inaccuracy if you had chosen higher amplitude of noise :)

      Delete
  16. @jiri Falta, You are right in saying that we should expect the answer to be 13.025 and we get 13.0275 which is pretty close.

    But our original model does not have noise. We generated this data ourselves so we knew the noise. But in general, when we work on experimental data, we don't know the noise. So in that case, we expect the correct answer because we have no knowledge about noise. That is why i expect 13 should be the answer.

    ReplyDelete
  17. Sir, i am working on offline signature verification using Neural Network.I have done feature extraction but how to work on neural network .I have to take 100 signatures .Please help.

    ReplyDelete
    Replies
    1. Once you know the features, form the input and target matrix and just use training of it and you are done, you may need to play with some different values of hidden layers and neurons.

      Delete
    2. sir
      i have a one proble when i train neural network in second step(using toolbox) selection of input&target values the target values are not shown in browse option sir please help............

      Delete
  18. can u tell me., we should compute square of some five numbers then using these input and output, i should compute output next input numbers..

    ReplyDelete
    Replies
    1. @keeti Hallur
      You can try the same approach. read the above blog. Just form your input and target matrix as some numbers and their square and train your network.

      then test it using the new values. should work fine.

      Delete
    2. Dear Sir,
      Could you kindly explain how to do system identification for a system that has 5 inputs and 2 outputs, using Neural Networks.

      Thanks in advance

      Delete
    3. Do you have pre information about the system? or is it totally unknown?

      Delete
    4. Thanks a lot for your reply, I do have all the information and data about the system if you could help.

      Delete
    5. I'm trying to make the identification of a MISO system by the time I'm using the tool NNSYSID but I could not move and I do not know what steps to take after training the network, I can collaborate with another alternative for the identification of the system. thanks

      Delete
  19. i have image and i want to segmented it into 4 cluster by using competitive learning alogrithm not som .it cpnsist from 2 layer but i dont know i the pixel of image represent the input i mean if image have sise of 200*300 so i have 50000 input ? or what a lot of quetion i have no one answer

    ReplyDelete
    Replies
    1. The whole image is never an input because then you will have lots of value. You first extract feature. feature is something which can give you most of the information. Like orienctation histogram if you are studying the guesture recognition. relative positions of face points if you are doing face recognition.

      you need to first decide some feature which define you clustering

      Delete
  20. in the last line u have written "Now test it again on some other data. What about a=1,b=1 and c=1;".r u want to say that in the example u have chosen random no. but now u r insisting with [1,1,1].matrix. One more question....how to adjust the weight???

    ReplyDelete
    Replies
    1. Yes , The data was generated randomly here. but in true sense, it is not generated. I tried to mimic the actaully experiement, So think in this way that data has been given to you by an real experiment done.

      Now I want to just test if my system is trained or not. So i just give one more random data. a=1,b-1 c=1 is just some random data. you could have given [a=2 b=3 c=5] and expected an different answer. This step cannot be done when you are in real world. because you didnt know what output to expect as you dont know that y=5a+bc+7c; (because this is unknown)

      Delete
    2. Hi.. I want to know how to write program in MATLAB for RBF neural netwroks. Especially for modeling. Please do help me. reply to satheeshnurgemar@gmail.com,

      Delete
  21. i m doing a project on career counselling using neural network. Can u suggest me how to map the data( means input, which is in string like introvert, extrovert) into matlab dataset.

    ReplyDelete
    Replies
    1. for a computer program, being an introvert or extovert does nt have any meaning.
      So you can just assign one of them as 1 and other as 0.
      if you have more than 2 options (eg lower class, middle class, upper class), you can use 0, 0. 1 0.2 like this.

      Delete
  22. This comment has been removed by the author.

    ReplyDelete
  23. I m designing an ANN for fault detection in transmission line. I have to identify 6 different types of faults, for this i m using 12x51 order matrix as input and have 7x51 order matrix as target/output. I m using 6 different sets of input-target combination to train the n/w. I m not able to train the n/w.

    Can anyone just help me to train the n/w and let me the best suitable training function for my application?

    PLZ HELP...

    ReplyDelete
    Replies
    1. Read the example, your question is no different
      as far as training is considered, there is no rule as of my knowledge
      try different functions and see what works best for you

      Delete
  24. Hi,

    How can I convert the neural network to state space model?
    Thank you..

    ReplyDelete
    Replies
    1. interesting . there are no dynamics in neural netwrk so why statespace?

      Delete
    2. I need a state space identified model in order to use it in MPC controller. But, never mind I have found a method on how to construct neural state space by using a customize neural network..Thank you for your reply..

      Delete
  25. Hi;
    in case i've trained my neural network and i've got results close to the target ,how can i fixe weights to get the the model i want,i mean weights are changing every time i train the model , is there any code to get that???

    ReplyDelete
    Replies
    1. you can save the model in a file after you trained.
      So first time you run it, after that type
      save net_trained net

      then next time instead of running the script
      just type
      load net_trained
      and then call directly
      y1=sim(net,[1 1 1]');

      Delete
  26. Probably a stupid question, since I'm new to both Matlab & NN.


    net = newff([0 1;0 1 ;0 1],[4 1],{'tansig','purelin'});

    Wat's the meaning of the [4,1]? Shouldn't it be the S matrix [5,1] ?

    Grtz

    ReplyDelete
    Replies
    1. yes theirry
      thanks for pointing out the mistake
      Corrected it, see the updates

      Delete
  27. I also get two errors, and I don't know how to fix them :)

    >>
    net = newff([0 1;0 1 ;0 1],[4 1],{'tansig','purelin'});
    Warning: NEWFF used in an obsolete way.
    > In obs_use at 18
    In newff>create_network at 127
    In newff at 102
    See help for NEWFF to update calls to the new argument list.

    >>
    net=train(net,I,O);
    Undefined function or variable 'O'.

    ReplyDelete
    Replies
    1. Updated the error
      T is same as O
      so define O=T

      I have updated the blog with T replaced by O

      Delete
  28. Hi! Can any one tell me, how do i sure that the performance goal has been reached. Is there a relation between MSE and performance goal......What is the meaning of the following code..............net.trainParam.goal=1e-3

    ReplyDelete
    Replies
    1. It depends on your system's requirement. generally you want to see that network is well trained but not over trained. generally low enough MSE is desired.

      Delete
  29. I am asking a foolish question....but plz help me.
    I am creating a neural net whose input set is a matrix (15 x 100) i.e. P. How can i initialize it when declaring newff.Should i take individual row? or is there any mechanism for matrix

    ReplyDelete
    Replies
    1. It depends on how you define your data
      generally 15 rows and 100 columns mean 15 inputs and 100 data samples
      See the example above, rest is same

      Delete
  30. I want to fit the curve with noisy data of range 0.5 using rbf network,may you help for this?

    ReplyDelete
  31. can anybody help on this topic? Its urgent for me...Please help....I want to reconstruct curve from the noisy data using rbf neural network or any other feedforward network....

    ReplyDelete
    Replies
    1. http://www.anc.ed.ac.uk/rbf/rbf.html
      may be this help . try using this toolbox

      Delete
  32. can anybody help me with the following problem: I want to train a neural network taking input and output from an external text file ....there are 5 inputs and a single output..how do i get the desired trained network??

    ReplyDelete
    Replies
    1. Read the data first and form I, O matrix and proceed as the examples says

      Delete
  33. Hi there, anyone knows another type of suprovised neural network similar to net = newff?

    Thank you.

    ReplyDelete
  34. I'm trying to make the identification of a MISO system by the time I'm using the tool NNSYSID but I could not move and I do not know what steps to take after training the network, I can collaborate with another alternative for the identification of the system. thanks

    ReplyDelete
  35. I have the 100 samples data. Each sample contains 3 inputs and relevant 12 outputs. The programme needs to train by these data and able to predict suitable outputs for new 3 inputs after that. Do I need to write the programme to read 3 inputs and train the network with all samples or have to consider all 15 data as inputs?

    ReplyDelete
    Replies
    1. you can use all 3 input, just where i had 3 rows in I matrix and one in T, you wuld have 3 rows in I but the output T (O) wuld have 12 rows

      -Abhishek

      Delete
  36. i have same output for different inputs... how can i overcome this issue

    ReplyDelete
    Replies
    1. dont worry about that, just code s you would do in any other case.
      -
      abhishek

      Delete
  37. hi
    my name is sina.im studing financial math.i wrote a programme in MATLAB.now I want to use formulas of this programme in neural network.how can I?let I have a (f(x)=x) now I want use tise function at a neural network,how can I? pleas help me.this is my address :
    www.sinababaie@yahoo.com

    ReplyDelete
    Replies
    1. describe more about your code. where do you want to use you function.
      --
      abhishek

      Delete
  38. i am a beginner in neural network in matlab. I want to detect the faulty condition of a single phase induction motor using ann. Can you tell me the procedure to perform the work, please?

    ReplyDelete
  39. Hi, I am working on a trivial experiment in which there are 4 categories of images of flowers say rose, tulip, daisy and sunflowers. i intend to train a simple ANN with about 25 images of each and then test the network. The underlying logic i intend to use is a) First, convert the images into simple binary b) then, based on the binaries of the training set and the desired target outputs, carry out the weight adjustments.c) thereafter, the weight can be directly applied to the test cases.
    My doubt is can in such trivial cases, this simple kind of a logic work? anything you would want to suggest in this context that would help in more accuracy?

    ReplyDelete
  40. they are all black and white images!

    ReplyDelete
  41. Hi, Iam Sharath Kumar. My project is image retreival system for which i had extracted the features of an image. Now i want to design retrieval system using neural network can any one help me. pleas help me.this is my email id: saikumar1310@gmail.com.

    ReplyDelete
    Replies
    1. after extracting the features from an image the output is as follows below a 1x4 matrix
      1.0e+011 * 1.0448 1.1249 0.0020 1.1254
      pls kindly guide me in assigning target and p value.

      Delete
  42. Dear sir,
    i have 4 inputs from the radar .like RCS,Velocity of object,Position and Acceleration.and 4 outputs like Pedestrian,bike,car and truck.using the given inputs i want to classify the output objects.its a online process where i just have one value of from the inputs at a time.and using these input i have to decide the output. i have already observed the inputs for each output.and i know that how RCS curves for each output look like.but there are lot of overlaping between car and truck RCS .mean is that i cant classify the track in linear way.can i used neural network in this case to classify the output objects.if yes how can specify the inputs and outputs for neural network.?
    thankyou

    ReplyDelete
    Replies
    1. Yes you can, the trick is to categorize output as number 1 2 3 4 and for each data, set ur output as [1 0 0 0] if its car, [0 1 0 0] if the output is bike and so on. input will beusual values .

      Delete
  43. hi im working on emotion recognition using neural network can anyone help me? thanks much

    ReplyDelete
  44. Hi this is nik smith:Summer Internship Program & summer training program in Noida, Delhi, NCR. Click to know more or Call us at 0120-3939220.

    ReplyDelete
  45. pls can anyone help me? I need a neural network solution for software reliability prediction. I have a failure history as input(30 elements, that is time that failure occurs) and predicted future failures as output(30 elements, that is corresponding number of failures(cumulative) for that time). i need a neural network structure,training and performance evaluation .

    ReplyDelete
  46. Thanks for your interesting little tutorial.. i need an example matlab code by cascade correlation neural network.. can u help me? it is so urgent and i need help for implementation..
    Thanks

    ReplyDelete
  47. Hi all

    I need Introduction to neural networks using MATLAB 6.0 by S N DEEPA, if any one has this book or has a direct link to get it please help me. and if also any one has an cascade correlation neural network matlab code example it will be so helpful.. thank u.

    Yours

    ReplyDelete
  48. Hi all,
    my project is load forecasting using neural networks..but I don't know anything about it.so,pls suggest me where to start..

    ReplyDelete
  49. hi all
    i work in project about speaker verfication by mlp but i donot know how design mlp and how many input and output please help me

    ReplyDelete
  50. Hi all,I am working on project STOCK PRICE PREDICTION USING ANN, using RBF and MLP. Please help me in how to start with coding and design in MATLAB. I am ready to pay if anyone individually write code and build&train model for me. Email-id- mayank.patel34@yahoo.co.in
    Mail me with ur contact number if interested.

    ReplyDelete
  51. hello, i am working on project ANALYSING MACULAR EDEMA IN DIABETIC PATIENTS,using matlab. so can anyone tell about how the nueral networks works on matlab.....?

    ReplyDelete
    Replies
    1. Hi all,I am working on project STOCK PRICE PREDICTION USING ANN using matlab so can anyone tell about how the nueral networks works on matlab.....? for more click here

      Delete
    2. Hi Chaitra and Nik, this example tells you a standard implementation of neural networl. Let me know if you any particular doubts about any step.

      Delete
  52. Hi,
    i need to design, train and simulate a radial basis function network without using neural network toolbox. Can anybody help me, Plz.. kkamaljeet47@gmail.com

    ReplyDelete
    Replies
    1. Learn the algorithm and implement it. Do you have doubt ina ny particular steps?

      Delete
  53. CAN YOU ENTER THE MATLAB GENERATION OF SINE WAVE GUSING NEURAL NETWORK

    ReplyDelete
    Replies
    1. Hi Megha, I didn't get you question. Can you clarify further?

      Delete
  54. hey, can you give an example for cascade feedforward neural network? when we train the net, are the target must have a same size with the input? thank you

    ReplyDelete