Singlevariate Model fitting for a Simulation Data without any prior i/o relation knowledge using Taylor Series Method

Singlevariate Model fitting for a Simulation Data without any prior i/o relation knowledge using Taylor Series Method

In this example we will talk about how we can fit a model/function in a single variate simulation data. In many practical cases, we get some observation data after changing an input variable and our aim is to find the relation between input and output. If we know the relation form atleast (like y=mx+c or y=d*sin(x)^2), then we can find the values of these coeefficients y,c or d by linear/non linear regression. But what if we dont know the relation form, we dont have any information about whether they follow exponential form or logarthim relation. Here is a trick which we are going to discuss in this example.

Taylor Series fit
Let i/o relation be represented by a function z=f(x). We assume here that f(x) is continuous and differentiable, which is generally the case with practical/real time systems. Now each such function can be represented using a taylor's series
\[ f(x) = \sum_n{b_n x^n} \] we will limit the summation to n=N
Since we know the data x and z=f(x), we can calculate the coefficients bi .
Let \[B=\left[\begin{array}{ccccc} b_0 & b_1 & b_2  & ... & b_N \end{array} \right]^T \] and \[ Xmat =\left[ \begin{array}{ccccc} 1 & x^1 & x^2 & ... & x^N \end{array} \right] \]
so \[ z=f(x)= Xmat B \]

This is also true when x is a column vector mx1 and contains m data points and z contains values of f at those m point.
So B can be calculated as \[B=Xmat^{-1} Z\]

function: taylorseriesfit()
Writing the function in matlab is very easy. we just need to follow the steps as explained above.
Since our function need to matrixs x and z, these will be input to the function and B will be the output of the function. We will take N=4 here. Increasing the value may result in accuracy of your fit.

function B= taylorseriesfit(x,z)
Xmat=[ones(size(x)) x x.^2 x.^3 x^4];
B=Xmat\z; 

Here x and z are column vectors representing data points.

Generation of Simulation Data 
We will generate a sample Simulation data for our example. In real example, you will get this data from some experiment after changing an input parameter(x) and observing the output(z).

x=[0:0.1:1]';

z=exp(x);

Calling the defined function

B=taylorseriesfit(x,z);
disp(B);

You will get B as

B =
1.0000
0.9988
0.5099
0.1400
0.0696

Now your model is
\[ z= x*B = B(1)+B(2)x+B(3)x^2+B(4)x^3+B(5)x^4 \] \[ z= 1+0.99x+0.50 x^2+0.14x^3+.07x^4 \]

Error
we can calculate the modelling error (MMSE) by comparing the actual output and the simulation output.

z_o = Xmat*B;
e=sqrt(sum((z-z_o).^2));
disp(e);

you will get something like

e= 6.9400e-005

which is pretty good.

Final Observation
Now if we observe the coefficients and plot them with respect with n, we may guess the actual function. Fig1 represents such a graph for N=7 case.

Fig1 : b_i plotted versus i

like for this example, it looks like 1/n!, so this is an exponential relations. Well, if we can guess, it is good. But if we cannot, still we have a relation which is sufficient for all numerical purposes.