Function approximation: Neural network great

tea asked: 2021-07-26 neural networks , generalization , trainbr , trainlm , simulation , matlab

Learn about function approximation using neural networks with MATLAB. Explore practical examples and techniques for building powerful models. Get started now!

Expert Answer

Prashant Kumar answered . 2025-07-29 20:37:40

% I need some help with NN because I don't understand what happened. One % hidden layer, I=4, H=1:20, O=1. I run each net architecture 10 times % with different initial weights (left default initnw). I have in total % 34 datasets

 Do you mean data points N = 34?

It typically takes ~ 10 to 30 data points per dimension to

adequately characterize a distribution. For a 4-D distribution I'd recommend

 40 <~ Ntrn <~ 120

% which were divided 60/20/20 when using Levenberg-Marquadt

 Ntrn = 34-2*round(0.2*34) = 20

 Hub = (20-1)/(4+1+1) = 3.2

indicating you really don't have enough data to adequately characterize a 4-D distribution.

You should consider

 1. Dimensionality reduction
 2. k-fold crossvalidation
 3. Adding new data with the same mean and covariance (stdv + 
correlations) matrix

% algorithm. Mse_goal = 0.01*mean(var(t',1)), i calculate NMSE and R^2, % choose best R^2, for that check performance of each subsample, check % regression plots, check rmse. R^2 is usually around 0,95; R for each % subset 0,98... But when I simulate network with completely new set of % data, estimations deviate quite a lot. It is not because of % extrapolation.

 No. It probably is. Your training data subset is insufficiently 
large for 4 dimensions.

 I would begin with minimizing H with dividetrain. Then consider 
k-fold crossvalidation.

% Data are normalized with mapminmax, transfer functions tansig, % purelin. % Trainbr was my first choice actually, since I have small dataset and % trainbr doesn't need validation set (Matlab2015a), but it is awfully % slow. I ran a net with trainbr and we are talking hours versus minutes % with trainlm.

This may be a BUG. Let MATLAB know. What version are you using?

>> ver

% I've read a ton of Greg Heath's posts and tutorials and found very % valuable information there, however, still nothing. I see no way out.

 It typically takes ~ 10 to 30 data points per dimension to adequately 
characterize a distribution,
I suggest calculating the means and stdv for each data set to see how 
much your training data is representative of the total 4-D 
distribution that includes the new datasets. 2 or 3-D 
color coded projections may be helpful.