The best Ann configuration
I have run Ann for prediction with hidden nodes from 2-17 about 50 times. My question is which criteria I should rely on to select the best Ann? Should I choose R squerd of Test or Mse of the Ann or validation performance?
Neeta Dsouza answered .
2025-11-20
1. Accept all default parameters except for the number of hidden nodes, H.
2. Minimize H subject to the constraint that the degree-of-freedom adjusted
mean-square-error of the training data is less than 1% of the average training target variance.
3. Design and test Ntrials >= 10 nets for each value of H in a range less than the upper
bound Hub (determined by not having more unknown weights Nw than training equations Ntrneq).
The untrained nets only differ by the random trn/val/tst data division AND the random initial weights.
4. Rank the nets via their slightly biased performance on validation data.
5. Obtain unbiased performance estimates on the nets using test data.
6. Statistically significant differences in performance can be estimated using the standard
deviation of the performance estimates.
I have posted zillions of examples in both the NEWSGROUP and ANSWERS using the same notation. Therefore searching with
greg fitnet Ntrials
should dig up enough references to clarify what I have written.