Cat Lady,
"please don't feel obligated to try explaining any further unless you really want to"
Yeah, that last posting got kinda out of control. It's just that understanding something about this business of overfitting is likely to be critical in our collective optimization of how to use NS. However, you verified my main point...if there are 2200 weights that must be determined, and you are only using 750 or so data points, then conventional wisdom says there is a potential problem! Unless of course, the net architecture solves the problem for you somehow, which, I suppose that in principle it could.
"Among the parameters I see in the file is in nodes and out nodes, knowing that and the number of neurons can the number of weights be calculated?"
Maybe. IF the net is a simple back prop type of network with a single "hidden layer", then the total number of neurons is the sum of the number of inputs, hidden neurons and outputs. All of our available neurons (40 in your case?) are in the hidden layer(s). The number of weights for such a network is the product of the same three numbers, or if, for example, you have 10 inputs, 10x40x2, or 800(!). There are other architectures that could result in a different number of weights, but the total number of neurons is always the same, 52 in this case.
In general, one wants to minimize the number of neurons in the hidden layer for such simple net architectures, but in the case of NS, it certainly seems to be different. I let the MEH net that Jay built with indices train for additional time, and allowed it to adjust the number of neurons, and it quickly got from 40 to my maximum, 80. This behavior does not ALWAYS occur, however, and I frankly don't have a clue about when/why it does. Maybe it doesn't matter, as long as the net predicts properly, like your COMS net, eh?
Well, before this gets out of control again...
Dave |
