Jay,
Interesting observation. I've formed a similar opinion myself, that the "forget" button, at least at times, doesn't actually forget old training completely. Nor does "file" and "new" from the menu.
But, shutting down and restarting NS, without rebooting seems to be enough, maybe?
Pure speculation on my part, but maybe whatever random process is used to "re-seed" the training process, maybe isn't a truly random process at all, but starts with the old weights. ???
Anyway, I've upgraded from a Cyrix 200MX to an Intel 233MMX and because of the better FPU, nets are training about 50% faster. I'm letting my old nets do some additional training and seeing some positive results. I'm also starting to think that it actually is OK to stop annealing before it's completed, as long as it's always followed by some more backprop time. In fact, a little annealing may be better than too much?
Overall, I found several of my nets were overly pessimistic over the last 2 to 4 weeks, and missed out on some very good gains. The overall market has been showing remarkable strength, so the nets seem to have been fooled by a less than common situation. I am going to start keeping copies of the nets with short or no verify periods in hopes of spotting such anomalies faster.
CL
|
