Silicon Investor (SI) -- The First Internet Community

STOCKTALK

We've detected that you're using an ad content blocking browser plug-in or feature. Ads provide a critical source of revenue to the continued operation of Silicon Investor. We ask that you disable ad blocking while on Silicon Investor in the best interests of our community. If you are not using an ad blocker but are still receiving this message, make sure your browser's tracking protection is set to the 'standard' level.

Strategies & Market Trends : NeuroStock -- Ignore unavailable to you. Want to Upgrade?

To: Bill Scoggin who wrote (484)	12/17/1998 7:08:00 PM
From: Vic Nyman	Read Replies (2) \| Respond to of 805

Possible verification data contamination? Maybe I'm doing something wrong, but I thought I would ask the expert community first... As a part of further checking out the capabilities of NeuroStock, I decided to test for forward contamination of the verification data in the training activity. So far, I have tried only 1 set of networks, but the results were shocking. Maybe one of you can tell me if I am off-track here. What did I do? 1. I picked one of my favorite stocks and built a network using one related stock and two related industry indices. The entries use a 1 day related for Long filters and a 15 day related for price and short filters for each of the stocks or indices. 2. I downloaded new data files which contained data from 12/02/1996 through 12/16/1998. Then I created copies of those files in another directory and removed the data from 08/17/1998 to now. 3. I trained the model TWICE. The first model was allowed to train on the files which contained all of the data, but used a training period of 12/02/1996 to 08/03/1998. The model trained very well and gave an excellent verification period showing predictive model behavior on the market moves from 08/04/98 to 12/16/98. 4. Next, I copied the shorter files into the NeuroStock directory. These files were exactly the same as the first set, with the exception of the removal of the data in the verification period. ( I did leave 10 days of data in the file beyond 08/03/98 so NeuroStock would be training against the exact same set of days. ) I built the exact same model on this data. Same order, same relateds, same #training cycles, same settings... Now, according to my knowledge of NeuroStock, this should result in the same training on the same data. 5. Lastly, I copied the full files back into the NeuroStock directory allowing the model trained on the shorter file to see the full data in the 08/04/98 to 12/16/98 verification period. The results are VERY different. Unlike the model trained against the full file which showed very predictive behavior, the model trained against the shorter file and then exposed to the data after the fact got lost on many of the market moves. In december, the models were completely in opposition with one showing 10 straight days of BUY recommendation and the other 10 days of SELL. Have any of you seen this phenomenon? Any advice or debunking? Vic