How to Blind Test and Forward Test a Trading Strategy
First of all what is blind testing and forward testing? When you design a trading strategy, you will normally look at a historical price chart and see how the strategy performs against some parts of that data series. The problem with this is that it introduces bias throughout the design process. It means the strategy is being designed to fit a certain data set, often unintentionally. Then when the strategy is deployed in the real world it fails because it was over-fitted to a historical subset of data.
Segmenting data series
The way around this problem is to split up the test data. Use different data for design, testing and optimization. A blind test is a test that is done on clean data. That is, data that wasn’t used as part of the training or optimization. A forward test is when the test is done on the newest part of the price series and the older part is used for design and optimization.
When creating a strategy, it’s best then to divide the data up into sections in advance and these will be used for design, testing and optimization. Then create training points inside of the section designated for training, rather than over the whole data set.
Running the back tester, will tell you whether the test is a blind test or not. To check this, select a strategy from your list and open the back tester page.
Click the toggle switch that says Show Data. Doing this will highlight each data segment as shown in the screen above. The green parts are blind tests. This means this part of the data was not used for training, or for optimization.
The blue section shows which segment was used for training and the brown which segment was used for optimization. In this example, the optimization and the training covered some of the same date range.
If you click the calendar icon on the toolbar, the date selector will show the date ranges that were used for training and optimization. Moving the selector will tell you if that date range will be a blind test.
On running the test again on dates that were not used for training, or optimizing, we are left with true blind test of the strategy as shown in the screen below. From this we can be sure there were no unintended biases because this segment of data was not used in the design.
If you optimize the strategy, it will show which date ranges have been used. The optimization can overlap the training data. However, the important thing is not to optimize on data that will be used for testing.