Checking Strategy Performance

When you use a trading strategy, the short term success of that strategy can depend on certain factors like stop losses, take profit, position sizing and so on. A poor strategy can sometimes be masked by creative trade management. Likewise, a good strategy can execute poorly if trade management isn’t done properly.

An objective way to measure the performance of a strategy is to measure the reliability of the signals it produces and strip out all other factors. This will tell you the rate of true positives, and false positives. This is known as hypothesis testing. True positives are when a buy signal is generated and the market did go up or a sell is generated and the market did go down over a predetermined time.

The goal then is to maximize the rate of true positives, and minimize false positives and false negatives. The false cases are where a strategy generates the wrong signal, or where the market moves and the strategy fails to create any signal at all.

Card image cap

Indicators page

After you’ve created a strategy, go to the Indicators page. Open the search box, by clicking the spyglass icon. Then click the uncheck box to deselect other indicator types. Click the box that is labeled Custom Strategy.

Card image cap

Custom Strategies

The list should show all of the strategies that you’ve created and which are valid to run. To be valid, the strategy must have been trained. Select from the list the strategy that you want to measure, then click the icon.

Card image cap

Strategy performance

On clicking the icon, you should see a chart visualization of the strategy performance. This marks the buy signals and the sell signals.

Hypothesis Classification

The metrics for the buy side signals are:

  • True positives: The market went up and the strategy correctly signaled buy
  • False positives (hold): The market was flat and the strategy falsely signaled buy
  • False positives (sell): The market went down and the strategy falsely signaled buy
  • Average profit: The average profit over all buy signals
  • Total: The total number of buy signals measured

The metrics for the sell side signals are:

  • True positives: The market went down and the strategy correctly signaled sell
  • False positives (hold): The market was flat and the strategy falsely signaled sell
  • False positives (sell): The market went up and the strategy falsely signaled sell
  • Average profit: The average profit over all sell signals
  • Total: The total number of sell signals measured

In the screen above, the true positive rate for buy signals was 78.9%. This means, of 33 signals that the strategy generated, 26 of them correctly predicted a market rise. The average market move, for all 33 cases was 7.8%. It’s important to check the average profit because for some strategies, the market move for false positives may be much bigger than that for true positives. This means losing trades potentially could lose more on average than winning trades gain. This would make for a losing trade system.

Card image cap

Calculation settings

You can change the settings for the performance test. To do this, click the expand icon at the top right of the chart. The two settings below determine how the measurements work.

  • Target trade time
  • Target trade profit

The target trade time sets the time in chart bars, over which the check is made. For example, on a daily chart, when this is set at 50, the measurement is done over a 50-day period. The target trade profit, sets the definition for a flat, up or down market. For example, if target profit is 1%, then buys must predict a market move of at least +1% over 50-days to be considered successful. Sells must predict a market move of at least -1% to be classed as successful (true positives). Buys and sells are classed as false if the market stays with a range of plus or minus 1% over 50 days.

Draw down

Unlike most other testing, hypothesis testing also accounts for draw down. For example, if a buy signal is made and the market moves up by the end of the period by at least +1%, but there was a fall in between that was greater than -1%, this would be classed as a false positive. This is because the strategy should have signalled a sell rather than a buy. Take profit and stop loss settings might have made the trade profitable, but the downside risk was greater than the upside reward.

Card image cap

Reviewing performance

Use the other settings above to test how the strategy performs for other markets and at other time frames. For example, the above chart shows how the same strategy performs on the Google weekly chart. The chart below shows the same result for a daily chart.

Card image cap

Daily chart performance

If a strategy doesn’t produce reliable signals, it’s unlikely to do well over the longer term. Classifying performance in this way should therefore be done as a baseline. It’s independent from other forms of testing like back testing and forward testing.

Risk Disclosure: Information given on this website is for general purposes only and should not be construed as investment advice. Certain instruments shown here are complex and may come with a high risk of losing money rapidly due to leverage. You should consider whether you understand how these instruments work and whether you can afford to take the high risk of losing your money. Market data displayed is indicative and is not a solicitation to buy or sell.