Backtesting and Forward Testing: The Importance of Correlation

Rate this post
Backtesting and Forward Testing: The Importance of Correlation

Traders who are eager to put a trading concept to the test in a live market sometimes make the error of relying only on backtesting results to assess if the system will be lucrative. While backtesting may give significant information to traders, it is often deceptive, and it is just one component of the assessment process.

Out-of-sample testing and forward performance testing give further assurance of a system’s efficacy and may reveal a system’s true colors before real money is at stake. A good connection between backtesting, out-of-sample, and forward performance testing data is critical for establishing a trading system’s feasibility.

Key Takeaways

  • Backtesting is the process of applying a trading system to historical data to see how it would have fared during a certain time period.
  • Backtesting is frequently supported by trading platforms, allowing traders to test ideas and gain insight into an idea without risking cash.
  • Some traders and investors may hire skilled programmers to turn their concepts into tested prototypes.
  • Optimization studies, in which a trader inputs a range of data to determine which concept would have performed best, and forward performance testing, in which a simulation of trading is done following the logic of the idea, are examples of related study into hypothetical situations.
  • Many brokers and trading platforms provide simulated accounts where investors may practice using the tools and doing research on hypotheticals without putting their own money at risk.

Backtesting Basics

Backtesting is the process of running a trading system against historical data to see how it would have fared during the chosen time period. Backtesting is commonly supported by today’s trading systems. Traders may test ideas with a few keystrokes and obtain insight into their efficacy without putting money at risk in their trading account. Backtesting may be used to analyze basic concepts such as how a moving average crossover might operate on historical data, as well as more complicated systems with many inputs and triggers.

Backtesting is possible as long as a concept can be quantified. Some traders and investors may seek the assistance of a trained programmer in developing the concept into a tested form. Typically, this entails a programmer writing the concept into the trading platform’s proprietary language. User-defined input variables may be included by the programmer, allowing the trader to “tweak” the system.

As an example, consider the following basic moving average crossover system: The trader would be allowed to enter (or adjust) the lengths of the system’s two moving averages. Backtesting might be used to identify which moving average lengths would have performed best on previous data.

  The Top 5 Impact Investing Firms

Optimization Studies

Many trading systems also provide optimization research. This requires specifying a range for the requested input and allowing the computer to “do the arithmetic” to determine which input performed the best. A multi-variable optimization may do the math for two or more variables to find which combinations produced the best results.

Traders, for example, may tell the computer whatever inputs they want to use in their strategy; these will then be adjusted to their optimal weights based on the tested historical data.

Backtesting may be thrilling since an unprofitable system can often be miraculously converted into a money-making machine with a few tweaks. Unfortunately, tinkering with a strategy to obtain the highest degree of historical success often results in a system that performs badly in actual trading. This over-optimization results in systems that merely appear nice on paper.

Curve fitting is the use of optimization analytics to generate the largest number of profitable trades based on the historical data utilized in the testing period. Although it seems good in backtesting results, curve fitting results in unstable systems since the findings are effectively custom-designed for that specific data and time period.

Backtesting and optimizing have several advantages for traders, but they are just one element of the process when assessing a possible trading system. The next stage for a trader is to apply the strategy to historical data that was not utilized in the original backtesting phase.

In-Sample vs. Out-of-Sample Data

It is useful to reserve a time period of historical data for testing reasons when testing a concept on historical data. The in-sample data is the original historical data on which the concept is tried and improved. The reserved data set is referred to as out-of-sample data. This configuration is critical to the assessment process because it allows you to test the concept on data that was not included in the optimization model.

As a consequence, the notion will remain unaffected by the out-of-sample data, and traders will be able to predict how well the system would perform on fresh data, i.e., in real-world trading.

Before beginning any backtesting or optimization, traders might put aside a fraction of the historical data for out-of-sample testing. One approach is to split the historical data into thirds and set aside one-third for out-of-sample testing. For early testing and optimization, only in-sample data should be utilized.

  The Two-Hour-a-Day Trading Plan

The graph below depicts a time line in which one-third of the historical data is set aside for out-of-sample testing and the other two-thirds is utilized for in-sample testing. Although the graphic below represents the out-of-sample data at the start of the test, in most cases, the out-of-sample section comes directly before the forward performance.

A time line representing the relative length of in-sample and out-of-sample data used in the backtesting process. Image by Julie Bang © Investopedia2020

Correlation refers to parallels between the two data sets’ performances and general trends. Correlation metrics may be used to assess strategy performance reports generated during testing (a feature that most trading platforms provide).The higher the connection, the more likely it is that a strategy will perform well in forward performance testing and actual trading.

The diagram below depicts two distinct systems that were tested and optimized on in-sample data before being applied to out-of-sample data. The graphic on the left depicts a system that was obviously curve-fit to perform well on in-sample data but failed utterly on out-of-sample data. The right-hand graphic depicts a system that did well on both in- and out-of-sample data.

Two equity curves. The trade data before each yellow arrow represents in-sample testing. The trades generated between the yellow and red arrows indicate out-of-sample testing. The trades after the red arrows are from the forward performance testing phases.

After developing a trading system using in-sample data, it is ready to be applied to out-of-sample data. Traders may assess and compare the performance outcomes of in-sample and out-of-sample data.

If there is no connection between in-sample and out-of-sample testing, as seen in the left chart in the picture above, the system has likely been over-optimized and will not perform well in actual trading. If there is a substantial connection in performance, as shown in the right figure, the next round of review will include an extra kind of out-of-sample testing known as forward performance testing.

Forward Performance Testing Basics

Forward performance testing, often known as paper trading, offers traders with additional out-of-sample data to assess a system. Forward performance testing is a simulation of real trading in which the system’s logic is followed in a live market. It is also known as paper trading since all deals are completed on paper only; that is, trade entries and exits, as well as any profit or loss for the system, are recorded, but no actual trades are made.

  The Potential of Low-Priced Options

Following the system’s logic correctly is a key component of forward performance testing; otherwise, properly evaluating this phase of the process becomes difficult, if not impossible. Traders should be honest about their trade entry and exits and avoid practices such as cherry-picking deals or not include a trade on paper because “I would never have made that trade.” If the deal happened in accordance with the system’s logic, it should be logged and analyzed.

Many brokers provide a virtual trading account where trades may be made and profits and losses assessed. A simulated trading account may provide a semi-realistic environment in which to practice trading and further evaluate the strategy.

The results of forward performance testing on two systems are also shown in the image above. Again, the system shown in the left figure falls short of expectations beyond the first testing on in-sample data. However, the system shown in the right chart continues to perform effectively throughout all stages, including forward performance testing. A system that produces favorable findings with a high correlation between in-sample, out-of-sample, and forward performance testing is ready for deployment in the market.

The Bottom Line

Backtesting is a useful feature that is accessible on most trading platforms. Dividing historical data into various sets to allow for in-sample and out-of-sample testing may provide traders a realistic and efficient way to evaluate a trading concept and strategy. Because most traders use optimization methods in their backtesting, it is critical to analyze the strategy on clean data to establish its feasibility.

Adding forward performance testing to the out-of-sample testing adds another degree of protection before releasing a system into the market and risking real money. Positive findings and a strong connection between in-sample and out-of-sample backtesting and forward performance testing raise the likelihood that a system will perform well in real-world trading.

Investopedia does not provide tax, investment, or financial advice. The material is offered without regard for any individual investor’s investing goals, risk tolerance, or financial circumstances, and may not be appropriate for all investors. Investing entails risk, including the possibility of losing money.

You are looking for information, articles, knowledge about the topic Backtesting and Forward Testing: The Importance of Correlation on internet, you do not find the information you need! Here are the best content compiled and compiled by the team, along with other related topics such as: Trading.

Similar Posts