Tharp's Thoughts Weekly Newsletter (View On-Line)

  • Article SQN® Optimization by Means of Step-Forward Testing by Douglas Rechia
  • Trading Education Van Tharp Signature Peak Performance Training
  • Trading Tip Don’t Throw the Baby Out with the Bath Water by D.R. Barton, Jr.
  • Offer Free Definitive Guide to Position Sizing with Purchase

$700 Discounts Expire Today On These Workshops


DougSQN® Optimization by Means of Step-Forward Testing

Part 1: The Process

Trading system optimization has become much more common in the last fifteen years with the development of numerous computerized platforms. Some argue that optimization is vital for trading performance and keeps you in tune with the most recent market conditions, while others argue that optimization only fits past data and thus does not benefit real trading.

I'd like to talk about an optimization process that may satisfy both groups. Step-Forward Testing (SFT for short) is an optimization technique that alternately tests in-sample and out-of-sample data. With this technique, out-of-sample test performance is closer to real trading than simple backtesting with optimized, or overfitted, parameters.

Background: Step-Forward Testing

Optimization, also called massive testing, amounts to an exploration of a search space through successive tests, while looking for the test that maximizes (or minimizes) a target function. The search space is the set of all possible combinations of values that can be assigned to the system parameters, such as the number of periods considered in moving averages for a trend-following system. The target function is the mathematical representation of what the trader wants to achieve through optimization, such as maximum profits, minimal risk, or possibly some combination of both.

Process definition. According to Perry Kaufman’s book New Trading Systems and Methods (Wiley, 4th edition), massive testing involves reserving some data from initial testing for validation. Data used for testing is called in-sample data and the reserved portion of data for validation is out-of-sample data. For example: when a trader has 10 years of data available for testing, a standard approach would be to use the oldest 9 years to find the best parameters, then validate the system performance using the selected values on the most recent year’s data.

There’s a refinement to this basic process that involves running tests on in-sample data and validation on out-of-sample data multiple times on thinner slices of data. This process is called Step-Forward Testing and follows the simplified sequence below:

  1. Select the total test period.
  2. Divide the total test period into multiple individual in-sample data intervals and out-of-sample data intervals (more on this process later).
  3. Do massive testing with the in-sample data and select the best parameter values from those in-sample test results.
  4. Then use the best parameter values to validate the performance of the system in each out-of-sample data interval.
  5. Examine the system performance.

In order to follow the SFT process and analyze its effectiveness, let's experiment using a trend-following system that I call Trend-I.

The Underlying System: Trend-I Rules

Trend-I is a long-only trend-following system that served as a proof of concept and benchmark reference for other non-optimized trend systems. Trend-I was tested over a period of about 15 years with all 67 stocks in the most well-known Brazilian index: Ibovespa (the index itself is not traded).

Trend-I is a fully automated, parameter-based trading system. Parameters for Trend-I are:

  • The number of periods used for the market SQN® technical indicator to measure the trend of the market (Ibovespa).
  • The maximum market volatility allowed.
  • The number of periods c used for SQN® to measure the trend of individual stocks within the index.
  • The coefficient d used to calculate the trailing stop.

Trend-I’s system rules are as follows:

  1. Market Type: Trade system only if (i) market direction is either bullish or strong bull, and (ii) market volatility is either normal or quiet. For our purposes, Ibovespa index defines the market (for a detailed analysis on how to assess the market type, refer to Van Tharp’s series of articles, "Understanding Market Type and Putting Today’s Volatility in Perspective," which appears in May and June 2009 issues of this newsletter).
  2. Setup: Identify bull or strong bull trends in individual Ibovespa stocks. Some minimal SQN® score of the daily price change percentages over the last ____ days identifies a bull or strong bull trend. The system then looks for a retracement as the setup. The retracement occurs when the Relative Strength Index (RSI) over the last 2 days reads below 30. Calculate entry price and initial stop loss. Enter only if entry price is higher than initial stop loss.
  3. Entry: Enter the day after the setup signal at a price higher than the mean of the daily median price of the last 3 days. The order should expire in 3 working days if it does not execute.
  4. Exit: After entry, use a trailing stop to both protect capital and take profits. First, find the highest high of the last 30 days. Subtract the Average True Range (ATR) of the latest 30 days times a multiplier d from the highest high (d is one of the system parameters). If the result is greater than yesterday's stop, adjust today's stop to the recently calculated value; otherwise, keep yesterday's stop. Re-calculate the stop daily, at the end-of-day, until the position is closed.

For the parameters above, I will assume that the:

  • Number of periods for market SQN® is 25 days.
  • Max market volatility is normal.
  • Number of periods to identify an individual stock trend will be selected by optimization.
  • ATR multiplier for the trailing stop will be selected by optimization.

Putting it All Together

First, we’ll select the number of periods for stock trend and the ATR multiplier; then we’All analyze the performance of Trend-I.

Test period. Let's start with a test period from January 1st, 1995 to October 18th, 2011.

Test interval size. Next, we select the size of individual in-sample intervals. We need to use some judgment when choosing between short and long intervals. Short intervals are more likely to produce overfitting because they generate just a few trades. I always assume that the more trades the in-sample data generates, the less likely the optimization will overfit past quotes. On the other hand, long intervals will probably generate statistically significant results, but they may fail to reflect the most recent conditions of the market. For my purposes, I decided to start by using a period of 60 months for the in-sample testing.

The size of out-of-sample data determines how often the optimization (i.e. massive testing) takes place. One hypothesis holds that the interval period depends mostly on the average number of days that a trade holds a position. Since preliminary tests showed that, on average, Trend-I holds positions for 28 trading days, out-of-sample data should last at least 3 months.

How would we apply these periods to in-sample and out-of-sample testing? Let's assume that our in-sample interval size is 60 months, and that our out-of-sample size is 12 months. In that case, every year, one optimization will be done using the last 5 years’ worth of data to find the optimal parameters. Table 1 shows the size of intervals along with the start and end dates for each test.

Table 1

On subsequent tests, the following time intervals were used for in-sample and out-of-sample data, respectively: (60, 6), (40, 4) and (30, 3). These intervals were selected according to Kaufman’s suggestion that the out-of-sample size should be about 10% of the in-sample size.

The Range of Values for Optimized Trend-I Parameters. It is necessary to define the range of values from which the parameters are chosen during massive testing. For this, discretion is needed. Ranges that are too wide will lead to very large search spaces—that is, to a high number of combinations. For instance, if the number of periods to be tested in parameter c is 10, and the number of constants in d is 15, there are 10 x 15 = 150 combinations. As more parameters are tested and the range of individual parameters increases, the number of combinations also increases. Consequently, the time it takes the computer to find the optimal parameters can get out of control.

Although using genetic algorithms and other techniques can shorten exploration time, they were beyond the scope of this basic experiment. You need to limit the size of the search space as much as possible. According to Kaufman, limiting the search space is actually an advantage because it forces a decision, in advance, about the range of parameters for the strategy. Without these restrictions, the test process moves away from validation and closer to indiscriminate exploration.

Now, back to Trend-I. The range of values chosen for the c parameter: {number of days to define a trend - 15, 20, 25, 30, 35, 40, 50, 60, 70, 80, 100}; and for d: {ATR multiplier - 2.00, 2.25, 2.50, … 5.00}. Such ranges amount to a search space with a size equal to 11 x 13 = 143 combinations.

Target function to be optimized. Optimization involves the search for the “best” results. What are the best results? I believe that each trader must decide how to define “best” prior to any system development work based on his or her objectives. I also believe that a proper position sizing algorithm plays a vital role in achieving those objectives, and that is why I chose to optimize on SQN.

According to Van Tharp in The Definitive Guide to Position Sizing™, the higher the SQN® score, the easier it is to use position sizing strategies to achieve your trading objectives. It seemed reasonable to me to use SQN® as the target function to maximize in the massive testing with in-sample data.

In next week’s newsletter, Douglas will review the results from his SFT experiment, discuss a big surprise and summarize his thoughts on the usefulness of the SFT process.

About the Author: Douglas N. Rechia is an experienced computer programmer, software architect and designer. He lives in Florianópolis, a small island city in the South of Brazil. About two years ago, he decided to apply his computing skills to the markets. He welcomes comments from readers as he has not yet put his experimental systems into production. You can reach him at rechia (at) gmail.com.

 

Disclaimer


Trading Education

Workshops

Today is your last chance to register for Core Trading Systems and Blueprint for Trading Success workshops at a discount.


Next we have Van’s signature workshop for traders, Peak Performance 101 followed by his advanced Peak Performance 202 workshop. Then in June we have our first three-day Forex workshop coupled with our system development workshop
.

Click the name of the workshop below to learn all the benefits for your trading advancement.

Apr
21-22

$2,295

$2,995

Core Trading Systems: Market Outperformance and Absolute Returns
Longer Term Systems for Consistent Trading Profits

Cary, NC

Apr
24-26

$2,295

$2,995

Blueprint for Trading Success
Build the Foundation of Successful Trading

Cary, NC

Apr
28-29

$195

Oneness Awakening Workshop
Become more positive, calm, and centered

Cary, NC
May
17-19

$2,295

$2,995

Peak Performance 101
Van Tharp's Signature Workshop

Cary, NC
May
21-24

$3,295

$3,995

Peak Performance 202
with Dr. Libby Adams

Cary, NC
June 2

$299

Tharp Think
Your chance to get a taste of a Van Tharp workshop for only $299

Cary, NC
June
14-16

$3,295

$3,995

Forex Trading
New! Three-Day Workshop

Cary, NC
June
18-20

$2,295

$2,995

How to Develop A Winning Trading System That Fits You
with Van and RJ Hixson

Cary, NC

To see our full workshop schedule including dates, prices, and location, click here.

Now posted, dates for Aug-Oct!


Trading Tip

drETPs—Don’t Throw the Baby Out with the Bath Water

Part 1


In the lead article for Tharp’s Thoughts last week, we talked about some quirks inherent in trading Exchange Traded Funds (ETFs) and Exchange Traded Notes (ETNs)—collectively known as Exchange Traded Products (ETPs). In particular, we dug into what happens when an ETP issuer chooses to stop creating new shares, as has been the case recently in the volatility ETN symbol TVIX and the natural gas ETN symbol GAZ. After new share issuance was suspended, these two products started trading like closed-end funds with a premium over their Net Asset Value. When Credit Suisse announced it would start issuing new TVIX shares again recently, traders and investors who were holding shares bought at a premium took large hits to their positions in a very short time. GAZ owners face a potential similar prospect at some point.

Many of you wrote to say thanks for the article and added some interesting commentary and/or questions. We really appreciate the feedback! A number of readers also asked if, with these types of problems hanging around, it was even worth trading or investing in ETPs.

My answer is a resounding yes! I said last week that ETPs are one of the top financial innovations of our generation—which brings us back to the idiom in the title: don’t throw the baby out with the bath water! That expression dates back to 16th century Germany and reminds us not to get rid of what’s important when disposing the bad.

Last week I likened ETPs to Disney World—both are really great and both have quirks. ETPs are so useful and popular, in fact, that 6 out of the top 10 volume stocks over the last 30 days on U.S. exchanges are ETPs! Those six ETPs are SPY (S&P 500), XLF (Financial Sector), EEM (Emerging Markets), VXX (Volatility), IWM (Russell 2000 Small Caps) and QQQ (NASDAQ). And for those who are curious—the four individual stocks that round up the top ten share volume stocks of Bank of America (BAC), Sirius XM Radio (SIRI), Ford (F) and Nokia (NOK).

The bottom line is that the benefits of trading and investing in ETPs far outweigh the downside. AND… if you use the tool we described in last week’s article—namely, reviewing the ETPs’ indicated value—you can easily protect yourself from one of the few downsides of trading these great vehicles.

Lastly, if you haven’t read last week’s article, “Exchange-Traded Funds and Notes: A Trick That Not 1 in 100 Traders Knows,” click here. It’s worth a few minutes of your time to add extra tool to your tool kit!

As always, I’d love to hear your comments and feedback. Send them to drbarton “at” vantharp.com.

Great Trading, D. R.

About the Author: A passion for the systematic approach to the markets and lifelong love of teaching and learning have propelled D.R. Barton, Jr. to the top of the investment and trading arena. He is a regularly featured guest on both Report on Business TV, and WTOP News Radio in Washington, D.C., and has been a guest on Bloomberg Radio. His articles have appeared on SmartMoney.com and Financial Advisor magazine. You may contact D.R. at "drbarton" at "vantharp.com".

 

 

Disclaimer


Package Price

Free Definitive Guide

Def Guide

For a limited time, get Van's comprehensive Definitive Guide to Position Sizing free, when you purchase the intro e-learning material and the trading simulation game.

  • Learn the basics of Position Sizing Strategies™ in the Introduction to Position Sizing E-Learning Course. ($149)
  • Apply what you learned (and have fun) with the Position Sizing Trading Game. ($195)
  • Discover new levels of position sizing strategies in the Definitive Guide to Position Sizing. ($199)

All three for only $344

You save $199!

Purchase Now

Learn More


Ask Van...

Everything we do here at the Van Tharp Institute is focused on helping you improve as a trader and investor. Consequently, we love to get your feedback, both positive and negative!

Click here to take our quick, 6-question survey.


Also send comments or ask Van a question by using the form below.

Click Here for Feedback Form »

Back to Top


Contact Us

Email us at [email protected]

The Van Tharp Institute does not support spamming in any way, shape or form. This is a subscription based newsletter.

To change your e-mail Address, click here

To stop your subscription, click on the "unsubscribe" link at the bottom left-hand corner of this email.

How are we doing? Give us your feedback! Click here to take our quick survey.

800-385-4486 * 919-466-0043 * Fax 919-466-0408

SQN® and the System Quality Number® are registered trademarks of the Van Tharp Institute

Back to Top

 

April 18, 2012 - Issue 573

VTI logo


 

ST expanded

A Must Read for All Traders

Super Trader

 


 

How are we doing?

Give us your feedback!

Click here to take our quick survey.

From our reader survey...
"I think the newsletter is extremely generous and it is a resource I utilize constantly. I have saved every single one since I first subscribed."

 


 

Trouble viewing this issue?

View On-line. »




 

Tharp Concepts Explained...

  • Trading Psychology

  • System Development

  • Risk and R-Multiples

  • Position Sizing

  • Expectancy

  • Business Planning

Learn the concepts...

 


 

Read what Van says about the mission of his training institute.

 



The Position Sizing Game Version 4.0

Picking the right stocks has nothing to do with trading success and neither do amazing trading systems with high percentage wins. The Position Sizing Game teaches you the key elements of trading success. Learn more.

To Download for Free or Upgrade Click Here

game


Download the 1st three levels of Version 4.0 for free.

Register now. »

 


 

Trouble viewing this issue?

View On-line. »

 


 

tywff

 


A Thousand Names for Joy: A Commentary

You can read Super Trader Curtis Wee's full review here.


 

RECOMMENDED READING

For a list of recommended reading click here for our Amazon store.

 


 

Dr. Tharp is on Facebook

FACEBOOK


 

Follow Van through

Twitter »


Van Tharp Trading Education Products are the best training you can get.

products

Check out our home study materials, e-learning courses, and best-selling books.

Click here for products and pricing


 

What kind of Trader Are You? Click below to take the test.

Tharp Trader Test

 

Back to Top


 

Ecourse position sizing

Introduction to Position Sizing™ Strategies E-Learning Course

Only $149

Learn More

Buy Now


 

SQN® and the System Quality Number® are registered trademarks of the Van Tharp Institute