Quantopian's community platform is shutting down. Please read this post for more information and download your code.
Back to Community
What is the difference between real trading and backtesting

I've read this thread :
https://www.quantopian.com/posts/seeking-feedback-on-rsi-based-algo

The algorithm is very short and simple, it outperforms the s&p500 on any period I've tested it (even 2007-2009 when the markets crashed).
For example in 2016 the performance is 37.7% while the s&p500 gained 13%.

If the algorithm was traded with real money, will the outcome was the same ?
Are the commissions payed to IB simulated somewhere ?
What are the major differences between real money trading and backtesting ?

7 responses

You make imaginary money with one and lose real money with the other.

🌋

:) Care to elaborate ?

Hello Dima -

There are differences between what the Quantopian backtester can model, and real-money trading:

  • Slippage & commissions. For example, see How Accurate is Our Slippage Model: Comparing Real and Simulated Transaction Costs (commissions are discussed, despite the title of the blog post).
  • Borrowing costs. I'm not familiar with the mechanics here, but any time you go short or the leverage is greater than 1.0, there are costs that are not captured, as I understand.
  • Margin calls.
  • Lack of availability of stocks for shorting.
  • Taxes.
  • "Day trading" and other regulatory restrictions.

Anthony and others may be able to list more.

For the algo you reference, it is not particularly attractive in my opinion, since SR ~ 1.0 with a max drawdown of 38%! There are also long periods where it does nothing. So, you'd have to have a real faith in the long-term viability of the strategy to stick with it. That said, in the end, you'd just be holding VTI and TLT, which are ETFs and not so crazy (I'd worry about TLT, though, since it is a long-duration bond ETF, which probably has more risk than one would think). Go for it, and post back here in 20 years to let us know how it worked out!

Hi Grant,

First of all thank you for your answer.

What is SR ?

SR stands for Sharpe ratio. It is a reward-to-risk ratio, so higher is better. My sense is that a long-term backtest that shows a SR of ~ 1.0 is o.k., but the volatility/drawdown is not so hot. I'd also be careful to make sure you are comparing apples-to-apples. If the leverage is > 1.0, then it is an unfair comparison to the unleveraged SPY, when talking about returns.

I've tested this algorithm without leverage. The returns are not as good, but still better then the S&P 500.

Can you elaborate on how the drawdown is calculated in quantopian. Let's say I run the algorithm on 10 years period, the drawdown will be the highest point vs the lowest point ? or is it computed in some window like one month ?

Not sure about the drawdown calculation. My guess is that it is overall. You could probably eyeball the plot of the returns to see if the reported drawdown is consistent with this assumption.