thanks for sharing. I might trade this one.

Try it with some of the reverse ETFs like SDOW. 50% more return with same drawdown

Hi Chris. The script shorts stocks when it is in down trend, so adding SDOW (3x leveraged short DOW index) may be redundant. For example, when DOW index is going down, it will short DIA and long SDOW, and it will bet too much (4x) DOW falling compared to all other assets. On the other hand, it might make sense to add leveraged bond ETFs, even though they are not allowed in the contest, to make it volatile enough compared to equities.

Good clean algo. Useful frame work you have.

Use of regression.! I am loving my stay at Quantopian.

Naoki,

Thanks very much for sharing this. Agree that it's super helpful / education for me to see coding and how people are using this. Your code solved one of my questions about creating large universes of ETF's and then checking to see if they existed at the time or not, so that I can run max. length backtests. And all the code is very simple and clean to me (as a novice programmer). Thanks!

I like the linear regression for trend analysis also. I've seen that on other Quantopian systems, but hadn't seen that much on other platforms I was on. I would suggest that you use limit or vwap orders on the entries. Even with large ETF's, can have very volatility intraminute pricing (and sometimes huge movements) that can lose you a lot of money without that. A single bad trade could really hurt. But, that will depend on 'time decay' of the pattern.

I am curious if you have studied the 'signal decay' (how important is it to get really rapid fills for returns). If so, how do returns 'fall off' with only checking signals once every 3 hours, once a day, once a week, once a month. If the fall off is large, what causes it? Is it the entries being delayed or the stop loss amounts?

I am also curious about Universe bias (although the universe feels fair). How much optimization has gone into the selection of these ETF's? Have you run any backtests / studies on various universes that are non-correlated with one another, but tend to 'exhibit trend behaviors' to see if it's a general pattern. The 'ideal' would be to have Universe ETF's selected 'mechanically' based on the degree to which they have trended over the past X period. So, I would think about a much larger Universe of ETF's to start. I did run some tests with a lot more emerging stock market indices in the mix, and the performance was much worse (large DD since 7/2011 or so). In theory these markets should work well (China, Brazil, Russia, Taiwan, etc). So, that would be concerning.

(Some of the indices I mixed in:) 'FXI', # My addition below here
'EWJ',
'EWZ',
'EWA',
'IWM',
'IWC',
'EWW', # Mexico
'EWT', #Taiwan
'EWY', # South Korea
'RSX', #Russia

I am also curious about any stop-loss testing that you did. Your current rule is pretty cool. I don't get it 100%, but see that it's generally the slope of the regression line modified in some way by time period. How sensitive are the results to various stop loss amounts and formulas? I would likely look at a 'constraint' for the maximum stop loss. Have you looked at 'volatility based stops' based on the volatility of the position itself and/or of the overall market? I have sometimes found these effective in the past. Have you also just looked at flat 'percentage based' stop losses that are applied equally to all positions? Curious what you found.

And how sensitive is the system to slippage assumptions and or various rebalance periods?

There are some markets that I have heard CTA's say trend more and exhibit more trend like behaviors (i.e. Asian markets and energies), and it might be interesting to test on groups of just them as well - however most CTA's trade a lot of different strat's.

It will also be very cool to combine this with more position and risk management logic - i.e. a 'constraint based' risk management at the position, total asset class and 'market exposure' levels.
These could be done, for example, with an algo that matches the 'slope based' position sizing logical with some volatility (risk) constraints per position (so after the initial 'linear regression based position sizing is selected it is put into 'risk budget' analyzer and re-adjusted - so that it stays within any position size and asset class level constraints). As well as potentially constrained 'beta' for the long and short sides each week or whatever (so there is a max. long and short beta component).

Most of the pieces for building this exist on the site.

But it's gonna take me a few weeks to build it. But thanks a LOT for sharing.

If I get time (won't be till early next week), I will check and report back on some of this. I am also curious about 'general effectiveness' of the signal versus it only being effective on this basket. Anyway, those are just some initial thoughts.

Best,
Tom

I am also curious about some position size approaches that might work, such as:
a) divide the trend line slop by the Abs value of peak DD over the period to get the raw scores (so this is taken into account)
b) Creating Omega Ratio, Sortino and Information Ratio over the lookback period and then multiplying the trend line by the average of these - so that we have more risk adjusted 'trend strength.' One or several of these should work to give better position sizing results that are taking account of 'risk' not just 'slope'.
c) Calculate average trailing X period correlations of all 'longs' and all 'shorts.' Then adjust ('slope weights') * (1.5-the average pairwise correlation of the ETF with all other ETF's) / (peak DD over lookback). Something like this should work better than A above as it takes into account the recent correlation of the longs with the other longs being bought and the shorts with the other shorts.

Has anyone coded these 'basic ratios' yet? They may not actually work, as the rebalacing of the volatility may be a core piece of the returns and the system would then need vol to achieve best results. Just some ideas.

Great Job!

Thank you for sharing this interesting algo.

It seems very sensitive to various parameters:
- slope_min (BTW I don't get how you figure it corresponds to 7%)
- ETF selection: adding VEU, TLT, or SLV worsens the returns considerably

Also context.multiple does not seem to add much to the results versus context.maxlever alone and again, I don't really get what it does.
In my tests, the performance worsens considerably when I simulate at minute level suggesting maybe the algo is too quick to react?

Any thoughts?

Thank you for comments, every one. After thinking about your what you all told me, I have rewrote the script. Here you go.
Major changes;

Consider max drawdown - When entering into a position, the algo looks into max drawdown over lookback (thanks to Tom Austin). Drawdown works better than volatility because volatility could also go up when the stock is trending sharply, but not drawdowns.

Much broader universe - Departing from the original basic ETFs I chose arbitrary (thanks to Tom, Pierre, Chris), I now use top 5% trading volume universe. It is free from look ahead bias.

The result yields quite humble return for 10-year, but with impressive low drawdown, even through the financial crisis.

Hello Naoki,
What would you say are the "optimizeable" parameters in this algo? Could you possibly identify them?

Thanks!
Adam

Naoki,

Great script, elegant code. Thanks for sharing.

Does the algo only enter new positions on trend change? So if a trend is established already, it won't jump onto it until the trend changes? Just looking at this bit from the entry section, which requires a cross of price action and regression line to enter a position.

 if slope > slope_min and delta[-1] > 0 and delta[-2] < 0 and dd < context.maxdrawdown:  
                context.weights[s] = slope  
                context.drawdown[s] = slope_min  
                log.info('/     Long  a = %+.2f%% %3s - %s' %(slope*100, s.symbol, s.security_name),gain)  

Actually 'delta' is the distance between the current price and the trend line, and not the change in trend line slope.

For example (with long position) it does not enter into position when:

• Trend is established, but price keep going up above and away from trend line (too late to enter)
  
• Trend is established, the price crosses the trend line downward. (wait till it comes up)
  
• Trend is established, the price is swinging a lot and crosses the trend line multiple times (because 'dd < context.maxdrawdown' will not be FALSE)
  

Thanks for clarifying the 'delta'. What I mean, is that if the algo is started today, we willl need to wait until trend changes to enter any positions, correct?

Naoki, thanks for sharing this algo. And thank you for making it easy to understand. I have learned a lot by reading your algo, and it has given me more ideas to try. At the very least, my Python skills are slowly improving! :)

BTW, I have copied your "get_gain" function to a Github community site that I have started. I hope you don't mind!

https://github.com/tristanbob/Quantopian-Community

Naoki

I'm really impressed with this algo for larger time frames. I was curious to see how it would perform on intraday trading, using minute bars, and obviously using different values for the context variables. I figured I would have to tweak it considerably to reduce the very high friction due to bid/ask spreads, but still expected to get moderately positive gains.

Unfortunately, I seem to be doing something very wrong, as it is buying and selling way beyond the limits of the portfolio. This despite your implementation of the "order_target" as opposed to "order" criteria.

I suspect the problem lies in the loop that runs every 15 minutes shown below.

 for i in range(total_minutes):  
        # Every 15 minutes run schedule  
        if i % 15 == 0:  
          # This will start at 9:31AM and will run every 15 minutes  
          schedule_function(regression, date_rules.every_day(),  time_rules.market_open(minutes=1))  
          schedule_function(trade, date_rules.every_day(),  time_rules.market_open(minutes=1))  
          schedule_function(trail_stop, date_rules.every_day(),  time_rules.market_open(minutes=1))  
          schedule_function(execute, date_rules.every_day(),  time_rules.market_open(minutes=1))  
    ```  
I will struggle with this further, but perhaps you are curious yourself and / or have already successfully dealt with this issue.

By the way, as you can see in the code, I intended to introduce a "limit" order as I have had bad experiences in day trading when dealing with market orders. I had to comment that part of code out because my limits were getting set to 0.00, which resulted in no orders ever getting placed. It's probably fairly trivial to decode that , so I'll get to that later.

I look forward to any hints/suggestions you may have.  

You might have already figured it out, but you forgot to add the 'i' here.

          schedule_function(regression, date_rules.every_day(),  time_rules.market_open(minutes=i+1))  
          schedule_function(trade, date_rules.every_day(),  time_rules.market_open(minutes=i+1))  
          schedule_function(trail_stop, date_rules.every_day(),  time_rules.market_open(minutes=i+1))  
          schedule_function(execute, date_rules.every_day(),  time_rules.market_open(minutes=i+1))  

Noticed something that makes an opportunity for a sidenote tip: You can see in the code that the metrics, like those above the chart in the IDE (or full backtest too) are copyable, good for everyone to know. Now, taking that another step, can copy/paste the metrics and also apply a macro to them in an editor for like:
tr121.3% br118.8% a0.05 b0.29 sh0.72 s0.88 ir0.02 v0.10 d16.7%

Naoki,

Good catch. That did the trick. Of course, as I suspected it does not make money on intraday, but I want to isolate this further, because if you
eliminate all friction and trading costs, in theory it should be profitable, yet it is not. I'll play around with my variable settings and let you know how I fare.

Thanks again.

Serge

Thanks for sharing, Naoki.

What's the motivation of setting the weight to be slope(ontext.weights[s] = slope)? is it a magic number?

Hello and first of all Thanks for Sharing!

I am trying to analize the algo and maybe this can help someone, i wanted to visualize the value used as a momentum indicator ( slope )
So this is my first question: why you consider a/b and not just the slope a?

(just SPY in this first notebok )

In this algo I used a/b to be the slope to normalize the price difference.

Stock X: If it is at $100 at day 0, went up average of 1% per day. a will be 1.0 and b will be 100. a/b will be 0.01.
Stock Y: If it is at $10 at day 0, went up average of 2% per day. a will be 0.2 and b will be 10. a/b will be 0.02.

From the slope = a/b, you can tell Stock Y is trending up twice more than Stock X. The algo then put more weight on Stock Y than Stock X by applying

context.weights[s] = slope  

This example is perfectly clear thank you very much -Giuseppe

well-written and brilliant algo! Thanks for sharing!

@Naoki - Great and simple code! Thanks for sharing! Have you tried trading it live?

@Naoki - I really enjoy your algo and the backtests look very promising. I am going to put it for live trading soon. However, my paper trading result shows it requires at least two months to "warm up" before it starts to place the first trade.
Do you have any suggestion how to reduce this period for this algo?
Thanks.

A way to "warm it up" beforehand is to run the backtest until the day before you run the live trade (say, Friday evening). Using the print function, write the current holding and the parameters (in this case, context.weights and context.drawdown.) Then, hard code the initial holdings in the 'initialize function.' (it will start off on Monday morning). You can check the current date and only output the holding on the last day.

hey all. does this algo work on Quantopian 2?

Very good algo! Thanks for sharing.

@Tybalt: If you clone the backtest at the top of this thread, you will get the Quantopian 2 version.

thank you sir

the algorithm migrated to Quantopian 2 version, have the obviously difference performance compare to original version!
Could give the advice to check the reason?

Big kudos to Nagai-san. indeed very good investment thesis behind the algo.
3 source of profit: asset alloc, stock selection and market timing. Nagai-san's v1 and v2 touched 2 of the 3. Thank you.

Yes, When back testing in Quantopian 2 The algo does not go short during the 2008 Lehman bankruptcy it just takes no positions.. Why is this different?

Hi Naoki,

I cloned your algo and start the backtesting. I haven't changed any code. But the "Total Return" is very terrible. :-/

It seems to be quite hard to write code that works in today's market. Perhaps all the quants have canceled all the information out of it?

Me too, the result is totally different now... someone may to explane why?
The code is Very well done for learning purpose for newbee as me.. thanks

It looks like the original algo was migrated to 2.0 incorrectly. See attached the up-to-date return of the original algo. Still need adjust the code to remove the depreciated functions.

Right!
Then, this is my small contribution to the Q community: removed all deprecated code.
Any ideas how to improve it?

Thanks Michele for fixing the code. I think that another great improvement can be the integration of the volatility based position sizing, for example as described in Robert Carver's book Systematic Trading (http://qoppac.blogspot.ca/p/systematic-trading-book.html).
Simon Thornington tried to implement this logic in his algo here https://www.quantopian.com/posts/exponentially-weighted-moving-average-and-standard-deviation-in-pipeline
Unfortunately, his algo didn't work as intended. Maybe someone can try implementing this logic to adjust position sizing of the algo?

The code doesn't look fixed at all. The new performance does not match the original performance nor the adjusted performance after Naoki modified to avoid look-ahead bias.

A.W. , please note that the diffrence in the performance may come from the different initial capital $.
The most recent version of the algo run with $10K of the initial capital vs. $100K in the original algo

Here is my attempt to plug the position volatilty adjustment (credit to Simon's algo here https://www.quantopian.com/posts/exponentially-weighted-moving-average-and-standard-deviation-in-pipeline)
The algo adjusts the leverage to keep the same volatility. In some cases the leverage can go up to 3x+, so might not be appropriate for many retail brokerage accounts, but it achieves 2x performance of the previous algo with very similar drawdowns

unbelievable stuff!

Just read the code. But the point is, the context.price is defined as none at first, so how to compare it with 0? I mean, if context.stopprice[s] < 0:, how does it work?

With a request of being pardoned for ignorance of python and statistics I have a silly question. I would appreciate an answer. If y=ax+b is the equation then 'a' is the slope and 'b' is the intercept. In the code segment I don't understand the part "slope = a / b * 252.0 # Daily return regression * 1 year"
+++++++++++++++++++++++++++++++++++++++++++++++

Run regression y = ax + b

    results = sm.OLS(Y,A).fit()  
    (b, a) =results.params  

    # Normalized slope  
    slope = a / b * 252.0        # Daily return regression * 1 year  

++++++++++++++++++++++++++++++++++++++++++++++++++++++

If 'a' is the slope what does a/b give, if a is not the slope what does the ols function return ?

Thanks!

Just ran a back test on this, is there a Robin Hood friendly version of this algo? this pumps through a ton of trades, you would very quickly be labeled a day trader on Robin Hood.

Let me know!

This algo was originally written with ETFs to be exposed to diversified asset classes, but now you can this on futures. Here is an example.

https://www.quantopian.com/posts/futures-trend-reversion-algo

That's awesome that you already have a futures version of the algorithm, Naoki. Thanks for posting it!

And....POOF....just like that, this algo went from looking GOOD....to not keeping pace with the benchmark.

Tried to run previously cloned algo TODAY and it gave me an error message on line 149. I thought OK, maybe I had change something, so i cloned it just now and a pop-up box came up and said the ALGO had been migrated to Q2......which seemed like a good idea, until I ran the algo....indeed the logic of the ALGO was still there, but the back test results are 'spectacularly' LOWER.....lower even than the benchmark.

I read the Q2 info and there could be a number of things that caused this, but in general, I am quite discouraged that even Q2 will not give accurate results as what may 'actually' occur in real time trading ?!

Hello Blaine,

We're constantly improving the accuracy of our backtesting system, one of the ways we do that is open sourcing our engine (amongst many other repos). https://github.com/quantopian/zipline

Algorithms are complex systems with serial state-dependent behavior, small changes can throw off algorithm. That said, good algorithms are generally more robust to small changes, and algorithms being massively different after a small change can imply (amongst other things) that
A: The algorithm was overfit and oversensitive to small parameter changes.
B: There was a bug in the backtesting system that was falsely causing the algorithm to look good.

This is more common than you might think. Overfitting is incredibly common and nobody is immune. As Marcos Lopez de Prado pointed out in his recent QuantCon keynote, backtest overfitting is likely the single biggest problem in finance right now (https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2308659). Part of the reason this occurs is that if you find an anomaly, you'll likely try to dial into it and take more advantage of that behavior in your data. A data bug or quirk of the data can look no different from a true returns inefficiency. We've also released some research about how common overfitting is here https://blog.quantopian.com/using-machine-learning-to-predict-out-of-sample-performance-of-trading-algorithms/.

Sometimes a small change to the algo is all that's needed to get it back on track after a code change like this. Sometimes what our improved backtest system revealed is that the algo was never going to work in the first place. Backtesting is always at best an estimate of what will happen, and that's why it's important to use many other techniques to determine whether your algorithm will be okay, and to determine where the failure point actually is. I discuss this more here: https://blog.quantopian.com/idea-algorithm/ In my opinion, backtesting should just be used to check if the algorithm survives real market conditions like slippage and trading costs. You should know your model is predictive before backtesting. That's what alphalens is for.

If you believe you've discovered a bug or inaccuracy, please let us know more about what's going on at [email protected]. It's incredibly helpful to us and everybody who uses Quantopian/Zipline to receive feedback that will make our systems better.

Lastly, for Quantopian's allocation purposes you shouldn't worry about beating the market. It's not a good benchmark unless that's precisely your target. What's interesting to us is algorithms with alpha -- returns which are not correlated with other risk factors or strategies. Even with low returns, an uncorrelated algorithm can be combined with other strategies or levered to the point of usefulness. In general if the strategy makes money and has well controlled risk exposures, it doesn't matter whether or not it beats the market. It's cheap to buy market exposure through ETFs, so the goal of a single strategy is to be an diversified investment.

Everyone has probably long moved on from the original code, but like Pierre Warnier, I'm having a lot of trouble understanding how a slope_min of 0.252 corresponds to yearly growth of 7%.

Slope = a/b * 252
Assume slope_min = 0.252
0.252 = a/b * 252
0.001 = a/b
Using the example from Naoki Nagai,
"Stock X: If it is at $100 at day 0, went up average of 1% per day. a will be 1.0 and b will be 100. a/b will be 0.01." Obviously in this case, a/b of 0.01 is greater than 0.001, so this 1% per day increase stock qualifies.
Now consider a hypothetical Stock Y: If it is at $100 at day 0, and went up 10% in a year (0.0378% per day average as this gives a 10% increase over 1 year). a will be 0.0378 and b will be 100. a/b will be 0.000378 which will be significantly less than 0.001 which means that it doesn't qualify as exceeding the slope_min, even though it rises 10% in a year which is over 7%.
Apologies for any confusion, but I still can't figure out how 0.252 slope_min corresponds to 7% increase per year.
Thanks.

It is interesting to read a thread on trend following, bearing in mind what has been happening to trend following CTAs over the last few years. Who knows why but a strategy which made hundreds of millions for the likes of JW Henry and Bill Dunn suddenly stopped working and may or may not "come back". At least so far as a diversified basket of futures is concerned.

Broadly speaking my belief is that successful "probability" and pattern trading requires the participant to be in the right place at the right time. It is about luck rather than skill; or at least given enough skill not to make a complete fool of yourself you will still get nowhere without luck.

It may be that trend following comes back into profit. It may be that it is too overcrowded.

Ironically of course all investment relies on trends. If a market does not trend upwards over time you will make no profit from it.

The question therefore becomes "to trade or not to trade".

If world economic growth continues over the next hundred years you can expect to profit from a simple buy and hold of an index. Preferably a world index and preferably an index which is leaning more towards equal weight that market cap.

The trouble is with this industry is that the vast majority of participants are wrongly diagnosing luck for skill.

Other than that the major problem is the incurable disease of overfitting a system to the data.

What would it take to get someone to explain what this algorithm is doing?

I can't understand any of it...

There is significant documentation in the source code.......anything in quotes " " is his documentation as to what each line of code or function is doing. I'm no python expert, but the logic seems straight forward if you read his documentation.

Also...if you click on Naokis name you and look through his previous posts there is quite a large amount of discussion on the algo as well.

Right off, I do not particularly like this trading strategy. Enough trades, but too few securities traded. It takes some 2 hours to run. However, money is money and in the CAGR department, it kept its own.

Based on its tearsheet we can notice the strategy has a relatively steady equity curve, but it has started to break down.

All I did was raise the initial capital to $1M, nothing else. Did not even try higher due to the low number of securities. See attached tearsheet.

Nonetheless, the securities traded are highly liquid ETFs, stuff like DIA and QQQ which are tracking indexes and thereby offer data averaging, built-in smoothing, and less volatility.

This second tearsheet increased the trading interval to 10.25 years (adding 14 months). You increase the duration to see how a trading strategy might behave under a different set of circumstances. It is like having a walk-forward for a strategy since it was showcased in October 2016. Making this test showing what would have happened during the added 14 months.

Based on the tearsheet, it did pretty good. Even raising the CAGR to 28.7%.

It is not what I expected. However, it does demonstrate that throwing away a strategy because it seems to be breaking down as in the previous post might not always be the best solution.

Nonetheless, I still do not like the strategy. However, its trading behavior just became more interesting. It showed sustainable alpha and that is worth investigating. Maybe some of the procedures could be used elsewhere. Still, it is a 28.7% CAGR over its 10.25-year simulation.

I had to test adding more capital to this strategy.

Handling $1M might not be enough for the trouble of monitoring the thing for 10 years. Also, the test would show scaling abilities and how trading units would be handled.

Raised the capital to $10M. No other change. If the trading strategy is scalable, the outcome should be in the vicinity of 10 times the previous test at $1M. The attached tearsheet attest to that.

I still have not looked at the code as yet. However, for the preliminary tests, it did not matter. What I wanted to know was: is it worth it to investigate? The answer continues to be yes. Now I know it can handle the pressure and the traded securities are more than sufficiently liquid to accommodate.

The tests of this strategy, up to now, have shown: it can handle more capital, it can handle more time, it can scale up, and it was relatively easy to find something in the vicinity of a 30% CAGR. In fact, it is the first strategy I looked at after my post to that effect in the thread:
https://www.quantopian.com/posts/the-gaming-of-stock-trading-strategies

I still do not like how it behaves, it is therefore up to me to see if I can make this strategy behave more in the way I would like and determine if it becomes more acceptable.

Nonetheless, the strategy still grew at a 28.2% CAGR over those 10.25 years, turning the initial $10M into $91M. It used leverage to do this which it could afford to pay.

What would it take to get this explained in plain language?

The code is too complicated. I can't understand what it's supposed to do.

Could this be explained in "trading rules" - as if a person were going to do it manually?

John.....the 'plain language' as mentioned earlier is contained within the quotes of the code itself. All the 'trading rules' are there.

Guy....thank you for your wonderful work on this, it has inspired me to expand my mind a bit further with Python and Quantopian.

One thing that greatly concerns me however, is the test period....a 10 year lookback really only covers what I would personally consider an artificial market place where market has had the luxury of unprecedented liquidity and FED intervention...

@Blaine, yes. Note that during the same period the majority of portfolio managers nevertheless did not outperform the market averages.

However, your concerns could be programmed in. Meaning you identify what you consider weaknesses and help the strategy navigate its maze to even better results in spite of what the FED might do or not do.

We are really too small to have an impact on what the market does. But, we are in control of what we allow our strategies to do.

The objective of any trading strategy is to have its portfolio grow in size with time. As a matter of fact, it should do this at a compounding rate. The task gets harder the larger the portfolio gets. It is here that the law of diminishing return enters the picture.

We should all ask the same question: can my strategy handle $1M, $10M, $50M, $100M or more? The answer can be had without even changing a line of code. It is sufficient to set the initially allocated capital and run the simulation.

The outcome will show how the strategy behaved and how it handled the added capital resources. For a strategy that is scalable, the outcome is simply scaled up by about the same amount or close to it. You go from $1M to $10M, then the payoff should be near 10 times. That is if the strategy is scalable. Any trading strategy that uses fixed-fractions as trading unit becomes scalable by design: \(u_t = F(t) / j_t\) where \(j_t\) is the number of securities traded at time \(t\).

Therefore, it should be expected that raising the stakes by a factor of 10 in this trading strategy should result in a corresponding increase in portfolio value. In the attached tearsheet, we can see the law of diminishing return taking its toll. There is still how the trading strategy handled its trades that should be analyzed. Nonetheless, the strategy maintained a 25.5% CAGR over those 10.25 years which translated into $693M in profits. Not so bad.

Presently, one Q member has a $50M allocation. And as a result for any trading strategies we may develop, we should also simulate if our strategies could handle it or not. And that is a very easy test to do! It is part of doing our homework and being able to say: it does.

Any change you make to a trading strategy should have some impact on its payoff matrix. Otherwise, the changes in question should be considered simply as cosmetic in nature. You can always do that kind of code. It is just that it does not produce any money. And that is not the object of the game.

I started reading the code and making modifications. Things like: if I increase this number the impact will be to increase volatility but also increase profits. If I insert a little delayed gratification, it should increase profits. If I use a little bit more cash, again it should increase profits by taking more trading opportunities. This should be gradual, spread out over the entire trading interval and still have a positive impact on the strategy's payoff matrix equation.

$$ F(t) = F_0 \cdot (1 + \bar g)^t = F_0 + \sum (H \cdot \Delta P) - \sum (Exp) = F_0 + n \cdot \bar x $$

The thing to do to demonstrate that it does, in fact, do as intended is to run the simulation, knowing beforehand that changing those numbers would have a direct impact on the portfolio's payoff matrix.

The attached tearsheet shows the results. The CAGR came in at 34.7%, meaning that the strategy transformed its $10M into $153M in 10.25 years. Admittedly, it continued to use some leverage to do so, just as in its prior $10M simulation which came in at a 28.2% CAGR.

I have not read all the logic of this program as yet (am at line 37). The strategy is getting more aggressive, trading more and raising its average net profit per trade \(\bar x\) as expected. Evidently, there is a cost to this. It translates into a higher volatility reading and a larger drawdown. I do not mind about that presently since I have not addressed that issue as yet. I do like the strategy's 0.17 beta reading.

The changes made have nothing to do with the logic of the trading strategy and on how it mixed its stuff to trigger its trades. I only “forced” the strategy to reconsider its assumptions. Also, it does make the point that the assumptions we give a trading strategy can have quite an impact on its outcome.

This trading strategy started as a throwaway strategy playing the largest EFTs out there, and it is keeping me interested.

Thanks Guy....its very insightful to see how one approaches these things from the deep thought perspective. I'm glad its keeping you interested because its immensely helpful to watch this evolve with detailed analysis.

Changed the timing of the trades in order to corroborate the point made in my last post in Portfolio Structure and Over-fitting.

Trades were given a different time of day to execute which should result in a different answer. Also, requested more profits as illustrated in the attached tearsheet.

A summary of the impact:

These minor changes raised the portfolio's matrix payout from $153.1M to $175.8M. Raised its CAGR from 34.7% to 36.5%, a 1.8% increase in its alpha. Not much, but still an added $22.7M for a few numbers.

To do so, the strategy increased its number of trades from 4,440 to 4,768 while also raising its average net profit per trade from $34,491 to $36.871. I still do not know how it trades, but I will get to that later. The alpha is compounded.

In the beginning iterations of this strategy, it was making peanuts on its short trades. That has increased considerably. The strategy now takes on 1,715 shorts which are mostly profitable. The strange thing is that this strategy is supposed to be long only. Therefore, when I'll get to the code, I will have to find out why and then determine if it is a desirable “feature”.

You have a trading strategy but know nothing as to its limits or its architecture. How far can it go? Would it be acceptable trading in this fashion?

Again, I only increased what could have an impact on the strategy's payoff matrix. When compared with the immediate prior simulation tearsheet we have (n) the number of trades increased from 4,768 to 4,872 while the average net profit per trade when from $36,871 to $45,948. The strategy's CAGR over the 10.25 years when from 36.5% to 39.9%.

All this translated into generating $223,860,013 in profits compared to $175,803,112 in the previous simulation. A 48M increase just for a few numbers. A 27% increase in net profits.

What is seen in this series of simulations is that I affected all the trades in each scenario, not just one here or there, but all of them by influencing and coercing the payoff matrix. It is like adding a function to the payoff matrix: \( \sum (f(t) \cdot H \cdot \Delta P) \). This function controls the strategy's restrictions.

Things I often see in trading strategies is conflicting and redundant procedures. Like having two profit target functions, one saying take the profit at 10% and the other at 20%. Surely, everyone would prefer the 20% profit target, but it is never executed since the other one has precedence. However, you can still add to the 10% profit target and push it higher, whereas even if you increase on the 20% mark nothing happens. If you push your 10% profit target above the 20% one then it becomes redundant since the other 20% will take over control.

Very little was changed in these simulations. None of the trading procedures or program logic was changed, and yet, performance increased considerably. Am I over-fitting or simply exploring the architectural limits of this program?

Hi Guy,

I would suspect that you are overfitting somewhat. Though you might still obtain a third of the CAGR you are getting on your backtest, and that would not be too bad!

I also significantly improved the algo returns by adding a few more trending ETFs, and changing the slope entry threshold to become asymmetric for longs and for shorts. I'm getting a CAGR of 26% with gross leverage never going above 2 with the initial lookback period of 126 days.

However, my main worry with this algorithm is that if you change the lookback period for the regression line from 126 days to 120 or 130, you get a very different performance (with the algo going nowhere for much longer periods - losing money for 18+ months stretches). While maybe 126 days is a magic number as it only captures 2 earnings calls at nearly all times, I'm always sceptical when a 5% move in one parameter can create such different results.

It was an interesting algo to study. I thought studying the effect of the volatility adjustment was particularly interesting (using pd.ewmstd() which I believe was added by maxim). I did not know this function beforehand and will likely use it in future algos...

@Mattias, interesting observations.

Over-fitting, in this case, might be a strong word. The program's code was not altered, nor its trading logic.

The intention was to appraise the nature of that at a later time. Once I've modified the code, maybe I'll encounter something that says it is a “misfit” for the task required. For now, I do not know if it is good or bad.

I did my usual initial exploration phase of someone else's program where I apply functions to the strategy's payoff matrix to see if it is scalable and sustainable. Most often, if a strategy cannot scale up and last for 10 years and more, I rapidly lose interest.

What I did was relatively simple, see it as a guiding function to the inventory matrix: \( \sum (k(t) \cdot H \cdot \Delta P) \) resulting in a positive impact on the outcome. Such modifications are available to anyone. It is where you preset part of your trading strategy's behavior.

I have not touched the code yet. Nonetheless, I do not expect the future to be like the past, and therefore, the strategy still has to prove itself going forward. At least, it showed that “under its set of conditions and trading procedures” it managed to outperform over those 10.25 years and doing over 4,000+ trades in the process. That is not negligible.

Will those “conditions” be the same going forward? As you know, probably not. However, without viewing the relevance of the code, I would be hard-pressed to answer. But, I do expect to see the strategy follow its code and behave in about the same manner as it did in the past. And that is generate about the same number of trades per time interval. This will tend to a constant as in: \(\Delta (n \cdot \bar {x}) / \Delta t \to constant \). And because of this, we will see a return degradation going forward. Measures will be needed to overcome this inherent alpha decay. But that is part of the code modification phase.

This exercise makes you wonder on the value of what is being missed by not exploring the limits of a trading strategy and what these controls \( k(t) \) might really be worth?

This could be viewed as opportunity costs for not having pursued those limits, barriers, or what have you. And, it could be expressed as having value, explicitly:

$$ \sum (k(t) \cdot H \cdot \Delta P) - \sum (H \cdot \Delta P) $$

You could take what you or I showed as performance improvements and compare that to the original script's payoff matrix to get an idea of these opportunity costs. This, just for a few numbers in a program that we did not even design ourselves.

Thanks to @Maxim for updating the strategy, @Michelle for removing the deprecated stuff, and @Naoki for originally putting it out for all to play with to explore its possibilities. There is more that can be done, but that will require reading and understanding the code, the core of the program which could lead to something “over-fitted”, and maybe not.

I find that term totally over-rated. You know the future will be different, and that, de facto, your trading strategy will process that data differently, and yet, people insist that their trading strategy, which will stay the same, somehow will behave differently in spite of knowing that the data will change.

If I had to use such a term, I would say that the original trading strategy was mostly under-fitted since so much more could have been extracted with relative ease, and by modifying just a few numbers at that.

The last few years should have shown us that trend following is a myth, generally speaking. In terms of commodities, analysis shows very little by way of long term trends over the past 150 years. Demand has largely been satisfied by increased supply in the commodities market through increasing productivity and better extraction methods.

In stocks it is true that the long term trend has been upwards. However even this rise has not been particularly spectacular when inflation is properly accounted for and you assume reinvestment of dividends. Tax free dividend investment is not a given unless you are investing for a fund which is not taxable

Regrettably, most if not all of the topics on this forum are so much time wasted. In the long term you would do as well flipping a coin and would be better off simply by diversifying widely and indexing your money.

Unless that is you have a method which is not dependent on prediction. Reaping the bid offered spread is one such, or collecting the premium on shorting volatility while protecting yourself from the downside.

Most of these schemes are merely pie in the sky and doomed to ignominious failure.

Zenothestoic makes a valid point......

Not to get offtopic, but just an honest question....
Have you guru strategy builders looked over at Quantiacs to see 'whats working' as it relates to their systems. It appears that many of the top performing systems (see link below) are likely 'sharpe ratio maximization' strategies which end up with quite a bit of commodity offset. Whats interesting is that I've seen a number of sharpe ratio maximization strategies on Quantopian but not many that are successful.....on the other hand, the ones over at Quantiacs appear to have astounding sharpe ratios, performance and minimal drawdowns over long periods. I guess the question is....what are they doing (would have to be a guess since nothing is public) that the wizards here on Quantopian are not to get such strong results ?

https://www.quantiacs.com/Systems.aspx

what are they doing [...] that the wizards here on Quantopian are not to get such strong results ?

While the answer may sound complex, hopefully it will be clear. A major part is simply the amount of money. Over there, $250k. Here, at $10M, with the modeling percentage that can be traded there is price drift in partial fills so a large number of stocks are required, and also with concentration limits for the contest, that means venturing into weaker alpha signals. Additionally leverage EOD must always be near 1, beta needs to be as close to zero as possible for a higher score, and then sector, turnover, risk & other constraints all contribute to the difference. At $250k here too, similar results with sharpe over 4 and so on would be seen, or even with $1M, and even without stop/limit which we aren't allowed to use in the contest.

Thank you Naoki.

Note that there's instances where the drawdown calculated for the stop-loss will exceed the drawdown at the open by a small margin, namely due to the fact that the drawdown at entry is done based on the open, whereas for the stop-loss we use the close. This can lead to early exits of positions on smaller timeframes / ranges.

Also, I added a take-profit function based on your 1.95*std that runs following any exit-long/short functions. Again, this is mostly due to the fact that in the crypto market pump and dumps happens frequently and you might be left out had you only taken a stance on the next candle's close.

Thank you very much Naoki.. This is a great start for newbie like me.