I welcome feedback (will post tear sheet next).
First off, there are a number of things I don't understand about how Q is approach "risk management" but I gather that it is a standard function in hedge fund construction and management, and this is their version of it. It seems that "risk" is anything that is not wanted in a particular fund in excess. The Q risk model somehow flows out of how the 1337 Street Fund is being constructed and managed, as a cobbled together fund of funds (licensed user algos). I guess the whole system is easier to manage, including compensating individual authors, if risk is managed at the algo level. Then licensed algos can be treated as a clean set of unexplained alphas, as input to the framework (and the more the 1337 Street Fund can be constructed of "new alpha" the more valuable it will be as a financial product). So, I guess the idea here is that authors need to deliver pure new alpha (as defined by the Q risk model). In the lingo of ML, authors will be paid for delivering new features (I have to think that ML will eventually play a role in the 1337 Street Fund construction, if it isn't already).
@ Karl - completely removing the Pipeline volatility factor reduces the SR from ~1 to ~0.8, over a longer backtest (back to 1/1/2010). Looking at the summary statistics, I can't tell if the algo is "good enough" or if I need to do more. It seems like the Specific stats are already pretty high, relative to the Common ones, but maybe there are other problems (the biggest, perhaps, is that I have no Strategic Intent explanation for the source of the specific returns, so the whole thing may be hopeless, until I'm a quant expert and know what I'm doing). I guess I could fiddle around trying to get the Common Returns curve to center on zero returns, but it might just be another form of over-fitting, I'm thinking, and not move the algo closer to being allocate-able.
Summary Statistics
Annualized Specific Return 4.83%
Annualized Common Return -0.59%
Annualized Total Return 4.22%
Specific Sharpe Ratio 2.25
@ Karl -
A brief note that there is a lot of commonality between this risk model/factor attribution jazz and the response surface methodology (dating back to 1951, by the way--nothing new under the sun). One thing I haven't quite sorted out is the potential role for so-called "interaction terms" in the context of Q alpha factor combination and the risk model (and risk mitigation techniques). If you and anyone has insight, I'd be interested. In the end, I think the interaction terms will need to be included (computationally, this is no big deal, if I'm thinking about it correctly).
Grant
@Grant The factor attribution tearsheet looks great. One thing I would try to improve is the 47.6% daily turnover rate. It's probably a little high for a 300 positions portfolio. Thanks for sharing the actual code!
@ Charles -
Thanks. No problem sharing the code--it is more of a template to improve and dink around with. I don't think there is anything magical in there.
Not sure what it all means, but my guess is that it is not suitable for the Q fund. Reading tea leaves to me, without a frame of reference.
That first version you posted, Grant, was pretty interesting. It has a persistent short-term mean reversal exposure, and also has a persistent underexposure to volatility (the portfolio tends to prefer low-volatility stocks). On one hand, the exposures aren't really that high, .2-.3 or so. On the other hand, does further analysis explain why those risk exposures are present? Is there a way to keep your overall returns while managing out that that risk exposure? (We are getting close to releasing a way for you to do more management in your algorithm code - it will soon be easier to answer the question).
The later version you posted had higher exposure to those style factors, more like .6-.7. That is a step backwards.
Another over-arching concern: do you have a way of keeping yourself from overfitting? I'd consider only running your tests over part of the history, and keep the last year or so hidden from yourself. Once you get close to a final product, then run it over a longer time frame, and see if you managed to avoid the overfitting trap. As it is, that first algo is interesting, but we won't know how interesting until many months have passed.
Disclaimer
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.
Here's another one for kicking the tires of the risk model. In this case I multiply the mean reversion and volatility factor together and end up with negligible risk factor exposure (I think...)
Will post sheets next.
Hi Dan,
Thanks for the feedback. Feel free to put a reminder on your calendar to check the backtest above (Backtest ID: 5a0f2f288b13eb440b93a10a) in six months and post the allocation suitability analysis here. I'd be curious how things play out. Since it is in the public domain, I'm guessing it won't be allocatable, but it would be an interesting exercise, nonetheless.
From first algo. Where are the 0.0033 weights decided? Bringing them down, will be fewer partial fills also. I typically look for rules that may reduce the magnitude of those at the bottom, PnL, second column here.
2017-11-16 13:00 _ti:1154 INFO 726 trading days 2 hr 33.3 min 2017-11-27 07:12 US/Pacific
Portfolio: 11265327 2015-01-02 to 2017-11-16 Alpha: 0.05
Initial Cash: 10000000 Buys: 715529 (2952944 partial) Beta: -0.06
Unused Cash: 9452437 Sells: 613683 (2886456 partial) Sharpe: 1.37
Max Margin: 0 Commissions: 69656 Drawdown: 2.88
Max Risk: 5926339 (59%) Shares Now: 211077 Stability: 0.295
Cash Profit: 1273782 Shares Val: -8455 Sortino: 2.18
Total Profit: 1265327 Cash Now: 11273782 Shrt/Lng Now: 1.00
Q Return: 12.65% Profit/Init Max Lvrg: 1.04 2015-10-05 Max Shorts: -5926339
PvR Return: 21.35% Profit/Risk PvR %/day: 0.03 Max Longs: 6020397
2017-11-16 13:00 ti:1326 INFO .
2017-11-16 13:00 _ti:1147 INFO Sort column 1 Positions: 302 of 591 traded
Profit Max Return Return Buy Price Buy|Sell Max Shares Shares
Symbol PnL Risked % %/day Hold Init|Now Orders Shrt|Lng Now %
CVE 48654 86283 56.4 0.12 -0.5 20|10 171|169 -3651|3746 0 0
CNHI 38067 76956 49.5 0.15 0.5 8|13 123|128 -6038|6139 3011 0.0034
TXN 36133 73574 49.1 0.11 0.8 54|98 162|152 -720|777 0 0
VRX 35488 72400 49.0 0.16 -0.9 144|14 107|106 -1270|1132 0 0
NWL 32520 69751 46.6 0.12 -0.3 42|28 157|144 -1259|1364 0 0
UHS 31091 68168 45.6 0.13 -0.1 112|98 134|125 -338|359 0 0
STJ 31033 65442 47.4 0.14 -1.0 65|0 101|99 -672|709 0 0
2017-11-16 13:00 _ti:1147 INFO _
APA 30863 73196 42.2 0.07 -0.3 62|40 218|215 -1070|967 922 0.0033
IMO 29643 67994 43.6 0.08 -0.3 43|31 233|202 -1338|1356 1208 0.0033
STX 28686 65797 43.6 0.12 -0.4 67|39 131|119 -1238|940 0 0
PXD 27995 71234 39.3 0.06 0.0 148|151 216|192 -331|336 250 0.0034
NOV 27799 64875 42.8 0.07 -0.5 63|32 208|197 -1334|1349 0 0
... 579 midrange rows omitted, x_high_low is 12 ...
FCX -25188 70477 -35.7 -0.06 -0.4 23|14 208|209 -4964|3858 2763 0.0033
ATVI -27336 60768 -45.0 -0.08 2.2 20|64 192|142 -1769|1786 -593 -0.0034
HPQ -29768 71009 -41.9 -0.08 -0.5 40|22 209|170 -3826|3739 0 0
AMZN -30596 52551 -58.2 -0.09 2.6 314|1137 148|109 -114|89 -33 -0.0033
2017-11-16 13:00 _ti:1154 INFO _
ADSK -32982 70401 -46.8 -0.1 1.1 60|127 216|165 -843|827 -301 -0.0034
FB -33275 60374 -55.1 -0.08 1.3 79|180 126|90 -444|264 -211 -0.0034
INCY -34622 78214 -44.3 -0.06 0.4 74|106 239|198 -641|585 -352 -0.0033
ABX -36539 82460 -44.3 -0.08 0.3 10|14 183|162 -4486|3627 -2711 -0.0033
PYPL -38863 63350 -61.3 -0.2 0.9 41|78 102|71 -1022|1052 -496 -0.0034
WDC -45740 85111 -53.7 -0.09 -0.2 111|91 211|174 -919|919 -415 -0.0034
NFLX -59231 89249 -66.4 -0.09 -0.4 348|196 190|137 -429|387 -191 -0.0033
NVDA -78045 57419 -136.0 -0.23 9.8 20|212 209|140 -1705|1730 -176 -0.0033
Info is a from a tool I wrote. If you're the hero with means who will make it open source to the community, ping.
Blue Seahawk -
Thanks. The .0033 comes from:
MAX_LONG_POSITION_SIZE = 2*1.0/NUM_TOTAL_POSITIONS
MAX_SHORT_POSITION_SIZE = MAX_LONG_POSITION_SIZE
NUM_TOTAL_POSITIONS = 600
You can clone the algo and play around with it. Either adjust NUM_TOTAL_POSITIONS or the multiplier (presently set to 2) or both.
I didn't have time to read past a few posts up, but just wanted to say that multiplying factors falls into the category of 'non-linear' combinations and is super interesting. Obviously you want to have some economic rationale when you do this for real (I get this is an example to kick the tires), but wanted to note that using non-linear combinations of factors is a real thing. What people will do sometimes is look at 'corners' in factor matrices. For instance if you have factors A and B, you can look at stocks where A is high and B is low, and the 3 other corners. There can be interesting anomalies hidden there from what pro quants have told me.
Disclaimer
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.
Thanks Delaney -
Well, it is just the idea that if one has multiple factors, a weighted sum of factors is just the simplest case of expressing the alpha as a multivariate polynomial of factors. Including the lowest-order interaction terms, the form is:
alpha = b0 + b1*alpha1 + b2*alpha2 + b12*alpha1*alpha2
This can be generalized to any number of factors.
Regarding the economic rationale jazz, it probably makes sense that the individual factors have some basis, but requiring a story for their interactions may be a stretch.
One thought is to determine the statistical significance of the b-coefficients on a rolling basis (akin to how one would approach the problem in design of experiments/response surface methodology).
About the numbers above: I had added my code with a paste to the cloned (first) algo, ran it, and at the end of the 3 hour backtest I copied the last output and pasted it above. Can be set to produce that output at any interval or condition for watching progress or examining downturns etc. The second column, PnL is all-time, not just the current position. A few unauthorized tea leaf readings: If you take a look at the last line, NVDA, you'll see your algorithm has a position that is currently short and an outlier in Return % at -136, and lowest in PnL. It's price went from 20 to 212 since first seen but they were both long and short.
Generally eleven of the twelve worst losers shown are still being held, that's unusual. Changes to factors can have big effects on profiles like that. I've also had some benefit from gathering current PnL's and dropping the lowest periodically, can use idxmin() on a Series as one way. Can also add a stock to a list fed to Optimze CannotHold() as another. Could even try a flip of long, short. The 12 highs and 12 lows shown could be changed, doubling it or something to find out if the number of losing stocks still being held might be even more dramatic than just those 11. Partials: Approaching 3 million partial buys and partial sells for a backtest of this length looks high to me. Couple of things that I find beneficial, a mask for top minimum volumes for some window length and/or canceling orders after a certain amount of time or after a certain number of partial fills (can treat positive and negative PnL stocks differently).
If this wraps you could copy to an editor. It's fine in the backtester.
A few partial fills:
Datetime Trading Minute Action Filled|Prv|Order Sym Prc Shrs_Now Stop|Limit PnL Lvrg oId Cash
2016-07-05 07:00 _t:620 INFO 30 Buy 1560 NVDA 46.61 -774 -22 0.99 d042 10946288
2016-07-05 07:01 _t:620 INFO 31 Bot 689|0|1560 NVDA 46.59 -85 -1 1.01 d042 10974839
2016-07-05 07:02 _t:620 INFO 32 Bot 401|689|1560 NVDA 46.63 316 -1 1.01 d042 10897848
2016-07-05 07:03 _t:620 INFO 33 Bot all 470|1090|1560 NVDA 46.61 786 -9 1.02 d042 10899336
2016-07-06 07:00 _t:620 INFO 30 Sell -1560 NVDA 47.28 786 518 1.00 1d13 10960303
2016-07-06 07:01 _t:620 INFO 31 Sold -1180|0|-1560 NVDA 47.27 -394 -2 0.97 1d13 11181913
2016-07-06 07:02 _t:620 INFO 32 Sold all -380|-1180|-1560 NVDA 47.47 -774 -81 0.97 1d13 10959183
2016-07-07 07:00 _t:620 INFO 30 Buy 12 NVDA 48.08 -774 -554 1.02 d77a 10962642
2016-07-07 07:01 _t:620 INFO 31 Bot 12 NVDA 48.06 -762 -530 0.99 d77a 10857137
2016-07-08 07:00 _t:620 INFO 30 Buy 28 NVDA 49.94 -762 -1963 1.01 48ed 10984581
2016-07-08 07:01 _t:620 INFO 31 Bot 28 NVDA 49.90 -734 -1861 1.00 48ed 11146479
2016-07-11 07:00 _t:620 INFO 30 Buy 31 NVDA 52.06 -734 -3447 1.01 3883 10998615
2016-07-11 07:01 _t:620 INFO 31 Bot 31 NVDA 52.09 -703 -3326 0.99 3883 10925876
2016-07-13 07:00 _t:620 INFO 30 Buy 14 NVDA 52.99 -703 -3955 1.00 8711 10949374
2016-07-13 07:01 _t:620 INFO 31 Bot 14 NVDA 52.97 -689 -3863 1.00 8711 10791296
2016-07-19 07:00 _t:620 INFO 30 Buy 689 NVDA 53.09 -689 -3942 1.00 3b25 10953302
2016-07-19 07:01 _t:620 INFO 31 Bot 238|0|689 NVDA 53.09 -451 -2585 0.97 3b25 10820012
2016-07-19 07:02 _t:620 INFO 32 Bot 325|238|689 NVDA 53.10 -126 -723 0.97 3b25 10920719
With NVDA, it was not affected by partial fills as often as some others. More often was like this, complete buying and selling at lower volumes, both long and short.
2017-09-14 07:00 _t:620 INFO 30 Buy 4 NVDA 169.80 -225 -11908 1.00 5762 11273249
2017-09-14 07:01 _t:620 INFO 31 Bot 4 NVDA 169.67 -221 -11668 1.00 5762 11204744
2017-09-15 07:00 _t:620 INFO 30 Buy 7 NVDA 175.90 -221 -13045 1.00 439e 11244305
2017-09-15 07:01 _t:620 INFO 31 Bot 7 NVDA 176.22 -214 -12700 1.00 439e 11308007
2017-09-18 07:00 _t:620 INFO 30 Buy 12 NVDA 185.60 -214 -14707 1.01 f16d 11261151
2017-09-18 07:01 _t:620 INFO 31 Bot 12 NVDA 185.53 -202 -13868 1.01 f16d 11031559
2017-09-21 07:00 _t:620 INFO 30 Sell -5 NVDA 181.15 -202 -12984 1.00 7821 11247529
2017-09-21 07:01 _t:620 INFO 31 Sold -5 NVDA 180.79 -207 -12911 0.98 7821 11237903
2017-09-25 07:00 _t:620 INFO 30 Sell -8 NVDA 174.01 -207 -11507 1.00 d1a6 11240518
2017-09-25 07:01 _t:620 INFO 31 Sold -8 NVDA 174.34 -215 -11575 1.03 d1a6 11352925
2017-09-26 07:00 _t:620 INFO 30 Buy 4 NVDA 176.93 -215 -12133 1.00 e677 11207084
2017-09-26 07:01 _t:620 INFO 31 Bot 4 NVDA 176.86 -211 -11892 0.98 e677 11225226
2017-09-27 07:00 _t:620 INFO 30 Sell -4 NVDA 173.60 -211 -11204 1.00 83d4 11207135
2017-09-27 07:01 _t:620 INFO 31 Sold -4 NVDA 173.87 -215 -11261 0.98 83d4 10988039
2017-09-29 07:00 _t:620 INFO 30 Buy 6 NVDA 178.37 -215 -12229 1.00 1523 11188759
2017-09-29 07:01 _t:620 INFO 31 Bot 6 NVDA 178.75 -209 -11967 1.00 1523 11068840
2017-10-09 07:00 _t:620 INFO 30 Buy 7 NVDA 184.68 -209 -13205 1.00 145a 11172453
2017-10-09 07:01 _t:620 INFO 31 Bot 7 NVDA 185.07 -202 -12843 1.00 145a 10993256
2017-10-10 07:00 _t:620 INFO 30 Buy 6 NVDA 190.44 -202 -13928 1.00 928c 11176835
2017-10-10 07:01 _t:620 INFO 31 Bot 6 NVDA 190.05 -196 -13437 1.01 928c 11037683
2017-10-16 07:00 _t:620 INFO 30 Buy 5 NVDA 195.18 -196 -14443 1.00 d262 11183832
2017-10-16 07:01 _t:620 INFO 31 Bot 5 NVDA 195.34 -191 -14105 1.00 d262 11378467
2017-10-30 07:00 _t:620 INFO 30 Buy 8 NVDA 205.18 -191 -15984 1.00 08e8 11246975
2017-10-30 07:01 _t:620 INFO 31 Bot 8 NVDA 205.50 -183 -15373 1.00 08e8 11339853
Please sign in or
join Quantopian to
post a reply.
Need help?
Contact support.
Company
Learn & Support
Notices
Follow Us
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian.
In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.