I have now been able to create a vector with historical data and save it as a context variable. The current challenge is reading the vector, in order to regress it to stocks returns and save the beta as a custom factor

I'm thinking it might be easier to do it outside the pipeline.

I decided to start again from scratch. It is to hard for me to get the pipeline to use a series of historical vix term structure data. So, i'll try using the pipeline only to get the S&P500 tickers, and do all the work later in the program.

Right now this new code gets, once a month, the beta from regressing 30 days of returns for tickers of S&P500, with the slope of the VIX term structure. Next challenge is doing something when some values are missing, as the regression then outputs a NaN.

After that, ranking the tickers according to beta and building a target portfolio with long and short positions based on the ranking.

Finally, I got a working algorithm.

Many challenges to get here. The last one being that doing regressions in a schedulled function timed-out, so I had to change the pipeline calling to before_trade_starts even though I only need it once per month. I also discovered that some quantopian features, like factors, can only be used inside the pipeline, so in order to mimic the pipeline output I had to learn some more python and research the zipline api.

Next step is checking that everything is working as it should be (or why does it not perform as it should). We need to read again the paper and carefully revisit the algorithm. If everything seems fine, then we will spend time with code optimization and playing with some parameters.

Ok. After checking we found out that the top 50 stocks ranked by beta changes everymonth. Furthermore, the daily turnover of 8% suggests that stocks from one month are completely independant from the ones choosen the next month.

As the idea of the paper is taking advantage of the fact that some stocks are more correlated to the VIX Term Structure, we know that we are doing something wrong. So we kept doing tests by:
1- Using VIX term structure slope differences instead of the actual values
2- Using actual slopes of the VIX term structure as proxies PCA slopes (doubting our PCA analysis)
3- Using differential of slopes in number 2
4- Using a percentual change of slopes in number 2

All these tests resulted in the same daily turnover. To proceed we are:
1- Contacting the author of the paper to confirm how the PCA analysis is used. Keep checking the code and PCA analysis for possible errors.

We are still improving the algorithm. We added 2 variables to the regression, changes in vix and market return:
1- Changes in vix again required to upload data in order to use historical, but it was faster as we already new how to do it. The Algo does the differenciation and builds the vector.
2- For the market return, we used SPY (the S&P500 ETF) as proxy. The Algo calculates the daily returns and builds the vector.

We tried to look at the p_values of the regressions, but most are not significant. As an additional test, we used the algorithm with the beta to market. We then get from the backtest a lower turnover (6.8% daily), an alpha of zero and a beta of 0.4. This results suggests that we are doing the regression right, but probably will need to clean the results according to the p_values.

Our last Algorithm, commented. We didn't find an statistically significant relation between the VIX Term Structure second PCA component and the return of the stocks, monthly regressing over the last month. Additional code optimizations or sock selecting filters can be added (for example using only the most statistically significant betas when building the portfolio).